Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added option to run multiple analytics workflows during history matching #337

Merged
merged 11 commits into from
Feb 26, 2021

Conversation

wouterjdb
Copy link
Contributor

@wouterjdb wouterjdb commented Feb 25, 2021

This PR adds the option to include multiple analytics workflows in the FlowNet config, like so:

ert:
  static_include_files: ./brugge_static
  realizations:
    num_realizations: 200
    required_success_percent: 80
    max_runtime: 600
  queue:
    system: LOCAL
    name: mr
    max_running: 5
  ensemble_weights: [4, 2, 1]
  analysis:
    - metric: [RMSE, MAE]
      quantity: ['WOPR:', 'WBHP:']
      start: 2000-01-01
      end: 2008-12-31
      outfile: analysis_metrics_iteration_2000_2008
    - metric: [RMSE, MAE]
      quantity: ['WOPR:', 'WBHP:']
      start: 2009-01-01
      end: 2010-01-01
      outfile: analysis_metrics_iteration_2009

This is a ❗ breaking change ❗ as you now must define the analysis workflows as list items under the analysis keyword.

A new issue was identified during this work #338.


Contributor checklist

  • 🎉 This PR closes Enable multiple analytics workflow in history matching mode #330.
  • 📜 I have broken down my PR into the following tasks:
    • Change config parser
    • Change template generation
    • Test
  • 🤖 I have added tests, or extended existing tests, to cover any new features or bugs fixed in this PR.
  • 📖 I have considered adding a new entry in CHANGELOG.md.
  • 📚 I have considered updating the documentation.

@wouterjdb wouterjdb added the enhancement New feature or request label Feb 25, 2021
@wouterjdb wouterjdb self-assigned this Feb 25, 2021
@wouterjdb wouterjdb marked this pull request as ready for review February 25, 2021 14:06
@edubarrosTNO
Copy link
Contributor

edubarrosTNO commented Feb 25, 2021

@wouterjdb , I did not get a request for review. But I did find a solution to allow it to work for quantity: [WOPR:BR-P-, WBHP:BR-P-] (in Brugge) (i.e., closes #338). The following code in _ahm_iteration_analytics.py worked for me locally:

        truth_data = (
            df_obs_filtered.pivot(
                index="DATE", columns="WELL_NAME", values=key.split(":")[0]
            )
            .add_prefix(key.split(":")[0] + ":")
            .fillna(0)
            .reset_index()
        )

        obs_opm = prepare_opm_reference_data(truth_data, key, nb_real)
        truth_data = truth_data.loc[:, truth_data.columns.str.startswith(key)]

        # Prepare data from ensemble of FlowNet
        ens_flownet = []
        ens_flownet.append(
            prepare_flownet_data(
                filter_dataframe(
                    df_sim[list(truth_data.columns) + ["DATE"]],
                    "DATE",
                    args.start,
                    args.end,
                ),
                key,
                nb_real,
            )
        )

Let's continue tomorrow. Hopefully we can merge this PR still in the morning

Copy link
Contributor

@edubarrosTNO edubarrosTNO left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code-wise, LGTM! Do not forget to revert .github/workflows/flownet.yml back to equinor:master.

I have checked the CI checks with your flownet-testdata branch and the workflow fails (never gets to [Done]) because the config files there have quantity: [WOPR:BR-P-, ...], so it falls into the fix of the next PR. But I guess that you have verified that this is the only issue.

One thing is: we should maybe add a comment somewhere making explicitly clear that the hyperopt option will use settings of the first instance of the analysis list.

@wouterjdb wouterjdb merged commit f0d5db3 into equinor:master Feb 26, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Enable multiple analytics workflow in history matching mode
2 participants