Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

remote-install: add "ana/" to the default install list #5137

Merged
merged 3 commits into from
Sep 27, 2022
Merged

remote-install: add "ana/" to the default install list #5137

merged 3 commits into from
Sep 27, 2022

Conversation

oliver-sanders
Copy link
Member

@oliver-sanders oliver-sanders commented Sep 15, 2022 •
edited
Loading

  • The ana/ directory is used by rose_ana to load "compotators" (custom comparison thinggies).
  • These are typically run where the data is generated which is often remote.
  • These directories typically contain a small number of Python files.

Check List

  • I have read CONTRIBUTING.md and added my name as a Code Contributor.
  • Contains logically grouped changes (else tidy your branch by rebase).
  • Does not contain off-topic changes (use other PRs for other changes).
  • Applied any dependency changes to both setup.cfg and conda-environment.yml.
  • Tests are included (or explain why tests are not needed).
  • CHANGES.md entry included if this is a change that can affect users
  • Cylc-Doc pull request opened if required at remote-install: document ana/ directory cylc-doc#535
  • If this is a bug fix, PRs raised to both master and the relevant maintenance branch.

@oliver-sanders oliver-sanders added the bug Something is wrong :( label Sep 15, 2022
@oliver-sanders oliver-sanders added this to the cylc-8.0.3 milestone Sep 15, 2022
@oliver-sanders oliver-sanders self-assigned this Sep 15, 2022
oliver-sanders added a commit to oliver-sanders/cylc-doc that referenced this pull request Sep 15, 2022
@oliver-sanders
remote-install: document ana/ directory
1895170 
* To accompany cylc/cylc-flow#5137
@oliver-sanders oliver-sanders mentioned this pull request Sep 15, 2022
Merged
1 task
@oliver-sanders oliver-sanders force-pushed the rose-ana-fix branch 2 times, most recently from 9d81f2a to 97d066a Compare September 15, 2022 12:05
@datamel
Copy link
Contributor

datamel commented Sep 15, 2022

Just CHANGES.md conflicts.

@datamel
Copy link
Contributor

datamel commented Sep 15, 2022

Actually, while looking at the sibling docs pr I remembered that there is a bit of documentation in cylc-flow. In

cylc-flow/cylc/flow/cfgspec/workflow.py

Line 223 in 83f258c

The following directories already get installed by default:
, you could add ana/, to the list of dirs included by default.

@oliver-sanders
remote-install: add "ana/" to the default install list
68d5670 
* The `ana/` directory is used by `rose_ana` to load comparison modules.
* These are typically run where the data is generated which is often
  remote.
* These directories typically contain a small number of Python files.
@oliver-sanders
Copy link
Member Author

Good spot, sorted.

oliver-sanders added a commit to oliver-sanders/cylc-doc that referenced this pull request Sep 20, 2022
@oliver-sanders
remote-install: document ana/ directory
8ecd6fc 
* To accompany cylc/cylc-flow#5137
@hjoliver
Copy link
Member

compotators

Is that a word?! 🤯

@datamel
Copy link
Contributor

datamel commented Sep 21, 2022

  • compotators

https://www.merriam-webster.com/dictionary/compotator 😄

@oliver-sanders
Copy link
Member Author

(I've called them "analysis modules" in the docs)

@oliver-sanders oliver-sanders requested review from hjoliver and MetRonnie and removed request for hjoliver September 21, 2022 12:19
MetRonnie
MetRonnie approved these changes Sep 26, 2022
View reviewed changes
Copy link
Member

@MetRonnie MetRonnie left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 Just a couple of optional formatting niceties suggested

cylc/flow/cfgspec/workflow.py Outdated Show resolved Hide resolved
cylc/flow/cfgspec/workflow.py Outdated Show resolved Hide resolved
oliver-sanders and others added 2 commits September 26, 2022 16:12
@oliver-sanders @MetRonnie
Apply suggestions from code review
dd1fc7c 
Co-authored-by: Ronnie Dutta <61982285+MetRonnie@users.noreply.github.com>
@MetRonnie
Merge branch '8.0.x' into rose-ana-fix
6fb3b9f
@MetRonnie
Copy link
Member

(fast test failures due to codecov upload fail)

@MetRonnie MetRonnie merged commit c44563a into cylc:8.0.x Sep 27, 2022
@oliver-sanders oliver-sanders deleted the rose-ana-fix branch September 27, 2022 08:48
datamel pushed a commit that referenced this pull request Sep 27, 2022
@oliver-sanders @dwsutherland
@MetRonnie @hjoliver
8.0.x (#5158)
f88f32e 
* fix reversed data-store edge source-target (#5156)

Co-authored-by: David Sutherland <davidwollow@gmail.com>

* `log_vc_info`: Redirect diff straight to file to avoid blocking pipe (#5139)

* log_vc_info: redirect diff straight to file to avoid blocking pipe
* Update changelog

* A no-flow task should not merge and retrigger incomplete children (#5146)

Prevent no-flow merge.

* remote-install: add "ana/" to the default install list (#5137)

* The `ana/` directory is used by `rose_ana` to load comparison modules.
* These are typically run where the data is generated which is often
  remote.
* These directories typically contain a small number of Python files.

* Db store force triggered (#5023)

Store force-triggered flag in the run DB.

* Add a new functional test.
* Remove some redundant DB updates.
* Update change log.

Co-authored-by: David Sutherland <davidwollow@gmail.com>
Co-authored-by: Ronnie Dutta <61982285+MetRonnie@users.noreply.github.com>
Co-authored-by: Hilary James Oliver <hilary.j.oliver@gmail.com>
datamel pushed a commit that referenced this pull request Sep 29, 2022
@oliver-sanders @dwsutherland
@MetRonnie @hjoliver
8.0.x (#5170)
27e866d 
* fix reversed data-store edge source-target (#5156)

Co-authored-by: David Sutherland <davidwollow@gmail.com>

* `log_vc_info`: Redirect diff straight to file to avoid blocking pipe (#5139)

* log_vc_info: redirect diff straight to file to avoid blocking pipe
* Update changelog

* A no-flow task should not merge and retrigger incomplete children (#5146)

Prevent no-flow merge.

* remote-install: add "ana/" to the default install list (#5137)

* The `ana/` directory is used by `rose_ana` to load comparison modules.
* These are typically run where the data is generated which is often
  remote.
* These directories typically contain a small number of Python files.

* Auto bump dev version on release

* Run GH Actions tests on push to `8.*.x` branches

* Db store force triggered (#5023)

Store force-triggered flag in the run DB.

* Add a new functional test.
* Remove some redundant DB updates.
* Update change log.

* remote: ensure all remote commands use a platform config (#5152)

Remote interfaces (SSH & rsync) now all require a platform object for configuration purposes (e.g. "ssh command").
Convenience interfaces added for remote calls to Cylc servers which use the localhost platform configuration. These are now used for:
* `cylc play` command re-invocation on a Cylc server.
* Evaluation of host selection rankings (via `cylc psutil`).
* The detect old contact file check, which tests whether a workflow is still running on the server recored in the contact file.

Co-authored-by: David Sutherland <davidwollow@gmail.com>
Co-authored-by: Ronnie Dutta <61982285+MetRonnie@users.noreply.github.com>
Co-authored-by: Hilary James Oliver <hilary.j.oliver@gmail.com>
wxtim added a commit to wxtim/cylc that referenced this pull request Oct 3, 2022
@wxtim
Merge remote-tracking branch 'upstream/8.0.x' into fix.rose-re-instal…
6f83653 
…l-failure.bk

* upstream/8.0.x:
  remote: ensure all remote commands use a platform config (cylc#5152)
  Db store force triggered (cylc#5023)
  Run GH Actions tests on push to `8.*.x` branches
  Auto bump dev version on release
  remote-install: add "ana/" to the default install list (cylc#5137)
  A no-flow task should not merge and retrigger incomplete children (cylc#5146)
  `log_vc_info`: Redirect diff straight to file to avoid blocking pipe (cylc#5139)
  fix reversed data-store edge source-target (cylc#5156)
  Fix Jinja2 support if HOME undefined.
  Assume Jinja2 might be used in global-tests.cylc.
  Fix post-reload trigger. (cylc#5104)
  Bump dev version
  Update changelog
  workflow_state xtrigger: infer run num
  Type annotations
  Prepare release 8.0.2
  Remove HOME Env Variable from get_remote_workflow_run_dir (cylc#5115)
wxtim added a commit that referenced this pull request Oct 6, 2022
@oliver-sanders @dwsutherland
@MetRonnie @hjoliver @wxtim
8.0.x.patch (#5178)
b68c98d 
* fix reversed data-store edge source-target (#5156)

Co-authored-by: David Sutherland <davidwollow@gmail.com>

* `log_vc_info`: Redirect diff straight to file to avoid blocking pipe (#5139)

* log_vc_info: redirect diff straight to file to avoid blocking pipe
* Update changelog

* A no-flow task should not merge and retrigger incomplete children (#5146)

Prevent no-flow merge.

* remote-install: add "ana/" to the default install list (#5137)

* The `ana/` directory is used by `rose_ana` to load comparison modules.
* These are typically run where the data is generated which is often
  remote.
* These directories typically contain a small number of Python files.

* Auto bump dev version on release

* Run GH Actions tests on push to `8.*.x` branches

* Db store force triggered (#5023)

Store force-triggered flag in the run DB.

* Add a new functional test.
* Remove some redundant DB updates.
* Update change log.

* remote: ensure all remote commands use a platform config (#5152)

Remote interfaces (SSH & rsync) now all require a platform object for configuration purposes (e.g. "ssh command").
Convenience interfaces added for remote calls to Cylc servers which use the localhost platform configuration. These are now used for:
* `cylc play` command re-invocation on a Cylc server.
* Evaluation of host selection rankings (via `cylc psutil`).
* The detect old contact file check, which tests whether a workflow is still running on the server recored in the contact file.

* host select: allow unary operators in ranking expressions (#5151)

* host select: allow unary operators in ranking expressions

* Whitelist unary operators (required for expressions like `-1 * x`)
  in `[scheduler][run hosts]ranking` expressions.
* Improve error formatting.

* Update tests/unit/test_exceptions.py

Co-authored-by: Tim Pillinger <26465611+wxtim@users.noreply.github.com>

Co-authored-by: Tim Pillinger <26465611+wxtim@users.noreply.github.com>

* reinstall changes to `rose-suite.conf` (#5125)

reinstall: ensure rose-suite.conf changes trigger reinstallation

Co-authored-by: David Sutherland <davidwollow@gmail.com>
Co-authored-by: Ronnie Dutta <61982285+MetRonnie@users.noreply.github.com>
Co-authored-by: Hilary James Oliver <hilary.j.oliver@gmail.com>
Co-authored-by: Tim Pillinger <26465611+wxtim@users.noreply.github.com>
datamel pushed a commit to datamel/cylc-flow that referenced this pull request Oct 19, 2022
@oliver-sanders @dwsutherland
@MetRonnie @hjoliver @datamel
8.0.x (cylc#5158)
99d75d0 
* fix reversed data-store edge source-target (cylc#5156)

Co-authored-by: David Sutherland <davidwollow@gmail.com>

* `log_vc_info`: Redirect diff straight to file to avoid blocking pipe (cylc#5139)

* log_vc_info: redirect diff straight to file to avoid blocking pipe
* Update changelog

* A no-flow task should not merge and retrigger incomplete children (cylc#5146)

Prevent no-flow merge.

* remote-install: add "ana/" to the default install list (cylc#5137)

* The `ana/` directory is used by `rose_ana` to load comparison modules.
* These are typically run where the data is generated which is often
  remote.
* These directories typically contain a small number of Python files.

* Db store force triggered (cylc#5023)

Store force-triggered flag in the run DB.

* Add a new functional test.
* Remove some redundant DB updates.
* Update change log.

Co-authored-by: David Sutherland <davidwollow@gmail.com>
Co-authored-by: Ronnie Dutta <61982285+MetRonnie@users.noreply.github.com>
Co-authored-by: Hilary James Oliver <hilary.j.oliver@gmail.com>
datamel pushed a commit to datamel/cylc-flow that referenced this pull request Oct 19, 2022
@oliver-sanders @dwsutherland
@MetRonnie @hjoliver @wxtim @datamel
8.0.x.patch (cylc#5178)
cb8f583 
* fix reversed data-store edge source-target (cylc#5156)

Co-authored-by: David Sutherland <davidwollow@gmail.com>

* `log_vc_info`: Redirect diff straight to file to avoid blocking pipe (cylc#5139)

* log_vc_info: redirect diff straight to file to avoid blocking pipe
* Update changelog

* A no-flow task should not merge and retrigger incomplete children (cylc#5146)

Prevent no-flow merge.

* remote-install: add "ana/" to the default install list (cylc#5137)

* The `ana/` directory is used by `rose_ana` to load comparison modules.
* These are typically run where the data is generated which is often
  remote.
* These directories typically contain a small number of Python files.

* Auto bump dev version on release

* Run GH Actions tests on push to `8.*.x` branches

* Db store force triggered (cylc#5023)

Store force-triggered flag in the run DB.

* Add a new functional test.
* Remove some redundant DB updates.
* Update change log.

* remote: ensure all remote commands use a platform config (cylc#5152)

Remote interfaces (SSH & rsync) now all require a platform object for configuration purposes (e.g. "ssh command").
Convenience interfaces added for remote calls to Cylc servers which use the localhost platform configuration. These are now used for:
* `cylc play` command re-invocation on a Cylc server.
* Evaluation of host selection rankings (via `cylc psutil`).
* The detect old contact file check, which tests whether a workflow is still running on the server recored in the contact file.

* host select: allow unary operators in ranking expressions (cylc#5151)

* host select: allow unary operators in ranking expressions

* Whitelist unary operators (required for expressions like `-1 * x`)
  in `[scheduler][run hosts]ranking` expressions.
* Improve error formatting.

* Update tests/unit/test_exceptions.py

Co-authored-by: Tim Pillinger <26465611+wxtim@users.noreply.github.com>

Co-authored-by: Tim Pillinger <26465611+wxtim@users.noreply.github.com>

* reinstall changes to `rose-suite.conf` (cylc#5125)

reinstall: ensure rose-suite.conf changes trigger reinstallation

Co-authored-by: David Sutherland <davidwollow@gmail.com>
Co-authored-by: Ronnie Dutta <61982285+MetRonnie@users.noreply.github.com>
Co-authored-by: Hilary James Oliver <hilary.j.oliver@gmail.com>
Co-authored-by: Tim Pillinger <26465611+wxtim@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@datamel datamel datamel approved these changes

@MetRonnie MetRonnie MetRonnie approved these changes

Assignees

@oliver-sanders oliver-sanders

Labels
bug Something is wrong :(
Projects
None yet
Milestone
cylc-8.0.3
Development

Successfully merging this pull request may close these issues.
4 participants
@oliver-sanders @datamel @hjoliver @MetRonnie