Include fully qualified scope in build invalidator key #4966
Labels
Comments
|
...hm, or perhaps this should include the entire |
cosmicexplorer
added a commit
to cosmicexplorer/pants
that referenced
this issue
Dec 5, 2017
stuhood
pushed a commit
that referenced
this issue
Dec 20, 2017
…ashes (#5170) ### Problem In #4966, it was mentioned how tasks in different option scopes can end up with the same `results_dir`, which results in an error if tasks with different scopes but the same `stable_name` both try to use the cache in the same pants run (e.g. `lint.scalastyle` and `compile.scalastyle`). ### Solution - Use `Task#fingerprint` to compute the build invalidation key (in `TaskBase#_build_invalidator()`) and to generate the name of the cache subdirectory where results from the task run are stored (in `CacheFactory.make_task_cache_dirname( )` - Incorporate the `stable_name()` and `implementation_version()` into the computation of the task fingerprint. - Incorporate the `options_scope` of the `Task` subclass and the scopes of its `subsystem_dependencies` into the computation of the task fingerprint. Traverse all registered option scopes and incorporate the values of relevant options with `OptionsFingerprinter.combined_options_fingerprint_for_scope()`. ### Result - The task fingerprint is now rigorously described and tested. - The task fingerprint is now used for the invalidation key and for the name of the results directory. A corollary of this is that e.g. `lint.scalastyle` and `compile.scalastyle` no longer use the same `results_dir`, which resolves the issue described in #4966. - The directory path containing the task results after execution can no longer be guessed without instantiating the task, because the task fingerprint requires fingerprinting the option values relevant to the task. In `test_cache_cleanup_integration.py`, we run the pants subprocess twice in a few tests to get around this.
cosmicexplorer
added a commit
to cosmicexplorer/pants
that referenced
this issue
Mar 6, 2019
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Because the
BuildInvalidatordirectory is computed using aTask.stable_name, the same task installed in two different locations will use the sameBuildInvalidatordirectory. This leads to errors where theBuildInvalidatorcan think that an earlier copy of a task running (which will put outputs in.pants.d/${scope1}/**/*) count as valid runs of the latter task (which will put outputs in.pants.d/${scope2}/**/*).The location where a task is installed (its "
scope") should also be included in the build invalidator location.The text was updated successfully, but these errors were encountered: