Testing: Prioritize changed unit tests #2832
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
This adds a short round of testing which is run in parallel with the header checks. It sees what files are changed relative to
develop
and runs only the tests which either changed, or the tests of the functions which changed, if it can determine them.The goal here is to make the math CI pipeline fail faster and with a more localized error for simple PRs. Obviously if you change a function which other functions depend on it might still fail in later stages, but this should help with iteration.
Note that these tests do get re-run later during the stage they normally would.
Other notes: OpenCL tests are excluded, and if more than 20 tests have changed we skip this to avoid it slowing down the whole CI pipeline - at that point, you probably want to move on to the later full stages anyway.
Thanks to @SteveBronder for the idea
Checklist
Math issue #(issue number)
Copyright holder: (fill in copyright holder information)
The copyright holder is typically you or your assignee, such as a university or company. By submitting this pull request, the copyright holder is agreeing to the license the submitted work under the following licenses:
- Code: BSD 3-clause (https://opensource.org/licenses/BSD-3-Clause)
- Documentation: CC-BY 4.0 (https://creativecommons.org/licenses/by/4.0/)
the basic tests are passing
./runTests.py test/unit
)make test-headers
)make test-math-dependencies
)make doxygen
)make cpplint
)the code is written in idiomatic C++ and changes are documented in the doxygen
the new changes are tested