Improve build caching on CI #1902
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
(rust_highfive has picked a reviewer for you, use r? to override) |
For some reason, incremental files are changing on CI even when no code changes. Turning off incremental compilation should help avoid uploading a new cache with every single build. This commit also removes `target/.rustc_info.json` from the CI cache
jtgeibel
force-pushed
the
ci-cache-improvements
branch
from
96d1cea
to
3e852e2
Compare
The CI build cache is currently updated with every build, even if there are no changes. This is because running `cargo install` will update the index causing some files under `$HOME/.cargo` to be updated, invalidating the cache. The drawback is that if we bump the diesel version, then the CLI will not be automatically updated. However, even an old version of the CLI application is expected to be able to run our migrations on CI and the cache can always be cleared if a new version is needed.
This is an attempt to further decrease the cache size on CI.
jtgeibel
force-pushed
the
ci-cache-improvements
branch
from
ee058fe
to
5e86f7b
Compare
The change to `RUSTFLAGS` changes build hashes, so wiping the target directory and doing a full rebuild.
jtgeibel
changed the title
|
@bors: r+ Merging because the cache sizes on |
|
📌 Commit 45675ec has been approved by |
bors
added a commit
that referenced
this pull request
Nov 21, 2019
Improve build caching on CI These changes started based on a discussion over on [internals](https://internals.rust-lang.org/t/evaluating-github-actions/11292/8?u=jtgeibel). Basically, on every single build we end up touching some files and invaliding the caches. This forces Travis to compress and upload a new cache on every single build. This PR includes several changes on CI: * Incremental compilation is disabled. It appears that incremental files can change even if nothing in the codebase changes. * Drop the `target/.rustc_info.json` file before caching, as it can change on each build. * Running `cargo install` for the diesel CLI acts similar to a `cargo update` causing changes to the index and some caches maintained under `$HOME/.cargo`. `cargo install` is now only run if the diesel binary is not on the path. The downside to this is that if we bump the `.diesel_version` file then Travis will not be using the exact same version as Heroku until we clear the CI cache. Since we only use this binary to run migrations and we haven't bumped versions since January, I think this change is low risk compared to the improvement. * Add `RUSTFLAGS="-C debuginfo=0"` to disable debug info, which helps shrink the cache size. Based on my observations while putting together this PR, it looks like disabling incremental compilation has an impact on compile times, but this is small relative to the overall win on caching behavior. Additionally, our cache size has been reduced from 3948.04MB to 2439.19MB.
|
☀️ Test successful - checks-travis |
|
Now I'm seeing 2157.20MB cache for master, and I don't see auto at all, so... yay? Not sure if auto being gone is expected (I guess we haven't run r+ since this was merged?)
|
|
Ok now that I've r+ed something I see auto at 560.20MB. |
|
Yeah, I ended up manually clearing the caches for auto and master. The PR had shrunk their size, but they were still around 7G each for some reason. My best guess is that something changed on Travis that caused the hash in cache filenames to change and that those branches still had old cache files that would never be used again. In any case, hopefully our cache size will now hover aroud 2.1GB per branch/PR, instead of 4GB. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
These changes started based on a discussion over on internals. Basically, on every single build we end up touching some files and invaliding the caches. This forces Travis to compress and upload a new cache on every single build.
This PR includes several changes on CI:
target/.rustc_info.jsonfile before caching, as it can change on each build.cargo installfor the diesel CLI acts similar to acargo updatecausing changes to the index and some caches maintained under$HOME/.cargo.cargo installis now only run if the diesel binary is not on the path. The downside to this is that if we bump the.diesel_versionfile then Travis will not be using the exact same version as Heroku until we clear the CI cache. Since we only use this binary to run migrations and we haven't bumped versions since January, I think this change is low risk compared to the improvement.RUSTFLAGS="-C debuginfo=0"to disable debug info, which helps shrink the cache size.Based on my observations while putting together this PR, it looks like disabling incremental compilation has an impact on compile times, but this is small relative to the overall win on caching behavior. Additionally, our cache size has been reduced from 3948.04MB to 2439.19MB.