Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: warn on empty cache #9236

Merged
merged 4 commits into from
Oct 10, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 23 additions & 1 deletion crates/turborepo-lib/src/run/cache.rs
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,9 @@ use std::{
time::Duration,
};

use itertools::Itertools;
use tokio::sync::oneshot;
use tracing::{debug, error};
use tracing::{debug, error, log::warn};
use turbopath::{
AbsoluteSystemPath, AbsoluteSystemPathBuf, AnchoredSystemPath, AnchoredSystemPathBuf,
};
Expand Down Expand Up @@ -47,6 +48,7 @@ pub enum Error {
pub struct RunCache {
task_output_logs: Option<OutputLogsMode>,
cache: AsyncCache,
warnings: Arc<Mutex<Vec<String>>>,
reads_disabled: bool,
writes_disabled: bool,
repo_root: AbsoluteSystemPathBuf,
Expand Down Expand Up @@ -80,6 +82,7 @@ impl RunCache {
RunCache {
task_output_logs,
cache,
warnings: Default::default(),
reads_disabled: opts.skip_reads,
writes_disabled: opts.skip_writes,
repo_root: repo_root.to_owned(),
Expand Down Expand Up @@ -122,12 +125,18 @@ impl RunCache {
log_file_path,
daemon_client: self.daemon_client.clone(),
ui: self.ui,
warnings: self.warnings.clone(),
}
}

pub async fn shutdown_cache(
&self,
) -> Result<(Arc<Mutex<UploadMap>>, oneshot::Receiver<()>), CacheError> {
if let Ok(warnings) = self.warnings.lock() {
for warning in warnings.iter().sorted() {
warn!("{}", warning);
}
}
// Ignore errors coming from cache already shutting down
self.cache.start_shutdown().await
}
Expand All @@ -144,6 +153,7 @@ pub struct TaskCache {
daemon_client: Option<DaemonClient<DaemonConnector>>,
ui: ColorConfig,
task_id: TaskId<'static>,
warnings: Arc<Mutex<Vec<String>>>,
}

impl TaskCache {
Expand Down Expand Up @@ -360,6 +370,18 @@ impl TaskCache {
globwalk::WalkType::All,
)?;

// If we're only caching the log output, *and* output globs are not empty,
// we should warn the user
if files_to_be_cached.len() == 1 && !self.repo_relative_globs.is_empty() {
let _ = self.warnings.lock().map(|mut warnings| {
warnings.push(format!(
"no output files found for task {}. Please check your `outputs` key in \
`turbo.json`",
self.task_id
))
});
}

let mut relative_paths = files_to_be_cached
.into_iter()
.map(|path| {
Expand Down
3 changes: 3 additions & 0 deletions crates/turborepo-lib/src/task_graph/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,9 @@ pub struct TaskOutputs {
}

impl TaskOutputs {
pub fn is_empty(&self) -> bool {
self.inclusions.is_empty() && self.exclusions.is_empty()
}
pub fn validated_inclusions(&self) -> Result<Vec<ValidatedGlob>, GlobError> {
self.inclusions
.iter()
Expand Down
2 changes: 2 additions & 0 deletions turborepo-tests/integration/tests/affected.t
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,7 @@ Validate that we only run `my-app#build` with change not committed
Cached: 0 cached, 1 total
Time:\s*[\.0-9]+m?s (re)

WARNING no output files found for task my-app#build. Please check your `outputs` key in `turbo.json`


Do the same thing with the `ls` command
Expand Down Expand Up @@ -80,6 +81,7 @@ Validate that we only run `my-app#build` with change not committed
Cached: 0 cached, 1 total
Time:\s*[\.0-9]+m?s (re)

WARNING no output files found for task my-app#build. Please check your `outputs` key in `turbo.json`

Do the same thing with the `ls` command
$ ${TURBO} ls --affected
Expand Down
2 changes: 2 additions & 0 deletions turborepo-tests/integration/tests/filter-run.t
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,8 @@ Setup
Cached: 0 cached, 2 total
Time:\s*[\.0-9]+m?s (re)

WARNING no output files found for task my-app#build. Please check your `outputs` key in `turbo.json`
WARNING no output files found for task util#build. Please check your `outputs` key in `turbo.json`

Non existent package name should error
$ ${TURBO} run build --filter="foo" --output-logs none
Expand Down
2 changes: 2 additions & 0 deletions turborepo-tests/integration/tests/global-deps.t
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ Run a build
Cached: 0 cached, 1 total
Time:\s*[\.0-9]+m?s (re)

WARNING no output files found for task my-app#build. Please check your `outputs` key in `turbo.json`

$ echo "new text" > global_deps/foo.txt
$ ${TURBO} build -F my-app --output-logs=hash-only
Expand All @@ -24,6 +25,7 @@ Run a build
Cached: 0 cached, 1 total
Time:\s*[\.0-9]+m?s (re)

WARNING no output files found for task my-app#build. Please check your `outputs` key in `turbo.json`
$ echo "Submit a PR!" >> global_deps/CONTRIBUTING.md
$ ${TURBO} build -F my-app --output-logs=hash-only
\xe2\x80\xa2 Packages in scope: my-app (esc)
Expand Down
3 changes: 3 additions & 0 deletions turborepo-tests/integration/tests/global-env.t
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ Setup
Cached: 0 cached, 1 total
Time:\s*[\.0-9]+m?s (re)

WARNING no output files found for task util#build. Please check your `outputs` key in `turbo.json`
# run again and ensure there's a cache hit
$ ${TURBO} run build --filter=util --output-logs=hash-only
\xe2\x80\xa2 Packages in scope: util (esc)
Expand All @@ -36,6 +37,7 @@ Setup
Cached: 0 cached, 1 total
Time:\s*[\.0-9]+m?s (re)

WARNING no output files found for task util#build. Please check your `outputs` key in `turbo.json`
# set env var with "THASH" and ensure cache miss
$ SOMETHING_THASH_YES=hi ${TURBO} run build --filter=util --output-logs=hash-only
\xe2\x80\xa2 Packages in scope: util (esc)
Expand All @@ -58,6 +60,7 @@ Setup
Cached: 0 cached, 1 total
Time:\s*[\.0-9]+m?s (re)

WARNING no output files found for task util#build. Please check your `outputs` key in `turbo.json`
# THASH deprecation doesn't break --dry=json
$ SOMETHING_THASH_YES=hi ${TURBO} run build --filter=util --dry=json | jq -r '.tasks[0].environmentVariables.global[0]'
null
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ Populate cache
Cached: 0 cached, 1 total
Time:\s*[\.0-9]+m?s (re)

WARNING no output files found for task a#build. Please check your `outputs` key in `turbo.json`
$ ${TURBO} build --filter=b
\xe2\x80\xa2 Packages in scope: b (esc)
\xe2\x80\xa2 Running build in 1 packages (esc)
Expand All @@ -25,6 +26,7 @@ Populate cache
Cached: 0 cached, 1 total
Time:\s*[\.0-9]+m?s (re)

WARNING no output files found for task b#build. Please check your `outputs` key in `turbo.json`

Bump dependency for b and rebuild
Only b should have a cache miss
Expand Down Expand Up @@ -53,6 +55,7 @@ Only b should have a cache miss
Cached: 0 cached, 1 total
Time:\s*[\.0-9]+m?s (re)

WARNING no output files found for task b#build. Please check your `outputs` key in `turbo.json`
Add lockfile changes to a commit
$ git add . && git commit -m "bump lockfile" --quiet
Only root and b should be rebuilt since only the deps for b had a version bump
Expand All @@ -76,6 +79,7 @@ Bump of root workspace invalidates all packages
Cached: 0 cached, 1 total
Time:\s*[\.0-9]+m?s (re)

WARNING no output files found for task a#build. Please check your `outputs` key in `turbo.json`
$ ${TURBO} build --filter=b
\xe2\x80\xa2 Packages in scope: b (esc)
\xe2\x80\xa2 Running build in 1 packages (esc)
Expand All @@ -87,6 +91,7 @@ Bump of root workspace invalidates all packages
Cached: 0 cached, 1 total
Time:\s*[\.0-9]+m?s (re)

WARNING no output files found for task b#build. Please check your `outputs` key in `turbo.json`
Add lockfile changes to a commit
$ git add . && git commit -m "global lockfile change" --quiet
Everything should be rebuilt as a dependency of the root package got bumped
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ Populate cache
Cached: 0 cached, 1 total
Time:\s*[\.0-9]+m?s (re)

WARNING no output files found for task a#build. Please check your `outputs` key in `turbo.json`
$ ${TURBO} build --filter=b
\xe2\x80\xa2 Packages in scope: b (esc)
\xe2\x80\xa2 Running build in 1 packages (esc)
Expand All @@ -33,6 +34,7 @@ Populate cache
Cached: 0 cached, 1 total
Time:\s*[\.0-9]+m?s (re)

WARNING no output files found for task b#build. Please check your `outputs` key in `turbo.json`

Bump dependency for b and rebuild
Only b should have a cache miss
Expand Down Expand Up @@ -69,6 +71,7 @@ Only b should have a cache miss
Cached: 0 cached, 1 total
Time:\s*[\.0-9]+m?s (re)

WARNING no output files found for task b#build. Please check your `outputs` key in `turbo.json`
Add lockfile changes to a commit
$ git add . && git commit -m "bump lockfile" --quiet
Only root and b should be rebuilt since only the deps for b had a version bump
Expand Down Expand Up @@ -96,6 +99,7 @@ Bump of root workspace invalidates all packages
Cached: 0 cached, 1 total
Time:\s*[\.0-9]+m?s (re)

WARNING no output files found for task a#build. Please check your `outputs` key in `turbo.json`
$ ${TURBO} build --filter=b
\xe2\x80\xa2 Packages in scope: b (esc)
\xe2\x80\xa2 Running build in 1 packages (esc)
Expand All @@ -111,6 +115,7 @@ Bump of root workspace invalidates all packages
Cached: 0 cached, 1 total
Time:\s*[\.0-9]+m?s (re)

WARNING no output files found for task b#build. Please check your `outputs` key in `turbo.json`
Add lockfile changes to a commit
$ git add . && git commit -m "global lockfile change" --quiet
Everything should be rebuilt as a dependency of the root package got bumped
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ Populate cache
Cached: 0 cached, 1 total
Time:\s*[\.0-9]+m?s (re)

WARNING no output files found for task a#build. Please check your `outputs` key in `turbo.json`
$ ${TURBO} build --filter=b
\xe2\x80\xa2 Packages in scope: b (esc)
\xe2\x80\xa2 Running build in 1 packages (esc)
Expand All @@ -33,6 +34,7 @@ Populate cache
Cached: 0 cached, 1 total
Time:\s*[\.0-9]+m?s (re)

WARNING no output files found for task b#build. Please check your `outputs` key in `turbo.json`

Bump dependency for b and rebuild
Only b should have a cache miss
Expand Down Expand Up @@ -69,6 +71,7 @@ Only b should have a cache miss
Cached: 0 cached, 1 total
Time:\s*[\.0-9]+m?s (re)

WARNING no output files found for task b#build. Please check your `outputs` key in `turbo.json`
Add lockfile changes to a commit
$ git add . && git commit -m "bump pnpm-lock" --quiet
Only root and b should be rebuilt since only the deps for b had a version bump
Expand Down Expand Up @@ -96,6 +99,7 @@ Bump of root workspace invalidates all packages
Cached: 0 cached, 1 total
Time:\s*[\.0-9]+m?s (re)

WARNING no output files found for task a#build. Please check your `outputs` key in `turbo.json`
$ ${TURBO} build --filter=b
\xe2\x80\xa2 Packages in scope: b (esc)
\xe2\x80\xa2 Running build in 1 packages (esc)
Expand All @@ -111,6 +115,7 @@ Bump of root workspace invalidates all packages
Cached: 0 cached, 1 total
Time:\s*[\.0-9]+m?s (re)

WARNING no output files found for task b#build. Please check your `outputs` key in `turbo.json`
Add lockfile changes to a commit
$ git add . && git commit -m "global lockfile change" --quiet
Everything should be rebuilt as a dependency of the root package got bumped
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ Populate cache
Cached: 0 cached, 1 total
Time:\s*[\.0-9]+m?s (re)

WARNING no output files found for task a#build. Please check your `outputs` key in `turbo.json`
$ ${TURBO} build --filter=b
\xe2\x80\xa2 Packages in scope: b (esc)
\xe2\x80\xa2 Running build in 1 packages (esc)
Expand All @@ -33,6 +34,7 @@ Populate cache
Cached: 0 cached, 1 total
Time:\s*[\.0-9]+m?s (re)

WARNING no output files found for task b#build. Please check your `outputs` key in `turbo.json`

Bump dependency for b and rebuild
Only b should have a cache miss
Expand Down Expand Up @@ -69,6 +71,7 @@ Only b should have a cache miss
Cached: 0 cached, 1 total
Time:\s*[\.0-9]+m?s (re)

WARNING no output files found for task b#build. Please check your `outputs` key in `turbo.json`
Add lockfile changes to a commit
$ git add . && git commit -m "bump lockfile" --quiet
Only root and b should be rebuilt since only the deps for b had a version bump
Expand Down Expand Up @@ -96,6 +99,7 @@ Bump of root workspace invalidates all packages
Cached: 0 cached, 1 total
Time:\s*[\.0-9]+m?s (re)

WARNING no output files found for task a#build. Please check your `outputs` key in `turbo.json`
$ ${TURBO} build --filter=b
\xe2\x80\xa2 Packages in scope: b (esc)
\xe2\x80\xa2 Running build in 1 packages (esc)
Expand All @@ -111,6 +115,7 @@ Bump of root workspace invalidates all packages
Cached: 0 cached, 1 total
Time:\s*[\.0-9]+m?s (re)

WARNING no output files found for task b#build. Please check your `outputs` key in `turbo.json`
Add lockfile changes to a commit
$ git add . && git commit -m "global lockfile change" --quiet
Everything should be rebuilt as a dependency of the root package got bumped
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -28,3 +28,4 @@
Cached: 0 cached, 1 total
Time:\s+[.0-9]+m?s (re)

WARNING no output files found for task app-a#dev. Please check your `outputs` key in `turbo.json`
1 change: 1 addition & 0 deletions turborepo-tests/integration/tests/pkg-inference.t
Original file line number Diff line number Diff line change
Expand Up @@ -17,3 +17,4 @@ Setup
Cached: 0 cached, 1 total
Time:\s*[\.0-9]+m?s (re)

WARNING no output files found for task util#build. Please check your `outputs` key in `turbo.json`
Original file line number Diff line number Diff line change
Expand Up @@ -22,5 +22,6 @@ Make sure that the internal util package is part of the prune output
Cached: 0 cached, 1 total
Time:\s*[\.0-9]+m?s (re)

WARNING no output files found for task docs#new-task. Please check your `outputs` key in `turbo.json`


3 changes: 3 additions & 0 deletions turborepo-tests/integration/tests/run-caching/cache-state.t
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,9 @@ Run a build to get a local cache.
Cached: 0 cached, 2 total
Time:\s+[.0-9]+m?s (re)

WARNING no output files found for task my-app#build. Please check your `outputs` key in `turbo.json`
WARNING no output files found for task util#build. Please check your `outputs` key in `turbo.json`


Do a dry run so we can see the state of the cache
$ ${TURBO} run build --dry=json > dry.json
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ Running build for my-app succeeds
Cached: 0 cached, 1 total
Time:\s*[\.0-9]+m?s (re)

WARNING no output files found for task my-app#build. Please check your `outputs` key in `turbo.json`
Update exluded file and try again
$ echo "new excluded value" > apps/my-app/excluded.txt
$ ${TURBO} run build --filter=my-app
Expand Down
2 changes: 2 additions & 0 deletions turborepo-tests/integration/tests/run-caching/global-deps.t
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ Run a build to get a local cache.
Cached: 0 cached, 1 total
Time:\s+[.0-9]+m?s (re)

WARNING no output files found for task my-app#build. Please check your `outputs` key in `turbo.json`

Run again to get cache hit
$ SOME_ENV_VAR=hi ${TURBO} run build --output-logs=none --filter=my-app
Expand All @@ -33,3 +34,4 @@ Run again without env var to get cache miss
Cached: 0 cached, 1 total
Time:\s+[.0-9]+m?s (re)

WARNING no output files found for task my-app#build. Please check your `outputs` key in `turbo.json`
2 changes: 2 additions & 0 deletions turborepo-tests/integration/tests/run-caching/root-deps.t
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ Warm the cache
Cached: 0 cached, 1 total
Time:\s+[.0-9]+m?s (re)

WARNING no output files found for task another#build. Please check your `outputs` key in `turbo.json`

Confirm cache hit
$ ${TURBO} build --filter=another --output-logs=hash-only
Expand Down Expand Up @@ -43,6 +44,7 @@ All tasks should be a cache miss, even ones that don't depend on changed package
Cached: 0 cached, 1 total
Time:\s+[.0-9]+m?s (re)

WARNING no output files found for task another#build. Please check your `outputs` key in `turbo.json`

Verify that all packages are in scope on a internal root dep change
$ ${TURBO} build --filter='[HEAD]' --dry=json | jq '.packages'
Expand Down
2 changes: 2 additions & 0 deletions turborepo-tests/integration/tests/run-logging/errors-only.t
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ Setup
Cached: 0 cached, 1 total
Time:\s*[\.0-9]+m?s (re)

WARNING no output files found for task app-a#build. Please check your `outputs` key in `turbo.json`



Expand All @@ -28,6 +29,7 @@ Setup
Cached: 0 cached, 1 total
Time:\s*[\.0-9]+m?s (re)

WARNING no output files found for task app-a#buildsuccess. Please check your `outputs` key in `turbo.json`


# [x] error exit
Expand Down
Loading
Loading