-
Notifications
You must be signed in to change notification settings - Fork 10
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Surface action cache hit rate #90
Labels
type/feat
Suggests new features.
Comments
saraadams
changed the title
Surface action cache hHit Rate
Surface action cache hit rate
Jan 26, 2023
If latency is high, then having many parallel check cache hit actions can help speed up getting remote cache hits (as the jobs are idle due to high latency). |
saraadams
added a commit
that referenced
this issue
Nov 25, 2023
This change * fixes the percentage shown, it was showing the inverse value (cache miss % instead of cache hit %) * adds absolute numbers for how many cache checks were performed and how many were misses * filters out local actions that don't do remote cache checks, as these are not relevant Contributes to #90 Signed-off-by: Sara Adams <sara.e.adams@gmail.com>
saraadams
added a commit
that referenced
this issue
Nov 25, 2023
This change * fixes the percentage shown, it was showing the inverse value (cache miss % instead of cache hit %) * adds absolute numbers for how many cache checks were performed and how many were misses * filters out local actions that don't do remote cache checks, as these are not relevant Contributes to #90 Signed-off-by: Sara Adams <sara.e.adams@gmail.com>
saraadams
added a commit
that referenced
this issue
Nov 25, 2023
This change * fixes the percentage shown, it was showing the inverse value (cache miss % instead of cache hit %) * adds absolute numbers for how many cache checks were performed and how many were misses * filters out local actions that don't do remote cache checks, as these are not relevant Contributes to #90 Signed-off-by: Sara Adams <sara.e.adams@gmail.com>
|
saraadams
added a commit
that referenced
this issue
Nov 25, 2023
If remote caching is used and there are remote cache misses, suggest investigating the misses with a link to Bazel documentation. Contributes to #90 Signed-off-by: Sara Adams <sara.e.adams@gmail.com>
saraadams
added a commit
that referenced
this issue
Nov 26, 2023
If remote caching is used and there are remote cache misses, suggest investigating the misses with a link to Bazel documentation. Contributes to #90 Signed-off-by: Sara Adams <sara.e.adams@gmail.com>
saraadams
added a commit
that referenced
this issue
Nov 27, 2023
This change * fixes the percentage shown, it was showing the inverse value (cache miss % instead of cache hit %) * fixes the "remote upload outputs" time, which mistakenly also included upload times of inputs (for RE). * filters out local actions that don't do remote cache checks, as these are not relevant and falsify the % * adds absolute numbers for how many cache checks were performed and how many were misses * adds some documentation on `disk_cache` also being a remote cache Contributes to #90 --------- Signed-off-by: Sara Adams <sara.e.adams@gmail.com>
saraadams
added a commit
that referenced
this issue
Nov 27, 2023
…c `CompleteEvent`s (#125) This is a refactor and code cleanup in preparation for addressing #90 * moves some static methods over to a new util class that helps identify the meaning of `CompleteEvent`s seen in Bazel profiles * adds some missing copyright notices --------- Signed-off-by: Sara Adams <sara.e.adams@gmail.com>
saraadams
added a commit
that referenced
this issue
Nov 27, 2023
If remote caching is used and there are remote cache misses, suggest investigating the misses with a link to Bazel documentation. Contributes to #90 Signed-off-by: Sara Adams <sara.e.adams@gmail.com>
saraadams
added a commit
that referenced
this issue
Nov 27, 2023
If remote caching is used and there are remote cache misses, suggest investigating the misses with a link to Bazel documentation. Contributes to #90 --------- Signed-off-by: Sara Adams <sara.e.adams@gmail.com>
saraadams
added a commit
that referenced
this issue
Nov 28, 2023
…on location This data provider scans all actions and splits them into: * remote cache hit * remote cache miss * remote cache not checked as well as (for non-cache-hits): * executed locally * executed remotely * execution location not reported "Internal" Bazel actions are included in "remote cache not checked" and "execution location not reported". While I invested ample time in trying to single internal actions out reliably, I did not succeed. I'm not sure it's possible with the information currently written to profiles. A TODO to look into separating out "internal" actions is in the code. Contributes to #90 Signed-off-by: Sara Adams <sara.e.adams@gmail.com>
saraadams
added a commit
that referenced
this issue
Nov 28, 2023
…on location (#138) This data provider scans all actions and splits them into: * remote cache hit * remote cache miss * remote cache not checked as well as (for non-cache-hits): * executed locally * executed remotely * execution location not reported "Internal" Bazel actions are included in "remote cache not checked" and "execution location not reported". While I invested ample time in trying to single internal actions out reliably, I did not succeed. I'm not sure it's possible with the information currently written to profiles. A TODO to look into separating out "internal" actions is in the code. Contributes to #90 --------- Signed-off-by: Sara Adams <sara.e.adams@gmail.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Problem
Surface the action cache hit rate, in particular if remote caching is used.
Suggested solution
The following events may help detect these:
check cache hit
(categoryremote action cache check
) within eventActionContinuation.execute
(categorygeneral information
), thereafter executionupload missing inputs
(categoryRemote execution upload time
) followed byexecute remotely
(categoryremote action execution
)check cache hit
within eventActionContinuation.execute
, no execution thereafterDataProvider
to provide rate and/or absolute numbers (cache checks, successful cache checks)SuggestionProvider
to suggest strategies to increase the cache hit rate, e.g. --incompatible_strict_action_envThe text was updated successfully, but these errors were encountered: