Add generic read-only storage mode for a storage that fails the write check

[jkoritzinsky]

Today, sccache can detect that a storage medium is read-only or read-write even if this capability was not specified by the config. However, it doesn't do anything with this information other than log it for diagnostics.

This PR changes sccache to use this information to avoid writing to a read-only storage. This is useful for cases where one might want to use sccache in a read-only configuration in some cases (e.g. public PR builds) while only updating the cache from a separate pipeline (that has write access) without using a separate configuration file.

Additionally, this change reduces the number of "access denied" errors that would be logged on the storage side when sccache detects that it only has read-only access (instead of continuing to attempt to write to the storage).

Fixes #1853

[codecov-commenter]

Codecov Report

Attention: 69 lines in your changes are missing coverage. Please review.

Comparison is base (ffe3070) 30.77% compared to head (6d906b9) 30.76%.
Report is 4 commits behind head on main.

Files	Patch %	Lines
src/cache/readonly.rs	31.48%	7 Missing and 30 partials ⚠️
tests/sccache_cargo.rs	21.62%	3 Missing and 26 partials ⚠️
src/server.rs	40.00%	0 Missing and 3 partials ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #2091      +/-   ##
==========================================
- Coverage   30.77%   30.76%   -0.02%     
==========================================
  Files          52       53       +1     
  Lines       19913    20037     +124     
  Branches     9630     9726      +96     
==========================================
+ Hits         6129     6164      +35     
+ Misses       7959     7950       -9     
- Partials     5825     5923      +98     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[sylvestre]

Could you please add a test and fix the clippy warning?
Thanks

[jkoritzinsky]

I've fixed the clippy warning and added a few unit tests.

[sylvestre]

Sorry, I was talking about an integration test like

sccache/tests/sccache_cargo.rs

Line 318 in ffe3070

fn test_rust_cargo_cmd_readonly(cmd: &str, test_info: SccacheTest) -> Result<()> {

thanks

[jkoritzinsky]

@sylvestre I've added a test that validates (through the sccache log) that in a scenario where the underlying cache is detected as read-only, the ReadOnlyStorage handles the write requests and the requests never go to the underlying storage.

[sylvestre]

thanks!