Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Compilation often fails on gocryptfs #93900

Closed
krumelmonster opened this issue Feb 11, 2022 · 3 comments
Closed

Compilation often fails on gocryptfs #93900

krumelmonster opened this issue Feb 11, 2022 · 3 comments
Assignees
Labels
A-incr-comp Area: Incremental compilation C-bug Category: This is a bug. I-ICE Issue: The compiler panicked, giving an Internal Compilation Error (ICE) ❄️ T-compiler Relevant to the compiler team, which will review and decide on the PR/issue.

Comments

@krumelmonster
Copy link
Contributor

I often get these errors when trying to compile any rust project on a gocryptfs filesystem (gocryptfs v2.2.1; go-fuse [vendored]; 2021-10-30 go1.17.2 linux/amd64). There have been such problems with nightly, beta and stable.

$ cargo build
   Compiling hello v0.1.0 (/home/c/rust-hello)
thread 'rustc' panicked at 'assertion failed: `(left == right)`
  left: `82`,
 right: `1002111927320821928687967599834759150`', compiler/rustc_query_impl/src/on_disk_cache.rs:547:5
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

error: internal compiler error: unexpected panic

note: the compiler unexpectedly panicked. this is a bug.

note: we would appreciate a bug report: https://github.com/rust-lang/rust/issues/new?labels=C-bug%2C+I-ICE%2C+T-compiler&template=ice.md

note: rustc 1.59.0-beta.6 (0426998f5 2022-02-02) running on x86_64-unknown-linux-gnu

note: compiler flags: -C embed-bitcode=no -C debuginfo=2 -C incremental --crate-type bin

note: some of the compiler flags provided by cargo are hidden

query stack during panic:
end of query stack
error: could not compile `hello`

and

$ cargo build
   Compiling hello v0.1.0 (/home/c/rust-hello)
error: could not compile `hello`

Caused by:
  process didn't exit successfully: `rustc --crate-name hello --edition=2018 src/main.rs --error-format=json --json=diagnostic-rendered-ansi,future-incompat --crate-type bin --emit=dep-info,link -C embed-bitcode=no -C debuginfo=2 -C metadata=2c3affb4eb36b7d2 -C extra-filename=-2c3affb4eb36b7d2 --out-dir /home/c/rust-hello/target/debug/deps -C incremental=/home/c/rust-hello/target/debug/incremental -L dependency=/home/c/rust-hello/target/debug/deps` (signal: 7, SIGBUS: access to undefined memory)
@krumelmonster krumelmonster added C-bug Category: This is a bug. I-ICE Issue: The compiler panicked, giving an Internal Compilation Error (ICE) ❄️ T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. labels Feb 11, 2022
@csmoe
Copy link
Member

csmoe commented Feb 11, 2022

@krumelmonster would you mind sharing your code here?

@krumelmonster
Copy link
Contributor Author

Any code I tested, for example https://github.com/spiegel-im-spiegel/rust-hello

@JohnTitor JohnTitor added E-needs-mcve Call for participation: This issue has a repro, but needs a Minimal Complete and Verifiable Example A-incr-comp Area: Incremental compilation and removed E-needs-mcve Call for participation: This issue has a repro, but needs a Minimal Complete and Verifiable Example labels Mar 3, 2023
@saethlin saethlin self-assigned this Apr 29, 2024
bors added a commit to rust-lang-ci/rust that referenced this issue May 6, 2024
Add a footer in FileEncoder and check for it in MemDecoder

We have a few reports of ICEs due to decoding failures, where the fault does not lie with the compiler. The goal of this PR is to add some very lightweight and on-by-default validation to the compiler's outputs. If validation fails, we emit a fatal error for rmeta files in general that mentions the path that didn't load, and for incremental compilation artifacts we emit a verbose warning that tries to explain the situation and treat the artifacts as outdated.

The validation currently implemented here is very crude, and yet I think we have 11 ICE reports currently open (you can find them by searching issues for `1002111927320821928687967599834759150` which this simple validation would have detected. The structure of the code changes here should permit the addition of further validation code, such as a checksum, if it is merited. I would like to have code to detect corruption such as reported in rust-lang#124719, but I'm not yet sure how to do that efficiently, and this PR is already a good size.

The ICE reports I have in mind that this PR would have smoothed over are:
rust-lang#124469
rust-lang#123352
rust-lang#123376 [^1]
rust-lang#99763
rust-lang#93900.

---

[^1]: This one might be a compiler bug, but even if it is I think the workflow described is pushing the envelope of what we can support
bors added a commit to rust-lang-ci/rust that referenced this issue May 22, 2024
Add a footer in FileEncoder and check for it in MemDecoder

We have a few reports of ICEs due to decoding failures, where the fault does not lie with the compiler. The goal of this PR is to add some very lightweight and on-by-default validation to the compiler's outputs. If validation fails, we emit a fatal error for rmeta files in general that mentions the path that didn't load, and for incremental compilation artifacts we emit a verbose warning that tries to explain the situation and treat the artifacts as outdated.

The validation currently implemented here is very crude, and yet I think we have 11 ICE reports currently open (you can find them by searching issues for `1002111927320821928687967599834759150`) which this simple validation would have detected. The structure of the code changes here should permit the addition of further validation code, such as a checksum, if it is merited. I would like to have code to detect corruption such as reported in rust-lang#124719, but I'm not yet sure how to do that efficiently, and this PR is already a good size.

The ICE reports I have in mind that this PR would have smoothed over are:
rust-lang#124469
rust-lang#123352
rust-lang#123376 [^1]
rust-lang#99763
rust-lang#93900.

---

[^1]: This one might be a compiler bug, but even if it is I think the workflow described is pushing the envelope of what we can support. This issue is one of the reasons this warning still asks people to file an issue.
github-actions bot pushed a commit to rust-lang/miri that referenced this issue May 23, 2024
Add a footer in FileEncoder and check for it in MemDecoder

We have a few reports of ICEs due to decoding failures, where the fault does not lie with the compiler. The goal of this PR is to add some very lightweight and on-by-default validation to the compiler's outputs. If validation fails, we emit a fatal error for rmeta files in general that mentions the path that didn't load, and for incremental compilation artifacts we emit a verbose warning that tries to explain the situation and treat the artifacts as outdated.

The validation currently implemented here is very crude, and yet I think we have 11 ICE reports currently open (you can find them by searching issues for `1002111927320821928687967599834759150`) which this simple validation would have detected. The structure of the code changes here should permit the addition of further validation code, such as a checksum, if it is merited. I would like to have code to detect corruption such as reported in rust-lang/rust#124719, but I'm not yet sure how to do that efficiently, and this PR is already a good size.

The ICE reports I have in mind that this PR would have smoothed over are:
rust-lang/rust#124469
rust-lang/rust#123352
rust-lang/rust#123376 [^1]
rust-lang/rust#99763
rust-lang/rust#93900.

---

[^1]: This one might be a compiler bug, but even if it is I think the workflow described is pushing the envelope of what we can support. This issue is one of the reasons this warning still asks people to file an issue.
flip1995 pushed a commit to flip1995/rust-clippy that referenced this issue May 24, 2024
Add a footer in FileEncoder and check for it in MemDecoder

We have a few reports of ICEs due to decoding failures, where the fault does not lie with the compiler. The goal of this PR is to add some very lightweight and on-by-default validation to the compiler's outputs. If validation fails, we emit a fatal error for rmeta files in general that mentions the path that didn't load, and for incremental compilation artifacts we emit a verbose warning that tries to explain the situation and treat the artifacts as outdated.

The validation currently implemented here is very crude, and yet I think we have 11 ICE reports currently open (you can find them by searching issues for `1002111927320821928687967599834759150`) which this simple validation would have detected. The structure of the code changes here should permit the addition of further validation code, such as a checksum, if it is merited. I would like to have code to detect corruption such as reported in rust-lang/rust#124719, but I'm not yet sure how to do that efficiently, and this PR is already a good size.

The ICE reports I have in mind that this PR would have smoothed over are:
rust-lang/rust#124469
rust-lang/rust#123352
rust-lang/rust#123376 [^1]
rust-lang/rust#99763
rust-lang/rust#93900.

---

[^1]: This one might be a compiler bug, but even if it is I think the workflow described is pushing the envelope of what we can support. This issue is one of the reasons this warning still asks people to file an issue.
@saethlin
Copy link
Member

As far as I can tell, the ICE reported in this issue is caused by another process damaging rustc's build artifacts or by a buggy filesystem. That other process could be anything from an IDE running the compiler at the same time as a manual build to a find-and-replace that accidentally edits files in the target directory.

#124686 contains a mitigation for this situation; the compiler now has a crude way to detect and ignore damaged incremental compilation files. You will still get a warning about the damaged file, but the build should proceed normally. That change shipped in Rust 1.80 which released today, so I am closing this issue and a few others which as far as I can tell all have the same root cause. You shouldn't see this ICE anymore, but please file a new issue if you see something like it on 1.80 or later.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-incr-comp Area: Incremental compilation C-bug Category: This is a bug. I-ICE Issue: The compiler panicked, giving an Internal Compilation Error (ICE) ❄️ T-compiler Relevant to the compiler team, which will review and decide on the PR/issue.
Projects
None yet
Development

No branches or pull requests

4 participants