-
Notifications
You must be signed in to change notification settings - Fork 1.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fatal: unable to save snapshot: Detected data corruption while saving blob... Corrupted blobs are either caused by hardware issues or software bugs. #4765
Comments
I have the same problem with an s3/minio backend while backuping many small files.
"restic check --read-data" finds no errors. Also testing against a new bucket with a new repo runs into the same error. |
Do those errors show up reproducibly or do they only occur at random times? Please run a memory / CPU stress test to check for problems with your hardware. In addition, please run @mmachner Your error looks like a bitflip in the memory/CPU on your host: @alexxtasi Is that error reproducible? |
It happens at every Backup run of this run. The same server has also another backup job with restic, which gets an big dbdump piped in, which runs fine. Only the job with with small files has the Problem. I also tried with a new, clean repo, same Problem. The Server has ECC Ram and RAID Disks. Fatal: unable to save snapshot: Detected data corruption while writing pack-file header: pack header entry mismatch got <Blob (tree) 1bfb952e, offset 0, length 999811890, uncompressed length 240300308> instead of <Blob (tree) 1bfb952e, offset 0, length 999811890, uncompressed length 4535267604> Fatal: unable to save snapshot: Detected data corruption while writing pack-file header: pack header entry mismatch got <Blob (tree) 800152cb, offset 0, length 1002453363, uncompressed length 250946913> instead of <Blob (tree) 800152cb, offset 0, length 1002453363, uncompressed length 4545914209> Fatal: unable to save snapshot: Detected data corruption while writing pack-file header: pack header entry mismatch got <Blob (tree) cf6c4e56, offset 0, length 1002683982, uncompressed length 251812809> instead of <Blob (tree) cf6c4e56, offset 0, length 1002683982, uncompressed length 4546780105> |
I don't have much time right now, so I'll just give a short reply.
That's 4.2GB, which completely changes the explanation: you've hit #2446 which will only be fixed in restic 0.18.0 (https://forum.restic.net/t/roadmap-for-restic-0-17-to-0-19/7197), which hopefully still happens this year. There's unfortunately no real workaround, restic versions before 0.16.4 will just panic with Do you have any way to reduce the number of small files in a folder (subfolders don't count!)? |
@MichaelEischer thanks for the suggestion
It is, in terms of whenever attempt to backup (the same command) the result is the same. But...
and on the other hand... you where right !! My pc has RAM issues !! ;) Thanks you very much. |
restic includes several layers of integrity checks to (hopefully) detect data corruption in a backup. That means it's usually pretty effective at detection RAM issues. |
Closing as there's nothing left to do here and the tree blob size problem is tracked in #2446 |
Output of
restic version
restic 0.16.4 compiled with go1.21.6 on linux/amd64
What backend/service did you use to store the repository?
A usb connected HDD
Problem description / Steps to reproduce
restic init --repo /run/media/alex/restic/restic-home/
restic --repo /run/media/alex/restic/restic-home --verbose backup /home/alex/ --exclude=/home/alex/VirtualBox\ VMs/
(tried to ...)Expected behavior
Proceed to backup
Actual behavior
The output:
Do you have any idea what may have caused this?
No !
It happened once again using the same HDD to create restic repos. Same
... Corrupted blobs are either caused by hardware issues or software bugs...
error happened. But that time I was trying to create restic backup of large amount of photo files.Did restic help you today? Did it make you happy in any way?
The text was updated successfully, but these errors were encountered: