Perhaps I'm doing something very stupid here, but Google hasn't been very helpful in resolving the problem.

I have an archive of files I made for backup purposes. I've generated an SHA1 checksum file from this archive:

sha1sum myarchive.tar > myarchive.tar.sha1

The contents of this file are as follows:

6f5d7bdd71fe25ed8e881265fdb8a8bbcdaa41c1  myarchive.tar

I have also run the SHA1 process in the terminal without piping to a file:

sha1sum myarchive.tar

This gives me the output:

6f5d7bdd71fe25ed8e881265fdb8a8bbcdaa41c1  myarchive.tar

Clearly these checksums are the same. However, when I run the verification command, with the archive and its SHA1 file next to each other in the same directory:

sha1sum -c myarchive.tar.sha1

I get an error message saying that the checksum does not match:

myarchive.tar: FAILED
sha1sum: WARNING: 1 computed checksum did NOT match

Obviously something is wrong here, but I have no idea what it could be. Can anyone enlighten me?

EDIT: Interestingly, doing two consecutive MD5s on the file comes up with two different checksums. Now I'm very confused.

$ md5sum myarchive.tar
9a15036eed341613bbcf2c4b53a09859  myarchive.tar
$ md5sum myarchive.tar
a662d6b469627c62f2b03ee0df067436  myarchive.tar

EDIT2: Additional context:

  • This is on real hardware (my Ubuntu MATE 19.10 desktop machine).
  • The archive I've made was destined for a Blu Ray backup disc. It's 22.6GB in size.
  • Performing an SHA1 verification of the file as burned to the Blu Ray disc ends up being successful.

EDIT3: In response to requests to see the output of dmesg, there appear to be a few errors like the following:

[ 7102.039819] perf: interrupt took too long (2502 > 2500), lowering kernel.perf_event_max_sample_rate to 79750
[ 8278.017874] sr 4:0:0:0: [sr0] tag#0 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
[ 8278.017876] sr 4:0:0:0: [sr0] tag#0 Sense Key : Blank Check [current] 
[ 8278.017877] sr 4:0:0:0: [sr0] tag#0 Add. Sense: No additional sense information
[ 8278.017878] sr 4:0:0:0: [sr0] tag#0 CDB: Read(10) 28 00 00 00 00 00 00 00 01 00
[ 8278.017879] blk_update_request: critical target error, dev sr0, sector 0 op 0x0:(READ) flags 0x80700 phys_seg 1 prio class 0
[ 8278.019391] sr 4:0:0:0: [sr0] tag#0 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
[ 8278.019392] sr 4:0:0:0: [sr0] tag#0 Sense Key : Blank Check [current] 
[ 8278.019392] sr 4:0:0:0: [sr0] tag#0 Add. Sense: No additional sense information
[ 8278.019393] sr 4:0:0:0: [sr0] tag#0 CDB: Read(10) 28 00 00 00 00 00 00 00 01 00
[ 8278.019394] blk_update_request: critical target error, dev sr0, sector 0 op 0x0:(READ) flags 0x0 phys_seg 1 prio class 0
[ 8278.019396] Buffer I/O error on dev sr0, logical block 0, async page read

I think this is to do with my USB Blu Ray drive, though someone may be able to correct me.

  • is sha1um a typo or an alias? Does sha1sum -c <(sha1sum myarchive.tar) work? It's a bit unfortunate that sha1sum does not print out which result it expected vs. which one it got, so you can only guess... – frostschutz Jul 19 '20 at 14:02
  • Sorry, sha1um was a typo on my part. sha1sum -c <(sha1sum myarchive.tar) validates correctly. – x6herbius Jul 19 '20 at 14:07
  • Are you in different directories (containing the same file names) or have you modified myarchive.tar between creating the checksum file and the test of it? – sudodus Jul 19 '20 at 14:08
  • Same directory, same terminal session, commands executed one after another. No modifications whatsoever. – x6herbius Jul 19 '20 at 14:09
  • @x6herbius then either the checksum really changed or something happened to your myarchive.tar.sha1 file... make a new one then compare them with cmp -l or diff -u. or run it as sha1sum -c <(sha1sum myarchive.tar | tee myarchive.tar.tee) and see what's in the .tee file compared to .sha1 file – frostschutz Jul 19 '20 at 14:13

There are 2 things that need some scrupulous investigation in your case.

Blu-Ray disc

As proven with your dmesg output snippet, your Blu-Ray disc has reached end-of-life. There is no way to fix that disc, unless the Blu-Ray drive manages to read the data on the Blu-Ray disc 100% fine, which is improbable at this point.

Personal recommendation regarding Blu-Ray discs: If ever I have to use for whatever reason my Blu-Ray Drive to burn something, I sure use the M-Disc (wiki) Blu-Ray discs. For highly sensitive small data it's close to perfect storage.

Storage of your computer

The varying consecutive checksum read from your computer storage would bother me more, if that was read from your storage, that is. You could in theory have bad RAM, but that is not much probable. My advice is to watch the output of dmesg (man page) for a long period of time, example of such command follows:

\dmesg --human --color=auto --ctime --level=err,warn --follow

That could help you identify if your storage, be it a classic SATA HDD, or modern M.2 NVMe SSD, it does not matter, is failing. If you find there actually is a problem with your storage, better have backups prepared.

I hope this answer helps someone in similar position.

  • It's a little disturbing that the Blu Ray drive/disc has reached the end of its life, since I unpacked both from Amazon brand new yesterday. It was also the SHA1s from the files on the BD that actually do validate, as opposed to when the files are on my desktop. I will check my hard disks, though, and continue backing everything up. Thanks for your help. – x6herbius Jul 19 '20 at 17:56
  • @x6herbius Happy that we got somewhere and didn't reset your BIOS in the meantime :) If there is something missing, just comment, and once I get up in the morning, I'll fill in the gaps. Cheers and good luck! – LinuxSecurityFreak Jul 19 '20 at 18:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.