Questions tagged [checksum]
The checksum tag has no usage guidance.
157
questions
0
votes
1
answer
86
views
gnome-software Unable to get list of updates: Failed to update metadata for lvfs: checksum failure
I sometimes use gnome-software to check for updates on Debian. My repositories are Debian Stable, Flathub and a few others each dedicated to a single program. When I do sudo apt-get update on Terminal ...
- 1
1
vote
1
answer
52
views
How to calculate a single checksum for each folder in a directory?
I am comparing two large directories between a server and a client machine. I run a command similar to the one below to determine if the two directories are the same or not.
find /path/to/dir/ -type f ...
- 43
1
vote
1
answer
185
views
How to determine which checksum `rsync` uses by default?
In the rsync image of output below:
I am unable to find the default checksum. Is it the first one,
or the one in brackets?
What is the bit-depth of just xxhash?
Found this DOCUMENTATION states the ...
- 25
0
votes
1
answer
75
views
Using rsync to query the checksum of a remote file
Can we use rsync to simply get the checksum of a remote file, without transferring it?
This could be useful for verifying the integrity of remote files without using a lot of bandwidth, if you don't ...
- 941
4
votes
1
answer
209
views
Checksumming on Ext4 (crc32c-intel) while formatting (5TB external HDD)
I bought a new 2.5-inch external hard drive of 5TB in size from Seagate.
On my Linux Mint 21.1 now, I need to format the newly created partition with gdisk:
Model: Expansion HDD
Sector size (...
- 26.8k
1
vote
1
answer
45
views
checksum a pipeline
I want to construct a huge blob of data (a backup of a sort) and send it over the network (ssh or rsync) to another host. There is enough space on the remote for the data, but not on the local host, ...
- 478
2
votes
2
answers
107
views
does rsync calculate a checksum even when it won't be transferring any data?
From man rsync:
--existing
skip creating new files on receiver
(emphasis mine)
So, what I've done with apparent success:
rsync --existing -rtvhP "remote.machine:/photos/*" /photos
This ...
- 41
7
votes
4
answers
407
views
Record count and cksum on compressed file
There is a file of 40 GB in gz format. I want to find the record count and cksum of this file in uncompressed format. One approach I have is:
Unzip the file using gunzip
Use wc, cksum commands on ...
- 183
1
vote
1
answer
113
views
rsync not detecting changed file with option --append-verify
I have invoked rsync with the options --checksum and --append-verify.
My understanding of the processing for this combination of options has been that the application would behave as in the ...
- 320
2
votes
0
answers
662
views
Does the tar format support checksums?
TLDR, my main questions:
Does the tar format itself support checksums?
How to create a tar (or even better, tar.gz file) with the tar format built-in checksum?
Long:
Before you ask why I am asking or ...
- 1,748
1
vote
1
answer
868
views
How to verify the downloaded files by .torrent?
Many Ubuntu versions like 18.04.1, 18.04.2, 16.04.3, kubuntu, lubuntu, xubuntu, couldn't find it's checksum of .iso file from websites, but the .iso can be downloaded from a torrent file providers ...
- 105
0
votes
2
answers
133
views
How to declare each outputs if there are multiple output in a single query in shell script?
I want to write a condition if the output matches by executing the below command.
diff -is <(echo 'curl https://get.gravitational.com/teleport-v9.3.4-linux-amd64-bin.tar.gz.sha256') \
<(...
0
votes
1
answer
107
views
Looking for a checksum-based, poor-man's solution to fighting silent data corruption in long term storage of family video files
Secure storage of my collection of family video is becoming problematic after a dramatic increase in size in the last years. I am using a set of 3 copies of the same collection stored in 2.5 USB thin ...
- 1,067
0
votes
0
answers
501
views
what are the causes for "checksum does not match inode" HD problem, even if it says the inode check passed?
Root became RO after some minutes using the PC.
On reboot it required manual fsck.
Then this problem happened:
it said the inode check passed fine ok, but said this problem: "checksum does not ...
- 4,029
0
votes
1
answer
278
views
How can I create a checksum for a folder full of files that I am copying to another hard disk to verify it?
I am on a 2015 MacBook Pro and have just updated to OS 12.01 Monterey, and to zsh on terminal.
I am able to create a checksum for individual files using
shasum -a 256 (then drag in a file).
However, I ...
2
votes
1
answer
565
views
Compare zip file and extracted directory
What is the simplest way to compare a zip file and a directory that has been extracted from the zip file? I would like to delete the zip file if they are the same.
Example. Need to check if Archive....
- 121
16
votes
1
answer
2k
views
Splitting a small file into 512 byte segments changes it, but splitting it in 1k segments doesn't
So I'm trying to split a 64MB file FileCarve.001 into 512 byte segments (each block is 512 bytes long). I need to make sure the file has the same data when split into smaller files, so I cat all the ...
- 171
-3
votes
1
answer
612
views
Difference between diff and sum utilities
We use diff for finding difference between two files. We use checksum utilities for finding hashes and compare them later. In this regards, I have some questions :
Can we diff any type of file ? For ...
- 331
1
vote
0
answers
341
views
Checksum or validate .tar file contents with directory in shell
Scenario:
Using Linux shell to back up a directory of many sub files and compressing the data with ‘tar -cvpJf filename.tar /savedir/’.
Goal:
Using shell — checksum or validate the newly created ‘.tar’...
- 11
3
votes
1
answer
601
views
Two supposedly same M4A files are giving different hash results. Why?
Two files containing same song, both in M4A format, output two different results when I calculate their hashes:
md5sum
f149e2d2a232a410fcf61627578c101a new.m4a
ad26ed675342f0f45dcb5f9de9d586df old....
- 301
1
vote
0
answers
321
views
can md5sum check ignore some warnings?
When I run md5sum -c I get errors such as "No such file or directory". I only want checksum that did not match and ignore files that could not be read. --quiet gives me same result.
- 21
3
votes
3
answers
4k
views
How to show the incorrect checksum when the computed checksum did not match
Suppose I try to verify the checksum of a file using:
echo '760382d5e8cdc5d0d079e8f754bce1136fbe1473be24bb885669b0e38fc56aa3 emacs-26.1.tar.gz' | \
sha256sum --check
If the file is corrupt and the ...
- 2,426
1
vote
1
answer
143
views
Make sure two folders are identical on compressed zfs and ext4
Summary
I have copied (rsync --archive) a folder from an ext4 filesystem to a zfs file system with compression on. Now, I'm trying to verify that both folders are identical so that I can safely delete ...
- 201
3
votes
1
answer
136
views
Checksum of binary changes after rsync-ing to remote machine [closed]
I'm attempting to copy a binary to a remote machine using rsync, however it appears to corrupt in some way, with both file and md5sum noting the files difference, rsync on the other hand does not ...
- 51
2
votes
2
answers
651
views
How to rename a file to its creation date and hash value combined in order to achieve unique filename?
I have bunch of .JPG, .NEF and .MOV files made with my Nikon DSLR, named as DSC_0001.JPG, DSC_0002.JPG and so on. After certain number is reached, new directory is made and number count restarted, ...
- 301
0
votes
1
answer
309
views
MD6 hash generation linux command for 128-bit, 256-bit, 512-bit MD6 hashes
there are multiple checksum commands in the GNU Core Utilities in most Linux distros for famous hashing algorithms like SHA2 or MD5, But I need a command for MD6 (message digest 6) checksum that can ...
- 153
11
votes
5
answers
11k
views
Why doesn't changing a file's name change its checksum?
As far as I know changing even a bit of a file, will change the whole checksum result, but when I change a file's name this does not affect its checksum (I've tried SHA-1, SHA-256 and MD5).
Why? file ...
- 793
1
vote
1
answer
230
views
MX Linux ISO checksum mismatch
I went to the MX Linux website and their "Direct Download" linked me to their Sourceforge downloads page. I selected the first option - MX-19.2_September_x64.iso then checked the download ...
- 13
0
votes
1
answer
116
views
Check installed packages + check effective deployement of packages [to detect problems caused by memory corruption]
My computer has a flawed memory stick for two years and I can't correct this (sticks aren't removable and computer cannot be changed : so it won't be the subject).
Sometimes a content is corrupted ...
- 1,279
2
votes
1
answer
1k
views
sha1sum reports that checksums do not match even though they are identical
Perhaps I'm doing something very stupid here, but Google hasn't been very helpful in resolving the problem.
I have an archive of files I made for backup purposes. I've generated an SHA1 checksum file ...
- 183
0
votes
0
answers
32
views
how to check if someone override a directory?
Lets say i have an existing directory and someone did cp -r over this and some files changed. I want to detect this change. How can i do it ?
I thought of using md5 or some checksum calculator for ...
0
votes
1
answer
194
views
Can GNU tar incremental backup use checksums?
According to GNU tar manual:
Incremental dumps depend crucially on time stamps, so the results are unreliable if you modify a file's time stamps during dumping (e.g., with the `--atime-preserve=...
- 617
1
vote
0
answers
87
views
Is the output of `zfs send` deterministic?
My goal is to generate a single checksum for the entirety of a ZFS snapshot.
If zfs send is deterministic (including across systems, meaning including on the system that receives the snapshot), then ...
- 1,481
0
votes
1
answer
481
views
Checksum during rsync questions
Simple questions, I hope. if I'm running checksum (as a -c flag) on rsync, like so -
rsync -c --log-file="/location/logfile.txt" \
-e "ssh -p {port number}" {source} {destination}
will detected ...
- 11
1
vote
0
answers
232
views
Checksum comparison file too large
I am trying to run a checksum comparison task on a large folder that I rsync from one Synology server to another.
What I am trying to get is an output that shows me any differences between the ...
- 11
3
votes
0
answers
39
views
A_Parity_Error_?
Now this is odd: While copying a tarred directory over my LAN using ssh from a Fedora machine to a FreeBSD machine. I get: Uncorrectable parity/CRC error
(All the disks are set with S.M.A.R.T – ...
- 31
3
votes
1
answer
2k
views
Linux Live USB MD5 checksum integrity check failed for 5 files, are these files relevant?
When running manually the MD5 checksum integrity check from a USB Live Boot medium (Linux Mint 19), the command line shows "failed" for these 5 files:
grub.cfg
loopback.cfg
chain.c32
isolinux.cfg
...
- 131
1
vote
0
answers
50
views
Adding UDP checksum in IP layer after Xfrm transformation
I am doing a custom transformation of the packet using xfrm tables (IPv6). As part of the transformation, I am adding a UDP header (basically putting my data into an UDP a.k.a UDP encapsulation). The ...
- 11
0
votes
1
answer
284
views
How to calculate the checksum of a file's name?
I tried with openssl as it is very fast:
openssl sha1 "$(basename "very_big_image.png")"
But this just substitutes the actual file for check summing.
- 3
2
votes
0
answers
264
views
Why do we need filesystem block checksum?
Each sector on the disk has its own ECC to detect (and can sometimes correct) data corruption.
Yet some filesystems like Btrfs and ZFS provide checksum for each of their data block. Is it really ...
- 445
0
votes
2
answers
2k
views
Count the sum of wc -l command output
i have a workspace tree like :
/Directory
/Dir1/file1, file2
/Dir2/file3, file4
/Dir3/file5, file6
...
I want to count the sum of the number of lines for each file in dir.
I have this script,...
0
votes
1
answer
2k
views
Request file checksum from FTP server?
I am downloading some large files from a remote FTP server. Is it possible to request the server to send me checksums (e.g. sha256, or related variants) for the files in question?
- 1,607
0
votes
0
answers
461
views
How can I validate the integrity of a torrent retrieved through DHT?
I've just downloaded a dataset made available through BitTorrent and (at length) transferred it to my home PC.
Now I want to validate that nothing's been corrupted along the way.
Unfortunately, the ...
- 967
0
votes
1
answer
475
views
Need help to create a checksum file of a directory on linux
I backup my data to an external hard drive regularly, and in the past, some of the files ended up getting corrupted and I haven't noticed it until it was too late. So to prevent this, I would like to ...
user397449
4
votes
2
answers
703
views
File Corruption between 2 HDD
I have a NAS running Ubuntu 16.04 LTS server i386 (kernel 4.4.211-0404211-generic). I use it with samba to share movies and music through my home network.
I've tried to copy ~ 100 Go of data from an ...
- 41
4
votes
2
answers
3k
views
How to verify checksum file of centos 8?
How to verify that the CHECKSUM file for the CentOS 8 ISO is real?
Download check sum and key:
$ wget https://www.centos.org/keys/RPM-GPG-KEY-CentOS-Official
$ wget http://www.gtlib.gatech.edu/pub/...
1
vote
2
answers
459
views
unmount SD card makes file checksum change
I have an SD card mounted by Lubuntu at /media/$USERNAME/COREELEC/
I need to copy a file from one directory to another and rename it. However if I check the md5sum (or sha512sum) after I unmount it ...
- 455
1
vote
2
answers
142
views
How to calculate n values in a column and the overall average?
I would really appreciate some help with this as I am relativity new to Linux. I'm using grep but I also want an average of all of my extracted values (irrespective of which column they are in) and ...
- 55
7
votes
1
answer
658
views
Can files compressed with bzip2 be relied upon to be deterministic (reproducible)?
I am trying to determine if there are any potential issues using bzip2 to compress files that need to be 100% reproducible. Specifically: can metadata (name / inode, lastmod date, etc) or anything ...
- 248
3
votes
1
answer
438
views
What checksum algorithm is used by Version 5 Unix?
I can't figure out, what algorithm was used in Unix V5 and V6 for the sum command.
At first, I thought that it is a simple sum of bytes modulus 2^16. However for a string "1111111111\n" repeated 320 ...
- 528