Skip to main content

Questions tagged [checksum]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
2 votes
1 answer
66 views

Continue a directory tree checksum from a given file

I have: A checksum.txt file which contains many lines of checksums of single files from a mountpoint in a huge directory, which mounted and then it disconnected, thereby not finishing the checksum....
Sir Muffington's user avatar
1 vote
1 answer
37 views

Check SHA256SUMS and exit non-zero on unexpected file (file not present in digest)

I'm trying to check the integrity of a set of downloaded files using sha256sum. I cryptographically signed a digest file (named SHA256SUMS) with PGP. I create the file by recursively calculating the ...
Michael Altfield's user avatar
0 votes
1 answer
36 views

How do you synchronize files with partially failed download using rsync?

I'm trying to synchronize literally thousands of files of various sizes and I would like to have a 1:1 copy of the files. That means that already present files should be checked for their integrity ...
Sir Muffington's user avatar
0 votes
0 answers
72 views

BASH Get input from STDIN while using | (pipe)

I'm creating a new script. I would like to implement a way to verify this script. So I'm using those commands to check the content of the file. remote_file="$(curl -m2 -s "$1")" ...
olive007's user avatar
  • 131
0 votes
2 answers
131 views

Are there any filesystems with builtin data repairing via checksums?

I've read that ZFS/BtrFS have a checksum check, but they don't use it for data recovery, only for recovering data from a full local copy or a mirror copy. On the other hand, RAR archives support data ...
bodqhrohro's user avatar
1 vote
1 answer
47 views

See if any of a number of zip files contains any of the original files in a directory structure

I have a pretty hard problem here. I have a photo library with a lot of photos in it in various folders. I then started using Google Photos for my photos, I put those originals into Google Photos, and ...
Albert's user avatar
  • 161
0 votes
1 answer
110 views

What checksums are involved/recalculated in a ZFS send/recv?

I read a while back that a zfs send/recv is more immune to memory errors than a standard rsync, because it retains all of the zfs on disk checksums. I can no longer find this post, but the more I ...
zfspan's user avatar
  • 1
0 votes
0 answers
60 views

Linux corrupts customized TCP option

I am facing a peculiar problem with TCP options in k8s/Cilium/Kind. I have added a customized TCP option to the SYN packets being sent out from an endpoint using Cilium's bpf_lxc program (an eBPF ...
yeyushengfan's user avatar
0 votes
1 answer
412 views

gnome-software Unable to get list of updates: Failed to update metadata for lvfs: checksum failure

I sometimes use gnome-software to check for updates on Debian. My repositories are Debian Stable, Flathub and a few others each dedicated to a single program. When I do sudo apt-get update on Terminal ...
igi8276's user avatar
1 vote
1 answer
274 views

How to calculate a single checksum for each folder in a directory?

I am comparing two large directories between a server and a client machine. I run a command similar to the one below to determine if the two directories are the same or not. find /path/to/dir/ -type f ...
ricardo3889's user avatar
1 vote
1 answer
1k views

How to determine which checksum `rsync` uses by default?

In the rsync image of output below: I am unable to find the default checksum. Is it the first one, or the one in brackets? What is the bit-depth of just xxhash? Found this DOCUMENTATION states the ...
Neail's user avatar
  • 51
1 vote
1 answer
984 views

Using rsync to query the checksum of a remote file

Can we use rsync to simply get the checksum of a remote file, without transferring it? This could be useful for verifying the integrity of remote files without using a lot of bandwidth, if you don't ...
Nate Eldredge's user avatar
4 votes
1 answer
293 views

Checksumming on Ext4 (crc32c-intel) while formatting (5TB external HDD)

I bought a new 2.5-inch external hard drive of 5TB in size from Seagate. On my Linux Mint 21.1 now, I need to format the newly created partition with gdisk: Model: Expansion HDD Sector size (...
Vlastimil Burián's user avatar
2 votes
1 answer
189 views

checksum a pipeline

I want to construct a huge blob of data (a backup of a sort) and send it over the network (ssh or rsync) to another host. There is enough space on the remote for the data, but not on the local host, ...
q.undertow's user avatar
2 votes
2 answers
286 views

does rsync calculate a checksum even when it won't be transferring any data?

From man rsync: --existing skip creating new files on receiver (emphasis mine) So, what I've done with apparent success: rsync --existing -rtvhP "remote.machine:/photos/*" /photos This ...
captain's user avatar
  • 51
7 votes
4 answers
587 views

Record count and cksum on compressed file

There is a file of 40 GB in gz format. I want to find the record count and cksum of this file in uncompressed format. One approach I have is: Unzip the file using gunzip Use wc, cksum commands on ...
Gopi's user avatar
  • 183
1 vote
1 answer
314 views

rsync not detecting changed file with option --append-verify

I have invoked rsync with the options --checksum and --append-verify. My understanding of the processing for this combination of options has been that the application would behave as in the ...
brainchild's user avatar
2 votes
0 answers
2k views

Does the tar format support checksums?

TLDR, my main questions: Does the tar format itself support checksums? How to create a tar (or even better, tar.gz file) with the tar format built-in checksum? Long: Before you ask why I am asking or ...
adrelanos's user avatar
  • 1,876
1 vote
1 answer
2k views

How to verify the downloaded files by .torrent?

Many Ubuntu versions like 18.04.1, 18.04.2, 16.04.3, kubuntu, lubuntu, xubuntu, couldn't find it's checksum of .iso file from websites, but the .iso can be downloaded from a torrent file providers ...
user536180's user avatar
0 votes
2 answers
216 views

How to declare each outputs if there are multiple output in a single query in shell script?

I want to write a condition if the output matches by executing the below command. diff -is <(echo 'curl https://get.gravitational.com/teleport-v9.3.4-linux-amd64-bin.tar.gz.sha256') \ <(...
user123 codebit's user avatar
0 votes
1 answer
189 views

Looking for a checksum-based, poor-man's solution to fighting silent data corruption in long term storage of family video files

Secure storage of my collection of family video is becoming problematic after a dramatic increase in size in the last years. I am using a set of 3 copies of the same collection stored in 2.5 USB thin ...
Mephisto's user avatar
  • 1,087
0 votes
0 answers
991 views

what are the causes for "checksum does not match inode" HD problem, even if it says the inode check passed?

Root became RO after some minutes using the PC. On reboot it required manual fsck. Then this problem happened: it said the inode check passed fine ok, but said this problem: "checksum does not ...
Aquarius Power's user avatar
0 votes
1 answer
435 views

How can I create a checksum for a folder full of files that I am copying to another hard disk to verify it?

I am on a 2015 MacBook Pro and have just updated to OS 12.01 Monterey, and to zsh on terminal. I am able to create a checksum for individual files using shasum -a 256 (then drag in a file). However, I ...
TracyuFitz's user avatar
4 votes
1 answer
1k views

Compare zip file and extracted directory

What is the simplest way to compare a zip file and a directory that has been extracted from the zip file? I would like to delete the zip file if they are the same. Example. Need to check if Archive....
vpk's user avatar
  • 141
16 votes
1 answer
2k views

Splitting a small file into 512 byte segments changes it, but splitting it in 1k segments doesn't

So I'm trying to split a 64MB file FileCarve.001 into 512 byte segments (each block is 512 bytes long). I need to make sure the file has the same data when split into smaller files, so I cat all the ...
FlashDaggerX's user avatar
-3 votes
1 answer
1k views

Difference between diff and sum utilities

We use diff for finding difference between two files. We use checksum utilities for finding hashes and compare them later. In this regards, I have some questions : Can we diff any type of file ? For ...
cheesesticks's user avatar
1 vote
0 answers
538 views

Checksum or validate .tar file contents with directory in shell

Scenario: Using Linux shell to back up a directory of many sub files and compressing the data with ‘tar -cvpJf filename.tar /savedir/’. Goal: Using shell — checksum or validate the newly created ‘.tar’...
CosmicFiasco's user avatar
3 votes
1 answer
660 views

Two supposedly same M4A files are giving different hash results. Why?

Two files containing same song, both in M4A format, output two different results when I calculate their hashes: md5sum f149e2d2a232a410fcf61627578c101a new.m4a ad26ed675342f0f45dcb5f9de9d586df old....
Igor V.'s user avatar
  • 321
1 vote
0 answers
534 views

can md5sum check ignore some warnings?

When I run md5sum -c I get errors such as "No such file or directory". I only want checksum that did not match and ignore files that could not be read. --quiet gives me same result.
bilig james's user avatar
3 votes
3 answers
7k views

How to show the incorrect checksum when the computed checksum did not match

Suppose I try to verify the checksum of a file using: echo '760382d5e8cdc5d0d079e8f754bce1136fbe1473be24bb885669b0e38fc56aa3 emacs-26.1.tar.gz' | \ sha256sum --check If the file is corrupt and the ...
Flux's user avatar
  • 2,978
1 vote
1 answer
213 views

Make sure two folders are identical on compressed zfs and ext4

Summary I have copied (rsync --archive) a folder from an ext4 filesystem to a zfs file system with compression on. Now, I'm trying to verify that both folders are identical so that I can safely delete ...
fiedl's user avatar
  • 221
3 votes
1 answer
184 views

Checksum of binary changes after rsync-ing to remote machine [closed]

I'm attempting to copy a binary to a remote machine using rsync, however it appears to corrupt in some way, with both file and md5sum noting the files difference, rsync on the other hand does not ...
JoshuaO's user avatar
  • 51
2 votes
2 answers
886 views

How to rename a file to its creation date and hash value combined in order to achieve unique filename?

I have bunch of .JPG, .NEF and .MOV files made with my Nikon DSLR, named as DSC_0001.JPG, DSC_0002.JPG and so on. After certain number is reached, new directory is made and number count restarted, ...
Igor V.'s user avatar
  • 321
0 votes
1 answer
465 views

MD6 hash generation linux command for 128-bit, 256-bit, 512-bit MD6 hashes

there are multiple checksum commands in the GNU Core Utilities in most Linux distros for famous hashing algorithms like SHA2 or MD5, But I need a command for MD6 (message digest 6) checksum that can ...
Ibraheem Alyan's user avatar
12 votes
5 answers
13k views

Why doesn't changing a file's name change its checksum?

As far as I know changing even a bit of a file, will change the whole checksum result, but when I change a file's name this does not affect its checksum (I've tried SHA-1, SHA-256 and MD5). Why? file ...
Amir reza Riahi's user avatar
2 votes
1 answer
307 views

MX Linux ISO checksum mismatch

I went to the MX Linux website and their "Direct Download" linked me to their Sourceforge downloads page. I selected the first option - MX-19.2_September_x64.iso then checked the download ...
SonOfEl's user avatar
  • 23
0 votes
1 answer
248 views

Check installed packages + check effective deployement of packages [to detect problems caused by memory corruption]

My computer has a flawed memory stick for two years and I can't correct this (sticks aren't removable and computer cannot be changed : so it won't be the subject). Sometimes a content is corrupted ...
Marc Le Bihan's user avatar
2 votes
1 answer
2k views

sha1sum reports that checksums do not match even though they are identical

Perhaps I'm doing something very stupid here, but Google hasn't been very helpful in resolving the problem. I have an archive of files I made for backup purposes. I've generated an SHA1 checksum file ...
NoodleCollie's user avatar
0 votes
0 answers
45 views

how to check if someone override a directory?

Lets say i have an existing directory and someone did cp -r over this and some files changed. I want to detect this change. How can i do it ? I thought of using md5 or some checksum calculator for ...
deepak jha's user avatar
0 votes
1 answer
271 views

Can GNU tar incremental backup use checksums?

According to GNU tar manual: Incremental dumps depend crucially on time stamps, so the results are unreliable if you modify a file's time stamps during dumping (e.g., with the `--atime-preserve=...
qwr's user avatar
  • 741
1 vote
0 answers
107 views

Is the output of `zfs send` deterministic?

My goal is to generate a single checksum for the entirety of a ZFS snapshot. If zfs send is deterministic (including across systems, meaning including on the system that receives the snapshot), then ...
mpb's user avatar
  • 1,641
0 votes
1 answer
584 views

Checksum during rsync questions

Simple questions, I hope. if I'm running checksum (as a -c flag) on rsync, like so - rsync -c --log-file="/location/logfile.txt" \ -e "ssh -p {port number}" {source} {destination} will detected ...
JohnB42's user avatar
  • 11
1 vote
0 answers
287 views

Checksum comparison file too large

I am trying to run a checksum comparison task on a large folder that I rsync from one Synology server to another. What I am trying to get is an output that shows me any differences between the ...
JohnB42's user avatar
  • 11
3 votes
0 answers
40 views

A_Parity_Error_?

Now this is odd: While copying a tarred directory over my LAN using ssh from a Fedora machine to a FreeBSD machine. I get: Uncorrectable parity/CRC error (All the disks are set with S.M.A.R.T – ...
G_Man's user avatar
  • 31
3 votes
1 answer
2k views

Linux Live USB MD5 checksum integrity check failed for 5 files, are these files relevant?

When running manually the MD5 checksum integrity check from a USB Live Boot medium (Linux Mint 19), the command line shows "failed" for these 5 files: grub.cfg loopback.cfg chain.c32 isolinux.cfg ...
johnsmiththelird's user avatar
1 vote
0 answers
79 views

Adding UDP checksum in IP layer after Xfrm transformation

I am doing a custom transformation of the packet using xfrm tables (IPv6). As part of the transformation, I am adding a UDP header (basically putting my data into an UDP a.k.a UDP encapsulation). The ...
rkbug's user avatar
  • 11
0 votes
1 answer
698 views

How to calculate the checksum of a file's name?

I tried with openssl as it is very fast: openssl sha1 "$(basename "very_big_image.png")" But this just substitutes the actual file for check summing.
mustaqim's user avatar
2 votes
0 answers
367 views

Why do we need filesystem block checksum?

Each sector on the disk has its own ECC to detect (and can sometimes correct) data corruption. Yet some filesystems like Btrfs and ZFS provide checksum for each of their data block. Is it really ...
Livy's user avatar
  • 455
0 votes
2 answers
2k views

Count the sum of wc -l command output

i have a workspace tree like : /Directory /Dir1/file1, file2 /Dir2/file3, file4 /Dir3/file5, file6 ... I want to count the sum of the number of lines for each file in dir. I have this script,...
Kate Maserati's user avatar
0 votes
1 answer
4k views

Request file checksum from FTP server?

I am downloading some large files from a remote FTP server. Is it possible to request the server to send me checksums (e.g. sha256, or related variants) for the files in question?
a06e's user avatar
  • 1,747