On my local box "machineA", I have two folders "/primary" and "/secondary". These two folders have some files in it. Now on the remote server "machineB" I have one folder "/bat/snap/" which contains lot of files.

All the files in "/primary" and "/secondary" folders in "machineA" should be there in "machineB" remote server in this directory "/bat/snap/". Now I need to compare checksum of all files in "/primary" and "/secondary" folder on local box "machineA" with remote server in this directory "/bat/snap/". If there is any mismatch in checksum then I want to report all those files that has issue in "machineA"

Do I need to use md5checksum here?


This is the command I am running on "machineA":

find /primary/ /secondary/ -type f | xargs md5sum | ssh machineB '(cd /bat/snap/ && md5sum -c)' | egrep -v 'OK$'

Below is the error I am getting and after that I stopped my above command. I checked both the servers and I can see this file is present so what's wrong then?

md5sum: /primary/abc_monthly_134_proc_7.data: No such file or directory
/primary/abc_monthly_134_proc_7.data: FAILED open or read
  • Do you have, or can you install, the mtree program? – Mark Plotnick Apr 19 '18 at 21:28
  • I am running Ubuntu and I have sudo access so I can install it but is it needed both on source and destination servers or just source? – user1950349 Apr 19 '18 at 21:30
  • It needs to read both sets of files. You don't have to install it on both servers if you can share one server's files to the other over the network. – Mark Plotnick Apr 19 '18 at 21:34
  • Ok but I don't need to copy anything just compare the checksum of files that's all. And if anything mismatches then report all those files. – user1950349 Apr 19 '18 at 21:41
  • 1
    You just need to copy the mtree output file, which contains the filenames and checksums, from one system to the other. – Mark Plotnick Apr 19 '18 at 22:06

This is what the various md*sum files are written for.

On machine A:

find primary secondary -type f | xargs md5sum > checksum.md5

(copy file to machine B)

Machine B:

md5sum -c checksum.md5

Edit: Combined into a single command: find primary secondary -type f | xargs md5sum | ssh machineB '(cd /location_on_B/ && md5sum -c)' | egrep -v 'OK$'

(Another option is to tell rsync to run in dry-run mode with --checksum.)

| improve this answer | |
  • Can it be done in one command and figure out which files are having issues? Because I want to run one command on local box "machineA" only if this is possible by any chance? – user1950349 Apr 20 '18 at 3:41
  • Also "machineB" which is a remote server will have lot of files. I mean they will have "primary" and "secondary" files for "machineA" along with lot of other files. – user1950349 Apr 20 '18 at 3:54
  • It will only check the files listed in the checksum listing. I just combined it into one command and even verified it on a personal machine. – Aaron D. Marasco Apr 20 '18 at 10:19
  • If paths change, you'll need to do some perl/sed magic to manipulate the listing. – Aaron D. Marasco Apr 20 '18 at 10:21
  • after running the command, I am getting an error. I updated my question with details and I am not sure whats wrong? – user1950349 Apr 20 '18 at 20:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.