6

There are various similar questions here, but I don't think any of them quite answer this.

I have a local directory (www) which has to be mirrored exactly on a remote server, and I need to confirm that they're exact copies.

www contains symlinks to other files inside www (the links are to both regular files and directories). This command will compare the two versions of www, excluding local svn directories as a bonus, and print only the different files:

[local]$ rsync -rvnc --exclude=.svn/ --delete www/ remote:/var/www/

But it doesn't follow symlinks, and just reports that they're not regular files. So, I need to dereference the symlinks, and compare their targets. Neither rsync -rvncL nor rsync -rvncK does this - -L prints out lots of files that don't differ (that I can see, anyway), and -K is doing nothing. Any ideas?

  • If your symlinks point to other files within www, the symlink targets themselves will (eventually) get copied independently of the symlinks. So they'll get checked as and of themselves, leaving the symlinks simply pointing to guaranteed-copied files. (In other words I don't think you need to worry - your rsync command is already doing the right thing.) – roaima Mar 19 '17 at 17:53
  • But unfortunately a symlink in local could point to file A, while the same symlink in remote could point to file B, which should be flagged as a failure. Not very likely, but I'd feel a lot better if I could actually check it, and dereferencing the symlink would do the check. – EML Mar 20 '17 at 8:32
  • it does get shown as a "failure". I can give you an answer that shows there's no problem. Any other explicit concerns? – roaima Mar 20 '17 at 8:53
  • I can't get it to do anything useful other than saying it's a non-regular file - if you think there's an option that actually derefences and shows mismatches as failures, then certainly post as an answer - that would be much appreciated - thanks – EML Mar 21 '17 at 8:11
  • I think you can't do better with rsync only, because either you sync the link, so it will tell if the link points to the same file (but in that case the file itself is not taken into account). Or you consider the target, and in that case you won't get the link. – Pierre-Alain TORET Apr 21 '17 at 12:39
0

Using normal commands with symlinks is very tricky. find command handles them pretty well. So the key is to use find, and I'd suggest using a fast CRC or cryptographic hash function depending on your needs.

So something like this should work (you can make supplements as needed)

find -L www -type f -exec cksum {} \; | cut -d ' ' -f1-2 | md5sum

If you want cryptographic backing for your checksums,

find -L www -type f -exec sha256sum {} \; | cut -d ' ' -f1 | sha256sum

The 'type' identifier is required because sha256 only works on files and errors out for directories, and the cut is only used to pass the checksum/hash to the final function and avoid false positives when in reality only the path names are different.

Note: This will fail in case the symlink is not relative and if the two systems don't have the exact same path which can happen if a symlink points outside the directory you are running find on.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.