I'm using diff -rq to compare directories on a Mac OS X 10.6 server. In folder1, we have 'filename'. In folder2, we have'filename.archive' because folder2 is created by a file sync app that also archives the files it syncs.

We've just pulled folder1 from a backup and I want to see what, in folder2, is more recent and could be copied over.

How would I do this? Specifically I'm asking how to do the diff in such a way that 'file' is evaluated equivalent to 'file.archive', and preferably by file content or checksum, not filename.


You cannot do this with "just" diff as the filenames do not match in both directories and diff doesn't have anything build in to do the mapping.

What you can do is change to the "new" directory and do:

for i in *; do diff "$i" /path/to/other/dir/"$i"; done

or, change to the restored backup dir and do:

for i in *.archive; do diff "$i" /path/to/new/dir/"${i%.archive}"; done

If this extends to a whole directory structure this becomes somewhat more complex. If the originals are under a and the backup restored temporarily under backup/b:

├── a
│   ├── p
│   │   ├── x
│   │   ├── y
│   │   └── z
│   └── q
│       ├── 1
│       ├── 2
│       └── 3
└── backup
    └── b
        ├── p
        │   ├── x.archive
        │   ├── y.archive
        │   └── z.archive
        └── q
            ├── 1.archive
            ├── 2.archive
            └── 3.archive

You can run the following script with python toddiff.py a backup/b > todiff (the todiff is not listed in the above tree structure).

import sys
import os

base = sys.argv[1]
if base[-1] != '/':
    base += '/'  # this last / needs to be removed to create bup_file_name
backup = sys.argv[2]

for root, directory_names, file_names in os.walk(base):
    for file_name in file_names:
        full_name = os.path.join(root, file_name)
        bup_file_name = os.path.join(backup, full_name.replace(base, '', 1))
        bup_file_name += '.archive'
        # adjust the diff options to your need in the following line
        print('diff -u "{}" "{}"'.format(full_name, bup_file_name))

This will generate a file of the form:

diff -u "a/q/3" "backup/b/q/3.archive"
diff -u "a/q/1" "backup/b/q/1.archive"
diff -u "a/q/2" "backup/b/q/2.archive"

which you can then source with: source ./todiff

In that case I would suggest your run the following program

  • It needs to be recursive though; there's dozens or hundreds of directories present that need to be combed through. – Harv Nov 13 '14 at 8:36
  • @Harv That can be solved in different ways, and the best way is depending on the circumstances. Do you expect many differences or are you looking for just a few files that have changed. Can you review all changes in one go? I'll try to expand my answer later today, but it would be good to have some better idea of your circumstances. – Anthon Nov 13 '14 at 9:28
  • We restored from a backup that was a few hours old. We also have a file syncing utility that when it syncs, archives the file it synced and appends a suffix. The syncing app runs live, and since the backup was a few hours old, we wanted to find out what was present in the sync archives that wasn't present (or simply different) in the backup archives. That will let us bring over only new/changed files from the sync archives, bringing our restored folder completely up to date. – Harv Nov 13 '14 at 19:04
  • @Harv I made a script that will generate the list of diff statements that needs to be executed, that should help you with tree structures that are of any depth. You can run the script in one go, or use split to generate chunks of e.g. 100 files to compare. – Anthon Nov 13 '14 at 21:11
  • Anthon, thanks for the effort you put into this! I didn't think it answered the question I asked, although you gave me a very useful tool that I could use. It sounds like the answer is that you can't use diff in the way I wanted to, the best you can do is generate a series of diff statements to compare one file to another and then maybe automate the running of those diff statements. Bummer. I will upvote this but if you submit an answer detailing that it can't be done, I will select that as the correct answer. Thanks again. – Harv Dec 6 '14 at 21:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.