I would like to copy a set of files from directory A to directory B, with the caveat that if a file in directory A is identical to a file in directory B, that file should not be copied (and thus its modification time should not be updated). Is there a way to do that with existing tools, without writing my own script to do it?

To elaborate a bit on my use-case: I am autogenerating a bunch of .c files in a temporary directory (by a method that has to generate all of them unconditionally), and when I re-generate them, I'd like to copy only the ones that have changed into the actual source directory, leaving the unchanged ones untouched (with their old creation times) so that make will know that it doesn't need to recompile them. (Not all the generated files are .c files, though, so I need to do binary comparisons rather than text comparisons.)

(As a note: This grew out of the question I asked on https://stackoverflow.com/questions/8981552/speeding-up-file-comparions-with-cmp-on-cygwin/8981762#8981762, where I was trying to speed up the script file I was using to do this operation, but it occurs to me that I really should ask if there's a a better way to do this than writing my own script -- especially since any simple way of doing this in a shell script will invoke something like cmp on every pair of files, and starting all those processes takes too long.)


7 Answers 7


You can use the -u switch to cp like so:

$ cp -u [source] [destination]

From the man page:

   -u, --update
       copy only when the SOURCE file is newer than the destination file or 
       when the destination file is missing
  • 6
    From a comment on a similar A that was already deleted: "This will not work since it would copy also identical files, if timestamp of source is newer (and so update timestamp of destination, against the OP request)."
    – slm
    Jun 27, 2014 at 17:48
  • 2
    Doesn't answer the question at all, but I still found it useful.
    – user31389
    Oct 21, 2016 at 11:19
  • 1
    Not available in macOS (11.2) Bash, unfortunately.
    – akauppi
    Apr 30, 2021 at 9:06

rsync is probably the best tool for this. There are a lot of options on this command so read man page. I think you want the --checksum option or the --ignore-times

  • I should have noted that I already tried that, with no success. Both of those options only affect whether rsync does a copy -- but, even when it does not do a copy, it either updates the target file's modification time to the same as the source (if the -t option is specified) or to the synchronization time (if -t is not specified). Jan 24, 2012 at 5:52
  • 4
    @Brooks Moses: It doesn't. At least my version of rsync doesn't. If I do this: mkdir src dest; echo a>src/a; rsync -c src/* dest; sleep 5; touch src/a; rsync -c src/* dest, then stat dest/a shows its mtime and ctime are 5 secs older than the ones of src/a.
    – angus
    Jan 24, 2012 at 8:48
  • 2
    @BrooksMoses: I think you can rely on it: rsync's workflow is: 1) check if the file need to be updated; 2) if so, update the file. The --checksum option say it should not be updated, so rsync should not proceed to step 2).
    – enzotib
    Jan 24, 2012 at 10:25
  • 2
    @BrooksMoses: --ignore-times without --checksum would copy every file, and so also update the timestamp, even if the files are identical.
    – enzotib
    Jan 24, 2012 at 21:22
  • 8
    This answer's high rating is unwarranted. Saying "read the man page" is barely more helpful than saying "go google the answer".
    – Dennis
    Aug 22, 2019 at 17:22

While using rsync --checksum is a good general way to "copy if changed", in your particular case there is an even better solution!

If you want to avoid unnecessarily recompiling files you should use ccache which was built for exactly this purpose! In fact, not only will it avoid unnecessary recompiles of your auto-generated files, it will also speed things up whenever you do make clean and re-compile from scratch.

Next I'm sure you'll ask, "Is it safe?" Well, yes, as the website points out:

Is it safe?

Yes. The most important aspect of a compiler cache is to always produce exactly the same output that the real compiler would produce. This includes providing exactly the same object files and exactly the same compiler warnings that would be produced if you use the real compiler. The only way you should be able to tell that you are using ccache is the speed.

And it's easy to use it by just adding it as a prefix in the CC= line of your makefile (or you can use symlinks, but the makefile way is probably better).

  • 1
    I initially misunderstood and thought you were suggesting I use ccache to do part of the generating, but now I understand -- your suggestion was that I simply copy all the files, and then use ccache in the build process, thereby avoiding rebuilding the ones that hadn't changed. It's a good idea, but it won't do well in my case -- I have hundreds of files, usually only change one or two at a time, and am running under Cygwin where simply starting the hundreds of ccache processes to look at each file would take several minutes. Nonetheless, upvoted because it's a good answer for most people! Jan 31, 2012 at 6:14
  • No, I was not suggesting that you copy all the files, rather you can just autogenerate your .c files in-place (remove the copy step and write to them directly). And then just use ccache. I don't know what you mean by starting hundreds of ccache processes... it is just a light-weight wrapper around gcc that is quite fast and will speed up re-building other parts of your project, too. Have you tried using it? I would like to see a comparison of the timing between using your copy-method vs ccache. You could, in fact, combine the two methods to get the benefits of both.
    – aculich
    Jan 31, 2012 at 16:07
  • 1
    Right, ok, I understand now about the copying. To clarify, what I mean is this: If I generate the files in place, I have to then call ccache file.c -o file.o or the equivalent, several hundreds of times because there are several hundred file.c files. When I was doing that with cmp, rather than ccache, it took several minutes -- and cmp is as lightweight as ccache. The problem is that, on Cygwin, starting a process takes non-negligible time, even for a completely trivial process. Feb 1, 2012 at 3:57
  • 1
    As a datapoint, for f in src/*; do /bin/true.exe; done takes 30 seconds, so yeah. Anyway, I prefer my Windows-based editor, and aside from this sort of timing issue Cygwin works quite well with my workflow as the lightweight place to test things locally if I'm not uploading to the build servers. It's useful to have my shell and my editor in the same OS. :) Feb 1, 2012 at 4:43
  • 1
    If you want to use your Windows-based editor you can do that quite easily with Shared Folders if you install Guest Additions... but hey, if Cygwin suits you, then who am I to say any different? It just seems a shame to have to jump through weird hoops like this... and compilation in general would be faster in a VM, too.
    – aculich
    Feb 1, 2012 at 4:55

This should do what you need

diff -qr ./x ./y | awk '{print $2}' | xargs -n1 -J% cp % ./y/


  • x is your updated/new folder
  • y is the destination you want to copy to
  • awk will take the second argument of the each line from the diff command (maybe you will need some extra stuff for filenames with space - can't try it now)
  • xargs -J% will insert the file name to cp at the proper place
  • 2
    -1 because this is overly-complicated, non-portable (-J is bsd-specific; with GNU xargs it is -I), and does not work correctly if the same set of files do not exist in both locations already (if I touch x/boo then grep gives me Only in ./x: boo which causes errors in the pipeline). Use a tool built for the job, like rsync --checksum.
    – aculich
    Jan 30, 2012 at 23:24
  • Or better yet, for this specific case use ccache.
    – aculich
    Jan 31, 2012 at 0:06
  • +1 because its a set of well known commands that I can break to use on similar tasks (came here for doing a diff), still rsync may be better for this particular task
    – ntg
    Nov 20, 2017 at 8:22

I like to use unison in favor of rsync because it supports multiple masters, having already setup my ssh keys and vpn separately.

So in my crontab of only one host I let them synchronize every 15 minutes:

*/15 * * * * [ -z "$(pidof unison)" ] && (timeout 25m unison -sortbysize -ui text -batch -times /home/master ssh:// -path dev -logfile /tmp/sync.master.dev.log) &> /tmp/sync.master.dev.log

Then I can be developing on either side and the changes will propagate. In fact for important projects I have up to 4 servers mirroring the same tree (3 run unison from cron, pointing to the one that doesn't). In fact, Linux and Cygwin hosts mixed--except don't expect sense out of soft links in win32 outside the cygwin environment.

If you go this route, make the initial mirror on the empty side without the -batch, i.e.

unison -ui text  -times /home/master ssh:// -path dev

Of course there is a config to ignore backup files, archives, etc.:

 ~/.unison/default.prf :
# Unison preferences file
ignore = Name {,.}*{.sh~}
ignore = Name {,.}*{.rb~}
ignore = Name {,.}*{.bak}
ignore = Name {,.}*{.tmp}
ignore = Name {,.}*{.txt~}
ignore = Name {,.}*{.pl~}
ignore = Name {.unison.}*
ignore = Name {,.}*{.zip}

    # Use this command for displaying diffs
    diff = diff -y -W 79 --suppress-common-lines

    ignore = Name *~
    ignore = Name .*~
    ignore = Path */pilot/backup/Archive_*
    ignore = Name *.o
  • I looked at that, but I couldn't find a unison option that means "don't update file-last-modified dates". Is there one? Otherwise, this is a great answer to an entirely different problem. Feb 2, 2012 at 20:46
  • 1
    -times does that for me. Unison has a dry-run mode too, me thinks.
    – Marcos
    Feb 2, 2012 at 22:29
  • Well, setting times=false (or leaving off -times) would do that. I don't know how I missed that in the documentation before. Thanks! Feb 2, 2012 at 23:56
  • Glad to help. I'm a stickler when it comes to preserving things like modtimes, permissions and soft links. Often overlooked
    – Marcos
    Feb 3, 2012 at 16:44

While rsync --checksum is the correct answer, note that this option is incompatible with --times, and that --archive includes --times, so if you want to rsync -a --checksum, you really need to rsync -a --no-times --checksum.

  • What do you mean by saying 'incompatible'?
    – o.v
    Jan 10, 2019 at 12:44

This is the perfect answer: numbered backup file and copy only if source (also multiple, also directories) and destination file doesn't matches, with the possibility to choose to do backup recursively or not (with the depth level of your choice or unlimited) (default: not recursively), in any case the copy (no-clobber) is always recursive (unlimited depth):

export NULLGLOB="$(shopt -p nullglob)"

copy () {
    command cp -a --no-preserve=mode,ownership --remove-destination "$@"
    return $?

backup-and-copy () {
    local exit_code
    local error_message
    local i
    local maxdepth=0
    local submaxdepth=-1
    local abs
    for (( i=1; i<=$#; i++ )); do
        if [[ ${!i} == -- ]]; then
            command set -- "${@:1:i-1}" "${@:i+1}"
        if [[ ${!i,,} == -r ]]; then
            command set -- "${@:1:i-1}" "${@:i+1}"
            if [[ ${!i} =~ ^-?[0-9]+$ ]]; then
                if [[ ${#abs} -gt 9 ]]; then
                    echo "backup-and-copy: $maxdepth: Numerical result out of range=[-999999999, 999999999]"
                    return 1
                command set -- "${@:1:i-1}" "${@:i+1}"
    if [[ $maxdepth -gt 0 ]]; then submaxdepth=$((maxdepth-1)); fi
    error_message="$(cp -n -- "$@" 2>&1)"
    if [[ $exit_code -ne 0 ]]; then
        echo "backup-and-copy: ${error_message}"
        return $exit_code
    if [[ -d ${!#} ]]; then
        for (( i=1; i<$#; i++ )); do
            if [[ -d ${!i} ]]; then
                if [[ $maxdepth -ne 0 ]]; then
                    shopt -s nullglob
                    backup-and-copy -R $submaxdepth -- "${!i}/"{,.[^.],..?}* "${!#}/$(command basename "${!i}")"
                command cmp -s -- "${!i}" "${!#}/$(command basename "${!i}")" 2> /dev/null || cp --backup=numbered -- "${!i}" "${!#}"
        command cmp -s -- "$@" 2> /dev/null || cp --backup=numbered -- "$@"
    return $exit_code

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .