Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems. It's 100% free, no registration required.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I have 3 distinct folders: history, inbox, backup.

I need to copy all the files from 'history' to 'inbox', only if they are not present in 'backup'.

How to do this?

share|improve this question
What have you tried already? Do you have any scripting experience? – lynxlynxlynx Sep 28 '12 at 9:22
up vote 3 down vote accepted

Just an example, is there any subfolder in history ?

for x in history/*;
  [[ -f backup/"$(basename "$x")" ]] || cp "$x" inbox

This script would loop through all possible files in history folder, and extract the basename of it (e.g the basename of /bin/ls is ls), and check if the file exists in backup folder; if not, do the copy procedure.

share|improve this answer
It works great!! Thanks a lot... I don't understand the [[ -f backup/"$(basename "$x")" ]] part. Could you please explain me this? – Alban Soupper Sep 28 '12 at 8:23
@AlbanSoupper updated answer – warl0ck Sep 28 '12 at 8:24

This simple script works only if the three directories don't contain any subdirectories (a more complicated recursive algorithm would be required to handle that case).

Warlock's answer is good, but the advantage of a script like this over Warlock's one-liner is that by using xargs we don't fork an instance of cp for every file that we copy. We only fork one instance of cp for every time xargs generates a command line that fills the shell's command-line length limit. That tends to be huge on modern systems, so it's quite likely we'd only run one instance of cp total, even for hundreds or possibly thousands of files in history/

#! /bin/sh

# use tempfile(1) if available, otherwise use:
# EXCLUDEFILE="/tmp/excl.$$"
EXCLUDEFILE=$(tempfile -p excl)

# generate an exclude file, changing backup/ to history/    
find  backup/ -type f | sed -e 's:^backup/:history/:' > "$EXCLUDEFILE"

find history/ -type f -print0 | \
  grep -z -Z -F -v -x -f "$EXCLUDEFILE" | \
  xargs -0r -i{} cp {} inbox/


FYI, the long-form of the grep options used here are:

grep --null-data --null --fixed-strings --invert-match \
      --line-regexp --file "$EXCLUDEFILE"

and an even more compact form is:

grep -zZFvxf "$EXCLUDEFILE"

Most of the options are commonly used or self-explanatory, but three may require some explanation:

-z or --null-data tells grep to expect null-terminated input (from find -print0)
-Z or --null tells grep to output null-terminated lines (to pipe into xargs -0)
-x or --line-regexp tells grep to match only on the whole full lines

share|improve this answer
BTW, it wouldn't be difficult to come up with variations on this that used tar or rsync (that have built-in support for exclude-files) instead of cp. inf fact, using tar or rsync would be a good way to make the script work recursively if history/, inbox/, and backup/ contained subdirectories. – cas Sep 28 '12 at 9:04

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.