Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

I have 3 distinct folders: history, inbox, backup.

I need to copy all the files from 'history' to 'inbox', only if they are not present in 'backup'.

How to do this?

share|improve this question
    
What have you tried already? Do you have any scripting experience? –  lynxlynxlynx Sep 28 '12 at 9:22
add comment

2 Answers

up vote 3 down vote accepted

Just an example, is there any subfolder in history ?

for x in history/*;
do
  [[ -f backup/"$(basename "$x")" ]] || cp "$x" inbox
done

This script would loop through all possible files in history folder, and extract the basename of it (e.g the basename of /bin/ls is ls), and check if the file exists in backup folder; if not, do the copy procedure.

share|improve this answer
    
It works great!! Thanks a lot... I don't understand the [[ -f backup/"$(basename "$x")" ]] part. Could you please explain me this? –  Alban Soupper Sep 28 '12 at 8:23
    
@AlbanSoupper updated answer –  warl0ck Sep 28 '12 at 8:24
add comment

This simple script works only if the three directories don't contain any subdirectories (a more complicated recursive algorithm would be required to handle that case).

Warlock's answer is good, but the advantage of a script like this over Warlock's one-liner is that by using xargs we don't fork an instance of cp for every file that we copy. We only fork one instance of cp for every time xargs generates a command line that fills the shell's command-line length limit. That tends to be huge on modern systems, so it's quite likely we'd only run one instance of cp total, even for hundreds or possibly thousands of files in history/

#! /bin/sh

# use tempfile(1) if available, otherwise use:
# EXCLUDEFILE="/tmp/excl.$$"
EXCLUDEFILE=$(tempfile -p excl)

# generate an exclude file, changing backup/ to history/    
find  backup/ -type f | sed -e 's:^backup/:history/:' > "$EXCLUDEFILE"

find history/ -type f -print0 | \
  grep -z -Z -F -v -x -f "$EXCLUDEFILE" | \
  xargs -0r -i{} cp {} inbox/

rm "$EXCLUDEFILE"

FYI, the long-form of the grep options used here are:

grep --null-data --null --fixed-strings --invert-match \
      --line-regexp --file "$EXCLUDEFILE"

and an even more compact form is:

grep -zZFvxf "$EXCLUDEFILE"

Most of the options are commonly used or self-explanatory, but three may require some explanation:

-z or --null-data tells grep to expect null-terminated input (from find -print0)
-Z or --null tells grep to output null-terminated lines (to pipe into xargs -0)
-x or --line-regexp tells grep to match only on the whole full lines

share|improve this answer
    
BTW, it wouldn't be difficult to come up with variations on this that used tar or rsync (that have built-in support for exclude-files) instead of cp. inf fact, using tar or rsync would be a good way to make the script work recursively if history/, inbox/, and backup/ contained subdirectories. –  cas Sep 28 '12 at 9:04
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.