Create hardlinks instead of copying files with rsync

Question

The following command finds recursively all files containing foo (but not bar) and preserves the directory structure in the target directory, for example if I have /path/to/2017/foo.ps and /path/to/2018/foo.pdf it creates the subdirectories 2017 and 2018 under ~/path/to/dir/ containing foo.ps resp. foo.pdf.

find . -iname \*foo\*pdf -exec rsync -R --exclude='*bar*' {} ~/path/to/dir/ \;

What is the equivalent to this command such that I end up with hardlinks instead of copies?

Answer 1

Since no one else did, to answer the title question, you can use --link-dest. Generally, if you want to sync s/ to d/ with rsync but using hardlinks instead of copies, you can do:

rsync -r --link-dest="$PWD/s/" s/ d/

As to your specific need, where you want to specify files and not whole directories, you can do:

find . \
  -iname \*foo\*pdf \
  -exec bash -c '
    rsync -R \
      --exclude="*bar*" \
      --link-dest="$PWD/$(dirname "$1")" \
      "$1" \
      ~/path/to/dir/
  ' sh {} \;

The reason why bash is introduced here is to be able to get the dirname of the filepath. --link-dest needs the directory and not the file itself. I also use $PWD because --link-dest is relative to the destination directory, not the current directory.

Answer 2

This doesn't look very much like a job for rsync. Rsync is mostly useful for recursive copies, but evidently you only intend to copy regular files. The only advantage of rsync over cp here is that it creates the parent directories as needed. You could do it with a small shell script calling mkdir instead.

find . -type f -iname '*foo*.pdf' ! -name '*bar*' \
       -exec sh -c 'mkdir -p "~/path/to/dir/${0%/*}" && cp "$0" ~/path/to/dir/"$0"' {} \;

• Add -type f to the find command to exclude non-regular files (e.g. directories). Use -xtype f to also process symbolic links to regular files.
• ! -name '*bar*' is equivalent to the rsync --exclude option.
• sh -c '…' {} executes the shell script … with $0 set to the file name found by find.
• mkdir -p … creates the target directory and any parent as necessary.

Here you can substitute ln for cp to create a hard link.

---

Alternatively you can use zsh's zmv function. In zsh:

setopt extended_glob
autoload zmv
mkdir_ln () {
  mkdir -p -- $2:h
  ln -- $1 $2
}
zmv -p mkdir_ln '**/(#i)*foo*.pdf~**/*bar*' ~/path/to/dir/'$f'

• zmv -p mkdir_ln PATTERN REPLACEMENT calls mkdir_ln FILENAME REPLACEMENT for each FILENAME that matches PATTERN, with $f in REPLACEMENT replaced by FILENAME.
• mkdir -p -- $2:h creates the directory part of $2 and any parent directory as needed.
• **/… matches files in subdirectories recursively.
• (#i) starts a case-insensitive match.
• ~**/*bar* excludes files whose name contains bar.

Answer 3

Sounds like a job for pax:

find . -iname '*foo*pdf' ! -name '*bar*' -print0 |
  pax -0rwl ~/path/to/dir/

If your pax doesn't support the non-standard -0, remove the 0 from -print0 and -0rwl but beware that won't work properly if you have file paths that contain newline character.

If you don't have pax (POSIX, but not GNU), try replacing with cpio -0pd (GNU, but not POSIX).