Copy directory but exclude files with certain names

Question

I'm not allowed to use rsync on the cluster I'm working on so I need to use things like cp or tar or find etc. I want to copy a large directory including all files and subfolders etc. while ignoring, i.e. not copying any files that for example have the names "file1", "file2" or "file3".

I saw this post on SO which allows me to include only certain files. I want basically that but I want to exclude certain files.

I also tried this solution but when I tried a command using tar which was suggested there I get the error tar: This does not look like a tar archive and tar: Cowardly refusing to create an empty archive

I also tried cp !(filename-to-ignore) new_dir/ from this post but this doesn't copy the subfolders.

I'm a beginner with bash so if you could explain your answer and what the flags etc. do that would really be helpfull, I've been trying forever now, on what I think shouldn't be that hard to do.

Answer 1

With GNU or libarchive's tar:

(cd src && tar --exclude='file[123]' -cf - .) |
  (cd dst && tar -xpf -)

Beware that GNU tar doesn't preserve the sparseness of files and depending on the version (or OS as some OS have some patches for that), not all extended attributes. libarchive's tar (often called bsdtar) does preserve extended attributes and the -S option upon extraction can be used to create sparse files.

That runs two subshells piped to each other. The first one generates a tar stream that represents the . (current) directory and all its contents recursively but ignoring the files whose name matches the test[123] pattern (similar to shell glob patterns, not regexps, here test followed by a character in the {1, 2, 3} set) after having changed the subshell's current working directory with cd. And the second has tar extract that stream on the fly in its own working directory (dst).

GNU tar also has a -C option to do that chdir() internally, as shown by Gilles in the Q&A you linked (there was a missing . in there which I've now added and explains your error), though that has little advantage over using the cd of your shell.

See also info -n exclude tar with GNU tar for the many different ways you can exclude files there and more generally info -n choosing tar to tune how files are selected/included in the archives it generates or extracts.