97

I have a folder structure with a bunch of *.csv files scattered across the folders. Now I want to copy all *.csv files to another destination keeping the folder structure.

It works by doing:

cp --parents *.csv /target
cp --parents */*.csv" /target
cp --parents */*/*.csv /target
cp --parents */*/*/*.csv /target
...

and so on, but I would like to do it using one command.

migrated from serverfault.com Jul 19 '13 at 2:46

This question came from our site for system and network administrators.

12

This one worked for me:

find -name "*.csv" | xargs cp --parents -t /target

  • Best answer with most simple syntax. Works just fine and is easy to remember. In many cases find [things] | xargs [do stuff] is very powerfull. – Everyday Astronaut Dec 4 '18 at 7:56
  • Agreed. Very straightforward compared to some of the other answers. – Tim B Mar 21 at 12:40
7

From rsync's manpage:

-R, --relative

Use relative paths. This means that the full path names specified on the command line are sent to the server rather than just the last parts of the filenames. This is particularly useful when you want to send several different directories at the same time. For example, if you used this command:

rsync -av /foo/bar/baz.c remote:/tmp/

... this would create a file named baz.c in /tmp/ on the remote machine. If instead you used

rsync -avR /foo/bar/baz.c remote:/tmp/

then a file named /tmp/foo/bar/baz.c would be created on the remote machine, preserving its full path. These extra path elements are called "implied directories" (i.e. the "foo" and the "foo/bar" directories in the above example).

So, this would work, too:

rsync -armR --include="*/" --include="*.csv" --exclude="*" /full/path/to/source/file(s) destination/
19

The cp command permits multiple source arguments:

cp **/*.csv --parents ../target

CAVEAT: I'm using a recursive glob here; this is the globstar option in Bash 4+ and ksh, and is supported by default in zsh. Recursive globs do not match hidden files and folders, and the some implementations follow symlinks while others do not.

If your shell doesn't support recursive globs, or if you'd prefer not to use them, you can do the following:

  • *.csv */*.csv */*/*.csv */*/*/*.csv -- this is of course very redundant and requires knowing how deep your directory structure is.
  • $(find . -name '*.csv') -- This will match hidden files and folders. find also supports specifying whether or not symlinks are followed, which may be useful.
  • this is exactly what i tried (the recursive glob) and it found some but not all? pretty weird. I got the same exact result when using the npm copyfiles script but if i use the find command it finds everything... – Randyaa Sep 2 '16 at 15:22
  • @Randyaa I'll need some more details on which files, exactly, weren't found in order to help you. You may find the discussion here and continued here about the precise behavior of the recursive glob useful. – Kyle Strand Sep 2 '16 at 15:50
  • 1
    turns out recursive glob wasn't enabled for some reason... I've never run into this before but i corrected it with just an execution of shopt -s globstar immediately before my command and all is well. Thanks for the follow up! – Randyaa Sep 2 '16 at 15:52
112

Is there any reason why people resist using find's -exec? It's very handy.

find . -name '*.csv' -exec cp --parents \{\} /target \;

Know your tools. ;-)

  • 60
    Probably because of this \{\} \; – igo Mar 4 '15 at 13:56
  • 11
    '{}' works just as well – OrangeDog Dec 7 '15 at 17:44
  • 3
    Even though -execdir is safer than -exec, simply replacing one with the other does not preserve the folder structure as intended. – Simon Shine Aug 23 '16 at 11:33
  • 2
    Why I get message 'Omitting directory' when I try to copy them with your command ? – Vicky Dev Sep 6 '16 at 7:32
  • 4
    Can you explain why the braces have to be escaped here? – Noumenon Oct 23 '16 at 3:26
41

You could also use rsync for this.

$ rsync -a --prune-empty-dirs --include '*/' --include '*.csv' --exclude '*' source/ target/

If you want to keep empty directories from the source tree, skip the --prune-empty-dirs option:

$ rsync -a --include '*/' --include '*.csv' --exclude '*' source/ target/

If you do not want symlinks, modification dates, file permissions, owners etc. preserved, please replace -a with another combination of -rlptgoD. ;-)

  • 1
    -m is a shortcut for --prune-emty-dirs. – Geremia Sep 24 '16 at 0:59
  • Also, the -R option can be added to copy the parent directory structure of source. (cf. my answer here.) – Geremia Sep 24 '16 at 1:52
31

You can use find and cpio in pass through mode

find . -name '*.csv' | cpio -pdm  /target

This will find all .csv files in the current directory and below and copy them to /target maintaining the directory structure rooted in ..

If you use

find /path/to/files -name '*.csv' | cpio -pdm /target

it will find all of the file in /path/to/files and below and copy them to /target/path/to/files and below.

  • 3
    I tried all the answers top down up to this one, and this was the only one that worked on the first attempt – OscarRyz Jun 24 '16 at 19:10
8

Assuming you want to replicate this structure from ./source to ./destination:

cd source
find . -name "*.csv" | xargs tar cvf - | (cd ../destination ; tar xfp -)

I'm prepared to count that as one line, the cd source being a shell builtin.

  • 2
    I don't think he really means one command - just not having to fag about like his example ;) – Iain Jul 18 '13 at 15:04
  • tar has the -C option to avoid having to change the directory first. – bart Nov 4 '16 at 11:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy