221

I have a folder structure with a bunch of *.csv files scattered across the folders. Now I want to copy all *.csv files to another destination keeping the folder structure.

It works by doing:

cp --parents *.csv /target
cp --parents */*.csv" /target
cp --parents */*/*.csv /target
cp --parents */*/*/*.csv /target
...

and so on, but I would like to do it using one command.

7 Answers 7

229

find has a very handy -exec option:

find . -name '*.csv' -exec cp --parents \{\} /target \;
15
  • 82
    Probably because of this \{\} \;
    – igo
    Commented Mar 4, 2015 at 13:56
  • 18
    '{}' works just as well
    – OrangeDog
    Commented Dec 7, 2015 at 17:44
  • 3
    Even though -execdir is safer than -exec, simply replacing one with the other does not preserve the folder structure as intended.
    – sshine
    Commented Aug 23, 2016 at 11:33
  • 3
    Why I get message 'Omitting directory' when I try to copy them with your command ?
    – Vicky Dev
    Commented Sep 6, 2016 at 7:32
  • 8
    Can you explain why the braces have to be escaped here?
    – Noumenon
    Commented Oct 23, 2016 at 3:26
97

You could also use rsync for this.

$ rsync -a --prune-empty-dirs --include '*/' --include '*.csv' --exclude '*' source/ target/

If you want to keep empty directories from the source tree, skip the --prune-empty-dirs option:

$ rsync -a --include '*/' --include '*.csv' --exclude '*' source/ target/

If you do not want symlinks, modification dates, file permissions, owners etc. preserved, please replace -a with another combination of -rlptgoD. ;-)

3
  • 3
    -m is a shortcut for --prune-emty-dirs.
    – Geremia
    Commented Sep 24, 2016 at 0:59
  • 1
    Also, the -R option can be added to copy the parent directory structure of source. (cf. my answer here.)
    – Geremia
    Commented Sep 24, 2016 at 1:52
  • Unfortunately it seems that rsync is excruciating slow at finding matching files though. It takes like 100 more time than find. Is there anything that can be done about this?
    – Kvothe
    Commented May 11, 2023 at 14:38
68

You can use find and cpio in pass through mode

find . -name '*.csv' | cpio -pdm  /target

This will find all .csv files in the current directory and below and copy them to /target maintaining the directory structure rooted in ..

If you use

find /path/to/files -name '*.csv' | cpio -pdm /target

it will find all of the file in /path/to/files and below and copy them to /target/path/to/files and below.

4
  • 4
    I tried all the answers top down up to this one, and this was the only one that worked on the first attempt
    – OscarRyz
    Commented Jun 24, 2016 at 19:10
  • 2
    This should be the accepted answer. Commented Aug 25, 2020 at 0:30
  • 1
    I found the GNU docs helpful: "In copy-pass mode, [requested by the -p option, cpio] reads the list of files to copy from the standard input; the directory into which it will copy them is given as a non-option argument." ... "-d Create leading directories where needed." ... "-m Retain previous file modification times when creating files."
    – Nickolay
    Commented Sep 2, 2020 at 21:48
  • I want to do the same as the first line but copy all the contents of a folder and not only the csv files.
    – seralouk
    Commented Nov 15, 2022 at 12:39
39

The cp command permits multiple source arguments:

cp **/*.csv --parents ../target

CAVEAT: I'm using a recursive glob here; this is the globstar option in Bash 4+ and ksh, and is supported by default in zsh. Recursive globs do not match hidden files and folders, and the some implementations follow symlinks while others do not.

If your shell doesn't support recursive globs, or if you'd prefer not to use them, you can do the following:

  • *.csv */*.csv */*/*.csv */*/*/*.csv -- this is of course very redundant and requires knowing how deep your directory structure is.
  • $(find . -name '*.csv') -- This will match hidden files and folders. find also supports specifying whether or not symlinks are followed, which may be useful.
3
  • this is exactly what i tried (the recursive glob) and it found some but not all? pretty weird. I got the same exact result when using the npm copyfiles script but if i use the find command it finds everything...
    – Randyaa
    Commented Sep 2, 2016 at 15:22
  • @Randyaa I'll need some more details on which files, exactly, weren't found in order to help you. You may find the discussion here and continued here about the precise behavior of the recursive glob useful. Commented Sep 2, 2016 at 15:50
  • 1
    turns out recursive glob wasn't enabled for some reason... I've never run into this before but i corrected it with just an execution of shopt -s globstar immediately before my command and all is well. Thanks for the follow up!
    – Randyaa
    Commented Sep 2, 2016 at 15:52
32

This one worked for me:

find -name "*.csv" | xargs cp --parents -t /target

If you have file names with spaces, add options -print0 and -0 like suggested in one of the comments:

find -name "*.csv" -print0 | xargs -0 cp --parents -t /target

6
  • 2
    Best answer with most simple syntax. Works just fine and is easy to remember. In many cases find [things] | xargs [do stuff] is very powerfull.
    – oh really
    Commented Dec 4, 2018 at 7:56
  • Agreed. Very straightforward compared to some of the other answers.
    – Tim B
    Commented Mar 21, 2019 at 12:40
  • 2
    Breaks if you have spaces in filenames Commented Sep 16, 2019 at 14:58
  • 2
    @MichelePiccolini Spaces in filenames can be handled with find -print0 and xargs -0. Commented Sep 19, 2019 at 15:22
  • 1
    @pasztorpisti, is there any downside to that? If not should the answer not simply be edited to include your suggestion.
    – Kvothe
    Commented Mar 9, 2021 at 18:10
12

From rsync's manpage:

-R, --relative

Use relative paths. This means that the full path names specified on the command line are sent to the server rather than just the last parts of the filenames. This is particularly useful when you want to send several different directories at the same time. For example, if you used this command:

rsync -av /foo/bar/baz.c remote:/tmp/

... this would create a file named baz.c in /tmp/ on the remote machine. If instead you used

rsync -avR /foo/bar/baz.c remote:/tmp/

then a file named /tmp/foo/bar/baz.c would be created on the remote machine, preserving its full path. These extra path elements are called "implied directories" (i.e. the "foo" and the "foo/bar" directories in the above example).

So, this would work, too:

rsync -armR --include="*/" --include="*.csv" --exclude="*" /full/path/to/source/file(s) destination/
1
  • 1
    Thanks for providing this answer. Compatible with BSD userland (macOS), and paths with spaces in them.
    – myxal
    Commented Feb 14, 2020 at 9:57
9

Assuming you want to replicate this structure from ./source to ./destination:

cd source
find . -name "*.csv" | xargs tar cvf - | (cd ../destination ; tar xfp -)

I'm prepared to count that as one line, the cd source being a shell builtin.

1
  • tar has the -C option to avoid having to change the directory first.
    – bart
    Commented Nov 4, 2016 at 11:54

You must log in to answer this question.