3

Is it possible, with find and rsync, to recursively copy a specific file type, but ignore the directory structure when writing to the destination? In other words, to copy every zip file from every subdirectory under the source, and drop them all into a single destination directory. The destination is an NTFS mount on the same machine being shared via Samba. Because source files may change or be removed, I'd like to use rsync.

I've been trying to put together something along these lines but am unsure if I'm on the right track, or if this is the best approach:

usr/bin/find /SRC -name "*.zip" -exec usr/bin/rsync -vgoth --delete {} /DEST/ \;

I'm using full paths because the command will eventually be run from a shell script executed via Crontab.

  • You might want to look into the --include filter rule available with rsync. man rsync should get you the info you need. Although something like rsync /SRC --include="*.zip" --delete destination:/patch/to/SRC/ should probably get you most of the way there. – Thomas N Feb 20 '17 at 19:16
  • One thing I'd like to understand is how you are going to sort out namespace collisions? rsync will overwrite the destination directory, unless you tell it not to, so you could end up losing data in this model, if two files happen to be named the same. – Thomas N Feb 20 '17 at 19:23
  • I probably should have included that info in the original question. The source files I am working with are all uniquely named, so there's no chance of accidental overwrites. – graphics.guy Feb 20 '17 at 19:38
  • If you don't find a solution here, you may want to ask on the rsync list lists.samba.org/mailman/listinfo/rsync They are very helpful and the developers answer a lot of questions. I have never seen a way to consolidate files like that. On second thought, since both source and destination are mounted on the same machine. It may even be slower. cp -au source dest will work better, but you can also just add a -exec to your find command to put the files anywhere you want to. – Joe Feb 24 '17 at 20:14
1

I don't know if I misunderstood your question but the line that you provide is working (at least on my side with rsync 3.1.2)!

$ tree
.
├── file1.zip
├── folder1
│   └── file2.zip
└── folder2
    └── subfolder
        └── file3.zip

3 directories, 3 files

$ tree $BACKUP_DESTINATION
0 directories, 0 files

$ find . -iname "*.zip" -exec rsync -vgoth {} ${BACKUP_DESTINATION} \;
[...]

$ tree $BACKUP_DESTINATION
.
├── file1.zip
├── file2.zip
└── file3.zip

0 directories, 3 files

As you can see the 3 zip files were copied without their path even though they were in different folders.

Did you find a better solution? (Maybe on the rsync list as suggested by Joe)

(Note: As BowlOfRed pointed out in his comment, the --delete is missing from my test (whereas you used it in your original suggestion). If I use this option, rsync is failing because it complains --recursive or --dirs is missing. This error exists only from version 2.6.7 so maybe you were using an old rsync version. Also you did not mention anything about deleting files in the destination so I don't know if you want to keep this option.)

  • Your command differs from the OP in that you don't have --delete. I'm not sure what graphics.guy thought it would do, but it can't be there. Otherwise any later rsync run is going to delete previously copied files that aren't in its tree. – BowlOfRed Sep 20 '17 at 21:33
  • You're right --delete is missing. I just did a test and it seems --delete makes the command fail because either --recursive or --dirs is missing. Apparently this error would not be displayed prior version 2.6.7 of rsync so maybe OP was using an old version. Anyway thank you for pointing this out, I will edit my answer to mention that. – Tom Sep 21 '17 at 17:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.