How do I only copy files to a remote folder on another server that don't already exist in the folder... from the command line in linux?

Question

I have two folders on two different servers.

I want to sync files between A and B, however, I only want to copy files that don't already exist in folder B because these files are huge. I don't care about updating files. I simply wants a copy of each in folder B.

How do I do this on Linux? (I suppose it'd be nice to know how to update files that have changed too)

Answer 1

rsync is able to do this.

rsync --ignore-existing <src> <dest>

You can perform also various kinds of updates. Just have a look at the man page.

Answer 2

Use rsync, and specify --ignore-existing if you do not want to update existing files:

rsync -e ssh -av --ignore-existing /my/source/directory [email protected]:/data/

A popular additional option I usually add is --progress so you see what's going on. If you expect interruptions, also say --partial, but I'm not sure how "partial" and "ignore-existing" mix if you do get cut off during a transmission.

Answer 3

Here's a trick that almost does this, but this trick only works when you are copying to folders on the same server. Below is the relevant content from the included link.

[jonesy@cranford testing]$ ls
bar  foo

[jonesy@cranford testing]$ ls foo
1  2  3

[jonesy@cranford testing]$ ls bar
1  2  3  4  5  6

[jonesy@cranford testing]$ yes n | cp -i bar/* foo 2>/dev/null

[jonesy@cranford testing]$ ls foo
1  2  3  4  5  6

[jonesy@cranford testing]$ ls bar
1  2  3  4  5  6

"I’m piping a constant “no” to the “cp -i” command, which asks you if you want to overwrite (or not) any files that already exist in the target directory. You don’t have to send STDERR to /dev/null — there’s just some messy output if you don’t. The end result is that the copy command will only copy files into the destination directory if they don’t already exist. It’ll skip files that exist in both directories (Well, technically the copy command won’t, but the operation will when you do it this way)."

"Of course, I could just forcibly overwrite the directory contents, but I don’t know if the files that exist in both directories are identical. Or I could move one sideways and the other into place, but the same issue exists."