Questions tagged [rsync]

rsync is a tool to efficiently copy directory hierarchies, locally or remotely, with powerful filters to decide what gets copied.

Filter by
Sorted by
Tagged with
410 votes
4 answers

Specify identity file (id_rsa) with rsync

I need to make periodic backups of a directory on a remote server which is a virtual machine hosted by a research organisation. They mandate that access to VMs is through ssh keys, which is all good, ...
Jangari's user avatar
  • 4,518
318 votes
10 answers

Can rsync resume after being interrupted?

I used rsync to copy a large number of files, but my OS (Ubuntu) restarted unexpectedly: sudo rsync -azvv /home/path/folder1/ /home/path/folder2 After reboot, I ran rsync again, but from the output ...
Tim's user avatar
  • 101k
259 votes
9 answers

scp without replacing existing files in the destination

How do I copy an entire directory into a directory of the same name without replacing the content in the destination directory? (instead, I would like to add to the contents of the destination folder)
user avatar
246 votes
5 answers

How to rsync only new files

I am trying to set up rsync to synchronize my main web server to the remote server by adding newly generated file to the latter. Here is the command that I use: rsync -avh --update -e "ssh -i /path/...
supermario's user avatar
  • 3,229
208 votes
11 answers

Rsync filter: copying one pattern only

I am trying to create a directory that will house all and only my PDFs compiled from LaTeX. I like keeping each project in a separate folder, all housed in a big folder called LaTeX. So I tried ...
Seamus's user avatar
  • 3,593
166 votes
1 answer

Understanding the output of --info=progress2 from rsync

If I run rsync with --info=progress2, I get an output like 105.45M 13% 602.83kB/s 0:02:50 (xfr#495, ir-chk=1020/3825) But what do the single numbers mean? I haven't found a matching entry in ...
muffel's user avatar
  • 2,808
138 votes
12 answers

How to rsync over ssh when directory names have spaces

I am trying to rsync from one server to another. The servers have the same directory structure but I am having trouble getting the remotes server to properly recognize the path name on the remote ...
PyNEwbie's user avatar
  • 1,559
130 votes
4 answers

How to rsync a directory to a new directory with different name?

I'm doing a data transfer, the old file system relies deeply on a directory which now is on different path. This is a git directory which stores code online. I have no rights to move it or rename it. ...
Zen's user avatar
  • 7,497
119 votes
7 answers

rsync compare directories?

Is it possible to compare two directories with rsync and only print the differences? There's a dry-run option, but when I increase verbosity to a certain level, every file compared is shown. ls -alR ...
chris's user avatar
  • 1,665
110 votes
8 answers

rsync ignore owner, group, time, and perms

I want to know how use rsync for sync to folders recursive but I only need to update the new files or the updated files (only the content not the owner, group or timestamp) and I want to delete the ...
alsanal's user avatar
  • 1,203
107 votes
7 answers

Does rsync verify files copied between two local drives?

I want to make a fresh new copy of a large number of files from one local drive to another. I've read that rsync does a checksum comparison of files when sending them to a remote machine over a ...
Frez's user avatar
  • 1,173
105 votes
8 answers

Using rsync to move (not copy) files between directories?

I have been using rsync to copy files for some time. My understanding is that rsync is faster than cp when some of the files to transfer are already in the destination directory, transferring only the ...
Amelio Vazquez-Reina's user avatar
99 votes
3 answers

How to rsync multiple source folders

I want to rsync multiple sources and I wonder the best way to achieve that. e.g. /etc/fstab /home/user/download I thought about 3 solutions : Solution 1 multiple call to rsync rsync -a /etc/...
user1437346's user avatar
  • 1,121
96 votes
15 answers

How to rsync files between two remotes?

I would like to transfer files between two remote hosts using on local shell, but it seems rsync doesn't support synchronisation if two remotes are specified as follow: $ rsync -vuar host1:/var/www ...
kenorb's user avatar
  • 20.9k
94 votes
9 answers

How to avoid password prompt with rsync (and without using public keys)?

I need to execute rsync, without it prompting me for password. I've seen in rsync manpage that it doesn't allow specifying the password as command line argument. But I noticed that it allows ...
Dor's user avatar
  • 2,525
88 votes
1 answer

Preserve the permissions with rsync

Let's say I have a file a.txt in LINUX with permission of 0664. When I use rsync to copy the file to my Mac with rsync -r -t -v LINUX MAC, the file's permission becomes 0644. How can I keep the ...
prosseek's user avatar
  • 8,528
86 votes
9 answers

Does compression option -z with rsync speed up backup

In rsync, --compress or -z will compress file data during the transfer. If I understand correctly, it compresses files before transfer and then decompress them after transfer. Does the time reduced ...
Tim's user avatar
  • 101k
82 votes
8 answers

rsync all files of remote machine over SSH without root user?

I have this command to backup a remote machine. The problem is that I need root rights to read and copy all files. I have no root user enabled for security reasons and use sudo the Ubuntu way. Would I ...
redanimalwar's user avatar
  • 1,057
77 votes
6 answers

Is there ever a reason to use scp instead of rsync?

Is there a reason to use scp instead of rsync? I can see no reason for using scp ever again, rsync does everything that scp does, with more safety (can preserve symlinks etc).
mikebloch's user avatar
  • 889
73 votes
1 answer

force rsync to overwrite files at destination even if they're newer

I have an rsync backup script I run, which also restores files back where they came from when I ask. But if the files at the destination are newer than those in the backup when I try to restore, it ...
jedipixel's user avatar
  • 839
72 votes
1 answer

How to exclude subdirectory from rsync?

I'm using rsync to recursively sync a remote folder tree that looks something like this: /folderA/a1/cache /folderA/a1/cache/A1 /folderA/a1/cache/A2 /folderA/a1/somefolder /folderA/a1/someotherfolder /...
TheEdge's user avatar
  • 823
68 votes
7 answers

"rsync: failed to set permissions on ..." error with rsync -a or -p option

When I use the -a option as is asked and answered in Preserve the permissions with rsync, I got a lot of "rsync: failed to set permissions on" errors. rsync: failed to set permissions on "/ata/text/...
prosseek's user avatar
  • 8,528
68 votes
2 answers

How to use rsync to backup a directory without git subdirectory

I want to copy my c directory with all subdirectories excluding ./git subdirectory. I do it using rsync : echo "copy c and sh files " rsync -a --include='*.c' --include='*.sh' --include='*/' --...
Adam's user avatar
  • 989
67 votes
2 answers

rsync "skipping non-regular file" when cloning my Aperture library

I am trying to use rsync to maintain a backup copy of my Aperture library. When I run the sync command to see what would happen this is the outcome: rsync --dry-run -r "/Volumes/Data/Aperture Library....
Nippysaurus's user avatar
67 votes
4 answers

How can I pause/resume rsync

How can I pause an rsync that's running? First time I did Ctrl+C to kill it and used the -P flag to run again. Is this prone to corrupt the current file transfer? Second time I simply put MacOS to ...
qnoid's user avatar
  • 833
66 votes
6 answers

Resume transfer of a single file by rsync

In Ubuntu, I want to copy a big file from my hard drive to a removable drive by rsync. For some other reason, the operation cannot complete in a single run. So I am trying to figure out how to use ...
Tim's user avatar
  • 101k
62 votes
4 answers

how to tell rsync to preserve time stamp on files when source tree has a mounted point

Related to this question Short description of the problem: When source tree has a mounted point inside it, then time stamps on files inside that mounted point when copied to target tree are not ...
Nasser's user avatar
  • 911
59 votes
5 answers

rsync exclude directory not working

I am running Fedora 17 64-bit and the rsync --exclude=/home/ben/<dir> is not working as expected. I am trying to rsync my home directory to a thumb drive, but I want to exclude certainly ...
Freedom_Ben's user avatar
  • 4,444
52 votes
1 answer

Delete extraneous files from dest dir via rsync?

Say I have SRC 001.jpg 002.jpg 001.txt DEST hello.jpg rsync -d --delete SRC:{*.jpg,*.txt} DEST It doesn't remove hello.jpg from DEST, any idea how to archive this?
Cheng's user avatar
  • 6,621
48 votes
3 answers

Rsync Error 23: Can I tell which files were not transferred?

I ran sudo rsync -va --progress from the root of one external drive to a folder on another external drive. The reason is that the source drive has an error-ful NTFS and I don't have access to a ...
themirror's user avatar
  • 6,968
48 votes
1 answer

What happens when you rsync without a destination?

To make the story short, I did an rsync: rsync -avP [email protected]:/tmp/ And I forgot to put in my source directory... I really meant to run: rsync -avP /tmp/ [email protected]:/tmp/ ...
Questionmark's user avatar
  • 3,925
42 votes
4 answers

How can I slow down rsync?

I'm trying to copy the contents of a failing USB thumb drive. If I read the data too fast, the drive's controller chip overheats and the drive vanishes from the system. When that happens, I need to ...
Mark's user avatar
  • 4,204
42 votes
1 answer

rsync to copy files which has changed

I need to copy some files from a particular directory to a different location on a daily basis. I want to copy the changes only, so planning to use rsync. These files follows the following naming ...
Zama Ques's user avatar
  • 3,286
41 votes
1 answer

Reuse rsync --dry-run output to speed up the actual transfer later on

I usually do rsync -n before doing an actual rsync. The thing is, how could I automate the second run to reuse exactly the list generated with --dry-run? I mean, if I run rsync -n then I get a ...
Manuel's user avatar
  • 513
37 votes
8 answers

Parallelise rsync using GNU Parallel

I have been using a rsync script to synchronize data at one host with the data at another host. The data has numerous small-sized files that contribute to almost 1.2TB. In order to sync those files, I ...
Mandar Shinde's user avatar
35 votes
1 answer

find and rsync?

I want to be able to search for files over 14 days and over 10k and than rsync those found files to a destination. Is there a way to combine these two commands? find ./ -mtime +14 -size +10k rsync -...
mkrouse's user avatar
  • 939
35 votes
1 answer

Reusing ssh session for repeated rsync commands [duplicate]

Possible Duplicate: Using an already established SSH channel I have a remote server accessible via a chain of SSH proxies. The resulting channel is wide and fast enough, but setting it up takes a ...
9000's user avatar
  • 1,629
35 votes
1 answer

rsync: colon in file names

Other than by renaming the files first, how can I rsync files that may have colons in their names?
WIZARDELF's user avatar
  • 548
34 votes
4 answers

How to copy directory structure without removing symlinks?

I need to "install" a bunch of files to another directory keeping the directory structure of the source files intact. For example, if I have ./foo/bar/baz.txt going to /var/www/localhost/webroot/ I ...
kojiro's user avatar
  • 4,634
34 votes
2 answers

Why is rsync not found?

rsync -avP /home/user/.profile hpux3:/home/user/.profile bash: rsync: command not found If I did ssh to hpux3 machine rsync version 3.1.1 protocol version 31 Copyright (C) 1996-2014 by Andrew ...
elbarna's user avatar
  • 12.6k
34 votes
5 answers

What is the need for rsync server in daemon mode

I don't understand the need for an rsync server in daemon mode. What are the benefits from it if I can use rsync with SSH or telnet?
Hanan's user avatar
  • 5,741
33 votes
1 answer

Does rsync delete files on the destination that were deleted from the sending side?

Does rsync delete files from the destination if they were deleted from the source? For example, say in the source /home/me I have three files: a.txt b.txt c.txt Now, I run the command rsync -v /...
perhapsmaybeharry's user avatar
32 votes
6 answers

Syncing directories in both directions with rsync

I happen to know about rsyn, and I use rsync to sync between my mac and a linux server as follows. rsync -r -t -v MAC LINUX rsync -r -t -v LINUX MAC I expected to run the first command to sync, but ...
prosseek's user avatar
  • 8,528
32 votes
1 answer

rsync recursively with a certain depth of subfolders

I want to rsync a folder recursively but want subfolders included only to a certain depth. For example I would want a depth of 1,2,3 or 4 subfolders like this: source/ ├── subfolder 1 │   ├── ...
nnn's user avatar
  • 1,181
31 votes
7 answers

tar + rsync + untar. Any speed benefit over just rsync?

I often find myself sending folders with 10K - 100K of files to a remote machine (within the same network on-campus). I was just wondering if there are reasons to believe that, tar + rsync + ...
Amelio Vazquez-Reina's user avatar
31 votes
2 answers

Asking rsync to delete files on the receiving side that don't exist on the sending side, with exceptions on the receiving side

I have read these threads: rsync --delete --files-from=list / dest/ does not delete unwanted files Delete extraneous files from dest dir via rsync? But, as far as I can tell (maybe I am missing ...
Amelio Vazquez-Reina's user avatar
30 votes
1 answer

Why rsync attempts to copy file that is already up-to-date?

I have two same files, on local machine and on remote one. Their sizes are equal, and file on local machine is newer than on remote one - but rsync still attempts to copy the file. I invoke rsync as ...
Rogach's user avatar
  • 6,273
29 votes
1 answer

Best way to sync files - copy only EXISTING files and only if NEWER than target

I'm doing this sync locally on Ubuntu 12.04. The files are generally small text files (code). I want to copy (preserving mtime stamp) from source directory to target but I only want to copy if the ...
MountainX's user avatar
  • 17.8k
27 votes
1 answer

Filesystem copied to new server is 60% bigger - why

I'm migrating a server from an Ubuntu Server 18.02 instance ("saturn") to a newly-built Debian Buster 10 system ("enceladus"). I have copied a complete filesystem across the ...
Henry Law's user avatar
  • 395
27 votes
8 answers

suppress rsync warning: some files vanished before they could be transferred

I get loads of warnings when backing up my running Postfix and Courier server files like: file has vanished: /var/kunden/mail/username/[email protected]/tmp/courier.lock How do I suppress those ...
rubo77's user avatar
  • 28.7k

2 3 4 5