Questions tagged [rsync]

rsync is a tool to efficiently copy directory hierarchies, locally or remotely, with powerful filters to decide what gets copied.

Filter by
Sorted by
Tagged with
410 votes
4 answers
458k views

Specify identity file (id_rsa) with rsync

I need to make periodic backups of a directory on a remote server which is a virtual machine hosted by a research organisation. They mandate that access to VMs is through ssh keys, which is all good, ...
Jangari's user avatar
  • 4,518
318 votes
10 answers
329k views

Can rsync resume after being interrupted?

I used rsync to copy a large number of files, but my OS (Ubuntu) restarted unexpectedly: sudo rsync -azvv /home/path/folder1/ /home/path/folder2 After reboot, I ran rsync again, but from the output ...
Tim's user avatar
  • 101k
259 votes
9 answers
373k views

scp without replacing existing files in the destination

How do I copy an entire directory into a directory of the same name without replacing the content in the destination directory? (instead, I would like to add to the contents of the destination folder)
user avatar
246 votes
5 answers
434k views

How to rsync only new files

I am trying to set up rsync to synchronize my main web server to the remote server by adding newly generated file to the latter. Here is the command that I use: rsync -avh --update -e "ssh -i /path/...
supermario's user avatar
  • 3,229
208 votes
11 answers
239k views

Rsync filter: copying one pattern only

I am trying to create a directory that will house all and only my PDFs compiled from LaTeX. I like keeping each project in a separate folder, all housed in a big folder called LaTeX. So I tried ...
Seamus's user avatar
  • 3,593
166 votes
1 answer
94k views

Understanding the output of --info=progress2 from rsync

If I run rsync with --info=progress2, I get an output like 105.45M 13% 602.83kB/s 0:02:50 (xfr#495, ir-chk=1020/3825) But what do the single numbers mean? I haven't found a matching entry in ...
muffel's user avatar
  • 2,808
138 votes
12 answers
124k views

How to rsync over ssh when directory names have spaces

I am trying to rsync from one server to another. The servers have the same directory structure but I am having trouble getting the remotes server to properly recognize the path name on the remote ...
PyNEwbie's user avatar
  • 1,559
130 votes
4 answers
132k views

How to rsync a directory to a new directory with different name?

I'm doing a data transfer, the old file system relies deeply on a directory which now is on different path. This is a git directory which stores code online. I have no rights to move it or rename it. ...
Zen's user avatar
  • 7,497
119 votes
7 answers
179k views

rsync compare directories?

Is it possible to compare two directories with rsync and only print the differences? There's a dry-run option, but when I increase verbosity to a certain level, every file compared is shown. ls -alR ...
chris's user avatar
  • 1,665
110 votes
8 answers
184k views

rsync ignore owner, group, time, and perms

I want to know how use rsync for sync to folders recursive but I only need to update the new files or the updated files (only the content not the owner, group or timestamp) and I want to delete the ...
alsanal's user avatar
  • 1,203
107 votes
7 answers
124k views

Does rsync verify files copied between two local drives?

I want to make a fresh new copy of a large number of files from one local drive to another. I've read that rsync does a checksum comparison of files when sending them to a remote machine over a ...
Frez's user avatar
  • 1,173
105 votes
8 answers
172k views

Using rsync to move (not copy) files between directories?

I have been using rsync to copy files for some time. My understanding is that rsync is faster than cp when some of the files to transfer are already in the destination directory, transferring only the ...
Amelio Vazquez-Reina's user avatar
99 votes
3 answers
119k views

How to rsync multiple source folders

I want to rsync multiple sources and I wonder the best way to achieve that. e.g. /etc/fstab /home/user/download I thought about 3 solutions : Solution 1 multiple call to rsync rsync -a /etc/...
user1437346's user avatar
  • 1,121
96 votes
15 answers
169k views

How to rsync files between two remotes?

I would like to transfer files between two remote hosts using on local shell, but it seems rsync doesn't support synchronisation if two remotes are specified as follow: $ rsync -vuar host1:/var/www ...
kenorb's user avatar
  • 20.9k
94 votes
9 answers
364k views

How to avoid password prompt with rsync (and without using public keys)?

I need to execute rsync, without it prompting me for password. I've seen in rsync manpage that it doesn't allow specifying the password as command line argument. But I noticed that it allows ...
Dor's user avatar
  • 2,525
88 votes
1 answer
191k views

Preserve the permissions with rsync

Let's say I have a file a.txt in LINUX with permission of 0664. When I use rsync to copy the file to my Mac with rsync -r -t -v LINUX MAC, the file's permission becomes 0644. How can I keep the ...
prosseek's user avatar
  • 8,528
86 votes
9 answers
163k views

Does compression option -z with rsync speed up backup

In rsync, --compress or -z will compress file data during the transfer. If I understand correctly, it compresses files before transfer and then decompress them after transfer. Does the time reduced ...
Tim's user avatar
  • 101k
82 votes
8 answers
136k views

rsync all files of remote machine over SSH without root user?

I have this command to backup a remote machine. The problem is that I need root rights to read and copy all files. I have no root user enabled for security reasons and use sudo the Ubuntu way. Would I ...
redanimalwar's user avatar
  • 1,057
77 votes
6 answers
98k views

Is there ever a reason to use scp instead of rsync?

Is there a reason to use scp instead of rsync? I can see no reason for using scp ever again, rsync does everything that scp does, with more safety (can preserve symlinks etc).
mikebloch's user avatar
  • 889
73 votes
1 answer
168k views

force rsync to overwrite files at destination even if they're newer

I have an rsync backup script I run, which also restores files back where they came from when I ask. But if the files at the destination are newer than those in the backup when I try to restore, it ...
jedipixel's user avatar
  • 839
72 votes
1 answer
86k views

How to exclude subdirectory from rsync?

I'm using rsync to recursively sync a remote folder tree that looks something like this: /folderA/a1/cache /folderA/a1/cache/A1 /folderA/a1/cache/A2 /folderA/a1/somefolder /folderA/a1/someotherfolder /...
TheEdge's user avatar
  • 823
68 votes
7 answers
215k views

"rsync: failed to set permissions on ..." error with rsync -a or -p option

When I use the -a option as is asked and answered in Preserve the permissions with rsync, I got a lot of "rsync: failed to set permissions on" errors. rsync: failed to set permissions on "/ata/text/...
prosseek's user avatar
  • 8,528
68 votes
2 answers
58k views

How to use rsync to backup a directory without git subdirectory

I want to copy my c directory with all subdirectories excluding ./git subdirectory. I do it using rsync : echo "copy c and sh files " rsync -a --include='*.c' --include='*.sh' --include='*/' --...
Adam's user avatar
  • 989
67 votes
2 answers
83k views

rsync "skipping non-regular file" when cloning my Aperture library

I am trying to use rsync to maintain a backup copy of my Aperture library. When I run the sync command to see what would happen this is the outcome: rsync --dry-run -r "/Volumes/Data/Aperture Library....
Nippysaurus's user avatar
67 votes
4 answers
112k views

How can I pause/resume rsync

How can I pause an rsync that's running? First time I did Ctrl+C to kill it and used the -P flag to run again. Is this prone to corrupt the current file transfer? Second time I simply put MacOS to ...
qnoid's user avatar
  • 833
66 votes
6 answers
58k views

Resume transfer of a single file by rsync

In Ubuntu, I want to copy a big file from my hard drive to a removable drive by rsync. For some other reason, the operation cannot complete in a single run. So I am trying to figure out how to use ...
Tim's user avatar
  • 101k
62 votes
4 answers
145k views

how to tell rsync to preserve time stamp on files when source tree has a mounted point

Related to this question Short description of the problem: When source tree has a mounted point inside it, then time stamps on files inside that mounted point when copied to target tree are not ...
Nasser's user avatar
  • 911
59 votes
5 answers
83k views

rsync exclude directory not working

I am running Fedora 17 64-bit and the rsync --exclude=/home/ben/<dir> is not working as expected. I am trying to rsync my home directory to a thumb drive, but I want to exclude certainly ...
Freedom_Ben's user avatar
  • 4,444
52 votes
1 answer
51k views

Delete extraneous files from dest dir via rsync?

Say I have SRC 001.jpg 002.jpg 001.txt a.zip DEST hello.jpg rsync -d --delete SRC:{*.jpg,*.txt} DEST It doesn't remove hello.jpg from DEST, any idea how to archive this?
Cheng's user avatar
  • 6,621
48 votes
3 answers
139k views

Rsync Error 23: Can I tell which files were not transferred?

I ran sudo rsync -va --progress from the root of one external drive to a folder on another external drive. The reason is that the source drive has an error-ful NTFS and I don't have access to a ...
themirror's user avatar
  • 6,968
48 votes
1 answer
8k views

What happens when you rsync without a destination?

To make the story short, I did an rsync: rsync -avP [email protected]:/tmp/ And I forgot to put in my source directory... I really meant to run: rsync -avP /tmp/ [email protected]:/tmp/ ...
Questionmark's user avatar
  • 3,925
42 votes
4 answers
11k views

How can I slow down rsync?

I'm trying to copy the contents of a failing USB thumb drive. If I read the data too fast, the drive's controller chip overheats and the drive vanishes from the system. When that happens, I need to ...
Mark's user avatar
  • 4,204
42 votes
1 answer
80k views

rsync to copy files which has changed

I need to copy some files from a particular directory to a different location on a daily basis. I want to copy the changes only, so planning to use rsync. These files follows the following naming ...
Zama Ques's user avatar
  • 3,286
41 votes
1 answer
108k views

Reuse rsync --dry-run output to speed up the actual transfer later on

I usually do rsync -n before doing an actual rsync. The thing is, how could I automate the second run to reuse exactly the list generated with --dry-run? I mean, if I run rsync -n then I get a ...
Manuel's user avatar
  • 513
37 votes
8 answers
126k views

Parallelise rsync using GNU Parallel

I have been using a rsync script to synchronize data at one host with the data at another host. The data has numerous small-sized files that contribute to almost 1.2TB. In order to sync those files, I ...
Mandar Shinde's user avatar
35 votes
1 answer
40k views

find and rsync?

I want to be able to search for files over 14 days and over 10k and than rsync those found files to a destination. Is there a way to combine these two commands? find ./ -mtime +14 -size +10k rsync -...
mkrouse's user avatar
  • 939
35 votes
1 answer
14k views

Reusing ssh session for repeated rsync commands [duplicate]

Possible Duplicate: Using an already established SSH channel I have a remote server accessible via a chain of SSH proxies. The resulting channel is wide and fast enough, but setting it up takes a ...
9000's user avatar
  • 1,629
35 votes
1 answer
14k views

rsync: colon in file names

Other than by renaming the files first, how can I rsync files that may have colons in their names?
WIZARDELF's user avatar
  • 548
34 votes
4 answers
62k views

How to copy directory structure without removing symlinks?

I need to "install" a bunch of files to another directory keeping the directory structure of the source files intact. For example, if I have ./foo/bar/baz.txt going to /var/www/localhost/webroot/ I ...
kojiro's user avatar
  • 4,634
34 votes
2 answers
115k views

Why is rsync not found?

rsync -avP /home/user/.profile hpux3:/home/user/.profile bash: rsync: command not found If I did ssh to hpux3 machine rsync version 3.1.1 protocol version 31 Copyright (C) 1996-2014 by Andrew ...
elbarna's user avatar
  • 12.6k
34 votes
5 answers
18k views

What is the need for rsync server in daemon mode

I don't understand the need for an rsync server in daemon mode. What are the benefits from it if I can use rsync with SSH or telnet?
Hanan's user avatar
  • 5,741
33 votes
1 answer
46k views

Does rsync delete files on the destination that were deleted from the sending side?

Does rsync delete files from the destination if they were deleted from the source? For example, say in the source /home/me I have three files: a.txt b.txt c.txt Now, I run the command rsync -v /...
perhapsmaybeharry's user avatar
32 votes
6 answers
50k views

Syncing directories in both directions with rsync

I happen to know about rsyn, and I use rsync to sync between my mac and a linux server as follows. rsync -r -t -v MAC LINUX rsync -r -t -v LINUX MAC I expected to run the first command to sync, but ...
prosseek's user avatar
  • 8,528
32 votes
1 answer
32k views

rsync recursively with a certain depth of subfolders

I want to rsync a folder recursively but want subfolders included only to a certain depth. For example I would want a depth of 1,2,3 or 4 subfolders like this: source/ ├── subfolder 1 │   ├── ...
nnn's user avatar
  • 1,181
31 votes
7 answers
69k views

tar + rsync + untar. Any speed benefit over just rsync?

I often find myself sending folders with 10K - 100K of files to a remote machine (within the same network on-campus). I was just wondering if there are reasons to believe that, tar + rsync + ...
Amelio Vazquez-Reina's user avatar
31 votes
2 answers
85k views

Asking rsync to delete files on the receiving side that don't exist on the sending side, with exceptions on the receiving side

I have read these threads: rsync --delete --files-from=list / dest/ does not delete unwanted files Delete extraneous files from dest dir via rsync? But, as far as I can tell (maybe I am missing ...
Amelio Vazquez-Reina's user avatar
30 votes
1 answer
48k views

Why rsync attempts to copy file that is already up-to-date?

I have two same files, on local machine and on remote one. Their sizes are equal, and file on local machine is newer than on remote one - but rsync still attempts to copy the file. I invoke rsync as ...
Rogach's user avatar
  • 6,273
29 votes
1 answer
50k views

Best way to sync files - copy only EXISTING files and only if NEWER than target

I'm doing this sync locally on Ubuntu 12.04. The files are generally small text files (code). I want to copy (preserving mtime stamp) from source directory to target but I only want to copy if the ...
MountainX's user avatar
  • 17.8k
27 votes
1 answer
4k views

Filesystem copied to new server is 60% bigger - why

I'm migrating a server from an Ubuntu Server 18.02 instance ("saturn") to a newly-built Debian Buster 10 system ("enceladus"). I have copied a complete filesystem across the ...
Henry Law's user avatar
  • 395
27 votes
8 answers
59k views

suppress rsync warning: some files vanished before they could be transferred

I get loads of warnings when backing up my running Postfix and Courier server files like: file has vanished: /var/kunden/mail/username/[email protected]/tmp/courier.lock How do I suppress those ...
rubo77's user avatar
  • 28.7k

1
2 3 4 5
42