rsync is a tool to efficiently copy directory hierarchies, locally or remotely, with powerful filters to decide what gets copied.

learn more… | top users | synonyms

5
votes
3answers
1k views

Copy from remote server which doesn't have rsync

I need to recursively copy a folder from a Ubuntu remote server where I have ssh access. I don't want to follow symbolic links, nor to copy permissions/owner/group, because my client system (Ubuntu ...
5
votes
3answers
9k views

rsync mkstemp failed Invalid argument (22) with davfs mount of Box.com cloud

I mounted Box.com cloud storage using davfs according to these instructions. I mounted my Box.com account under /home/me/Cloud/Box I can access the mounted filesystem via Dolphin as well as via the ...
5
votes
2answers
2k views

creating a device image for a network location

I have a dedicated ubuntu server with hetzner.de. hetzner also provides a seperate backup space accessible via samba/ftp/sftp/scp. In their support document about backups, the following line is ...
5
votes
1answer
363 views

Can I exclude /run from my backup scheme?

I have written a rsync-based system backup for my Linux system. The aim is to be able to redeploy the system entirely from the saves, so I don't only backup homedirs. I have excluded a few top-level ...
5
votes
3answers
2k views

Performing accurate snapshot and incremental backups to a remote server

I have a few Ubuntu (and potentially RHEL) servers I'd like to back up to a central backup server (via rsh/SSH). I'd like to do both snapshot and incremental backups. The goal is that if one of the ...
5
votes
2answers
6k views

How to examine the fast output of rsync

I am using rsync for backup about 100GB files and directories. My command is sudo rsync -azvv /home/path/folder1/ /home/path/folder2 However, the output for each file goes up so quickly and there ...
5
votes
4answers
8k views

How to exclude numeric directories with rsync?

I know rsync has an --exclude option which I use quite frequently. But how can I specify that it should exclude all "numeric" directories? In the directory listing below I would like to only have it ...
5
votes
1answer
3k views

rsync recursively with a certain depth of subfolders

I want to rsync a folder recursively but want subfolders included only to a certain depth. For example I would want a depth of 1,2,3 or 4 subfolders like this: source/ ├── subfolder 1 │   ├── ...
5
votes
2answers
8k views

Reasons for rsync NOT transfering all files?

Does anyone know common reasons for such a large deficit difference in the number of files transfered when backing up my LARGE home directory using rsync on a ubuntu 10.04 LTS setup.. machine is ...
5
votes
2answers
115 views

mv with rsync functionality

I'm trying to merge two directory trees that have many common elements but also each have elements that are only present in one of the two trees. The main issue I am having is that when mv encounters ...
5
votes
2answers
3k views

inotify and rsync on large number of files

I am using inotify to watch a directory and sync files between servers using rsync. Syncing works perfectly, and memory usage is mostly not an issue. However, recently a large number of files were ...
5
votes
3answers
6k views

suppress rsync warning: some files vanished before they could be transferred

I get loads of warnings when backing up my running Postfix and Courier server files like: file has vanished: /var/kunden/mail/username/name@mymail.de/tmp/courier.lock How do I suppress those ...
5
votes
3answers
944 views

Two-way rsync results in oscillating content

I am trying to sync content between two servers. The servers are identical builds (Ubuntu 12.04LTS) and have this cron job running to sync the content: rsync -arzc --update --delete /htdocs/testing/...
5
votes
2answers
2k views

rsync-like tool that supports compression and maintains hardlinks?

I use an rsync-based backup scheme to maintain daily (and soon weekly and monthly as well) "snapshots" of my file server, using rsync's link-dest feature to store hardlinks between the snapshots for ...
5
votes
1answer
38 views

prevent backup from running in parallel

In my script /usr/local/bin/backup, that I call every hour from /etc/crontab, I use rsync to copy data to an off-site server. That all worked fine, even in cases where we had somewhat more new data ...
5
votes
1answer
1k views

Rsync folder while --exclude-from'ing .gitignore files at different depths

So I'm trying to work on a command to rsync a source folder with a structure like so: - app.js - .gitignore - src/ - src/logs/ - src/logs/.gitignore The first .gitignore would have a series of ...
5
votes
2answers
2k views

Can't share an ssh connection with rsync

In my .ssh/config file I have the following: Host xxx User yyy HostName zzzz ControlMaster auto ControlPath ~/.ssh/%r@%h:%p This works great for multiplexing my ssh connections (i.e. login once, ...
5
votes
1answer
268 views

Reuse rsync --dry-run output

I usually do rsync -n before doing an actual rsync. The thing is, how could I automate the second run to reuse exactly the list generated with --dry-run? I mean, if I run rsync -n then I get a ...
5
votes
1answer
293 views

“hard-linking” parts of a big file in which only a small part has changed

I use rsync to make backups: rsync -a --link-dest=PATHTO/$PREVIOUSBACKUP $SOURCE $CURRENTBACKUP This way I save space due to using hard links. The problem appears when I need to backup a huge file ...
5
votes
1answer
3k views

Rsync --backup-dir seems to be ignored

I want to use rsync to backup a directory from a local location to a remote location, and store changed files in another remote location. I did use: rsync -rcvhL --progress --backup \ --backup-dir=...
5
votes
1answer
4k views

Using setfacl to create recursive permissions for Apache with rsync

I have a Dokuwiki installation locally, which I regularly sync to my server with rsync. I'm also going to give a friend of mine an ssh account, and host his public Dokuwiki installation. However, I'm ...
5
votes
2answers
857 views

Sync files from a mac to a flash drive - automatically?

I have a flash drive, let's name it FLASH. I want, when on my mac, when FLASH is plugged (and automatically mounted), execute a specific script and make ~/Documents to be automatically copied to /...
5
votes
2answers
53 views

Can you scp, sftp, or rsync, a pipe?

I want to tar and send a ~700GiB directory to a remote drive I don't control. I don't have the HDD space locally to create the tarball and then copy that over. The remote is also protected by rssh - ...
5
votes
3answers
307 views

rsync: continue or in random order

I use rsync over tor. It works for a while, then it drops the connection. That in itself is not a big problem and I suppose it is not entirely unexpected when running long sessions over tor that takes ...
5
votes
1answer
727 views

easy rsync solution with file manager (thunar or nautilus or…)

A common tasks I teach new users in UNIX is how to make backups. Since most are not comfortable with the command line (CLI) and I want to have them make backups (more than forcing them to use the CLI),...
5
votes
1answer
5k views

Rsync — Itemized List of Changes

I've learned that with the -i option, I can get rsync to list all the changes it makes. I've been using it along with the -n or --dry-run options so that I can always learn about all expected changes ...
5
votes
1answer
283 views

How can I synchronize all PDFs from one directory with Dropbox?

I want to synchronize all the PDFs from one directory (my Zotero library) to Dropbox. Finally, I want to have a list of all the PDFs, not the directory names. I successfully synchronized all my PDFs ...
5
votes
1answer
495 views

How to merge multiple WebDAV accounts for backup?

I use davfs, aufs, rsync: sudo mount -t davfs -o uid=wd,gid=wd,dir_mode=777,rw https://wd1.wd.wd/ /mnt/webdav1 sudo mount -t davfs -o uid=wd,gid=wd,dir_mode=777,rw https://wd1.wd.wd/ /mnt/webdav2 ...
5
votes
2answers
92 views

rsync delete only excluded files

rsync option --delete-excluded implies --delete as well. From man page: --delete-excluded In addition to deleting the files on the receiving side that are not on the sending side, this tells ...
5
votes
3answers
3k views

One-way-sync a directory, but leave deleted files deleted on the destination

I want to sync a directory between two systems. To make it more interesting the syncing must only be done in one direction, i.e.: if a file is deleted in the source directory, it must also be ...
4
votes
3answers
1k views

rsync server/daemon security

I have a NAS (Netgear Readynas NV+) at home and a linux (Arch) box at work that is always on. The linux box is already running an ssh server/daemon and the NAS is already running an rsync server/...
4
votes
1answer
7k views

Why rsync attempts to copy file that is already up-to-date?

I have two same files, on local machine and on remote one. Their sizes are equal, and file on local machine is newer than on remote one - but rsync still attempts to copy the file. I invoke rsync as ...
4
votes
4answers
2k views

The most *robust* remote file copy?

How would I go about to copy files over a very unstable internet connection? Sometimes, the connection is lost, other times the IP of one machine or the other machine is changed, sometimes both, ...
4
votes
2answers
3k views

Meaning of Crossing filesystem boundaries, --one-file-system, etc

I'm planning to use rsnapshot to backup my whole Linux system, though I'm confused by -x option (same of one_fs in rsnapshot.conf). The man page says: -x one filesystem, don't cross partitions ...
4
votes
4answers
4k views

rsync over ssh tunnel

---------- /| /| ------------ | Home | | | | | | Work | | |--------| |---------------------| |-------------...
4
votes
2answers
2k views

How to use rsync with a remote remote host

I am a happy user of rsync -e ssh and use it to synchronize data between my machine and a remote host using rsync -avz -e ssh me@hostA:~/folder ~/folder And this did what I needed until now. I am ...
4
votes
3answers
54 views

Is there a way to send custom arguments to an rsync server?

I have a backup server that clients use with rsync over SSH. The server is configured to run a custom shell script that wraps the invocation of the rsync server side. The client side is unaware of ...
4
votes
1answer
221 views

Should the key for an automatically running cron job that runs over ssh not have a passphrase?

I'm reading an article in Master Linux Now 2013 called OpenSSH: Easy Logins and it uses ssh-agent to allow you to enter a passphrase for your key once, and then you're able to connect to a remote ...
4
votes
1answer
656 views

Is rsync really bidirectional or more unidirectional?

So I understand that rsync works by using a checksum algorithm that updates files according to file size and date. But isn't this synchronization dependent on how source and destination are called? ...
4
votes
2answers
8k views

How to see the total progress while copying the files

We know that if we give --progress parameter to rsync it will show the progress of files copied. But issue is that is shows the progress for each single file not total or overall progress. So how to ...
4
votes
2answers
3k views

rsync: how to include files with name ending in a specific string [duplicate]

Possible Duplicate: Rsync filter: copying one pattern only I would like to use rsync to transfer all files from a server (or server-via-ssh) which have a specific ending string, such as a file ...
4
votes
1answer
9k views

Real Time Local File Sync

How can I set up an instant file sync for two local directories? The catch is I need it in real time (15, 10 or even 5 seconds is too slow), and I don't want the target root directory created. For ...
4
votes
3answers
2k views

Help Shell script to pass variables to rsync

I'm trying to create a simple wrapper shell script for rsync. When I send file names to my script, rsync can never seem to identify the correct location. The file names have spaces in them. I've tried ...
4
votes
1answer
212 views

rsync move files, leave directories?

I'm using rsync to copy data between my internal and external hard drives. It's done what I need, but I'm running out of room on my internal hard drive. So, is there any way to get rsync to function ...
4
votes
3answers
803 views

Monitor directory and if a file is requested then rsync on demand

Is there a tool that monitors a certain directory and if a program tries to access a file in that directory, the program loads that file from another place transparent for any application that tries ...
4
votes
1answer
691 views

What does all this extra data from rsync mean?

When I move directories with rsync, I get some extra data to the right of the progress bar right after each file has been copied: $ rsync -aP assembly /media/andreas/backup sending incremental file ...
4
votes
1answer
2k views

How do I enable rsync in libcurl?

I'm on Arch. When pacman tries to download a package from an ftp server, It fails with the error message Protocol rsync not supported or disabled in libcurl This has been bugging me for a little ...
4
votes
1answer
2k views

how to use rsync (using password-less ssh), with sudo just on the local side?

This command works fine for me with password-less ssh: rsync -aqve ssh ${user}@${remote_host}:/etc/bind /tmp/etc/bind Since I would like to rsync directly to my own local /etc/bind with root ...
4
votes
2answers
525 views

Rsync won't Give Up Writing to an Unmounted Remote Drive

I've been having issues with a external hard drive staying mounted to a mount point. That's not the topic of this question, but background info. The issue is with rsync. It doesn't seem to recognize ...
4
votes
1answer
2k views

How to rsync to different directory?

Here is what I want to achieve: Run Rsync to compare two directories recursively like this: rsync -r /source /target but don't actually rsync between the directories. I want the resulting ...