rsync is a tool to efficiently copy directory hierarchies, locally or remotely, with powerful filters to decide what gets copied.

learn more… | top users | synonyms

4
votes
1answer
2k views

Incremental system backup and restore w/ rsync or rdiff-backup issues

I've been reading about using rsync and rdiff-backup to make incremental backups of my entire Linux system. I am planning to practise that by setting up a Linux system, making some changes to it, ...
4
votes
2answers
148 views

How do I sync files between two drives?

What command can check that a directory contains the same files as another directory, all the files are up to date and copy any updated files or new files to the first directory. It should not care ...
4
votes
1answer
3k views

how to get rsync to link identical files with --link-dest option if an old file already exists?

One might think that --link-dest'ing to an identical file would work in all cases. But it does not when the file exists, even if the file is out of date/has different contents. It is because of this, ...
4
votes
1answer
1k views

Rsync folder while --exclude-from'ing .gitignore files at different depths

So I'm trying to work on a command to rsync a source folder with a structure like so: - app.js - .gitignore - src/ - src/logs/ - src/logs/.gitignore The first .gitignore would have a series of ...
4
votes
1answer
2k views

sudo askpass: ask from other tty

Context I want to setup rsync backups of my server, so I run the following command rsync -r -e ssh --rsync-path="sudo rsync" rsyncuser@example.com:/ /backup/ Ideally, that command would ask my ...
4
votes
3answers
6k views

suppress rsync warning: some files vanished before they could be transferred

I get loads of warnings when backing up my running Postfix and Courier server files like: file has vanished: /var/kunden/mail/username/name@mymail.de/tmp/courier.lock How do I suppress those ...
4
votes
3answers
3k views

Rsync --bwlimit not working

I am currently using rsync to transfert large file from a host to another. I would like to limit the debit for this transfert. After reading the man page I found the option --bwlimit which is ...
4
votes
1answer
783 views

Only use mtime comparison with rsync?

Is it possible to sync files via rsync and let rsync only compare mtime informations? Or is there another tool for this job? That means that such a tool only copies src to destination if the src is ...
4
votes
1answer
3k views

Asking rsync to create *any* necessary directories

I have the following loop in zsh: for x in my_folders*; do rsync -adrv $x/foo/bar/ /foo2/bar2/new_dir/$x/ done new_dir does not exist before calling rsync, and I would like rsync to create ...
4
votes
1answer
487 views

rsync's relationship with special files

I'm trying to transfer some existing backups from one system to another. The older backup system seems to have managed to back up special files like /dev/hda, /dev/tty0, and /dev/null which look like ...
4
votes
1answer
404 views

rsync 3.1.0 is creating empty backup directories

I just noticed that there is a discrepancy between version 3.0.9 and 3.1.0 of rsync. Running the command $ rsync -a --delete --backup --backup-dir=../old source/ dest/ with version 3.1.0 gives the ...
4
votes
1answer
88 views

Use rsync to just get a folder and its content with a pattern

I've looked all around the internet, looked in regexp sites and everything else, but I'm unable to write a rsync command to fetch what I want to. I need to mirror the FreeBSD release, but I don't ...
4
votes
3answers
6k views

Alternative to diff with progress for massive directory compare?

I just rsync-ed 2,000,000 files (3TB) from one RAID to another. I want to make sure my data is intact. rsync -c takes a really long time. diff doesn't show me what it's doing. Is there an ...
4
votes
3answers
1k views

How to avoid unnecessary deletion and recopying of moved folders when using rsync? [duplicate]

I've read that rsync cannot detect moved folders, and simply delete and recopy them into the new folder structure. Since the only change I did to my master copy is the simple introduction of a top ...
4
votes
2answers
8k views

How do I rsync only some files in some directories

I'm trying to sync all *.jpg files from /var/www/data/models/days/ + some other directories from /var/www/data root. I do: rsync -az --delete --include='/models' --include='*.jpg' --exclude='*' $...
4
votes
2answers
736 views

How to rsync a chroot without breaking symlinks?

I'd like to rsync (backup) a chroot environment from outside the chroot. For that I first make a lvm snapshot of the chroot volume and then run rsync on that. The only problem with this approach: ...
4
votes
1answer
1k views

Is there any way to rsync absolute symlinks alone?

I want to sync a big folder hierarchy between two machines. I want to copy the contents of the symlinks if they are absolute, for eg it points to /some/folder/someFile I want to retain the symlinks if ...
4
votes
1answer
62 views

Different rsync behavior for identical directory name

I want to synchronize 2 codebases. The below command is working great, except that it doesn't account for one use case that I have with 2 directories named test in different locations: /test/ /start/...
4
votes
1answer
87 views

Edge cases in filesystem operations during backup and file copy

I have a question about file access in Linux that I cannot sum it up completely. Consider that I have a home directory that I backup it using rsync triggered via cron. My home directory is on an EXT4 ...
4
votes
1answer
630 views

rsync: how to relocate absolute symlinks?

I've pored over the rsync manpage as well as on several related questions here on serverfault. I've played with -R and -L and -l options, but no luck. Here is the question: when sending a dir from ...
4
votes
1answer
1k views

How to recursively synchronize file timestamps across servers?

Suppose I run an rsync command to mirror a server, but forget to supply the -t option. The mirror will now have different timestamps to the original. What is the easiest way to fix this, without ...
4
votes
1answer
104 views

How do I mount an external SD card without using sudo?

I wrote a bash-script with the intention to back up a specific (truecrypt-)file onto a SD-card (and a folder shared over the cloud) using the rsync command. The problem was that the SD-card usually ...
4
votes
2answers
173 views

synchronize only the newest files and folders, limit by size

I would like to sync Folder A to Folder B with the following restrictions Folder A contains thousands of files in hundreds of folders and many hundreds of GBs of files. Folder B will contain the ...
4
votes
1answer
76 views

Does rsync over any type of checksum? [duplicate]

I need to transfer files from several servers to a single remote host using Rsync over SSH. However, I need to be sure that before the file is deleted on the source using the --remove-from-source ...
4
votes
1answer
222 views

How do you give a developer group access to apache, MySQL, php and the webroot?

We have 3 web designers that are currently logging in to our server via root to configure apache and setup websites. I have very basic linux knowledge (I'm a web designer) and have created users and ...
4
votes
2answers
544 views

light weight disk based backup solution

Currently I am using Dirvish to backup a handful remote machines, both Linux and Windows to local disks. But Dirvish hasn't seen any updates for many years and it has a couple of small annoyances (e.g....
4
votes
2answers
168 views

SSH broken pipe, message authentication code incorrect

I have set up a Ubuntu server that I plan on using to backup my macbook air using rsync. But every time I use rsync, or even scp, the connection drops, with one of the following errors: ...
3
votes
3answers
4k views

How do I backup via rsync to a remote machine, preserving permissions and ownership?

Drive A is 2TB in a closet at home. Drive B is 2TB in my office at work. I'd like drive A to be the one I use regularly and to have rsync mirror A to B nightly/weekly. The problem I have with this ...
3
votes
4answers
198 views

Comments in argument lists?

Some commands, such as rsync, take in lists as an argument. Can these files contain unix comments like #?
3
votes
1answer
490 views

rsync'ing multiple {src,dest} pairs

I have a remote server on which I am deploying code. I am currently using scp to upload code, but scp is very slow and I'd like to switch to rsync (or something else which is more efficient). The ...
3
votes
2answers
3k views

Meaning of Crossing filesystem boundaries, --one-file-system, etc

I'm planning to use rsnapshot to backup my whole Linux system, though I'm confused by -x option (same of one_fs in rsnapshot.conf). The man page says: -x one filesystem, don't cross partitions ...
3
votes
3answers
2k views

Rsync problem with cron and no problem manually

I have a backup.sh file that launch an rsync command. This rsync is to backup my dedicated server on a RaspberryPi running raspbian. (I use keychain so I don't need to type any password etc ...) The ...
3
votes
3answers
522 views

Backup and synchronization

I want to synchronize my personal document repository between my different computers in my home. Today this folder is under a dedicated partition of the hard drive of a dual boot workstation. My ...
3
votes
1answer
4k views

How should I rsync files in /var/www if I want them to be owned by www-data?

I have set up a new ec2 ubuntu box and installed apache and php5. By default you log onto the box with the ubuntu user who has sudo privileges. I believe apache runs as root and php5 runs as www-data....
3
votes
2answers
747 views

How can I invoke a prompt for an ssh key passphrase during the execution of a script?

In my ssh_config file there are multiple entries for sites on the same server such as: Host site1 HostName 123.1.1.1 User myuser Port 13245 GSSAPIAuthentication no ...
3
votes
1answer
442 views

How do I backup a non-empty directory and preserve metadata like permissions/user-group ownership/time file was modified?

I used rsync for creating a backup copy: bash-3.2$ rsync -tvrog /export/home/kent/Scripts /export/home/kent/TEST_Scripts_rsync With these options: Options: -v, --verbose ...
3
votes
2answers
38 views

How to copy just timestamps, or “I should have used cp -p”

I foolishly copied a very large amount of data using cp -r. Looking at it now, I've realized that this means I lost my file modification times. I should have used cp -rp. My directory structure and ...
3
votes
1answer
30 views

Rsync unable to remove remote file not exist in local

Consider my local folder /tmp/foo/ /tmp/foo/1.txt /tmp/foo/2.txt /tmp/foo/3.txt and my remote folder /tmp/foo/ /tmp/foo/1.txt /tmp/foo/2.txt /tmp/foo/4.txt and in my local side, I run the ...
3
votes
1answer
174 views

Can rsync fix time stamps without redownloading?

I have mirrored 100 GiB of files. On my mirror, some of the timestamps and maybe file permissions are wrong. I would like to use rsync to fix this. The server from which I have downloaded has more ...
3
votes
1answer
379 views

How does rysnc work over ssh?

When the rsync program is invoked over ssh what does the interaction between rsync and ssh look like? What feature of ssh is rsync using? I am interested in the interaction between rsync and ssh.
3
votes
1answer
644 views

rsync exclude-from directory with space

I am trying to exclude "Mendeley Desktop" directory for rsync. The exclude-from= is as: Downloads *.o *.iso Dropbox Documents/Mendeley\ Desktop but rsync is still backing up the Mendeley Desktop. ...
3
votes
3answers
904 views

What should I include and exclude with rsync to back up my system?

I have spent all day experimenting with rsync exclude-from and include-from, no success and completely lost. None of the examples or help I can find answer my exact, but quite simple requirement, and, ...
3
votes
2answers
967 views

Preserve ownership with rsync and nonexistent user

I have two computers with the same user, me@Home and me@Work. I usually kept their folders synced by bringing Work home (it's a laptop) and rsyncing over LAN. However, now I'm not bringing Work home ...
3
votes
3answers
2k views

Decreasing hard drive sync time with Unison

I'm using Unison to synchronize my music collection to a few different sources, namely a couple computers over SSH and a hard drive. One thing I've noticed is that SSH updates are incredibly fast in ...
3
votes
1answer
2k views

Rsync protocol name full path,not a module

Is it possible to name the full path to source directory in rsync command instead of the name of the module. For example: rsync -av rsync://10.171.73.23/home/user/work/tests . instead of : ...
3
votes
1answer
2k views

rsync time comparison - what's the precision of the Modified times comparison

I'm doing some synching with rsync using: rsync --rLvv --times --size-only For my initial sync. My idea, now is to use: rsync --rLvv --times To synch files that have a new modified time. ...
3
votes
2answers
1k views

Deduplication tool for rsync?

Is there a Linux / Unix tool that does de-duplication similar to the way Veeam does it before compressing and sending a backup with rsync to a mirrored location?
3
votes
1answer
141 views

rsync file copy including files that change

I have to copy files between 2 servers using rsync (the network is slow). Can mirroring/ FXP with lftp be used here? Also there will be a continuous flow of files in the source directory & the ...
3
votes
1answer
3k views

Incremental file back up by date

Is there any backup tool that supports partial back up by date? What I mean is that I have some small drives such as on a laptop that can't hold a full copy of all the files on an external 2TB drive. ...
3
votes
2answers
2k views

Rsyncing files with special characters to USB FAT32

I want to copy a large number of files to a USB drive that is formatted FAT32 (and cannot be formatted anything else, unfortunately). Many of these files have names with characters such as : and ? ...