rsync is a tool to efficiently copy directory hierarchies, locally or remotely, with powerful filters to decide what gets copied.

learn more… | top users | synonyms

5
votes
2answers
5k views

How to examine the fast output of rsync

I am using rsync for backup about 100GB files and directories. My command is sudo rsync -azvv /home/path/folder1/ /home/path/folder2 However, the output for each file goes up so quickly and there ...
5
votes
4answers
8k views

How to exclude numeric directories with rsync?

I know rsync has an --exclude option which I use quite frequently. But how can I specify that it should exclude all "numeric" directories? In the directory listing below I would like to only have it ...
5
votes
1answer
3k views

rsync recursively with a certain depth of subfolders

I want to rsync a folder recursively but want subfolders included only to a certain depth. For example I would want a depth of 1,2,3 or 4 subfolders like this: source/ ├── subfolder 1 │   ├── ...
5
votes
2answers
7k views

Reasons for rsync NOT transfering all files?

Does anyone know common reasons for such a large deficit difference in the number of files transfered when backing up my LARGE home directory using rsync on a ubuntu 10.04 LTS setup.. machine is ...
5
votes
2answers
113 views

mv with rsync functionality

I'm trying to merge two directory trees that have many common elements but also each have elements that are only present in one of the two trees. The main issue I am having is that when mv encounters ...
5
votes
2answers
3k views

inotify and rsync on large number of files

I am using inotify to watch a directory and sync files between servers using rsync. Syncing works perfectly, and memory usage is mostly not an issue. However, recently a large number of files were ...
5
votes
3answers
924 views

Two-way rsync results in oscillating content

I am trying to sync content between two servers. The servers are identical builds (Ubuntu 12.04LTS) and have this cron job running to sync the content: rsync -arzc --update --delete ...
5
votes
2answers
2k views

rsync-like tool that supports compression and maintains hardlinks?

I use an rsync-based backup scheme to maintain daily (and soon weekly and monthly as well) "snapshots" of my file server, using rsync's link-dest feature to store hardlinks between the snapshots for ...
5
votes
1answer
27 views

prevent backup from running in parallel

In my script /usr/local/bin/backup, that I call every hour from /etc/crontab, I use rsync to copy data to an off-site server. That all worked fine, even in cases where we had somewhat more new data ...
5
votes
1answer
383 views

Using inotifywait along with vim

I have simple script that monitors file for changes and rsyncs it with remote copy: #!/bin/bash while inotifywait -e close_write somefile do rsync somefile user@host.domain:./somefile done It ...
5
votes
2answers
2k views

Can't share an ssh connection with rsync

In my .ssh/config file I have the following: Host xxx User yyy HostName zzzz ControlMaster auto ControlPath ~/.ssh/%r@%h:%p This works great for multiplexing my ssh connections (i.e. login once, ...
5
votes
1answer
158 views

Reuse rsync --dry-run output

I usually do rsync -n before doing an actual rsync. The thing is, how could I automate the second run to reuse exactly the list generated with --dry-run? I mean, if I run rsync -n then I get a ...
5
votes
1answer
278 views

“hard-linking” parts of a big file in which only a small part has changed

I use rsync to make backups: rsync -a --link-dest=PATHTO/$PREVIOUSBACKUP $SOURCE $CURRENTBACKUP This way I save space due to using hard links. The problem appears when I need to backup a huge file ...
5
votes
1answer
4k views

Using setfacl to create recursive permissions for Apache with rsync

I have a Dokuwiki installation locally, which I regularly sync to my server with rsync. I'm also going to give a friend of mine an ssh account, and host his public Dokuwiki installation. However, I'm ...
5
votes
2answers
850 views

Sync files from a mac to a flash drive - automatically?

I have a flash drive, let's name it FLASH. I want, when on my mac, when FLASH is plugged (and automatically mounted), execute a specific script and make ~/Documents to be automatically copied to ...
5
votes
3answers
292 views

rsync: continue or in random order

I use rsync over tor. It works for a while, then it drops the connection. That in itself is not a big problem and I suppose it is not entirely unexpected when running long sessions over tor that takes ...
5
votes
1answer
681 views

easy rsync solution with file manager (thunar or nautilus or…)

A common tasks I teach new users in UNIX is how to make backups. Since most are not comfortable with the command line (CLI) and I want to have them make backups (more than forcing them to use the ...
5
votes
1answer
5k views

Rsync — Itemized List of Changes

I've learned that with the -i option, I can get rsync to list all the changes it makes. I've been using it along with the -n or --dry-run options so that I can always learn about all expected changes ...
5
votes
1answer
280 views

How can I synchronize all PDFs from one directory with Dropbox?

I want to synchronize all the PDFs from one directory (my Zotero library) to Dropbox. Finally, I want to have a list of all the PDFs, not the directory names. I successfully synchronized all my PDFs ...
5
votes
1answer
488 views

How to merge multiple WebDAV accounts for backup?

I use davfs, aufs, rsync: sudo mount -t davfs -o uid=wd,gid=wd,dir_mode=777,rw https://wd1.wd.wd/ /mnt/webdav1 sudo mount -t davfs -o uid=wd,gid=wd,dir_mode=777,rw https://wd1.wd.wd/ /mnt/webdav2 ...
5
votes
3answers
3k views

One-way-sync a directory, but leave deleted files deleted on the destination

I want to sync a directory between two systems. To make it more interesting the syncing must only be done in one direction, i.e.: if a file is deleted in the source directory, it must also be ...
4
votes
3answers
1k views

rsync server/daemon security

I have a NAS (Netgear Readynas NV+) at home and a linux (Arch) box at work that is always on. The linux box is already running an ssh server/daemon and the NAS is already running an rsync ...
4
votes
1answer
6k views

Why is rsync not found?

rsync -avP /home/user/.profile hpux3:/home/user/.profile bash: rsync: command not found If I did ssh to hpux3 machine rsync version 3.1.1 protocol version 31 Copyright (C) 1996-2014 by Andrew ...
4
votes
1answer
7k views

Why rsync attempts to copy file that is already up-to-date?

I have two same files, on local machine and on remote one. Their sizes are equal, and file on local machine is newer than on remote one - but rsync still attempts to copy the file. I invoke rsync as ...
4
votes
4answers
4k views

rsync over ssh tunnel

---------- /| /| ------------ | Home | | | | | | Work | | |--------| |---------------------| ...
4
votes
2answers
2k views

How to use rsync with a remote remote host

I am a happy user of rsync -e ssh and use it to synchronize data between my machine and a remote host using rsync -avz -e ssh me@hostA:~/folder ~/folder And this did what I needed until now. I am ...
4
votes
1answer
177 views

Should the key for an automatically running cron job that runs over ssh not have a passphrase?

I'm reading an article in Master Linux Now 2013 called OpenSSH: Easy Logins and it uses ssh-agent to allow you to enter a passphrase for your key once, and then you're able to connect to a remote ...
4
votes
1answer
608 views

Is rsync really bidirectional or more unidirectional?

So I understand that rsync works by using a checksum algorithm that updates files according to file size and date. But isn't this synchronization dependent on how source and destination are called? ...
4
votes
4answers
2k views

The most *robust* remote file copy?

How would I go about to copy files over a very unstable internet connection? Sometimes, the connection is lost, other times the IP of one machine or the other machine is changed, sometimes both, ...
4
votes
2answers
8k views

How to see the total progress while copying the files

We know that if we give --progress parameter to rsync it will show the progress of files copied. But issue is that is shows the progress for each single file not total or overall progress. So how to ...
4
votes
2answers
3k views

rsync: how to include files with name ending in a specific string [duplicate]

Possible Duplicate: Rsync filter: copying one pattern only I would like to use rsync to transfer all files from a server (or server-via-ssh) which have a specific ending string, such as a ...
4
votes
1answer
9k views

Real Time Local File Sync

How can I set up an instant file sync for two local directories? The catch is I need it in real time (15, 10 or even 5 seconds is too slow), and I don't want the target root directory created. For ...
4
votes
3answers
2k views

Help Shell script to pass variables to rsync

I'm trying to create a simple wrapper shell script for rsync. When I send file names to my script, rsync can never seem to identify the correct location. The file names have spaces in them. I've tried ...
4
votes
3answers
780 views

Monitor directory and if a file is requested then rsync on demand

Is there a tool that monitors a certain directory and if a program tries to access a file in that directory, the program loads that file from another place transparent for any application that tries ...
4
votes
1answer
580 views

What does all this extra data from rsync mean?

When I move directories with rsync, I get some extra data to the right of the progress bar right after each file has been copied: $ rsync -aP assembly /media/andreas/backup sending incremental file ...
4
votes
1answer
2k views

how to use rsync (using password-less ssh), with sudo just on the local side?

This command works fine for me with password-less ssh: rsync -aqve ssh ${user}@${remote_host}:/etc/bind /tmp/etc/bind Since I would like to rsync directly to my own local /etc/bind with root ...
4
votes
2answers
514 views

Rsync won't Give Up Writing to an Unmounted Remote Drive

I've been having issues with a external hard drive staying mounted to a mount point. That's not the topic of this question, but background info. The issue is with rsync. It doesn't seem to recognize ...
4
votes
1answer
206 views

rsync move files, leave directories?

I'm using rsync to copy data between my internal and external hard drives. It's done what I need, but I'm running out of room on my internal hard drive. So, is there any way to get rsync to function ...
4
votes
3answers
993 views

from a sparse file to a block device over the network

If I have a sparse file representing a block device, then how can I efficiently restore the file over a network? Ideally, I'd use rsync -S host:file /dev/sdb, but rsync won't write to block devices. ...
4
votes
1answer
527 views

Using rsync + cron to sync a machine behind a firewall with my dropbox

I have a machine (A) behind a firewall with no access to the Internet, on this machine I can NFS mount directories on another machine (B) which can access internet, and is accessible from Internet, ...
4
votes
1answer
1k views

Do any Linux distributions have a folder sync option (like briefcase in windows)?

I want to keep my local folder synchronized with a network folder. Is there any folder synchronization utility or feature in Linux?
4
votes
2answers
594 views

How to synchronize a networked drive and an external drive (Ubuntu & OS X)?

In light of a recent hardware failure, I've decided that it's time to be a little less lax regarding back-ups. What I have: An Ubuntu 10.10 'server/NAS' A MacBook 5,2 A WD external HDD The server ...
4
votes
1answer
2k views

Incremental system backup and restore w/ rsync or rdiff-backup issues

I've been reading about using rsync and rdiff-backup to make incremental backups of my entire Linux system. I am planning to practise that by setting up a Linux system, making some changes to it, ...
4
votes
2answers
147 views

How do I sync files between two drives?

What command can check that a directory contains the same files as another directory, all the files are up to date and copy any updated files or new files to the first directory. It should not care ...
4
votes
1answer
2k views

how to get rsync to link identical files with --link-dest option if an old file already exists?

One might think that --link-dest'ing to an identical file would work in all cases. But it does not when the file exists, even if the file is out of date/has different contents. It is because of this, ...
4
votes
1answer
1k views

Rsync folder while --exclude-from'ing .gitignore files at different depths

So I'm trying to work on a command to rsync a source folder with a structure like so: - app.js - .gitignore - src/ - src/logs/ - src/logs/.gitignore The first .gitignore would have a series of ...
4
votes
1answer
1k views

How do I enable rsync in libcurl?

I'm on Arch. When pacman tries to download a package from an ftp server, It fails with the error message Protocol rsync not supported or disabled in libcurl This has been bugging me for a little ...
4
votes
1answer
2k views

sudo askpass: ask from other tty

Context I want to setup rsync backups of my server, so I run the following command rsync -r -e ssh --rsync-path="sudo rsync" rsyncuser@example.com:/ /backup/ Ideally, that command would ask my ...
4
votes
3answers
5k views

suppress rsync warning: some files vanished before they could be transferred

I get loads of warnings when backing up my running Postfix and Courier server files like: file has vanished: /var/kunden/mail/username/name@mymail.de/tmp/courier.lock How do I suppress those ...
4
votes
3answers
3k views

Rsync --bwlimit not working

I am currently using rsync to transfert large file from a host to another. I would like to limit the debit for this transfert. After reading the man page I found the option --bwlimit which is ...