rsync is a tool to efficiently copy directory hierarchies, locally or remotely, with powerful filters to decide what gets copied.

learn more… | top users | synonyms

7
votes
0answers
127 views

Synchronising millions of files between two Linux servers

I have a server which exports a directory containing ~7 million files (mostly images) from its local disk to network clients via NFS. I need to add a second one for the sake of HA, and to keep it in ...
7
votes
0answers
290 views

How to convert existing gz (gzip) files to rsyncable

I am using rsync to back up a repository that contains many gz files including many new ones each day. The rsync backup proceeds more slowly than it should because these gz files are not built with ...
3
votes
0answers
45 views

rsync --verbose always shows directory structure when syncing to NTFS volume

When you run rsync with the -v option, for example rsync -av "/from/this/dir/" "/to/this/dir" ... you will get a list of what was updated/deleted etc. So, the first run will give you a bunch of ...
3
votes
0answers
84 views

Sync with Onedrive

I would like to sync (and backup) my /home directory with OneDrive. I found some great terminal clients, where you can even mount your directory, but nowhere did I found an option to this with some ...
3
votes
0answers
65 views

Speed up / avoid calculation of rsync file lists

I use rsync 3.1.1 to keep in sync two discs, one of which is on the network and mounted as a samba share on /mnt/ROUTER_WD_2TB/. This is the source, with a speed of 30-40MB/s maximum due to network ...
3
votes
0answers
321 views

sync two zip archives (with minimal I/O)?

zip itself can freshen an archive to reflect the current filesystem (e.g. -FS). It only updates zip entries with outdated timestamps. Is there a tool that updates a 2.zip file so that it becomes ...
3
votes
0answers
419 views

Rsync - Corrupted MAC on input

Background I have a script on my NAS which is a Synology DS212J which is running on a special distro where uname -a gives me Linux DS212J 2.6.32.12 #4493 Wed Jul 16 11:37:16 CST 2014 armv5tel GNU/...
3
votes
0answers
793 views

How to use rsync to backup large files by either appending or replacing

I am using rsync to backup some large files regularly: rsync --archive --update --append-verify [source] [destination] Most of the times, the files increase in size (like a log file), but sometimes ...
2
votes
0answers
38 views

Why do two directories differ after syncing with rsync?

I copied some directories on one removable drive to another removable one using rsync. The command used was: rsync -avPS <source> <destination> rsync doesn't report any errors. After ...
2
votes
0answers
147 views

rsync's “Read-only file system (30)” error

NB: This is a 2-part question: why does an error happen, and how to work around the error. (The two parts are related, inasmuch as the answer to the "why"-question may shed light on the "how"-...
2
votes
0answers
25 views

Certain folders are always missing using Rsync to server

I have tried several different ways using rsync modules and ssh trying to pull my linux desktop from my freenas server and no matter how I do it, certain folders are always missing. I am trying to ...
2
votes
0answers
83 views

Ecryptfs problem with rsync

I use rsync to sync my files between my home laptop and my office desktop, with the -Cauvb --delete options. Recently, on my laptop, I encrypted my home directory with ecryptfs, which lives on an ...
2
votes
0answers
343 views

Estimate time or work left to finish for rsync?

When running rsync on a directory with lots of files and directories at multiple levels, can we estimate the amount of work or time to finish? the progress option only shows the progress of ...
2
votes
0answers
578 views

“ls -lh” directories reported size : Linux vs CIFS mounted drives

From a Linux box, I recently mounted a Windows share using CIFS. The intent was to "locally" use rsync to backup my Windows machine. The command line used to mount the Windows drive is something ...
2
votes
0answers
94 views

How to output rsync sent data in realtime using standard tools

I'm developing a small tool for some people who need to send files over to a server. I'd like to use rsync to a rsync deamon to do so. Is it possible to have rsync to output how much data it has ...
2
votes
0answers
2k views

rsync multiple files to multiple directories

I need to move a large number of files that need to go to different directories i.e. file1.mpg to /mnt/s3/directory1/file1.mpg file2.mpg to /mnt/s3/directorya/file2.mpg file3.mpg to /mnt/s3/...
2
votes
0answers
4k views

Occasional rsync error code 12 “Connection reset by peer (104)” from cron task that runs every 20 minutes

I have a script run by cron that rsync's various folders to a server in a different city to publish files to websites. In case it matters, the source server is within the corporate firewall and the ...
2
votes
0answers
359 views

BackupPC can't finish backup

I'm trying to backup /etc/ directory on CentOS 5.2 using BackupPC. It takes several hours and then aborted by signal ALRM. There is a partial backup created with all data from /etc But BackupPC thinks ...
2
votes
0answers
309 views

Cannot get rsync to keep source file ownership, tried many tricks

I'm trying to rsync some directories from a webserver running RHEL5 to a system running Xubuntu. I have rsync, ssh, and keys setup for the machines. I've tried everything so that when i backup from ...
1
vote
0answers
33 views

Proper way to create a compressed, rsyncable mirror of a directory?

I would like to create a compressed mirror of some directories in such a way that it can be rsynced again, both from the original destination and to a remote destination. I can do it inefficiently in ...
1
vote
0answers
22 views

Rsync link-dest - unsure if its creating hard links

This is my first attempt at a shell script. I'm simply trying to make incremental copies of certain folders from a server to another server using Rsync, but using the link-dest option to create ...
1
vote
0answers
50 views

BTRFS send only new files without deleting anything on remote

I'm looking into replacing rsync with BTRFS tools, but I'm not sure if what I want to do is even possible. I have a small local volume and a large remote volume used for archiving. Files will be ...
1
vote
0answers
21 views

How to reconfigure grub on a backed up arch linux on a USB flash disk

i managed to back up my arch linux suing rsync : # rsync -aAXv --exclude={"/dev/*","/proc/*","/sys/*","/tmp/*","/run/*","/mnt/*","/media/*","/lost+found"} / /run/media/username/archlinux edited ...
1
vote
0answers
56 views

Does a file move operation over SAMBA preserve the inode number?

If I have a directory exposed over a SAMBA share and a user, from either a Linux or Windows system, moves a file from one place to another on that share, does SAMBA know to ask the operating system ...
1
vote
0answers
50 views

rsync + chmod only selected files

I'm trying to sync specific cron files between 2 servers and change the permissions for them. However, the command I am trying is also changing the permissions on the main directory. Command used: ...
1
vote
0answers
301 views

rsync only files newer than given date, with --delete

I'm trying to use rsync to backup only the files that have been created or changed after a certain date. The reason is that on that date, the whole system has been backed up. However, those global ...
1
vote
0answers
41 views

rsync: --fuzzy doesn't detect name changes with link-dest

So these are my test files: $ ls -li live/ total 2048 804974 -rwxrwx--- 1 ariel ariel 1048576 Feb 22 00:45 live/file1 804975 -rwxrwx--- 1 ariel ariel 1048576 Feb 22 00:47 live/file2 And a first ...
1
vote
0answers
82 views

Why is my rsync slow?

I run a rsync job transferring about 45 million files/approximately 1.8 TB data (a Mac OS X Time Machine backup) over a 100 MBit connection. I use rsync 3.1.1 from MacPorts (I first tried the built ...
1
vote
0answers
21 views

Does rsync\duplicity real write files on mounted storage?

I've got an external storage, which is mounted as cifs: $: mount ... //somespace/backup on /mnt/backup type cifs (rw) ... And I have write some file: $: rsync /home/user/files /mnt/backup or $: ...
1
vote
0answers
51 views

How to keep remote directory up-to-date with local directory automatically?

Say, I have a directory A on my local machine and B on remote machine(Both are CentOS 7). I am using rsync to sync between them. But I want to track my changes on local directory and automatically ...
1
vote
0answers
212 views

Can rsync command run using multiple cpu?

I executed this command but it took long time when checking checksum. Load average was 1.00 at all times. Can rsync command run using multiple cpu? $ rsync --checksum -av -e ssh /usr/local/xxx/* ...
1
vote
0answers
71 views

Live OS freezes while copying files

I was trying to backup many files (~10,000) from a hard drive on an OS-less notebook to an external 500 GB HDD, so I booted up the Fedora 22 Live XFCE spin using a USB, started the copy and after 5 ...
1
vote
0answers
486 views

rsync + renaming the copied files

Is it possible to change the name of the destination file (add some timestamp as a prefix) when copy is done by rsync ? For example, I can use the following command to change the name of the ...
1
vote
0answers
35 views

Exchanging huge file trees between disks

I have two 1TB disks and a huge file tree on each. Each takes up about 85% of their disk. I want to swap/exchange the trees between disks. The only spare (USB) disk that big I have is nearly full. ...
1
vote
0answers
45 views

Using rsync with the verbose option and filtering the information displayed

I am using rsync with the -a -r and -v options to copy files and directories from one server to another. This all works fine. But when I make a change, for example, to a single file on the source ...
1
vote
0answers
75 views

keep same inodes metadata with rsync or ignore metadata with du

I'm trying to synchronize two folders from a remote computer. I want to use rsync to send the folder, and du to check if the space used is identical on the remote computer. The problem is even with a ...
1
vote
0answers
48 views

How can you recover from disk corruption on a backup drive whilst maintaining all hard links?

So, I have a scenario that is similar to How to copy directories with preserving hardlinks? that I wish to use however I have corruption from the backup hardlink source. My Scenario: 1. I backup my ...
1
vote
0answers
110 views

delete .gitignored files with rsync

I often need to use rsync to synchronize a local with a possible remote directory, while ignoring exactly all the files git would ignore because of its .gitignore configuration. To do so, I am ...
1
vote
0answers
160 views

Absolute path in rsync dry-run statistics

I have executed following command in order to get the list of files that would be transferred to remote server if rsync is executed: rsync -avzm --stats --safe-links --ignore-existing --dry-run --...
1
vote
0answers
337 views

Samba File Server Configuration and Sync for Redundancy

I have two boxes which I installed Samba on. For now, I have a single user that access the server, however, I need to expand as to deploy in a production network. Below is what I want to achieve. ...
1
vote
0answers
260 views

Restore from incremental backup?

I read some books which use dump and restore. They said that during restore, first restore the latest full backup, and then restore each incremental backup created after the full backup. But I am ...
1
vote
0answers
274 views

Using Rsync to SSH and Find files

Hello I am trying to use Rsync to SSH into a remote server and pull a certain type of file that has been modified within the last 24 hours. Here is what I am trying: rsync -avx --timeout=480 --...
1
vote
0answers
91 views

What folders should I exclude / not exclude when migrating a server using rsync

I've got a requirement to migrate a web server onto a fresh server (both 32 bit and Fedora 15). I'm using this guide as I've very little experience with linux, coming from a windows background. So ...
1
vote
0answers
59 views

Can I change the exclude file list while running rsync under a cron job

I am using Fedora 20, MATE desktop, zshell. I have started a cronjob to run, every evening, an incremental backup using rsync, closely following the suggestions here. After the first run yesterday, ...
1
vote
0answers
131 views

rsync daemon max connections unlimited

I'm creating a gui app to upload files to a particular place on a server. the user is able to drop files and these files are 'moved' to the server, if the file already exists, then it is only updated ...
1
vote
0answers
33 views

Using rsync, how can I know what attribute of source file differed from the dest which caused the trasfer

I heavily depend on rsync for remote data transfer and backup purposes, occasionally on different filesystem types. AFAIK rsync by default compares file size and modification timestamps to find out ...
1
vote
0answers
4k views

@ERROR: chdir failed but directory exists

I have a D-Link DNS-320 and I've installed FFP and I'm trying to get rsync running on it. So far I've got the daemon up and running and I've defined a module in my rsyncd.conf like so: max ...
1
vote
0answers
120 views

How can I limit the bandwith to a davfs-mounted cloud storage?

I need to copy approx 400 files of approx. 25 MB each to a davfs-mounted cloud storage. I have tried the following commands to limit the CPU-load and IO-load: nice -n 15 ionice -c 3 rsync -avhW --no-...
1
vote
0answers
34 views

After restoring a backup my permissions seem to be borked

I made a complete backup of the system with "rsync -aAXv", and set up my encryption. After restoring everything, and booting up and trying to login, openbox didn't want to start, and "startx" didn't ...
1
vote
0answers
962 views

Run multiple rsync's at once?

I'm moving over about a million of files (800GB, containing) multiple system backups. And it's only running at ~9MB at a time, often. Enough bandwidth, since it can run up to about 50MB/s. However, ...