rsync is a tool to efficiently copy directory hierarchies, locally or remotely, with powerful filters to decide what gets copied.
18
votes
7answers
11k views
Rsync filter: copying one pattern only
I am trying to create a directory that will house all and only my PDFs compiled from LaTeX. I like keeping each project in a separate folder, all housed in a big folder called LaTeX. So I tried ...
13
votes
5answers
2k views
What is the need for rsync server in daemon mode
I don't understand the need for an rsync server in daemon mode. What are the benefits from it if I can use rsync with SSH or telnet?
13
votes
1answer
6k views
Delete extraneous files from dest dir via rsync?
Say I have
SRC
001.jpg
002.jpg
001.txt
a.zip
DEST
hello.jpg
rsync -d --delete SRC:{*.jpg,*.txt} DEST
It doesn't remove hello.jpg from DEST, any idea how to archive this?
12
votes
3answers
6k views
scp without replacing existing files in the destination
How do I copy an entire directory into a directory of the same name without replacing the content in the destination directory? (instead, I would like to add to the contents of the destination folder)
...
11
votes
4answers
3k views
Does rsync verify files copied between two local drives?
I want to make a fresh new copy of a large number of files from one local drive to another.
I've read that rsync does a checksum comparison of files when sending them to a remote machine over a ...
11
votes
1answer
1k views
Making a progressbar with “dialog” from rsync output
I'm looking for a way to filter/redirect rsync output in a manner where it can be fed to the "dialog --gauge" command, so I can get a nice looking progressbar during file sync.
Currently I have only ...
10
votes
3answers
8k views
How can I pause/resume rsync
How can I pause an rsync that's running?
First time I did Ctrl+C to kill it and used the -P flag to run again. Is this prone to corrupt the current file transfer?
Second time I simply put MacOS to ...
10
votes
3answers
4k views
Resume transfer of a single file by rsync
In Ubuntu, I want to copy a big file from my hard drive to a removable drive by rsync. For some other reason, the operation cannot complete in a single run. So I am trying to figure out how to use ...
9
votes
6answers
4k views
tar + rsync + untar. Any speed benefit over just rsync?
I often find myself sending folders with 10K - 100K of files to a remote machine (within the same network on-campus).
I was just wondering if there are reasons to believe that,
tar + rsync + ...
8
votes
3answers
10k views
Sync a local directory with a remote directory in Linux
I have a folder with a few files in it, and some space on a web server. I want to do a bi-directional sync between the local folder and the remote one in linux, just on modification time. How do I ...
8
votes
5answers
423 views
Directory sync with automatic peer discovery
I'm working with a cluster of Linux servers sitting behind the Amazon cloud autoscaling feature (documentation). There are files that accumulate on each server, they are cache files for the ...
7
votes
3answers
3k views
Give user permissions to all files and folders
I want to have a user who has access to all the files and folders on the system. This is for the purpose of using RSYNC on a local machine to backup a remote machine.
At the moment we are using the ...
6
votes
4answers
4k views
Is there ever a reason to use scp instead of rsync?
Is there a reason to use scp instead of rsync? I can see no reason for using scp ever again, rsync does everything that scp does, with more safety (can preserve symlinks etc).
6
votes
3answers
226 views
Why does rsync fail to copy files from /sys in Linux?
I have a bash script which uses rsync to backup files in Archlinux. I noticed that rsync failed to copy a file from /sys, while cp worked just fine:
# rsync /sys/class/net/enp3s1/address /tmp
...
6
votes
2answers
461 views
sync files recursively between two folders where files are less than 24 hours old
I want to find all the files under a directory which are within 24 hours then rsync those files with another server.
There are some old files that I do not want to transfer, however if those old ...
6
votes
5answers
917 views
force copying a corrupted home directory
I have a home directory in a disk that has a lot of corrupted files. I am trying to save most files as possible. I am trying to create a backup of my home directory on an external drive using
tar ...
6
votes
1answer
606 views
rsync: how to exclude the topmost directory
So, I have a seemingly simple issue, but so far I haven't found a solution: I want to exclude the topmost directory from an rsync, while still syncing all its children. The particular issue is that I ...
5
votes
3answers
1k views
cp or rsync, is cp really worth it?
I hope this does not count as a question without a real answer, as I can't seem to find a good reason to use cp(1) over rsync(1) in virtually all circumstances. Should one typically favour rsync over ...
5
votes
3answers
293 views
Copying large tree from one machine to another, maintaining ownership
I am trying to copy a large folder structure between machines. I want to maintain the ownership/rights during the copy as it is not reasonable to 'fix' the privs afterwards.
Therefore, I am using the ...
5
votes
4answers
4k views
view stdout for another pts
Here is the situation. I left my pc at home doing an rsync from a 2TB hard drive to another 2TB hard drive (it's going to take a while since they are both USB 2.0). I am now at work and I have ssh-ed ...
5
votes
3answers
5k views
Syncing directories in both directions with rsync
I happen to know about rsyn, and I use rsync to sync between my mac and a linux server as follows.
rsync -r -t -v MAC LINUX
rsync -r -t -v LINUX MAC
I expected to run the first command to sync, but ...
5
votes
2answers
1k views
How do I save changed files?
I have two folders:
ORIGINAL/
ORIGINAL_AND_MY_CHANGES/
My friend has a copy of ORIGINAL/. I would like to generate MY_CHANGES.tgz -- it should contain only new/changed files from ...
5
votes
2answers
3k views
root user denied access to .gvfs in rsnapshot?
I was running rsnapshot as root and I got the following error. Why would this happen? what is .gvfs?
rsnapshot weekly slave-iv
...
5
votes
1answer
110 views
rsync: skip files for which I don't have permissions
I'm using rsync -rlptD to copy a directory from another user. There are a few files (I have no way of knowing these in advance) which I don't have permission to copy. Is there a way have rsync ignore ...
5
votes
1answer
415 views
What keeps one side of an rsync so busy?
I've got a debian machine in my lan serving as a backup server for the others. It has four HDDs combined into a software RAID 5 md device, on that an lvm, and on that a btrfs. Backups are made using ...
5
votes
2answers
130 views
Rsync different behavior when syncing folders HDD-HDD or HDD-Ext HD
When I sync two folders between two HDDs, rsync only lists the changes made at the end.
However if I sync a folder between a HDD and an external HD, while the sync is running it is listing each ...
5
votes
1answer
143 views
How can I synchronize all PDFs from one directory with Dropbox?
I want to synchronize all the PDFs from one directory (my Zotero library) to Dropbox. Finally, I want to have a list of all the PDFs, not the directory names.
I successfully synchronized all my PDFs ...
5
votes
1answer
3k views
Copy symlink AND where it points to using rsync
It doesn't need to be done by rsync, but it would be nice not have to write a script to do this, and rsync is very close to what I want.
"rsync -a" (or -l) can preserve symlinks, and -L can ...
4
votes
2answers
861 views
How can I send single file to multiple remote sites at the same time?
Can scp be used to send a single file to multiple remote servers at the same time? If so, how? If not, what's the alternative?
4
votes
1answer
71 views
Can I exclude /run from my backup scheme?
I have written a rsync-based system backup for my Linux system. The aim is to be able to redeploy the system entirely from the saves, so I don't only backup homedirs. I have excluded a few top-level ...
4
votes
1answer
457 views
Reusing ssh session for repeated rsync commands [duplicate]
Possible Duplicate:
Using an already established SSH channel
I have a remote server accessible via a chain of SSH proxies. The resulting channel is wide and fast enough, but setting it up ...
4
votes
2answers
269 views
How to use rsync with a remote remote host
I am a happy user of rsync -e ssh and use it to synchronize data between my machine and a remote host using
rsync -avz -e ssh me@hostA:~/folder ~/folder
And this did what I needed until now.
I am ...
4
votes
1answer
2k views
rsync not using --files-from option?
I'm trying to rum rsync -a --files-from=~/.rsync_file_list ~/destination and it tells me: rsync error: syntax or usage error (code 1) at options.c(1652) [client=3.0.7]. Can anyone enlighten me as to ...
4
votes
3answers
633 views
Performing accurate snapshot and incremental backups to a remote server
I have a few Ubuntu (and potentially RHEL) servers I'd like to back up to a central backup server (via rsh/SSH). I'd like to do both snapshot and incremental backups. The goal is that if one of the ...
4
votes
3answers
354 views
How can I exclude files by default with rsync?
How can I exclude files by default with rsync? Here is how my normal rsync syntax starts out:
rsync --exclude ".ht*" --exclude "error_log" --exclude ".DS*" --exclude "old" ...
I've seen a lot of ...
4
votes
3answers
226 views
Monitor directory and if a file is requested then rsync on demand
Is there a tool that monitors a certain directory and if a program tries to access a file in that directory, the program loads that file from another place transparent for any application that tries ...
4
votes
1answer
498 views
Automated incremental backups to an external drive
Background
A client needs an automated backup solution to an external drive (to be taken offsite weekly). I'd typically suggest rsnapshot (or perhaps rolling out a custom rsync script), but this ...
4
votes
2answers
298 views
How to synchronize a networked drive and an external drive (Ubuntu & OS X)?
In light of a recent hardware failure, I've decided that it's time to be a little less lax regarding back-ups.
What I have:
An Ubuntu 10.10 'server/NAS'
A MacBook 5,2
A WD external HDD
The server ...
4
votes
1answer
243 views
Only use mtime comparison with rsync?
Is it possible to sync files via rsync and let rsync only compare mtime informations?
Or is there another tool for this job?
That means that such a tool only copies src to destination if the src is ...
4
votes
2answers
1k views
rsync-like tool that supports compression and maintains hardlinks?
I use an rsync-based backup scheme to maintain daily (and soon weekly and monthly as well) "snapshots" of my file server, using rsync's link-dest feature to store hardlinks between the snapshots for ...
4
votes
2answers
57 views
Two-way rsync results in oscillating content
I am trying to sync content between two servers. The servers are identical builds (Ubuntu 12.04LTS) and have this cron job running to sync the content:
rsync -arzc --update --delete ...
4
votes
2answers
237 views
Restrict file access to append only
I have a directory with log files and I'm putting
logs from script launched by users into them. Logging with syslog doesn't seem
possible in this case. (non-daemon rsync)
I want the users to have ...
4
votes
2answers
273 views
How to rsync a chroot without breaking symlinks?
I'd like to rsync (backup) a chroot environment from outside the chroot.
For that I first make a lvm snapshot of the chroot volume and then run rsync on that.
The only problem with this approach: ...
4
votes
2answers
355 views
Sync files from a mac to a flash drive - automatically?
I have a flash drive, let's name it FLASH.
I want, when on my mac, when FLASH is plugged (and automatically mounted), execute a specific script and make ~/Documents to be automatically copied to ...
4
votes
2answers
4k views
rsync gives “chown <…> failed: Invalid argument (22)” with nfs share
I'm trying to back up my entire system to an external disk using rsync, via a shell script which is run as root:
#!/bin/bash
rsync -vSHPhhaX --numeric-ids --delete ...
4
votes
1answer
294 views
Rsync — Itemized List of Changes
I've learned that with the -i option, I can get rsync to list all the changes it makes. I've been using it along with the -n or --dry-run options so that I can always learn about all expected changes ...
4
votes
0answers
275 views
Can't share an ssh connection with rsync
In my .ssh/config file I have the following:
Host xxx
User yyy
HostName zzzz
ControlMaster auto
ControlPath ~/.ssh/%r@%h:%p
This works great for multiplexing my ssh connections (i.e. login once, ...
3
votes
4answers
133 views
Comments in argument lists?
Some commands, such as rsync, take in lists as an argument. Can these files contain unix comments like #?
3
votes
2answers
445 views
Rsync two file types in one command?
How to write those into one line, also without repeat the same path?
rsync -a root@somewhere:/folder/remote/*.txt .
rsync -a root@somewhere:/folder/remote/*.jpg .
3
votes
3answers
327 views
Backup and synchronization
I want to synchronize my personal document repository between my different computers in my home. Today this folder is under a dedicated partition of the hard drive of a dual boot workstation.
My ...
