rsync is a tool to efficiently copy directory hierarchies, locally or remotely, with powerful filters to decide what gets copied.
5
votes
1answer
418 views
What keeps one side of an rsync so busy?
I've got a debian machine in my lan serving as a backup server for the others. It has four HDDs combined into a software RAID 5 md device, on that an lvm, and on that a btrfs. Backups are made using ...
3
votes
3answers
327 views
Backup and synchronization
I want to synchronize my personal document repository between my different computers in my home. Today this folder is under a dedicated partition of the hard drive of a dual boot workstation.
My ...
4
votes
2answers
270 views
How to use rsync with a remote remote host
I am a happy user of rsync -e ssh and use it to synchronize data between my machine and a remote host using
rsync -avz -e ssh me@hostA:~/folder ~/folder
And this did what I needed until now.
I am ...
6
votes
1answer
606 views
rsync: how to exclude the topmost directory
So, I have a seemingly simple issue, but so far I haven't found a solution: I want to exclude the topmost directory from an rsync, while still syncing all its children. The particular issue is that I ...
0
votes
1answer
172 views
How do I escape a dot character for an rsync command? [closed]
I'm trying to do...
ssh -av -e deploy@domain.com:/var/www/domain.com /Users/user/workspace/domain
But it's outputting this (I presume because of the period character):
OpenSSH_5.6p1, OpenSSL ...
1
vote
1answer
450 views
IO Error in rsync while doing everyday backup
I have a sample NAS server (QNAP TS-210), with very limited Linux on-board (although a bit reinforced with Optware/IPKG). I'm a Linux newbie. After digging through Internet I was able to write my own ...
3
votes
1answer
946 views
How should I rsync files in /var/www if I want them to be owned by www-data?
I have set up a new ec2 ubuntu box and installed apache and php5. By default you log onto the box with the ubuntu user who has sudo privileges.
I believe apache runs as root and php5 runs as ...
1
vote
1answer
315 views
Synchronize directory tree with deduplication?
I have two Linux servers that have a large amount of data (1TB+) that need to be synchronized over a slow connection (100 KB/s).
A lot of the data overlaps, but are in different locations.
I would ...
2
votes
2answers
501 views
Sync local files to remote git repository
I have my git repository on a linux located on the company server. I ssh into my linux machine and edit files there.
Problem
This method is great until I have a good connection to the server. in ...
0
votes
1answer
814 views
How to sync two directories?
I have an "A" HDD (1,5TB) next to me (on a netbook running OpenBSD 5.1, it's a fileserver)
And I have a "B" HDD (1,5TB) 200km's away from me on an OpenWrt 10.04 router (also "fileserver").
I always ...
4
votes
3answers
229 views
Monitor directory and if a file is requested then rsync on demand
Is there a tool that monitors a certain directory and if a program tries to access a file in that directory, the program loads that file from another place transparent for any application that tries ...
6
votes
4answers
4k views
Is there ever a reason to use scp instead of rsync?
Is there a reason to use scp instead of rsync? I can see no reason for using scp ever again, rsync does everything that scp does, with more safety (can preserve symlinks etc).
7
votes
3answers
3k views
Give user permissions to all files and folders
I want to have a user who has access to all the files and folders on the system. This is for the purpose of using RSYNC on a local machine to backup a remote machine.
At the moment we are using the ...
1
vote
1answer
515 views
Trying to create a cron to Rsync then tar the folder
I'm stuck at the first hurdle!
I want to run a couple of commands from a bash script. First something like this to rsync some directories:
rsync -e ssh -az user-whatever@website.com:/home ...
1
vote
0answers
344 views
ssh: “Error reading response length from authentication socket”
I have a Makefile with some ssh and rsync commands. All of them are executed correctly, but give the following error message:
Error reading response length from authentication socket.
I don't like ...
0
votes
0answers
139 views
rsync giving “Directory has vanished” when there are multiple (existing) source directories
This is happening to me:
$ rsync -av ./foo ./bar $DEST
sending incremental file list
directory has vanished: "/path/to/foo"
foo exists and is regular a regular dir, and rsync -av ./foo $DEST works, ...
1
vote
1answer
697 views
UID mapping with rsync
I'm migrating files between servers and have to map UIDs (add 500) and GIDs. As I'd like to have it done in one single pass, is there a way to execute a command for each file rsync copies?
1
vote
3answers
436 views
Decreasing hard drive sync time with Unison
I'm using Unison to synchronize my music collection to a few different sources, namely a couple computers over SSH and a hard drive.
One thing I've noticed is that SSH updates are incredibly fast in ...
0
votes
1answer
349 views
rsync with absolute paths and excluding subpaths
Say I have the following structure on path origin_path
origin_path/X=1/A/...
origin_path/X=1/B/...
origin_path/X=1/C/...
...
origin_path/X=2/A/...
origin_path/X=2/B/...
origin_path/X=2/C/...
...
...
...
1
vote
1answer
3k views
Real Time Local File Sync
How can I set up an instant file sync for two local directories? The catch is I need it in real time (15, 10 or even 5 seconds is too slow), and I don't want the target root directory created.
For ...
11
votes
4answers
3k views
Does rsync verify files copied between two local drives?
I want to make a fresh new copy of a large number of files from one local drive to another.
I've read that rsync does a checksum comparison of files when sending them to a remote machine over a ...
9
votes
6answers
4k views
tar + rsync + untar. Any speed benefit over just rsync?
I often find myself sending folders with 10K - 100K of files to a remote machine (within the same network on-campus).
I was just wondering if there are reasons to believe that,
tar + rsync + ...
3
votes
2answers
754 views
Setting nice and ionice for rsync via xinetd
I am attempting to set nice and ionice for rsync via xinetd. I am running Fedora 16. The reason I would like to use these values is to reduce the rsync process to an idle state so other processes run ...
5
votes
3answers
293 views
Copying large tree from one machine to another, maintaining ownership
I am trying to copy a large folder structure between machines. I want to maintain the ownership/rights during the copy as it is not reasonable to 'fix' the privs afterwards.
Therefore, I am using the ...
6
votes
2answers
461 views
sync files recursively between two folders where files are less than 24 hours old
I want to find all the files under a directory which are within 24 hours then rsync those files with another server.
There are some old files that I do not want to transfer, however if those old ...
3
votes
1answer
313 views
Export a m3u playlist to a directory in order they appear in the playlist
There is a way to rsync a m3u playlist into a destination directory.
How can I process the list and number the output so it orders the files to match the playlist order?
For example the m3u:
...
6
votes
5answers
918 views
force copying a corrupted home directory
I have a home directory in a disk that has a lot of corrupted files. I am trying to save most files as possible. I am trying to create a backup of my home directory on an external drive using
tar ...
1
vote
2answers
714 views
Using rsync in a cronjob when a password is needed
Basically I have a bash script that fetches data from my server to perform a backup.
As it is now I have to start that script manually, enter the password, and then wait for it to finish.
I would ...
1
vote
2answers
1k views
Using cygwin and rsync — “password file must not be other-accessible”
I've finally got an old computer of mine running as a backup server with Ubuntu 11.10. I'm planning on using it to backup my Windows 7 machine weekly using rsync. I've got cygwin installed, and rsync ...
13
votes
5answers
2k views
What is the need for rsync server in daemon mode
I don't understand the need for an rsync server in daemon mode. What are the benefits from it if I can use rsync with SSH or telnet?
3
votes
2answers
326 views
How to implement a rotation scheme with rsync
I have an ever growing remote directory of files that I'd like to sort and filter before I rsync them, with the goal of always keeping only the latest N files in my destination directory (i.e. a ...
3
votes
1answer
307 views
Using rsync + cron to sync a machine behind a firewall with my dropbox
I have a machine (A) behind a firewall with no access to the Internet, on this machine I can NFS mount directories on another machine (B) which can access internet, and is accessible from Internet, ...
4
votes
1answer
243 views
Only use mtime comparison with rsync?
Is it possible to sync files via rsync and let rsync only compare mtime informations?
Or is there another tool for this job?
That means that such a tool only copies src to destination if the src is ...
5
votes
2answers
1k views
How do I save changed files?
I have two folders:
ORIGINAL/
ORIGINAL_AND_MY_CHANGES/
My friend has a copy of ORIGINAL/. I would like to generate MY_CHANGES.tgz -- it should contain only new/changed files from ...
4
votes
2answers
4k views
rsync gives “chown <…> failed: Invalid argument (22)” with nfs share
I'm trying to back up my entire system to an external disk using rsync, via a shell script which is run as root:
#!/bin/bash
rsync -vSHPhhaX --numeric-ids --delete ...
2
votes
1answer
877 views
Asking rsync to create *any* necessary directories
I have the following loop in zsh:
for x in my_folders*;
do
rsync -adrv $x/foo/bar/ /foo2/bar2/new_dir/$x/
done
new_dir does not exist before calling rsync, and I would like rsync to create ...
3
votes
1answer
461 views
How to recursively synchronize file timestamps across servers?
Suppose I run an rsync command to mirror a server, but forget to supply the -t option. The mirror will now have different timestamps to the original. What is the easiest way to fix this, without ...
3
votes
4answers
133 views
Comments in argument lists?
Some commands, such as rsync, take in lists as an argument. Can these files contain unix comments like #?
3
votes
1answer
335 views
Rsync ssh include & exclude
Here is my current rsync shell script
upload.sh
SOURCEDIR=/Users/thomas/sites/dreamhost/
DESTDIR=******@reggi.com:/home/thomasreggi/sites/
##rsync -avzL --delete --include '*/' ...
3
votes
4answers
2k views
How to exclude numeric directories with rsync?
I know rsync has an --exclude option which I use quite frequently. But how can I specify that it should exclude all "numeric" directories?
In the directory listing below I would like to only have it ...
4
votes
3answers
354 views
How can I exclude files by default with rsync?
How can I exclude files by default with rsync? Here is how my normal rsync syntax starts out:
rsync --exclude ".ht*" --exclude "error_log" --exclude ".DS*" --exclude "old" ...
I've seen a lot of ...
2
votes
1answer
339 views
Rsyncing multiple files with white spaces in the names
I'm trying to back up files onto a server, but I'm having problems with files that have white spaces in the names.
I'm using this command in terminal:
FILES="testing/space\ in\ filename testing/and\ ...
2
votes
2answers
418 views
Backup a Windows PC to a Linux Machine
What is a good backup strategy to backup a Windows 7 box to a Ubuntu box that are on the same network?
The solution I am thinking would be to mount a Windows folder to Ubutnu then use rsync. But I'm ...
2
votes
1answer
190 views
Grant permissions using rysnc
I want to use rsync (as a user) from "me" on a local computer to "another" user on a distant computer. However, I also want "another" user (on the distant computer) to own the files and to be able to ...
4
votes
1answer
498 views
Automated incremental backups to an external drive
Background
A client needs an automated backup solution to an external drive (to be taken offsite weekly). I'd typically suggest rsnapshot (or perhaps rolling out a custom rsync script), but this ...
1
vote
1answer
245 views
Asking rsync to transfer files without connecting to the remote rsync server?
If I understand correctly, rsync tries to connect to a remote rsync server when transferring files via -e ssh. This worked well for me most of the time, but I am having problems rsyncing with a ...
3
votes
1answer
602 views
rsync pattern copy: dotfiles
I followed the advice given in the question
Rsync filter: copying one pattern only to setup a command line I need to backup only the dotfiles .inF*
Yet with the command:
rsync -av --include='.inF*' ...
3
votes
1answer
2k views
Asking rsync to delete files on the receiving side that don't exist on the sending side, with exceptions on the receiving side
I have read these threads:
rsync --delete --files-from=list / dest/ does not delete unwanted files
Delete extraneous files from dest dir via rsync?
But, as far as I can tell (maybe I am missing ...
2
votes
1answer
1k views
What is the interaction of the rsync --size-only and --sparse options?
If I use rsync with the --size-only and --sparse options together, if a file already exists at the destination as a full file (otherwise the same) it's not overwritten.
If I omit the --size-only ...
1
vote
1answer
616 views
Backing up hard links (rsync)?
My question begins with: Do I actually have hard links on my disk at all (except for "." and ".." of course)? I'm not sure how I would find that out?
If no, the question is already answered.
If yes, ...
