rsync is a tool to efficiently copy directory hierarchies, locally or remotely, with powerful filters to decide what gets copied.

learn more… | top users | synonyms

3
votes
2answers
30 views

rsync claims different files, but diff does not?

I have a file both locally and on server. I first try running rsync in "dry-run" mode, to see if there are some differences between the files: $ rsync -aP --dry-run USER@192.168.0.55:/home/dir . USER@...
0
votes
1answer
17 views

Rsync: Prioritise files and folders

I'm copying a large directory from one computer to another using rsync. Is there a way to tell rsync I would like it to copy some files and / or folders first, before others so that I can start ...
0
votes
1answer
36 views

Can rsync be used to make only hard links, not copies? [duplicate]

I'm working on a system to automatically back up my computer using rsync and the Google Drive app (I have a Macintosh computer). (This app creates a directory whose contents are saved to the Google ...
39
votes
4answers
27k views

Does rsync verify files copied between two local drives?

I want to make a fresh new copy of a large number of files from one local drive to another. I've read that rsync does a checksum comparison of files when sending them to a remote machine over a ...
0
votes
1answer
25 views

Expect Rsync with Double quotes

Below is my code: #!/usr/bin/expect -f set timeout -1 spawn rsync -arvz -e 'ssh -p 1690' --protect-args --progress /home/pappu/ 'backup@xx.xx.xx.xx:/volume1/56 - Backup Server/pappu' expect "password:...
1
vote
2answers
625 views

How can i keep the mtime of files and directories when I use rsync or cp? For use in Backup/Incremental Backup

I have a new NAS box and after looking through numerous backup solutions I've found doing it manually through PuTTy to work the best. After logging in to the box through PuTTy as root, I first need ...
1
vote
1answer
30 views

copy_only_missing? - Copy from /A to /C only paths missing on /B and /C - concise way

I'd like to copy from /A to /C only paths missing on /B and /C. (Please assume those are paths, and can represent also remote locations e.g. mounted via sshfs) I wonder... Is there more concise way ...
6
votes
5answers
9k views

Does compression option -z with rsync speed up backup

In rsync, -z will compress file data during the transfer. If I understand correctly, -z compress files before transfer and then decompress them after transfer. Does the time reduced during ...
0
votes
1answer
36 views

Rsync with destination owner and permission possible?

Source server have one owner is root. ----/home/sub1/test1 ----/home/sub2/test1 My local machine is destination. ----/home/sub1/test1 owner by user1 ----/home/sub2/test1 owner by user2 How I ...
0
votes
1answer
58 views

rsync complication on sync

We have 2 Web Server node Primary and secondary, if the primary is down by any reason secondary will act as Prim. Now if we talk about the codes we which is on both the host. we should be ...
1
vote
3answers
3k views

Unexplained rsync disconnect

I'm rsyncing in to a shared web host from a Yosemite client. I've set up my authorized keys and can ssh in just fine, and scp files over no problem. When I rsync with the -n (dry-run) option, it ...
0
votes
1answer
35 views

Rsync to copy specific subfolders and files to new directory

Original path, RootFolderI: RootFolderI/FolderA/Subfolder1/Subsub1/JPG1.jpg -> want this jpg RootFolderI/FolderA/Subfolder2/Subsub1/JPG2.jpg -> want this jpg RootFolderI/FolderA/Subfolder2/...
5
votes
2answers
17 views

Can you scp, sftp, or rsync, a pipe?

I want to tar and send a ~700GiB directory to a remote drive I don't control. I don't have the HDD space locally to create the tarball and then copy that over. The remote is also protected by rssh - ...
2
votes
1answer
55 views

why rsync does not do delta transfer

I have binary file which has approx 77MB: nupic@nupic-virtualbox:~/VboxSharedFolder/experiments/sync/exp2$ ls -lah src/ total 77M drwxrwx--- 1 root vboxsf 0 Jun 21 13:31 . drwxrwx--- 1 root vboxsf ...
0
votes
0answers
29 views

Does changing your OS (fedora>OpenSuse) or FS (ext4>btrfs) “break” rsync?

I am using a rather simple script to do incremental backups of my laptop to my external HDD's, where the core line is probably the following: rsync -arbuv --delete --progress -exclude-from=/backup-...
0
votes
1answer
44 views

Using Rsync include and exclude options to include directory and subdirectory but exlude files in subdirectory

I'm wondering if my scenario can actually be handled with rsync. Rsync working fine between my local and my remote. htdocs/ folderswithfilesIneed/ somefilesandfolders.jpg afolder/ ...
14
votes
2answers
39k views

Asking rsync to delete files on the receiving side that don't exist on the sending side, with exceptions on the receiving side

I have read these threads: rsync --delete --files-from=list / dest/ does not delete unwanted files Delete extraneous files from dest dir via rsync? But, as far as I can tell (maybe I am missing ...
2
votes
0answers
19 views

flock command wont run through cron [duplicate]

I am trying to cron an rsync sync over ssh, but for some reason, as soon as I put the command in cron, it won't run. I don't know what is causing it to break, but it is certainly something related to ...
0
votes
2answers
58 views

rsync --link-dest not working as expected with symlinks

I'm using rsync to backup some of my files: rsync -aEN --delete --link-dest="$CURR/" "$SOURCE/" "$NEW/" The --link-dest option works fine with most files, but not with symlinks. When I was writing a ...
23
votes
4answers
29k views

Using rsync to move (not copy) files between directories?

I have been using rsync to copy files for some time. My understanding is that rsync is faster than cp when some of the files to transfer are already in the destination directory, transferring only the ...
0
votes
1answer
14 views

Download with rsync through bastion/jumper, without .ssh/config changes

I'd like to download a file from server C, through server B (bastion) to my local server A. Normally I'd handle the mess in ssh/config with ProxyCommand, but in this case I can't. Can we make a one-...
1
vote
1answer
17 views

Rsync files-from problem

rsync -av --files-from=list.txt A/* B/ Inside the list.txt I have A/text1.txt When executing the command, I get an error refering me to the man page, does anyone know why? Please see the rsync(1)...
1
vote
0answers
34 views

After restoring a backup my permissions seem to be borked

I made a complete backup of the system with "rsync -aAXv", and set up my encryption. After restoring everything, and booting up and trying to login, openbox didn't want to start, and "startx" didn't ...
0
votes
1answer
34 views

Use of Verbose in a cron job

I'm busy setting up a back-up script to run on my Pi using rsync. I see that a number of people use the -v option in their cron jobs. Why? It's going to be run as root, and not in a terminal where ...
5
votes
0answers
99 views

Synchronising millions of files between two Linux servers

I have a server which exports a directory containing ~7 million files (mostly images) from its local disk to network clients via NFS. I need to add a second one for the sake of HA, and to keep it in ...
1
vote
3answers
47 views

rsync not overwriting newer files [closed]

Replication details: mkdir test && cd test mkdir folder && cd folder echo "#line1" > file rsync -a -I . .. echo "#line2" >> ../file rsync -a -I . .. Output: Content of ../...
21
votes
2answers
28k views

rsync compare directories?

Is it possible to compare two directories with rsync and only print the differences? There's a dry-run option, but when I increase verbosity to a certain level, every file compared is shown. ls -alR ...
1
vote
5answers
4k views

Save a list of files over SFTP to a text file

I have provided my own answer below. We have a number of bash scripts that work fine using SFTP, what I would like to do is to simply re-direct an ls or dir of a folder to a file on our SFTP Server.. ...
4
votes
2answers
200 views

SSH broken pipe, message authentication code incorrect

I have set up a Ubuntu server that I plan on using to backup my macbook air using rsync. But every time I use rsync, or even scp, the connection drops, with one of the following errors: ...
5
votes
1answer
10k views

rsync all directories that start with a specific digit

I have directory loaded with thousands of sub directories: /home/tmp/ 1 12 123 1234 2345 234 3456 345 34 ...
1
vote
1answer
28 views

how to rsync certain directory and files - problem

I have looked at previous posts but cannot get this working. I want to just rsync all the server03 directories and any sub directories of them to a new server. I tried the following but it was picking ...
12
votes
8answers
11k views

How to rsync files between two remotes?

I would like to transfer files between two remote hosts using on local shell, but it seems rsync doesn't support synchronisation if two remotes are specified as follow: $ rsync -vuar host1:/var/www ...
0
votes
0answers
101 views

Rsync freezes while copying

I'm having trouble using rsync to back-up my system locally. I'm using this short guide from the arch wiki. Here's the modified one-liner I'm using : rsync -aAX --debug=all5 --delete --modify-window=...
2
votes
1answer
37 views

Recursive move (`mv -rn`, like `cp -rn`), a move that will only move not present files

Context I have user uploaded content that needs to be backed up. The content is over 3 separate servers at /var/www/domain/media/ (on each server it is in the same place). The backup is a NFS ...
1
vote
1answer
27 views

rsync wants to copy files that already exist in the destination directory

I am trying the following command using rsync to copy only files that are new have a different size rsync -niav --size-only /home/me/source/Electronica /media/externalDrive/dest/Electronica ...
2
votes
0answers
37 views

Why do two directories differ after syncing with rsync?

I copied some directories on one removable drive to another removable one using rsync. The command used was: rsync -avPS <source> <destination> rsync doesn't report any errors. After ...
2
votes
1answer
2k views

rsync specific directories

Below are the directories of my directory structure. I only want to run rsync on the content under the archive directory of any subdirectory and exclude everything else: ./ ap/ ap/aploader/ ap/...
0
votes
1answer
26 views

To rsync -rtvuc in Two Filesystems?

I am newbie with rsync. Systems OS_X_Extended/Source/ FAT32/Destination/ I want to rsync everything from OSX extended to Fat32 directory and delete those files/directories which I deleted from ...
2
votes
1answer
37 views

Rsync files, excluding a list

I am working on setting up cron jobs,and I am new to this. The purpose is to analyze the new data that is generated.The data is present in directories. I want to sync only the latest data/that has ...
2
votes
0answers
26 views

How to make rsync delete files/folders on backup server that no longer exist on main server? [closed]

I am using this command to backup a main server to a backup server. rsync -avz --delete-after --exclude-from '/usr/order/exclude.txt' --password-file=/usr/prod/masterpass.rsync /usr/order/ order@...
1
vote
2answers
161 views

rsync --delete did not remove file in destination while with ssh

On my RHEL6 pair of servers, I tested the rsync from server-1 to Server-2 with SSH. Everything was good as expected. However, when I moved a file under A directory, A/x.file to B/ of /home/user on ...
1
vote
2answers
46 views

rsync on AIX gives popt The module has an invalid magic number error

[SERVER:root:/home/user:] rsync -avz --delete ... Could not load program rsync: Dependent module /opt/freeware/lib/libpopt.a(libpopt.so.0) could not be loaded. The module has an invalid magic ...
1
vote
1answer
744 views

How rsync --compare-dest works

I know there's a very similar question here, but the accepted answer has nothing to do with rsync, and the OP's solution doesn't apply in my case. I'm trying to get all the updates from a CentOS ...
1
vote
2answers
18 views

Rsync rotating logfiles

currently I have an troublesome problem. I needed quite a time for identifying this problem. Anyhow that's what I'm doing: I collect log files with rsync from multiple servers to archive them. Its ...
2
votes
3answers
6k views

Parallelise rsync using GNU Parallel

I have been using a rsync script to synchronize data at one host with the data at another host. The data has numerous small-sized files that contribute to almost 1.2TB. In order to sync those files, ...
1
vote
1answer
30 views

rsync fails/hangs when copying from a Linux ext4 to a FreeBSD ZFS (via SSH)

When I try to copy directories from an ext4 on Linux to a ZFS on FreeBSD (via SSH), rsync immediately fails without copying anything: # rsync -haxHAX /mnt/source/ root@lethe:/mnt/dest/ -vvv opening ...
3
votes
2answers
39 views

How to copy just timestamps, or “I should have used cp -p”

I foolishly copied a very large amount of data using cp -r. Looking at it now, I've realized that this means I lost my file modification times. I should have used cp -rp. My directory structure and ...
2
votes
1answer
51 views

Transfer Size limit for SCP, SFTP, RSYNC?

Is there a maximum size limit to the size of collective size of the files that are to be transferred in one go for the above protocol. My intent is to know if for example after 1 GB any of these ...
3
votes
0answers
64 views

Sync with Onedrive

I would like to sync (and backup) my /home directory with OneDrive. I found some great terminal clients, where you can even mount your directory, but nowhere did I found an option to this with some ...
2
votes
2answers
19k views

rsync: error starting client-server protocol

I have rsyncd running on both the systems . When I am trying to check the connection it giving me following error: From host2: user@host2#rsync username@host1::share password: [password] @...