Questions tagged [rsync]
rsync is a tool to efficiently copy directory hierarchies, locally or remotely, with powerful filters to decide what gets copied.
1,981
questions
0
votes
1
answer
10
views
SCP from a distant server with a cumulative size limit
I want to copy a number of files from a distant server to local computer, such that the total size of the files does not exceed certain limit, as I do not locally have enough disk space; and the files ...
0
votes
2
answers
31
views
Methodology to copy data from one machine to many others in a fast way
I found some bottleneck on my workflow which is as follows. I do have one master computer which needs to send data to other node machines. This is done in a for loop such as:
for all nodes: rsync <...
- 153
1
vote
1
answer
20
views
Directory usage from remote server that can be accessible only via rsync
Need use directory from remote server that can be accessible only via rsync.
Can it be done without sync remote dir localy? (like sshfs)
Now this required so many attention just for some changed files ...
- 71
22
votes
3
answers
2k
views
rsync the file `a`b
Yeah, I know what you are thinking: "Who on earth names their file `a`b?"
But let us assume you do have a file called `a`b (possibly made by a crazy Mac user - obviously not by you), and you ...
- 32.7k
0
votes
1
answer
32
views
How to copy directory with rsync to remote USB pen drive correctly
How to copy dir-tree form Debian 11 to remote pen drive in a way, that I can restore this directory to the OS without any permission modifications?
A usb pen drive is mounted to /home/<user>/mnt ...
- 1,088
0
votes
0
answers
29
views
Good security practice for remote to local backup using rsync
I would like to perform a manual backup of a remote VPS (Debian 12) to my laptop using rsync. I want to backup the single files and directories to easily inspect them locally, and not e.g. a VM ...
- 1
0
votes
1
answer
25
views
How do I show progress on rsync without --progress similar to dd SIGUSR1
How do I show progress on rsync without --progress similar to sending SIGUSR1 to dd?
When I accidentally start rsync without parameters on a large file and I neither pass --progress or --partial. Is ...
- 341
0
votes
0
answers
21
views
preserve target's permissions on rsync?
I am updating files for directory A to directory B like this:
rsync -vcru --delete /directoryA/ directoryB
Can I preserve the ownerships and permissions of the (existing) target files?
directoryB is ...
- 398
1
vote
1
answer
51
views
Would rsync overwrite the current files if I loose access to the files?
Would rsync overwrite to the current files that I have if I loose the access to certain data directories?
rsync -amHAXxzvR --update --numeric-ids --progress --include='*/' --include=*.sas7bdat --...
- 25
1
vote
1
answer
44
views
Rsync ignore files and folders by wildcards
I would like to ignore any files or folders under a specific folder when I am downloading from remove server with a foldername and wildcard.
I want to exclude all files or folders under "/*.old/&...
- 25
1
vote
1
answer
34
views
Moving root partition to a new drive, how do I mount both drives?
I currently have my root partition on a 480GB Sata SSD. I recently bought a new NVME 2TB drive which I would like to boot off of.
Using Arch Linux btw. What I plan to do is the following:
Create a ...
- 55
2
votes
2
answers
72
views
How to make rsync dst timestamp reflect time at which content was last transferred from src?
I am rsyncing a file from src to dst across a network. I’m navigating the many rsync options on the Linux man page, and need some help please.
A process on the destination responds to "newly ...
- 125
1
vote
1
answer
37
views
Transferring very large dataset from cluster to a storage server
We have to move a set of very large data (in petabytes) from HPC cluster to a storage server. We have a high capacity communication link between the devices. However, the bottleneck seems to be a fast ...
- 111
1
vote
1
answer
46
views
How to determine which checksum `rsync` uses by default?
In the rsync image of output below:
I am unable to find the default checksum. Is it the first one,
or the one in brackets?
What is the bit-depth of just xxhash?
Found this DOCUMENTATION states the ...
- 25
0
votes
1
answer
27
views
How to Tranfser huge amount of Files to slow USB Drive (dd, cp, rsync, 7z, tar)
I have like 5GB of data and a very slow USB-Drive.
Should I use dd, cp or rsync?
Should I compress them first into for example 7z / tar or not?
In short: What is the best way and the best practices ...
- 27
0
votes
1
answer
18
views
Using rsync to query the checksum of a remote file
Can we use rsync to simply get the checksum of a remote file, without transferring it?
This could be useful for verifying the integrity of remote files without using a lot of bandwidth, if you don't ...
- 911
-1
votes
0
answers
50
views
Running rsync on Windows return error, but SCP doesn't
I'm trying to use a Windows server as a backup for my remote files.
Using SCP I can transfer files via SSH, but with rsync it returns error 12. Here are both commands:
scp -r user@000.000.000.000:/...
-2
votes
0
answers
33
views
rsync returning error 12
I'm trying to run a rsync command, but I'm getting error 12 (usually used when the remote dir doesn't exist) but when I run the scp command with the same remote dir it works and copies the file to my ...
0
votes
0
answers
40
views
SSH server allows login but refuses connection when trying to copy a file
I am able to log into my ssh server both using a password and also with key. But when I try to copy a local file to the ssh server (rsync or scp) it refuses the connection which is very strange.
So ...
- 1,111
0
votes
0
answers
36
views
Why does transfer of a TAR file slow down so significantly mid-transfer?
I'm trying to transfer a 30GB TAR file between two servers. My problem is transfer starts at a fast rate of roughly 10s MB/s, but after 11% of the file transfer the speed slows enormously to 100s KB/s....
- 1
0
votes
0
answers
29
views
migrate from rsync to cp
We are currently using "rsync" to copy a directory recursively. We are moving to a newer platform where "rsync" is not available and have been advised to use "cp" instead....
- 1
0
votes
1
answer
41
views
What to back-up in home partition?
The hard drive in the notebook I own is quite old. So, I have a habit of doing regular back-ups to avoid losing everything should it fail.
Rsync does fine job of backing-up the partition (it is a ...
- 81
0
votes
0
answers
14
views
rsync close after sending some files and then start over again
I have a Python program which continuously generates video fragments or chunks. Another Python program running in parallel sends these video fragments by rsync to my server. I am using:
rsync -avzhe ...
- 1
1
vote
2
answers
71
views
Using sshpass with rsync to accept a fingerprint
I'm using rsync to copy a file to a device:
sshpass -p <password> rsync -avzP <file> <user>@<IP address>:<folder>
This has worked fine in the past but I was trying to ...
- 137
0
votes
2
answers
42
views
How to add file markers to check if script is already running
I wonder if I can get some help with a project I'm working on. I have a Synology NAS. I found a Community Package that autoruns a script of my creation anytime a USB drive is plugged in to one of the ...
- 3
0
votes
0
answers
36
views
Copy folder from one Linux server to another - cloning files onto a backup server
I would like to copy selected data files/folders from our a production server to a backup server. I want to do this every night at midnight. The servers are basically side-by-side, on the same ...
- 13
0
votes
2
answers
40
views
Copy only the folders (and all their content) at the level of source path with rsync
I have the following tree, where f are files while src and d are directories (f could be hidden or not):
src/f1
src/f2
...
src/d1
src/d2
src/d3
...
I want to replicate the same structure in a dst ...
- 25
0
votes
0
answers
21
views
rsync: Get diff output of file and something similar as a merge
Use-Case
I use rsync to sync my files between two machines. Assuming I want to sync /important/file.txt from machine A to B, I did some changes on machine A t /important/file.txt and somenone else ...
- 157
2
votes
0
answers
79
views
How to make rsync skip existing files on destination
I have one Raspberry NAS and one backup server (192.168.25.25).
I would like to regularly backup the Raspebery NAS into the backup server where until now I have copied the files manually.
I drafted ...
- 21
2
votes
2
answers
64
views
does rsync calculate a checksum even when it won't be transferring any data?
From man rsync:
--existing
skip creating new files on receiver
(emphasis mine)
So, what I've done with apparent success:
rsync --existing -rtvhP "remote.machine:/photos/*" /photos
This ...
- 41
-1
votes
2
answers
81
views
rsync does not create destination folder as I want
I have the following text file
...
/a/b/c/f1
/a/b/c/f2
...
/a/b/c/d1
...
where f1 and f2 are files, while d1 is a directory. I would like to copy the exact hiearchy in my destination folder (/z).
...
- 25
0
votes
0
answers
20
views
How can I run an rsync demon in the private machine?
I have never used rsync before.
Suppose I have the following two machines:
I have to run rsync with a Python script in Machine-B so that it can periodically fetch new files from Machine-A to Machine-...
- 375
0
votes
2
answers
60
views
Using rsync for specific files and directories
I am trying to find the optimum way to back up dotfiles, and I have devised a way to do it. I am using git with rsync to back up all the dotfiles. The rsync script looks like this right now:
rsync -...
2
votes
0
answers
32
views
Rsync between systems with different symlink permission semantics (MacOS -> Ubuntu)?
I’d like to use rsync for backup of a MacOS machine to an Ubuntu server running ZFS. The problem is that on MacOS symlinks can have file permissions distinct from their referenced file; whereas on ...
- 141
0
votes
0
answers
78
views
How can I use rsync filters properly?
I'm having trouble in getting rsync to respect my .rsync-filter files; which, I guess, it is due to me not properly understanding how they work.
Basically, I have made a filter inside my $HOME folder ...
- 1,675
0
votes
0
answers
74
views
rsync was working, now I get "failed: Read-only file system (30)" when using
I'm using rsync and crontab to regularly backup a remote directory to my home directory. It was working as intended for around 48 hours. When I attempted to manually backup, I got this error:
root@me/...
- 103
0
votes
0
answers
28
views
how to backup remote location containing hard links, without rsync
I want to backup backintime-data from an internet-facing, remote machine running debian 11 (src). That is the hardlinks-part from the title.
I want to back it up to a (very dumb) NAS (dest) in the ...
1
vote
1
answer
65
views
rsyncd service ProtectSystem=off has no effect
I'm running Debian 11 with rsyncd 3.2, which has ProtectSystem=full as default.
I'd like to override that, so am using systemctl edit to do so. Here's what I'm putting in the override:
### Editing /...
- 182
0
votes
0
answers
17
views
Re-use existing files from another directory on the target site
Sometimes i just rearrange files on the source site, meaning, I move them around to other subdirectories. In such cases it would be nice if rsync can try to see, if some of the files it would have to ...
- 101
2
votes
1
answer
94
views
How to rsync only newer (not new) files that match a pattern
I'm trying to get rsync to copy only *.ko files that are newer than their match on a target local directory. I'm using --update but it seems that rsync still wants to copy files that are identical to ...
- 133
1
vote
1
answer
41
views
Failed to Start After Moving /var to Another Drive
I am running CentOS 7.
I have /var on a separate hard drive, and smartctl shows that it has some read error. Hence, I would like to move /var to another hard drive.
For the new hard drive, I used ...
- 213
0
votes
0
answers
47
views
can't log in via both GNOME Display Manager and tty
I reinstalled (Fedora 37) system and restored some vital files to my home directory by using rsync's --archive flag. Potentially after replacing some .bash* files new terminal sessions were closed ...
- 363
0
votes
0
answers
49
views
sync: chgrp <dir/file> failed: Permission denied (13)
Trying to sync two directories with the command
sudo rsync -avh --progress /src/ /dst/
It gives the error:
sync: chgrp <dir/file> failed: Permission denied (13)
I understand -a is trying to ...
3
votes
1
answer
96
views
Restarting rsync
I am using rsync to transfer some files using the command
rsync -acv --progress ./d-planetr ./c3-remotsn/
Suppose I stop a transfer and restart it again, will I be ensured that all file transfers ...
- 313
0
votes
1
answer
67
views
rsync size differences and checksums
I am using the rsync command
rsync -av dir_a dir_b
Does the command check whether the size is different and also does checksums ?
- 313
1
vote
1
answer
74
views
rsync update from FAT32 not working correctly
I'm using a FAT32 usb-key to mirror my data from work (linux machine) to home (macOS machine).
I discovered that FAT32 doesnt work well with the --archive option, so I'm using on macOS the command ...
- 203
0
votes
0
answers
41
views
rsync is ignoring empty file
I am using rsync -avz to backup files on remote server always to same directory once a week with incremental rsync. But my client insists that existing files which have some data is never overwriten ...
- 358
0
votes
1
answer
53
views
File bytes was increased after copying files from centos to centos server
I copied files from one server to another server using rsync. The folder file byte size, as measured with du -sb <folder>, increased from 31,755,289,904 bytes to 31,755,505,579.
I'm not sure how ...
- 1
1
vote
0
answers
79
views
Rsync NTFS and SMB share not working
I'm struggling with my rsync script to sync data (on Ubuntu) from an NTFS drive to a NAS share.
What I do:
Mount the NAS share:
sudo mount -t cifs -o user=$DST_USER,password=$DST_USERPW,rw,uid=$...
- 11
1
vote
1
answer
29
views
Saving the output of find to a variable in a remote server
# Declare the array
declare -a LC_lines
# Read the lines of the file into the array
readarray -t LC_lines < unique_running_jobs_directory.log
sshpass -p password ssh -n dummy@dummydomain.xx.yy.zz ...
- 117