Questions tagged [rsync]

rsync is a tool to efficiently copy directory hierarchies, locally or remotely, with powerful filters to decide what gets copied.

Filter by
Sorted by
Tagged with
0 votes
2 answers
120 views

Are Rsnapshot backups self contained - how to save into zip?

I have some old backups done by rsnapshot. The directories are: /backups/week.0 /backups/week.1 /backups/week.2 /backups/week.3 /backups/week.4 I'd like to keep only one copy and delete the others. ...
Danijel's user avatar
  • 188
0 votes
2 answers
74 views

Copy gitignored files from host A to host B (rsync?)

I have a git repo where several files containing sensitive info are gitignored. These files are located in various subdirectories at various depths. Host A has all these files in place. I've cloned ...
ivan's user avatar
  • 1,868
0 votes
1 answer
184 views

Similar program to rsync but to instead use a shared cache directory or source?

With rsync, the files or data delta (difference) is compared directly between the source and destination. Is there an alternative program that can compare the delta between the source, destination and ...
Chris Stryczynski's user avatar
0 votes
1 answer
152 views

Copy new files from NAS to External Hard Drive with rsync

I want to copy new files from my NAS to my external hard drive. I did copy the new files manually up to this point. Now I want to use a script to automate this for me. However everything I tried so ...
dehlen's user avatar
  • 103
-1 votes
1 answer
401 views

Why is rsync/scp slow from one server but fast from another?

I have 300 GB of large files located in server A. I am at home trying to copy those files, and both scp and rsync are painfully slow (80 KB/s or less). I also have access to another server B. I can ...
Daniel Duque's user avatar
2 votes
0 answers
151 views

Download tool to keep multiple ISOs up to date

I currently use this script (https://gist.github.com/Phill93/3c2b3503810a71bb98d6edb950d7aab2) to keep my ISO collection up to date. I use it to download ISOs via Rsync or HTTP/S. Now I would like to ...
Phill93's user avatar
  • 21
0 votes
0 answers
16 views

Make Rsync log its decision about hardlink creation

This is an X-Post also asked on rsync mailing list. I know it is a often discussed topic how rsync decide about using hardlinks or copy a file. Even if content is unchanged problems are often file ...
buhtz's user avatar
  • 865
1 vote
1 answer
118 views

rsync include exclude is excruciatingly slow

When I use rsync to copy files matching a pattern à la https://unix.stackexchange.com/a/2503/288916 this works but it is excruciatingly slow. find locates the matching files much faster. (Like more ...
Kvothe's user avatar
  • 413
0 votes
0 answers
74 views

rsync between two machines with ssh and MFA without password

I have two machines that are accessible with ssh + MFA. I need to rsync regularly between these two but would like to avoid doing MFA each time. Setting up a keypair is not an option since MFA will ...
Ketan's user avatar
  • 9,226
0 votes
1 answer
248 views

Rsync with date filter over ssh

I use the following command successfully rsync -e 'ssh' -avr [email protected]:/home/mikrotik /bck/mikrotik/ How can I add date filter to this command? I would like to sync only files that are newer ...
mikia's user avatar
  • 103
0 votes
2 answers
54 views

rsync and delete other dir

My script has --delete --delete-after for rsync. I have a directory with name releases in same dir of basedir of rsync. I don't want to delete releases directory after each running my script. Which ...
PersianGulf's user avatar
  • 10.8k
0 votes
0 answers
17 views

can rsync recognise files have been moved? [duplicate]

Scenario: I back up to external (ext) drive 1 and sync it to ext drive 2 (to have 2 backups). I archived a large directory (dir) on ext drive 1 by moving it into a new dir named 'archive'. Then i ...
neverdimed's user avatar
0 votes
0 answers
403 views

NFS4 performance tuning for use with rsync

To open, I know that this is a bad idea, and should be avoided. However, we currently have no choice as our NFS server is a Netapp SAN, which means we can't just run rsync to the file server. What I ...
Thegs's user avatar
  • 686
3 votes
0 answers
182 views

MacOS files to NAS with rsync --iconv repeats the sync over and over

According to this hint and similar advice I am using the --iconv option in rsync (version 3.2.7) to sync file with umlauts (ä ü ö ...) to my Synology NAS. However the --iconv option does not work as ...
Haegar's user avatar
  • 31
0 votes
2 answers
102 views

How to rsync multiple enumerated files

Suppose I have files data_1.mat, data_2.mat, ..., data_n.mat. How to I rsync those files to another folder in one line of code? The only way I know is do it in a for loop: for i in 1..n do rsync '...
Eleven Chen's user avatar
1 vote
1 answer
44 views

Why does rsync always marks all folders as changed?

I'm on macOS Ventura and use rsync 3.2.7 to synchronize files between a NAS and a external hard drive. When I run rsync -avP --delete /Volumes/NAS/photos /Volumes/EXTERNAL_DRIVE rsync marks all ...
user15110545's user avatar
1 vote
1 answer
298 views

rsync: excluding a directory from the source

It appears in the code below, rsync ignores --exclude. How would I correct that, and extend it to multiple directories? $ rsync --help | grep Usage Usage: rsync [OPTION]... SRC [SRC]... DEST $ man ...
Erwann's user avatar
  • 677
0 votes
0 answers
558 views

rsync permission denied - but they have full 777?

It is getting late and I am loosing my mind ... I have a data drive (NTFS) which I used to share in an Windows/Ubuntu dual boot system. After an upgrade it is currently Windows 10 and Ubuntu running ...
territrades's user avatar
0 votes
1 answer
105 views

rsync: [sender] out of memory: my_alloc caller (file=lib/pool_alloc.c, line=143)

I am trying to rsync a dir that includes a subdir with a ridiculous amount of files. So it was not a complete surprise when I got: [sender] out of memory: my_alloc caller (file=lib/pool_alloc.c, line=...
Ole Tange's user avatar
  • 35.5k
0 votes
1 answer
135 views

Rsync script running manually but not via systemd service/timer

I've got a handful of bash scripts (all essentially clones of each other) which each does an rsync to a different dnf/yum repo to pull that repo down to our local mirror. The scripts all run correctly ...
Dulux-Oz's user avatar
0 votes
0 answers
30 views

I need to rsync from Ora1 vpn (as user michal) and rsync files into Ora2 vpn (as user srv_data). User 'michal' is in wheel grp on both Ora1 and Ora2

Hello Unix enthusiasts. I have two vpms called Ora1 and Ora2. I need to synchronize the contents of /DATA/ directory from Ora1, into the same /DATA/ directory located on the Ora2 vpm. Ora1 /DATA/ ...
michal roesler's user avatar
0 votes
0 answers
16 views

How to enforce rsync client-side transfer options via rsyncd daemon (such as --max-delete)?

Client is backing up with a QNAP which supports rsync to an rsyncd target remote storage I run. The QNAP has limited ability (esp via the GUI) to choose options for the rsync xfer. I would like to ...
math's user avatar
  • 311
2 votes
2 answers
801 views

rsync to exclude everything except specific files

How to exclude all files except specific files with rsync? I'm trying to only backup specific files with rsync, to exclude all files , except some specific files, namely the .md files, and solutions ...
xpt's user avatar
  • 1,520
2 votes
1 answer
389 views

Restore Rsync Backup Of System On Any Linux Distribution

I used the following alias to back up my root directory on Ubuntu 22.04 LTS to an external flash drive alias backup='sudo rsync -aAXHS --info=progress2 --delete --exclude={"/dev/*","/...
Growing My Roots's user avatar
0 votes
3 answers
197 views

Compare directories ignoring time differences of 1h +- 2s

I often want to synchronize my Music folders between my Android mobile phone and my Linux PC. The files on my phone are stored on an SD card using FAT32 and can come from either my Windows laptop or ...
DodgyCodeException's user avatar
0 votes
0 answers
178 views

rsync implications of ACL backups between different source & dest file systems

I'm having an issue using rsyncd (the daemon-ized version of rsync) to back up files from my macOS to my Synology server. Here's the command that fails: % rsync -rlcAXtgoDiv --fake-super ~/scripts ...
Seamus's user avatar
  • 2,894
0 votes
0 answers
43 views

Random rsync running in the background on Debian

I have recently had some lag in my system, when saving a file in vim. I noticed that an rsync process was running. It was copying the ~/.openshot_qt dir (openshot is not currently installed on my ...
Zoltan K.'s user avatar
  • 493
0 votes
1 answer
47 views

SCP from a distant server with a cumulative size limit

I want to copy a number of files from a distant server to local computer, such that the total size of the files does not exceed certain limit, as I do not locally have enough disk space; and the files ...
Radek Vavřička's user avatar
0 votes
2 answers
74 views

Methodology to copy data from one machine to many others in a fast way

I found some bottleneck on my workflow which is as follows. I do have one master computer which needs to send data to other node machines. This is done in a for loop such as: for all nodes: rsync <...
Tobi's user avatar
  • 155
1 vote
1 answer
31 views

Directory usage from remote server that can be accessible only via rsync

Need use directory from remote server that can be accessible only via rsync. Can it be done without sync remote dir localy? (like sshfs) Now this required so many attention just for some changed files ...
tuytuy20's user avatar
22 votes
3 answers
2k views

rsync the file `a`b

Yeah, I know what you are thinking: "Who on earth names their file `a`b?" But let us assume you do have a file called `a`b (possibly made by a crazy Mac user - obviously not by you), and you ...
Ole Tange's user avatar
  • 35.5k
0 votes
1 answer
359 views

How to copy directory with rsync to remote USB pen drive correctly

How to copy dir-tree form Debian 11 to remote pen drive in a way, that I can restore this directory to the OS without any permission modifications? A usb pen drive is mounted to /home/<user>/mnt ...
Hölderlin's user avatar
  • 1,186
0 votes
0 answers
99 views

Good security practice for remote to local backup using rsync

I would like to perform a manual backup of a remote VPS (Debian 12) to my laptop using rsync. I want to backup the single files and directories to easily inspect them locally, and not e.g. a VM ...
tomgag's user avatar
  • 1
0 votes
1 answer
203 views

How do I show progress on rsync without --progress similar to dd SIGUSR1

How do I show progress on rsync without --progress similar to sending SIGUSR1 to dd? When I accidentally start rsync without parameters on a large file and I neither pass --progress or --partial. Is ...
Philippe's user avatar
  • 431
1 vote
1 answer
121 views

Would rsync overwrite the current files if I loose access to the files?

Would rsync overwrite to the current files that I have if I loose the access to certain data directories? rsync -amHAXxzvR --update --numeric-ids --progress --include='*/' --include=*.sas7bdat --...
Ozioh's user avatar
  • 25
1 vote
1 answer
292 views

Rsync ignore files and folders by wildcards

I would like to ignore any files or folders under a specific folder when I am downloading from remove server with a foldername and wildcard. I want to exclude all files or folders under "/*.old/&...
Ozioh's user avatar
  • 25
1 vote
1 answer
976 views

Moving root partition to a new drive, how do I mount both drives?

I currently have my root partition on a 480GB Sata SSD. I recently bought a new NVME 2TB drive which I would like to boot off of. Using Arch Linux btw. What I plan to do is the following: Create a ...
JLCarveth's user avatar
2 votes
2 answers
329 views

How to make rsync dst timestamp reflect time at which content was last transferred from src?

I am rsyncing a file from src to dst across a network. I’m navigating the many rsync options on the Linux man page, and need some help please. A process on the destination responds to "newly ...
Tomasso's user avatar
  • 125
1 vote
1 answer
236 views

Transferring very large dataset from cluster to a storage server

We have to move a set of very large data (in petabytes) from HPC cluster to a storage server. We have a high capacity communication link between the devices. However, the bottleneck seems to be a fast ...
Ikram Ullah's user avatar
1 vote
1 answer
1k views

How to determine which checksum `rsync` uses by default?

In the rsync image of output below: I am unable to find the default checksum. Is it the first one, or the one in brackets? What is the bit-depth of just xxhash? Found this DOCUMENTATION states the ...
Neail's user avatar
  • 41
0 votes
1 answer
93 views

How to Tranfser huge amount of Files to slow USB Drive (dd, cp, rsync, 7z, tar)

I have like 5GB of data and a very slow USB-Drive. Should I use dd, cp or rsync? Should I compress them first into for example 7z / tar or not? In short: What is the best way and the best practices ...
Bog's user avatar
  • 989
1 vote
1 answer
778 views

Using rsync to query the checksum of a remote file

Can we use rsync to simply get the checksum of a remote file, without transferring it? This could be useful for verifying the integrity of remote files without using a lot of bandwidth, if you don't ...
Nate Eldredge's user avatar
0 votes
0 answers
121 views

SSH server allows login but refuses connection when trying to copy a file

I am able to log into my ssh server both using a password and also with key. But when I try to copy a local file to the ssh server (rsync or scp) it refuses the connection which is very strange. So ...
Martin Zeltin's user avatar
0 votes
0 answers
106 views

Why does transfer of a TAR file slow down so significantly mid-transfer?

I'm trying to transfer a 30GB TAR file between two servers. My problem is transfer starts at a fast rate of roughly 10s MB/s, but after 11% of the file transfer the speed slows enormously to 100s KB/s....
wziard's user avatar
  • 1
0 votes
1 answer
139 views

What to back-up in home partition?

The hard drive in the notebook I own is quite old. So, I have a habit of doing regular back-ups to avoid losing everything should it fail. Rsync does fine job of backing-up the partition (it is a ...
S. N.'s user avatar
  • 287
1 vote
2 answers
780 views

Using sshpass with rsync to accept a fingerprint

I'm using rsync to copy a file to a device: sshpass -p <password> rsync -avzP <file> <user>@<IP address>:<folder> This has worked fine in the past but I was trying to ...
parsley72's user avatar
  • 289
1 vote
2 answers
74 views

How to add file markers to check if script is already running

I wonder if I can get some help with a project I'm working on. I have a Synology NAS. I found a Community Package that autoruns a script of my creation anytime a USB drive is plugged in to one of the ...
Matt's user avatar
  • 13
0 votes
0 answers
253 views

Copy folder from one Linux server to another - cloning files onto a backup server

I would like to copy selected data files/folders from our a production server to a backup server. I want to do this every night at midnight. The servers are basically side-by-side, on the same ...
RDK's user avatar
  • 23
0 votes
2 answers
105 views

Copy only the folders (and all their content) at the level of source path with rsync

I have the following tree, where f are files while src and d are directories (f could be hidden or not): src/f1 src/f2 ... src/d1 src/d2 src/d3 ... I want to replicate the same structure in a dst ...
user9952796's user avatar
2 votes
0 answers
265 views

How to make rsync skip existing files on destination

I have one Raspberry NAS and one backup server (192.168.25.25). I would like to regularly backup the Raspebery NAS into the backup server where until now I have copied the files manually. I drafted ...
Daniele's user avatar
  • 21

1 2
3
4 5
42