Questions tagged [rsync]

rsync is a tool to efficiently copy directory hierarchies, locally or remotely, with powerful filters to decide what gets copied.

Filter by
Sorted by
Tagged with
0
votes
0answers
18 views

Move or copy selective files, while recreating directory structure

I'm looking for a command or commandline that can take a list of files of a subtree (via a pipe, etc.) and copy/move them to a new destination, while recreating the source directory structure. It ...
1
vote
0answers
12 views

How to sync specific files with unison?

I originally posted on SO but i realized this might be more suited here. I'm using unison on Linux (Manjaro) to keep some files synced between two local folders but I'm running into a problem. I have ...
0
votes
1answer
10 views

Rsync delta update, skip first N blocks (or %)

I have a bunch of very large "almost-append-only" files (meaning that a few bytes at the end are changed every day). I know of the -B, --block-size=BLOCKSIZE to check deltas in blocks. Is it ...
1
vote
1answer
36 views

Rsync include exclude

I know there are a billion questions of this out there. But I'm positive I have tried many of them and i am not able to make this work, so please, don't just mark this as duplicate I have a file ...
0
votes
0answers
5 views

rsync with sshpass to remote server and --from-files argument gives “sudo: no tty present and no askpass program specified” error

I know the correct answer here is to set up an ssh key, but I'm still curious to figure out how to solve this so I can learn something about how this works. I have this command: sudo su -c "...
0
votes
1answer
16 views

rsyncing with --files-from and -mtime option downloading old files that are in a newer directory

I've got the following command to try to sync only files less than about 36 hours old (2200 minutes): sudo su -c "rsync -vrt --delete --rsync-path='sudo rsync' --files-from=<(find '/home/dir1' ...
0
votes
0answers
18 views

rsync how to properly backup each destination file that it's replacing

I'm working on an script to easily install my dotfiles on a new machine. The basic functionality of installing all dependencies and my personal dotfiles is pretty much done. There's a few things that ...
0
votes
1answer
27 views

Remote Server: Permission denied (publickey) when using rsync with ssh with gpg via cronjob

I want to do regular backups of my remote VPS via a cronjob. Both systems run Debian 10. I have been following this guide and tweaked it to my liking. Relevant parts of the script: /root/.local/bin/...
0
votes
0answers
15 views

How to solve the add/delete problem when rsyncing?

I work on two different computers and want to keep certain directories synced between the computers. For example, I have a ~/scripts folder that has a lot of commonly used scripts in it. If I make a ...
0
votes
2answers
61 views

rsync not reporting on files to delete

I've had a look at this (and the forum thread here) and this. I've tried running in Python and also at the command line. I've double-checked: some files have definitely been deleted from the source, ...
4
votes
2answers
75 views

Why rsync on Linux does not preserve all timestamps (creation time)?

I am using the current rsync 3.2.3 version and when I run stat command it shows me this info for my file. stat '/test.txt' File: /test.txt Size: 0 Blocks: 0 IO Block: 4096 regular empty file ...
0
votes
0answers
20 views

Move out files via rsync, but keep local copies for a while - how?

I have a Linux server that periodically creates new data files I need elsewhere. I need to move any new data files to another PC periodically, but also keep local copies of them for a limited amount ...
0
votes
0answers
23 views

rsync endless creation of recursing directories

I've used rsync a bit before, but I'm a bit baffled with this. I've just started exploring WSL (installed Linux is Ubuntu 20.04) and this is one of the main things I'm interested in setting up. This ...
0
votes
0answers
19 views

How to rsync files between 2 remotes without ssh access to either?

Very similar to this question, but I have no ssh access to either remote. How to rsync files between two remotes? How do I transfer (rsync or otherwise) files between remote source and remote ...
0
votes
1answer
36 views

rsync exclude mid level directories

I am trying to create an rsync job that syncs src /foo/bar/.jpg to dest /foo/.jpg, skipping mid level directories but preserving the top level. i'm sure there is a way to do this with --include --...
0
votes
1answer
21 views

No space left on device error when using rsync to copy a directory to an external drive

I'm experiencing an issue in which rsync fails when I try to mirror my home directory to an external drive. The rsync invocation is (as a normal user): rsync -avz --progress --delete-before /home /mnt/...
1
vote
1answer
18 views

rsync not detecting out of disk space condition

I am copying a number of large files (50-500MB each). What I'm observing is that rsync continues to think it's syncing files even when df reports no disk space left. (Or actually, it reports some ...
1
vote
1answer
38 views

Find files in all directories by name in and copy to another directory using rsync

I have listed file names each per line in list.txt like below filename1.pdf file456.pdf file33.txt ....... I want to search by file name in a directory (including all subfolders in source) ...
0
votes
0answers
19 views

RSync Get Block by number

Does RSync allow to query, say, 145th block of the file avoiding iterating through the entire file? Something like random access
3
votes
1answer
16 views

Default rsync options per remote host

Is there a way to specify default rsync options per remote host? E.g. I have a NAS at 192.168.100.12; its rsync path is /opt/bin/rsync and its SSH port is 2222. It sucks to have to manually specify --...
1
vote
0answers
21 views

Why did rsync halt with errors but not display them?

I have a very large (> 18tb) backup operation over rsync that halted due to a misconfiguration. The progress display locked up mid-print: 289,792 100% 2.36MB/s 0:00:00 (xfr#8494957, ir-chk=...
0
votes
1answer
22 views

rsync delete older `--backup-dir` directories if sufficient space is not available to make the current backup

I create backup using rsync with the following two scripts: Script 1: #!/bin/bash rsync --dry-run --stats -ab --human-readable --inplace --delete-delay --debug=NONE --log-file=/media/blueray/...
1
vote
2answers
35 views

rsync --whole-file vs delta-transfer protocol CPU usage and copy speed

I was testing the copy speed of files over a LAN. I have generated 1GB of /dev/urandom data and copied if over 1Gbps network to another machine. The rsync transfer speed (just -a option) was around ...
1
vote
1answer
42 views

rsync exclude *2021* works but not *2020*

I have this rsync command that I use for years to backup photos from my smartphone. It runs normally except if I swap --exclude *2021* to --exclude *2020* In my real scenario, the error message is ...
0
votes
1answer
34 views

Synchronizing with rsync outputs error “Invalid argument (22)” for directories with dots and other symbols in their name

I tried to synchronize /dir1 (ext4) and /dir2 (ntfs) using rsync -azP, but got these errors: rsync: recv_generator: mkdir "dir2/X.Y." failed: Invalid argument (22) rsync: recv_generator: ...
0
votes
0answers
23 views

copying files to different directories that file names match with the content of that directory

I have a folder containing these files (some of them are symlinks) SP116651.h38.bam -> SP116651.h38.cram SP116651.h38.bam.bai -> SP116651.h38.cram.crai SP116651.h38.cram SP116651.h38.cram.crai ...
0
votes
1answer
26 views

Make sure two folders are identical on compressed zfs and ext4

Summary I have copied (rsync --archive) a folder from an ext4 filesystem to a zfs file system with compression on. Now, I'm trying to verify that both folders are identical so that I can safely delete ...
1
vote
2answers
31 views

FreeBSD 12.x: rsync a specific folder to my destination folder

as above I think it's best explained with an example. /mnt/source/source-folder -> sfolder1 -> sfolder2 -> sfolder3 -> sfolder4 The "sfolder" contains 10s of folders ...
0
votes
0answers
21 views

Copy recursively multiple directories each to its own path

Task: copy recursively multiple directories from local to remote, each to its own path: copy lib1 to /loc/user/src/lib copy lib2 to /loc/user/src/lib/libx I'm trying to use rsync but get the ...
0
votes
1answer
31 views

rsync: read list of files from pipe

I can transfer file /tmp/file using rsync to remote server: rsync -R -av /tmp/file root@server:/ but how can I provide the list of files to be transfert from a pipe? I tried using the --files-from= ...
0
votes
2answers
18 views

Rsync progress waits a long time after finish

When I am running rsync to copy big file (within directory on local hard drive) + rsync -P bigfile1.img bigfile2.img bigfile1.img 6,512,705,536 100% 413.76MB/s 0:00:15 (xfr#1, to-chk=0/1) it ...
0
votes
0answers
11 views

Rsync option (ignore-missing-args) for remote source not working

I'm trying to use rsync actually inside loop to backup a list of directories from remote. I just want to ignore if src directory do not exists. I got from most of the forums that --ignore-missing-args ...
0
votes
0answers
14 views

lsyncd dont work properly

Config-file "syncd.conf.lua" is located at: /etc/lsyncd "syncd.conf.lua" content: settings { logfile = "/var/log/lsyncd/lsyncd.log", statusFile = "/var/log/...
1
vote
1answer
65 views

rsync keep access time (atime), how?

I am using rsync-3.2.3 on Fedora 33 (GNOME). But how can I keep the access time (atime) for my files and folders? I can only keep the modified time (mtime) with this command: rsync -t
7
votes
3answers
507 views

How would you gracefully handle this snippet to allow for spaces in directories?

I have a series of commands, e.g.: ssh -i key 10.10.10.10 mkdir -p "${DEST_PATH}/subdir1" "${DEST_PATH}/subdir2" rsync "${SOURCE_PATH}" "$DEST_HOST:${DEST_PATH}/...
3
votes
1answer
47 views

Checksum of binary changes after rsync-ing to remote machine [closed]

I'm attempting to copy a binary to a remote machine using rsync, however it appears to corrupt in some way, with both file and md5sum noting the files difference, rsync on the other hand does not ...
2
votes
2answers
39 views

How to get rid of .d..t (timestamp change) outputs from rsync?

when I compare two directories with rsync app, I can see there are many outputs about timestamp change. For example: $ rsync --dry-run -avh -i --delete --size-only --checksum --stats dir1/ dir2/ ... ....
0
votes
0answers
47 views

Need rsync options for changing permissions of a folder + files together

I give the user the option to enter 3 digit number for permission say 775 I use ansible where I provide rsync_opts: --chmod:F775 (synchronize module) to change the permission of the transferred file ...
0
votes
1answer
17 views

lsyncd for files-mirroring at localhost: Parameter “targetdir” unknown

I'm trying to use lsync for mirroring files. Either source and target files located at localhost. This is settings: settings { logfile = "/var/log/lsyncd/lsyncd.log", statusFile = &...
0
votes
3answers
34 views

Sync huge number of files from one server through ssh

I need to speed up the process of syncing log files between servers. The machine that generates the logs (LOGMACHINE) creates them in a tree like this: /Files /Files/LOGS1/ /Files/LOGS1/logFiles/ /...
0
votes
1answer
39 views

SSH No route to host in local network after using rsync

So, this is my setup: I have computer A and a computer B, both with Ubuntu 20.01. Each computer has openssh-server working just fine. Yesterday, I used rsync to copy a large file from A to B, and it ...
1
vote
2answers
74 views

why is rsync ignoring a newer directory here?

I have two directories containing alot of subdirectories. I want to compare the directories and find the the subdirectories which are missing in the second one and those who are newer in the first ...
0
votes
0answers
18 views

Syncing renamed files without copying

I've renamed a bunch of files and moved some to other folders. Now I want to sync all these changes to another filesystem, that already contains the original files. Something like rsync with --delete. ...
2
votes
2answers
75 views

How to bypass RemoteCommand option in ssh_config

I have defined a ssh_config file with all the hosts to which I connect on a regular basis. I like to start/connect to a tmux session upon connection to the host, so I've added the line RemoteCommand ...
0
votes
1answer
67 views

Rsync fails to create symlinks when syncing with an ExFAT partition

I tried to sync with an ExFAT partion on an external HDD using the command: rsync -avvP --remove-source-files /path/to/sourceDir /path/to/destDir However I got this type of error several times: rsync:...
1
vote
2answers
62 views

rsync - What is the difference between using * and ** in the exclude directive?

Following on from my other question about rsync. I am trying to understand how include/exclude directives use patterns to match against the names of files in a source directory to decide which are to ...
1
vote
1answer
37 views

Birth time after Modify time in stat command

I have a folder with a Birth time later than its Modify time: File: Pictures Size: 4096 Blocks: 8 IO Block: 4096 directory Device: 804h/2052d Inode: 31724953 Links: 4 Access: ...
1
vote
1answer
106 views

Detecting Directory Moves with Rsync

The command I am currently using to backup one HDD to another (locally, not remotely) is rsync --info=PROGRESS2,BACKUP,DEL -ab --human-readable --inplace --delete-after --debug=NONE --log-file=/media/...
0
votes
0answers
27 views

rsync 100% cpu on remote machine

I observed that rsync shoot up to 100% cpu on my local server. I've machine-A and machine-B . On machine-B I've certain files which I do compress them before moving them to an other directory and from ...
0
votes
3answers
99 views

rsync only certain files types from specified sub-directories

Here is my source dir structure: ../openwrt ../openwrt/BackupOfPackages ../openwrt/BackupOfSettings ../openwrt/BackupOfImages I want to use rsync to mirror that to the destination dir. I only want to ...

1
2 3 4 5
34