Rsync operation not permitted

Question

I have backed up my data previously using rsync -avz on my late Linux Mint installation. Now when I try to backup my data with the same command on Windows 10 Ubuntu subsystem, the rsync just ignores my destination files and tries to copy all the files from source to destination. I also get 2 different error messages on multiple files:

rsync: chgrp "/mnt/p/file" failed: Operation not permitted (1)

and

rsync: mkstemp "/mnt/p/file" failed: Operation not permitted (1)

I don't really care about permissions, all I want to do is backup my data to external hard drive.

I tried to google a solution for this, but none of these helped:

sudo chown -R user:user /mnt/p # No effect

rsync -rlptgoD --chmod=ugo=rwX # Throws same errors and does not copy any files at all

rsync -avz --no-o --no-g --no-perms # Throws errors "failed to set times on"

EDIT:

I tried one more provided option: rsync -rtDvz, it throws rsync: failed to set times on "/mnt/p/file": Operation not permitted (1) on every file. The transfer also takes super long time, since it tries to modify all the files even though it is not able.

I also noticed one new thing, when I run the command sudo chown -R user /mnt/p and check the file permissions with ls -l p, it shows that all the file permissions are root root. For some reason I can't change the file permissions on my external hard drive files.

Oh yeah and I should have told you that I am trying to backup data to external USB stick which is Fat32 file system... Sorry! I don't always realize what stuff is important and what is not when asking questions.

Answer 1

The rsync man page says that -a is the same as -rlptgoD. I recommend making this replacement in your rsync command, i.e. rsync -rlptgoDvz, and then removing individual options which break under NTFS. I expect you will at least need to remove -p (permissions), -g (group), -o (owner), and maybe also -l (symbolic links). Removing all of these would leave you with:

rsync -rtDvz

Answer 2

I've been searching for a solution to this issue for a while when using rsync in WSL Ubuntu (Windows 10) to sync with a network attached volume and I think this solution may work for your USB stick as well. What worked for me was setting the user and group when mounting the drive. For example:

sudo mount -t drvfs -o rw,noatime,uid=1000,gid=1000 'P:\' /mnt/p

Using the --update/-u parameter is a neat trick to get it working by skipping files with dates/times that are newer on the destination, but if you want to more accurately match the dates/times on the files (like I did), then it was a less desirable solution. Of course, the values on your system for uid and gid, may differ, so check /proc/mounts to see what other mounts may already be using:

cat /proc/mounts

Once permissions for dates/times are resolved, you can drop the --update and shorten the command to something like:

rsync -rt --progress --modify-window=1 "$SOURCE" "$DEST"

I think the --modify-window=1 might still be useful, since it seemed like in other research I did that it might be needed in some cases in Windows:

https://pserver.samba.org/rsync/daylight-savings.html

Answer 3

Using this command solved the problem:

rsync --progress --modify-window=1 --update --recursive --times

It still throws the "failed to set times on" errors, but at least the syncing is now working correctly. I found the answer from here: https://serverfault.com/a/144475

Answer 4

I had this same problem when trying to transfer files from my desktop computer to a phone with Android running a SSH daemon (SimpleSSHD version 27) The problem was being caused by some files containing the ? character in their names. I removed the character and everything worked fine.

Below you can see two commands that I run to confirm that ? (question mark) was causing the problem.

In the command shown below, I tried to transfer a file named foo.txt. The command is successfully executed (see exit code 0 in the last line)

$ rsync -a --progress -vv ~/Downloads/foo.txt myphone:/sdcard/Download
opening connection using: ssh myphone rsync --server -vvlogDtpre.iLsfxCIvu . /sdcard/Download  (7 args)
sending incremental file list
delta-transmission enabled
foo.txt
              2 100%    0.00kB/s    0:00:00 (xfr#1, to-chk=0/1)
total: matches=0  hash_hits=0  false_alarms=0 data=2

sent 115 bytes  received 66 bytes  120.67 bytes/sec
total size is 2  speedup is 0.01
$ echo $?
0

In the command shown below, I tried to transfer a file named foo?.txt. The command results in an error (see exit code 23 in the last line)

$ rsync -a --progress -vv ~/Downloads/foo?.txt myphone:/sdcard/Download
opening connection using: ssh myphone rsync --server -vvlogDtpre.iLsfxCIvu . /sdcard/Download  (7 args)
sending incremental file list
delta-transmission enabled
foo?.txt
              2 100%    0.00kB/s    0:00:00 (xfr#1, to-chk=0/1)
rsync: mkstemp "/sdcard/Download/.foo?.txt.Vn5pbV" failed: Operation not permitted (1)
total: matches=0  hash_hits=0  false_alarms=0 data=2

sent 116 bytes  received 157 bytes  182.00 bytes/sec
total size is 2  speedup is 0.01
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1326) [sender=v3.2.4]
$ echo $?
23