1

I want to find those files which I cannot copy to another location such that I could do Rsync. I have an external HDD and my system´s HDD. The external HDD has files which my system is missing. I am thinking how to do this.

I can find those files/directories with permission denied -messages by 

find / ! -perm /u+r

where some permissions are 

drwx------ 1  501 root       6 Oct 22  2014 PortableCV_WebCruiter_20141022

The following command may work from commandlinefu.com, since remote data can be considered here as external HDD

rsync --rsync-path 'sudo rsync' username@source:/folder/ /local/

which

Rsync remote data as root using sudo If your user has sudo on the remote box, you can rsync data as root without needing to login as root. This is very helpful if the remote box does not allow root to login over SSH (which is a common security restriction).

How can you rsync files with permission denied problems between external HDD and system HDD?

Improve this question

3 Answers 3

Reset to default
2

If you want to skip the unreadable files:

find /external/hd/folder ! -readable -print >unreadable_files
rsync -a --exclude-from=unreadable_files /external/hd/folder/ /local/folder

This assumes that you're using GNU find, and that your filenames don't have embedded newlines.

If you want to copy all files, regardless of whether they are readable or not on the external disk:

sudo rsync -a /external/hd/folder/ /local/folder
sudo chown -R masi:masi /local/folder

This assumes you have enough privileges to run sudo rsync and sudo chown.

Improve this answer
7
  • 1
    @Masi Well in the vast majority of cases files are unreadable because of ownership, not because of permissions. There are uses for files that aren't readable by their owners, but those tend to be rare and pretty specialized. I can't think of any reason for an usual data file to be unreadable by its owner.
    – lcd047
    Commented Jul 16, 2015 at 6:31
  • One experiment done with 0.5TB transfer from external HDD to my HDD 1TB Lenovo G30-50. The system Debian 8.1 fails to handle so much data such that the login is disturbed. There must be some lower level problems in the system. I opened a new thread about this experiment here unix.stackexchange.com/q/216628/16920
    – Léo Léopold Hertz 준영
    Commented Jul 17, 2015 at 5:57
  • 1
    The problem with rsync is that it uses a lot of memory. The more files you want to synch, the more memory it needs. That's very likely the reason why your system crashed: rsync run out of memory, and OOM started committing "random acts of terrorism" (as DJB likes to put it). People that recommend using it to backup entire disks should keep this in mind. You might still be able to copy the files with cp -a. It's less efficient than rsync, but it also uses a lot less memory. If that still doesn't work you could try copying subdirectories one at a time.
    – lcd047
    Commented Jul 17, 2015 at 6:13
  • Any control system to control the memory?
    – Léo Léopold Hertz 준영
    Commented Jul 17, 2015 at 6:15
  • The memory needed by rsync? The only way AFAICT is to copy fewer files. shrug
    – lcd047
    Commented Jul 17, 2015 at 6:17
1

If you mount the external HDD to your system locally, then it's not remote. Unless I misunderstood and your HDD is running a rsync daemon and say, connecting over a network? But forget that, can you just mount the drive locally and as @mjturner says, all you need to do is make sure you run rsync as root and it's a direct copy sudo rsync -a /mount/for/hdd/folder/ /local/ ??

Improve this answer
0

Why not just use rsync to copy all of the files on the external HDD to the system HDD? It will not re-copy files that already exist.

Of course, be sure to do the copy as root to ensure that you copy all files and preserve permissions.

Improve this answer

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .