0

Goal: synch a large number of large files (filesize>500MB) from one NAS (WD Mycloud) to another. rysnc is available on both. Both are connected via a Gigabit switch. NAS files can be moved with Windows Explorer at 40+MB/s, however the file metadata (dates) are changed to the current date. rsync runs at ~10-12MB/s.

If the files were copied with Windows Explorer (for speed) the data's creation date would reflect today's date. Could the file date creation and other related file meta-data be restored with a specific rsync option to match the source files?

Update: I have been advised by @RomeoNinov to update this question: An attempt to use rsync was half-successful and restored the dates of files, but not the folders.

rsync -ax --size-only --progress [email protected]:/shares/Public/Drivers /shares/Public 

I was advised that adding -v to the statement and this link would help explain why it was not changed. My attempt to decode was not successful.

The request is for a command line example (similar to above) that will sync the files and folder meta-data. Looking forward to clever responses and a good narrative.

4
  • 1
    For reference, the rsync specifics relating to speed are (also) being discussed at Diagnosing a rsync bottleneck. Aug 31, 2016 at 15:45
  • I guess you are using Explorer to copy locally between two (network) mount points but you are using rsync via ssh - that's slow. Try to use rsync (or just cp) like you use the Explorer.
    – rudimeier
    Aug 31, 2016 at 16:27
  • @rudimeier My gut tells me you have something, however, I did not udnerstand and am confused. Could you clarify with a command line example? Thanks
    – gatorback
    Sep 1, 2016 at 13:03
  • @gatorback Your rsync command line connects via ssh protocol to your NAS box. ssh is encrypted and slow because your NAS box probably does not have much cpu power. Your Explorer is probably just copying files from one "windows network drive" to another "windows network drive" (both smb protocol), thats much faster. So you should mount both of your NAS box shares via smb or nfs on your linux box. And then try rsync -a --size-only /mnt/shares1/... /mnt/shares2/ (local, no IP adresses or hostnames).
    – rudimeier
    Sep 1, 2016 at 17:32

1 Answer 1

4

You can use rsync -aX to have it sync the metadata (including permissions and timestamps — adjust flags as necessary to sync what you want), and --size-only to disable checking the timestamps for the quick-check algorithm. From the man page:

--size-only This modifies rsync’s "quick check" algorithm for finding files that need to be transferred, changing it from the default of transferring files with either a changed size or a changed last-modified time to just looking for files that have changed in size. This is useful when starting to use rsync after using another mirroring system which may not preserve timestamps exactly.

... which sounds like basically your situation. There is the possibility, of course, of missing some change which updated the file but left it the same size (say, switching some words around) — but if you're sure you run it right after the copy with no chance of the source files changing, that won't be an issue.

Note, though, that Unix has no standard "creation" date. (See this answer.) I don't know this will interact with whatever filesystem and metadata your NAS uses and provides.

3
  • @rudimeier Sure — updated to say "no standard" instead of "no".
    – mattdm
    Aug 31, 2016 at 16:24
  • Excited to see that the files were changes, but unsure why the folders were not changed. Not sure if should create a new thread or answer. rsync -axn --size-only --progress [email protected]:/shares/Public/Drivers /shares/Public
    – gatorback
    Sep 1, 2016 at 0:02
  • To understand the rsync process add a few -v (verbosity) flags and use andreafrancia.it/2010/03/… to understand the log output. That should help you understand why certain changes are not being propagated. Sep 1, 2016 at 1:09

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .