0

Goal: synch a large number of large files (filesize>500MB) from one NAS (WD Mycloud) to another. rysnc is available on both. Both are connected via a Gigabit switch. NAS files can be moved with Windows Explorer at 40+MB/s, however the file metadata (dates) are changed to the current date. rsync runs at ~10-12MB/s.

If the files were copied with Windows Explorer (for speed) the data's creation date would reflect today's date. Could the file date creation and other related file meta-data be restored with a specific rsync option to match the source files?

Update: I have been advised by @RomeoNinov to update this question: An attempt to use rsync was half-successful and restored the dates of files, but not the folders.

rsync -ax --size-only --progress root@192.168.8.109:/shares/Public/Drivers /shares/Public 

I was advised that adding -v to the statement and this link would help explain why it was not changed. My attempt to decode was not successful.

The request is for a command line example (similar to above) that will sync the files and folder meta-data. Looking forward to clever responses and a good narrative.

  • 1
    For reference, the rsync specifics relating to speed are (also) being discussed at Diagnosing a rsync bottleneck. – roaima Aug 31 '16 at 15:45
  • I guess you are using Explorer to copy locally between two (network) mount points but you are using rsync via ssh - that's slow. Try to use rsync (or just cp) like you use the Explorer. – rudimeier Aug 31 '16 at 16:27
  • @rudimeier My gut tells me you have something, however, I did not udnerstand and am confused. Could you clarify with a command line example? Thanks – gatorback Sep 1 '16 at 13:03
  • @gatorback Your rsync command line connects via ssh protocol to your NAS box. ssh is encrypted and slow because your NAS box probably does not have much cpu power. Your Explorer is probably just copying files from one "windows network drive" to another "windows network drive" (both smb protocol), thats much faster. So you should mount both of your NAS box shares via smb or nfs on your linux box. And then try rsync -a --size-only /mnt/shares1/... /mnt/shares2/ (local, no IP adresses or hostnames). – rudimeier Sep 1 '16 at 17:32
4

You can use rsync -aX to have it sync the metadata (including permissions and timestamps — adjust flags as necessary to sync what you want), and --size-only to disable checking the timestamps for the quick-check algorithm. From the man page:

--size-only This modifies rsync’s "quick check" algorithm for finding files that need to be transferred, changing it from the default of transferring files with either a changed size or a changed last-modified time to just looking for files that have changed in size. This is useful when starting to use rsync after using another mirroring system which may not preserve timestamps exactly.

... which sounds like basically your situation. There is the possibility, of course, of missing some change which updated the file but left it the same size (say, switching some words around) — but if you're sure you run it right after the copy with no chance of the source files changing, that won't be an issue.

Note, though, that Unix has no standard "creation" date. (See this answer.) I don't know this will interact with whatever filesystem and metadata your NAS uses and provides.

  • @rudimeier Sure — updated to say "no standard" instead of "no". – mattdm Aug 31 '16 at 16:24
  • Excited to see that the files were changes, but unsure why the folders were not changed. Not sure if should create a new thread or answer. rsync -axn --size-only --progress root@192.168.8.109:/shares/Public/Drivers /shares/Public – gatorback Sep 1 '16 at 0:02
  • To understand the rsync process add a few -v (verbosity) flags and use andreafrancia.it/2010/03/… to understand the log output. That should help you understand why certain changes are not being propagated. – SACHIN GARG Sep 1 '16 at 1:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.