5

Had a question on NFS mounts and how they interact with transferring files on a low level. I'm trying to understand the latency involved with transferring files from within the same mount.

Say you SSH into a VM that has a mount setup. The VM is in USA and the mount is in Europe. Now execute the following command:

sudo mv /mnt/serverInEurope/dir1/file.txt /mnt/serverInEurope/dir2/file.txt 

Does the VM in USA read the file, just to write it back to the Europe mount?

Second question is very similiar:

sudo mv /mnt/serverOneInEurope/file.txt /mnt/serverTwoInEurope/file.txt 

If I'm transferring from one mounted server in Europe to another using a VM in the USA, will the VM read the data locally before it transfers from Europe mount to Europe mount? Or is the mv'ing of a file intelligent enough to execute the transfer entirely between the mounts in Europe?

These are very crucial distinctions because I'm transferring petabytes or more of information within different servers in Europe.

Thanks for your time.

  • Are these mounts NFS? SMB? Something else? – roaima May 27 '15 at 22:47
  • They are NFS mounts – abeauchamp May 27 '15 at 22:53
8

Using mv for a file or folder within an NFS mount will apply the operation remotely. (See this list of API functions or this overview.) This example will execute almost immediately regardless of the size of the file, provided that dir1 and dir2 are part of the same mountpoint:

mv /mnt/serverInEurope/dir1/file.txt /mnt/serverInEurope/dir2/file.txt

Using mv to move a file or folder between mountpoints will require the client to process the data. In this scenario the data will perform a double hop across the Atlantic, even if serverOneInEurope and serverTwoInEurope are in the same physical rack:

mv /mnt/serverOneInEurope/file.txt /mnt/serverTwoInEurope/file.txt

In this second instance it would be preferable to gain access to serverOneInEurope and have it directly transfer data to serverTwoInEurope. Failing that, spin up a VM in the same datacentre and mount both shares "locally".

  • Awesome, thanks for the clear and concise answer. – abeauchamp May 27 '15 at 23:09
  • using s3fs from aws, it did just hanged while renaming a folder (mv folder folder_new) containing 30k items. Not even possible to kill it. Now 1 hour and going on... – Mehdi Mar 14 at 13:32
  • @Mehdi that sounds totally different to the question/answer we have here. I suggest you ask a new question. Ping me here with a link and I'll suggest an answer for you. – roaima Mar 14 at 13:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.