2
  • RHEL 7.9 x86-64
  • high end dell servers with Xeon cpu's, 512gb ram, intel nic card
  • I am the only user on the server(s), and there is no other work load on them
  • cisco 1gbps wired LAN
  • data.tar is ~ 50 gb
  • /bkup is NFS mounted as vers=4.1 and sync
  • a scp data.tar backupserver:/bkup/ runs at 112 MB/sec consistently; I've seen this for 5+ years and believe this to be correct
  • a rsync -P data.tar /bkup runs at 55 MB/sec consistently; this one is copying over NFS
  • running both the copies at the same time, scp drops from 112 to 55, and rsync over NFS drops from 55 to 35
    • when one finishes, the other copy speed resumes to the original rate

why? and how can I improve speed over NFS?

Improve this question
4
  • If you can scp to the backupserver then a better use of rsync would be across the network rather than through NFS. In your scenario rsync thinks it's copying local to local and switches off many of its optimisations
    – Chris Davies
    Jan 4 at 20:01
  • I only used rsync -P as an easy way to show copy speed. If I use cp then I have to do math and look at my watch, but I'm still seeing half the speed of SSH. why?
    – ron
    Jan 4 at 20:21
  • Ok. What (client-side) mount options are you using? Particularly but not exclusively *sync and [rw]size ? Have you tried async instead of sync?
    – Chris Davies
    Jan 4 at 20:38
  • 2
    Try async nfs instead of sync.
    – Edward
    Jan 4 at 20:40

1 Answer 1

Reset to default
1

The primary cause is probably the fact that the NFS share is mounted with the sync option. This theoretically improves data safety in scenarios where the server might suddenly disappear or the client might unexpectedly disconnect, but it also hurts performance.

Using the sync mount option is equivalent to the application calling fsync() on the file it is writing to after every call to write(). IOW, every time the client submits an IO request, it has to wait for that to finish before it can submit another. This has a nontrivial impact on how fast data can be written even when used with local filesystems, but network filesystems make it much worse (because at least a full network round-trip is required after each IO request before the next one can be issued). If you have the time, you can actually see this type of effect yourself by trying to copy a file that is a few hundred MB in size using TFTP as compared to SCP. TFTP bakes this type of synchronization into the protocol at a basic level, and does so in a way that each individual packet has to be acknowledged before the next can be sent, so it will likely get even less performance than you’re seeing from NFS.

Provided you are using responsible software that atomically replaces files and handles copies sanely, you can probably safely switch to async mode for NFS to avoid this issue.

Improve this answer
2
  • I edited /etc/fstab and changed sync to async and did a mount -o remount /bkup` and an existing running copy going at 55 MB/sec ramped up within 10 seconds to ~ 270 MB/sec shown via rysnc -P, over a wired 1gbps LAN.
    – ron
    Jan 7 at 13:45
  • doing a mount | grep bkup to see the mount options, shows neither sync nor async. comments? How can I tell, or see, specifically if a mount is one or the other?
    – ron
    Jan 7 at 13:48

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .