3

I am currently using rsync to copy a 73GB file from a Samsung portable SSD T7 to an HPC cluster.

rsync -avh path/to/dataset [email protected]:/path/to/dest

The following applies:

  1. My local machine (where my T7 is connected) is a VirtualBox VM running Ubuntu 20.
  2. The T7 transfer speeds should be up to approx. 1000MB/s.
  3. Network gives me an approximate upload speed of 7.9Mbps.
  4. Rsync transfer speed is probably bottlenecking this to 1-5MB/s according to this answer.

The problem is that the move is still not done after 9 hours. According to 1, using cp instead is better with an empty directory (for the first time). I do not understand this or whether it is actually true. Can someone explain this?

8
  • It's going over a network? You need to look at the path and find which link has the minimum speed. That 1000MB/s also represents 8Gbps.
    – Bib
    May 15, 2022 at 21:05
  • @Bib, sorry I've only just begun using Linux heavily recently. What do you mean by finding which link please?
    – mesllo
    May 15, 2022 at 21:14
  • I reach the HPC cluster via password-protected ssh, if that is what you asked for. With regards to what the network is, sorry I'm not sure what that means.
    – mesllo
    May 15, 2022 at 21:52
  • 1
    8Mbps is around 1 MB per second. Whatever tool you use to copy 73GB (uncompressed) over this network (upload from your local box), time will be more than 73000/60/60 ~ 20 hours, It would be 1 day. Seems better rsync with -P and probably -z.
    – thanasisp
    May 15, 2022 at 23:52
  • 1
    The post you linked refers a case with many files, between a hard disk and a usb device, locally. Your case is different.
    – thanasisp
    May 15, 2022 at 23:54

1 Answer 1

5

You say that,

  1. Your network speed is approximately 8 Mb/s.
  2. Your rsync runs at about 1-5 MB/s.

Given that 1 MB/s is approximately 10 Mb/s I'd say that rsync is doing you a big favour.

Me, I'd probably have added compression with -z, and since you're using rsync with sensible flags over a network connection it's probably with interrupting it and restarting with compression. It'll just pick up where it left off.

Quick calculation: 73 GB is 73,000 MB*, which is 730,000 Mb. Approximately. You've got a network speed of 8 Mb/s, so that means the copy should take around 730,000/8 = 91,250 seconds. 25 hours, assuming theoretical maximum use of network bandwidth.

* Why 1000:1 instead of 1024:1? Partly because GB:MB strictly is 1000:1, but largely because this is an approximation calculation
‡ Why 10:1 instead of 8:1 as suggested by 8 bits to 1 byte? Two reasons: (a) it's an O(n) approximation, and (b) there's packet/protocol overhead to consider.


Now to try and answer the question strictly as asked, which is, "Is cp faster than rsync during the first run?". If you're using cp you need to have something managing the transport between the local and remote servers; that could be something like sshfs or NFS, or alternatively you might mean scp.

  1. cp over an NFS mounted filesystem. With a well-tuned network this is probably fairly efficient.
  2. cp over sshfs. This will include not only encryption overheads of ssh but also the translation between kernel and userspace for the FUSE implementation of the filesystem. Inefficient.
  3. scp (implicitly over ssh). This will include encryption overhead from ssh. Acceptable, and might futher benefit from link compression (-C).
  4. rsync (implicitly over ssh since we've not mentioned rsyncd). This will include encryption overhead from ssh. Acceptable, and might futher benefit from protocol compression (-z).

I've not taken quantitative performance metrics; these are qualitative assessments. However, note the comments on a similar answer Why is scp so slow and how to make it faster?, although to be fair that's discussing comparative transfer rates of multiple files rather than one large one.

However, where rsync wins across a network connection is that it's restartable. With the right flags (i.e. --partial) even mid-way through a transfer. For a single 73GB file taking around 25 hours to transfer that's a massive potential advantage.

15
  • 73 GB = 584000 Mbit --> 20h 17m at 8 Mbit/s (21h 46m if it's 73 GiB).
    – Kusalananda
    May 17, 2022 at 6:04
  • 1
    Just so you know, the rsync took a little just under 25 hours :)
    – mesllo
    May 17, 2022 at 13:21
  • 1
    @mesllo I've attempted to answer the question for you strictly as asked May 17, 2022 at 13:34
  • 1
    Don't use --append unless you really understand exactly under what circumstances it's safe to do so. It is always safe not to use --append. May 17, 2022 at 20:49
  • 1
    If you want a sanity check on the file contents, md5sum will provide it May 17, 2022 at 20:50

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .