18

I am currently using rsync to transfert large file from a host to another. I would like to limit the debit for this transfert. After reading the man page I found the option --bwlimit which is expecting a KB/s value.

I want to limit the transfert to arround 100Mbit/s So I used the following settings :

rsync --progress -v --bwlimit=10000 --temp-dir=/tmp srv:/tmp/* /tmp/

But this doesn't seems to be respected according to the output of rsync :

file.sql.gz
   570851702 100%   33.90MB/s    0:00:16 (xfer#5, to-check=34/39)
file.sql.gz
    11195140 100%    7.45MB/s    0:00:01 (xfer#6, to-check=33/39)
file.sql.gz
    51439169 100%   19.41MB/s    0:00:02 (xfer#7, to-check=32/39)
file.sql.gz
       34022 100%    1.55MB/s    0:00:00 (xfer#8, to-check=31/39)
file.sql.gz
     2305698 100%    6.96MB/s    0:00:00 (xfer#9, to-check=30/39)
file.sql.gz
     2152942 100%    2.73MB/s    0:00:00 (xfer#10, to-check=29/39)
file.sql.gz
    86655633 100%   19.33MB/s    0:00:04 (xfer#11, to-check=28/39)
file.sql.gz
  3614476032  40%   34.82MB/s    0:02:28

If I'm not wrong : 19.33MB/s ~= 154Mbit/s

So what is wrong here ?

PS : rsync version 3.0.6 protocol version 30

Thanks for your help !

8
  • Here it works with rsync 3.0.4 (Ubuntu), 3.0.6 (openSUSE) and 3.0.9 (Ubuntu) using the same parameters. You have to provide more information about your system.
    – scai
    Mar 12, 2013 at 15:42
  • Does this help? stackoverflow.com/questions/10064316/…
    – mreithub
    Mar 12, 2013 at 17:32
  • @mreithub The answer in this post is indeed interesting but actually Hugo seems to transfer rather large files.
    – scai
    Mar 12, 2013 at 18:52
  • 1
    Have you tried to measure the overall time and then figure out what's KB/S?
    – user55518
    Mar 5, 2014 at 17:38
  • 1
    It does not function everytime, but I had better luck with trickle --- see tuxradar.com/content/control-your-bandwidth-trickle
    – Rmano
    Mar 6, 2014 at 3:15

4 Answers 4

16

I think you experienced some "burstiness" as mentioned in the manpage:

--bwlimit=RATE
          ... 
          Some "burstiness" may be seen where rsync writes out a block 
          of data and then sleeps to bring the average rate into compliance.

So as I asked before the overall ratio at the end of your job was at last limited to 10M.

1
  • "burstiness" might be the best thing I've heard in a long time. Mar 24, 2021 at 13:53
7

As mentioned in rsync man page, it transfer in KByte/sec:

--bwlimit=KBPS          limit I/O bandwidth; KBytes per second
3
  • 3
    That's what I used : 10 000 kilobytes = 78.12500 megabits
    – Hugo
    Dec 10, 2013 at 13:33
  • 3
    You didn't pay attention to what i said. The rsync use Byte instead of bit, so you have to calculate the bandwidth in bit and covert it to byte
    – shgnInc
    Mar 8, 2014 at 6:07
  • I have the same issues, when i put --bwlimit=100, i obtain 275kB/s, very regular between 265-285 kB/s, no burstiness. What is wrong in here?
    – Brethlosze
    Jul 14, 2017 at 15:05
3

I want to limit the transfer to arround 100Mbit/s So I used the following settings:

rsync --progress -v --bwlimit=10000 --temp-dir=/tmp srv:/tmp/* /tmp/

100 Mbit/s is (approximately) 10 MByte/s, which is indeed 10,000 KByte/s. If you're hitting rsync burstiness then maybe the separate tool trickle will help:

trickle -s -d 10000 -u 10000 -t 10 -l 100 rsync --progress -v --temp-dir=/tmp srv:/tmp/* /tmp/
1

The value shown is the effective transfer rate which includes speeds up from compression and the delta algorithm, this can be easily observed by seeing large numbers if an interrupted rsync session is restarted with partial transfers enabled (ex --append-verify).

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .