1

I need to copy approximately 400 files of approximately 25 MB each to a davfs-mounted cloud storage. I have tried the following commands to limit the CPU-load and IO-load:

nice -n 15 ionice -c 3 rsync -avhW --no-compress --progress /src/ /dst/

My destination folder dst is a davfs-mounted cloud storage. Whenever I transfer a single file, the rsync just takes a few seconds -- at least it shows up to be this fast:

sending incremental file list
xxx
26.70M 100%   15.75MB/s    0:00:01 (xfer#1, to-check=0/1)
sent 26.70M bytes  received 31 bytes  7.63M bytes/sec
total size is 26.70M  speedup is 1.00

I don't see any rsync or nice process running, but the system reacts very slowly, as the data transfer is still running in the background. There is only one davfs process for the mounted cloud storage. After a few minutes, my system is responsive again and the file transfer is finished.

netstat shows an active connection to the cloud storage.

How can I limit the bandwidth to the davfs-mounted cloud storage to prevent my system from slowing down?

0

Rsync (now) has a bwlimit option:

   --bwlimit=RATE
          This option allows you to specify the maximum transfer rate for the data  sent  over  the
          socket,  specified  in units per second.  The RATE value can be suffixed with a string to
          indicate a size multiplier, and may be a fractional value (e.g.   "--bwlimit=1.5m").   If
          no suffix is specified, the value will be assumed to be in units of 1024 bytes (as if "K"
          or "KiB" had been appended).  See the --max-size option for  a  description  of  all  the
          available suffixes. A value of zero specifies no limit.

          For  backward-compatibility  reasons,  the  rate limit will be rounded to the nearest KiB
          unit, so no rate smaller than 1024 bytes per second is possible.

          Rsync writes data over the socket in blocks, and this option both limits the size of  the
          blocks  that  rsync  writes, and tries to keep the average transfer rate at the requested
          limit.  Some "burstiness" may be seen where rsync writes out a block  of  data  and  then
          sleeps to bring the average rate into compliance.

          Due  to  the  internal  buffering  of  data, the --progress option may not be an accurate
          reflection on how fast the data is being sent.  This is because some files can show up as
          being  rapidly  sent  when  the data is quickly buffered, while other can show up as very
          slow when the flushing of the output buffer occurs.  This may be fixed in a  future  ver‐
          sion.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.