Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

I'm running CentOS 5.7 and I have a backup utility that has the option of dumping its backup file to stdout. The backup file is rather large (multiple gigabytes). The target is an SSHFS filesystem. To ensure that I don't hog the bandwidth and degrade the performance of the network, I would like to limit the speed with which data is written to the "disk".

How can I limit the ability of stdout based on a byte number? For example, limiting a process's ability to write to about 768Bps.

share|improve this question
5  
@msw this question is about rate-limiting stdout, the previous question was about limiting network traffic. Different beasts... –  voretaq7 Mar 14 '12 at 19:14
5  
@msw Last I checked, the point of the SE network is to collect a massive amount of knowledge in specific areas. In this case, it's about *nix. Limiting the output of stdout is different than limiting bandwidth. Just because the underlying problem is the same doesn't mean that future visitors won't find each question useful independent of the other. Remember that the Qs&As are just as much for googlers as for the actual askers, which is why deleting answered questions is such a cardinal sin here. –  MDMarra Mar 14 '12 at 19:27
add comment

3 Answers

up vote 21 down vote accepted

You can add a rate limiting tool to your pipeline.

For example there is pv which has a rate-limiting option:

-L RATE, --rate-limit RATE

Limit the transfer to a maximum of RATE bytes per second. A suffix of "k", "m", "g", or "t" can be added to denote kilobytes (*1024), megabytes, and so on

An alternative is the tool buffer which has:

   -u microseconds

After every write pause for this many microseconds. Defaults to zero. (Surprisingly a small sleep, 100 usecs, after each write can greatly enhance throughput on some drives.)

share|improve this answer
add comment

I would be tempted to dump this on a local disk instead of writing it directly to the remote volume. This way you can use rsync with --bwlimit to easily limit the transfer.

One of the benefits to this approach is that if there is a connectivity hiccup to the remote machine, the transfer is interrupted, but the backup itself doesn't explode in a blaze of glory.

share|improve this answer
    
Noted. I do so wish that rsync was a viable solution in this rather limited situation I find myself in. =| –  Wesley Mar 14 '12 at 19:33
add comment

pv (pipe view)

echo asdlfkjasdf | pv -q -L 12
share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.