Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

Is it possible to throttle (limit) the download speed of wget or curl ?

Is it possible to change the throttle value while it is downloading ?

share|improve this question
3  
No it is not possible to change the speed while downloading. Have a look at gui download managers like fatrat or multiget –  Ulrich Dangel May 23 '12 at 15:48
2  
You can however suspend the process via SIGSTOP or ctrl+z and resume it later with SIGCONT or fg. This should pause the download process. –  Ulrich Dangel May 23 '12 at 15:54
    
I wonder if there's a generic way to throttle any running program, i.e. via setting an environment variable. –  Ehtesh Choudhury Oct 21 at 13:57

4 Answers 4

up vote 34 down vote accepted

Yes both wget and curl support limiting your download rate. Both options are directly mentioned in the man page.

curl

   --limit-rate <speed>
          Specify the maximum transfer rate you want curl to use. 
           This feature is useful  if you  have a limited pipe and 
           you'd like your transfer not to use your entire bandwidth.

          The given speed is measured in bytes/second, unless a suffix 
          is appended.  Appending  'k'  or 'K' will count the number
          as kilobytes, 'm' or M' makes it megabytes, while 'g' or 'G' 
          makes it gigabytes. Examples: 200K, 3m and 1G.

E.g: curl --limit-rate 423K

wget

   --limit-rate=amount
       Limit the download speed to amount bytes per second.  Amount may
       be expressed in bytes, kilobytes with the k suffix, or 
       megabytes with the m suffix.  For example, --limit-rate=20k will limit 
       the retrieval rate to 20KB/s.  This is useful when, for
       whatever reason, you don't want Wget to consume 
       the entire available bandwidth.

E.g: wget --limit-rate=423k

share|improve this answer
2  
Is it possible to change it dynamically while a download is in progress ? –  Gautam May 23 '12 at 14:34
1  
@GautamK No as neither wget nor curl are interactive programs. –  Ulrich Dangel May 23 '12 at 15:28
3  
@GautamK For one large file, if the server accepts it, you can kill the wget or curl process and resume with wget -c or curl -C. If you really need to reconfigure a running process, use trickle with a daemon — but the setup is a bit complicated. Alternatively, look into traffic shaping — again, the setup if complicated. –  Gilles May 23 '12 at 23:08

You have to throttle the network-link that is being used for the transfer.

share|improve this answer
1  
How would one achieve this? –  uther May 23 '12 at 20:32

It is possible to limit the traffic rate using the tc and netem tools but this will limit the rate for the network interface of the computer. I am assuming that you use only wget or curl and no other application is exchanging traffic through the network interface.

tc uses Token Bucket Filter (TBF) to control the rate.

One example of TBF would be as follows (ref. http://www.lartc.org/manpages/tc-tbf.html):

To attach a TBF with a sustained maximum rate of 0.5mbit/s, a peakrate of 1.0mbit/s, a 5kilobyte buffer, with a pre-bucket queue size limit calculated so the TBF causes at most 70ms of latency, with perfect peakrate behaviour, issue:

# tc qdisc add dev eth0 root tbf rate 0.5mbit \ burst 5kb latency 70ms peakrate 1mbit \ minburst 1540

Another example of usign tc and netem would be as follows (found in http://www.linuxfoundation.org/collaborate/workgroups/networking/netem):

There is no rate control built-in to the netem discipline, instead use one of the other disciplines that does do rate control. In this example, we use Token Bucket Filter (TBF) to limit output.

To add the delay of each packet going/coming through the interface eth0

 # tc qdisc add dev eth0 root handle 1:0 netem delay 100ms

to add the data rate in tbf, packet buffer size and maximum burst limit

 # tc qdisc add dev eth0 parent 1:1 handle 10: tbf rate 256kbit buffer 1600 limit 3000

To see the list of rules assigned in tc for interface eth0

 # tc -s qdisc ls dev eth0

The output of the above command would be as below

 qdisc netem 1: limit 1000 delay 100.0ms
  Sent 0 bytes 0 pkts (dropped 0, overlimits 0 )
 qdisc tbf 10: rate 256Kbit burst 1599b lat 26.6ms
  Sent 0 bytes 0 pkts (dropped 0, overlimits 0 )

Check on the options for buffer and limit as you might find you need bigger defaults than these (they are in bytes)

share|improve this answer

2 years later I will throw this tidbit in, while wget and curl are not interactive, at least wget (and possibly curl but i do not know for sure) has the -c switch. So if you need to change your speed in the middle of a download and you presumably used the -c switch with the --limit-rate=x (which stands for continue from where I left off downloading earlier, by the way) then you could stop wget and restart it with a different speed and it would change.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.