I'm attempting to limit the bandwidth available to a running ruby process to prevent it flooding our link and affecting other traffic. The process is downloading large files over HTTP.
I've found numerous people suggesting trickle
as a convenient userland-only option (like https://unix.stackexchange.com/a/34123/160146).
I've tested it with an scp download and confirmed it limits the download rate:
trickle -s -d 50 scp [email protected] localfile.dat
I've also tested a python 2.7 script and confirmed it limits the download rate:
# cat test.py
import urllib
testfile = urllib.URLopener()
testfile.retrieve("http://example.com/bigfile.dat")
# trickle -s -d 50 python test.py
Finally, I've tested with a basic ruby script and trickle has no effect. The scripts maxes out my downstream bandwidth:
# cat test.rb
require 'net/http'
Net::HTTP.get_response(URI.parse("http://example.com/bigfile.dat"))
# trickle -s -d 50 ruby test.rb
I've tested with ruby 2.1, 2.2 and 2.3 with no difference in behaviour.
I notice the trickle documentation says it only works with dynamically linked programs, but I'm fairly sure thats the case for me. I'm using the ruby provided by Debian in /usr/bin
, and ldd
reports linked libraries:
# ldd /usr/bin/ruby2.3
linux-vdso.so.1 (0x00007fffdcdc6000)
libruby-2.3.so.2.3 => /usr/lib/x86_64-linux-gnu/libruby-2.3.so.2.3 (0x00007f277e11a000)
libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f277defd000)
libgmp.so.10 => /usr/lib/x86_64-linux-gnu/libgmp.so.10 (0x00007f277dc79000)
libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f277da75000)
libcrypt.so.1 => /lib/x86_64-linux-gnu/libcrypt.so.1 (0x00007f277d83e000)
libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f277d538000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f277d194000)
/lib64/ld-linux-x86-64.so.2 (0x0000558796396000)
Interestingly, trickle can rate limit my ruby process if I use the curb (libcurl bindings) gem:
# cat test.rb
require 'curb'
http = Curl.get("http://example.com/bigfile.dat"))
# trickle -s -d 50 ruby test.rb
Does that suggest that ruby's default Net::HTTP
library uses sockets in a way that's incompatible with trickle?
I'm out of ideas. Is there a fixed reason why trickle doesn't work with ruby or is there something I can do to get them working together?