I'm attempting to limit the bandwidth available to a running ruby process to prevent it flooding our link and affecting other traffic. The process is downloading large files over HTTP.
I've found numerous people suggesting
trickle as a convenient userland-only option (like https://unix.stackexchange.com/a/34123/160146).
I've tested it with an scp download and confirmed it limits the download rate:
trickle -s -d 50 scp [email protected] localfile.dat
I've also tested a python 2.7 script and confirmed it limits the download rate:
# cat test.py import urllib testfile = urllib.URLopener() testfile.retrieve("http://example.com/bigfile.dat") # trickle -s -d 50 python test.py
Finally, I've tested with a basic ruby script and trickle has no effect. The scripts maxes out my downstream bandwidth:
# cat test.rb require 'net/http' Net::HTTP.get_response(URI.parse("http://example.com/bigfile.dat")) # trickle -s -d 50 ruby test.rb
I've tested with ruby 2.1, 2.2 and 2.3 with no difference in behaviour.
I notice the trickle documentation says it only works with dynamically linked programs, but I'm fairly sure thats the case for me. I'm using the ruby provided by Debian in
ldd reports linked libraries:
# ldd /usr/bin/ruby2.3 linux-vdso.so.1 (0x00007fffdcdc6000) libruby-2.3.so.2.3 => /usr/lib/x86_64-linux-gnu/libruby-2.3.so.2.3 (0x00007f277e11a000) libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f277defd000) libgmp.so.10 => /usr/lib/x86_64-linux-gnu/libgmp.so.10 (0x00007f277dc79000) libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f277da75000) libcrypt.so.1 => /lib/x86_64-linux-gnu/libcrypt.so.1 (0x00007f277d83e000) libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f277d538000) libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f277d194000) /lib64/ld-linux-x86-64.so.2 (0x0000558796396000)
Interestingly, trickle can rate limit my ruby process if I use the curb (libcurl bindings) gem:
# cat test.rb require 'curb' http = Curl.get("http://example.com/bigfile.dat")) # trickle -s -d 50 ruby test.rb
Does that suggest that ruby's default
Net::HTTP library uses sockets in a way that's incompatible with trickle?
I'm out of ideas. Is there a fixed reason why trickle doesn't work with ruby or is there something I can do to get them working together?