50

I have a CentOS 5.7 server that will be backing up its files nightly. I am concerned that visitors to the various sites that the server hosts will experience degraded performance while the backup is transferring across the network.

Is it possible to limit a process's maximum allowed throughput to a network interface? I would like to limit the SSH-based file transfer to only half of my available bandwidth. This could be on the server or client side; that is, I'd be happy to do this on either the client that initiates the connection or the server that receives the connection.

(Unfortunately, I can't add an interface to dedicate to backups. I could increase my available throughput, but that would merely mean that the network transfer would complete faster, but still max the total capacity of the connection while doing it.)


Some Background

Perhaps some background is in order. Stepping back, I had a problem with not having enough local space to create the backup itself. Enter SSHFS! The backup is saved to what is ostensibly a local drive so that no backup bits are ever on the web server itself.

Why is that important? Because that would seem to invalidate the use of the venerable rsync --bwlimit. rsync isn't actually doing the transfer nor can it because I can't even spare the space to save the backup file.

I can hear you ask: "So wait, why do you even need to make a backup file? Why not just rsync the source files and folders?" Because an annoying thing called "Plesk" is in the mix! This is my client-facing web host which uses Plesk for convenience. As such, I use Plesk to initiate the backups because Plesk adds all sorts of extra magic to the backup that makes consuming it during a restoration procedure very safe.

sad face

3
  • 1
    See serverfault.com/questions/52027/… Mar 14, 2012 at 0:52
  • 1
    Another possibility for my situation, which incidentally doesn't exactly answer the specific question, is to use ionice to throttle the writes that a process can make. Since I'm writing to an SSHFS filesystem, I can drop the class of the backup process to 3 to make it completely give way to any other process that wants to write. That way I get the effect that I want which is to never degrade a site visitor's experience because of the backup hogging bandwidth.
    – Wesley
    Mar 14, 2012 at 4:20
  • One question, your ssh uses compression? "Compression yes" to your .ssh/config?
    – Zlatko
    Aug 4, 2012 at 0:40

5 Answers 5

52

One option that I just discovered is to use trickle.

trickle is a portable lightweight userspace bandwidth shaper. It can run in collaborative mode (together with trickled) or in stand alone mode.

trickle works by taking advantage of the unix loader preloading. Essentially it provides, to the application, a new version of the functionality that is required to send and receive data through sockets. It then limits traffic based on delaying the sending and receiving of data over a socket. trickle runs entirely in userspace and does not require root privileges.

3
  • 1
    This is the one that solved my problem. I was having a problem with the bitcoin daemon randomly sucking all my bandwdith when a client decided that I was the person they should download the whole blockchain from. Nov 12, 2012 at 21:09
  • 1
    or sudo apt-get install trickle
    – ggll
    Jan 6, 2017 at 16:06
  • trickle doesn't always work though... rtfm for more details
    – Eric
    May 3, 2020 at 19:41
28

You can use iptables to mark a packet (--pid-owner ...), then use tc to shape the traffic. Also "--sid-owner" can be used to include threads and children of that process.

http://www.frozentux.net/iptables-tutorial/iptables-tutorial.html#OWNERMATCH

Match --pid-owner
Kernel 2.3, 2.4, 2.5 and 2.6
Example iptables -A OUTPUT -m owner --pid-owner 78
Explanation This match is used to match packets based on the Process ID (PID) that was responsible for them. This match is a bit harder to use, but one example would be only to allow PID 94 to send packets from the HTTP port (if the HTTP process is not threaded, of course). Alternatively we could write a small script that grabs the PID from a ps output for a specific daemon and then adds a rule for it. For an example, you could have a rule as shown in the Pid-owner.txt example

5
  • I think this ties with my solution for the best way to do this. --pid-owner doesn't actually select based on the process, but the process owner. I'd have to create a special user to launch the process as and then filter based on that owner just so I can be sure I'm shaping traffic just from that specific process and not, say, multiple daemons that might have been launched from a generic user.
    – Wesley
    Mar 14, 2012 at 18:41
  • @Wesley That's not what the man page says: "--pid-owner processid Matches if the packet was created by a process with the given process id." linux.die.net/man/8/iptables
    – Ajedi32
    Apr 22, 2016 at 15:38
  • 1
    This answer would be much better if it included an example of how to use iptables and tc for this purpose. The given iptables -A OUTPUT -m owner --pid-owner 78 example doesn't seem to be complete (as it only matches packets, it doesn't say how to "mark" them) and tc isn't explained at all.
    – Ajedi32
    Apr 22, 2016 at 15:45
  • To mark a packet you need to add something like -j MARK --set-mark 1. For more details see: wiki.archlinux.org/index.php/… Jul 14, 2016 at 18:30
  • Instead of iptables, it's possible to use cgroups with net_cls classifier: unix.stackexchange.com/a/328349/1254 May 23 at 13:41
23

If you can write to a pipe (or stdout), you can install the pv (pipe viewer) command. It was originally written to display the progress of data transferred through a pipe.

tar cvf - /files/to/backup | pv -L 512k > /your/file/on/sshfs

   -L RATE, --rate-limit RATE
          Limit the transfer to a maximum of RATE  bytes  per  second.   A
          suffix of "k", "m", "g", or "t" can be added to denote kilobytes
          (*1024), megabytes, and so on.
3
  • This is actually the answer I think I'll use! However, it's not exactly an answer to the specific question I originally asked. Alas, the question kind of morphed but was still focused on limiting the network speed of a process. However, you can contribute to this new question I asked: unix.stackexchange.com/q/34174/4232
    – Wesley
    Mar 14, 2012 at 18:53
  • Thanks! I was doing a similar thing and ended up doing ssh my-remote-server bash -c "'find / -xdev|cpio -o|gzip -c1'"|pv --rate-limit 1M > my-remote-root.cpio.gz.
    – clacke
    Nov 14, 2012 at 8:45
  • 1
    Could have combined it with ionice as well, but it wasn't allowed on that particular server. ssh my-remote-server ionice -c3 bash -c "'find / -xdev|cpio -o|gzip -c1'"|pv --rate-limit 1M > my-remote-root.cpio.gz
    – clacke
    Nov 14, 2012 at 8:47
8

I use rsync with the --bwlimit=KBPS option for the same reason.

Our 1 Gbit ethernet is easily able to swamp our old SCSI320 DAS RAID, and essentially DOS's some our older production boxes that rely on it for its NFS stores.

0
4

How are you transferring the data? (rsync over ssh? scp? sftp? something else?)

rsync will allow you to limit bandwidth (see the option --bwlimit=KBPS). rsync -e ssh --bwlimit ..

Alternatively, you could setup a qdisc or equivalent to do fancy rate limiting, but I suspect that in your case this would be severe overkill. Documentation on this is available at The Linux Advanced Routing and Traffic Control HOWTO

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.