I have 2 services, both operate over the same interface.
Service A goal is keep high bandwidth while sending massive amount of data.
Service B goal is low latency.

Service B packets should always be in favor of Service's A packets.
I need a TC structure to be able to :

  • Rate limit both service A & B
  • Give service B packets priority with 0% latency affect by service A packets.
  • Let each service utilize the whole line (or up to its limit) if the other service isn't transmitting.

I tried about an htb structure where I have class htb classid x which may be rate/ceil limit and qdisc prio (say handle y:0) below as child (it shall auto create i.e. classes y:1, y:2 & y:3) and use filters by src ip to redirect packets to y:1 / y:2.
However, it doesn't seem to work.
Both class x and it's children traffic seem to be 0. (used tc -s class/qdisc/filter show dev dev to see)
When watching the filters I can clearly see the "hits" so the data was supposed to get redirected correctly.

Here are the commands I execute :

tc qdisc add dev dev root handle 1: htb
tc class add dev dev parent 1:0 classid 1:1 htb rate 10gbit ceil 10gbit
# class x
tc class add dev dev parent 1:1 classid 1:2 htb rate 10gbit ceil 10gbit
# auto creates classes 21:1, 21:2 and 21:3
tc qdisc add dev dev parent 1:2 handle 21: prio
# example for service b filter (latency driven)
tc filter add dev dev parent 1:0 prio 2 u32 match ip src x.x.x.x/32 flowid 21:1
# example for service a filter
tc filter add dev dev parent 1:0 prio 2 u32 match ip src x.x.x.x/32 flowid 21:2

Haven't done this in a long time... take my answer with a grain of salt.

The filter parent might have to be the PRIO qdisc itself (so you have HTB filters, and PRIO filters, ...). Otherwise PRIO might reclassify packets by itself according to priomap.

This is what it looked like in an old script of mine (FairNAT, you can find the whole thing on GitHub) and I'm pretty sure it used to work back then... packets were not filtered by IP but marked by iptables in this case (which is also worth a try if you suspect the ip filter to be dodgy).

# Create a prio qdisc with 4 classes. All P2P traffic goes into class 4.
        $BIN_TC qdisc add dev $UC_DEV parent 1:$UC_MARK handle $UC_MARK: prio \
                          bands 4

# Add a filter for IPP2P to this qdisc. The rest depends on TOS.
        $BIN_TC filter add dev $UC_DEV parent $UC_MARK: protocol ip \
                       handle $(($UC_MARK+1)) fw flowid $UC_MARK:4

PRIO is not really a good choice most of the time. It is too aggressive, might completely starve all other traffic.

Having just a single HTB class might also not have the desired effect. Limiting to 10GE also sounds a bit strange to me, if that's the theoretical link speed which isn't ever reached in practice, it just won't do anything.

HFSC might be more suitable to the task. Documentation is even worse than HTB but it is better able to both rate limit and prioritize (HTB does not really prioritize anything at all).

  • 1. If I use PRIO as root qdisc I can't do any rate limit for the following classes. 2. I know the src ip filtering works fine. 3. I've tested myself the same scenario you describe here and it works fine, but it doesn't answer a lot of the requirements. – SagiLow Jul 18 '16 at 21:19
  • 4. I know about the starvation possibility, this is one of the requirements, all service B packets are favored over service's A packets. 5. 10gbit is my line bandwidth, I can't see why this is a bad idea. (it works great btw). 6. Ill read about HFSC but since I already have the HTB hierarchy as part of my system I prefer sticking with it (unless of course it's not possible to do what I need). Thank you very much ! – SagiLow Jul 18 '16 at 21:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.