I have an interesting problem.
We have a Linux server with Debian Stretch installed. The server has a single CPU E5-2680 v2 and Intel 82599ES 10G Dual Port NIC's. There are no other services (like HTTP and so on) running on it.
All seems good, until I enable traffic shaping. The shaping rules consist of
htb qdisc on the
There is a three step hashing filter:
Four rule filter to select a large subnet. The rules look like this:
protocol ip u32 ht 800:: match ip src 10.15.0.0/18 hashkey mask 0x0000ff00 at 12 link 1:
60 or so rule tables to select a /24 out of that large subnet
protocol ip u32 ht 1:10 match ip src 10.15.16.0/24 hashkey mask 0x000000ff at 12 link 100:
256 rule tables to match the IP
parent 1:0 prio 100 protocol ip u32 ht 100:20: match ip src 10.15.16.32 flowid 1:20 class add dev bond0 parent 1:0 classid 1:1 htb rate 24Gbit ceil 24Gbit burst 4096k cburst 256k class add dev bond0 parent 1:1 classid 1:20 htb rate 102400Kbit burst 4096k cburst 256k qdisc add dev bond0 parent 1:20 pfifo_fast
Now, when I enable these rules, I get at most around 10Gbps total traffic and then ping starts to go up to 100ms or more. However, during that time, the CPU load is almost none -
uptime shows a load of 0.07 and each CPU core is only about 5% utilized.
ping instantly becomes OK once I clear the shaping rules.
Previously I used
sfq instead of
pfifo_fast, and that did result in a large CPU load. I then switched to red, which produced a bit better results with no CPU load, then I switched to
pfifo_fast which produces almost the same results as red.
- What other bottleneck could be there?
- Has anyone tried using Linux traffic shaper for a network that's over 10gbps?