Setting traffic class on return packets

Question

I have a network topology:

Server <-> router1 <-> router2 <-> router3 <-> edgeRouter <-> "internet"

All routers are linux based, and support iptables.

The server sets traffic classes with iptables (--set class X:Y), and routers do some "routing" based on the class that is set. (Class depends on the originating application).

The edge routers forwards the packets via our ISP to the internet, and recieves the return (reply) packets. The recieves replies ofcourse have no traffic class set.

Is it possible to use an iptables rule on the edge router (mangle, or something simmilar), to track the return packets (NAT-style, packets from "ESTABLISHED" connections) and to mark the returning packets with the same traffic class as the originating packet? Enabling NAT on the edge router is not a problem.

TLDR: How to use iptables to classify ingress packets with the same class as egress for the same connection.

Answer 1

TLDR: How to use iptables to classify ingress packets with the same class as egress for the same connection.

It's not entirely clear by your question what method of classification you're referring to, but in general if we're talking about traffic shaping using tc and queuing disciplines, the following applies.

act_connmark

As ingress qdisc processing is done before netfilter, you cannot directly classify ingress traffic using iptables (without recompiling your kernel with IMQ, see below). You can however indirectly classify it by using connection tracking. If available on your kernel you can use the act_connmark module, designed for this exact purpose, which adds a connmark action to tc filters that support it.

# 0. Load modules and IFB device
modprobe act_connmark
modprobe ifb
ip link set ifb0 up

# 1. Classify packets by marking them
iptables -t mangle -A POSTROUTING -p tcp --sport 22 -j MARK --set-mark 1

# 2. Append rule to save the packet mark to the connection mark
iptables -t mangle -A POSTROUTING -j CONNMARK --save-mark

# 3. Restore the connection mark to the packet mark with 'action connmark'
#    before redirecting to the ifb-device
tc qdisc add dev eth0 handle ffff: ingress
tc qdisc add dev ifb0 handle 1: root
tc filter add dev eth0 parent ffff: prio 1 \
   protocol ip u32 match u32 0 0 flowid ffff:1 \
   action connmark \
   action mirred egress redirect dev ifb0

# 4. Apply filters to classify packets based on their mark
# ... setup qdiscs and classes as usual on ifb0... then
tc filter add dev ifb0 parent 1: prio 1 protocol ip handle 1 fw classid 1:01

• Example at the ArchLinux Wiki

IMQ

IMQ (Intermediate Queueing Device) circumvents the normal flow of traffic in the kernel by, as I understand it, looping it back through a virtual device after netfilter processing. It is not merged with the kernel tree, thus is not included in most distributions, and requires you the patch and compile the kernel yourself. If you do so, it would work something like this:

# classify and save mark in POSTROUTING as before... then
iptables -t mangle -A PREROUTING -j CONNMARK --restore-mark
iptables -t mangle -A PREROUTING -j IMQ --todev 0

# ... setup qdiscs and classes as usual on imq0 ... then
tc filter add dev imq0 parent 1: prio 1 protocol ip handle 1 fw classid 1:01

This would also enable you to do more advanced classifications on ingress using iptables, that might be very cumbersome using plain u32 filters, such as arbitrary port ranges. I cannot speak to the performance or elegance of this solution though, I'm guessing there's a reason it never got merged.

• IMQ at GitHub

Answer 2

Since your egress packets have the class set based on application (and I'm guessing that each application uses a certain set of TCP/UDP ports) you could re-classify incoming packets based on those ports.

eg. to re-classify an established (outbound) HTTP session, on edgeRouter:

iptables -t mangle -A INPUT -i [WANIF] -m state --state ESTABLISHED,RELATED -p tcp -m tcp --sport 80 -j DSCP --set-dscp-class cs3

NB: May need to use the FORWARD table instead of INPUT...

But - To track egress packets, determine what class they had on the way out then apply that same class to ingress packets of the same stream - still possible, but a mountain of work and possibly a custom net-filter module which interfaces with conntrack.

Answer 3

What is it that you set with --set class X:Y? Class of what exactly? I've searched the man page of iptables but have not found similar to what you describe. I think that you may want to do something like this:

1. Mark the TOS field of IP packets in "Server".
2. Let the routers treat the packet specially.

3. In "edgeRouter" do the following:

     # If a packet arrives from LAN, is marked and we know nothing about the
     #+ connection, then mark the connection
   iptables -t mangle -A PREROUTING -i $LAN_IF -m tos --tos $TOS_VAL -m \
     connmark \! --mark $TOS_VAL -j CONNMARK --set-xmark $TOS_VAL
   
     # Reset the TOS value when going out to prevent strange interpretation
   iptables -t mangle -A POSTROUTING -o $WAN_IF -m tos --tos $TOS_VAL \
     -j TOS --set-tos 0x00
   
     # If a packet arrives from WAN and the connection is marked, then mark
     #+ the packet so that the routers in LAN know how to deal with it
   iptables -t mangle -A PREROUTING -i $WAN_IF -m connmark --mark $TOS_VAL \
     -j TOS --set-tos $TOS_VAL