11

On Netfilter, you have the option --set-mark for packets that pass through the mangle table.

The majority of tutorials and examples over the Internet, say that this just adds a mark on the packet, like this, but there's no additional detail of what mark is set and where it resides on the packet:

iptables -A PREROUTING -t mangle -i eth0 -p tcp --dport 80 -j MARK --set-mark 1

My question is:

  • What kind of mark is set and exactly where in the packet this mark resides?

1 Answer 1

24

The mark is a 32 bits integer value attached to a network packet. Some network parts interacting with it (see below) can do bitwise operations on this value, it can then be interpreted between one single 32 bits value up to a collection of 32 flags, or a mix of flags and smaller values, depending on how one chooses to organise its use (tc can't do this). Of course this mark exists only as long as it's handled by the Linux kernel. It's only purely virtual and internal, as it can have no existence on the wire. Depending on where's it's used, it may be called firewall mark, fwmark or simply mark.

Each network packet processed by the kernel, is handled by a structure called sk_buff, defined in linux/include/linux/skbuff.h. This structure includes various meta-data related to the packet when applicable, like IPsec information if any, related conntrack entry once looked up, ... and also its mark.

Various parts of the network stack can interact with or alter this mark, such as:

  • tc,
  • the routing stack can have special rules set with ip rule (eg ip rule add fwmark 1 lookup 42), to alter its routing decisions with this fwmark (eg to use a routing table sending those packets to an other interface than default),
  • of course iptables,
  • its candidate successor nftables,
  • Various (usually tunnel) interfaces types can set a mark or interact with it, such as vti, erspan, gre, gretap, ipip, sit, WireGuard ... or XFRM transformations (ip xfrm state/policy). Depending on types, the mark can be set on the payload or on the envelope (eg: WireGuard) and can sometimes be inherited during decapsulation/encapsulation,
  • socket option SO_MARK,
  • eBPF,

and probably a few other places...

The main goal of this mark is to have all these network parts interact with each other by using it as a kind of message. The Packet flow in Netfilter and General Networking can help see in what order those elements will receive handling of the packet and thus its mark.

There are other related marks beside fwmark:

  • connmark, which isn't stored with a packet's sk_buff, but in a conntrack entry tracking packet flows. Its connmark can of course be used by iptables with its connmark match and CONNMARK target, with an usage example there: Netfilter Connmark To Linux and beyond !. It allows the decision made based on one single packet to be memorized and then applied to all the packets of the same connection.
  • secmark and likewise its associated connsecmark which are intended to interact with Linux Security Modules such as SELinux.
3
  • Thank you for your wonderful explanation and important documentation of netfilter and its limitations this mark exists only as long as it's handled by the Linux kernel First place I found information about this limitation
    – nelaaro
    Feb 25, 2019 at 12:53
  • Is this true for eg GRE tunnels? It looks as though GRE tunnels use key to actually modify the packet, whereas vti tunnele reappropriate key as a fwmark. Which seems to be why xfrm tunnels are the preferred approach nowadays?
    – MattM
    Sep 14, 2023 at 23:33
  • @MattM ip link add type gre help 2>&1 | grep -i mark
    – A.B
    Sep 15, 2023 at 0:39

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .