History of programmatic interfaces to iptables, ipchains, and ipfw

Question

I had to do some fiddling with iptables rules from Go recently, and I noticed both docker's and coreos's wrapper libraries exec() out to the iptables command and screen scrape the standard output. This seemed surprising to me.

In Python-land, there's python-iptables:

Interoperability with iptables is achieved via using the iptables C libraries (libiptc, libxtables, and the iptables extensions), not calling the iptables binary and parsing its output.

I guess a Python library is allowed to load C libraries at runtime, and Go can't [edit: I'm wrong], but couldn't Go statically link to those libraries? Because the Go library would have to get libxtables.a etc. from somewhere? (I'm dpkg -Ling the relevant -dev packages in Debian, and I only see .sos.)

Anyway, according to the Netfilter FAQ:

4.5 Is there an C/C++ API for adding/removing rules?

The answer unfortunately is: No.

Now you might think 'but what about libiptc?'. As has been pointed out numerous times on the mailinglist(s), libiptc was NEVER meant to be used as a public interface. We don't guarantee a stable interface, and it is planned to remove it in the next incarnation of linux packet

We are well aware that there is a fundamental lack for such an API, and we are working on improving that situation. Until then, it is recommended to either use system() or open a pipe into stdin of iptables-restore. The latter will give you a way better performance.filtering. libiptc is way too low-layer to be used reasonably anyway.

Ok, so maybe you're not supposed to treat those C libraries as stable. Then I thought "why not just talk to /proc or whatever directly?" Turns out, iptables mostly doesn't use /proc to talk to the kernel (just to read table names?), and the main interface is actually getsockopt() and setsockopt() on an unbound socket.

This was another surprise! (But maybe it's just because I'm unfamiliar with this kind of code that it seems like a funny way to interface with the kernel.)

I did a bit of archaeology backwards from iptables to ipchains to ipfw and found an ipfw man page from 1997 which made me feel a bit better:

BUGS
       The  setsockopt(2)  interface  is a crock.  This should be
       put under /proc/sys/net/ipv4 and the world would be a bet-
       ter place.

You can find elsewhere that "The ipfw utility first appeared in FreeBSD 2.0" and

HISTORY
      Initially this utility was written for BSDI by:
       Daniel Boulet    <[email protected]>
      The FreeBSD version is written completely by:
       Ugen J.S.Antsilevich <[email protected]>
      while synopsis partially compatible with old one.

Here's the FreeBSD 2.0 version of ipfw from 1994. It still uses setsockopt(), but seems to use kvm_read() instead of getsockopt().

My Questions

• Please correct anything I said wrong above :)
• Why did they choose {get,set}sockopt() in the first place? Is it uncommon, or just new to me?
• Why didn't it get changed at some point? Like to something under /proc as the 1997 man page suggests.
• Why did they never come up with a stable/public C interface to iptables?

My Follow-up Questions

"Not quite, the main interface is netlink’s NETLINK_NETFILTER family"

Thanks for this pointer!

I'm looking at https://git.netfilter.org/iptables/tree and only libipq seems to be using a netlink socket, unless I'm missing something.

$ ick "socket\("
include/libiptc/libiptc.h
158:int iptc_get_raw_socket(void);

include/libiptc/libip6tc.h
152:int ip6tc_get_raw_socket(void);

utils/nfsynproxy.c
129:    fd = socket(AF_INET, SOCK_STREAM, 0);

libiptc/libiptc.c
1312:   sockfd = socket(TC_AF, SOCK_RAW, IPPROTO_RAW);

extensions/libxt_set.h
14: int res, sockfd = socket(AF_INET, SOCK_RAW, IPPROTO_RAW);

libipq/libipq.c
223:                h->fd = socket(PF_NETLINK, SOCK_RAW, NETLINK_FIREWALL);
225:                h->fd = socket(PF_NETLINK, SOCK_RAW, NETLINK_IP6_FW);

libxtables/xtables.c
881:    sockfd = socket(afinfo->family, SOCK_RAW, IPPROTO_RAW);

The libipq stuff looks "normal" to me: make a socket, bind it with a sockaddr, sendto(), recvfrom(), etc.

Also xtables-monitor seems to be using some mnl_socket_* wrappers which do netlink stuff.

But I can't figure out libiptc and xtables -- no mention of netlink, and they don't even bind the socket.

https://git.netfilter.org/iptables/tree/libxtables/xtables.c#n881

https://git.netfilter.org/iptables/tree/libiptc/libiptc.c#n1312

There are other examples that don't (seem to?) use netlink, like this code from mozilla's rr debugger: https://github.com/mozilla/rr/blob/master/src/test/netfilter.c

And netlink is Linux-only, from the year 2000 (2.2), while this {get,set}sockopt() stuff in (at least) ipfw is older, and in BSD-land.

I'm not really sure what my question is anymore :) How do the non-netlink sockets somehow communicate with the correct kernel-side thing without even bind() and a sockaddr?

Answer 1

I guess a Python library is allowed to load C libraries at runtime, and Go can't

Go can use shared C libraries.

Turns out, iptables mostly doesn't use /proc to talk to the kernel (just to read table names?), and the main interface is actually getsockopt() and setsockopt().

Why did they choose {get,set}sockopt() in the first place? Is it uncommon, or just new to me?

(This answer previously stated, incorrectly, that the main interface is netlink’s NETLINK_NETFILTER family.)

You’re correct, iptables uses calls to getsockopt on unbound sockets of the appropriate type to interact with the kernel, which ends up calling into the kernel’s netfilter code. I suspect this is for historical reasons...

Incidentally, getsockopt and setsockopt work on sockets in a variety of states, including unbound, and they have to because they’re used to set options before connecting. The netfilter-via-sockopt connection is effectively a side-channel.

Why didn't it get changed at some point? Like to something under /proc as the 1997 man page suggests.

There are probably a number of reasons, but one is no doubt backward compatibility — in Linux, user-space interfaces are never supposed to break. So while a new interface could be provided, the existing one would remain, which reduces the incentive to improve it or rewrite programs which already work.

As long as the iptables command-line tool could be used with not too much effort, and providing a better interface involved a lot more effort, change would be difficult. A properly-maintained libiptc-style library would have been a possibility too, but there was never much incentive for the iptables developers to do that.

Why did they never come up with a stable/public C interface to iptables?

Because “they” never got round to it. Now iptables is being phased out in favour of nftables, which includes a number of libraries providing access at different levels of abstraction (libnftables, libnftnl, libnfnetlink). This uses netlink, which was designed as a sockets-based interface to transfer information between the kernel and user-space, and is a natural fit for tools such as nftables (and iptables). The netlink documentation lists the other families available; these cover quite a lot of kernel-hosted functionality. netlink.

You might find https://github.com/vishvananda/netlink useful to get started driving iptables from Go. (It needs a fair amount of work for that, but it’s better than starting from scratch.)