I am trying to build a little project using Linux network namespaces, but am a bit overwhelmed by all the linux networking features and containerization-tech available, and thus unsure if i'm approaching this problem the right way.
THE PROBLEM / PROJECT
I currently have a simple Linux box with a single network device (
eth0) which has a dozen static public IPv4 addresses assigned to it (
184.108.40.206). I wish to create a situation where i have a bunch of (network) namespaces, each of those effectively having its own single exclusive public IPv4 addr/interface.
My goal is to spin up multiple shells, each isolated to their own network namespace (and also pid,ipc,... namespaces). So for example, the 7th shell would use network namespace
ns7 which has a single (virtual) ethernet interface which has static ip
220.127.116.11. Within that shell i could (for example) start apache/nginx, let it listen on *:80 which would then serve websites to the public on
18.104.22.168:80. Any traffic directed at port 80 on any of the other IPv4 addresses would never reach
ns7, and likewise any traffic directed at
22.214.171.124 should only reach processes in namespace
The basic idea is that the namespaces are effectively permanent. The namespaces themselves, and the related virtual network devices, will be (re)created and spun up when the system boots.
POTENTIAL SOLUTION (am i on the right track ?)
From what i have been able to piece together without hands-on experimentation, the solution should be something like outlined below, am i on the right track here ?
Create and bring up a (virtual) L2 Bridge device
br0. Making sure we operate at L2 (and not L3).
Change the current ethernet device
eth0configuration, so that it still comes up on boot but doesn't set any IP address anymore (neither static nor DHCP), also assign
Create some network namespaces
ip netns add ns2,
ip netns add ns3, ...,
ip netns add ns12. I will treat the default/root namespace as if it is
Create multiple virtual ethernet interface pairs
net4b, ... For each pair, hook up the
a-version to the
br0Bridge, and assign the
b-version to it's respective namespace.
Within each namespace, assign the local veth device (like
ns2) the appropriate IPv4 address info and bring the device up.
I perhaps may have to enable IPv4 forwarding (?) (
/proc/sys/net/ipv4/ip_forward) and/or enable ARP filtering (?) (
Each namespace would have it's own firewall configuration (as i understand it), so within each namespace run some iptables/nftables shell script that sets some sane defaults and adjust them to local needs.
Does this sound like a (roughly) workable plan, to those more familiar with these virtual linux networking devices ?
EXTRA DETAILS (in case they matter)
OS Info: I run CentOS 8, which notably includes
SELinux, only i use
nftablesfor manual firewall configuration (instead of firewalld stuff).
Background: All the addresses provided (like
126.96.36.199) and interface names (like
eth0) and such were all hopefully obviously fictional, to some degree for privacy reasons, but more so for brevity/simplicity sake, the same goes for my stated intention of just "running a shell in each namespace".
Actual Requirements: There is a bunch of different kinds of software that will be running in these namespaced environments, each namespace will have a unique job and often involve multiple services; One of my main wishes is to completely isolate the IPv4 addresses from each other; Also, many of the target programs are servers/daemons (like my apache httpd example), i want them to be able to bind to the actual public-facing interface/port, instead of say binding to a port on a private ipv4 or unixsocket, and then having software in the root-namespace act as reverse proxy middleware;
Why not just docker: (TL/DR) It is an option, but i want to custom tailor something together for fun (Longer Rant) Almost all of these namespaced environments are for personal use, one will run my private mailserver, one webserver for hosting some personal low-traffic sites, one webserver for some live webdev work, a couple running their own sshd+webstack to act as a free mini-VPS for some hobbyist friends, some running some largely automated processes, that kind of stuff. I know there is a huge overlap between what i'm trying to do and what common containerization stacks like docker offer, infact most of the things i've described the system already does, much of it using Podman (near identical to docker) and much of the rest just running side by side in the common root namespace. For various reasons i like to cleanly micromanage some of it, which would be a lot easier if i can implement the separation of stuff into multiple permanent namespaces, i have found that the more i try to tune things to my wishes, the more the containersoftware gets in my way. And since the only things containersoftware provides, that i actually make use of, are actually linux kernel features, i felt it's a worthy project to ditch that wrapper and figure out how to accomplish these things without it. And i also enjoy the education value that comes with diving deeper in this stuff, as it's not something my day job usually touches on.
IPC: I have no significant reasons for the (processes inside the individual) namespaces to have to communicate over IP with the other namespaces (nor with the default namespace). Though, if i change my mind on that (and assuming my idea so far was mostly correct), i imagine i could just repeat part of the proces by setting up an additional L2 bridge with additional veth-pairs for each namespace, and assign those private
VPS / Cloud: The machine in question is not a physical machine but a VPS / virtual server / cloud server. The only/single network device that's currently present on the machine. This
eth0, which in reality is called
ens3, automatically functioned right away, even before completing the CentOS installation, and is thus already a virtual device itself.
virtio_netare loaded. I think my hosting provider provides this VPS using Qemu. I'm not sure if any of this matters or not, i imagine it shouldn't. Though i did spend some time looking into if it was possible to just clone (or create more) interfaces like the current
ens3and then simply assigning them all a single IPv4 and namespace directly, eliminating the need for a bridge device and veth-pair. Nothing really came out of that search, i kind of just ended up assuming this wouldn't be possible without help from the hosting providers personnel changing settings at the hypervisor level. And while they are generally helpful, and i imagine potentially open to such actions, it would make my solution less flexible to future changes, so if possible i'd prefer to handle this on the device that i can fully control.
IPv6: I have avoided mentioning IPv6 for brevity, i do have a block of IPv6 addresses too and plan to use those in a similar fashion. But i think it's likely best to get IPv4-only working correctly first, then enable IPv6 and go from there, i can't imagine it'll be too different.
My implementation: I'm not entirely sure yet on how to implement it all, once i do actually manage to get things working for the first time. I imagine that for the network part of the project, i'll create a
network-setup.shshell script that tests for the presence of all the network-related neccecities (like the namespaces, bridge device, veth devices, ...) and then recreates or sets up whatever is missing. Then accompany it with a systemd unit file that runs that shell script, and flag that unit file as required by
network-online.target. And then another shell script and unit file that run later in the (boot)process, that uses
unshareor the systemd-version of
initto actually start the relevant processes. But if anyone has a better idea i'd love to hear it.
Different namespace implementations: One potential complication i can smell coming is that i am under the impression that what util-linux (
man unshare) calls a network namespace is not the same as what iproute2 (
man ip-netns) means by that same term. I'm however still not sure if the former is a superset/extension over the latter, or if it is an entirely different incompatible implementation. Infact this seems to be a recurring problem when reading up on container related tech.