For future readers, I wrote this email to some colleagues of mine who are working with me on configuring LXC and figuring out the bridging.
Well, I spent most of the day today fussing with networking in LXC, and now things are a lot clearer, so I thought I’d pass on what I learned.
First is a couple of definitions so we know what we’re looking at (I didn’t really understand the difference between a bridge like br0 or virbr0 and an interface like eth0, and I definitely didn’t know what those interfaces like
vethILNaLo were for).
A bridge is shown as an interface in ifconfig, but a bridge does nothing but connect 2 OTHER networks, it’s not a network interface its self. You can see and change bridges by using
brctl show, and configure them permanently in /etc/networking/interfaces.
A bridge is best thought of like a switch.
Examples of bridges we see:
Interfaces are actually assigned to a single host, a single interface can’t be assigned to multiple hosts. The host sends and receives network traffic to an interface. The interface it chooses depends on the routing table,
route will show you this.
The reason we need bridging is that eth0 is physically attached to the host OS and cannot also be attached for a container or other virtual machine.
Examples of physical interfaces we see:
Some notes about that second one: vethILNaLo
- This is a physical interface that is created for each LXC container that is started, it’s attached to the container and seen inside the container as eth0
- Each time you define
lxc.network.type=veth in the LXC config file it creates another one of these physical interfaces and attaches it in the container, you can use
lxc.network.name = eth0 to set the name of the interface inside the container (the default of eth0, eth1, etc generally works)
- When this interface is created it’s not connected to anything, it’s like it’s not plugged in at all
Connecting vethILNaLo to a network using a bridge
Remember that a bridge is like a software based switch. If you plug two interfaces into it, they’re connected in the same way that a switch connects physical computers.
/etc/network/interfaces we defined and created br0, we basically defined a switch. In that same file we also connected eth0 to our “switch” br0:
iface br0 inet static
In the LXC config file we connected the containers physical interface to that same swtich:
/etc/network/interfaces is set up when the computer starts (that makes sense, the hosts eth0 interface should always be connected to our “switch”), and LXC takes care of creating the physical port for our LXC container and plugging it into our “switch” named br0 (the difference between a bridge and a switch is very small, so for this purpose it’s ok to think of them as the same, a switch is only a little smarter than a bridge).
What’s lxcbr0 and virbr0?
lxcbr0 and virbr0 are both automatically added by LXC, these are both the same thing (virbr0 is an older version kept for backwards compatability, lxcbr0 is newer, but they do exactly the same thing). These two bridges (aka “switches”) also provide NAT/routing capabilities. So if you connected your virtual machine to these bridges it would be like connecting it to a router.
Since we don’t need another router (all of our LXC containers will have their own IP on the network) I’ve removed both of these un-used bridges from the host OS in the build doc