2

I am trying to understand the internal mechanism of how Open vSwitch leverages the linux infrastructure, specifically difference between OVS bridges and br-xx (linux bridge). I am new to this field, so I could be wrong on many levels.

Background: So far I know that to create a virtual network, Network namespaces can be used to emulate hosts (end-points of the network) and veth pairs can be used as links to connect two nodes. Also, using the iproute2 package, one can set up bridges (br-xx) and add virtual interfaces to it which completes the virtual network.

To understand the difference between OVS Bridges and linux bridges (created by iproute2), I went through mininet code. One thing that I could make out is that mininet relies on ovs-vsctl to create OVS bridges. I explored OVS code to figure out how are they actually being setup and how are they different from linux bridges without much success.

Question: How are OVS bridges (created using ovs-vsctl) and linux bridges (created using iproute2) different internally? I intuitively feel that they both based on two different kernel modules in the back (In the context of Open vSwitch is it called datapath, not sure about the name of kernel module in case of linux bridge). Is my intuition right? If it is, how can I create my own kernel module and set up a bridge using it as the "backend"?

2

Share some understandings here, this may not as low-level as you would like, since I havn't devoted too much effort to this level/aspects.

First, Linux bridge relies on kernel stack for simple L2 forwarding. Put it in other words: forwarding packets according to src_mac and in_port mapping rules, which is stored in system ARP cache.

List the forwarding rules (mapping) with iproute2:

$ ip neigh show
192.168.157.2 dev ens33 lladdr f0:50:54:fd:b2:34 STALE
192.168.157.254 dev ens33 lladdr f0:50:54:fd:b2:34 STALE

With old fashioned (but pretty printing) arp command:

$ arp -n
Address                  HWtype  HWaddress           Flags Mask            Iface
192.168.157.2            ether   f0:50:54:fd:b2:34   C                     ens33
192.168.157.254          ether   f0:50:54:fd:b2:34   C                     ens33

And more, all regular network tools works well on the devices (veth pair, tun/tap) residing on a Linux bridge.

In contrast, OVS maintains its own forwarding table and forwarding rule, called flow table and flow, respectively. Once a packet enters an OVS bridge, it will match against the flows (rules), then executing the actions specified in the rules. This forwarding mechanism is more flexible, extensible and most importantly - programmable.

Take the second rule in following flow table as example:

$ ovs-ofctl dump-flows br-int
NXST_FLOW reply (xid=0x4):
 cookie=0x9661, duration=8986958.206s, table=0, n_packets=2285, n_bytes=82852, idle_age=0, hard_age=65534, priority=1 actions=NORMAL
 cookie=0x9661, duration=2944224.063s, table=0, n_packets=148, n_bytes=32018, idle_age=0, hard_age=65534, priority=3,in_port=1,dl_vlan=18 actions=mod_vlan_vid:43,NORMAL
 cookie=0x9661, duration=8986823.648s, table=0, n_packets=9151, n_bytes=17148, idle_age=0, hard_age=65534, priority=3,in_port=1,dl_vlan=21 actions=mod_vlan_vid:7,NORMAL
  • matching criteria: ingress packets (to this OVS bridge) from port 1 (in_port=1) and with vlan tag 18 (dl_vlan=18)
  • action: modify packet's vlan tag to 43 (mod_vlan_vid=43), then do normal L2 forwarding (NORMAL) within this OVS bridge (br-int)

corresponding src_mac:in_port mapping stored also in its own fdb (forwarding database):

$ ovs-appctl fdb/show br-int
 port  VLAN  MAC                Age
    1     5  fb:26:3f:e8:1e:1c  298
    1     8  fb:26:3f:b7:26:55  297

Specifically, if no other rules are specified in the flow table, an OVS bridges will following the default rule (action=NORMAL), which performs normal L2 forwarding, just like a normal Linux bridge.

Secondly, as OVS lifts the packets up from kernel stack, So network tools relying on kernel stack may cease to work on OVS devices (ports), e.g. tcpdump cann't capture any packets on an OVS patch port, and iptables rules won't work on OVS ports either.

Thirdly, regarding the implementation: Linux bridge is very straight forwarding, probably one of the earlist network devices in kernel stack, and only hundreds lines of code, check code here in Kernel 5.1. On the contrary, OVS code is much more complex as it handles so much things to work as a programmable, powerful, highly efficient and full-featured OpenFlow switch. Check out an early version (less features) if you want to have a quick glimpse.

| improve this answer | |
  • That pretty much answered my question. The only part that is still unclear to me is how I would go about creating my own software switch (like making my kernel module as the "backend" of my custom bridge (I was searching for a more mechanical answer). Maybe I will get to that after reading the source code that you have linked. – sbhTWR Jul 9 '19 at 4:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.