Connecting two clients on different physical interfaces using smcroute

Question

I am very new to networking. I have two clients connected on two different physical interfaces which has upnp running on them. I want to add both of them to the same multicast group, so that they both will discover each other and i will be able to ping each other. Is that possible? How can I achieve this using smcroute?

This is what I have tried:

I created two bridge interfaces(that is the requirement) and connected them to corresponding physical interfaces.

Added following rules to smcroute.conf

mgroup from br1 group 239.255.255.250
mgroup from br2 group 239.255.255.250
mroute from br1 group 239.255.255.250 to br2
mroute from br2 group 239.255.255.250 to br1

ip -s mroute is showing this

# ip -s mroute
(x.x.x.x, 239.255.255.250) Iif: br2    Oifs: br1
  242 packets, 46509 bytes
(x.x.x.x, 239.255.255.250) Iif: br1     Oifs: br2
  243 packets, 46740 bytes
(x.x.x.x, 239.255.255.250) Iif: unresolved
#

But my clients are not able to discover each other. Am I doing it in a wrong way?

/proc/net/ip_mr_vif shows that there are packets moving in and out of br1 and br2 interfaces.

This is the requirement. I have two physical interfaces which I don't want them to tag to the same bridge due to some organizational restrictions. There will be some clients which will be connected to these interfaces where they have upnp stack running on them. I want them to discover each other.

Solution I am trying here is using arp proxy and smcroute. I am using arp proxy so both the clients will be able to detect the other one. I am using smcroute to tag all the clients connected to these two interface into the multicast group 239.255.255.250 and forward the packets to and fro. Is this the correct approach?

Adding diagram of my setup.

        Device 1                      Router                   Device 2
+-----------------+     +----------------------------+    +-----------------+
|                 |     |                            |    |                 |
|           eth1  |     | br2                    br1 |    |  wlan0          |
|   169.254.10.10 |-----| 169.254.50.1      10.0.0.1 |----| 169.254.168.11  |
| (self assigned) |     |                            |    | (self assigned) |
+-----------------+     +----------------------------+    +-----------------+

Commands used to enable proxy arp:

arp -i br2 -Ds 169.254.168.11 br1 pub
arp -i br1 -Ds 169.254.10.10 br2 pub
ip route add 169.254.168.0/24 dev br1
ip route add 169.254.10.0/24 dev br2

I am able to see packets in ip -s mroute, But devices are not discovered each other:

# ip -s mroute
(169.254.10.10, 239.255.255.250) Iif: br2    Oifs: br1
  3 packets, 549 bytes
(169.254.168.11, 239.255.255.250) Iif: br1    Oifs: br2
  12 packets, 2196 bytes
(169.254.168.11, 239.255.255.250) Iif: unresolved
(169.254.10.10, 239.255.255.250) Iif: unresolved
#

Tcpdump from the router:

# tcpdump -i br2 -vvv port 1900
tcpdump: listening on br2, link-type EN10MB (Ethernet), capture size 262144 bytes
21:29:20.867399 IP (tos 0x0, ttl 4, id 0, offset 0, flags [DF], proto UDP (17), length 183)
    169.254.10.10.50759 > 239.255.255.250.1900: [udp sum ok] UDP, length 155
21:29:21.368865 IP (tos 0x0, ttl 4, id 0, offset 0, flags [DF], proto UDP (17), length 183)
    169.254.10.10.50759 > 239.255.255.250.1900: [udp sum ok] UDP, length 155
21:29:21.869556 IP (tos 0x0, ttl 4, id 0, offset 0, flags [DF], proto UDP (17), length 183)
    169.254.10.10.50759 > 239.255.255.250.1900: [udp sum ok] UDP, length 155
21:29:24.614276 IP (tos 0x50, ttl 3, id 6384, offset 0, flags [DF], proto UDP (17), length 183)
    169.254.168.11.50759 > 239.255.255.250.1900: [udp sum ok] UDP, length 155
21:29:25.114268 IP (tos 0x50, ttl 3, id 6393, offset 0, flags [DF], proto UDP (17), length 183)
    169.254.168.11.50759 > 239.255.255.250.1900: [udp sum ok] UDP, length 155
21:29:25.614997 IP (tos 0x50, ttl 3, id 6680, offset 0, flags [DF], proto UDP (17), length 183)
    169.254.168.11.50759 > 239.255.255.250.1900: [udp sum ok] UDP, length 155
^C
6 packets captured
6 packets received by filter
0 packets dropped by kernel

# tcpdump -i br1 -vvv port 1900
tcpdump: listening on br1, link-type EN10MB (Ethernet), capture size 262144 bytes
21:29:40.869434 IP (tos 0x50, ttl 3, id 0, offset 0, flags [DF], proto UDP (17), length 183)
    169.254.10.10.50759 > 239.255.255.250.1900: [udp sum ok] UDP, length 155
21:29:41.371016 IP (tos 0x50, ttl 3, id 0, offset 0, flags [DF], proto UDP (17), length 183)
    169.254.10.10.50759 > 239.255.255.250.1900: [udp sum ok] UDP, length 155
21:29:41.871953 IP (tos 0x50, ttl 3, id 0, offset 0, flags [DF], proto UDP (17), length 183)
    169.254.10.10.50759 > 239.255.255.250.1900: [udp sum ok] UDP, length 155
21:29:44.616742 IP (tos 0x0, ttl 4, id 17080, offset 0, flags [DF], proto UDP (17), length 183)
    169.254.168.11.50759 > 239.255.255.250.1900: [udp sum ok] UDP, length 155
21:29:45.138486 IP (tos 0x0, ttl 4, id 17334, offset 0, flags [DF], proto UDP (17), length 183)
    169.254.168.11.50759 > 239.255.255.250.1900: [udp sum ok] UDP, length 155
21:29:45.622226 IP (tos 0x0, ttl 4, id 17487, offset 0, flags [DF], proto UDP (17), length 183)
    169.254.168.11.50759 > 239.255.255.250.1900: [udp sum ok] UDP, length 155
^C
6 packets captured
6 packets received by filter
0 packets dropped by kernel
#

Command Outputs:

# route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         x.x.x.x         0.0.0.0         UG    0      0        0 erouter0
10.0.0.0        0.0.0.0         255.255.255.0   U     0      0        0 br1
169.254.0.0     0.0.0.0         255.255.0.0     U     0      0        0 br2
169.254.168.0   0.0.0.0         255.255.255.0   U     0      0        0 br1
239.255.255.250 0.0.0.0         255.255.255.255 UH    0      0        0 br1
#

# ifconfig br2
br2       Link encap:Ethernet  HWaddr xxxxxxxx
          inet addr:169.254.50.1  Bcast:169.254.255.255  Mask:255.255.0.0
          inet6 addr: fe80::d02d:5dff:fe68:8e60/64 Scope:Link
          UP BROADCAST RUNNING ALLMULTI MULTICAST  MTU:1500  Metric:1
          RX packets:24012 errors:0 dropped:0 overruns:0 frame:0
          TX packets:23477 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:4779091 (4.5 MiB)  TX bytes:5154708 (4.9 MiB)
#

# ifconfig br1
br1       Link encap:Ethernet  HWaddr xxxxxxxx
          inet addr:10.0.0.1  Bcast:10.0.0.255  Mask:255.255.255.0
          inet6 addr: xxxxxxxxxxxxxxxxxxxxxxxxxx/64 Scope:Global
          inet6 addr: fe80::16b7:f8ff:fefe:faf6/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:44444 errors:0 dropped:0 overruns:0 frame:0
          TX packets:55860 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:6780562 (6.4 MiB)  TX bytes:11041592 (10.5 MiB)
#

Answer 1

Networking ABC:

The basic unit of a LAN network is the segment. The original 10Base5 Ethernet used a long (usually yellow) coax cable and vampire taps, so a segment would look like this:

... ----------------------------- ...
        |         |         |
     Client 1  Client 2  Client 3

By construction, every device on one segment sees all ethernet packages for all devices on this segment, and just filteres out those it is interested in. That allows broadcast (to all devices on a segment) and also multicast (to some of the devices on a segment). For these reason, a LAN segment is also called a broadcast domain. A segment will be assigned an IP address range determined by the netmask. So in the above example, the segment could have 192.168.1.0/24 (i.e., 24 bits for the "segment name", and 8 bits at the end for each client), and Client 1 could be 192.168.1.1, Client 192.168.1.2 etc.

When Ethernet changed to point-to-point connection instead of vampire taps, a switch was used to form a segment:

      +-----------------------+
      |         Switch        |
      +-----------------------+
        |         |         |
     Client 1  Client 2  Client 3

So conceptually, a switch just sends out a packet it receives on one port to all other ports. (In reality, there are optimazations).

So the simplest way to connect your two clients so they can send multicasts to each other is to put them in the same segment with a switch, like in the picture above.

A Linux computer can act as a switch if it has several ethernet ports, and you put those in a bridge:

    +-------------- Linux PC---------------+
    |                                      |
    |              192.168.1.4             |
    |  <------------  br0 ------------>    |
    |    eth0         eth1         eth2    |
    |     |            |            |      |
    +--------------------------------------+
          |            |            |
      Client 1     Client 2     Client 3
     192.168.1.1  192.168.1.2  192.168.1.3

The bridge is called bridge, because originally such a construction was used to bridge two LAN segments together, but here it is used like a switch. The bridge can have an optional "internal" interface, so conceptually this is the same as

      +------------------------------------------------+
      |                     Switch                     |
      +------------------------------------------------+
          |            |            |           |
      Client 1     Client 2     Client 3     Linux PC
     192.168.1.1  192.168.1.2  192.168.1.3  192.168.1.4

So to allow your two clients to see each other on a Linux PC, you use one bridge (I have no idea why you use two, or why this should be a requirement).

If you clients have to be in two different segments (for organizational purposes, which you didn't tell us about), then instead of bridging (OSI level 2), you have to route (OSI level 3).

Writing a 1 into /proc/sys/net/ipv4/ip_forward (or setting a configuration file of your distro to the equivalent at boot) enables routing, ip route allows you to inspect and add routes if the automatically added routes are not sufficient.

This is for unicast traffic, for multicast traffic you then indeed need something like smcroute.

An arp proxy is a cludge for very special situations where for some reason you can't bridge (e.g. WLAN in 3-address mode and ethernet), but want to sort-of bridge. This pretends to each segment that the devices on the other segement are really in the same segment by spoofing arp messages, and the routes unicast traffic. It doesn't work for broadcast (DHCP), and it doesn't work for multicast.

If you are using an arp proxy without having special circumstances you didn't tell us about, you are likely doing it wrong. If you are using an arp proxy on a bridge you are almost certainly doing it wrong (you could just bridge everything instead), unless you have some really crazy circumstances you didn't tell us about.

So:

1. Decide whether you can bridge your two clients (or just use a switch), or not. If you can't bridge, update the question and explain why.

2. If you can't bridge, enable IP forwarding, check the routes and test if ping works. Then you can set up smcroute. Test it with ssmping or asmping, not with ping. Once ssmping/asmping works, try out if SSDP gets routed despite having local scope by definition (I didn't try). If not, there may be more fiddling.

3. If you really have to use an arp proxy, please edit question and explain the circumstances, completely and throughly, with all details.

Edit

So assume the following setup on Server:

• eth1 with 10.0.0.1/24
• eth2 with 10.0.1.1/24
• Client 1 behind eth1 with address 10.0.0.2
• Client 2 behind eth2 with address 10.0.1.2

Kill the arp proxy.

Enable forwarding:

echo "1" | sudo tee /proc/sys/net/ipv4/ip_forward

Check routes with ip route, you should see the routes to eth1 and eth2.

On Client 1, do ping 10.0.0.1, then ping 10.0.1.1, then ping 10.0.1.2. Check that all 3 work. Same for Client 2, with the appropriate addresses.

If it doesn't work, use tcpdump -ni eth1 in one window, tcpdump -ni eth2 in another window, do the pings, and have a look at what goes wrong.

If it works, install ssmping on both clients. Start sudo smcrouted -n on server in new window, so you can see the messages. Let's use multicast group 226.1.1.234 for testing. Do sudo smcroutectl add eth1 226.1.1.2 eth2. Run ssmpingd on Client 1, then asmping -4 10.0.0.2 226.1.1.234 on Client 2. Similarly, ssmpingd on Client 2, then asmping -4 10.0.1.2 226.1.1.234 on Client 1. If it doesn't work, debug as above.

Finally try sudo smcroutectl add eth1 239.255.255.250 eth2 and test if UPNP discovery works. I just tested this with minidlna and gupnp-universal-cp with two additional network namespaces, and it works fine here.

Once it works, set up configuration files as required.

Edit

I looked up RFC3927, and clearly says

if the destination address is in the 169.254/16 prefix [...], then the sender MUST ARP for the destination address and then send its packet directly to the destination on the same physical link.

The host MUST NOT send a packet with an IPv4 Link-Local destination address to any router for forwarding.

and AFAIK this is enforced in the Linux kernel, completely with the extra ARP rules (must be the same interface) So it's not possible to route the 169.254.*.* addresses. If we'd try to route them, we'd have to fight the Linux kernel all steps along the way. Maybe it's possible, but I don't even want to try that. And that's just for unicast, not even for multicast.

Devices with 169.254.*.*, UPNP, Apple Bonjour etc. are meant to be used on the same LAN segment. That also applies to all devices that want to use them. That's how it is.

You have the following choices:

• Touch the devices and configure a static IP, or convince them to accept DHCP. All my media devices can do that. You didn't say which devices you have, so I can't help you there.

• Put one or both media devices behind some sort of tunnel, VLAN, extra SSID or whatever, so you don't have to bridge everything, but can keep all devices that need to see the media devices in one LAN segment. You didn't explain your network setup in spite of me asking multiple times, so I can't help you finding a solution for that.

• Implement your own network stack (in user space like arp-proxy, or however) that ignores the RFCs and networking practices and does what you want. You not only have to pass along ARP, you also have to pass along broadcasts and multicasts, so it's already most part of a self-implemented NIH bridge.

• Actually bridge everything, but use ebtables etc. to enforce separation according to your administrative concerns. You didn't explain your administrative concerns despite me asking multiple times, so I can't help you with that.

If none of these options work for you, then what you want is not possible. Period.

Answer 2

Issue was with rp_filter. rp_filter was turned ON. So that kernel was filtering out all the multicast packets from reaching user space. Hence smcroute was not able to route the multicast packets.

echo 0 > /proc/sys/net/ipv4/conf/br1/rp_filter
echo 0 > /proc/sys/net/ipv4/conf/br2/rp_filter

Turned rp_filter OFF as above and it started working.