2

I have a following very simple topology where router r1 sends ICMP "echo request" messages to r2 with 2000 byte payloads and router r2 replies to those messages with ICMP "echo reply" messages:

passive network tap setup

Routers Gi0/0/0 interfaces are with 9000 byte MTU. As seen on the drawing, there is also a passive network tap between those two routers which mirrors traffic to bond0 interface in PC. When bond0(and thus eth2 and eth3) has smaller MTU than Ethernet frames containing ICMP messages on wire, then packet capture utilities like tcpdump or tshark see only part of the payload part of the ICMP messages. For example lets say that r1 sends ICMP messages padded with 0xabcd data, then packet capture utilities listening either on bond0, eth2 or eth3 interfaces in PC see following data:

0x0000:  abcd abcd abcd abcd abcd abcd abcd abcd  ................
0x0010:  abcd abcd abcd abcd abcd abcd abcd abcd  ................
0x0020:  abcd abcd abcd abcd abcd abcd abcd abcd  ................
0x0030:  abcd abcd abcd abcd abcd abcd abcd abcd  ................
0x0040:  abcd abcd abcd abcd abcd abcd abcd abcd  ................
/* further data removed for brevity */

What causes such behavior?

Improve this question
4
  • 1
    I have no idea, but you should specify at least which device driver is used.
    – sourcejedi
    Commented Mar 7, 2016 at 9:17
  • 1
    Also I assume PC has a normal connection too. (eth0 and eth1?). Presumably with the same device driver. Are you saying you've tested pinging PC directly, and it doesn't have the same problem?
    – sourcejedi
    Commented Mar 7, 2016 at 9:20
  • All I know is the tap can't literally be sending you packets like that, because they don't even have the ethernet headers that specify the packet length :).
    – sourcejedi
    Commented Mar 7, 2016 at 9:25
  • 1
    Driver in use for both eth2 and eth3 is cdc_ether(USB CDC Ethernet devices) which depends on usbnet, i.e those eth2 and eth3 interfaces are USB NICs. If I bond eth0 and eth1 interfaces(both use e1000e driver), then this behavior does not occur- frames which are larger than interface MTU are simply dropped. So I guess this behavior is specific to USB NICs.
    – Martin
    Commented Mar 9, 2016 at 17:32

1 Answer 1

Reset to default
1
+150

Indeed, usbnet seems the most likely culprit.

     /* rx and tx sides can use different message sizes;
      * bind() should set rx_urb_size in that case.
      */
     dev->hard_mtu = net->mtu + net->hard_header_len;

     net->netdev_ops = &usbnet_netdev_ops;
     net->watchdog_timeo = TX_TIMEOUT_JIFFIES;
     net->ethtool_ops = &usbnet_ethtool_ops;

     // allow device-specific bind/init procedures
     // NOTE net->name still not usable ...
     if (info->bind) {
             status = info->bind (dev, udev);
             ...
     }
     ....
     if (!dev->rx_urb_size)
             dev->rx_urb_size = dev->hard_mtu;

http://lxr.free-electrons.com/source/drivers/net/usb/usbnet.c?v=4.4#L1661

cdc_ether doesn't set rx_urb_size itself, as far as I can tell.

I'm assuming the URB size gets passed to the USB controller and the ethernet frames get their heads chopped off to fit (don't ask me why not the other end). I mean, hopefully your experiment isn't causing the hardware to DMA outside of the allocated buffers :).

Even cdc_ether did set rx_urb_size, there's this weird bit in the ndo_change_mtu callback in usbnet. It looks like it could end up being pegged back to mtu, if mtu was ever set to the maximum.

Honestly I don't know how this works, it just looks odd at a first glance.

     if (dev->rx_urb_size == old_hard_mtu) {
             dev->rx_urb_size = dev->hard_mtu;

http://lxr.free-electrons.com/source/drivers/net/usb/usbnet.c?v=4.4#L393

Improve this answer

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .