8

I have a small, embedded, completely self cross-compiled installation on a mips SoC.

SoC is based on a Mediatek MT7628 sporting a Wifi interface (2.4MHz only), a 100Mbps Ethernet MAC and an internal 5port Ethernet switch (one of the ports is connected to MAC).

I compiled Ethernet device driver (mtk-eth.ko) as installable module.

If I boot without the module (I actually renamed it so udevd doesn't manage to install it "automatically") I don't have an Ethernet device, exactly as expected:

# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
3: wlan0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue qlen 1000
    link/ether b8:d8:12:64:b9:a0 brd ff:ff:ff:ff:ff:ff
    inet 192.168.7.130/24 brd 192.168.7.255 scope global noprefixroute wlan0
       valid_lft forever preferred_lft forever

So far nothing strange. Next I manually install the module:

# modprobe mtk-eth
[ 6591.429607] mtk_soc_eth 10100000.ethernet: generated random MAC address 0a:90:77:c7:1d:f3
[ 6591.461683] mtk_soc_eth 10100000.ethernet eth0: mediatek frame engine at 0xb0100000, irq 5
[ 6591.659810] libphy: dsa slave smi: probed
[ 6591.674778] mt7628-esw 10110000.esw: nonfatal error -122 setting MTU on port 0
[ 6591.693746] mt7628-esw 10110000.esw lan (uninitialized): PHY [dsa-0.0:00] driver [Generic PHY] (irq=POLL)
[ 6591.728356] mt7628-esw 10110000.esw: configuring for fixed/mii link mode
[ 6591.736381] DSA: tree 0 setup
[ 6591.751095] mt7628-esw 10110000.esw: Link is Up - 100Mbps/Full - flow control off
[ 6595.094324] mt7628-esw 10110000.esw: port 0 link changed 0x1
# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
3: wlan0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue qlen 1000
    link/ether b8:d8:12:64:b9:a0 brd ff:ff:ff:ff:ff:ff
    inet 192.168.7.130/24 brd 192.168.7.255 scope global noprefixroute wlan0
       valid_lft forever preferred_lft forever
4: eth0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop qlen 1000
    link/ether 0a:90:77:c7:1d:f3 brd ff:ff:ff:ff:ff:ff
5: lan@eth0: <BROADCAST,MULTICAST,M-DOWN> mtu 1500 qdisc noop qlen 1000
    link/ether 0a:90:77:c7:1d:f3 brd ff:ff:ff:ff:ff:ff

This gives me two devices: the expected eth0, an unexpected lan@eth0 but still no connection. This is right because I have no DHCP client running (WiFi is handled separately by IWD).

Starting the DHCP client gives me connection:

# dhcpcd -b eth0
dhcpcd-9.4.0 starting
sandbox unavailable: seccomp
DUID 00:01:00:01:c7:92:bc:8f:b8:d8:12:64:b9:a2
dhcp_vendor: No such process
sandbox unavailable: seccomp
forked to background, child pid 850
[ 6671.439622] mtk_soc_eth 10100000.ethernet eth0: configuring for fixed/mii link mode
[ 6671.447753] mtk_soc_eth 10100000.ethernet eth0: Link is Up - 100Mbps/Full - flow control rx/tx
# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
3: wlan0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue qlen 1000
    link/ether b8:d8:12:64:b9:a0 brd ff:ff:ff:ff:ff:ff
    inet 192.168.7.130/24 brd 192.168.7.255 scope global noprefixroute wlan0
       valid_lft forever preferred_lft forever
5: lan@if4: <BROADCAST,MULTICAST,M-DOWN> mtu 1500 qdisc noop qlen 1000
    link/ether 0a:90:77:c7:1d:f3 brd ff:ff:ff:ff:ff:ff
6: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast qlen 1000
    link/ether 9a:cf:60:d9:2e:a1 brd ff:ff:ff:ff:ff:ff
    inet 192.168.7.106/24 brd 192.168.7.255 scope global dynamic noprefixroute eth0
       valid_lft 43191sec preferred_lft 37791sec

Notice only "standard" eth0 actually gets an IP address and "auxiliary" interface changed name lan@eth0 -> lan@if4.

What is this second "device"? What is its purpose and how am I supposed to use it (IFF I'm supposed to use it). Notice also bringing down eth0 actually seems to act on "auxiliary" device:

# ip l set eth0 down
[ 8558.176702] mtk_soc_eth 10100000.ethernet eth0: Link is Down
# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
3: wlan0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue qlen 1000
    link/ether b8:d8:12:64:b9:a0 brd ff:ff:ff:ff:ff:ff
    inet 192.168.7.130/24 brd 192.168.7.255 scope global noprefixroute wlan0
       valid_lft forever preferred_lft forever
5: lan@if4: <BROADCAST,MULTICAST,M-DOWN> mtu 1500 qdisc noop qlen 1000
    link/ether 0a:90:77:c7:1d:f3 brd ff:ff:ff:ff:ff:ff
6: eth0: <BROADCAST,MULTICAST> mtu 1500 qdisc pfifo_fast qlen 1000

Attempt to remove the mtk-eth module results in Kernel error:

# rmmod mtk-eth
[ 6634.791361] ------------[ cut here ]------------
[ 6634.796077] WARNING: CPU: 0 PID: 841 at drivers/net/phy/phylink.c:1151 0x80302674
[ 6634.809938] RTNL: assertion failed at drivers/net/phy/phylink.c (1151)
[ 6634.819931] Modules linked in: mtk_eth(-) des_generic libdes cbc ecb algif_skcipher hmac sha512_generic sha1_generic md5 md4 algif_hash af_alg mt7603e mt76 mac80211 sha256_generic libsha256 ehci_platform ohci_platform ohci_hcd ehci_hcd cfg80211 rfkill usbcore libarc4 usb_common
[ 6634.848585] CPU: 0 PID: 841 Comm: rmmod Not tainted 5.8.0 #2
[ 6634.857694] Stack : 80450000 80410e8c 804493fc 864d1d24 00000000 00000000 00000000 00000000
[ 6634.869806]         00000000 00000000 00000000 00000000 00000000 00000001 864d1cd0 80490000
[ 6634.881851]         864d1d68 00000000 00000000 ffff00fe 00000038 8025e724 00000006 876d5e58
[ 6634.890364]         7fca7028 7fca706c 7fca7070 7fca7074 8041e310 80490000 80000000 00000009
[ 6634.905154]         80467cc0 7fc3e284 00420000 004f0950 80495220 32cc152e 08072098 80500000
[ 6634.917055]         ...
[ 6634.919556] Call Trace:
[ 6634.919579] [<8025e724>] 0x8025e724
[ 6634.931103] [<8000abf4>] 0x8000abf4
[ 6634.934661] [<8000abfc>] 0x8000abfc
[ 6634.938196] [<8001d990>] 0x8001d990
[ 6634.955157] [<80302674>] 0x80302674
[ 6634.958716] [<80302674>] 0x80302674
[ 6634.968796] [<80302674>] 0x80302674
[ 6634.975843] [<8001da04>] 0x8001da04
[ 6634.979408] [<80302674>] 0x80302674
[ 6634.989141] [<80158110>] 0x80158110
[ 6634.996066] [<c00b10c4>] 0xc00b10c4
[ 6634.999625] [<c00b12c4>] 0xc00b12c4
[ 6635.009387] [<802c9c5c>] 0x802c9c5c
[ 6635.016419] [<802c991c>] 0x802c991c
[ 6635.019974] [<802c9dc0>] 0x802c9dc0
[ 6635.029705] [<802c86e0>] 0x802c86e0
[ 6635.036620] [<8007849c>] 0x8007849c
[ 6635.040176] [<800bc5a4>] 0x800bc5a4
[ 6635.049933] [<8002ec50>] 0x8002ec50
[ 6635.056920] [<8000f14c>] 0x8000f14c
[ 6635.060492] 
[ 6635.068472] ---[ end trace a60217f8a7ca79c8 ]---
[ 6635.076697] ------------[ cut here ]------------
[ 6635.084835] WARNING: CPU: 0 PID: 841 at drivers/net/phy/phylink.c:1030 0x80301a4c
[ 6635.095828] RTNL: assertion failed at drivers/net/phy/phylink.c (1030)
[ 6635.105831] Modules linked in: mtk_eth(-) des_generic libdes cbc ecb algif_skcipher hmac sha512_generic sha1_generic md5 md4 algif_hash af_alg mt7603e mt76 mac80211 sha256_generic libsha256 ehci_platform ohci_platform ohci_hcd ehci_hcd cfg80211 rfkill usbcore libarc4 usb_common
[ 6635.134359] CPU: 0 PID: 841 Comm: rmmod Tainted: G        W         5.8.0 #2
[ 6635.145331] Stack : 80450000 80410e80 804493fc 864d1d14 00000000 00000000 00000000 00000000
[ 6635.157222]         00000000 00000000 00000000 00000000 00000000 00000001 864d1cc0 80490000
[ 6635.169427]         864d1d58 00000000 00000000 ffff00fe 00000038 8025e724 00000006 876d5e58
[ 6635.181471]         7fca7028 7fca706c 7fca7070 7fca7074 8041e310 80490000 80000000 00000009
[ 6635.190097]         80467cc0 0000007c 00420000 004f0950 80495220 32cc152e 08072098 80500000
[ 6635.204965]         ...
[ 6635.207470] Call Trace:
[ 6635.207493] [<8025e724>] 0x8025e724
[ 6635.221955] [<8000abf4>] 0x8000abf4
[ 6635.225509] [<8000abfc>] 0x8000abfc
[ 6635.229045] [<8001d990>] 0x8001d990
[ 6635.241166] [<80301a4c>] 0x80301a4c
[ 6635.244725] [<80301a4c>] 0x80301a4c
[ 6635.248258] [<80301a4c>] 0x80301a4c
[ 6635.262154] [<8001da04>] 0x8001da04
[ 6635.265718] [<c00af0c0>] 0xc00af0c0
[ 6635.269251] [<80406134>] 0x80406134
[ 6635.282048] [<80301a4c>] 0x80301a4c
[ 6635.285608] [<80302690>] 0x80302690
[ 6635.289142] [<80158110>] 0x80158110
[ 6635.301514] [<c00b1120>] 0xc00b1120
[ 6635.305074] [<c00b12c4>] 0xc00b12c4
[ 6635.308611] [<802c9c5c>] 0x802c9c5c
[ 6635.321116] [<802c991c>] 0x802c991c
[ 6635.324673] [<802c9dc0>] 0x802c9dc0
[ 6635.328209] [<802c86e0>] 0x802c86e0
[ 6635.341418] [<8007849c>] 0x8007849c
[ 6635.344976] [<800bc5a4>] 0x800bc5a4
[ 6635.348511] [<8002ec50>] 0x8002ec50
[ 6635.361159] [<8000f14c>] 0x8000f14c
[ 6635.364729] 
[ 6635.366243] ---[ end trace a60217f8a7ca79c9 ]---
[ 6635.380166] ------------[ cut here ]------------
[ 6635.385575] WARNING: CPU: 0 PID: 841 at lib/refcount.c:28 0xc00b1258
[ 6635.397959] refcount_t: underflow; use-after-free.
[ 6635.406183] Modules linked in: mtk_eth(-) des_generic libdes cbc ecb algif_skcipher hmac sha512_generic sha1_generic md5 md4 algif_hash af_alg mt7603e mt76 mac80211 sha256_generic libsha256 ehci_platform ohci_platform ohci_hcd ehci_hcd cfg80211 rfkill usbcore libarc4 usb_common
[ 6635.434814] CPU: 0 PID: 841 Comm: rmmod Tainted: G        W         5.8.0 #2
[ 6635.445430] Stack : 80450000 80410e80 804493fc 864d1d44 00000000 00000000 00000000 00000000
[ 6635.457327]         00000000 00000000 00000000 00000000 00000000 00000001 864d1cf0 80490000
[ 6635.469233]         864d1d88 00000000 00000000 ffff00fe 00000038 8025e724 00000006 876d5e58
[ 6635.481410]         7fca7028 7fca706c 7fca7070 7fca7074 8041e310 80490000 80000000 00000009
[ 6635.489923]         8043f744 0000007c 00420000 004f0950 80495220 32cc152e 08072098 80500000
[ 6635.504762]         ...
[ 6635.507267] Call Trace:
[ 6635.507290] [<8025e724>] 0x8025e724
[ 6635.521960] [<8000abf4>] 0x8000abf4
[ 6635.525515] [<8000abfc>] 0x8000abfc
[ 6635.529050] [<8001d990>] 0x8001d990
[ 6635.541211] [<c00b1258>] 0xc00b1258
[ 6635.544767] [<c00b1258>] 0xc00b1258
[ 6635.548302] [<c00b1258>] 0xc00b1258
[ 6635.562011] [<8001da04>] 0x8001da04
[ 6635.565574] [<c00af0c0>] 0xc00af0c0
[ 6635.569108] [<80406134>] 0x80406134
[ 6635.579120] [<c00b1258>] 0xc00b1258
[ 6635.588795] [<c00b12c4>] 0xc00b12c4
[ 6635.593036] [<802c9c5c>] 0x802c9c5c
[ 6635.596590] [<802c991c>] 0x802c991c
[ 6635.600126] [<802c9dc0>] 0x802c9dc0
[ 6635.615364] [<802c86e0>] 0x802c86e0
[ 6635.618920] [<8007849c>] 0x8007849c
[ 6635.628652] [<800bc5a4>] 0x800bc5a4
[ 6635.635599] [<8002ec50>] 0x8002ec50
[ 6635.639157] [<8000f14c>] 0x8000f14c
[ 6635.649356] 
[ 6635.650891] ---[ end trace a60217f8a7ca79ca ]---

but system seems functional and even Ethernet works (if I reload the module).

I would like to understand a bit better what's really happening.

Documentation pointers and explanations welcome.

3
  • The MAC of eth0 changes. Interesting.
    – Nils
    Jan 5 at 22:29
  • I really have no idea, but years back when I was using openwrt the network interfaces was a programmable switch, so when you enabled the interface it got separated from being a switch port to an interface. Maybe this is doing something like this.
    – Fredrik
    Mar 24 at 9:53
  • Looks like an implementation of switchdev (there is an embedded switch associated with the Ethernet device). Some links: (an alternate architecture: kernel.org/doc/html/latest/networking/dsa/dsa.html and ...) , kernel.org/doc/html/latest/networking/switchdev.html . Having never laid a hand on such device (there is a "rocker" device that appears to be a test driver usable with QEMU but...) , I can hardly help further.
    – A.B
    Aug 23 at 18:17

1 Answer 1

0

Constructs like foo@eth0 are generally virtual interfaces for 802.11q VLANs. The right-hand-side of the name is the physical interface. The left-hand-side of the name is the VLAN. The kernel picks frames from the physical device with the appropriate VLAN tag, and presents them to the IP layer as if they were a separate interface. You can then configure IP addresses, etc., on the VLAN interface.

For the kind of consumer Internet gateway device you describe, typically all the external RJ-45 Ethernet ports are connected to a single switch ASIC. The processor running Linux is also connected to an "internal" port on the same switch ASIC. (They might all be part of the same SoC, but logically, they're separate components.) VLAN assignments are used to designate physical ports as "WAN" or "LAN" or whatever, and bring them to the Linux kernel separately.

As for why it's working that way, we would need more information on the software that is running. You don't describe the software environment at all (distribution, userland config automation like NetworkManager, etc.). A lot of software kits (e.g. OpenWrt) targeted at consumer gateways automatically set things up on the assumption that you'll be building a NAT between an ISP and your LAN.

The kernel errors are almost certainly an unrelated issue. It sounds like a buggy driver. assertion failed means the code did an internal check and found things were not as expected, so aborted the process/thread. refcount_t: underflow; use-after-free indicates the kernel was keeping a count of references to some resource, and something tried to use the resource after it had been released. (I think this specifically means the kernel discovered the driver code's idea of the reference count was higher than the kernel's, but I'm not 100% sure of that.)

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.