1

I have a Server with multiple (four) network interfaces running Arch Linux when I encountered something strange that I cannot explain.

I have two interfaces configured to the the ip addresses 192.168.100.11/24 and 192.168.100.12/24 (using two netctl profiles).

Both interfaces are connected to a switch which also connects to my computer.

When I enable the profiles both interface seem to work quite okay.

But when I'm pinging the address of the main interface (192.168.100.11) and removing its cable (while having the secondary interface up and running) I continue to getting replies from 192.168.100.11 even though no cable is attached to the interface. I even can successfully ssh 192.168.100.11 into the machine.

The output of ip addr shows that the interface has no carrier and is down but still has it's IP address:

2: enp7s0f0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN group default qlen 1000
link/ether 00:1e:67:a3:7f:b6 brd ff:ff:ff:ff:ff:ff
inet 192.168.100.11/24 brd 192.168.100.255 scope global enp7s0f0
   valid_lft forever preferred_lft forever
inet6 fd00::21e:67ff:fea3:7fb6/64 scope global mngtmpaddr dynamic 
   valid_lft 6836sec preferred_lft 3236sec
inet6 fe80::21e:67ff:fea3:7fb6/64 scope link 
   valid_lft forever preferred_lft forever
3: enp7s0f1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
link/ether 00:1e:67:a3:7f:b7 brd ff:ff:ff:ff:ff:ff
inet 192.168.100.12/24 brd 192.168.100.255 scope global enp7s0f1
   valid_lft forever preferred_lft forever
inet6 fd00::21e:67ff:fea3:7fb7/64 scope global mngtmpaddr dynamic 
   valid_lft 6836sec preferred_lft 3236sec
inet6 fe80::21e:67ff:fea3:7fb7/64 scope link 
   valid_lft forever preferred_lft forever

When I do netctl status main to check for the status the profile is still active:

# netctl status main
● netctl@main.service - Main interface
   Loaded: loaded (/etc/systemd/system/netctl@main.service; enabled; vendor preset: disabled)
   Active: active (exited) since Thu 2015-01-08 11:29:07 UTC; 25min ago
     Docs: man:netctl.profile(5)
 Main PID: 55293 (code=exited, status=0/SUCCESS)

Jan 08 11:29:03 timingserver1 network[55293]: Starting network profile 'main'...
Jan 08 11:29:07 timingserver1 network[55293]: Started network profile 'main'

How is this even possible?

It is important to me to understand the mechanics behind this. The plan is that I have two redundant servers offering a service on the ip 192.168.100.11. One of the servers will have the main interface down and both are connected to the same switch. The second interface on both servers is used for maintanance when the server is standby or degraded (of course they will not have the same IP).

So in case of a failover I will set the interface of the main-server down (or disconnect the cable) and activate the one on the backup-server. Of course, if the main-server still answers to 192.168.100.11 even if it is down, this would be bad... :-/

My netctl profile files:

  • main interface

    Description='Main interface'
    Interface=enp7s0f0
    Connection=ethernet
    IP=static
    Address=('192.168.100.11/24')
    Gateway='192.168.100.1'
    DNS=('192.168.100.1' '8.8.8.8')
    
  • secondary interface

    Description='Secondary interface'
    Interface=enp7s0f1
    Connection=ethernet
    IP=static
    Address=('192.168.100.12/24')
    DNS=('192.168.100.2' '8.8.8.8')
    

I would appreciate it very much if anybody can direct me as how to archive my desired solution. Maybe it's just something stupid I've overlooked or don't know about but at the moment this is bothering me quite a bit... :-/

  • With the cable disconnected, but the IP 192.168.100.11 still working, run arp -n and check the mac address for 192.168.100.11. – Christopher Neylan Jan 8 '15 at 14:50
  • @ChristopherNeylan: interrestingly, as long as the second interface is up, both ip addresses have the same mac (the one of the second interface) in my computers arp cache. when i disconnect the second interface as long as its up, both ip addresses become unavailable. when i shut the second interface down (netctl stop second) the first interface answers the pings for 192.168.100.11 again and 192.168.100.12 becomes unavailable. – deepthought-64 Jan 8 '15 at 18:38
1

Okay, so the problem seems to be, that any interface answers the ARP-requests because they are all in the same subnet and therefore even if the network interfaces have different ip addresses the ARP entries point to the same interface. So all the packets for the different IPs are sent to the same MAC address (because all the IPs have the same MAC in the ARP entry)

My arp cache looks like this:

$ ip neigh
192.168.100.11 dev enp2s0 lladdr 00:1e:67:a3:7f:b7 STALE
192.168.100.12 dev enp2s0 lladdr 00:1e:67:a3:7f:b7 STALE

Thanks @ChristopherNeylan for giving me the clue to look at ARP.

It seems that it is not easily possible to have more than one nic on the same subnet and archive the behavior I'd like.

The problem is that linux uses a weak host model but a strong host model would be needed here. I've found a question on serverfault that addresses this issue should anybody else be interrested: multiple physical interfaces with IPs on the same subnet

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.