2

Here's the story: I have a server(x3650M5) with several NIC's (eno1,2,3,4 and others). OS is CentOS7. Eno1 is off, eno2 is connected to local network (10.10.10.11/24). I'd like to connect the server to QNAP ts453 NAS via eno3 and eno4. QNAP NAS also has 4 NICs: eth0,1,2,3. Eth0 faces local network (10.10.10.12/24) and works. Server and NAS are connected directly eno3<->eth2 and eno4<->eth3.

If I use single connection (eg eno3 to eth2) and set addresses to eg. 10.10.11.20/24 on eno3 on server and 10.10.11.21 on eth2 on NAS, connection works (ping, LUNs mapping, anything).

When I try to use teaming with eno3 and 4 (preferably loadbalance mode set on Centos) and make bond on NAS (eth2 and 3 balance-tldb mode) with 10.10.11.20/24 on server and 10.10.11.21/24 on NAS respectively, connection doesn't work.

When I used NetworkManager on server sometimes connection did work but ping lasted for exacty 9 packets and went dead. So I decided to get rid of NetworkManager (stop&disable) and turned to editing ifcfg files:

ifcfg-team0:

DEVICE=team0
DEVICETYPE=Team
TEAM_CONFIG="{\"runner\": {\"name\": \"loadbalance\"}}"
BOOTPROTO=none
IPADDR=10.10.11.20
PREFIX=24
NAME=team0
ONBOOT=yes
NM_CONTROLLED=no

Slaves:
ifcfg-team0-eno3:

DEVICE=eno3
DEVICETYPE=TeamPort
TEAM_MASTER=team0
NAME=team0-eno3
ONBOOT=yes
NM_CONTROLLED=no

ifcfg-team0-eno4:

DEVICE=eno4
DEVICETYPE=TeamPort
TEAM_MASTER=team0
NAME=team0-eno4
ONBOOT=yes
NM_CONTROLLED=no

teamdctl team0 state:

setup:
  runner: loadbalance
ports:
  eno3
    link watches:
      link summary: up
      instance[link_watch_0]:
        name: ethtool
        link: up
        down count: 0
  eno4
    link watches:
      link summary: up
      instance[link_watch_0]:
        name: ethtool
        link: up
        down count: 0

ifconfig team0:

team0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 10.10.11.20  netmask 255.255.255.0  broadcast 10.10.11.255
        inet6 fe80::a94:efff:fe58:70f0  prefixlen 64  scopeid 0x20<link>
        ether 08:94:ef:58:70:f0  txqueuelen 1000  (Ethernet)
        RX packets 7  bytes 707 (707.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 131  bytes 8329 (8.1 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

and finally route -n:

Kernel IP routing table

Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         10.10.10.138    0.0.0.0         UG    0      0        0 eno2
10.10.10.0      0.0.0.0         255.255.255.0   U     0      0        0 eno2
10.10.11.0      0.0.0.0         255.255.255.0   U     0      0        0 team0
169.254.0.0     0.0.0.0         255.255.0.0     U     1002   0        0 eno1
169.254.0.0     0.0.0.0         255.255.0.0     U     1003   0        0 eno2
169.254.0.0     0.0.0.0         255.255.0.0     U     1004   0        0 ens4f0
169.254.0.0     0.0.0.0         255.255.0.0     U     1005   0        0 eno3
169.254.0.0     0.0.0.0         255.255.0.0     U     1006   0        0 eno4
169.254.0.0     0.0.0.0         255.255.0.0     U     1007   0        0 ens4f1
169.254.0.0     0.0.0.0         255.255.0.0     U     1008   0        0 ens4f2
169.254.0.0     0.0.0.0         255.255.0.0     U     1009   0        0 ens4f3
169.254.0.0     0.0.0.0         255.255.0.0     U     1011   0        0 team0
192.168.122.0   0.0.0.0         255.255.255.0   U     0      0        0 virbr0

On the NAS: ifconfig:

bond0     Link encap:Ethernet  HWaddr 24:5E:BE:26:6F:60
          inet addr:10.10.11.21  Bcast:10.10.11.255  Mask:255.255.255.0
          UP BROADCAST RUNNING MASTER MULTICAST  MTU:1500  Metric:1
          RX packets:1105 errors:0 dropped:574 overruns:0 frame:0
          TX packets:105005 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:103142 (100.7 KiB)  TX bytes:6566084 (6.2 MiB)

eth0      Link encap:Ethernet  HWaddr 24:5E:BE:26:6F:5E
          inet addr:10.10.10.12  Bcast:10.10.10.255  Mask:255.255.255.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:503686 errors:0 dropped:77 overruns:0 frame:0
          TX packets:370782 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:83802168 (79.9 MiB)  TX bytes:248700488 (237.1 MiB)
          Memory:91400000-9141ffff

eth1      Link encap:Ethernet  HWaddr 24:5E:BE:26:6F:5F
          UP BROADCAST MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)
          Memory:91300000-9131ffff

eth2      Link encap:Ethernet  HWaddr 24:5E:BE:26:6F:60
          UP BROADCAST RUNNING SLAVE MULTICAST  MTU:1500  Metric:1
          RX packets:3361082 errors:0 dropped:34 overruns:0 frame:0
          TX packets:380048 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:5024263161 (4.6 GiB)  TX bytes:45039739 (42.9 MiB)
          Memory:91200000-9121ffff

eth3      Link encap:Ethernet  HWaddr 24:5E:BE:26:6F:61
          UP BROADCAST NOTRAILERS RUNNING SLAVE MULTICAST  MTU:1500  Metric:1
          RX packets:776 errors:0 dropped:461 overruns:0 frame:0
          TX packets:54162 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:73093 (71.3 KiB)  TX bytes:3934987 (3.7 MiB)
          Memory:91100000-9111ffff

lo        Link encap:Local Loopback
          inet addr:127.0.0.1  Mask:255.0.0.0
          UP LOOPBACK RUNNING  MTU:65536  Metric:1
          RX packets:1238295 errors:0 dropped:0 overruns:0 frame:0
          TX packets:1238295 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:352816577 (336.4 MiB)  TX bytes:352816577 (336.4 MiB)

route -n:

Kernel IP routing table

Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         10.10.10.138    0.0.0.0         UG    0      0        0 eth0
10.10.10.0      0.0.0.0         255.255.255.0   U     0      0        0 eth0
10.10.11.0      0.0.0.0         255.255.255.0   U     0      0        0 bond0
127.0.0.0       0.0.0.0         255.0.0.0       U     0      0        0 lo

Should I add something to ifcfg? I have tried also LACP (802.3ad) and Balance-alb modes but to no avail.

1

Solved by properly setting LACPI and vlans on cisco switch connecting server and NAS.

| improve this answer | |
  • Well done for solving it, but your answer lacks enough detail to be widely useful. What was the "proper setting"? What exact lines did you add or change, in which config files, on which machines? Did you have to run any additional commands, or restart anything, to make the changes take effect? – JigglyNaga Oct 5 '18 at 10:39

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.