802.3ad on FreeBSD and Linux using crossover cables?

Question

I have setup a NFS server on FreeBSD 8.1, which will be serving one Linux client. Each box has a dual-port 1gig fiber card, and each box is directly connected to the other over this link. There is no switch involved.

I suppose that my question is, since both FreeBSD and Linux support 802.3ad, is the following the right way to configure this, when not using a switch? Or is there a better way? Is this even supported?

With this setup, I am able to ping between the hosts, transfer files over NFS, etc.

FreeBSD  em2 <-> eth2 Linux 
         em3 <-> eth3

I have configured em2 and em3 for 802.3ad (lacp):

lagg0: flags=8843 metric 0 mtu 1500
 options=9b
 ether 00:04:23:c2:c2:e0
 inet 192.168.0.1 netmask 0xfffffffc broadcast 192.168.0.3
 media: Ethernet autoselect
 status: active
 laggproto lacp
 laggport: em3 flags=1c
 laggport: em2 flags=1c

if_lagg_load="YES"
ifconfig_em2="UP"
ifconfig_em3="UP"
#ifconfig_lagg0="create"
cloned_interfaces="lagg0"
ifconfig_lagg0="laggproto lacp laggport em2 laggport em3 192.168.0.1 netmask 255.255.255.252"

And eth2 and eth3 for 802.3ad bonding:

[root@xbox ~]# ifconfig bond0
bond0     Link encap:Ethernet  HWaddr 00:04:23:C2:C4:DA  
          inet addr:192.168.0.2  Bcast:192.168.0.3  Mask:255.255.255.252
          inet6 addr: fe80::204:23ff:fec2:c4da/64 Scope:Link
          UP BROADCAST RUNNING MASTER MULTICAST  MTU:1500  Metric:1
          RX packets:140000208 errors:0 dropped:0 overruns:0 frame:0
          TX packets:101560577 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:3281219872 (3.0 GiB)  TX bytes:3609990240 (3.3 GiB)

[root@xbox network-scripts]# cat ifcfg-bond0 
DEVICE=bond0
IPADDR=192.168.0.2
NETWORK=192.168.0.0
NETMASK=255.255.255.252
USERCTL=no
BOOTPROTO=none
ONBOOT=yes
[root@xbox network-scripts]# cat ifcfg-eth2
DEVICE=eth2
USERCTL=no
ONBOOT=yes
MASTER=bond0
SLAVE=yes
BOOTPROTO=none
[root@xbox network-scripts]# cat ifcfg-eth3
DEVICE=eth3
USERCTL=no
ONBOOT=yes
MASTER=bond0
SLAVE=yes
BOOTPROTO=none
[root@xbox network-scripts]# tail -n2 /etc/modprobde.conf
alias bond0 bonding
options bond0 mode=802.3ad miimon=100

EDIT: I've found that, on the FreeBSD box, the MAC addresses of em2 and em3 are the same, where-as on the Linux box, the MAC addresses of eth2 and eth3 are different.

Answer 1

You can do that, but it won't do what you want. Due to an esoteric requirement of the 802.11ad standard that requires packets sent to the same host over the trunk to arrive in the same order they were sent in, the kernel can not load balance packets to the same host over multiple links -- it can only send packets to some hosts over one link, and packets to some other hosts over the other. Since you only have one host ( on each end ), all of the packets will end up going over one link anyhow.

Answer 2

Your setup is supported. If there's a switch between the hosts using link aggregation, the switch needs to be LACP-aware, but if there's no switch, there's no problem.