1

I have two machines both network cards speed is 10Gbps.

  • The inbound bandwidth between the two machines is 10Gbps, the outbound bandwidth internet bandwidth is 500Mpbs.
  • Both machines are using public IP addresses in the public and private network.
  • Both machines transfer and connecti on Nginx's port 80, and machine B is used for streaming media, like Youtube streamed videos.

I checked the transfer speed using iperf utility From machine A to machine B:

# iperf -c 0.0.0.2 -p 8777
------------------------------------------------------------
Client connecting to 0.0.0.2, TCP port 8777
TCP window size: 85.3 KByte (default)
------------------------------------------------------------
[  3] local 0.0.0.1 port 38895 connected with 0.0.0.2 port 8777
[ ID] Interval       Transfer     Bandwidth
[  3]  0.0-10.8 sec    528 KBytes    399 Kbits/sec

My Current Connections in machine B:

# netstat -an|grep ":8777"|awk '/tcp/ {print $6}'|sort -nr| uniq -c
   2072 TIME_WAIT
     28 SYN_RECV
      1 LISTEN
    189 LAST_ACK
    139 FIN_WAIT2
    373 FIN_WAIT1
   3381 ESTABLISHED
     34 CLOSING

machine A Network Card Information:

Settings for eth0:
        Supported ports: [ TP ]
        Supported link modes:   100baseT/Full
                                1000baseT/Full
                                10000baseT/Full
        Supported pause frame use: No
        Supports auto-negotiation: Yes
        Advertised link modes:  10000baseT/Full
        Advertised pause frame use: No
        Advertised auto-negotiation: Yes
        Speed: 10000Mb/s
        Duplex: Full
        Port: Twisted Pair
        PHYAD: 0
        Transceiver: external
        Auto-negotiation: on
        MDI-X: Unknown
        Supports Wake-on: d
        Wake-on: d
        Current message level: 0x00000007 (7)
                               drv probe link
        Link detected: yes

machine B Network Card Information:

Settings for eth2:
        Supported ports: [ FIBRE ]
        Supported link modes:   10000baseT/Full
        Supported pause frame use: No
        Supports auto-negotiation: No
        Advertised link modes:  10000baseT/Full
        Advertised pause frame use: No
        Advertised auto-negotiation: No
        Speed: 10000Mb/s
        Duplex: Full
        Port: Direct Attach Copper
        PHYAD: 0
        Transceiver: external
        Auto-negotiation: off
        Supports Wake-on: d
        Wake-on: d
        Current message level: 0x00000007 (7)
                               drv probe link
        Link detected: yes

ifconfig machine A:

eth0      Link encap:Ethernet  HWaddr 00:25:90:ED:9E:AA
          inet addr:0.0.0.1  Bcast:0.0.0.255  Mask:255.255.255.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:1202795665 errors:0 dropped:64334 overruns:0 frame:0
          TX packets:2313161968 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:893413096188 (832.0 GiB)  TX bytes:3360949570454 (3.0 TiB)

lo        Link encap:Local Loopback
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:65536  Metric:1
          RX packets:2207544 errors:0 dropped:0 overruns:0 frame:0
          TX packets:2207544 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:247769175 (236.2 MiB)  TX bytes:247769175 (236.2 MiB)

ifconfig machine B:

eth2      Link encap:Ethernet  HWaddr 00:25:90:82:C4:FE
          inet addr:0.0.0.2  Bcast:0.0.0.2  Mask:255.255.255.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:39973046980 errors:0 dropped:1828387600 overruns:0 frame:0
          TX packets:69618752480 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:3013976063688 (2.7 TiB)  TX bytes:102250230803933 (92.9 TiB)

lo        Link encap:Local Loopback
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:65536  Metric:1
          RX packets:1049495 errors:0 dropped:0 overruns:0 frame:0
          TX packets:1049495 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:129012422 (123.0 MiB)  TX bytes:129012422 (123.0 MiB)

The problem

As you can see from iperf utility, the transfer speed from machine A to machine B slow. When I restart network service on machine B the transfer to machine A is at full speed, after 2 minutes, it's getting slow.

How could I troubleshoot the slow speed issue and fix it on machine B?

Notice: if there are any other commands I should execute on either machine for more information, so it might help resolve the problem, please let me know in comments.

  • The output from B's ethtool looks off to me. I'd start there. – slm Jun 1 '14 at 14:54
  • @slm what exactly should i start to do with machine B ? – Linux Guy Jun 1 '14 at 15:04
  • Is the output right that it's a fibre connection? – slm Jun 1 '14 at 17:04
  • yes its on fiber connection , the two machines are located in a data center – Linux Guy Jun 1 '14 at 21:07
  • Given the sensitivity of these things when at high speeds, I would be a little suspicious of the cabling. – slm Jun 2 '14 at 6:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.