1

For some time I have strange network lags when using virtual NICs created by VPNs like ZeroTier and Hamachi, especially when sending large amounts of data. I use Ubuntu 20.04, kernel 5.4.0-67-generic.

Wireshark shows a lot of TCP retransmissions, ACK dupes and out-of-order packets.

1   0.000000000 172.27.0.110    172.27.0.179    TCP 74  44602 → 25565 [SYN] Seq=0 Win=63480 Len=0 MSS=2760 SACK_PERM=1 TSval=2974512735 TSecr=0 WS=128
2   0.045201707 172.27.0.179    172.27.0.110    TCP 62  25565 → 44602 [SYN, ACK] Seq=0 Ack=1 Win=63480 Len=0 MSS=2760 SACK_PERM=1
3   0.045262797 172.27.0.110    172.27.0.179    TCP 54  44602 → 25565 [ACK] Seq=1 Ack=1 Win=63480 Len=0
4   0.049124813 172.27.0.110    172.27.0.179    TCP 74  44602 → 25565 [PSH, ACK] Seq=1 Ack=1 Win=63480 Len=20
5   0.049473924 172.27.0.110    172.27.0.179    TCP 65  44602 → 25565 [PSH, ACK] Seq=21 Ack=1 Win=63480 Len=11
6   0.093782891 172.27.0.179    172.27.0.110    TCP 54  25565 → 44602 [ACK] Seq=1 Ack=32 Win=63449 Len=0
7   0.096647589 172.27.0.179    172.27.0.110    TCP 58  25565 → 44602 [PSH, ACK] Seq=1 Ack=32 Win=63449 Len=4
8   0.096672987 172.27.0.110    172.27.0.179    TCP 54  44602 → 25565 [ACK] Seq=32 Ack=5 Win=63476 Len=0
9   0.096770408 172.27.0.179    172.27.0.110    TCP 82  25565 → 44602 [PSH, ACK] Seq=5 Ack=32 Win=63449 Len=28
10  0.096783346 172.27.0.110    172.27.0.179    TCP 54  44602 → 25565 [ACK] Seq=32 Ack=33 Win=63448 Len=0
11  0.138199037 172.27.0.179    172.27.0.110    TCP 149 [TCP Previous segment not captured] 25565 → 44602 [PSH, ACK] Seq=2793 Ack=32 Win=63449 Len=95
12  0.138217852 172.27.0.110    172.27.0.179    TCP 66  [TCP Dup ACK 10#1] 44602 → 25565 [ACK] Seq=32 Ack=33 Win=63448 Len=0 SLE=2793 SRE=2888
13  0.138274049 172.27.0.179    172.27.0.110    TCP 81  25565 → 44602 [PSH, ACK] Seq=2888 Ack=32 Win=63449 Len=27
14  0.138281634 172.27.0.110    172.27.0.179    TCP 66  [TCP Dup ACK 10#2] 44602 → 25565 [ACK] Seq=32 Ack=33 Win=63448 Len=0 SLE=2793 SRE=2915
15  0.138309371 172.27.0.179    172.27.0.110    TCP 59  25565 → 44602 [PSH, ACK] Seq=2915 Ack=32 Win=63449 Len=5
16  0.138315649 172.27.0.110    172.27.0.179    TCP 66  [TCP Dup ACK 10#3] 44602 → 25565 [ACK] Seq=32 Ack=33 Win=63448 Len=0 SLE=2793 SRE=2920
17  0.138342033 172.27.0.179    172.27.0.110    TCP 66  25565 → 44602 [PSH, ACK] Seq=2920 Ack=32 Win=63449 Len=12
18  0.138348253 172.27.0.110    172.27.0.179    TCP 66  [TCP Dup ACK 10#4] 44602 → 25565 [ACK] Seq=32 Ack=33 Win=63448 Len=0 SLE=2793 SRE=2932
19  0.138374146 172.27.0.179    172.27.0.110    TCP 58  25565 → 44602 [PSH, ACK] Seq=2932 Ack=32 Win=63449 Len=4 [TCP segment of a reassembled PDU]
20  0.138380017 172.27.0.110    172.27.0.179    TCP 66  [TCP Dup ACK 10#5] 44602 → 25565 [ACK] Seq=32 Ack=33 Win=63448 Len=0 SLE=2793 SRE=2936
21  0.139014360 172.27.0.179    172.27.0.110    TCP 2814    [TCP Fast Retransmission] 25565 → 44602 [ACK] Seq=33 Ack=32 Win=63449 Len=2760
22  0.139025513 172.27.0.110    172.27.0.179    TCP 54  44602 → 25565 [ACK] Seq=32 Ack=2936 Win=60720 Len=0
23  0.141892063 172.27.0.179    172.27.0.110    TCP 2814    25565 → 44602 [ACK] Seq=2936 Ack=32 Win=63449 Len=2760 [TCP segment of a reassembled PDU]
24  0.141913064 172.27.0.110    172.27.0.179    TCP 54  44602 → 25565 [ACK] Seq=32 Ack=5696 Win=60720 Len=0
25  0.303248158 172.27.0.110    172.27.0.179    TCP 68  44602 → 25565 [PSH, ACK] Seq=32 Ack=5696 Win=60720 Len=14
26  0.303554866 172.27.0.110    172.27.0.179    TCP 81  44602 → 25565 [PSH, ACK] Seq=46 Ack=5696 Win=60720 Len=27
27  0.303716262 172.27.0.110    172.27.0.179    TCP 59  44602 → 25565 [PSH, ACK] Seq=73 Ack=5696 Win=60720 Len=5
28  0.349692404 172.27.0.179    172.27.0.110    TCP 54  [TCP Previous segment not captured] 25565 → 44602 [ACK] Seq=32380 Ack=73 Win=63408 Len=0
29  0.402913460 172.27.0.179    172.27.0.110    TCP 54  25565 → 44602 [ACK] Seq=32380 Ack=78 Win=63403 Len=0
30  1.101234508 172.27.0.179    172.27.0.110    TCP 2814    [TCP Retransmission] 25565 → 44602 [ACK] Seq=5696 Ack=78 Win=63403 Len=2760
31  1.101261884 172.27.0.110    172.27.0.179    TCP 54  44602 → 25565 [ACK] Seq=78 Ack=8456 Win=60720 Len=0
32  1.147180805 172.27.0.179    172.27.0.110    TCP 2814    25565 → 44602 [PSH, ACK] Seq=32380 Ack=78 Win=63403 Len=2760
33  1.147213877 172.27.0.110    172.27.0.179    TCP 66  [TCP Dup ACK 31#1] 44602 → 25565 [ACK] Seq=78 Ack=8456 Win=60720 Len=0 SLE=32380 SRE=35140
34  3.300628848 172.27.0.179    172.27.0.110    TCP 2814    [TCP Retransmission] 25565 → 44602 [ACK] Seq=8456 Ack=78 Win=63403 Len=2760
35  3.300676657 172.27.0.110    172.27.0.179    TCP 66  44602 → 25565 [ACK] Seq=78 Ack=11216 Win=60720 Len=0 SLE=32380 SRE=35140
36  3.350777564 172.27.0.179    172.27.0.110    TCP 2814    [TCP Previous segment not captured] 25565 → 44602 [PSH, ACK] Seq=37900 Ack=78 Win=63403 Len=2760
37  3.350813605 172.27.0.110    172.27.0.179    TCP 74  [TCP Dup ACK 35#1] 44602 → 25565 [ACK] Seq=78 Ack=11216 Win=60720 Len=0 SLE=37900 SRE=40660 SLE=32380 SRE=35140
38  4.313838376 172.27.0.179    172.27.0.110    TCP 2814    [TCP Retransmission] 25565 → 44602 [ACK] Seq=11216 Ack=78 Win=63403 Len=2760
39  4.313878478 172.27.0.110    172.27.0.179    TCP 74  44602 → 25565 [ACK] Seq=78 Ack=13976 Win=57960 Len=0 SLE=37900 SRE=40660 SLE=32380 SRE=35140
40  4.360138604 172.27.0.179    172.27.0.110    TCP 2814    [TCP Previous segment not captured] 25565 → 44602 [PSH, ACK] Seq=43420 Ack=78 Win=63403 Len=2760
41  4.360199131 172.27.0.110    172.27.0.179    TCP 82  [TCP Dup ACK 39#1] 44602 → 25565 [ACK] Seq=78 Ack=13976 Win=57960 Len=0 SLE=43420 SRE=46180 SLE=37900 SRE=40660 SLE=32380 SRE=35140
42  5.325241275 172.27.0.179    172.27.0.110    TCP 2814    [TCP Retransmission] 25565 → 44602 [ACK] Seq=13976 Ack=78 Win=63403 Len=2760
43  5.325266889 172.27.0.110    172.27.0.179    TCP 82  44602 → 25565 [ACK] Seq=78 Ack=16736 Win=57960 Len=0 SLE=43420 SRE=46180 SLE=37900 SRE=40660 SLE=32380 SRE=35140
44  5.371542444 172.27.0.179    172.27.0.110    TCP 2814    [TCP Previous segment not captured] 25565 → 44602 [PSH, ACK] Seq=48940 Ack=78 Win=63403 Len=2760
45  5.371570142 172.27.0.110    172.27.0.179    TCP 90  [TCP Dup ACK 43#1] 44602 → 25565 [ACK] Seq=78 Ack=16736 Win=57960 Len=0 SLE=48940 SRE=51700 SLE=43420 SRE=46180 SLE=37900 SRE=40660 SLE=32380 SRE=35140
46  5.724447336 172.27.0.179    172.27.0.110    TCP 2814    [TCP Retransmission] 25565 → 44602 [PSH, ACK] Seq=16736 Ack=78 Win=63403 Len=2760
47  5.724488523 172.27.0.110    172.27.0.179    TCP 90  44602 → 25565 [ACK] Seq=78 Ack=19496 Win=55200 Len=0 SLE=48940 SRE=51700 SLE=43420 SRE=46180 SLE=37900 SRE=40660 SLE=32380 SRE=35140
50  6.079468803 172.27.0.179    172.27.0.110    TCP 2814    [TCP Retransmission] 25565 → 44602 [PSH, ACK] Seq=19496 Ack=78 Win=63403 Len=2760
51  6.079512860 172.27.0.110    172.27.0.179    TCP 90  44602 → 25565 [ACK] Seq=78 Ack=22256 Win=55200 Len=0 SLE=48940 SRE=51700 SLE=43420 SRE=46180 SLE=37900 SRE=40660 SLE=32380 SRE=35140
52  7.039116534 172.27.0.179    172.27.0.110    TCP 2814    [TCP Retransmission] 25565 → 44602 [PSH, ACK] Seq=22256 Ack=78 Win=63403 Len=2760
53  7.039152631 172.27.0.110    172.27.0.179    TCP 90  44602 → 25565 [ACK] Seq=78 Ack=25016 Win=55200 Len=0 SLE=48940 SRE=51700 SLE=43420 SRE=46180 SLE=37900 SRE=40660 SLE=32380 SRE=35140
54  7.053728613 172.27.0.110    172.27.0.179    TCP 94  44602 → 25565 [PSH, ACK] Seq=78 Ack=25016 Win=57960 Len=4 SLE=48940 SRE=51700 SLE=43420 SRE=46180 SLE=37900 SRE=40660 SLE=32380 SRE=35140
55  7.053890617 172.27.0.110    172.27.0.179    TCP 126 44602 → 25565 [PSH, ACK] Seq=82 Ack=25016 Win=57960 Len=36 SLE=48940 SRE=51700 SLE=43420 SRE=46180 SLE=37900 SRE=40660 SLE=32380 SRE=35140
56  7.087735464 172.27.0.179    172.27.0.110    TCP 2814    [TCP Previous segment not captured] 25565 → 44602 [PSH, ACK] Seq=65500 Ack=78 Win=63403 Len=2760
57  7.087768694 172.27.0.110    172.27.0.179    TCP 90  [TCP Dup ACK 53#1] 44602 → 25565 [ACK] Seq=118 Ack=25016 Win=57960 Len=0 SLE=65500 SRE=68260 SLE=48940 SRE=51700 SLE=43420 SRE=46180 SLE=37900 SRE=40660
58  7.150668480 172.27.0.179    172.27.0.110    TCP 54  25565 → 44602 [ACK] Seq=33296 Ack=118 Win=63363 Len=0
59  7.444215482 172.27.0.179    172.27.0.110    TCP 2814    [TCP Retransmission] 25565 → 44602 [PSH, ACK] Seq=25016 Ack=118 Win=63363 Len=2760
60  7.444256497 172.27.0.110    172.27.0.179    TCP 90  44602 → 25565 [ACK] Seq=118 Ack=27776 Win=55200 Len=0 SLE=65500 SRE=68260 SLE=48940 SRE=51700 SLE=43420 SRE=46180 SLE=37900 SRE=40660
61  8.398822152 172.27.0.179    172.27.0.110    TCP 2814    [TCP Retransmission] 25565 → 44602 [PSH, ACK] Seq=27776 Ack=118 Win=63363 Len=2760
62  8.398869702 172.27.0.110    172.27.0.179    TCP 90  44602 → 25565 [ACK] Seq=118 Ack=30536 Win=52440 Len=0 SLE=65500 SRE=68260 SLE=48940 SRE=51700 SLE=43420 SRE=46180 SLE=37900 SRE=40660
63  8.447356900 172.27.0.179    172.27.0.110    TCP 2814    [TCP Previous segment not captured] 25565 → 44602 [ACK] Seq=76540 Ack=118 Win=63363 Len=2760
64  8.447393248 172.27.0.110    172.27.0.179    TCP 90  [TCP Dup ACK 62#1] 44602 → 25565 [ACK] Seq=118 Ack=30536 Win=52440 Len=0 SLE=76540 SRE=79300 SLE=65500 SRE=68260 SLE=48940 SRE=51700 SLE=43420 SRE=46180
65  8.810606201 172.27.0.179    172.27.0.110    TCP 2814    [TCP Retransmission] 25565 → 44602 [PSH, ACK] Seq=30536 Ack=118 Win=63363 Len=2760
66  8.810641092 172.27.0.110    172.27.0.179    TCP 90  44602 → 25565 [ACK] Seq=118 Ack=35140 Win=52440 Len=0 SLE=32380 SRE=33296 SLE=76540 SRE=79300 SLE=65500 SRE=68260 SLE=48940 SRE=51700
67  8.856490459 172.27.0.179    172.27.0.110    TCP 330 [TCP Retransmission] 25565 → 44602 [PSH, ACK] Seq=40660 Ack=118 Win=63363 Len=276
68  8.856514714 172.27.0.110    172.27.0.179    TCP 90  [TCP Window Update] 44602 → 25565 [ACK] Seq=118 Ack=35140 Win=55200 Len=0 SLE=40660 SRE=40936 SLE=76540 SRE=79300 SLE=65500 SRE=68260 SLE=48940 SRE=51700
69  9.775748115 172.27.0.179    172.27.0.110    TCP 2814    [TCP Retransmission] 25565 → 44602 [PSH, ACK] Seq=35140 Ack=118 Win=63363 Len=2760
70  9.775791032 172.27.0.110    172.27.0.179    TCP 82  44602 → 25565 [ACK] Seq=118 Ack=40936 Win=52440 Len=0 SLE=76540 SRE=79300 SLE=65500 SRE=68260 SLE=48940 SRE=51700
71  9.821968969 172.27.0.179    172.27.0.110    TCP 2814    [TCP Previous segment not captured] 25565 → 44602 [ACK] Seq=87580 Ack=118 Win=63363 Len=2760
72  9.821992406 172.27.0.110    172.27.0.179    TCP 90  [TCP Window Update] 44602 → 25565 [ACK] Seq=118 Ack=40936 Win=55200 Len=0 SLE=87580 SRE=90340 SLE=76540 SRE=79300 SLE=65500 SRE=68260 SLE=48940 SRE=51700
73  10.176466157    172.27.0.179    172.27.0.110    TCP 2814    [TCP Retransmission] 25565 → 44602 [PSH, ACK] Seq=40936 Ack=118 Win=63363 Len=2760
74  10.176511928    172.27.0.110    172.27.0.179    TCP 90  44602 → 25565 [ACK] Seq=118 Ack=46180 Win=52440 Len=0 SLE=43420 SRE=43696 SLE=87580 SRE=90340 SLE=76540 SRE=79300 SLE=65500 SRE=68260
75  10.533491944    172.27.0.179    172.27.0.110    TCP 2814    [TCP Retransmission] 25565 → 44602 [PSH, ACK] Seq=46180 Ack=118 Win=63363 Len=2760
76  10.533531151    172.27.0.110    172.27.0.179    TCP 82  44602 → 25565 [ACK] Seq=118 Ack=51700 Win=55200 Len=0 SLE=87580 SRE=90340 SLE=76540 SRE=79300 SLE=65500 SRE=68260
79  11.487938977    172.27.0.179    172.27.0.110    TCP 2814    [TCP Retransmission] 25565 → 44602 [PSH, ACK] Seq=51700 Ack=118 Win=63363 Len=2760
80  11.487983949    172.27.0.110    172.27.0.179    TCP 82  44602 → 25565 [ACK] Seq=118 Ack=54460 Win=57960 Len=0 SLE=87580 SRE=90340 SLE=76540 SRE=79300 SLE=65500 SRE=68260
81  11.533904915    172.27.0.179    172.27.0.110    TCP 2814    [TCP Previous segment not captured] 25565 → 44602 [ACK] Seq=104140 Ack=118 Win=63363 Len=2760
82  11.533940582    172.27.0.110    172.27.0.179    TCP 90  [TCP Dup ACK 80#1] 44602 → 25565 [ACK] Seq=118 Ack=54460 Win=57960 Len=0 SLE=104140 SRE=106900 SLE=87580 SRE=90340 SLE=76540 SRE=79300 SLE=65500 SRE=68260
83  11.885482472    172.27.0.179    172.27.0.110    TCP 2814    [TCP Retransmission] 25565 → 44602 [PSH, ACK] Seq=54460 Ack=118 Win=63363 Len=2760
84  11.885521789    172.27.0.110    172.27.0.179    TCP 90  44602 → 25565 [ACK] Seq=118 Ack=57220 Win=55200 Len=0 SLE=104140 SRE=106900 SLE=87580 SRE=90340 SLE=76540 SRE=79300 SLE=65500 SRE=68260
85  12.853577770    172.27.0.179    172.27.0.110    TCP 2814    [TCP Retransmission] 25565 → 44602 [PSH, ACK] Seq=57220 Ack=118 Win=63363 Len=2760
86  12.853620652    172.27.0.110    172.27.0.179    TCP 90  44602 → 25565 [ACK] Seq=118 Ack=59980 Win=55200 Len=0 SLE=104140 SRE=106900 SLE=87580 SRE=90340 SLE=76540 SRE=79300 SLE=65500 SRE=68260
87  12.906044223    172.27.0.179    172.27.0.110    TCP 2814    [TCP Window Full] [TCP Previous segment not captured] 25565 → 44602 [PSH, ACK] Seq=112420 Ack=118 Win=63363 Len=2760
88  12.906065406    172.27.0.110    172.27.0.179    TCP 90  [TCP Dup ACK 86#1] 44602 → 25565 [ACK] Seq=118 Ack=59980 Win=55200 Len=0 SLE=112420 SRE=115180 SLE=104140 SRE=106900 SLE=87580 SRE=90340 SLE=76540 SRE=79300
89  12.981535818    172.27.0.110    172.27.0.179    TCP 90  44602 → 25565 [FIN, ACK] Seq=118 Ack=59980 Win=55200 Len=0 SLE=112420 SRE=115180 SLE=104140 SRE=106900 SLE=87580 SRE=90340 SLE=76540 SRE=79300
90  13.026084367    172.27.0.179    172.27.0.110    TCP 54  25565 → 44602 [ACK] Seq=115180 Ack=119 Win=63363 Len=0
91  13.057004830    172.27.0.110    172.27.0.179    TCP 74  44606 → 25565 [SYN] Seq=0 Win=63480 Len=0 MSS=2760 SACK_PERM=1 TSval=2974525792 TSecr=0 WS=128
92  13.101638708    172.27.0.179    172.27.0.110    TCP 62  25565 → 44606 [SYN, ACK] Seq=0 Ack=1 Win=63480 Len=0 MSS=2760 SACK_PERM=1
93  13.101687138    172.27.0.110    172.27.0.179    TCP 54  44606 → 25565 [ACK] Seq=1 Ack=1 Win=63480 Len=0
94  13.103774881    172.27.0.110    172.27.0.179    TCP 74  44606 → 25565 [PSH, ACK] Seq=1 Ack=1 Win=63480 Len=20
95  13.103992979    172.27.0.110    172.27.0.179    TCP 56  44606 → 25565 [PSH, ACK] Seq=21 Ack=1 Win=63480 Len=2
96  13.155155809    172.27.0.179    172.27.0.110    TCP 54  25565 → 44606 [ACK] Seq=1 Ack=23 Win=63458 Len=0
97  13.577219433    172.27.0.179    172.27.0.110    TCP 2814    25565 → 44606 [ACK] Seq=1 Ack=23 Win=63458 Len=2760
98  13.577258814    172.27.0.110    172.27.0.179    TCP 54  44606 → 25565 [ACK] Seq=23 Ack=2761 Win=60720 Len=0
99  13.869278968    172.27.0.179    172.27.0.110    TCP 2814    [TCP Retransmission] 25565 → 44602 [PSH, ACK] Seq=59980 Ack=119 Win=63363 Len=2760
100 13.869308377    172.27.0.110    172.27.0.179    TCP 54  44602 → 25565 [RST] Seq=119 Win=0 Len=0
101 14.544962843    172.27.0.179    172.27.0.110    TCP 2814    25565 → 44606 [ACK] Seq=2761 Ack=23 Win=63458 Len=2760
102 14.544993395    172.27.0.110    172.27.0.179    TCP 54  44606 → 25565 [ACK] Seq=23 Ack=5521 Win=60720 Len=0
103 15.501396856    172.27.0.179    172.27.0.110    TCP 2814    25565 → 44606 [ACK] Seq=5521 Ack=23 Win=63458 Len=2760
104 15.501426755    172.27.0.110    172.27.0.179    TCP 54  44606 → 25565 [ACK] Seq=23 Ack=8281 Win=60720 Len=0
105 15.551232714    172.27.0.179    172.27.0.110    TCP 2656    25565 → 44606 [PSH, ACK] Seq=8281 Ack=23 Win=63458 Len=2602
106 15.551258749    172.27.0.110    172.27.0.179    TCP 54  44606 → 25565 [ACK] Seq=23 Ack=10883 Win=60720 Len=0
107 15.554164780    172.27.0.110    172.27.0.179    TCP 64  44606 → 25565 [PSH, ACK] Seq=23 Ack=10883 Win=60720 Len=10
108 15.600773646    172.27.0.179    172.27.0.110    TCP 64  25565 → 44606 [PSH, ACK] Seq=10883 Ack=33 Win=63448 Len=10
109 15.600819407    172.27.0.110    172.27.0.179    TCP 54  44606 → 25565 [ACK] Seq=33 Ack=10893 Win=60720 Len=0
110 15.600929437    172.27.0.179    172.27.0.110    TCP 54  25565 → 44606 [FIN, ACK] Seq=10893 Ack=33 Win=63448 Len=0
111 15.601400939    172.27.0.110    172.27.0.179    TCP 54  44606 → 25565 [FIN, ACK] Seq=33 Ack=10894 Win=60720 Len=0
112 15.646026620    172.27.0.179    172.27.0.110    TCP 54  25565 → 44606 [ACK] Seq=10894 Ack=34 Win=63448 Len=0

I have tried:

I want to avoid:

  • Reinstalling the whole system

1 Answer 1

0

it might be an issue with the linux tun/tap driver (or the way it is implemented in zerotier/hamachi), which I believe is what zerotier uses and is possibly what hamachi uses as well. see this and this

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.