4

On a server wget-1.16 takes 8 minutes to complete:

$ wget http://http.debian.net/debian/dists/stable/Release -O -
--2017-06-12 23:44:40--  http://http.debian.net/debian/dists/stable/Release         [4693/5569]
Resolving http.debian.net (http.debian.net)... 2001:4f8:1:c::15, 2605:bc80:3010:b00:0:deb:166:202, 2001:610:1908:b000::148:14, ...                                                            
Connecting to http.debian.net (http.debian.net)|2001:4f8:1:c::15|:80... failed: Connection timed out.                 
Connecting to http.debian.net (http.debian.net)|2605:bc80:3010:b00:0:deb:166:202|:80... failed: Connection timed out. 
Connecting to http.debian.net (http.debian.net)|2001:610:1908:b000::148:14|:80... failed: Connection timed out.       
Connecting to http.debian.net (http.debian.net)|140.211.166.202|:80... connected.
HTTP request sent, awaiting response... 302 Found
Location: http://cdn-fastly.deb.debian.org/debian/dists/stable/Release [following]
--2017-06-12 23:51:02--  http://cdn-fastly.deb.debian.org/debian/dists/stable/Release
Resolving cdn-fastly.deb.debian.org (cdn-fastly.deb.debian.org)... 2a04:4e42:3::204, 151.101.12.204                   
Connecting to cdn-fastly.deb.debian.org (cdn-fastly.deb.debian.org)|2a04:4e42:3::204|:80... failed: Connection timed out.
Connecting to cdn-fastly.deb.debian.org (cdn-fastly.deb.debian.org)|151.101.12.204|:80... connected.
...

Because it is trying to connect using IPv6 address. curl-7.38.0 on the same machine responds instantly. Because it uses IPv4 address. Do they resolve domain differently? How do they do it? How can I make wget use IPv4 address?

UPD

$ ip a
...
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link/ether d8:cb:8a:37:cf:57 brd ff:ff:ff:ff:ff:ff
    inet 188.40.99.4/26 brd 188.40.99.63 scope global eth0
       valid_lft forever preferred_lft forever
    inet6 2a01:4f8:100:738b::2/64 scope global 
       valid_lft forever preferred_lft forever 
    inet6 fe80::dacb:8aff:fe37:cf57/64 scope link                                                     valid_lft forever preferred_lft forever 

$ ip route
default via 188.40.99.1 dev eth0 
10.0.0.0/24 dev br0  proto kernel  scope link  src 10.0.0.1 
188.40.99.0/26 via 188.40.99.1 dev eth0 
188.40.99.0/26 dev eth0  proto kernel  scope link  src 188.40.99.4 
  • Use --inet4-only perhaps? – DopeGhoti Jun 12 '17 at 21:30
  • Or maybe --bind-address= in case your server has multiple IPv4 addresses. – Deathgrip Jun 12 '17 at 21:53
  • Actually, the bigger picture is, the issue manifested itself when creating lxc container with lxc-create. So I can't possibly add extra switches to wget. But more importantly, why curl succeeds fast, but wget doesn't? Doesn't it all come down to some system call they both should use? – x-yuri Jun 12 '17 at 22:00
  • Your system seems to think that it has IPv6 connectivity without actually having it. Can you show your interface addresses and routing table? – Sander Steffann Jun 12 '17 at 22:43
  • 1
    You have a global IPv6 address. You only show the IPv4 routing table, so I can't see if you have a default IPv6 route, but I'm pretty sure you do. You can check with ip -6 route. The problem is that something isn't working. The fast failover of curl masks the problem. The real problem is that your IPv6 configuration isn't working. You might need to talk to your network admin or Idaho to find our what exactly is wrong. – Sander Steffann Jun 13 '17 at 3:53
7

curl and wget do not use different mechanisms for resolving domains (they're using getaddrinfo()). However, curl implements a fast fallback algorithm to improve the user experience in cases where IPv6 connectivity is less than good.

This algorithm is described in detail in RFC 6555 (Happy Eyeballs): https://tools.ietf.org/html/rfc6555

According to curl/lib/connect.h this timeout is set to 200ms: https://github.com/curl/curl/blob/a8e523f086c12e7bb9acb18d1ac84d92dde0605b/lib/connect.h#L43

Both curl and wget support -4/-6 options which will force the connection to either IPv4 or IPv6 respectively.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.