5

I am working on CentOS 6.4 and I am new to this operating system.

I was downloading a 5 GB file using wget command. I observed that it was trying to download the file from different IP addresses (54.240.168.41), which was blocked by the proxy server. So I got this specific IP address opened by the network support and the download started working.

Since it was a huge file, I left it to complete the execution overnight. Next morning, due to some network error, the download stopped. only 42% was downloaded. I tried to download the file using -c option of the wget command. However, wget keeps trying to connect to different IP addresses starting with 54.xxx.xxx.xxx, except the IP address 54.240.168.41.

My question is, how I would tell wget to download from a specific IP addresses which is NOT blocked by the network?.

This is the command that I am executing

wget --continue http://public-repo-1.hortonworks.com/HDP/centos6/2.x/updates/2.3.2.0/HDP-2.3.2.0-centos6-rpm.tar.gz
2
  • 2
    What's the actual command you used?
    – muru
    Nov 16, 2015 at 3:27
  • I have edited the question to have actual command.
    – KurioZ7
    Nov 16, 2015 at 3:32

3 Answers 3

Reset to default
15

This worked for me when switching DNS, and needed to access old server by IP but specified host header to route to my account at old server.

wget http://198.38.82.5/something.tar.gz --header "Host: domain-at-server.net"
3
  • My linux box exposes 3 IP4 addresses and a range of IP6 addresses. Can I set host header to one of these addresses, and wget will download through that network interface?
    – P i
    Oct 3, 2021 at 21:16
  • technically you could download by using IP directly. It depends how the server is configured, my solution was for shared hosting service where multiple domains are hosted from same IP. If you use Nginx on your VPS then you can configure host names per IP, and then you provide wget with the host name assigned to IP you want to use
    – Pawel Cioch
    Oct 5, 2021 at 21:00
  • 1
    As a side note, if you are trying to access a secure server (https://...) then it's going to try to verify the certificate and "198.38.82.5" is not going to validate... One way is to ignore the fact using --no-check-certificate.
    – Alexis Wilke
    Nov 1 at 3:43
3

I resolved this by using the option --no-dns-cache. When this option was given, it started listing the IP address 54.240.168.41 in the list of IPs it was trying to connect to.

wget --no-dns-cache --continue http://public-repo-1.hortonworks.com/HDP/centos6/2.x/updates/2.3.2.0/HDP-2.3.2.0-centos6-rpm.tar.gz
2

If you use a hostname instead of an IP in your command, you run the risk that the DNS entry for that hostname changes between runs. Instead, use the actual IP:

wget --continue http://54.240.168.41/HDP/centos6/2.x/updates/2.3.2.0/HDP-2.3.2.0-centos6-rpm.tar.gz

But this won't help if the server has actually moved to a separate IP. Consider asking your proxy admins to provide an exemption by hostname instead of by IP.

1
  • 4
    You'd actually probably need to provide --header 'Host: public-repo-1.hortonworks.com' as well, since the server side may serve different content under different names.
    – Ulrich Schwarz
    Nov 16, 2015 at 5:28

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .