3

I am working on CentOS 6.4 and I am new to this operating system.

I was downloading a 5 GB file using wget command. I observed that it was trying to download the file from different IP addresses (54.240.168.41), which was blocked by the proxy server. So I got this specific IP address opened by the network support and the download started working.

Since it was a huge file, I left it to complete the execution overnight. Next morning, due to some network error, the download stopped. only 42% was downloaded. I tried to download the file using -c option of the wget command. However, wget keeps trying to connect to different IP addresses starting with 54.xxx.xxx.xxx, except the IP address 54.240.168.41.

My question is, how I would tell wget to download from a specific IP addresses which is NOT blocked by the network?.

This is the command that I am executing

wget --continue http://public-repo-1.hortonworks.com/HDP/centos6/2.x/updates/2.3.2.0/HDP-2.3.2.0-centos6-rpm.tar.gz 
  • 2
    What's the actual command you used? – muru Nov 16 '15 at 3:27
  • I have edited the question to have actual command. – KurioZ7 Nov 16 '15 at 3:32
6

This worked for me when switching DNS, and needed to access old server by IP but specified host header to route to my account at old server.

wget http://198.38.82.5/something.tar.gz --header "Host: domain-at-server.net"
| improve this answer | |
2

If you use a hostname instead of an IP in your command, you run the risk that the DNS entry for that hostname changes between runs. Instead, use the actual IP:

wget --continue http://54.240.168.41/HDP/centos6/2.x/updates/2.3.2.0/HDP-2.3.2.0-centos6-rpm.tar.gz 

But this won't help if the server has actually moved to a separate IP. Consider asking your proxy admins to provide an exemption by hostname instead of by IP.

| improve this answer | |
  • 4
    You'd actually probably need to provide --header 'Host: public-repo-1.hortonworks.com' as well, since the server side may serve different content under different names. – Ulrich Schwarz Nov 16 '15 at 5:28
2

I resolved this by using the option --no-dns-cache. When this option was given, it started listing the IP address 54.240.168.41 in the list of IPs it was trying to connect to.

wget --no-dns-cache --continue http://public-repo-1.hortonworks.com/HDP/centos6/2.x/updates/2.3.2.0/HDP-2.3.2.0-centos6-rpm.tar.gz
| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.