Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

I have a server in USA (Linux box B), and my home PC (Linux box A), and I need download a file from website C,

The issue is, it is very slow to download a file direct from A, so I need download the file when I log in B, and sftp get the file from A.

Is there any way that I can download file and use B as proxy directly through only one line command?

share|improve this question
add comment

5 Answers 5

up vote 16 down vote accepted

(Strange situation, doesn't something like the triangle inequality hold for internet routing?)

Anyway, try the following, on A, ssh into B with a -D argument,

ssh -D 1080 address-of-B

which acts as a SOCKS5 proxy on 127.0.0.1:1080, which can be used by anything supporting SOCKS5 proxied connections. Apparently, wget can do this, by using the environment variable

export SOCKS_SERVER=127.0.0.1:1080
wget http://server-C/whatever

Note that sometimes curl is more handy (i.e. I'm not sure if wget can do hostname lookups via SOCKS5; but this is not one of your concerns I suppose); also Firefox is able to work completely through such a SOCKS5 proxy.

Edit I've just now noticed that you're looking for a one-line solution. Well, how about

ssh address-of-B 'wget -O - http://server-C/whatever' >> whatever

i.e. redirection the wget-fetched output to stdout, and redirecting the local output (from ssh running wget remotely) to a file.

This seems to work, the wget output is just a little confusing ("saved to -"), you can get rid of it by adding -q to the wget call.

share|improve this answer
add comment

Another approach could be that you normally log in into B, where you start a screen session. There you do the wget of your files - all into one directory.

And there the program can happily run; you just detach from screen, but let it run in the background.

If the downloads are finished (maybe even earlier), you can fech the data from B to A using rsync (my preference).

share|improve this answer
add comment

You can do a ssh tunnel from box A to box B and add to the routing table in box A, that website C is reachable via tunnel to box B. You have to allow packet forwarding on the box B.

Here you can see a very good step-by-step tutorial...

share|improve this answer
add comment

You would need to create a tunnel on machine B tha would redirect the call to website C. But I'm puzzled as why this would be faster, unless your ISP as some restrictions.

I don't know a oneliner, but this isn't much more complicated.

On machine A, you do (I took 11111 randomly, you can take whatever you want as long as it is > 1024, or you would need to be root)

ssh -f -C -N -L 11111:C:80 username@B

The username on B is the one you use to connect to B. This should create a tunnel on port 11111 on machine B that redirect to port 80 (web site in HTTP use 443 for HTTPS) on machine C (I hope I did not mess the order ;) )

Then you can download the file directly from machine A via machine B. I'm assuming the file is at http://C/path/to/file so you would then use:

wget http://B:11111/path/to/file
share|improve this answer
    
Wouldn't this work only if the file is accessible without HTTP 1.1 name-based virtual hosting? Since the GET request issued by wget would state the hostname as B, not C. –  Michael Kjörling May 16 '12 at 11:39
    
@MichaelKjörling I don't have enough knowledge to answer you. –  Huygens May 21 '12 at 7:36
add comment

You can do this via port forwarding (ssh tunneling). Here's a resource: http://www.jfranken.de/homepages/johannes/vortraege/ssh2_inhalt.en.html#ToC9

Essentially, you should set up port forwarding on B. When A issues wget to B, B will forward the packets to C and send the results back to A.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.