Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

I can access a web page just fine by directly hitting my web server as follows:

$ echo "GET /sample" | nc web-server 80
This is contents of /sample...
$

Now, I would like netcat to go via a Squid HTTP proxy (listening on port 3128), much like I can configure my Firefox browser via its proxy preferences and have it go via an HTTP proxy.

I tried the following, but it did not work:

$ echo "GET /sample" | nc -x squid-proxy:3128 web-server 80
    <Seemed to be blocked FOREVER on input, so I killed it.>
<Ctrl-C>
$

Note: I'm using RHEL 5.3 version of netcat that has the following options:

$ nc --help
nc: invalid option -- -
usage: nc [-46DdhklnrStUuvzC] [-i interval] [-p source_port]
  [-s source_ip_address] [-T ToS] [-w timeout] [-X proxy_version]
  [-x proxy_address[:port]] [hostname] [port[s]]

Excerpt from the man page of nc:

 EXAMPLES
    <snip>
 Connect to port 42 of host.example.com via an HTTP proxy at 10.2.3.4, port 8080. 
 This example could also be used by ssh(1); see the ProxyCommand directive in
 ssh_config(5) for more information.
       $ nc -x10.2.3.4:8080 -Xconnect host.example.com 42

Now, because mine is not an ssh/SSL usecase, I'm not sure how to use the -x / -X options, or even whether I should be using them at all!

If there's more than one way to achieve the above goal (namely, routing netcat traffic via an HTTP proxy), then I would greatly appreciate if you could share them all.

Many thanks in advance.

share|improve this question
add comment

2 Answers

up vote 6 down vote accepted

Netcat is not a specialized HTTP client. Connecting through a proxy server for Netcat thus means creating a TCP connection through the server, which is why it expects a SOCKS or HTTPS proxy with the -x argument, specified by -X:

 -X proxy_protocol
         Requests that nc should use the specified protocol when talking
         to the proxy server.  Supported protocols are “4” (SOCKS v.4),
         “5” (SOCKS v.5) and “connect” (HTTPS proxy).  If the protocol is
         not specified, SOCKS version 5 is used.

connect specifies a method for creating SSL (HTTPS) connections through a proxy server. Since the proxy is not the other end point and the connection is endpoint-wise encrypted, a CONNECT request allows you to tunnel a point-to-point connection through an HTTP Proxy (if it is allowed). (I might be glossing over details here, but it's not the important point anyway; details on "HTTP CONNECT tunneling" here)

So, to connect to your webserver using a proxy, you'll have to do what the web browser would do - talk to the proxy:

$ nc squid-proxy 3128
GET http://webserver/sample HTTP/1.0

(That question has similarities to this one; I don't know if proxychain is of use here.)

Addendum A browser using an ordinary HTTP proxy, e.g. Squid (as I know it), does what more or less what the example illustrated, as Netcat can show you: after the nc call, I configured Firefox to use 127.0.0.1 port 8080 as proxy and tried to open google, this is what was output (minus a cookie):

$ nc -l 8080
GET http://google.com/ HTTP/1.1
Host: google.com
User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:6.0.2) Gecko/20100101 Firefox/6.0.2
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip, deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
DNT: 1
Proxy-Connection: keep-alive

By behaving this way, too, you can use Netcat to access a HTTP server through the HTTP proxy. Now, what should happen if you try to access a HTTPS webserver? The browser surely should not reveal the traffic to anyone in the middle, so a direct connection is needed; and this is where CONNECT comes into play. When I again start nc -l 8080 and try to access, say, https://google.com with the proxy set to 127.0.0.1:80, this is what comes out:

CONNECT google.com:443 HTTP/1.1
User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:6.0.2) Gecko/20100101 Firefox/6.0.2
Proxy-Connection: keep-alive
Host: google.com

You see, the CONNECT requests asks the server for a direct connection to google.com, port 443 (https). Now, what does this request do?

$ nc -X connect -x 127.0.0.1:8080 google.com 443

The output from the nc -l 8080 instance:

CONNECT google.com:443 HTTP/1.0

So it uses the same way to create a direct connection. However, as this can of course be exploited for almost anything (using for example corkscrew), CONNECT requests are usually restricted to the obvious ports only.

share|improve this answer
    
@sr_ Could you elaborate just a little bit more on how a browser might be achieving the ... TCP connection through the server part? Just like socks4, socks5, and connect, it seems there is a protocol for plain HTTP proxying as well. Would you please confirm this? Many thanks. –  Harry May 16 '12 at 12:49
1  
@BruceEdiger, yes, indeed. I subtracted 0.1 to mask my ignorance. :) –  sr_ May 16 '12 at 12:51
1  
@Harry Does my playing around with nc and Firefox clarify things a little? –  sr_ May 16 '12 at 13:19
    
@sr_ I tried nc squid-proxy 3128 followed by GET http://webserver/sample HTTP/1.0, but got an HTTP/1.0 403 Forbidden (Access denied error). –  Harry May 16 '12 at 13:42
1  
Ugh, sorry, no idea. Not an expert here, you see :) You can probably use tcpdump or wireshark to tap your own CONNECT connection and gain insights... –  sr_ May 16 '12 at 13:54
show 3 more comments

Take a look at socat: http://www.dest-unreach.org/socat/doc/README

share|improve this answer
    
Thanks, +1. Looks like a VERY sophisticated and complex tool. Tried echo "GET /sample HTTP/1.0" | socat PROXY:squid-proxy:web-server:80,proxyport=3128 STDIN, but got this error: 2012/05/17 06:56:23 socat[3135] E CONNECT web-server:80: Forbidden. Now, just to see if I understood this tool right, I next tried the basic, non-proxy usage: echo "GET /sample HTTP/1.0" | socat TCP4:web-server:80 STDIN, but got no response back! So, obviously, I'm missing something here in the socat usage. Would appreciate if you could point out my mistake. –  Harry May 17 '12 at 1:32
    
Ok, I could get the non-proxy version invocation to work by using STDIO (or, -) instead of STDIN. But the proxy version is still giving me the same error. –  Harry May 17 '12 at 2:40
    
Another update: I tried echo "GET http://web-server/sample" | socat - TCP:squid-proxy:3128 and it worked. Would you confirm if this is how I am supposed to use socat (instead of the the PROXY:... address specification)? –  Harry May 17 '12 at 3:35
1  
Hi, i haven't used it so much myself, but i think it's correct. Socat opens a plain TCP-Connection to the http-proxy and squid parses the http-GET and does the rest. Does it work for https btw.? And finally another link: technostuff.blogspot.com/2008/10/… –  Fabian Zeindl May 17 '12 at 13:08
    
No, I haven't gotten to trying https yet. I will soon be worrying about https as well. Thanks. –  Harry May 18 '12 at 3:13
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.