Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

Can anybody tell me why I am getting bad request while executing this command

echo -e "GET http://www.yellowpages.com.eg/Mjg3NF9VUkxfMTEwX2h0dHA6Ly93d3cubG90dXMtYWlyLmNvbV8=/Lotus-Air/profile.html HTTP/1.1\n\n" | nc www.yellowpages.com 80 

The same web site opens fine in the browser.

share|improve this question
    
Your example domains are inconsistent. You are telling nc to use a .com and sending a get request for .com.eg. Typo? –  Caleb Jun 29 '11 at 10:28
    
Your example URL does NOT work in a browser for me, it gets forwarded somewhere else. What result do you expect back? –  Caleb Jun 29 '11 at 10:31
    
@Caleb : I was expecting 302 instead of 400 –  Madhur Ahuja Jun 29 '11 at 14:57
add comment

3 Answers

up vote 4 down vote accepted

The headers in an HTTP request must use CRLF (Windows) line endings. (See Wikipedia or RFC 2616.) Many servers support LF (Unix) line endings as an extension, but not this one.

In addition, HTTP 1.1 requires a Host: header line, as Warren Young pointed out. (See Wikipedia or RFC 2616).

echo -e "GET http://www.yellowpages.com.eg/Mjg3NF9VUkxfMTEwX2h0dHA6Ly93d3cubG90dXMtYWlyLmNvbV8=/Lotus-Air/profile.html HTTP/1.1\r\nHost: www.yellowpages.com.eg\r\n\r\n" | nc www.yellowpages.com 80

or more legibly

sed $'s/$/\r/' <<EOF | nc www.yellowpages.com 80
GET http://www.yellowpages.com.eg/Mjg3NF9VUkxfMTEwX2h0dHA6Ly93d3cubG90dXMtYWlyLmNvbV8=/Lotus-Air/profile.html HTTP/1.1
Host: www.yellowpages.com.eg

EOF

But why not use wget or curl, which will construct a valid request without sweating and still allow you to specify custom headers if necessary?

share|improve this answer
add comment

HTTP 1.1 requires that you send at least a Host header in GET requests. That is, the minimum legal request looks like this:

GET http://www.example.com/noise/and/junk HTTP/1.1
Host: www.example.com

(Plus an additional CRLF to terminate the header section, of course.)

There may be HTTP servers that will cope with a request claiming to require HTTP 1.1 but that doesn't include a Host header, but your server is correct to reject such a request.

Host is an HTTP 1.1 extension which was needed to support name-based virtual hosting. If the site you're trying to access has dedicated servers (or at least, a dedicated IP), you can safely drop back to HTTP 1.0, which lets you make a single-line HTTP request:

GET http://www.example.com/noise/and/junk HTTP/1.0
share|improve this answer
add comment

You need to include the domain name in your GET request. You have told nc the domain name you are connecting to do it knows where to go find the server, but nc doesn't pass that on to the server. If the server is hosting multiple domains, it will not know which one to send you. The request header you are passing with echo should include this full domain like this:

echo "GET http://domain.tld/path" | nc domain.tld 80

Note that you can also drop the -e argument to your echo and the escaped newlines at the end. The -e is suppressing echo's natural tendency to add a newline, then you are adding one yourself.

Edit 1: Is there some reason you are not using a normal download tool like curl that can handle all the header possibilities and give you useful output? Do you really need to handle the header chat yourself? curl http://domain.tld/path should give you much more reliable output because the programmers have already worked through all the possibilities for you.

Edit 2: See Warren's answer for information about the protocol specification. TL;DR: If you specify 1.1, you have to then comply with that protocal. If you pecify 1.0, you can usually make the reqest as above.

The make a requestion with echo and netcat using HTTP/1.1, try this:

echo "GET http://domain.tld/path HTTP/1.1\nHost: domain.tld\n" | nc domain.tld 80
share|improve this answer
    
I did even then the same error –  Madhur Ahuja Jun 29 '11 at 10:11
    
echo -e "GET yellowpages.com.eg/… HTTP/1.1\n\n" | nc www.yellowpages.com 80 –  Madhur Ahuja Jun 29 '11 at 10:13
1  
Your examples aren't full things I can copy and paste to test, but this does work. Try this example EXACTLY: echo "GET http://www.google.com.tr/" | nc www.google.com.tr 80 –  Caleb Jun 29 '11 at 10:24
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.