64

What would be the most straightforward way of making a GET request to a url over HTTPS, and getting the raw, unparsed response?

Could this be achieved with curl? If so, what options would you need to use?

4 Answers 4

83

If you want to use curl, this should work:

curl -D - https://www.google.com/

Note, however, that this is not exactly the raw response. For instance chunked transfer encoding will not be visible in the response. Using --raw solves this, also verbose mode (-v) is useful, too and -i shows the headers before the response body:

curl -iv --raw https://www.google.com/

If you want to use a pager like less on the result, it is also necessary to disable the progress-bar (-s):

curl -ivs --raw https://www.google.com/ | less

Depending on what you want to do this may or may not be a problem.

What you do get is all HTTP response headers and the document at the requested URL.

0
17

Here's a simple way that comes to mind

echo 'GET / HTTP/1.1
Host: google.com

' | openssl s_client -quiet -connect google.com:443 2>/dev/null
2
7

It's not curl, but it should be available on almost all Unices:

wget -S --spider https://encrypted.site

If the status messages bother you:

wget -S --spider https://encrypted.site 2>&1 | awk '/^  /'

If you want CRLF line endings:

wget -S --spider https://encrypted.site 2>&1 | awk '/^  / { sub(/$/,"\r"); print }'
3
  • Do you know if this would preserve the \r\n characters in the response?
    – Acorn
    Jan 19, 2012 at 0:04
  • @Acorn - First of all, not all webservers will respond with \r\n as a line ending. I don't believe this will preserve it in any case, but if that matters to you, I'll add a way of getting that result in the answer.
    – Chris Down
    Jan 19, 2012 at 8:05
  • any idea how to parse the response headers in a { json: "object" } ? Mar 13, 2022 at 6:58
3
$ GET -e https://www.google.com

On Debian/Ubuntu distros belongs to the package lwp-request.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .