How to access http server from bash script with existing tcp connection?

Question

In a bash shell scipt, I use several command line tools (wget, curl, httpie) to test my http server.

When using e.g. curl to invoke a GET request, I see that a tcp connection is opened to my server and closed immediatly after the http communication has been completed.

$ curl http://10.5.1.1/favicon.ico -o /dev/null 

For better testing the keep-alive behaviour of my server, I want to keep the tcp connection open over multiple http request / response cycles.

I am able to perform poor http requests directly from bash like this:

exec 3<>/dev/tcp/10.5.1.1/80
printf "GET /\r\n\r\n" >&3
while IFS= read -r -u3 -t2 line || [[ -n "$line" ]]; do echo "$line"; done
exec 3>&-
exec 3<&-

But because my testing communication is much more complex (incl. authentication et cetera), I don't want to code this completely myself in bash or other scripting language but want to use existing tools instead. The idea is now to first open a tcp socket in bash and then forcing some command line tools to use this connection instead of open their own sockets. Maybe like this;

exec 3<>/dev/tcp/10.5.1.1/80
curl http://10.5.1.1/message.txt --use-existing-socket-fd=3
# ... do some other testing stuff ...
wget http://10.5.1.1/message.txt --use-existing-socket-fd=3
# ... do some other testing stuff ...
http http://10.5.1.1/message.txt --use-existing-socket-fd=3
exec 3>&-
exec 3<&-

Might this be possible using the above mentioned tools or with other tools?

Answer 1

The tool you are looking for is socat. Since you are testing one single web server, you can ask it to establish a permanent connection to that server (as long as it doesn't choose to close it -- adjust your timeouts accordingly) and once this is done, you can use the connection as a tunnel. Below are two ways to accomplish this.

Querying through a Unix socket

curl has a --unix-socket option that allows you to send HTTP requests and receive HTTP replies through a Unix socket (thank you, thrig, for your enlightening comment)

You would use it like this:

socat TCP:10.5.1.1:80 UNIX-LISTEN:/tmp/curl.sock,fork

Then, on another terminal:

curl --unix-socket /tmp/curl.sock http://10.5.1.1/message1.txt
curl --unix-socket /tmp/curl.sock http://10.5.1.1/message2.txt
...

Querying through a pseudo-HTTP-proxy

You can also make your tunnel available as a pseudo-proxy through which wget, curl,... would connect. This solution has the advantage to not be limited to curl.

This time, socat listens to a local TCP port (say 3128):

socat TCP:10.5.1.1:80 TCP-LISTEN:3128,fork,reuseaddr

Then, on another terminal:

export http_proxy='http://localhost:3128'
curl http://10.5.1.1/message.txt
wget http://10.5.1.1/message.txt
....

Note that since the HTTP client is using a proxy, the HTTP request will be slightly altered and this may not be desirable.

Of course, none of those two solutions are intended to be used with multiple HTTP servers since this is your web-server at the end of the channel that receives all the requests.