44

I am looking for a command line tool that listens on a given part, happily excepts every HTTP POST request and dumps it.

I want to use it for testing purposes, i.e. for testing clients that issue HTTP POST requests.

That means I am searching the counterpart to curl -F (which I can use to send test HTTP POSTs to a HTTP server).

Perhaps something like socat TCP4-LISTEN:80,fork,bind=127.0.0.1 ... - but socat is not enough because it does not talk HTTP.

3
  • 3
    Maybe I don't understand the question correctly, but if all you need is to dump the POST request, you can use netcat (nc on some systems) with the -l (listen) and -p (port number) options.
    – peterph
    Dec 8, 2012 at 0:23
  • 2
    @peterph, you can use nc for partial testing - but I can see following disadvantages: 1) it does not send HTTP status code 2) I have to hit Ctrl+D after I see the request to close the connection 3) it does not know how to react upon then 'Expect: 100-continue' header 4) it does not know how to handle the 'Transfer-Encoding: chunked' header - it probably just displays the first (probably) empty chunk Dec 8, 2012 at 0:43
  • Similar question on stackoverflow: stackoverflow.com/questions/5725430/… Jul 22, 2015 at 18:33

5 Answers 5

71

I was looking for this myself as well and ran into the Node.js http-echo-server:

npm install http-echo-server -g
PORT=8081 http-echo-server

It accepts all requests and echos the full request including header to the command-line.

0
17

Simple core command line tools like nc, socat seem not to be able to handle the specific HTTP stuff going on (chunks, transfer encodings, etc.). As a result this may produce unexpected behaviour compared to talking to a real web server. So, my first thought is to share the quickest way I know of setting up a tiny web server and making it just do what you want: dump all output.

The shortest I could come up with using Python Tornado:

#!/usr/bin/env python

import tornado.ioloop
import tornado.web
import pprint

class MyDumpHandler(tornado.web.RequestHandler):
    def post(self):
        pprint.pprint(self.request)
        pprint.pprint(self.request.body)

if __name__ == "__main__":
    tornado.web.Application([(r"/.*", MyDumpHandler),]).listen(8080)
    tornado.ioloop.IOLoop.instance().start()

Replace the pprint line to output only the specific fields you need, for example self.request.body or self.request.headers. In the example above it listens on port 8080, on all interfaces.

Alternatives to this are plenty. web.py, Bottle, etc.

(I'm quite Python oriented, sorry)


If you don't like its way of outputting, just run it anyway and try tcpdump like this:

tcpdump -i lo 'tcp[32:4] = 0x484f535420'

to see a real raw dump of all HTTP-POST requests. Alternatively, just run Wireshark.

2
  • 2
    For others who find this very helpful snippet - it does what was asked - but if you want to see the POST body, it's pprint.pprint(self.request.body). Note self.request.body rather than self.body. Same for self.request.headers. See tornado.readthedocs.org/en/latest/…
    – mozz100
    Sep 25, 2014 at 10:17
  • 1
    FWIW, even without tornado, with just the Python standard library, such a minimal dumper would be just 3 lines or so longer. See also my answer - it supports more request types, json dumping and command line parsing, thus, it's a bit longer - but when restricting it to raw POST requests like this you just need to explicitly read the request body and send a status code, as additional work. Jul 11, 2022 at 12:05
5

https://hub.docker.com/r/jmalloc/echo-server/

Run

$ docker run -t --rm -p 8080:8080 jmalloc/echo-server
Unable to find image 'jmalloc/echo-server:latest' locally
latest: Pulling from jmalloc/echo-server
fbf67b0844fa: Pull complete
Digest: sha256:617a99b927c3b761621681eb4716582260391c0853b6da904e0f9f1d37785e7a
Status: Downloaded newer image for jmalloc/echo-server:latest
Echo server listening on port 8080

Post

$ curl -XPOST -H"ThisTook: 2 minutes to find" localhost:8080/asdf
Request served by a2d8fa109b92

HTTP/1.1 POST /asdf

Host: localhost:8080
User-Agent: curl/7.54.0
Accept: */*
Thistook: 2 minutes to find
10
  • 1
    That docker container just contains a single binary, so you can also docker cp <cid>:/bin/echo-server . to grab the binary. Oct 22, 2020 at 17:09
  • 1
    And do what with it.. and why? Nov 15, 2020 at 7:24
  • 1
    maybe i am misunderstanding intent here - you want to cp a binary from a docker instance/container and run it directly on the host? As opposed to running it in the container, which is a controlled environment that satisfies any/all needed dependencies of said binary? Nov 17, 2020 at 20:42
  • 2
    It's a go binary, there are no external dependencies, it's statically linked like virtually all go binaries. Apr 9, 2021 at 20:39
  • 1
    That's cool, to bring up months later and for the first time; statically linked, or not, doesn't mean "works on alpine" or "works on anything that it wasn't specifically compiled against (see link)". The reason I am being a dick is because wtf are you pulling a binary out of a container, as opposed to just running the binary in the gd container? Dockerd is universally available, bulletproofed ad nauseam, etc etc. gist.github.com/asukakenji/f15ba7e588ac42795f421b48b8aede63 Jun 1, 2021 at 23:55
5

Use nc (pronounced "netcat").

You tell it which port to listen on

nc -kl 8888

Then in a separate terminal window send a request to it

curl localhost:8888 -d hello=world

and it will print the data you sent in to it, in this case an HTTP request:

POST / HTTP/1.1
Host: localhost:8888
User-Agent: curl/7.84.0
Accept: */*
Content-Length: 11
Content-Type: application/x-www-form-urlencoded

hello=world

The -k option means it will listen for requests and print them indefinitely. Without that option (i.e. nc -l 8888), it will exit after the first request.

1
  • 2
    You nc command doesn't speak HTTP. Thus, an HTTP client (here the curl command) blocks on waiting for the HTTP-server's reply (such as some common status code) that never comes. Sep 5, 2022 at 19:19
4

The Python standard library comes with batteries included, i.e. it even includes an HTTP server package that can be used to write a simple HTTP request dumper:

#!/usr/bin/env python3

import argparse
import http.server
import json
import sys

class Dumper(http.server.BaseHTTPRequestHandler):
    def do_GET(self, method='GET'):
        print(f'\n{method} {self.path}\n{self.headers}')
        self.send_response(200)
        self.end_headers()

    def do_DELETE(self):
        return self.do_GET('DELETE')

    def do_POST(self, method='POST'):
        n = int(self.headers.get('content-length', 0))
        body = self.rfile.read(n)
        print(f'\n{method} {self.path}\n{self.headers}{body}\n')
        if self.headers.get('content-type') == 'application/json':
            d = json.loads(body)
            print(json.dumps(d, indent=4, sort_keys=True))
            print()
        self.send_response(200)
        self.end_headers()

    def do_PUT(self):
        return self.do_POST('PUT')

    def log_message(self, format, *args):
        pass

Some boilerplate:


def main():
    p = argparse.ArgumentParser(description='Dump HTTP requests to stdout')
    p.add_argument('address', help='bind address')
    p.add_argument('port', type=int, help='bind port')
    xs = p.parse_args();
    s = http.server.HTTPServer((xs.address, xs.port), Dumper)
    s.serve_forever()

if __name__ == '__main__':
    sys.exit(main())

See also: my gist

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .