Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

For debugging purposes I want to monitor the http requests on a network interface.

Using a naive tcpdump command line I get too much low-level information and the information I need is not very clearly represented.

Dumping the traffic via tcpdump to a file and then using wireshark has the disadvantage that it is not on-the-fly.

I imagine a tool usage like this:

$ monitorhttp -ieth0 --only-get --just-urls
2011-01-23 20:00:01 GET http://foo.example.org/blah.js
2011-01-23 20:03:01 GET http://foo.example.org/bar.html

I am using Linux.

share|improve this question
There is same question answered on superuser.com/questions/67428/possible-to-catch-urls-in-linux – AlexD Jan 23 '11 at 9:32
up vote 34 down vote accepted

Try tcpflow:

tcpflow -p -c -i eth0 port 80 | grep -oE '(GET|POST|HEAD) .* HTTP/1.[01]|Host: .*'

Output is like this:

GET /search?q=stack+exchange&btnI=I%27m+Feeling+Lucky HTTP/1.1
Host: www.google.com

You can obviously add additional HTTP methods to the grep statement, and use sed to combine the two lines into a full URL.

share|improve this answer
An advantage of tcpflow is that it is already available in the default repositories in Ubuntu 10.04 (justsniffer, httpry are not). The package info states that IP fragments are not recorded properly - don't know, if this matters for this use case - perhaps justsniffer can handle them better. – maxschlepzig Jan 22 '11 at 23:11
Since you're just grabbing the URL it doesn't seem like it'll matter. Tcpflow will display packets in the order they were received on the interface. Thus, if you were trying to capture file contents you can get packets that arrive out of order and will produce a corrupt file. But your use case listed in the question I think this will work for you. You can also widen your grep (or remove the -o) to see more of the packet data for sorting or whatnot later. – bahamat Jan 23 '11 at 0:01

You can use httpry or Justniffer to do that.

httpry is available e.g. via the Fedora package repository.

Example call:

# httpry -i em1

(where em1 denotes an network interface name)

Example output:

2013-09-30 21:35:20    >    POST    unix.stackexchange.com    /posts/6281/editor-heartbeat/edit    HTTP/1.1
2013-09-30 21:35:20       < HTTP/1.1   200    OK
2013-09-30 21:35:49    >    POST    unix.stackexchange.com    /posts/validate-body                 HTTP/1.1
2013-09-30 21:35:49       < HTTP/1.1   200    OK
2013-09-30 21:33:33    >    GET     cdn4.spiegel.de    /images/image-551203-breitwandaufmacher-fgoe.jpg    HTTP/1.1

(output is a little bit shortened)

share|improve this answer
+1 for httpry. Hadn't heard about it. – 0xC0000022L Mar 11 '11 at 14:11

I was looking for something similar, with the added requirement that it should work for https too.

pcap based tools like tcpflow httpry urlsnarf and other tcpdump kung fu work well for http, but for secure requests you're out of luck.

I came up with urldump, which is a small wrapper around mitmproxy.
iptables is used to redirect traffic to the proxy, so it works transparently.

$ sudo urldump   

See README for more info.

share|improve this answer

Another good option might be nethogs

On fedora is available among the core packages, and on centos you can get it through the epel repo.

share|improve this answer

I think Wireshark is capable of doing what you want

On the plus side, it's very powerful, you can install it via apt-get, and it comes with a GUI.

However, the filter system is complicated - but there are good tutorials built in, and it will give you a live or start/stop overview of the traffic.

Typing the word 'http' into the filter will probably give you what you are looking for (i.e. the main traffic generated by users).

share|improve this answer
Would like to know why this was downvoted. Wireshark can read the interface on the fly and filter to just http traffic. – Kevin M Jan 22 '11 at 17:10
@Kevin M, Well, I did not downvote your answer. But to be fair your answer is a bit incomplete and off-topic. 1) It misses details on how exactly wireshark should be used, i.e. that a filter should be used, the exact filter expression, etc. 2) it does not allow for command line usage like sketched in the question - even if I am ok with the GUI approach, the default view displays GET requests, where the domain name is not displayed side by side - with is not that convenient for the sketched use case. – maxschlepzig Jan 22 '11 at 18:18
I mean: s/your answer/Phobia's answer/ – maxschlepzig Jan 22 '11 at 18:27

There is also the command line program urlsnarf which is part of the dsniff package (which is also packaged with e.g. Fedora 19).


# urlsnarf -i em1
urlsnarf: listening on em1 [tcp port 80 or port 8080 or port 3128]
jhost - - [29/May/2014:10:25:09 +0200] "GET http://unix.stackexchange.com/questions HTTP/1.1" - - "-" "Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 Firefox/24.0"
jhost - - [29/May/2014:10:25:36 +0200] "GET http://www.spiegel.de/ HTTP/1.1" - - "-" "Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 Firefox/24.0"
jhost - - [29/May/2014:10:25:36 +0200] "GET http://www.spiegel.de/layout/css/style-V5-2-2.css HTTP/1.1" - - "-" "Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 Firefox/24.0"
jhost - - [29/May/2014:10:25:36 +0200] "GET http://www.spiegel.de/layout/jscfg/http/global-V5-2-2.js HTTP/1.1" - - "-" "Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 Firefox/24.0"
jhost - - [29/May/2014:10:25:36 +0200] "GET http://www.spiegel.de/layout/js/http/javascript-V5-2-2.js HTTP/1.1" - - "-" "Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 Firefox/24.0"
jhost - - [29/May/2014:10:25:36 +0200] "GET http://www.spiegel.de/layout/js/http/interface-V5-2-2.js HTTP/1.1" - - "-" "Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 Firefox/24.0"
jhost - - [29/May/2014:10:25:36 +0200] "GET http://www.spiegel.de/layout/js/http/netmind-V5-2-2.js HTTP/1.1" - - "-" "Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 Firefox/24.0"
jhost - - [29/May/2014:10:25:36 +0200] "GET http://www.spiegel.de/favicon.ico HTTP/1.1" - - "-" "Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 Firefox/24.0"
jhost - - [29/May/2014:10:25:36 +0200] "POST http://ocsp.thawte.com/ HTTP/1.1" - - "-" "Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 Firefox/24.0"
jhost - - [29/May/2014:10:25:36 +0200] "POST http://ocsp.thawte.com/ HTTP/1.1" - - "-" "Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 Firefox/24.0

(when browsing first to SE and then to spiegel.de)

Limitations: dsnarf does not support IPv6. I can reproduce this bug report with 0.17 on Fedora 19. Also seems to be broken under Ubuntu trusty atm (works fine under lucid).

share|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.