Hot answers tagged

166

Try SimpleHTTPServer: python -m SimpleHTTPServer # or the Python 3 equivalent python3 -m http.server It should will serve whatever's in the CWD (e.g. index.html) at http://0.0.0.0:8000.


66

I was looking for a way to run a program with modified DNS resolution for testing purposes. For me, the solution was using the HOSTALIASES environment variable: $ echo "foo www.google.com" >> ~/.hosts $ HOSTALIASES=~/.hosts wget foo See hostname(7). (Side note: In the example the HOSTALIASES environment variable only affects the wget process. Of ...


45

There is a Big list of http static server one-liners: To get on this list, a solution must: serve static files using your current directory (or a specified directory) as the server root be able to be run with a single, one line command (dependencies are fine if they're a one-time thing) serve basic file types (html, css, js, images) with ...


44

Use node.js , fast and lightweight. Or just use simple nc netcat command to start a quick webserver on a port and serve the content of a file including the server response headers. Reference from Wikipedia: http://en.wikipedia.org/wiki/Netcat#Setting_up_a_one-shot_webserver_on_port_8080_to_present_the_content_of_a_file { echo -ne "HTTP/1.0 200 OK\r\n\...


44

wget -qO- 'http://www.youtube.com/watch?v=Dd7dQh8u4Hc' | perl -l -0777 -ne 'print $1 if /<title.*?>\s*(.*?)\s*<\/title/si' You can pipe it to GNU recode if there are things like &lt; in it: wget -qO- 'http://www.youtube.com/watch?v=Dd7dQh8u4Hc' | perl -l -0777 -ne 'print $1 if /<title.*?>\s*(.*?)\s*<\/title/si' | recode html.. ...


40

I was looking for this myself as well and ran into the Node.js http-echo-server: npm install http-echo-server -g PORT=8081 http-echo-server It accepts all requests and echos the full request including header to the command-line.


34

You can use wget over ssh to open an HTTP connection and output stream data, then mplayer on the local machine to play it. ssh <user>@<host> "wget <stream> -O -" | mplayer -cache 512 - The subtleties: wget's -O (capital letter oh) option sets its output file; - (dash) implies stdout. mplayer's -cache option causes a configurable amount ...


33

lftp (Wikipedia) is good for that. It supports a number of protocols, can download files using several concurrent parallel connections (useful where there's a lot of packet loss not caused by congestion), and can automatically resume downloads. It's also scriptable. Here including the fine-tuning you came up with (credits to you): lftp -c 'set net:idle 10 ...


32

Yes, nweb. Can be found here: nweb.c (previously at ibm.com) To compile nweb.c: gcc -O -DLINUX nweb.c -o nweb


30

Since version 5.4.0 PHP also has a built-in web server: php -S localhost:8000 You can Specify the web server's documents directory with -t, for example: php -S localhost:8000 -t /var/lib/www If you want to be able to access the server over the network then: php -S 0.0.0.0:8000 -t /var/lib/www


28

This is a gzip compressed file. You can find this out by running the file command, which figures out the file format from magic numbers in the data (this is how programs such as Text Wrangler figure out that the file is compressed as well): file output.html wget -O - … | file - The server (I guessed it from the content you showed) is sending gzipped data ...


27

You can also try hxselect (from HTML-XML-Utils) with wget as follows: wget -qO- 'http://www.youtube.com/watch?v=Dd7dQh8u4Hc' | hxselect -s '\n' -c 'title' 2>/dev/null You can install hxselect in Debian based distros using: sudo apt-get install html-xml-utils. STDERR redirection is to avoid the Input is not well-formed. (Maybe try normalize?) message. ...


25

mod_security is great, but you don't really need it to achieve your goal. after all mods have been included in httpd.conf you can simply unset the headers of your choosing. Header unset Server ServerSignature Off ServerTokens Prod http://httpd.apache.org/docs/2.4/mod/core.html#serversignature


25

Congratulations, you've just delved into the concept of networking layers by realizing that ports and protocols are not directly connected with each other. As others are saying, telnet can be used to connect to any TCP port. However to understand why this is possible you need to understand a bit about networking layers. If you've ever heard of the OSI 7 ...


20

Try mitmproxy. mitmproxy is an SSL-capable man-in-the-middle proxy for HTTP. It provides a console interface that allows traffic flows to be inspected and edited on the fly. mitmdump is the command-line version of mitmproxy, with the same functionality but without the user interface. Think tcpdump for HTTP. Features Intercept HTTP requests and responses ...


20

You might be able to use curl's --fail option, though you should test it once first. man curl -f, --fail (HTTP) Fail silently (no output at all) on server errors. This is mostly done to better enable scripts etc to better deal with failed attempts. In normal cases when an HTTP server fails to ...


19

You can use ssh's -L flag to create a tunnel from your system to a remote: ssh -L <localport>:<streamhost>:<streamport> <user>@<remote> You can then connect to your end of the tunnel with mplayer as you would otherwise: mplayer localhost:<localport>


19

Use telnet domain.name.server.com 80 then HEAD /~USER HTTP/1.1 Host: domain.name.server.com (Then you have to hit Enter once more.) Now it should show you the header of this page. For a real life example: $ telnet unix.stackexchange.com 80 Trying 198.252.206.16... Connected to unix.stackexchange.com. Escape character is '^]'. HEAD /...


18

You can also use curl and grep to do this. You'll need to enlist the use of PCRE (Perl Compatible Regular Expressions) in grep to get the look behind and look ahead facilities so that we can find the <title>...</title> tags. Example $ curl 'http://www.youtube.com/watch?v=Dd7dQh8u4Hc' -so - | \ grep -iPo '(?<=<title>)(.*)(?=</...


16

Simple core command line tools like nc, socat seem not to be able to handle the specific HTTP stuff going on (chunks, transfer encodings, etc.). As a result this may produce unexpected behaviour compared to talking to a real web server. So, my first thought is to share the quickest way I know of setting up a tiny web server and making it just do what you ...


15

Node has a simple, fast, light HTTP server module. To install: sudo npm install http-server -g (Assuming you have node and npm already installed.) To run it, using the current directory as the website root: http-server This creates a server on http://0.0.0.0:8080/.


15

wget is a great tool. Use wget -m http://somesite.com/directory -m --mirror Turn on options suitable for mirroring. This option turns on recursion and time-stamping, sets infinite recursion depth and keeps FTP directory listings. It is currently equivalent to -r -N -l inf --no-remove-listing.


15

This file is still compressed with gzip. You can see that the first two bytes 0x1f8b match the gzip signature. So to read the data you need to uncompress it. mv file.dat file.dat.gz gunzip file.dat.gz Or zcat file.dat


15

Try something like this in your Makefile: .PHONY: local.dat local.dat: [ -e example.gz ] || touch -d '00:00' example.gz curl -z example.gz -s http://example.org/example.gz -o example.gz [ -e $@ ] || touch -d 'yesterday 00:00' $@ if [ "$(shell stat --printf '%Y' example.gz)" \ -gt "$(shell stat --printf '%Y' $@)" ] ; ...


13

Just updating this for people who are still looking. I was having trouble getting the Server line in the HTTP header changed. This advice should work for Debian branch distros with systemd and Apache 2.4.7. Specifically, I am using Ubuntu Server LTS 14.04.03. Some advice I found was to do grep -Ri servertokens /etc/apache2 This led me to /etc/apache2/conf-...


12

The server ID/token header is controlled by "ServerTokens" directive (provided by mod_core). Aside from modifying the Apache HTTPD source code, or using mod_security module, there is no other way to fully suppress the server ID header. With the mod_security approach, you can disable all of the module's directives/functions in the modsecurity.conf file, and ...


12

I can't test this for you in your situation, but you should not be using --range with -C -. Here's what the man page has to say on the subject: Use -C - to tell curl to automatically find out where/how to resume the transfer. It then uses the given output/input files to figure that out. Try this instead: curl -s --retry 9999 --retry-delay 3 --speed-...


11

You can use lsof and watch to do this, like so: $ watch -n1 lsof -i TCP:80,443 Example output dropbox 3280 saml 23u IPv4 56015285 0t0 TCP greeneggs.qmetricstech.local:56003->snt-re3-6c.sjc.dropbox.com:http (ESTABLISHED) thunderbi 3306 saml 60u IPv4 56093767 0t0 TCP greeneggs.qmetricstech.local:34788->ord08s09-in-f20.1e100.net:...


10

RFC 1945 (1996) says 5.1.1 Method The Method token indicates the method to be performed on the resource identified by the Request-URI. The method is case-sensitive. Method = "GET" ; Section 8.1 | "HEAD" ; Section 8.2 | "POST" ; Section 8.3 ...


Only top voted, non community-wiki answers of a minimum length are eligible