Hot answers tagged

74

python -m SimpleHTTPServer should do it.


42

I was looking for a way to run a program with modified DNS resolution for testing purposes. For me, the solution was using the HOSTALIASES environment variable: $ echo "foo www.google.com" >> ~/.hosts $ HOSTALIASES=~/.hosts wget foo See hostname(7). (Side note: In the example the HOSTALIASES environment variable only affects the wget process. Of ...


37

wget -qO- 'http://www.youtube.com/watch?v=Dd7dQh8u4Hc' | perl -l -0777 -ne 'print $1 if /<title.*?>\s*(.*?)\s*<\/title/si' You can pipe it to GNU recode if there are things like &lt; in it: wget -qO- 'http://www.youtube.com/watch?v=Dd7dQh8u4Hc' | perl -l -0777 -ne 'print $1 if /<title.*?>\s*(.*?)\s*<\/title/si' | recode html.. ...


34

You can use wget over ssh to open an HTTP connection and output stream data, then mplayer on the local machine to play it. ssh <user>@<host> "wget <stream> -O -" | mplayer -cache 512 - The subtleties: wget's -O (capital letter oh) option sets its output file; - (dash) implies stdout. mplayer's -cache option causes a configurable amount ...


24

You can also try hxselect (from HTML-XML-Utils) with wget as follows: wget -qO- 'http://www.youtube.com/watch?v=Dd7dQh8u4Hc' | hxselect -s '\n' -c 'title' 2>/dev/null You can install hxselect in Debian based distros using: sudo apt-get install html-xml-utils. STDERR redirection is to avoid the Input is not well-formed. (Maybe try normalize?) message....


19

Use node.js , fast and lightweight. Or just use simple nc netcat command to start a quick webserver on a port and serve the content of a file including the server response headers. Reference from Wikipedia: http://en.wikipedia.org/wiki/Netcat#Setting_up_a_one-shot_webserver_on_port_8080_to_present_the_content_of_a_file { echo -ne "HTTP/1.0 200 OK\r\n\...


18

You can use ssh's -L flag to create a tunnel from your system to a remote: ssh -L <localport>:<streamhost>:<streamport> <user>@<remote> You can then connect to your end of the tunnel with mplayer as you would otherwise: mplayer localhost:<localport>


18

Use telnet domain.name.server.com 80 then HEAD /~USER HTTP/1.1 Host: domain.name.server.com (Then you have to hit Enter once more.) Now it should show you the header of this page. For a real life example: $ telnet unix.stackexchange.com 80 Trying 198.252.206.16... Connected to unix.stackexchange.com. Escape character is '^]'. HEAD /...


16

You have the symptoms of an MTU problem: some TCP connections freeze, more or less reproducibly for a given command or URL but with no easily discernible overall pattern. A telltale symptom is that interactive ssh sessions work well but file transfers almost always fail. Furthermore pppoe is the number one bringer of MTU problem for home users. So I ...


15

Since version 5.4.0 PHP also has a built-in web server: php -S localhost:8000


11

You can also use curl and grep to do this. You'll need to enlist the use of PCRE (Perl Compatible Regular Expressions) in grep to get the look behind and look ahead facilities so that we can find the <title>...</title> tags. Example $ curl 'http://www.youtube.com/watch?v=Dd7dQh8u4Hc' -so - | \ grep -iPo '(?<=<title>)(.*)(?=</...


11

mod_security is great, but you don't really need it to achieve your goal. after all mods have been included in httpd.conf you can simply unset the headers of your choosing. Header unset Server ServerSignature Off ServerTokens Prod http://httpd.apache.org/docs/2.4/mod/core.html#serversignature


9

You can write a wrapper around the libc function to resolve hostnames and look them up in a different file than /etc/hosts. Then run any application you'd like to use your hosts file with LD_PRELOAD=wrapper.so firefox


9

Use tcpdump. tcpdump -w httpdebug.pcap -i eth0 port 80 will sniff all packets heading to or from port 80 on the eth0 interface and output them to httpdebug.pcap, which you can then read at your leisure, either with tcpdump again (with multiple -x options, refer to the tcpdump manpage ) in console if you're feeling masochistic, or with wireshark. I really ...


9

There is a Big list of http static server one-liners: To get on this list, a solution must: serve static files using your current directory (or a specified directory) as the server root be able to be run with a single, one line command (dependencies are fine if they're a one-time thing) serve basic file types (html, css, js, images) with ...


9

Simple core command line tools like nc, socat seem not to be able to handle the specific HTTP stuff going on (chunks, transfer encodings, etc.). As a result this may produce unexpected behaviour compared to talking to a real web server. So, my first thought is to share the quickest way I know of setting up a tiny web server and making it just do what you ...


9

RFC 1945 (1996) says 5.1.1 Method The Method token indicates the method to be performed on the resource identified by the Request-URI. The method is case-sensitive. Method = "GET" ; Section 8.1 | "HEAD" ; Section 8.2 | "POST" ; Section 8.3 ...


8

Most browsers enable you to get a warning if you have HTTP content on an HTTPS page. This can be very annoying if you visit sites that mix HTTP content on their HTTPS pages. From your question it appears Wikipedia is one of those. When properly set, Firefox warns me visiting this page. A web server is not required to offer HTTPS. Many sites do not ...


8

The difference is in how DNS and the HTTP "Host" header work. The site you're going to may have multiple sites hosted using the same server. In DNS, all the names for the sites hosted on that IP list the single server IP address. When you enter the name in the browser, the browser sends the hostname to the server using the "Host" header. If you enter only ...


8

You can do this by using tool netcat(also called as nc). Just from command line, do the following steps to send the modified request. $ nc <ip-address-of-webserver> 80 FOOBAR NOT REALLY HTTP Host: example.com


8

You can set up a SOCKS proxy: ssh -D localhost:8080 user@remotehost mplayer http_proxy://localhost:8080/http://stream You can use tell any other application, such as your web browser, to use this proxy too.


8

Try mitmproxy. mitmproxy is an SSL-capable man-in-the-middle proxy for HTTP. It provides a console interface that allows traffic flows to be inspected and edited on the fly. mitmdump is the command-line version of mitmproxy, with the same functionality but without the user interface. Think tcpdump for HTTP. Features Intercept HTTP requests and ...


8

Congratulations, you've just delved into the concept of networking layers by realizing that ports and protocols are not directly connected with each other. As others are saying, telnet can be used to connect to any TCP port. However to understand why this is possible you need to understand a bit about networking layers. If you've ever heard of the OSI 7 ...


7

wget is a great tool. Use wget -m http://somesite.com/directory -m --mirror Turn on options suitable for mirroring. This option turns on recursion and time-stamping, sets infinite recursion depth and keeps FTP directory listings. It is currently equivalent to -r -N -l inf --no-remove-listing.


7

Both Perl and Python (and probably Ruby as well) have simple kits that you can use to quickly build simple HTTP proxies. In Perl, use HTTP::Proxy. Here's the 3-line example from the documentation. Add filters to filter, log or rewrite requests or responses; see the documentation for examples. use HTTP::Proxy; my $proxy = HTTP::Proxy->new( port => ...


7

I was looking for this myself as well and ran into the Node.js http-echo-server: npm install http-echo-server -g export PORT=8081 http-echo-server It accepts all requests and echo the full request including header to the command-line.


7

You can do this using WebDAV. This is an HTTP extension that is supported by most web servers, including IIS and Apache. WebDAV can be mounted in linux via the davfs2 FUSE module.


7

The server ID/token header is controlled by "ServerTokens" directive (provided by mod_core). Aside from modifying the Apache HTTPD source code, or using mod_security module, there is no other way to fully suppress the server ID header. With the mod_security approach, you can disable all of the module's directives/functions in the modsecurity.conf file, and ...



Only top voted, non community-wiki answers of a minimum length are eligible