Hot answers tagged

59

python -m SimpleHTTPServer should do it.


41

I was looking for a way to run a program with modified DNS resolution for testing purposes. For me, the solution was using the HOSTALIASES environment variable: $ echo "foo www.google.com" >> ~/.hosts $ HOSTALIASES=~/.hosts wget foo See hostname(7). (Side note: In the example the HOSTALIASES environment variable only affects the wget process. Of ...


36

wget -qO- 'http://www.youtube.com/watch?v=Dd7dQh8u4Hc' | perl -l -0777 -ne 'print $1 if /<title.*?>\s*(.*?)\s*<\/title/si' You can pipe it to GNU recode if there are things like &lt; in it: wget -qO- 'http://www.youtube.com/watch?v=Dd7dQh8u4Hc' | perl -l -0777 -ne 'print $1 if /<title.*?>\s*(.*?)\s*<\/title/si' | recode html.. ...


33

You can use wget over ssh to open an HTTP connection and output stream data, then mplayer on the local machine to play it. ssh <user>@<host> "wget <stream> -O -" | mplayer -cache 512 - The subtleties: wget's -O (capital letter oh) option sets its output file; - (dash) implies stdout. mplayer's -cache option causes a configurable amount ...


24

You can also try hxselect (from HTML-XML-Utils) with wget as follows: wget -qO- 'http://www.youtube.com/watch?v=Dd7dQh8u4Hc' | hxselect -s '\n' -c 'title' 2>/dev/null You can install hxselect in Debian based distros using: sudo apt-get install html-xml-utils. STDERR redirection is to avoid the Input is not well-formed. (Maybe try normalize?) ...


18

Use node.js , fast and lightweight. Or just use simple nc netcat command to start a quick webserver on a port and serve the content of a file including the server response headers. Reference from Wikipedia: http://en.wikipedia.org/wiki/Netcat#Setting_up_a_one-shot_webserver_on_port_8080_to_present_the_content_of_a_file { echo -ne "HTTP/1.0 200 ...


18

You can use ssh's -L flag to create a tunnel from your system to a remote: ssh -L <localport>:<streamhost>:<streamport> <user>@<remote> You can then connect to your end of the tunnel with mplayer as you would otherwise: mplayer localhost:<localport>


18

Use telnet domain.name.server.com 80 then HEAD /~USER HTTP/1.1 Host: domain.name.server.com (Then you have to hit Enter once more.) Now it should show you the header of this page. For a real life example: $ telnet unix.stackexchange.com 80 Trying 198.252.206.16... Connected to unix.stackexchange.com. Escape character is '^]'. HEAD ...


15

You have the symptoms of an MTU problem: some TCP connections freeze, more or less reproducibly for a given command or URL but with no easily discernible overall pattern. A telltale symptom is that interactive ssh sessions work well but file transfers almost always fail. Furthermore pppoe is the number one bringer of MTU problem for home users. So I ...


13

Since version 5.4.0 PHP also has a built-in web server: php -S localhost:8000


10

You can also use curl and grep to do this. You'll need to enlist the use of PCRE (Perl Compatible Regular Expressions) in grep to get the look behind and look ahead facilities so that we can find the <title>...</title> tags. Example $ curl 'http://www.youtube.com/watch?v=Dd7dQh8u4Hc' -so - | \ grep -iPo ...


9

You can write a wrapper around the libc function to resolve hostnames and look them up in a different file than /etc/hosts. Then run any application you'd like to use your hosts file with LD_PRELOAD=wrapper.so firefox


9

Use tcpdump. tcpdump -w httpdebug.pcap -i eth0 port 80 will sniff all packets heading to or from port 80 on the eth0 interface and output them to httpdebug.pcap, which you can then read at your leisure, either with tcpdump again (with multiple -x options, refer to the tcpdump manpage ) in console if you're feeling masochistic, or with wireshark. I really ...


8

Most browsers enable you to get a warning if you have HTTP content on an HTTPS page. This can be very annoying if you visit sites that mix HTTP content on their HTTPS pages. From your question it appears Wikipedia is one of those. When properly set, Firefox warns me visiting this page. A web server is not required to offer HTTPS. Many sites do not ...


8

There is a Big list of http static server one-liners: To get on this list, a solution must: serve static files using your current directory (or a specified directory) as the server root be able to be run with a single, one line command (dependencies are fine if they're a one-time thing) serve basic file types (html, css, js, images) with ...


8

The difference is in how DNS and the HTTP "Host" header work. The site you're going to may have multiple sites hosted using the same server. In DNS, all the names for the sites hosted on that IP list the single server IP address. When you enter the name in the browser, the browser sends the hostname to the server using the "Host" header. If you enter only ...


8

Simple core command line tools like nc, socat seem not to be able to handle the specific HTTP stuff going on (chunks, transfer encodings, etc.). As a result this may produce unexpected behaviour compared to talking to a real web server. So, my first thought is to share the quickest way I know of setting up a tiny web server and making it just do what you ...


8

You can set up a SOCKS proxy: ssh -D localhost:8080 user@remotehost mplayer http_proxy://localhost:8080/http://stream You can use tell any other application, such as your web browser, to use this proxy too.


8

Try mitmproxy. mitmproxy is an SSL-capable man-in-the-middle proxy for HTTP. It provides a console interface that allows traffic flows to be inspected and edited on the fly. mitmdump is the command-line version of mitmproxy, with the same functionality but without the user interface. Think tcpdump for HTTP. Features Intercept HTTP requests and ...


7

You can do this by using tool netcat(also called as nc). Just from command line, do the following steps to send the modified request. $ nc <ip-address-of-webserver> 80 FOOBAR NOT REALLY HTTP Host: example.com


7

You can do this using WebDAV. This is an HTTP extension that is supported by most web servers, including IIS and Apache. WebDAV can be mounted in linux via the davfs2 FUSE module.


6

I think the best way to do this would be to set up a SOCKS5 proxy and tell firefox to send the DNS requests over the SOCKS5 proxy (network.proxy.socks_remote_dns). You could set up a socks5 proxy with openssh fairly easily (the -D option) and have a remote host running with a custom /etc/hosts, or something like DNSMasq for more complex DNS settings. ...


6

Yes, nweb: http://www.ibm.com/developerworks/systems/library/es-nweb/index.html To compile nweb.c: gcc -O -DLINUX nweb.c -o nweb


6

wget is a great tool. Use wget -m http://somesite.com/directory -m --mirror Turn on options suitable for mirroring. This option turns on recursion and time-stamping, sets infinite recursion depth and keeps FTP directory listings. It is currently equivalent to -r -N -l inf --no-remove-listing.


6

Both Perl and Python (and probably Ruby as well) have simple kits that you can use to quickly build simple HTTP proxies. In Perl, use HTTP::Proxy. Here's the 3-line example from the documentation. Add filters to filter, log or rewrite requests or responses; see the documentation for examples. use HTTP::Proxy; my $proxy = HTTP::Proxy->new( port => ...


6

With avfs: mkdir ~/.avfs avfsd ~/.avfs ln -s ~/.avfs/'#http:server.com|~me|uber_vimrc' ~/.vimrc For writing, you could use webdav. avfs supports a (great) number of network protocols and other methods to access files and adding more is relatively easy. For mounting over sftp, look at sshfs.


6

sshfs is a wonderful tool for cases like this. It hides many a firewall and allows you to use whatever player you choose (though mplayer is still a nice choice). sudo apt-get install sshfs # your favorite packager here sshfs me@mydomain.org:/home/arthur/Music ~/Music I also run Subsonic listening to port 4040 on localhost and then ssh-tunnel home with ...


6

The server ID/token header is controlled by "ServerTokens" directive (provided by mod_core). Aside from modifying the Apache HTTPD source code, or using mod_security module, there is no other way to fully suppress the server ID header. With the mod_security approach, you can disable all of the module's directives/functions in the modsecurity.conf file, and ...


5

Check out following question at superuser: http://superuser.com/questions/184643/override-dns-in-firefox If the discussed options and the SO link are not viable solutions then check out: http://superuser.com/questions/100239/hostname-override-in-firefox Especially check out: https://addons.mozilla.org/en-US/firefox/addon/redirector/ It sounds like this ...



Only top voted, non community-wiki answers of a minimum length are eligible