Tag Info

Hot answers tagged

34

You can use wget over ssh to open an HTTP connection and output stream data, then mplayer on the local machine to play it. ssh <user>@<host> "wget <stream> -O -" | mplayer -cache 512 - The subtleties: wget's -O (capital letter oh) option sets its output file; - (dash) implies stdout. mplayer's -cache option causes a configurable amount ...


31

python -m SimpleHTTPServer should do it.


30

wget -qO- 'http://www.youtube.com/watch?v=Dd7dQh8u4Hc' | perl -l -0777 -ne 'print $1 if /<title.*?>\s*(.*?)\s*<\/title/si' You can pipe it to GNU recode if there are things like &lt; in it: wget -qO- 'http://www.youtube.com/watch?v=Dd7dQh8u4Hc' | perl -l -0777 -ne 'print $1 if /<title.*?>\s*(.*?)\s*<\/title/si' | recode html.. ...


25

I was looking for a way to run a program with modified DNS resolution for testing purposes. For me, the solution was using the HOSTALIASES environment variable: $ echo "foo www.google.com" >> ~/.hosts $ HOSTALIASES=~/.hosts $ wget foo See hostname(7).


21

You can also try hxselect (from HTML-XML-Utils) with wget as follows: wget -qO- 'http://www.youtube.com/watch?v=Dd7dQh8u4Hc' | hxselect -s '\n' -c 'title' 2>/dev/null You can install hxselect in Debian based distros using: sudo apt-get install html-xml-utils. STDERR redirection is to avoid the Input is not well-formed. (Maybe try normalize?) ...


18

You can use ssh's -L flag to create a tunnel from your system to a remote: ssh -L <localport>:<streamhost>:<streamport> <user>@<remote> You can then connect to your end of the tunnel with mplayer as you would otherwise: mplayer localhost:<localport>


11

Use node.js , fast and lightweight. Or just use simple nc netcat command to start a quick webserver on a port and serve the content of a file including the server response headers. Reference from Wikipedia: http://en.wikipedia.org/wiki/Netcat#Setting_up_a_one-shot_webserver_on_port_8080_to_present_the_content_of_a_file { echo -ne "HTTP/1.0 200 ...


10

You can also use curl and grep to do this. You'll need to enlist the use of PCRE (Perl Compatible Regular Expressions) in grep to get the look behind and look ahead facilities so that we can find the <title>...</title> tags. Example $ curl 'http://www.youtube.com/watch?v=Dd7dQh8u4Hc' -so - | \ grep -iPo ...


9

You can write a wrapper around the libc function to resolve hostnames and look them up in a different file than /etc/hosts. Then run any application you'd like to use your hosts file with LD_PRELOAD=wrapper.so firefox


9

Use tcpdump. tcpdump -w httpdebug.pcap -i eth0 port 80 will sniff all packets heading to or from port 80 on the eth0 interface and output them to httpdebug.pcap, which you can then read at your leisure, either with tcpdump again (with multiple -x options, refer to the tcpdump manpage ) in console if you're feeling masochistic, or with wireshark. I really ...


8

Most browsers enable you to get a warning if you have HTTP content on an HTTPS page. This can be very annoying if you visit sites that mix HTTP content on their HTTPS pages. From your question it appears Wikipedia is one of those. When properly set, Firefox warns me visiting this page. A web server is not required to offer HTTPS. Many sites do not ...


8

The difference is in how DNS and the HTTP "Host" header work. The site you're going to may have multiple sites hosted using the same server. In DNS, all the names for the sites hosted on that IP list the single server IP address. When you enter the name in the browser, the browser sends the hostname to the server using the "Host" header. If you enter only ...


8

You can set up a SOCKS proxy: ssh -D localhost:8080 user@remotehost mplayer http_proxy://localhost:8080/http://stream You can use tell any other application, such as your web browser, to use this proxy too.


7

Since version 5.4.0 PHP also has a built-in web server: php -S localhost:8000


7

You can do this by using tool netcat(also called as nc). Just from command line, do the following steps to send the modified request. $ nc <ip-address-of-webserver> 80 FOOBAR NOT REALLY HTTP Host: example.com


7

You can do this using WebDAV. This is an HTTP extension that is supported by most web servers, including IIS and Apache. WebDAV can be mounted in linux via the davfs2 FUSE module.


6

I think the best way to do this would be to set up a SOCKS5 proxy and tell firefox to send the DNS requests over the SOCKS5 proxy (network.proxy.socks_remote_dns). You could set up a socks5 proxy with openssh fairly easily (the -D option) and have a remote host running with a custom /etc/hosts, or something like DNSMasq for more complex DNS settings. ...


6

Both Perl and Python (and probably Ruby as well) have simple kits that you can use to quickly build simple HTTP proxies. In Perl, use HTTP::Proxy. Here's the 3-line example from the documentation. Add filters to filter, log or rewrite requests or responses; see the documentation for examples. use HTTP::Proxy; my $proxy = HTTP::Proxy->new( port => ...


6

With avfs: mkdir ~/.avfs avfsd ~/.avfs ln -s ~/.avfs/'#http:server.com|~me|uber_vimrc' ~/.vimrc For writing, you could use webdav. avfs supports a (great) number of network protocols and other methods to access files and adding more is relatively easy. For mounting over sftp, look at sshfs.


6

sshfs is a wonderful tool for cases like this. It hides many a firewall and allows you to use whatever player you choose (though mplayer is still a nice choice). sudo apt-get install sshfs # your favorite packager here sshfs me@mydomain.org:/home/arthur/Music ~/Music I also run Subsonic listening to port 4040 on localhost and then ssh-tunnel home with ...


6

you can simply use curl for it.i have written a simple script for it. url=$1 curl -sL $url -w "%{http_code} %{url_effective}\n" "URL" -o /dev/null where URL is your URL which you have to test output will gives you status. Example


5

If you really want to use command line for this there is tcpflow. It saves TCP streams to different files. The HTTP request and responses will be stored separately. tcpflow -i wlan0 tcp port 80 If you can use GUI try Wireshark. You can right click any packet and pick "Follow TCP stream".


5

Check out following question at superuser: http://superuser.com/questions/184643/override-dns-in-firefox If the discussed options and the SO link are not viable solutions then check out: http://superuser.com/questions/100239/hostname-override-in-firefox Especially check out: https://addons.mozilla.org/en-US/firefox/addon/redirector/ It sounds like this ...


5

wget is a great tool. Use wget -m http://somesite.com/directory -m --mirror Turn on options suitable for mirroring. This option turns on recursion and time-stamping, sets infinite recursion depth and keeps FTP directory listings. It is currently equivalent to -r -N -l inf --no-remove-listing.


5

Like rsync, but use zsync to get from an httpd server.


5

This may not be the best solution, but if you use any proxy then it will have a specific host:port so the netcat solution with still work, albeit you'll have to pick apart the proxy meta-data to make sense of it. The easiest way to do this might be to use any random anonymization proxy out there and just channel all the traffic through netcat. (I.e., set ...


5

What you want is a HEAD request, but wget does not support it; curl does. Your distribution most probably has curl in repositories. curl -s -I $URL -I | awk '/Location: (.*)/ {print $2}' | tail -n 1 $ URL=http://unix.stackexchange.com/questions/89282/ $ curl -s -I $URL | awk '/Location: (.*)/ {print $2}' | tail -n 1 ...


5

Use the --trust-server-names option: --trust-server-names: If this is set to on, on a redirect the last component of the redirection URL will be used as the local file name. By default it is used the last component in the original URL. Note that in some cases this can be a security risk: you have no way of knowing what the created filename will be ...


4

Your question is quite subjective, and configuring the port should be very easy with any http server. Though, I would recommend lighttpd with server.port = 1080 (or whatever above 1024 for non-root users) in a configuration file: server.document-root = "/home/foo/www" server.port = 1080 It's a complete enough http server, and for serving static ...


4

The headers in an HTTP request must use CRLF (Windows) line endings. (See Wikipedia or RFC 2616.) Many servers support LF (Unix) line endings as an extension, but not this one. In addition, HTTP 1.1 requires a Host: header line, as Warren Young pointed out. (See Wikipedia or RFC 2616). echo -e "GET ...



Only top voted, non community-wiki answers of a minimum length are eligible