All about programs and services that use the Hypertext Transfer Protocol.

learn more… | top users | synonyms

39
votes
14answers
38k views

Simple command line http server

I have a script which generates a daily report which I want to serve to the so called general public. The problem is I don't want to add to my headaches maintance of a http server (e.x. apache) with ...
35
votes
7answers
10k views

How do I get a websites title using command line?

I want a command line program that prints the title of a website. For e.g.: Alan:~ titlefetcher http://www.youtube.com/watch?v=Dd7dQh8u4Hc should give: Why Are Bad Words Bad? You give it the ...
27
votes
4answers
4k views

How do I play an HTTP music stream over SSH?

I know I can just do mplayer <stream> to open and play a stream on the local machine. However, I'm behind a firewall. Instead, I'd like to open the stream on a remote machine, but still play it ...
26
votes
7answers
16k views

Can I create override dns similar to writing in /etc/hosts without root access

I want to set a dns record that my browser will use, but I don't have root access, so I can't modify /etc/hosts. I need to do this for testing vhosts with apache, whose dns hasn't yet been set up. I ...
18
votes
4answers
23k views

Change Apache httpd “Server:” HTTP header

One of the HTTP headers that the Apache httpd sends back with response data is "Server". For example, my web server machine is relatively up-to-date Arch Linux. It sends back headers closely ...
10
votes
3answers
6k views

Can't access select https sites on Linux over PPPoE

My internet connection used to be a direct LAN connection to my provider. Back then, everything would load fine on both Windows and Ubuntu (dual boot). However, a while ago they started needing me to ...
10
votes
2answers
3k views

Generic HTTP server that just dumps POST requests?

I am looking for a command line tool that listens on a given part, happily excepts every HTTP POST request and dumps it. I want to use it for testing purposes, i.e. for testing clients that issue ...
9
votes
2answers
5k views

What tool can I use to sniff HTTP/HTTPS traffic?

I'm looking for a command line tool that can intercept HTTP/HTTPS requests, extract information such as: (content, destination, etc.), perform various analysis tasks, and finally determine if the ...
8
votes
2answers
689 views

Are there any good tools besides SeleniumRC that can fetch webpages including content post-painted by JavaScript?

One major shortcoming of curl is that more and more wepages are having their main piece of content painted by a JavaScript AJAX response that occurs after the initial HTTP response. curl never picks ...
7
votes
2answers
4k views

Is there any command-line, generic HTTP proxy (like Squid)?

I can easily use Netcat (or, Socat) to capture traffic between my browser and a specific host:port. But for Linux, does there exist any command-line counterpart of a Squid-like HTTP proxy that I can ...
7
votes
2answers
18k views

How to check whether a command such as curl completed without error?

I am using curl to upload a file to a server via an HTTP post. curl -X POST -d@myfile.txt server-URL When I manually execute this command on the command line, I get a response from the server like ...
6
votes
3answers
1k views

Using telnet to get website header

I have question, is it possible to get header of website by using telnet? Website looks like this domain.name.server.com/~USER (just example). And I want to get it header by telnet. telnet domain....
6
votes
1answer
1k views

IPtables - new vs new, established, related

I'm «pretty» new to IPtables, or I have used IPtables a few years, but never with advanced features. I understand the concept of stateful IPtables rules, but recently I discovered that some people ...
5
votes
2answers
4k views

Sync with a Directory Listing of Web Server

Is there an easy way to keep a folder synced with a directory listing via HTTP? Edit: Thanks for the tip with wget! I created a shell script and added it as a cron job: remote_dirs=( "http://...
5
votes
3answers
384 views

Send first line of the HTTP Header

A normal HTTP request may look something like this: GET / HTTP/1.1 Host: example.com I would like to change the first line of this request, with a custom one, e.g.: FOOBAR NOT REALLY HTTP Host: ...
5
votes
1answer
297 views

What tools allow me to present man pages as formatted HTML on a web server?

What tools exist to make man pages available on a system available via HTTP and link topics together such that references in the SEE ALSO sections and elsewhere will become hyperlinks in the HTML ...
4
votes
4answers
3k views

how am i able to telnet to http port 80

After searching in google I found out that we can telnet to web-server to its http port and use GET to retrieve a html page. For ex: $ telnet web-server-name 80 But I am not able to understand ...
4
votes
2answers
1k views

How can I see dumps of wholе HTTP packets?

I need to see whole HTTP packets sent and recieved by an application for debugging purposes. How can this be done in command-line?
4
votes
2answers
7k views

Mount HTTP Server As File System

I have a machine on which I wish to mount multiple remote servers to access them all centrally. For remote Linux based systems I am using SSHFS which works fine. But for Windows systems, or systems ...
4
votes
1answer
119 views

Getting a website's ip with 'host'?

Why is it that when I run the command host examplewebsite.com it gives me website's ip, but for some websites, when I type that IP address into the address bar and hit enter, it shows me a 404 page? ...
4
votes
1answer
238 views

Map file to URL

I'm looking for a way to do something like this: some_tool --go-to-background http://server.com/~me/uber_vimrc ~/.vimrc After which, reading ~/.vimrc would give the data downloaded from that URL. ...
4
votes
1answer
7k views

How to configure pfsense to NAT servers?

How can I configure pfsense so it will NAT servers so they can be accessed outside of the company? I tried to play around with the NAT settings, but didn't get anywhere. I want to be able to access ...
4
votes
2answers
3k views

How to make wget download recursive combining --accept with --exclude-directories?

I'm trying to download some directories from an Apache server, but I need to ignore some directories that have huge files I don't care about The dir structure in the server is somewhat like this (...
4
votes
1answer
2k views

Why wget does not work via ssh tunnel? What does proxy prevent ssh-client to do?

To test things I'm trying to establish a local ssh tunnel from my laptop to a site via ssh-server, and download a page of the site (or view it in a browser). The tunnel is made like this: $ ssh -L ...
4
votes
1answer
926 views

Stream system audio through mpd

I use mpd to stream music to my phone (connected to Hi-Fi) via http. Yet, this only works for songs in my mpd database. I oftentimes want to play stuff in my browser and would like that to be ...
4
votes
2answers
192 views

Port Routing/DNS for individual program

Background Flickr has closed its http api leaving an https only api. My favourite uploader (the official Flickr Uploadr version 2.5 running under wine) is no longer maintained, so this change broke ...
3
votes
3answers
295 views

True HTTPS? How?

/Using Google Chrome/ if i visit: https://secure.wikimedia.org/wikipedia/en/wiki/File:Nokota_Horses_cropped.jpg then the: http://upload.wikimedia.org/wikipedia/commons/d/de/Nokota_Horses_cropped....
3
votes
4answers
701 views

How to examine POST request over HTTPS?

I'm trying to automate access to a website via scraping, and I'm trying to determine what parameters are being sent to the server in a POST request. I've currently got Wireshark open, but is there a ...
3
votes
3answers
133 views

How can I list every port serving a web page

I have a number of servers, all with various HTTP(S) services accessible on a variety of ports. It's incredibly difficult to keep track of which service is available under which host and what port. ...
3
votes
1answer
1k views

Monitor outgoing web requests as they’re happening

I want to see a list of all outgoing HTTP requests from my desktop. I think it should be possible to monitor HTTPS hostnames as well for local clients using Server Name Indication (SNI). OS X has a ...
3
votes
1answer
111 views

What ports need to be open on a firewall to access the internet?

Let's say as the example that I have a firewall that blocks ALL ports from all sources/destinations. What ports would I need to open to be able to successfully run: ping google.com ...and are ...
3
votes
2answers
1k views

How to redirect http requests to local Apache server on internet connection lost?

I have a little Linux (Ubuntu based) server with 2 network cards, which acts as a router, dhcp, dns server, firewall, etc. One network card is connected to the LAN, while the other is connected to my ...
3
votes
1answer
35 views

Storing stderr output in a variable without modifing stdout, and doing so repeatedly

there are many "almost duplicate" questions, but none gave me the answer I'm looking for, so bear with me. consider a simple server API, in which you can consume data in iterations, using a "...
3
votes
1answer
494 views

Does using epub2pdf pose a potential security risk? Is there an alternative?

epub2pdf is a fairly platform-independent program. Based on Java, it executes without a hitch on my machine, with no dependencies. However, strangely enough, the program requires an active internet ...
3
votes
1answer
510 views

Allow non-root unmodifiable sofware to bind to ports 80 & 443

The company I'm in uses the Java Secure Application Manager to "secure" non-internal connections their webmail and electronic collaboration software. Aside from using expired certificates, and ...
3
votes
2answers
805 views

Local DNS rewrite from host to host for web requests

Is there some de-facto way for rewriting certain hostnames to other ones? Something like /etc/hosts for host to host instead of ip to host. Is this possible or should I create a local dns cname for ...
3
votes
0answers
121 views

Using socat to Tunnel/Proxy TCP want to split send/receive into separate files

I'm using socat to dump plain TCP (e.g. HTTP) traffic: socat -v tcp-listen:$PORT,reuseaddr,fork,keepalive tcp:$TARGET_HOST:$TARGET_PORT At the moment all send and receive traffic is dumped into ...
3
votes
0answers
132 views

How to stream audio from /dev/dsp?

What is the best way to stream audio from /dev/dsp (I'm using OSS on Debian). I would prefer HTTP streaming, but if there is a way on another protocol, I can deal with that.
3
votes
0answers
203 views

WGET - Authenticating root AND all children folders

Before I start the question off, I want to say that a similar question helped me get past the initial login. My issue is as stated below. There's a website that I'm trying to mirror. It is something ...
3
votes
0answers
65 views

Use command line to download file which is accessible only in a given session [duplicate]

Possible Duplicate: Does anybody here have experience in automating some tasks in web applications using curl? Sorry for the lengthy title. I am trying use command line to download a file,...
2
votes
3answers
4k views

How do I get a URL over HTTP with netcat?

Can anybody tell me why I am getting bad request while executing this command echo -e "GET http://www.yellowpages.com.eg/Mjg3NF9VUkxfMTEwX2h0dHA6Ly93d3cubG90dXMtYWlyLmNvbV8=/Lotus-Air/profile.html ...
2
votes
4answers
6k views

How to change user agent on all http requests made from my machine with Squid intercepting proxy?

I'd like to change the user agent on all HTTP requests made from this machine. It's easy enough to change the user agent in each browser, but other apps also make http requests. It's vital I catch ...
2
votes
1answer
1k views

Capture web traffic grouped by individual TCP streams

As far as I know a HTTP request and response constructs one TCP connection. To debug a web application on a GUI-less server, I'd like to be able to capture these TCP streams in a single ...
2
votes
2answers
489 views

What is the “Online Certificate Status Protocol”?

I use privoxy. In the user.action file i have a redirect rule and a few websites: { +redirect{s@http://@https://@} } .twitter.com .facebook.com Ok! it's working great, e.g.: if i visit any "*...
2
votes
2answers
516 views

Run grep or other commands on files on a web server

I have files on another domain. What's the best way to use commands on them like grep? wget http://path/to/file.txt | grep search However this doesn't work. I guess I can use curl but is there a ...
2
votes
1answer
2k views

Telnet to port 80 is not showing results

I'm trying to a test request/response to www.google.com on port 80 as below : $ telnet www.google.com 80 After that I typed in : GET‌ / HTTP/1.1 Host: www.google.com But nothing happens! Any ...
2
votes
1answer
118 views

catch http requests from another process

Suppose a process, among other things, is posting https requests, for example with curl. Assuming I have root privileges, is there a way to intercept such requests and respond with an http error or,...
2
votes
1answer
1k views

Run my own HTTP fast-browsing proxy a la Opera Mini?

Opera Mini is a great browser that works well on feature phones and iPhones. Unlike most browsers, it uses Opera's servers to transcode every page you browse to into a small, easy-to-render chunk that'...
2
votes
1answer
139 views

Large downloads interrupted on nginx reload

I'm running nginx 1.9.3. When downloading a large file (in this case, 1GiB), and reloading nginx via "nginx -s reload", the download gets interrupted in numerous browsers, including (but potentially ...
2
votes
1answer
434 views

Problem with pkg behind a chunking proxy

I have a freshly installed FreeBSD 10.1 at work in a VM. As it is behind a corporate proxy, I had to set HTTP_PROXY in environment and it began to run fine. But no way to get pkg to work correctly. I ...