All about programs and services that use the Hypertext Transfer Protocol.

learn more… | top users | synonyms

31
votes
7answers
6k views

How do I get a websites title using command line?

I want a command line program that prints the title of a website. For e.g.: Alan:~ titlefetcher http://www.youtube.com/watch?v=Dd7dQh8u4Hc should give: Why Are Bad Words Bad? You give it the ...
26
votes
4answers
3k views

How do I play an HTTP music stream over SSH?

I know I can just do mplayer <stream> to open and play a stream on the local machine. However, I'm behind a firewall. Instead, I'd like to open the stream on a remote machine, but still play it ...
24
votes
6answers
13k views

Can I create override dns similar to writing in /etc/hosts without root access

I want to set a dns record that my browser will use, but I don't have root access, so I can't modify /etc/hosts. I need to do this for testing vhosts with apache, whose dns hasn't yet been set up. I ...
23
votes
10answers
21k views

Simple command line http server

I have a script which generates a daily report which I want to serve to the so called general public. The problem is I don't want to add to my headaches maintance of a http server (e.x. apache) with ...
12
votes
2answers
9k views

Change Apache httpd “Server:” HTTP header

One of the HTTP headers that the Apache httpd sends back with response data is "Server". For example, my web server machine is relatively up-to-date Arch Linux. It sends back headers closely ...
9
votes
3answers
5k views

Can't access select https sites on Linux over PPPoE

My internet connection used to be a direct LAN connection to my provider. Back then, everything would load fine on both Windows and Ubuntu (dual boot). However, a while ago they started needing me to ...
8
votes
2answers
554 views

Are there any good tools besides SeleniumRC that can fetch webpages including content post-painted by JavaScript?

One major shortcoming of curl is that more and more wepages are having their main piece of content painted by a JavaScript AJAX response that occurs after the initial HTTP response. curl never picks ...
7
votes
2answers
2k views

Generic HTTP server that just dumps POST requests?

I am looking for a command line tool that listens on a given part, happily excepts every HTTP POST request and dumps it. I want to use it for testing purposes, i.e. for testing clients that issue ...
5
votes
2answers
2k views

Is there any command-line, generic HTTP proxy (like Squid)?

I can easily use Netcat (or, Socat) to capture traffic between my browser and a specific host:port. But for Linux, does there exist any command-line counterpart of a Squid-like HTTP proxy that I can ...
5
votes
2answers
7k views

How to check whether a command such as curl completed without error?

I am using curl to upload a file to a server via an HTTP post. curl -X POST -d@myfile.txt server-URL When I manually execute this command on the command line, I get a response from the server like ...
5
votes
1answer
282 views

What tools allow me to present man pages as formatted HTML on a web server?

What tools exist to make man pages available on a system available via HTTP and link topics together such that references in the SEE ALSO sections and elsewhere will become hyperlinks in the HTML ...
5
votes
1answer
73 views

IPtables - new vs new, established, related

I'm «pretty» new to IPtables, or I have used IPtables a few years, but never with advanced features. I understand the concept of stateful IPtables rules, but recently I discovered that some people ...
4
votes
2answers
776 views

How can I see dumps of wholе HTTP packets?

I need to see whole HTTP packets sent and recieved by an application for debugging purposes. How can this be done in command-line?
4
votes
2answers
3k views

Sync with a Directory Listing of An Apache Server

Is there an easy way to keep a folder synced with a directory listing via HTTP? Edit: Thanks for the tip with wget! I created a shell script and added it as a cron job: remote_dirs=( ...
4
votes
2answers
4k views

Mount HTTP Server As File System

I have a machine on which I wish to mount multiple remote servers to access them all centrally. For remote Linux based systems I am using SSHFS which works fine. But for Windows systems, or systems ...
4
votes
3answers
292 views

Send first line of the HTTP Header

A normal HTTP request may look something like this: GET / HTTP/1.1 Host: example.com I would like to change the first line of this request, with a custom one, e.g.: FOOBAR NOT REALLY HTTP Host: ...
4
votes
1answer
116 views

Getting a website's ip with 'host'?

Why is it that when I run the command host examplewebsite.com it gives me website's ip, but for some websites, when I type that IP address into the address bar and hit enter, it shows me a 404 page? ...
4
votes
1answer
166 views

Map file to URL

I'm looking for a way to do something like this: some_tool --go-to-background http://server.com/~me/uber_vimrc ~/.vimrc After which, reading ~/.vimrc would give the data downloaded from that URL. ...
4
votes
1answer
3k views

What tool can I use to sniff HTTP/HTTPS traffic?

I'm looking for a command line tool that can intercept HTTP/HTTPS requests, extract information such as: (content, destination, etc.), perform various analysis tasks, and finally determine if the ...
4
votes
2answers
2k views

How to make wget download recursive combining --accept with --exclude-directories?

I'm trying to download some directories from an Apache server, but I need to ignore some directories that have huge files I don't care about The dir structure in the server is somewhat like this ...
4
votes
1answer
2k views

Why wget does not work via ssh tunnel? What does proxy prevent ssh-client to do?

To test things I'm trying to establish a local ssh tunnel from my laptop to a site via ssh-server, and download a page of the site (or view it in a browser). The tunnel is made like this: $ ssh -L ...
4
votes
1answer
559 views

Stream system audio through mpd

I use mpd to stream music to my phone (connected to Hi-Fi) via http. Yet, this only works for songs in my mpd database. I oftentimes want to play stuff in my browser and would like that to be ...
4
votes
1answer
5k views

How to configure pfsense to NAT servers?

How can I configure pfsense so it will NAT servers so they can be accessed outside of the company? I tried to play around with the NAT settings, but didn't get anywhere. I want to be able to access ...
4
votes
2answers
157 views

Port Routing/DNS for individual program

Background Flickr has closed its http api leaving an https only api. My favourite uploader (the official Flickr Uploadr version 2.5 running under wine) is no longer maintained, so this change broke ...
3
votes
3answers
289 views

True HTTPS? How?

/Using Google Chrome/ if i visit: https://secure.wikimedia.org/wikipedia/en/wiki/File:Nokota_Horses_cropped.jpg then the: ...
3
votes
4answers
660 views

How to examine POST request over HTTPS?

I'm trying to automate access to a website via scraping, and I'm trying to determine what parameters are being sent to the server in a POST request. I've currently got Wireshark open, but is there a ...
3
votes
3answers
63 views

How can I list every port serving a web page

I have a number of servers, all with various HTTP(S) services accessible on a variety of ports. It's incredibly difficult to keep track of which service is available under which host and what port. ...
3
votes
1answer
399 views

Monitor outgoing web requests as they’re happening

I want to see a list of all outgoing HTTP requests from my desktop. I think it should be possible to monitor HTTPS hostnames as well for local clients using Server Name Indication (SNI). OS X has a ...
3
votes
2answers
834 views

How to redirect http requests to local Apache server on internet connection lost?

I have a little Linux (Ubuntu based) server with 2 network cards, which acts as a router, dhcp, dns server, firewall, etc. One network card is connected to the LAN, while the other is connected to my ...
3
votes
1answer
449 views

Does using epub2pdf pose a potential security risk? Is there an alternative?

epub2pdf is a fairly platform-independent program. Based on Java, it executes without a hitch on my machine, with no dependencies. However, strangely enough, the program requires an active internet ...
3
votes
1answer
294 views

Allow non-root unmodifiable sofware to bind to ports 80 & 443

The company I'm in uses the Java Secure Application Manager to "secure" non-internal connections their webmail and electronic collaboration software. Aside from using expired certificates, and ...
3
votes
2answers
689 views

Local DNS rewrite from host to host for web requests

Is there some de-facto way for rewriting certain hostnames to other ones? Something like /etc/hosts for host to host instead of ip to host. Is this possible or should I create a local dns cname for ...
3
votes
0answers
19 views

Using socat to Tunnel/Proxy TCP want to split send/receive into separate files

I'm using socat to dump plain TCP (e.g. HTTP) traffic: socat -v tcp-listen:$PORT,reuseaddr,fork,keepalive tcp:$TARGET_HOST:$TARGET_PORT At the moment all send and receive traffic is dumped into ...
3
votes
0answers
96 views

How to stream audio from /dev/dsp?

What is the best way to stream audio from /dev/dsp (I'm using OSS on Debian). I would prefer HTTP streaming, but if there is a way on another protocol, I can deal with that.
3
votes
0answers
170 views

WGET - Authenticating root AND all children folders

Before I start the question off, I want to say that a similar question helped me get past the initial login. My issue is as stated below. There's a website that I'm trying to mirror. It is something ...
3
votes
0answers
65 views

Use command line to download file which is accessible only in a given session [duplicate]

Possible Duplicate: Does anybody here have experience in automating some tasks in web applications using curl? Sorry for the lengthy title. I am trying use command line to download a ...
2
votes
3answers
4k views

Simple problem with netcat tool

Can anybody tell me why I am getting bad request while executing this command echo -e "GET http://www.yellowpages.com.eg/Mjg3NF9VUkxfMTEwX2h0dHA6Ly93d3cubG90dXMtYWlyLmNvbV8=/Lotus-Air/profile.html ...
2
votes
3answers
4k views

How to change user agent on all http requests made from my machine with Squid intercepting proxy?

I'd like to change the user agent on all HTTP requests made from this machine. It's easy enough to change the user agent in each browser, but other apps also make http requests. It's vital I catch ...
2
votes
1answer
670 views

Capture web traffic grouped by individual TCP streams

As far as I know a HTTP request and response constructs one TCP connection. To debug a web application on a GUI-less server, I'd like to be able to capture these TCP streams in a single ...
2
votes
2answers
333 views

What is the “Online Certificate Status Protocol”?

I use privoxy. In the user.action file i have a redirect rule and a few websites: { +redirect{s@http://@https://@} } .twitter.com .facebook.com Ok! it's working great, e.g.: if i visit any ...
2
votes
2answers
257 views

Run grep or other commands on files on a web server

I have files on another domain. What's the best way to use commands on them like grep? wget http://path/to/file.txt | grep search However this doesn't work. I guess I can use curl but is there a ...
2
votes
1answer
81 views

catch http requests from another process

Suppose a process, among other things, is posting https requests, for example with curl. Assuming I have root privileges, is there a way to intercept such requests and respond with an http error ...
2
votes
1answer
729 views

Run my own HTTP fast-browsing proxy a la Opera Mini?

Opera Mini is a great browser that works well on feature phones and iPhones. Unlike most browsers, it uses Opera's servers to transcode every page you browse to into a small, easy-to-render chunk ...
2
votes
1answer
177 views

Problem with pkg behind a chunking proxy

I have a freshly installed FreeBSD 10.1 at work in a VM. As it is behind a corporate proxy, I had to set HTTP_PROXY in environment and it began to run fine. But no way to get pkg to work correctly. I ...
2
votes
1answer
1k views

Telnet to port 80 is not showing results

I'm trying to a test request/response to www.google.com on port 80 as below : $ telnet www.google.com 80 After that I typed in : GET‌ / HTTP/1.1 Host: www.google.com But nothing happens! Any ...
2
votes
2answers
171 views

Using Wireshark on Linux to observe coupled HTTP-request and HTTP-respond

I use Wireshark on Linux to observe my HTTP traffic. Other programms like Firefox Live Http Headers show coupled http-request from my browser and http-respond from any server. How using programs like ...
2
votes
1answer
420 views

How to implement retry mechanism while executing the server url in the shell script?

I am working on a project in which I need to make a url call to one of my server from the bash shell script.. http://hostname.domain.com:8080/beat After hitting the above url, I will be getting the ...
2
votes
2answers
541 views

Apache server redirects main domain to https

I have this server with multiple domains running CentOS 6.2 and Apache. This server has a SSL certificate installed and working well. Every time I browse to the main domain, without adding http:// it ...
2
votes
2answers
4k views

Fatal error: Class 'HttpRequest' not found

Whilst trying to make this request I have this error. I have tried to run "sudo pecl install pecl_http" but always fails. Could not find curl/curl.h After I have tired(yum instead of apt-get ...
2
votes
1answer
2k views

Install php_http on Arch Linux

First I installed the curl extension for PHP: $ sudo pacman -S php-curl and $ grep curl /etc/php/php.ini extension=curl.so Then I installed the PHP PEAR $ sudo pacman -S php-pear Having now ...