Skip to main content

Questions tagged [http]

All about programs and services that use the Hypertext Transfer Protocol.

Filter by
Sorted by
Tagged with
226 votes
22 answers
345k views

Simple command line HTTP server

I have a script which generates a daily report which I want to serve to the so called general public. The problem is I don't want to add to my headaches maintance of a HTTP server (e.g. Apache) with ...
Cid's user avatar
  • 2,475
56 votes
9 answers
42k views

Can I create override dns similar to writing in /etc/hosts without root access

I want to set a dns record that my browser will use, but I don't have root access, so I can't modify /etc/hosts. I need to do this for testing vhosts with apache, whose dns hasn't yet been set up. I ...
xenoterracide's user avatar
44 votes
5 answers
35k views

Generic HTTP server that just dumps POST requests?

I am looking for a command line tool that listens on a given part, happily excepts every HTTP POST request and dumps it. I want to use it for testing purposes, i.e. for testing clients that issue ...
maxschlepzig's user avatar
  • 57.8k
41 votes
2 answers
182k views

How to check whether a command such as curl completed without error?

I am using curl to upload a file to a server via an HTTP post. curl -X POST [email protected] server-URL When I manually execute this command on the command line, I get a response from the server like "...
Wes's user avatar
  • 831
38 votes
4 answers
121k views

Change Apache httpd "Server:" HTTP header

One of the HTTP headers that the Apache httpd sends back with response data is "Server". For example, my web server machine is relatively up-to-date Arch Linux. It sends back headers closely ...
user avatar
38 votes
3 answers
55k views

What tool can I use to sniff HTTP/HTTPS traffic?

I'm looking for a command line tool that can intercept HTTP/HTTPS requests, extract information such as: (content, destination, etc.), perform various analysis tasks, and finally determine if the ...
slm's user avatar
  • 372k
36 votes
7 answers
35k views

Download big file over bad connection

Is there an existing tool, which can be used to download big files over a bad connection? I have to regularly download a relatively small file: 300 MB, but the slow (80-120 KBytes/sec) TCP connection ...
Crouching Kitten's user avatar
30 votes
4 answers
8k views

How do I play an HTTP music stream over SSH?

I know I can just do mplayer <stream> to open and play a stream on the local machine. However, I'm behind a firewall. Instead, I'd like to open the stream on a remote machine, but still play it ...
Anko's user avatar
  • 4,556
21 votes
3 answers
11k views

Can't access select https sites on Linux over PPPoE

My internet connection used to be a direct LAN connection to my provider. Back then, everything would load fine on both Windows and Ubuntu (dual boot). However, a while ago they started needing me to ...
Mussnoon's user avatar
  • 353
20 votes
4 answers
65k views

How am I able to telnet to HTTP port 80?

After searching in google I found out that we can telnet to web-server to its http port and use GET to retrieve a html page. For ex: $ telnet web-server-name 80 But I am not able to understand how ...
sps's user avatar
  • 1,436
19 votes
2 answers
28k views

Sync with a Directory Listing of Web Server

Is there an easy way to keep a folder synced with a directory listing via HTTP? Edit: Thanks for the tip with wget! I created a shell script and added it as a cron job: remote_dirs=( "http://...
Lenar Hoyt's user avatar
18 votes
2 answers
19k views

Monitor outgoing web requests as they’re happening

I want to see a list of all outgoing HTTP requests from my desktop. I think it should be possible to monitor HTTPS hostnames as well for local clients using Server Name Indication (SNI). OS X has a ...
Daniel's user avatar
  • 1,207
17 votes
3 answers
30k views

Mount HTTP Server As File System

I have a machine on which I wish to mount multiple remote servers to access them all centrally. For remote Linux based systems I am using SSHFS which works fine. But for Windows systems, or systems ...
Baldrick's user avatar
  • 7,732
14 votes
3 answers
14k views

Is there any command-line, generic HTTP proxy (like Squid)?

I can easily use Netcat (or, Socat) to capture traffic between my browser and a specific host:port. But for Linux, does there exist any command-line counterpart of a Squid-like HTTP proxy that I can ...
Harry's user avatar
  • 812
13 votes
3 answers
15k views

How to get http server response in case of 401 using wget?

I need to test my http server responses in various cases, even for failed authentication. In case of failed authentication, my server returns 401 Unauthorized and also a response body which simple ...
Joe's user avatar
  • 759
12 votes
2 answers
5k views

Wget returning binary instead of html?

I am using wget to download a static html page. The W3C Validator tells me the page is encoded in UTF-8. Yet when I cat the file after download, I get a bunch of binary nonsense. I'm on Ubuntu, and ...
ΓΓIICK's user avatar
  • 123
11 votes
2 answers
3k views

How can I use files from HTTP as prerequisites in GNU make?

I want to use files from the World Wide Web as prerequisites in my makefiles: local.dat: http://example.org/example.gz curl -s $< | gzip -d | transmogrify >$@ I only want to "transmogrify" ...
pipe's user avatar
  • 925
11 votes
3 answers
32k views

Download Windows ISO from microsoft using wget or curl

Objective: download the official Win10_1909 iso into linux directly via command line Source: https://www.microsoft.com/en-in/software-download/windows10ISO After selecting edition and language you ...
Curi0usM3's user avatar
  • 123
9 votes
1 answer
43k views

What ports need to be open on a firewall to access the internet?

Let's say as the example that I have a firewall that blocks ALL ports from all sources/destinations. What ports would I need to open to be able to successfully run: ping google.com ...and are there ...
Cestarian's user avatar
  • 2,071
9 votes
1 answer
4k views

Display webpage with unix domain socket

Is there a way to serve a webpage from a locally running tcp server listening on a unix domain socket instead of localhost:<port>? something like: file:///tmp/webpage.sock my only real ...
Alexander Mills's user avatar
8 votes
2 answers
1k views

Are there any good tools besides SeleniumRC that can fetch webpages including content post-painted by JavaScript?

One major shortcoming of curl is that more and more wepages are having their main piece of content painted by a JavaScript AJAX response that occurs after the initial HTTP response. curl never picks ...
dan's user avatar
  • 4,097
7 votes
4 answers
21k views

Using telnet to get website header

I have question, is it possible to get header of website by using telnet? Website looks like this domain.name.server.com/~USER (just example). And I want to get it header by telnet. telnet domain....
MyUnix's user avatar
  • 71
7 votes
1 answer
17k views

Wget: downloading files selectively and recursively?

Question about wget, subfolder, and index.html. Let's say I am inside "travels/" folder and this is in "website.com": "website.com/travels/". Folder "travels/" contains a lot of files and other (sub)...
T. Caio's user avatar
  • 129
7 votes
1 answer
8k views

IPtables - new vs new, established, related

Let's take an example, let's assume that I would allow only outgoing HTTP[S] traffic to pass through the firewall. I know I would also need to allow outgoing DNS traffic in a real world scenario, but ...
BufferOverflow's user avatar
6 votes
3 answers
33k views

How do I get a URL over HTTP with netcat?

Can anybody tell me why I am getting bad request while executing this command echo -e "GET http://www.yellowpages.com.eg/Mjg3NF9VUkxfMTEwX2h0dHA6Ly93d3cubG90dXMtYWlyLmNvbV8=/Lotus-Air/profile.html ...
Madhur Ahuja's user avatar
  • 1,499
6 votes
2 answers
9k views

How to access http server from bash script with existing tcp connection?

In a bash shell scipt, I use several command line tools (wget, curl, httpie) to test my http server. When using e.g. curl to invoke a GET request, I see that a tcp connection is opened to my server ...
Joe's user avatar
  • 759
6 votes
1 answer
17k views

logrotate every 2 hours

I'm trying to make httpry logs every 2 hours. I added this to my logrotate.conf file: /var/log/httpry/*.log { rotate 1 weekly create 640 root admin missingok notifempty } I tried ...
o.o's user avatar
  • 173
6 votes
0 answers
1k views

Using socat to Tunnel/Proxy TCP want to split send/receive into separate files

I'm using socat to dump plain TCP (e.g. HTTP) traffic: socat -v tcp-listen:$PORT,reuseaddr,fork,keepalive tcp:$TARGET_HOST:$TARGET_PORT At the moment all send and receive traffic is dumped into ...
Amin Abbaspour's user avatar
5 votes
2 answers
4k views

How can I see dumps of wholе HTTP packets?

I need to see whole HTTP packets sent and recieved by an application for debugging purposes. How can this be done in command-line?
Eugene Yarmash's user avatar
5 votes
3 answers
679 views

Send first line of the HTTP Header

A normal HTTP request may look something like this: GET / HTTP/1.1 Host: example.com I would like to change the first line of this request, with a custom one, e.g.: FOOBAR NOT REALLY HTTP Host: ...
Tyilo's user avatar
  • 6,041
5 votes
1 answer
28k views

How to decompress gzipped HTTP response?

File req contains the request header: GET /cd/E11882_01/server.112/e41084/toc.htm HTTP/1.1^M Host: docs.oracle.com^M Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8^M Accept-...
Utku's user avatar
  • 1,433
5 votes
1 answer
905 views

Map file to URL

I'm looking for a way to do something like this: some_tool --go-to-background http://server.com/~me/uber_vimrc ~/.vimrc After which, reading ~/.vimrc would give the data downloaded from that URL. ...
hyde's user avatar
  • 1,288
5 votes
1 answer
5k views

Why wget does not work via ssh tunnel? What does proxy prevent ssh-client to do?

To test things I'm trying to establish a local ssh tunnel from my laptop to a site via ssh-server, and download a page of the site (or view it in a browser). The tunnel is made like this: $ ssh -L ...
xealits's user avatar
  • 2,163
5 votes
5 answers
23k views

wget is returning an html page instead of original file

In my case the problem is with SourceForge.net. Actually I was trying to download these files: http://sourceforge.net/projects/wxpython/files/wxPython/2.9.4.0/wxPython-src-2.9.4.0.tar.bz2/download ...
Santosh Kumar's user avatar
5 votes
1 answer
423 views

What tools allow me to present man pages as formatted HTML on a web server?

What tools exist to make man pages available on a system available via HTTP and link topics together such that references in the SEE ALSO sections and elsewhere will become hyperlinks in the HTML ...
0xC0000022L's user avatar
  • 16.7k
5 votes
3 answers
12k views

Why isn't Android's Captive Portal detection triggering a browser window?

I have a Raspberry Pi that is hosting a simple website using nginx. The RPi is acting as a Wireless Access Point - users can connect to its wireless network, the RPi gives them an IP (it runs a DHCP ...
Tal's user avatar
  • 2,172
5 votes
1 answer
20k views

What's the difference between using netcat (nc) and curl for HTTP requests?

I'm using curl to request a specific URL and getting 200 OK response: curl -v www.youtypeitwepostit.com * About to connect() to www.youtypeitwepostit.com port 80 (#0) * Trying 54.197.246.21... * ...
heytar's user avatar
  • 53
5 votes
1 answer
14k views

How to use authentication with negiotiation (e.g. Kerberos) to HTTP proxy?

Generally accepted is the use of HTTP_PROXY/HTTPS_PROXY environment variables to specify the use of a proxy server. Authentication can be included in this URL, e.g. HTTP_PROXY=http://user:pass@myproxy....
gertvdijk's user avatar
  • 14k
4 votes
3 answers
37k views

echo text after curl command on same line

#!/bin/bash echo "$(curl -s -I https://google.com|grep Server)" abc returns: abcer: gws and bash -x shows: ++ grep Server ++ curl -s -I https://google.com ' abco 'Server: gws abcer: gws but ...
w00t's user avatar
  • 151
4 votes
3 answers
6k views

Simple server that triggers script and responds

I need to have a server running that can: Take rest call. It triggers a script when a call is made The script checks if database is running or not. If running then reply back to client as Success ...
Jatin's user avatar
  • 141
4 votes
4 answers
19k views

How to change user agent on all http requests made from my machine with Squid intercepting proxy?

I'd like to change the user agent on all HTTP requests made from this machine. It's easy enough to change the user agent in each browser, but other apps also make http requests. It's vital I catch ...
Colonel Panic's user avatar
4 votes
1 answer
152 views

Getting a website's ip with 'host'?

Why is it that when I run the command host examplewebsite.com it gives me website's ip, but for some websites, when I type that IP address into the address bar and hit enter, it shows me a 404 page? ...
user1091856's user avatar
4 votes
3 answers
2k views

How can I list every port serving a web page

I have a number of servers, all with various HTTP(S) services accessible on a variety of ports. It's incredibly difficult to keep track of which service is available under which host and what port. ...
chris6654's user avatar
4 votes
2 answers
8k views

Stream system audio through mpd

I use mpd to stream music to my phone (connected to Hi-Fi) via http. Yet, this only works for songs in my mpd database. I oftentimes want to play stuff in my browser and would like that to be ...
Profpatsch's user avatar
  • 1,799
4 votes
1 answer
12k views

How to configure pfsense to NAT servers?

How can I configure pfsense so it will NAT servers so they can be accessed outside of the company? I tried to play around with the NAT settings, but didn't get anywhere. I want to be able to access ...
user avatar
4 votes
1 answer
15k views

How to set up a localhost server with http protocol on apache [closed]

I'm novice in linux world and I'm french, I don't have a good English. I want to create a data platform with a table in an intranet network! So, How to set up localhost server with http protocol on ...
Micael André's user avatar
4 votes
1 answer
2k views

OpenBSD httpd.conf conditionals

Note: this question is about the built-in OpenBSD http server named httpd and its configuration. It does not apply to any other web servers. Is it possible to make runtime conditional configurations ...
Bex's user avatar
  • 758
4 votes
3 answers
9k views

How to make wget download recursive combining --accept with --exclude-directories?

I'm trying to download some directories from an Apache server, but I need to ignore some directories that have huge files I don't care about The dir structure in the server is somewhat like this (...
elias's user avatar
  • 532
4 votes
1 answer
2k views

mplayer not sending the correct user-agent

I am trying to play a stream through mplayer. Here is the command I execute: mplayer -user-agent "PS4 libhttp/1.60" -cache 8192 -v ffmpeg://http://the_stream_url.m3u8 The stream does not play due to ...
Vincent Savard's user avatar
4 votes
1 answer
2k views

How to make an HTTP request from raw data?

I have some HTTP requests in raw format such as GET /docs/index.html HTTP/1.1 Host: www.nowhere123.com Accept: image/gif, image/jpeg, */* Accept-Language: en-us Accept-Encoding: gzip, deflate User-...
Emiliano S.'s user avatar

1
2 3 4 5
7