All about programs and services that use the Hypertext Transfer Protocol.

learn more… | top users | synonyms

4
votes
2answers
3k views

How to make wget download recursive combining --accept with --exclude-directories?

I'm trying to download some directories from an Apache server, but I need to ignore some directories that have huge files I don't care about The dir structure in the server is somewhat like this (...
1
vote
1answer
35 views

How to decompress gzipped HTTP response?

File req contains the request header: GET /cd/E11882_01/server.112/e41084/toc.htm HTTP/1.1^M Host: docs.oracle.com^M Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8^M Accept-...
2
votes
3answers
4k views

How do I get a URL over HTTP with netcat?

Can anybody tell me why I am getting bad request while executing this command echo -e "GET http://www.yellowpages.com.eg/Mjg3NF9VUkxfMTEwX2h0dHA6Ly93d3cubG90dXMtYWlyLmNvbV8=/Lotus-Air/profile.html ...
1
vote
0answers
33 views

cURL: Java website not loading

I can access the configuration page of my device with no problems via web browser (on the GUI). But it doesn't seem to work by using cURL: luis@Fresoncio:~$ curl http://192.168.1.232 <html><...
3
votes
1answer
35 views

Storing stderr output in a variable without modifing stdout, and doing so repeatedly

there are many "almost duplicate" questions, but none gave me the answer I'm looking for, so bear with me. consider a simple server API, in which you can consume data in iterations, using a "...
1
vote
1answer
298 views

Grab http requests by lynx

I need to grab and debug all HTTP-requests and responds which i make from lynx or another text-based browser. How can i save all request url's, redirects, headers and page sources for analyze and ...
0
votes
1answer
49 views

iptables redirect via url to localhost vhost

I'm trying to set up a test environment for a mobile app. I have a machine acting as a gateway and providing dhcp on 192.168.10.0/24, and serving pages from apache. How would I set up iptables rules ...
1
vote
0answers
11 views

DD WRT restrict access but exclude subdomain

I know it's possible to restrict access to some websites in Access restrictions section. I'd like to know if it possible to exclude from restriction subdomain of a site. For example: http://...
0
votes
2answers
48 views

Can not change HTTPS to HTTP on Nginx

My problem is described on the title of this topic. I have a web server running by nginx. I registered a free domain name .tk and installed Let's Encrypt certificate. The following configuration is ...
0
votes
0answers
31 views

How do I use imagemagik with URL locations?

The command below can be used to genereate a gif animation from a bunch of files stored locally: convert -delay 10 -loop 0 *.png animation.gif Is there a way to tell convert to wget the images from ...
26
votes
7answers
16k views

Can I create override dns similar to writing in /etc/hosts without root access

I want to set a dns record that my browser will use, but I don't have root access, so I can't modify /etc/hosts. I need to do this for testing vhosts with apache, whose dns hasn't yet been set up. I ...
5
votes
2answers
4k views

Sync with a Directory Listing of Web Server

Is there an easy way to keep a folder synced with a directory listing via HTTP? Edit: Thanks for the tip with wget! I created a shell script and added it as a cron job: remote_dirs=( "http://...
0
votes
2answers
64 views

How to get a correct error code from curl when transfer was closed before getting the whole file

I have a bash-script which calls curl using a proxy. Today there is a problem with the download location. Calling curl ends up with: curl: (18) transfer closed with 7512708716 bytes remaining to ...
1
vote
2answers
407 views

Why are HTTP operations usually written in all caps? [closed]

I'm looking for the history of this. Why do we usually write HTTP POST or OPTIONS operations in all caps?
0
votes
0answers
108 views

Google Drive HTTP Error when downloading .zip file with Firefox

I'm using openSUSE 42.1 and opened a fresh install of Firefox, but somehow when I try to download a zip file from my Google Drive, an error is shown: Could not download "[filename]". HTTP error. ...
1
vote
0answers
56 views

python weather script not working [closed]

i am using archlinux with openbox. I created a pipe menu with a python script to get the weather from yahoo. at the begining it worked fine, but then it stopped working. the python file: http://...
3
votes
1answer
111 views

What ports need to be open on a firewall to access the internet?

Let's say as the example that I have a firewall that blocks ALL ports from all sources/destinations. What ports would I need to open to be able to successfully run: ping google.com ...and are ...
0
votes
1answer
55 views

Dose iptables NAT overwrite user agent of HTTP headers?

If i use iptables to NAT all HTTP traffic behind another network, does it overwrite user agent of HTTP headers? The rules are as follows: /usr/sbin/iptables -t nat -A POSTROUTING -o eth0 -j ...
2
votes
4answers
6k views

How to change user agent on all http requests made from my machine with Squid intercepting proxy?

I'd like to change the user agent on all HTTP requests made from this machine. It's easy enough to change the user agent in each browser, but other apps also make http requests. It's vital I catch ...
39
votes
14answers
38k views

Simple command line http server

I have a script which generates a daily report which I want to serve to the so called general public. The problem is I don't want to add to my headaches maintance of a http server (e.x. apache) with ...
1
vote
2answers
43 views

How to treat remote txt file as stream?

A web service writes its output in a text file. I can fetch that log via: curl -u user:password -k https://log.my-app.com/log.2016-04-04.txt Yet this always only fetches a current "full" file even ...
-1
votes
1answer
64 views

Unable to execute shell from web server

When ever I am trying to execute php script from web server, it fails: cat test.sh echo "Hello World" [root@yumserver ~]# I am trying to build a button on webpage through which I can push the ...
0
votes
0answers
6 views

SSL of domain redirecting to plesk panel default screen

I have plesk setup on my linux server, where we have a subscription setup for a website 4ti.co.uk We are getting a strange error where if you go to the domain using https://4ti.co.uk it directs you ...
0
votes
0answers
22 views

frequently visit a website then can not continue to visit until reboot

my server is centos 7, whose firewall has been stopped. I use a spider to collect data from a website frequently(about several times per second). After some time, I find I can't continue to visit that ...
0
votes
1answer
169 views

Lots of TIME_WAIT in netstat on apache server

What would be the reason for many TIME_WAIT here? Even we have very few ESTABLISHED, SYN connection. [root@www ~]# netstat -natp | grep "210.XX.XX.XX:80" | awk '{print $6}' | sort | uniq -c 1 ...
0
votes
1answer
32 views

how to see a http reponse message using telnet command? [duplicate]

I am trying to telnet to different web servers and see what http response messages I get. I've tried telnet some_host_name 80, and then the shell says connected to some_host_name escape character is '^...
35
votes
7answers
10k views

How do I get a websites title using command line?

I want a command line program that prints the title of a website. For e.g.: Alan:~ titlefetcher http://www.youtube.com/watch?v=Dd7dQh8u4Hc should give: Why Are Bad Words Bad? You give it the ...
1
vote
1answer
56 views

Apache Web Server Load Testing ( Running on Centos6) [closed]

How can i easily test how many requests my web server ( Apache) setup can handle without dropping a connection or timing out. Is there any simple bash script or i have to use some sophisticated ...
0
votes
0answers
68 views

Scanning files over http by suricata or snort

I want to use IDS or IPS system like Snort or Suricata for the purpose of scanning files which are send by HTTP (or different way). Let's assume that files are not encrypted by the way. Any rule or ...
0
votes
1answer
303 views

can not access HTTP site with squid3

I have a problem here, i have installed squid3 but it cant access http site like http://kompas.com , but it can access https like https://google.com here is my squid.conf #acl localnet src 192.168....
2
votes
1answer
885 views

mplayer not sending the correct user-agent

I am trying to play a stream through mplayer. Here is the command I execute: mplayer -user-agent "PS4 libhttp/1.60" -cache 8192 -v ffmpeg://http://the_stream_url.m3u8 The stream does not play due ...
1
vote
1answer
47 views

Mount HTTP directory on Raspbian?

Is there a way to mount an HTTP resource in Raspbian (default Raspberry Pi OS)? I have an HTTP server hosting music, and I would like to be able to have read-only filesystem access to that music on ...
0
votes
1answer
40 views

Can't access web sites when running iptables script!

I have wrote script for protecting my machine: #!/bin/bash ssh=1.1.1.1 http='1.1.1.1 2.2.2.2' # Clear any previous rules. iptables -F # Default drop policy. iptables -P INPUT DROP iptables -P ...
1
vote
1answer
100 views

Managing iptables for specific ips and ports

I disable firewall like so: #!/bin/bash iptables -F iptables -X iptables -P INPUT ACCEPT iptables -P OUTPUT ACCEPT iptables -P FORWARD ACCEPT Update I also enable my firewall: #!/bin/bash ssh=x....
18
votes
4answers
23k views

Change Apache httpd “Server:” HTTP header

One of the HTTP headers that the Apache httpd sends back with response data is "Server". For example, my web server machine is relatively up-to-date Arch Linux. It sends back headers closely ...
0
votes
0answers
1k views

wget failing with 404 File not found

I have two instances of jboss running on the same server with two different ports. ( 8280 & 10580). I am facing issue while downloading a file using wget on Port 10580 , but the file is getting ...
0
votes
0answers
36 views

How to enable http for Synology via ssh

I have problems connecting to my Synology NAS via https, to fix it I'd like to connect to the NAS by HTTP, but it's currently disabled. Is there a way to enable HTTP connections from terminal ...
2
votes
1answer
878 views

OpenWRT redirect traffic based on requested domain/port pair

If we would have an OpenWRT router, would we have a solution for redirecting the users traffit to HTTPS if they would request a plain HTTP website based on domain name? Example: If the traffic wants ...
1
vote
2answers
82 views

99% file download

We have server under Ubuntu 12.04 with Apache HTTP 2.2 installed there. Kernel 3.2.0. Faced with weird behavior during dowloading some files. Virtualhost config: <Directory /var/www/name/*> ...
2
votes
2answers
126 views

Find IP owners from apache log file

I have the top 10 IPs making the most requests and the number of requests made from Apache log file example. I used this bash command to handle a request: cat access_log|cut -d' ' -f1|uniq -c|sort -r|...
0
votes
2answers
75 views

Apache map a port to a public directory

I have this directory in a vm /www/app1 and I want to be able to go to http://192.168.1.1:7778 and go to that directory how do I configure that using CentOS 7 minimal install? I just went through ...
1
vote
1answer
189 views

GET URI from webserver by IP address not matching hostname's IP

Using curl, I would like to request a URI from a web server via a specific IP address that does not necessarily match the forward DNS of the hostname. For example, suppose the hostname www.example....
0
votes
1answer
98 views

Download files from the file with same path directory

I need a simple command/script which allows me to download hundreds of files from the file in the same path structure as on the remote server. For example: http://download.com/path/to/the/folder/...
9
votes
2answers
5k views

What tool can I use to sniff HTTP/HTTPS traffic?

I'm looking for a command line tool that can intercept HTTP/HTTPS requests, extract information such as: (content, destination, etc.), perform various analysis tasks, and finally determine if the ...
0
votes
2answers
204 views

How to get the contents of a webpage but after running javascript?

I am trying to download the contents of a web page using curl but it seems l am missing some content that l suspect to be generated by JavaScript. If the page runs on the browser l am able to get all ...
0
votes
1answer
306 views

Puppet Master: TCPServer Error: Address already in use - bind(2) - WEBrick 1.3.1

I rebooted Puppet Master several times but always this error [root@puppet-master04 ~]# service httpd restart Stopping httpd: [ OK ] Starting httpd: ...
4
votes
4answers
3k views

how am i able to telnet to http port 80

After searching in google I found out that we can telnet to web-server to its http port and use GET to retrieve a html page. For ex: $ telnet web-server-name 80 But I am not able to understand ...
0
votes
2answers
25 views

try download a file from server if the pathname is known

I access a webpage by this url http://someRandomName.no.meaning:20025/hello.html The location of the hello.html that is displayed is /var/some/fulder/hello.html. (I mean in the server it is /var/...
6
votes
3answers
1k views

Using telnet to get website header

I have question, is it possible to get header of website by using telnet? Website looks like this domain.name.server.com/~USER (just example). And I want to get it header by telnet. telnet domain....
6
votes
1answer
1k views

IPtables - new vs new, established, related

I'm «pretty» new to IPtables, or I have used IPtables a few years, but never with advanced features. I understand the concept of stateful IPtables rules, but recently I discovered that some people ...