2

I have several questions about the wget tool in GNU/Linux:

# wget http://www.jdfschool.com

--2019-04-05 02:33:44-- (Trys: 3) http://www.jdfschool.com/
Connecting www.jdfschool.com|115.28.223.13|:80... Connected.
An HTTP request has been issued, waiting for a response... Connection reset by peer.
Retrying.
  1. Does the Connected in Connecting www.jdfschool.com|115.28.223.13|:80... Connected. represent a successful TCP connection?

  2. You see the Connection reset by peer in the log, are there any more detailed logs of it? I can't know the reason of the reset issue.

1

1 Answer 1

1

As you correctly assume, 80... Connected means a (web) service is listening on port 80, and you were able to connect to it.

If the site is working well when using a browser (and also from other IP address) connection reset by peer only gives you a feedback whatever you are trying to do, does not somewhat work. If you want more details, you can try to sniff the connection.

However, short of asking the owner of site for logs/debugging, I pretty much doubt you will get more data about the reasons behind it being reset otherwise.

As for the causes, it might be that they have rules to not accept your user agent/spiders/have specific pages/or configurations that trow purposefully that error or also block you after n tries/pages in a defined time frame (defined by them).

As commented before, this is more related to specific site security measures and configurations than a Unix issue per se.

For listening to the actual HTTP requests, you can also run in your machine:

# ngrep -q "." "port 80"

or

# ngrep -q "." "port 80 and host www.jdfschool.com"

Mind you that depending also on configurations, HTML, DNS and infra-structure setup, it is not a given the second ngrep will listen to all HTTP requests.

As @muru states in comments, using

wget -v http://www.jdfschool.com

also might output more useful data.

As talked before, the odds of pinpoint exactly why is happening is slim. (In Apache web servers it is common to setup mod_evasive and mod_security to curb on spiders/abusive behaviour, for instance)

TLDR From the point of view of the site owner, what is happening to your command might well be intended behaviour.

3
  • 1
    More verbose output wget -v may help to see at what stage the connection was closed, maybe. (Thanks, by the way!)
    – muru
    Apr 4, 2019 at 6:58
  • 2
    wget -v get no more logs.
    – 244boy
    Apr 4, 2019 at 8:24
  • wget -v is the default, so adding the -v wont change anything. wget -d is supposed to print debugging info, but didnt for me. i wound up just using curl Dec 2, 2020 at 13:47

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .