I'm being trolled by China, and don't know why I can't block their request to my server.


ALL: item.taobao.com
ALL: 117.25.128.*

But when I watch the error log on my webserver tail -f /var/log/apache2/error.log the requests are still being allowed through.

Question: Why isn't my /etc/hosts.deny config working?

  1. The file is called /etc/hosts.deny, not host.deny

  2. Not all services use tcp-wrappers. sshd, for example, doesn't by default. Neither does apache.

  3. You can use iptables to block all packets from 117.25.128/24, e.g.:

     iptables -I INPUT -s -j DROP
  4. Even better, you can use fail2ban to monitor a log file (such as apache's access.log and/or error.log) and automatically block IP addresses trying to attack your server.

From the debian fail2ban package description:

Fail2ban monitors log files (e.g. /var/log/auth.log, /var/log/apache/access.log) and temporarily or persistently bans failure-prone addresses by updating existing firewall rules.

Fail2ban allows easy specification of different actions to be taken such as to ban an IP using iptables or hosts.deny rules, or simply to send a notification email.

By default, it comes with filter expressions for various services (sshd, apache, qmail, proftpd, sasl etc.) but configuration can be easily extended for monitoring any other text file. All filters and actions are given in the config files, thus fail2ban can be adopted to be used with a variety of files and firewalls.

  • Ok yea I'll edit the tile. Ah okay, well I tried iptables -A INPUT -s -j DROP but that didn't block it either. – Jordan Davis Nov 17 '15 at 4:10
  • 1
    -I INPUT inserts the new rule at the top of the INPUT chain. -A INPUT adds it to the bottom of the chain. if -A INPUT didn't work, there must have been some other rule earlier in the chain which allowed those packets. – cas Nov 17 '15 at 4:11
  • ahhh okay, I'm assuming precedence is top to bottom then? – Jordan Davis Nov 17 '15 at 4:12
  • yep. first matching rule wins. – cas Nov 17 '15 at 4:13
  • Perfect it looks to be working! Thank you. Quick question though so alot of the requests are coming from the same refer ie. (item.taobao.com) with different client IP's, so I'm assuming they're using some type of redirect. Is there a way to block by referer, or block by 117.* using like a wildcard selector? – Jordan Davis Nov 17 '15 at 4:30

As for your original question. My Apache in debian is not configured with libwrap, so it won't consult hosts.deny. [the previous answer already mentions it - the reality is that tcpwrappers is not the epitome of security as it was in the 90s, specially when it comes to blacklisting]. You would have to run it not as a daemon, but from (x)inetd, which would slow it down (considerably).

You can block/allow access at Apache level, and do not need tcp wrappers for Apache [nor iptables for that matter]. You have not mentioned ssh, however I never leave ssh servers open to the outside directly. However keep reading on.

We have a vhost with 300+ domains, and also similar problems, with taobao, baidu, and often even with google spiders. Notably baidu spiders can be quite aggressive and persistent.

As you have already figured out, they have farms of servers, and even if you block an IP they will appear again shortly from some other IP addresses.

It is not practical at all to try and maintain lists of IP addresses/netblocks by hand.

What it works for us rather well is modsecurity blocking user agent strings permanently while mod_evasive is blocking temporarily IPs that are being abusive.

This setup, besides slowing down spiders from search engines, also has the advantage of throttling down zombies trying to guess passwords on CMSes.

The relevant part of our modsecurity.conf

SecRule REQUEST_HEADERS:User-Agent "Yandex" phase:1,deny,nolog,id:'6972'
SecRule REQUEST_HEADERS:User-Agent "ichiro" phase:1,deny,nolog,id:'6973'
SecRule REQUEST_HEADERS:User-Agent "Baiduspider" phase:1,deny,nolog,id:'6974'
SecRule REQUEST_HEADERS:User-Agent "Baiduspider/.*" phase:1,deny,nolog,id:'6975'
SecRule REQUEST_HEADERS:User-Agent "Baiduspider-video" phase:1,deny,nolog,id:'6976'
SecRule REQUEST_HEADERS:User-Agent "Baiduspider-image" phase:1,deny,nolog,id:'6977'
SecRule REQUEST_HEADERS:User-Agent "sogou spider" phase:1,deny,nolog,id:'6978'
SecRule REQUEST_HEADERS:User-Agent "YoudaoBot" phase:1,deny,nolog,id:'6979'
SecRule REQUEST_HEADERS:User-Agent "bingbot(at)microsoft.com" phase:1,deny,nolog,id:'6980'
SecRule REQUEST_HEADERS:User-Agent "msnbot(at)microsoft.com" phase:1,deny,nolog,id:'6981'
SecRule REQUEST_HEADERS:User-Agent "BLEXBot/1.0" phase:1,deny,nolog,id:'6982'
SecRule REQUEST_HEADERS:User-Agent "Bot.*" phase:1,deny,nolog,id:'6984'
SecRule REQUEST_HEADERS:User-Agent "AhrefsBot.*" phase:1,deny,nolog,id:'6985'

And our mod-evasive.conf

DOSHashTableSize 2048
DOSPageCount 10
DOSSiteCount 300
DOSPageInterval 2.0
DOSSiteInterval 1.0   
DOSBlockingPeriod 600.0   
DOSLogDir /var/log/apache2/evasive
DOSWhitelist 1xx.xxx.xxx.xx

I also forgot a very real possibility. If you do not deal with China or are running your home server, just block the whole country. The level of attacks and malware that comes from them has justified many professionals to do that.


I also forgot to add to this rather lengthy answer a footnote. Often people suggest me at work to use robots.txt for these kind of problems. The point is robots.txt is only a suggestion for remote programs.Rogue actors certainly ignore them, and it is not guaranteed other web crawlers honor them nowadays. From our tests, for instance, Baidu seems to not honor them. (robots.txt it tantamount to ask a gangster, please tickle me instead of punching me)

  • Thanks man! yea I'm def going to use mod_evasive, those configure file settings work good for you? – Jordan Davis Nov 17 '15 at 5:54
  • They work quite well, most of our vhosts do not have much visitors. Otherwise you have to increase dospagecount (max visits to a single page from an IP), and dossitecount (max visits of and IP) accordingly. – Rui F Ribeiro Nov 17 '15 at 6:08
  • Look to my first lines again please. – Rui F Ribeiro Nov 17 '15 at 6:11
  • Got it makes sense, I'll read up on the settings. Thanks again for all your help! – Jordan Davis Nov 17 '15 at 6:14
  • (do not take it wrongly, just suggesting blocking addresses in web server or Apache could be a better title)...nevertheless, it was not for this comment that I came here again, read the answer again please. Besides comments in the beggining, you will find something more interesting in the end. – Rui F Ribeiro Nov 17 '15 at 6:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.