fail2ban log parsing too slow on Raspberry Pi - options?

Question

I'm running fail2ban on a Raspberry Pi at 950MHz which I cannot overclock further.

The Pi is occasionally subject to SYN floods on particular ports. I've set up iptables to throttle the rate of SYNs on the port of interest; when the throttle limits are exceeded, hosts which send SYNs are dropped into the REJECT chain and the particular SYN packet which exceeded the limit is logged.

fail2ban then watches for these logged SYNs and, after seeing a few, temporarily bans the host for a short time (this is a transient issue in the app I'm working with).

The problem is that the SYN floods can occasionally reach rates which are too fast for fail2ban to keep up with; I'll see 20-40 log messages per second, and eventually fail2ban falls behind and becomes ineffective. To add insult to injury, it continues consuming a LOT of CPU as it tries to catch up.

I have verified that DROP chained packets from hosts already banned by fail2ban are not logged, and thus do not add to its load.

What are my options here? I have a few ideas, but no clear path forward.

1. Could I make the log-parse regex "easier" so it takes fewer cycles? Would using iptables --log-prefix to put a token near the start of the log message, and/or otherwise simplifying/altering the fail2ban regex help? Here is the current fail2ban config line containing a regex:
   failregex = kernel:.*?SRC=(?:::f{4,6}:)?(?P<host>[\w\-.^_]+) DST.*?SYN

2. Is there a faster way for fail2ban to watch for the packets exceeding the limits than parsing kern.log?

3. Could fail2ban be run under PyPy instead of CPython with minimal nonstandard wizardry (the OS is Raspbian 7, so, mostly Debian 7)?

4. Is there something better than fail2ban that I could use to watch for the packets which exceed the SYN limits, and after N exceeds in X seconds, temporarily put the offending IP into the iptables DROP bucket, and take it out when the ban timer expires? Again, I'd vastly prefer a solution that uses as much software available in Debian as possible, though I can build Debian packages in a pinch.

Answer 1

Slightly off-topic to your question, but SYN cookies might be a suitable and simple mitigation when confronted with moderate SYN flooding.

# echo 1 > /proc/sys/net/ipv4/tcp_syncookies

Google some for reasons why this might be a good or bad idea.

Alternatively, since you already have a kernel firewall configured, you could use the "recent" iptables module as limiter circumventing fail2ban entirely:

# iptables -A INPUT -p tcp -m state --state NEW -m recent --name ftp_limit --update --seconds 60 --hitcount 20 -j DROP
# iptables -A INPUT -p tcp -m state --state NEW -m recent --set --name ftp_limit -m tcp --dport 21 -j ACCEPT

These rules limit the rate of SYN requests (new connections) from one IP to 20 per minute (line 1) for IP-addresses in the list named ftp_limit and DROP any additional requests. Line 2 will add the source IP-address to the ftp_limit list whenever a new connection is being established on port 21 (ftp).

You run the risk you block legitimate traffic originating from networks behind NAT, since those appear to originate from a single ip-address. And you might want to whitelist some trusted networks as well.

Answer 2

Like I said in my comments.

Switch to CSF, it has a larger memory footprint, but if I had to choose between high ARM-CPU usage and high overall memory usage, I would choose the latter.

Other alternative would be to drop using any firewall software and switch to iptables rules. Move the server to nonstandard port (8030, 24544, etc.) and use iptables to drop connections on all other ports.

If you're behind NAT, you could use your router to disable portforwarding on all ports except the one for the webserver (assuming you set it up to listen on other port).