6

I know how to user fail2ban and how to configure a jail, but I'm not comfortable about how it actually works.

The thing is, there's a particular jail option that pique my curiosity: findtime.

When I configure a filter, it is necessary to use the HOST keyword (match IP address), so that fail2ban can know the IP to compare and ban if necessary. Alright.

But there's no such thing for time: fail2ban can't know the exact time a line was added to a log file, because there's no TIME keyword, right? Actually, it can scan files without any time on any line and it will still work.

I guess it means fail2ban is scanning files periodically: it set a scan time internally so it can handle options like findtime by comparing its own scan dates.

First, am I right? If so, what is the scan frequency? Can't it be a bottleneck if there are lots of big log files to scan often?

Then, what happened if the scan frequency is superior to the findtime options? Does it means fail2ban adapts to the minimal findtime option it found to set its minimal scan frequency?

  • Which log files are we talking about? Most log files will have some sort of timestamp for each entry. dmesg is the only log that I know of that doesn't have one. – Bratchley Jan 16 '15 at 19:47
8

First off. This is (perhaps) not an answer but perhaps better then a comment (and a bit long for it).

Time stamps

Find your statement:

Actually, it can scan files without any time on any line and it will still work.

conflicting with the documentation. What do you mean by work?

The manual#filters (v 0.8) states:

If you're creating your own failregex expressions, here are some things you should know:

  • [...]

  • In order for a log line to match your failregex, it actually has to match in two parts: the beginning of the line has to match a timestamp pattern or regex, and the remainder of the line has to match your failregex. If the failregex is anchored with a leading ^, then the anchor refers to the start of the remainder of the line, after the timestamp and intervening whitespace.

  • The pattern or regex to match the time stamp is currently not documented, and not available for users to read or set. See Debian bug #491253. This is a problem if your log has a timestamp format that fail2ban doesn't expect, since it will then fail to match any lines. Because of this, you should test any new failregex against a sample log line, as in the examples below, to be sure that it will match. If fail2ban doesn't recognize your log timestamp, then you have two options: either reconfigure your daemon to log with a timestamp in a more common format, such as in the example log line above; or file a bug report asking to have your timestamp format included.

Note here that log files can be configured to include time stamps as well as the format of the time stamps. (That include dmesg as mentioned in comment.)

Also see this thread, Message #14 and #19 in particular:

Two examples:

Note that you can also test with commands like:

fail2ban-regex /var/log/auth.log /etc/fail2ban/filter.d/sshd.conf

1 No time stamp:

$ fail2ban-regex ' [1.2.3.4] authentication failed' '\[<HOST>\] authentication failed'

Running tests
=============

Use   failregex line : \[<HOST>\] authentication failed
Use      single line :  [1.2.3.4] authentication failed


Results
=======

Failregex: 0 total

Ignoreregex: 0 total

Date template hits:

Lines: 1 lines, 0 ignored, 0 matched, 1 missed
|- Missed line(s):
|   [1.2.3.4] authentication failed
`-

2 With time stamp:

$ fail2ban-regex 'Jul 18 12:13:01 [1.2.3.4] authentication failed' '\[<HOST>\] authentication failed'

Running tests
=============

Use   failregex line : \[<HOST>\] authentication failed
Use      single line : Jul 18 12:13:01 [1.2.3.4] authentication failed


Results
=======

Failregex: 1 total
|-  #) [# of hits] regular expression
|   1) [1] \[<HOST>\] authentication failed
`-

Ignoreregex: 0 total

Date template hits:
|- [# of hits] date format
|  [1] MONTH Day Hour:Minute:Second
`-

Lines: 1 lines, 0 ignored, 1 matched, 0 missed

Scan times

Manual#Reaction time:

It is quite difficult to evaluate the reaction time. Fail2ban waits 1 second before checking for new logs to be scanned. This should be fine in most cases. However, it is possible to get more login failures than specified by maxretry.

In that regard also see this thread: Re: Bug#481265: fail2ban: Poll interval is not configurable.

But under optional but recommended software one find Gamin.

Gamin is a file alteration monitor. Gamin greatly benefits from a "inotify"-enabled kernel. Thus, active polling is no longer required to get the file modifications.

If Gamin is installed and backend in jail.conf is set to auto (or gamin) - Gamin will be used.

0

It is important to notice that the date doesn't have to be at the beginning of the line, if and only if, you let the regex "recognize" it with a wildcard.

For example: suppose that in tomcat7 logfiles are some lines like these:

1.2.3.4 - - [13/Feb/2017:02:47:44 -0300] "GET /manager/html HTTP/1.1" 401 2486
1.2.3.4 - - [13/Feb/2017:02:47:45 -0300] "GET /manager/html HTTP/1.1" 401 2486
1.2.3.4 - - [13/Feb/2017:02:47:45 -0300] "GET /manager/html HTTP/1.1" 401 2486
1.2.3.4 - - [13/Feb/2017:02:47:45 -0300] "GET /manager/html HTTP/1.1" 401 2486
1.2.3.4 - - [13/Feb/2017:02:47:45 -0300] "GET /manager/html HTTP/1.1" 401 2486
1.2.3.4 - - [13/Feb/2017:02:47:46 -0300] "GET /manager/html HTTP/1.1" 401 2486

looking at them, the date is not at the beginning, and that should be a problem.

Nonetheless, if you test it with a regex that has a "room for the date", the test will match:

$ fail2ban-regex '1.2.3.4 - - [13/Feb/2017:02:47:44 -0300] "GET /manager/html HTTP/1.1" 401 2486' '<HOST> - - \[.*\] "GET .* HTTP/1.1" 40\d \d+$'

Running tests
=============

Use regex line : <HOST> - - \[.*\] "GET .* HTTP/1.1" 40\d \d+$
Use single line: 1.2.3.4 - - [13/Feb/2017:02:47:44 -0300] "GET /man...

Matched time template Day/MONTH/Year:Hour:Minute:Second
Got time using template Day/MONTH/Year:Hour:Minute:Second

Results
=======

Failregex: 1 total
|- #) [# of hits] regular expression
|  1) [1] <HOST> - - \[.*\] "GET .* HTTP/1.1" 40\d \d+$
`-

Ignoreregex: 0 total

Summary
=======

Addresses found:
[1]
    1.2.3.4 (Mon Feb 13 02:47:44 2017)

Date template hits:
2 hit(s): Day/MONTH/Year:Hour:Minute:Second

Success, the total number of match is 1

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.