Is there a special set of permissions that must be set on a log file read by fail2ban?

Question

I've experienced some spam recently on my Minecraft server that I suspect to be from some kind of port-scanner or other bot. So far whatever it is hasn't posed any real threat but I figured better safe than sorry. I've been trying to setup fail2ban to ban any IPs from which 3 or more failed connections attempts originate. I've seen some success from other people on Reddit using the same tool so I thought I'd give it a shot.

However, I'm kind of stuck. Whenever I try to start up the fail2ban service, it fails with the following error(s):

ERROR   Failed during configuration: Have not found any log file for minecraft jail
ERROR   Async configuration of server failed

Some googling reveals this is typically because the path referenced by the logpath config option in the respective jail.conf file for this "minecraft" either isn't valid or points to a non-existent file. I've checked and double checked and can confirm that the path I've specified does in fact exist so I'm thinking it's something else.

If I disable the "minecraft" unit via its config file, start the fail2ban systemd service, re-enable the "minecraft" unit, and reload the fail2ban server using the fail2ban-client command, I get the following error instead:

[root@fedora ~]# fail2ban-client reload
2023-06-30 21:37:28,314 fail2ban                [4565]: ERROR   NOK: (13, 'Permission denied')
[Errno 13] Permission denied: '/path/omitted.log'

Hmm, okay... maybe the permissions are bad? Just for the sake of ensuring it's not the permissions getting in the way, I made a temporary test file to use in the /tmp directory with the following permissions:

[root@fedora ~]# ll -d /tmp
drwxrwxrwt. 29 root root 640 Jun 30 22:10 /tmp
[root@fedora ~]# ll /tmp/testing.log 
-rwxrwxrwx. 1 root root 0 Jun 30 21:29 /tmp/testing.log

Repeating the reload process from above shows no dice... what am I missing? Anybody have any ideas for me to try out? Any help is much appreciated! Please let me know if I need to provide any more details.

Below is the log output from the fail2ban server (with DEBUG logging enabled) after doing the reload process described above:

[root@fedora ~]# tail -n 50 /var/log/fail2ban.log 
<output removed for brevity>
2023-06-30 21:37:28,186 fail2ban.server         [4553]: INFO    Start Fail2ban v1.0.2
2023-06-30 21:37:28,187 fail2ban.server         [4553]: INFO    Changed logging target to /var/log/fail2ban.log for Fail2ban v1.0.2
2023-06-30 21:37:28,187 fail2ban.ipdns          [4553]: DEBUG   IPv6 is auto
2023-06-30 21:37:28,187 fail2ban.jail           [4553]: INFO    Creating new jail 'minecraft'
2023-06-30 21:37:28,311 fail2ban.jail           [4553]: DEBUG   Backend 'pyinotify' failed to initialize due to No module named 'pyinotify'
2023-06-30 21:37:28,312 fail2ban.jail           [4553]: DEBUG   Backend 'gamin' failed to initialize due to No module named 'gamin'
2023-06-30 21:37:28,312 fail2ban.jail           [4553]: INFO    Jail 'minecraft' uses poller {}
2023-06-30 21:37:28,312 fail2ban.filter         [4553]: DEBUG   Setting usedns = warn for FilterPoll(Jail('minecraft'))
2023-06-30 21:37:28,312 fail2ban.filter         [4553]: DEBUG   Created FilterPoll(Jail('minecraft'))
2023-06-30 21:37:28,312 fail2ban.filterpoll     [4553]: DEBUG   Created FilterPoll
2023-06-30 21:37:28,312 fail2ban.jail           [4553]: INFO    Initiated 'polling' backend
2023-06-30 21:37:28,312 fail2ban.filter         [4553]: DEBUG   Setting usedns = warn for FilterPoll(Jail('minecraft'))
2023-06-30 21:37:28,312 fail2ban.server         [4553]: DEBUG     failregex: '\\(\\/<HOST>\\:'
2023-06-30 21:37:28,313 fail2ban.filter         [4553]: INFO      maxRetry: 3
2023-06-30 21:37:28,313 fail2ban.filter         [4553]: INFO      findtime: 86400
2023-06-30 21:37:28,313 fail2ban.actions        [4553]: INFO      banTime: 2592000
2023-06-30 21:37:28,313 fail2ban.filter         [4553]: INFO      encoding: UTF-8
2023-06-30 21:37:28,313 fail2ban.server         [4553]: INFO    Reload finished.
2023-06-30 21:37:28,313 fail2ban.transmitter    [4553]: ERROR   Command ['reload', '--all', [], [['set', 'syslogsocket', 'auto'], ['set', 'loglevel', 'DEBUG'], ['set', 'logtarget', '/var/log/fail2ban.log'], ['set', 'allowipv6', 'auto'], ['set', 'dbfile', '/var/lib/fail2ban/fail2ban.sqlite3'], ['set', 'dbmaxmatches', 10], ['set', 'dbpurgeage', '1d'], ['add', 'minecraft', 'auto'], ['set', 'minecraft', 'usedns', 'warn'], ['set', 'minecraft', 'addfailregex', '\\(\\/<HOST>\\:'], ['set', 'minecraft', 'maxretry', 3], ['set', 'minecraft', 'maxmatches', 3], ['set', 'minecraft', 'findtime', '1d'], ['set', 'minecraft', 'bantime', '30d'], ['set', 'minecraft', 'ignorecommand', ''], ['set', 'minecraft', 'logencoding', 'auto'], ['set', 'minecraft', 'addlogpath', '/tmp/testing.log', 'head'], ['set', 'minecraft', 'addaction', 'firewallcmd-rich-rules'], ['multi-set', 'minecraft', 'action', 'firewallcmd-rich-rules', [['actionstart', ''], ['actionstop', ''], ['actioncheck', ''], ['actionban', 'ports="0:65535"; for p in $(echo $ports | tr ", " " "); do firewall-cmd --add-rich-rule="rule family=\'<family>\' source address=\'<ip>\' port port=\'$p\' protocol=\'tcp\' reject type=\'<rejecttype>\'"; done'], ['actionunban', 'ports="0:65535"; for p in $(echo $ports | tr ", " " "); do firewall-cmd --remove-rich-rule="rule family=\'<family>\' source address=\'<ip>\' port port=\'$p\' protocol=\'tcp\' reject type=\'<rejecttype>\'"; done'], ['port', '0:65535'], ['protocol', 'tcp'], ['chain', '<known/chain>'], ['name', 'minecraft'], ['actname', 'firewallcmd-rich-rules'], ['family', 'ipv4'], ['zone', 'public'], ['service', 'ssh'], ['rejecttype', 'icmp-port-unreachable'], ['blocktype', 'REJECT --reject-with <rejecttype>'], ['rich-blocktype', "reject type='<rejecttype>'"], ['family?family=inet6', 'ipv6'], ['rejecttype?family=inet6', 'icmp6-port-unreachable']]], ['start', 'minecraft']]] has failed. Received PermissionError(13, 'Permission denied')
Traceback (most recent call last):
  File "/usr/lib/python3.11/site-packages/fail2ban/server/transmitter.py", line 58, in proceed
    ret = self.__commandHandler(command)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/fail2ban/server/transmitter.py", line 109, in __commandHandler
    self.__commandHandler(cmd)
  File "/usr/lib/python3.11/site-packages/fail2ban/server/transmitter.py", line 89, in __commandHandler
    return self.__commandSet(command[1:])
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/fail2ban/server/transmitter.py", line 258, in __commandSet
    self.__server.addLogPath(name, value, tail)
  File "/usr/lib/python3.11/site-packages/fail2ban/server/server.py", line 382, in addLogPath
    filter_.addLogPath(fileName, tail)
  File "/usr/lib/python3.11/site-packages/fail2ban/server/filter.py", line 1006, in addLogPath
    log = FileContainer(path, self.getLogEncoding(), tail)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/fail2ban/server/filter.py", line 1327, in __init__
    handler = open(filename, 'rb')
              ^^^^^^^^^^^^^^^^^^^^
PermissionError: [Errno 13] Permission denied: '/tmp/testing.log'

Answer 1

Thanks to @roaima for sharing the link to Installing fail2ban on CentOS 7, which led me down the path to figure this out.

TL;DR:

The problem was SELinux. Disabling it, changing its access control policy, or changing where you store your log files are the only ways (I have found) to get around this.

The Solution(s):

A quick and dirty fix and/or verification of SELinux as the problem is to set SELinux's enforcement policy to "permissive" instead of "enforced". Based on the explanation provided from this Red Hat help article, setting the enforcement policy of SELinux to "permissive" effectively disables the enforcement of any access control policies. So, should an SELinux access policy be the reason the fail2ban-server process can't access a log file, disabling it should give you an idea of whether SELinux was the problem or not. This can be done by doing something similar to the following:

[root@kaleb-desktop ~]# sestatus | grep "Current mode"
Current mode:                   enforcing
[root@kaleb-desktop ~]# setenforce 0
[root@kaleb-desktop ~]# sestatus | grep "Current mode"
Current mode:                   permissive
[root@kaleb-desktop ~]#

Now if you find yourself in a similar boat to me and doing the above suddenly enables fail2ban to access your desired log file, you're in luck. If not, then your problem might be something else unfortunately. If you don't care for keeping SELinux around, I guess doing the above is a sufficient solution. However, I didn't necessarily want to get rid of SELinux entirely.

If you dig around in the system audit logs, you'll find that SELinux actually gives you a more permanent solution to the problem. Running sealert -l "*" (I had to install this manually on my Fedora 38 system) will output something similar to the following:

SELinux is preventing fail2ban-server from open access on the file /tmp/testing.log.

*****  Plugin catchall (100. confidence) suggests   **************************

If you believe that fail2ban-server should be allowed open access on the testing.log file by default.
Then you should report this as a bug.
You can generate a local policy module to allow this access.
Do
allow this access for now by executing:
# ausearch -c 'fail2ban-server' --raw | audit2allow -M my-fail2banserver
# semodule -X 300 -i my-fail2banserver.pp

As noted in the output, the suggested solution is to create a local policy module that can be applied on top of the existing policy. However, as noted in the accepted solution of the link @roaima shared, this policy might get overwritten by future updates to the selinux-policy package.

Thus, the final, most permanent solution is to simply move the logs to a directory that the existing SELinux access control policy is happy with. In my case, I was able to move my logs to a subdirectory of /var/log which SELinux seems to be content with. YMMV.

I would also like to mention that the output from journalctl -lfu fail2ban on my system did not reflect the output from that of the link @roaima shared. That question was asked 8 years ago so obviously a lot could have changed in terms of how things are logged so just be aware of that.

Apologies for the wordiness but I enjoyed this opportunity to dive into something I previously didn't understand and come out the other side with a better understanding. I hope this explanation helps someone else in a similar situation!