MySQL binlogs on cifs share getting OS errno 13 - Permission denied

Question

I am creating a Rocky Linux 9 server to replace a CentOS 7 server. The original CentOS 7 server mounts a Windows shared directory where it stores the binlogs. When I try to duplicate this for the Rocky system, it fails with error OS errno 13 - Permission denied.

2024-07-01T20:31:31.048050Z 0 [System] [MY-010116] [Server] /usr/libexec/mysqld (mysqld 8.0.36) starting as process 1052
mysqld: File '/var/lib/mysql/binlog/rckpts-mysql-bin.index' not found (OS errno 13 - Permission denied)
2024-07-01T20:31:31.389776Z 0 [ERROR] [MY-010119] [Server] Aborting

The Rocky root user can view the mounted directory and edit a file in it. I confirmed that file was edited from the Windows and CentOS servers. It appears that the mysql user and group cannot write to mysql/binlog directory. The CentOS server uses the same Windows user credentials to successfully write binlogs to that shared directory. As far as I know, only the .smbcredentials file is required to use the directory.

ls -laFh /var/lib/mysql shows that the mysql user and group own the mysql directories and subdirectories. Root owns one subdirectory for mysql dumps.

total 78M
drwxr-xr-x.  9 mysql mysql 4.0K Jul  1 17:22  ./
drwxr-xr-x. 59 root  root  4.0K Jun 27 16:22  ../
-rw-r-----.  1 mysql mysql   56 Jun 27 17:00  auto.cnf
drwxr-xr-x.  2 mysql mysql  32K Jul  1 01:30  binlog/
-rw-r-----.  1 mysql mysql  180 Jun 27 17:02  binlog.000001 # binlog from successful start
-rw-r-----.  1 mysql mysql  180 Jun 27 17:42  binlog.000002
-rw-r-----.  1 mysql mysql  180 Jun 27 17:50  binlog.000003
-rw-r-----.  1 mysql mysql   48 Jun 27 17:42  binlog.index

If I umount the /binlog directory, MySQL will start successfully.

Where possible, I copied configuration files directly from the working CentOS server.

Configuration includes:

FSTAB

//XX.XX.XX.100/LinuxDumpFiles/PTS/binlogs /var/lib/mysql/binlog    cifs   credentials=/root/.smbcredentials,uid=27,forceuid,gid=27,forcegid,domain=DOMAIN.org 0 0
## I confirmed that mysql uid=27 and gid=27.

//XX.XX.XX.100/LinuxDumpFiles/PTS/dbdumps /var/lib/mysql/dumps    cifs   credentials=/root/.smbcredentials 0 0
## for nightly DB dumps;

.smbcredentials File

/root/.smbcredentials
username=user_account
password=user_password
domain=DOMAIN.ORG  # note DOMAIN.ORG added to Rocky file, but not CentOS file.

MTAB

cat /etc/mtab | grep binlog
//XX.XX.XX.100/LinuxDumpFiles/PTS/binlogs /var/lib/mysql/binlog cifs rw,relatime,vers=3.1.1,cache=strict,username=mntlinux,domain=DOMAIN.org,uid=27,forceuid,gid=27,forcegid,addr10.210.6.100,file_mode=0755,dir_mode=0755,soft,nounix,serverino,mapposix,rsize=4194304,wsize=4194304,bsize=1048576,echo_interval=60,actimeo=1,closetimeo=1 0 0

MY.CNF

# /etc/my.cnf.d/mysql-server.cnf
log-bin=/var/lib/mysql/binlog/rckpts-mysql-bin
log-bin-index=/var/lib/mysql/binlog/rckpts-mysql-bin
## appended a long prefix to the binlog names to prevent filename conflicts with existing server

I have updated the Rocky /etc/fstab file so that the resulting /etc/mtab file closely resembles the CentOS /etc/mtab. The differences are in wsize, bsize, and closetimeo.

The CentOS fstab and .smbcredentials do not include the domain. I have added domain entries to those files, trying DOMAIN.org and DOMAIN in different combinations.

Both servers are VMs on the same host.

Impersonate mysql user

This appears to confirm the mysql user cannot write:

runuser -u mysql -- 'echo 2024-07-01 > /var/lib/mysql/binlog/ztest.txt'
runuser: failed to execute echo 2024-07-01 > /var/lib/mysql/binlog/ztest.txt: Permission denied

Firewalld I allowed samba services through fireall-cmd. I do not know if this is correct. If I turn off the firewall, I get the same error.

I have used https://linux.die.net/man/8/mount.cifs as a reference. Perhaps the commands differ between Rocky and CentOS?

Answer 1

Turning off SeLinux is not a solution for a server. On new server builds where you know the application(s) installed are rusted you can baseline it with SELinux like so:

# Install necessary packages
sudo dnf install -y policycoreutils policycoreutils-python-utils

# Generate a custom SELinux policy module from current denials
sudo ausearch -m avc -ts recent | audit2allow -M custom_policy

# Review the generated policy module
cat custom_policy.te

# Install the custom policy module
sudo semodule -i custom_policy.pp

# Verify the applied policy and monitor for new denials
sudo ausearch -m avc -ts recent

Answer 2

The problem was SELinux. I reviewed the log here in /var/log/messages.

I tested with

sudo setenforce 0       #Switch SELinux to permissive mode.

Then I shut off SELinux. In /etc/sysconfig/selinux, set SELINUX=disabled.