Configuring ulimit of `max open files` for daemonized nginx master process

Question

I have an nginx-daemon running on a Debian (8.3). The nginx process occasionally runs into resource limitations when trying to write log files: too many open files.

The nginx master process is executed with root, while each of the four worker processes is executed with www-data user permissions.

When I checked for the nginx-master and each worker process limit configuration I discovered something odd.

cat /proc/{nginx-master-process-id}/limits
Limit                     Soft Limit           Hard Limit           Units
…
Max open files            1024                 4096                 files
…

cat /proc/{nginx-any-worker-process-id}/limits
…
Max open files            30000                30000                files
…

Each nginx worker is allowed to open 30000 files.

The nginx master process though is only allowed to open 1024 files, respectively 4096 files regarding the hard limit.

When I check for the root user ulimit settings I see no such limit defined! Where does this 1024/4096 setting may come from?

root ulimit settings

# logged in as root
ulimit -H
unlimited

Additionally I checked the daemon config:

/lib/systemd/system/nginx.service

[Unit]
Description=A high performance web server and a reverse proxy server
After=network.target

[Service]
Type=forking
PIDFile=/run/nginx.pid
ExecStartPre=/usr/sbin/nginx -t -q -g 'daemon on; master_process on;'
ExecStart=/usr/sbin/nginx -g 'daemon on; master_process on;'
ExecReload=/usr/sbin/nginx -g 'daemon on; master_process on;' -s reload
ExecStop=-/sbin/start-stop-daemon --quiet --stop --signal QUIT --retry QUIT/5 --pidfile /run/nginx.pid
# Give Passenger a chance to clean up before being killed by systemd.
ExecStop=/bin/sleep 1
TimeoutStopSec=5
KillMode=mixed

[Install]
WantedBy=multi-user.target

I see no ulimit configuration here, either.

What places else can I check to modify the 1024/4096 nofile limit for the nginx-master process?

Answer 1

It seems the problem, so to speak, was my wrong assumption that a systemd service respects the ulimit configured in /etc/security/limits.conf.

As it turns out a daemon configured via systemd does intentionally ignore settings in the limits.conf and requires a LimitNOFILE configuration in the service configuration file.

Updating my systemd unit file fixed the problem:

/lib/systemd/system/nginx.service

[Unit]
Description=A high performance web server and a reverse proxy server
After=network.target

[Service]
Type=forking
PIDFile=/run/nginx.pid
ExecStartPre=/usr/sbin/nginx -t -q -g 'daemon on; master_process on;'
ExecStart=/usr/sbin/nginx -g 'daemon on; master_process on;'
ExecReload=/usr/sbin/nginx -g 'daemon on; master_process on;' -s reload
ExecStop=-/sbin/start-stop-daemon --quiet --stop --signal QUIT --retry QUIT/5 --pidfile /run/nginx.pid
# Give Passenger a chance to clean up before being killed by systemd.
ExecStop=/bin/sleep 1
TimeoutStopSec=5
KillMode=mixed
LimitNOFILE=30000 # <= This line was added

[Install]
WantedBy=multi-user.target

Here are some links and resources regarding this issue:

• https://serverfault.com/questions/628610/increasing-nproc-for-processes-launched-by-systemd-on-centos-7
• https://bugzilla.redhat.com/show_bug.cgi?id=754285

Thanks to @ijaz-khan for pointing me in that direction.

Answer 2

Best way to achieve this, is to either copy the service file

cp /lib/systemd/system/nginx.service /etc/systemd/system

and then edit it in /etc/systemd/system. In case /etc/systemd/system/nginx.service.d exists, a simple override can be made by creating /etc/systemd/system/nginx.service.d/ulimit.conf with the following content

[Service]
LimitNOFILE=30000

Overriding in /etc is the recommended way when dealing with systemd definition files. Don't forget to finally systemctl daemon-reload