3

I have an nginx-daemon running on a Debian (8.3). The nginx process occasionally runs into resource limitations when trying to write log files: too many open files.

The nginx master process is executed with root, while each of the four worker processes is executed with www-data user permissions.

When I checked for the nginx-master and each worker process limit configuration I discovered something odd.

cat /proc/{nginx-master-process-id}/limits
Limit                     Soft Limit           Hard Limit           Units
…
Max open files            1024                 4096                 files
…

cat /proc/{nginx-any-worker-process-id}/limits
…
Max open files            30000                30000                files
…

Each nginx worker is allowed to open 30000 files.

The nginx master process though is only allowed to open 1024 files, respectively 4096 files regarding the hard limit.

When I check for the root user ulimit settings I see no such limit defined! Where does this 1024/4096 setting may come from?

root ulimit settings

# logged in as root
ulimit -H
unlimited

Additionally I checked the daemon config:

/lib/systemd/system/nginx.service

[Unit]
Description=A high performance web server and a reverse proxy server
After=network.target

[Service]
Type=forking
PIDFile=/run/nginx.pid
ExecStartPre=/usr/sbin/nginx -t -q -g 'daemon on; master_process on;'
ExecStart=/usr/sbin/nginx -g 'daemon on; master_process on;'
ExecReload=/usr/sbin/nginx -g 'daemon on; master_process on;' -s reload
ExecStop=-/sbin/start-stop-daemon --quiet --stop --signal QUIT --retry QUIT/5 --pidfile /run/nginx.pid
# Give Passenger a chance to clean up before being killed by systemd.
ExecStop=/bin/sleep 1
TimeoutStopSec=5
KillMode=mixed

[Install]
WantedBy=multi-user.target

I see no ulimit configuration here, either.

What places else can I check to modify the 1024/4096 nofile limit for the nginx-master process?

4

It seems the problem, so to speak, was my wrong assumption that a systemd service respects the ulimit configured in /etc/security/limits.conf.

As it turns out a daemon configured via systemd does intentionally ignore settings in the limits.conf and requires a LimitNOFILE configuration in the service configuration file.

Updating my systemd unit file fixed the problem:

/lib/systemd/system/nginx.service

[Unit]
Description=A high performance web server and a reverse proxy server
After=network.target

[Service]
Type=forking
PIDFile=/run/nginx.pid
ExecStartPre=/usr/sbin/nginx -t -q -g 'daemon on; master_process on;'
ExecStart=/usr/sbin/nginx -g 'daemon on; master_process on;'
ExecReload=/usr/sbin/nginx -g 'daemon on; master_process on;' -s reload
ExecStop=-/sbin/start-stop-daemon --quiet --stop --signal QUIT --retry QUIT/5 --pidfile /run/nginx.pid
# Give Passenger a chance to clean up before being killed by systemd.
ExecStop=/bin/sleep 1
TimeoutStopSec=5
KillMode=mixed
LimitNOFILE=30000 # <= This line was added

[Install]
WantedBy=multi-user.target

Here are some links and resources regarding this issue:

Thanks to @ijaz-khan for pointing me in that direction.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.