Modify ulimit (open files) of a specific process

Question

Is it possible to change the soft - and hard limit of a specific process?

In my case, my process is mongod and a lot of web resources tell me to simply execute:

ulimit -n <my new value>

My current thoughts:

• How will the command know the limit of the process that I'll be modifying? Won't this modify the whole systems open file limit?
• I'm guessing that this command only changes the soft limit. So is there a way to increase the hard limit too?

Answer 1

To change the limits of a running process, you may use the utility command prlimit.

prlimit --pid 12345 --nofile=1024:1024

What that does internally is to call setrlimit(2). The man page of prlimit should contain some useful invocation examples.

Source: https://sig-io.nl/posts/run-time-editing-of-limits-in-linux/

Answer 2

A process can change its limits via the setrlimit(2) system call. When you run ulimit -n you should see a number. That's the current limit on number of open file descriptors (which includes files, sockets, pipes, etc) for the process. The ulimit command executed the getrlimit(2) system call to find out what the current value is.

Here's the key point: a process inherits its current limit from its parent process. So if you ran ulimit -n 64 you would set that shell's limit of open file descriptors to 64. Any process that shell starts would have the same limit, unless that new process calls setrlimit() appropriately.

To change mongodb's open file descriptor limit, you'd run ulimit -n 2048 (or whatever large number your kernel allows) in a shell. You'd then use that shell to start mongodb. As a child process, mongodb would inherit the (large) limit on open file descriptors.

To modify the system's open file limit, which seems more like the sum of all processes open file descriptor limit, you have to do something like modify /etc/sysctl.conf and run sysctl -p. Look at the value of the fs.file-max parameter in /etc/sysctl.conf.

Answer 3

On Linux at least, most distros seem to use pam for authentication. One module that comes with pam is the limits module. Quoting from the README for pam_limits:

The pam_limits PAM module sets limits on the system resources that can be obtained in a user-session. Users of uid=0 are affected by this limits, too.

As a result, you can set per user, per group, and default limits, in both categories hard (root sets this and the process cannot request higher) and soft limits. The soft limits are typically set lower than the hard limits, and the app can increase it upwards until it hits the hard limit.

In your case, if the process you want to increase the limits of runs as a regular user, you can increase the limits for that user or group. As an example, I have some mysql cron jobs on some servers that require extra file handles to be opened, so I set this:

$ cat /etc/security/limits.d/mysql.conf
@mysql           soft    nofile          100000
@mysql           hard    nofile          200000

There is nothing that needs to be restarted; when you su to that user, you can immediately see that the new limits take effect.

Another thing that you can do if you are on a typical RedHat derived system is put the ulimit call that you want into the /etc/sysconfig/$SERVICE script. As an example, apache's init script is named /etc/init.d/httpd, and it sources the /etc/sysconfig/httpd configuration file if it's found. I have found it easier to manage by doing this instead of editing the init script itself because init scripts get updated when the rpm is upgraded, but sysconfig files are only updated if they are not changed from the default.

Answer 4

How will the command know the limit of the process that I'll be modifying? Won't this modify the whole systems open file limit?

The command will change the limit for the current process (your shell) and any child process (anything you run from the shell subsequently).

Processes are organized in a tree. Every process has a parent (which invoked it). The process number 1, init, is a special process in that it is its own parent. Tools like htop or top can show you processes displayed as a tree of parents and children.