Is it possible to change the soft - and hard limit of a specific process?

In my case, my process is mongod and a lot of web resources tell me to simply execute:

ulimit -n <my new value>

My current thoughts:

  • How will the command know the limit of the process that I'll be modifying? Won't this modify the whole systems open file limit?
  • I'm guessing that this command only changes the soft limit. So is there a way to increase the hard limit too?
  • Have you read this valuable piece of information? docs.mongodb.org/manual/reference/ulimit – tink May 15 '13 at 22:03
  • Yes I learnt it all by heart.. I'm not sure if I'm missing something but that page only tells you how to VIEW the details. It only tells you to write ulimit -n <value>... – test May 15 '13 at 22:09

To change the limits of a running process, you may use the utility command prlimit.

prlimit --pid 12345 --nofile=1024:1024

What that does internally is to call setrlimit(2). The man page of prlimit should contain some useful invocation examples.

Source: https://sig-io.nl/posts/run-time-editing-of-limits-in-linux/

  • This saved my day, thanks. I had a long running Python process that was producing too many files open errors. Your answer resolved my issue without stopping the process. – Visionscaper Aug 11 '19 at 21:21

A process can change its limits via the setrlimit(2) system call. When you run ulimit -n you should see a number. That's the current limit on number of open file descriptors (which includes files, sockets, pipes, etc) for the process. The ulimit command executed the getrlimit(2) system call to find out what the current value is.

Here's the key point: a process inherits its current limit from its parent process. So if you ran ulimit -n 64 you would set that shell's limit of open file descriptors to 64. Any process that shell starts would have the same limit, unless that new process calls setrlimit() appropriately.

To change mongodb's open file descriptor limit, you'd run ulimit -n 2048 (or whatever large number your kernel allows) in a shell. You'd then use that shell to start mongodb. As a child process, mongodb would inherit the (large) limit on open file descriptors.

To modify the system's open file limit, which seems more like the sum of all processes open file descriptor limit, you have to do something like modify /etc/sysctl.conf and run sysctl -p. Look at the value of the fs.file-max parameter in /etc/sysctl.conf.

  • What a great explanation! Thank you very much. I will try this right away! – test May 15 '13 at 22:14
  • This answer helped me a lot, yet the problem now is that the hard limit is 2048, and this is not enough in my case. 1) how can I find the limit that my kernel allows? 2) Is it possible to change the HARD LIMIT now? Thanks again! – test May 15 '13 at 22:22
  • @test - I've never raised the hard limit. Found this: blog.samat.org/2011/04/05/… and this wiki.brekeke.com/wiki/Increase-File-Descriptor-Limit-on-Linux They say the same thing. – Bruce Ediger May 16 '13 at 2:23

On Linux at least, most distros seem to use pam for authentication. One module that comes with pam is the limits module. Quoting from the README for pam_limits:

The pam_limits PAM module sets limits on the system resources that can be obtained in a user-session. Users of uid=0 are affected by this limits, too.

As a result, you can set per user, per group, and default limits, in both categories hard (root sets this and the process cannot request higher) and soft limits. The soft limits are typically set lower than the hard limits, and the app can increase it upwards until it hits the hard limit.

In your case, if the process you want to increase the limits of runs as a regular user, you can increase the limits for that user or group. As an example, I have some mysql cron jobs on some servers that require extra file handles to be opened, so I set this:

$ cat /etc/security/limits.d/mysql.conf
@mysql           soft    nofile          100000
@mysql           hard    nofile          200000

There is nothing that needs to be restarted; when you su to that user, you can immediately see that the new limits take effect.

Another thing that you can do if you are on a typical RedHat derived system is put the ulimit call that you want into the /etc/sysconfig/$SERVICE script. As an example, apache's init script is named /etc/init.d/httpd, and it sources the /etc/sysconfig/httpd configuration file if it's found. I have found it easier to manage by doing this instead of editing the init script itself because init scripts get updated when the rpm is upgraded, but sysconfig files are only updated if they are not changed from the default.

  • Best and most practical solution. – nelaaro Mar 15 '17 at 12:16

How will the command know the limit of the process that I'll be modifying? Won't this modify the whole systems open file limit?

The command will change the limit for the current process (your shell) and any child process (anything you run from the shell subsequently).

Processes are organized in a tree. Every process has a parent (which invoked it). The process number 1, init, is a special process in that it is its own parent. Tools like htop or top can show you processes displayed as a tree of parents and children.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.