Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

There are plenty of questions and answers about constraining the resources of a single process, e.g. RLIMIT_AS can be used to constrain the maximum memory allocated by a process that can be seen as VIRT in the likes of top. More on the topic e.g. here Is there a way to limit the amount of memory a particular process can use in Unix?

setrlimit(2) documentation says:

A child process created via fork(2) inherits its parent's resource limits. Resource limits are preserved across execve(2).

It should be understood in the following way:

If a process has a RLIMIT_AS of e.g. 2GB, then it cannot allocate more memory than 2GB. When it spawns a child, the address space limit of 2GB will be passed on to the child, but counting starts from 0. The 2 processes together can take up to 4GB of memory.

But what would be the useful way to constrain the sum total of memory allocated by a whole tree of processes?

share|improve this question
    
Releated: unix.stackexchange.com/questions/1424/… –  slm Jun 4 at 12:37
4  
I'd take a look at cgroups. –  slm Jun 4 at 12:38
    
@slm Thanks! Sounds like cgroups is something to try. The only working solution this far (besides an ugly way of summing memory using PS and killing the parent process if above limit) that might work is using some form of a container (lxc or the likes). –  jpe Jun 4 at 12:55
    
Yeah - the tools I'm aware of do not do a group, just single processes, but given how cgroups work for VM technologies like LXC and Docker I'd expect it to do what you want. –  slm Jun 4 at 13:00
1  
If it's Linux put the parent PID in its own namespace and control it and all its children that way. Here's an introductory answer to that concept: unix.stackexchange.com/a/124194/52934 –  mikeserv Jun 7 at 20:53

2 Answers 2

up vote 1 down vote accepted
+50

I am not sure if this answers your question, but I found this perl script that claims to do exactly what you are looking for. The script implements its own system for enforcing the limits by waking up and checking the resource usage of the process and its children. It seems to be well documented and explained, and has been updated recently.

As slm said in his comment, cgroups can also be used for this. You might have to install the utilities for managing cgroups, assuming you are on Linux you should look for 'libcgroups'.

sudo cgcreate -t $USER:$USER -a $USER:$USER -g memory:myGroup

Make sure $USER is your user. Your user should then have access to the cgroup memory settings in /sys/fs/cgroup/memory/myGroup.

You can then set the limit to, lets say 500 MB, by doing this:

echo 500000000 > /sys/fs/cgroup/memory/myGroup/memory.limit_in_bytes

Now lets run Vim!

cgexec -g memory:myGroup vim

The vim process and all its children should now be limited to using 500 MB of RAM. However, I think this limit only applies to RAM and not swap. Once the processes reach the limit they will start swapping. I am not sure if you can get around this, I can not find a way to limit swap usage using cgroups.

I hope this helps you!

share|improve this answer
    
The proposed solution does make it possible to limit the resident set size of a tree of processes. The behaviour seems to be different from RLIMIT_AS: it is possible to malloc more memory than is the limit, however it seems not to be possible to actually use more. –  jpe Jun 13 at 17:03

I created a script that does this, using cgmanager which is used in Ubuntu. One advantage is that this does not require root access. Note that cgroup management is a bit distro specific, so I don't know if this works on distros that use systemd cgroup management.

#!/bin/sh

set -eu

if [ "$#" -lt 2 ]
then
    echo Usage: `basename $0` "<limit> <command>..."
    exit 1
fi

limit="$1"
shift
cgname="limitmem_$$"
echo "limiting memory to $limit (cgroup $cgname) for command $@"

cgm create memory "$cgname" >/dev/null
cgm setvalue memory "$cgname" memory.limit_in_bytes "$limit" >/dev/null
# try also limiting swap usage, but this fails if the system has no swap
cgm setvalue memory "$cgname" memsw.limit_in_bytes "$limit" >/dev/null 2>&1 || true

# spawn subshell to run in the cgroup
set +e
(
set -e
cgm movepid memory "$cgname" `sh -c 'echo $PPID'` > /dev/null
exec "$@"
)
# grab exit code
exitcode=`echo $?`

set -e

echo -n "peak memory used: "
cgm getvalue memory "$cgname" memory.max_usage_in_bytes | tail -1 | cut -f2 -d\"

cgm remove memory "$cgname" >/dev/null
exit $exitcode

usage: save as limitmem in your path and make it executable. Then run e.g. limitmem 1G command.... This limit the actually used memory. If that is reached the OOM killer will kill the process or one of its children, but not something random that has nothing to do with this command.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.