I have the following situation: (The following functions are ones taken from python)
I have a process A which is running and has a cgroup memory limit set on it.
I fork a child process from A using os.fork(). Let us call it B. Then I execute os.execvp to load a shell script inside B. As per http://www.csl.mtu.edu/cs4411.ck/www/NOTES/process/fork/exec.html this process runs in the same address space as the caller (i.e. B).
The shell script running in B creates a Java program which runs idefinitely with a given heap size and automatic kill on OOM. This is achieved by passing -XX:OnOutOfMemoryError='kill -9 %p' to the java command. This creates another process C as child of B.
Looking from top, I see B as child of A and C as child of B as expected.
Below are the doubts:
Does the cgroup limit apply on only A or (A+B) or (A+B+C)?
If memory limit is reached, which all process are killed: A or (A+B) or (A+B+C)? And why? Is it identified using process pids or addresses space of the process on which limit is applied?
If in the above point, not all the process are killed is there a way to fine-tune cgroup setting to kill all the child processes as well since we would be left with orphaned processes in such cases?
I am working on Centos7 as the underlying operating system.