I was checking possibilities of pid leaks about a bash script which continuously create background jobs yet not call wait command, I happened found (by strace) that Bash monitors SIGCHLD and automatically calls wait4(...), despite that my script have not called wait command. This is why no pid leaks, that is good. But then I started think that what if I call wait command for that background pid? it does not exist in /proc, it should return error, how does Bash handle that? I did some experiments on Bash 4.4.19 and 5.1.16, turns out that Bash wait command actually gets result from background job cache, I also checked the source code, e.g., Bash 5.1.16 , see the builtins/wait.def line 253
status = wait_for_single_pid (pid, wflags|JWAIT_PERROR);
then job.c line 2611
r = bgp_search (pid);
which means
/* Search for PID in the list of saved background pids; return its status if
found. If not found, return -1. We hash to the right spot in pidstat_table
and follow the bucket chain to the end. */
.
My experiment is
test1:
bash <<'EOF'
bash -c 'sleep 1; exit 9' &
PID=$!
echo $PID
sleep 2
ls -d /proc/$PID
wait $PID
echo wait result: $?
EOF
The result is:
16079
ls: cannot access '/proc/16079': No such file or directory
wait result: 9
This is an evidence that the Bash wait command use the cache (I'v also use strace to verify that, it clearly showed the last syscall of wait4
returned -1.
wait4(-1, [{WIFEXITED(s) && WEXITSTATUS(s) == 9}], 0, NULL) = 397
wait4(-1, [{WIFEXITED(s) && WEXITSTATUS(s) == 0}], 0, NULL) = 399
wait4(-1, 0x7ffd79fc3fd0, WNOHANG, NULL) = -1 ECHILD (No child processes)
Of course if I run disown -a
before wait
, then the wait
will return code 127: wait: pid xxxxxx is not a child of this shell
, this also verified that once removed the background pid from background job list then the wait command will not the exit code correctly.
So that is the conclusion that Bash wait command is using the cached result in background job management info.
Then my question become: if a script creates background jobs continuously, e.g.,
test2:
while true; do
echo hi &
done
then the background job cache will be bigger and bigger, then will it become a memory leak?
I've tested the script, seems no memory leak, then why no leak?
EDIT: Let me make it more clear, the above script is supposed to run out of memory, but it actually does not as my observation, why?
EDIT: The above test2
is still the most interesting question, why it does not run out of memory?
EDIT: I made another test, it does run out of memory several seconds later:
test3:
bash <<'EOF'
while true; do
sleep 10 &
echo $!
done
EOF
The result is
...
bash: fork: retry: Resource temporarily unavailable
bash: fork: retry: Resource temporarily unavailable
bash: fork: Interrupted system call
OK, now it behaves as expected: running out of memory.
Sorry then my question becomes: is this by design? I have never heard of the caveat of continuously create background job. The only solution I know so far is use disown
to stop managing the background jobs, or use other tricks such as (cmd&)
to start a process without being managed as background job.
EDIT: answered by myself: this should be by design, it just means Bash tracking all active jobs, if in a short period, there are a lot of active jobs, then it runs out of memory. So this is not contradictory to the test2
.
EDIT: added another test to show that the Bash background job exit code cache does not just cache the exit code of last job besides of active jobs, it do cache exit code of all jobs.
test4:
bash -x <<'EOF'
bash -c '/bin/sleep 3; exit 1' &
PID1=$!
bash -c '/bin/sleep 6; exit 2' &
PID2=$!
wait $PID1
echo exit code of first process is: $?
wait $PID2
echo exit code of second process is: $?
wait $PID1
echo Get exit code of first process again, result is: $?
EOF
result is:
+ PID1=2357449
+ bash -c '/bin/sleep 3; exit 1'
+ PID2=2357450
+ wait 2357449
+ bash -c '/bin/sleep 6; exit 2'
+ echo exit code of first process is: 1
exit code of first process is: 1
+ wait 2357450
+ echo exit code of second process is: 2
exit code of second process is: 2
+ wait 2357449
+ echo Get exit code of first process again, result is: 1
Get exit code of first process again, result is: 1
wait $PID1
AGAIN at the end, after you waited for PID2, it would show the exit status of PID1, which proves thatbash
keeps the results of the the children in the memory, and that's your main point.test2
does not run out of memory? based on my observation, Bash caches all exit codes. There might be something I still don't know.test3
didn't "run out of memory", it just hit thenproc
limit. So it wasn't related tobash
or to your question at all.