4

Why does this shell script gradually consume more memory in bash? I’m not using any local variables. Although the value of cpid is changing at every iteration because ls exits immediately, it should never grow too large.

The shell script in question just monitors a subprocess and checks if it’s up. This script is named run-always

set -euf

# monitor a child process
# see if it's up

cpid=
while : ; do
    # if cpid isn't set, spawn child
    if [ -z "$cpid" ]; then
       "$@" &
       cpid="$!"
    fi

    # if child isn't active unset cpid
    if ps -o pid= -p "$cpid"; then
        :
    else
        cpid=
    fi

    sleep 1
done

If I use it with a process that exits immediately (like ls) then we get the following memory usage when I execute the script with dash run-always ls

4476kb and it never grows.

This is the command I’m using to get the memory usage once per second on Linux … there’s probably a way to get the memory and peak memory usage directly from ps, but this does the job.

 while : ; do cat /proc/"$(ps a | grep run-always | grep -v grep | awk '{print $1}')"/status | grep -i 'vmsize\|vmpeak' ; sleep 1; done

When I execute the above script with dash, the memory usage never grows, it hovers at a constant 4476kb on my machine.

However, when I use bash (bash run-always ls), every ten seconds or so it allocates another few kilobytes of memory and never frees it.

Why is bash doing this?

Example output:

VmPeak:    16676 kB
VmSize:    16676 kB
VmPeak:    16676 kB
VmSize:    16676 kB
VmPeak:    16676 kB
VmSize:    16676 kB
VmPeak:    16676 kB
VmSize:    16676 kB
VmPeak:    16676 kB
VmSize:    16676 kB
VmPeak:    16676 kB
VmSize:    16676 kB
VmPeak:    16676 kB
VmSize:    16676 kB
VmPeak:    16676 kB
VmSize:    16676 kB
VmPeak:    16676 kB
VmSize:    16676 kB
VmPeak:    16676 kB
VmSize:    16676 kB
VmPeak:    16676 kB
VmSize:    16676 kB
VmPeak:    16680 kB
VmSize:    16680 kB
VmPeak:    16680 kB
VmSize:    16680 kB
VmPeak:    16680 kB
VmSize:    16680 kB
VmPeak:    16680 kB
VmSize:    16680 kB
VmPeak:    16680 kB
VmSize:    16680 kB
VmPeak:    16680 kB
VmSize:    16680 kB
VmPeak:    16680 kB
VmSize:    16680 kB
VmPeak:    16680 kB
VmSize:    16680 kB
VmPeak:    16680 kB
VmSize:    16680 kB
VmPeak:    16684 kB
VmSize:    16684 kB
VmPeak:    16684 kB
VmSize:    16684 kB
VmPeak:    16684 kB
VmSize:    16684 kB
VmPeak:    16684 kB
VmSize:    16684 kB
VmPeak:    16684 kB
VmSize:    16684 kB
VmPeak:    16684 kB
VmSize:    16684 kB
VmPeak:    16684 kB
VmSize:    16684 kB
VmPeak:    16684 kB
VmSize:    16684 kB
VmPeak:    16684 kB
VmSize:    16684 kB
VmPeak:    16684 kB
VmSize:    16684 kB
VmPeak:    16684 kB
VmSize:    16684 kB
VmPeak:    16684 kB
VmSize:    16684 kB
VmPeak:    16684 kB
VmSize:    16684 kB
VmPeak:    16684 kB
VmSize:    16684 kB
VmPeak:    16688 kB
VmSize:    16688 kB
VmPeak:    16688 kB
VmSize:    16688 kB
VmPeak:    16688 kB
VmSize:    16688 kB
VmPeak:    16688 kB
VmSize:    16688 kB
VmPeak:    16688 kB
VmSize:    16688 kB
VmPeak:    16688 kB
VmSize:    16688 kB
VmPeak:    16688 kB
VmSize:    16688 kB
VmPeak:    16688 kB
VmSize:    16688 kB
VmPeak:    16688 kB
VmSize:    16688 kB
VmPeak:    16688 kB
VmSize:    16688 kB
VmPeak:    16692 kB
VmSize:    16692 kB
1
  • 1
    This is unrelated to your problem, but what you are doing in this script could probably be done better with the wait command - or in a case this simple, just not backgrounding the process at all. – Score_Under Jul 29 '16 at 14:28
2

Bash saves each backgrounded process in a table of active jobs. Because you're spawning new jobs without explicitly checking the old ones, the table potentially increases without bound. The problem goes away if you disown the process after backgrounding it or check the exit status using jobs before launching the next background process.

For example, this version of your script does not increase in memory usage:

set -euf

# monitor a child process
# see if it's up

cpid=
while : ; do
    # if cpid isn't set, spawn child
    if [ -z "$cpid" ]; then
       "$@" &
       disown $!       # Don't waste memory on this red-headed child
       cpid="$!"
    fi

    # if child isn't active unset cpid
    if ps -o pid= -p "$cpid"; then
        :
    else
        cpid=
    fi

    sleep 1
done
2

As you already discovered that this problem does not appear with dash, it seems to be obvious that there is a high probability that this is caused by a bug.

It does also grow if you are using bash-3.x but very slowly - still faster than with other shells.

I recommend you to make a bug-report against your version of bash.

BTW: I did some tests and there are only two shells where there is no growth at all - regardless how long you wait: mksh and the original Bourne Shell that was not yet converted to use malloc() instead of sbrk().

All other shells grow very slowly.

2
  • Did you also check the behavior of dash? I ran that script for half an hour and it never allocated more memory. – Gregory Nisbet Apr 22 '16 at 2:52
  • Dash is slowly growing like ksh88 and ksh93. I removed the sleep and waited for some minutes. – schily Apr 22 '16 at 7:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.