1

Bash 4.4.19(1)-release

I have below a simple script which is the basis for a logging app. For various reasons I had to use process substitution.

The runner is the heart of the app and since process substitution is asynchronous, I have managed to get it to a good degree of coherence by the while loop. It works perfectly.

Unfortunately I found a case where it will not work: when I execute 'bash <filename> <function>'

So we need 2 files to reproduce.

Requirement:

  1. Why does this happen?
  2. How to modify my while loop to accommodate similar cases?

Simplified script is:

test.sh

#!/bin/bash

2sub() {
local in=$(cat); echo -e "$in";
}   
runner () {
 "${@}" 1> >(2sub)
 while [ -e /proc/$! ]; do sleep 0.1; done     # <<< LOOP WAIT FOR $!
}
remotesub() {
 bash ./test2.sh remotesub2
}

echo -e "running\n"; 
    runner bash ./test2.sh remotesub2 # LOOPS
    # runner remotesub # A POSSIBLE BYPASS/SOLUTION? But why?
echo -e "done!\n"

test2.sh

     remotesub2() {
         echo -e "'${BASH_VERSION}'"
         return 0
     }

     "$@"

Bypass:

As you can see from the script, there is a bypass for the problem, by including bash <filename> <function> inside a function, and passing the function to the runner. Why this works and not the direct way, I am sure somebody here knows.

Please shed some light on this issue and if there are some better ways to do the waiting loop in order to cover these cases.

Solution:

The best solution is what mosvy suggested. Thank you. Using { "${@}"; } removes the need to package the commands in separate small functions which is a pain. Also after many hours of testing with my larger code, I came to the conclusion that careful killing of sub-processes makes this while [ -e /proc/$! ]; do sleep 0.1; done unnecessary. That line was replaced with wait $!;

  • Which bash version are you using? – Inian Jun 14 at 3:25
  • 4.4.19(1)-release – conanDrum Jun 14 at 3:25
3

If I understand you exactly, you're wondering why $! will be set to the PID of a process run inside >(...) only when that is part of the command line of a built-in command or function, but not when it's part of the command line of an external command.

Simplified example:

$ bash -c 'true > >(echo in=$BASHPID; sleep .1); echo psubst=$!'
psubst=12392
in=12392

$ bash -c '/bin/true > >(echo in=$BASHPID; sleep .1); echo psubst=$!'
in=12751
psubst=

That happens because in the case where an external command is used bash will fork a separate process to run it in, and the process running inside the >(...) will be run as a child of that process, and so as a grandchild of your script, completely outside of its control.

By the time the external command terminates, its child (if still running) will be adopted by pid 1 (init), and so any link that could still be used to retrieve its PID from your script is broken.

A workaround may be to use a wrapper function which will cause all the process substitutions from its command line to be run as children of your script, so their PIDs could be retrieved via pgrep -P "$$".

Also, putting the external command in {...} block and redirecting the ouput of the block also seems to work:

$ bash -c 'func(){ /bin/true; }; func > >(echo in=$BASHPID; sleep .1); echo psubst=$!'
in=3574
psubst=3574
$  bash -c '{ /bin/true; } > >(echo in=$BASHPID; sleep .1); echo psubst=$!'
in=3435
psubst=3435

Both workarounds rely on the way the current implementation works; eg. bash may decide one day to optimize away trivial group commands or functions, breaking these assumptions.

Notice that $! being set to the PID from the last process substitution is an undocumented feature, which also does not work in other shells than bash.

  • so my 'POSSIBLE BYPASS/SOLUTION' mentioned in the script is the only solution to this issue? – conanDrum Jun 14 at 7:12
  • I guess so. Using a function wrapper will make sure that the subshell from the process substitution will be a child of your script. – mosvy Jun 14 at 7:15
  • I'll update this answer when I get time (or remove it ;-)). In the meanwhile, wait $! or wait $(pgrep -P $$) would be a better idea than your loop (the latter since I'm not sure that $! is documented anywhere to be also set to the pid of a process substitution) – mosvy Jun 14 at 7:19
  • perfect thanks for taking the time – conanDrum Jun 14 at 7:19
  • See... I am even pioneering in areas you never knew existed!! )) – conanDrum Jun 14 at 7:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.