16

I'm been having some weird problems with bash lately. While trying to simplify my script, I came up with this small piece of code:

$ o(){ echo | while read -r; do return 0; done; echo $?;}; o
0
$ o(){ echo | while read -r; do return 1; done; echo $?;}; o
1

return should have exited the function without printing $?, shouldn't it? Well, then I checked if I can return from a pipe alone:

$ echo | while read -r; do return 1; done
bash: return: can only `return' from a function or sourced script

The same happens without a while loop:

$ foo(){ : | return 1; echo "This should not be printed.";}
$ foo
This should not be printed.

Is there something I'm missing here? A Google search brought nothing about this! My bash version is 4.2.37(1)-release on Debian Wheezy.

  • Anything wrong with the settings I suggested in my reply that allow your script to behave the intuitive way you expected it to do? – jlliagre Sep 19 '15 at 20:42
  • @jlliagre It is a rather complex script on the thousands of lines. With the concern of breaking something else, I prefer to just avoid running a pipe within the function, so I replaced it with a process substitution. Thanks! – Teresa e Junior Sep 19 '15 at 21:21
  • Why not remove the first two examples, if the while is not needed for reproduction? It distracts from the point. – Lightness Races with Monica Sep 20 '15 at 0:29
  • @LightnessRacesinOrbit A while loop is a very common usage for a pipe with return. The second example is more straight to the point, but it is something I don't believe anyone would ever use... – Teresa e Junior Sep 20 '15 at 3:08
  • 1
    Unfortunately my correct answer has been deleted... You are in a grey zone as you do something that is unspecified. The behavior depends on how the shell interprets pipes and this is even different between the Bourne Shell and the Korn Shell even though ksh was derived from sh sources. In the Bourne Shell, the while loop is in a subshell therefore you see the echo as with bash, In ksh the while loop is the foreground process and thus ksh does not call echo with your example. – schily Sep 20 '15 at 10:04
10

Related: https://stackoverflow.com/a/7804208/4937930

It's not a bug that you cannot exit a script or return from a function by exit or return in subshells. They are executed in another process and not affecting the main process.

Besides that, I suppose you are seeing undocumented behaviors of bash on (probably) undefined spec. In a function, no errors are asserted for return at top level of subshell commands and it just behaves like exit.

IMHO it's a bash bug for the inconsistent behavior of return depending on whether the main statement is in a function or not.

#!/bin/bash

o() {
    # Runtime error, but no errors are asserted,
    # each $? is set to the return code.
    echo | return 10
    echo $?
    (return 11)
    echo $?

    # Valid, each $? is set to the exit code.
    echo | exit 12
    echo $?
    (exit 13)
    echo $?
}
o

# Runtime errors are asserted, each $? is set to 1.
echo | return 20
echo $?
(return 21)
echo $?

# Valid, each $? is set to the exit code.
echo | exit 22
echo $?
(exit 23)
echo $?

Output:

$ bash script.sh 
10
11
12
13
script.sh: line 20: return: can only `return' from a function or sourced script
1
script.sh: line 22: return: can only `return' from a function or sourced script
1
22
23
  • The lack of error verbosity may be undocumented. But the fact that return doesn't work from a top-level command sequence in a subshell, and in particular doesn't exit the subshell, is what existing documents have already made me expect. The OP could use exit 1 || return 1 where they are trying to use return, and should then get the expected behavior. EDIT: @herbert's answer indicates that toplevel return in subshell is functioning as exit (but only from the subshell). – dubiousjim Sep 19 '15 at 7:07
  • 1
    @dubiousjim Updated my script. I mean return in a simple subshell sequence should be asserted as a runtime error in any cases, but actually it's not when it occurs in a fucntion. This problem has also been discussed in gnu.bash.bug, but there's no conclusion. – yaegashi Sep 19 '15 at 10:15
  • 1
    Your answer is not correct as it is unspecified whether the while loop is in a subshell or is the foreground process. Regardless of the implementation of the actual shell, the return statement is in a function and thus legal. The resulting behavior however is unspecified. – schily Sep 20 '15 at 10:06
  • You shouldn't write it is an undocumented behavior while the fact pipe components are in a subshell is documented in the bash manual page. You shouldn't write the behavior is probably based on undefined specs while POSIX specify the allowed behaviors. You shouldn't suspect a bash bug while bash is following the POSIX standard by allowing return in a function but not outside. – jlliagre Sep 23 '15 at 11:46
17

It is not a bug in bash but its documented behavior:

Each command in a pipeline is executed in its own subshell

The return instruction is valid being inside a function definition but being in a subshell as well, it doesn't affect its parent shell so the next instruction, echo, is executed regardless. It is nevertheless a non portable shell construction as the POSIX standard allows the commands composing a pipeline to be executed either in a subshell (the default) or the top one (an allowed extension).

Additionally, each command of a multi-command pipeline is in a subshell environment; as an extension, however, any or all commands in a pipeline may be executed in the current environment. All other commands shall be executed in the current shell environment.

Hopefully, you can tell bash to behave the way your expect with a couple of options:

$ set +m # disable job control
$ shopt -s lastpipe # do not run the last command of a pipeline a subshell 
$ o(){ echo | while read -r; do return 0; done; echo $?;}
$ o
$          <- nothing is printed here
  • 1
    Since return won't quit the function, wouldn't it make more sense if the shell just printed bash: return: can only `return' from a function or sourced script, instead of giving the user a false sense that the function might have returned? – Teresa e Junior Sep 19 '15 at 12:42
  • 2
    I don't see anywhere in documentation said that the return inside subshell is valid. I bet that this feature was copied from ksh, the return statement outside function or sourced script behave like exit. I'm not sure about the original Bourne shell. – cuonglm Sep 19 '15 at 14:01
  • 1
    @jlliagre: Maybe Teresa is confused about the terminology of what she's asking for, but I don't see why it would be "tricky" for bash to issue a diagnostic if you execute a return from a subshell.  After all, it does know that it's in a subshell, as evidenced by the $BASH_SUBSHELL variable.  The biggest problem is that this could lead to false positives; a user who understands how subshells work could have written scripts that use return in lieu of exit to terminate a subshell.  (And, of course, there are valid cases where one might want to set variables or do a cd in a subshell.) – Scott Sep 19 '15 at 16:19
  • 1
    @Scott I think I understand the situation well. A pipe creates a subshell, and return is returning from the subshell instead of failing, since it is inside an actual function. The problem is that help return specifically states: Causes a function or sourced script to exit with the return value specified by N. From reading the documentation, any user would expect it to at least fail or print a warning, but never to behave like exit. – Teresa e Junior Sep 19 '15 at 16:41
  • 1
    It seems to me that anybody who expects a return in a subshell in a function to return from the function (in the main shell process) doesn’t understand subshells very well.  Conversely, I would expect a reader who understands subshells to expect return in a subshell in a function to terminate the subshell, just as exit would. – Scott Sep 19 '15 at 17:13
6

Per POSIX documentation, using return outside of function or sourced script is unspecified. So, it depends on your shell to handle.

SystemV shell will report error, while in ksh, return outside of function or sourced script behave like exit. Most other POSIX shells and schily's osh also behave like that:

$ for s in /bin/*sh /opt/schily/bin/osh; do
  printf '<%s>\n' $s
  $s -c '
    o(){ echo | while read l; do return 0; done; echo $?;}; o
  '
done
</bin/bash>
0
</bin/dash>
0
</bin/ksh>
</bin/lksh>
0
</bin/mksh>
0
</bin/pdksh>
0
</bin/posh>
0
</bin/sh>
0
</bin/yash>
0
</bin/zsh>
</opt/schily/bin/osh>
0

ksh and zsh didn't output because the last part of pipe in these shells was executed in current shell instead of subshell. The return statement affected the current shell environment which called the function, cause the function return immediately without printing anything.

In interactive session, bash only report the error but didn't terminated the shell, schily's osh reported the error and terminated the shell:

$ for s in /bin/*sh; do printf '<%s>\n' $s; $s -ci 'return 1; echo 1'; done
</bin/bash>
bash: return: can only `return' from a function or sourced script
1
</bin/dash>
</bin/ksh>
</bin/lksh>
</bin/mksh>
</bin/pdksh>
</bin/posh>
</bin/sh>
</bin/yash>
</bin/zsh>
</opt/schily/bin/osh>
$ cannot return when not in function

(zsh in interactive session and output is terminal do not terminated, bash, yash and schily's osh reported the error but didn't terminate the shell)

  • 1
    It can be argued return is used inside a function here. – jlliagre Sep 19 '15 at 11:25
  • 1
    @jlliagre: Not sure what you mean, return was use inside subshell inside function, except ksh and zsh. – cuonglm Sep 19 '15 at 11:31
  • 2
    I mean being inside a subshell which is itself inside a function doesn't necessarily mean being outside that function, i.e. nothing in the standard states pipeline components are to be considered to be outside the function where they are located. This would deserve to be clarified by the Open Group. – jlliagre Sep 19 '15 at 13:09
  • 3
    I think no. That's outside the function. The shell which called the function and the subshell which executed return are different. – cuonglm Sep 19 '15 at 13:45
  • I understand your reasoning which rightly explains the issue, my point is according to shell grammar described in the POSIX standard, the pipeline is part of the compound-list which is part of the compound-command which is the body of the function. Nowhere is it stated pipeline components are to be considered outside the function. Just like if I am in a car and that car is parked in a garage, I can assume I am in that garage too ;-) – jlliagre Sep 19 '15 at 20:03
4

I think that you got the expected behavior, in bash, each command in a pipeline is executed in a subshell. You can convice yourself by trying to modify a global variable of your function:

foo(){ x=42; : | x=3; echo "x==$x";}

By the way, the return is working but it return from the subshell. Again you can check that:

foo(){ : | return 1; echo$?; echo "This should not be printed.";}

Will output the following:

1
This should not be printed.

So return statement correctly exited the subshell

.

  • 2
    Hence, to exit the function, use foo(){ : | return 1 || return 2; echo$?; echo "This should not be printed.";}; foo; echo $? and you'll get a result of 2. But for clarity I would make the return 1 be exit 1. – dubiousjim Sep 19 '15 at 7:10
  • By the way, is there some substantiation for the fact that all members of a pipeline (not all but one) are executed in subshells? – Incnis Mrsi Sep 19 '15 at 17:33
  • @IncnisMrsi: See jlliagre's answer. – Scott Sep 19 '15 at 19:07
1

The more general answer is that bash and some other shells normally put all the elements of a pipeline into separate processes.  This is reasonable when the command line is

program1 | program2 | program3

since programs are normally run in separate processes anyway (unless you say exec program).  But it can come as a surprise for

command1 | command2 | command3

where some or all of the commands are built-in commands.  Trivial examples include:

$ a=0
$ echo | a=1
$ echo "$a"
0
$ cd /
$ echo | cd /tmp
$ pwd
/

A slightly more realistic example is

$ t=0
$ ps | while read pid rest_of_line
> do
>     : $((t+=pid))
> done
$ echo "$t"
0

where the entire whiledodone loop is put into a subprocess, and so its changes to t aren’t visible to the main shell after the loop ends.  And that’s exactly what you’re doing — piping into a while loop, causing the loop to run as a subshell, and then trying to return from the subshell.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.