7

I have a perl script that runs a system() call to execute shell commands and I would like to run more than one command and pipe data between them. Something like (in perl):

system("command1 | command2");

Perl's system() uses /bin/sh and since these are running on an Ubuntu Server system, /bin/sh is dash. Like many other shells, the exit value of a pipeline in dash is the exit value of the right-most command. This means that something like this will return 0 (success):

system("false | true")

If I were on a system whose /bin/sh were bash, I could fix it easily by adding the pipefail option:

system("set -o pipefail; false | true");

Indeed, this works as expected on my local Arch system whose /bin/sh points to bash:

$ perl -le '$status = system("false | true"); print $status>>8'
0
$ perl -le '$status = system("set -o pipefail; false | true"); print $status>>8'
1

(Yes, I know this looks weird, but if you don't know Perl, just take my word for it: we need to shift by 8 to get the actual exit status)

However, since the machine it is supposed to run on has dash, the same thing fails:

$ perl -le '$status = system("set -o pipefail; false | true"); print $status>>8'
sh: 1: set: Illegal option -o pipefail
2

Using set -e is also not a solution:

$ perl -le '$status = system("set -e; false | true"); print $status>>8'
0

I can think of some Perl workarounds, such as not piping or using the IPC::System::Simple module, but I was hoping there might be a simpler way to do this directly in dash.

So, is there some trick, hack or option that lets me tell dash that the exit status of a set of piped commands should only be 0 (success) if all commands worked and should be non-0 if any of them failed the way that set -o pipefail works in bash?

7
  • 1
    Not a solution, but an idea: You could run a bash from the dash that's started by perl. Something like system("bash -c 'set -o pipefail; false | true; echo $?'").
    – anick
    Jul 21, 2022 at 17:43
  • What programs do you need to run as a pipe using system? Wouldn't it be possible to implement parts of the pipeline (or even everything) in Perl?
    – Bodo
    Jul 21, 2022 at 17:48
  • 1
    @anick yeah, I could do that. It's ugly, but it should work. I might resort to it if I can't find anything simpler, thanks!
    – terdon
    Jul 21, 2022 at 17:56
  • 1
    @Bodo yes, it would be, but I'm largely wondering if there's some way of doing it in dash itself. I can think of various workarounds, but the thrust of this question is about whether there's some dash trick I don't know about.
    – terdon
    Jul 21, 2022 at 17:57
  • 2
    Related: stackoverflow.com/q/17757039/19456361 Jul 21, 2022 at 18:55

2 Answers 2

1

I don't know about doing it in Dash, but you can bypass /bin/sh completely.

Perl's system() tries to guess what the programmer wants, and uses sh -c only if there's just one argument, and that argument contains shell metacharacters. Otherwise it runs the given command directly, and you can specify the exact arguments (including the command name that goes in argv[0] or $0 in the shell, that's passed through an "indirect object").

The docs say:

Note that argument processing varies depending on the number of arguments. If there is more than one argument in LIST, or if LIST is an array with more than one value, starts the program given by the first element of the list with arguments given by the rest of the list.

If there is only one scalar argument, the argument is checked for shell metacharacters, and if there are any, the entire argument is passed to the system's command shell for parsing (this is /bin/sh -c on Unix platforms, but varies on other platforms).

These two would make the same execve() call:

system{"/bin/sh"} ("sh", "-c", 'false |true; echo $?')
system('false |true; echo $?')

Namely, this one, as shown by strace:

execve("/bin/sh", ["sh", "-c", "false |true; echo $?"], [/* 39 vars */]) = 0

(system("sh", "-c", '...') would be similar but would look in $PATH for sh, and system("/bin/sh", "-c", '...') would differ ever so slightly in that it'd pass /bin/sh in argv[0] for the launched shell instead of just sh. No big differences.)

Hence, we can just run this in Perl:

system("bash", "-o", "pipefail", "-c", 'false |true; echo $?');
# or a bit shorter
system(qw/bash -o pipefail -c/, 'false |true; echo $?');

and we avoid the round-trip through sh, and the associated double-quoting for the Bash command line.

0
1

Here's a really really ugly yet functional solution for you:

system(q{
  exec 3>&1; \
  status1="$(((false 2>&1 1>&3 3>&- 4>&-; echo $? >&4) \
    | true 1>&2 3>&- 4>&-) 4>&1)"; \
  status2=$? \
  [ $status2 != 0 ] && exit $status2; \
  exit $status1
});

You can remove those backslashes, line breaks, and indentation if you want it all on one line. This was adapted from the Csh Programming Considered Harmful essay, § 1d "More Elaborate Combinations" [of things Posix shell can do and Csh cannot].

This opens file descriptor 3 to temporarily store stdout, then runs the first command and re-plumbs where its output goes: FD4 and FD3 are closed, FD1 (standard output) goes into FD3, and FD2 (standard error) goes to FD1. Then it puts the exit code into FD4 before running the second command. The second command gets its FD4 and FD3 closed, then puts its standard output into FD2 (I believe its stderr actually prints and this may include stdout, but I may not be reading the magic correctly). With the output printed, it's safe to return FD4 to stdout and then that is what's stored in $status1. Assigning $status2 is more straightforward since the second command's status populates $?. Now we just have to implement pipefail. If $status2 is not a clean exit of 0, exit with it. Otherwise, exit with whatever $status1 stored.

Add new file descriptors (5, 6, etc) and $status variables for more pipes.

That gives you Posix compliance and portability, but you're far better off doing this more natively in perl or else using anick's advice and invoking bash (assuming that's an option), which certainly seems elegant in comparison:

system("bash", "-o", "pipefail", "-c", 'false |true; echo $?')

(Arguments broken out for safer implementation within perl, see ilkkachu's answer for details regarding that invocation.)

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .