I want to pipe two programs into one. If my shell supports it, I can use Process substitution. For example, to list the common lines of two files in indifferent order, I can use

comm -12 <(sort a) <(sort b)

However process substitution doesn't exist in plain sh. I can do it with full POSIX portability by creating a named pipe, but this is cumbersome as it requires finding a directory for the FIFO and cleaning up afterwards. A good practical compromise is to use two shell pipe constructs and use file descriptor shuffling to move one pipe to another file descriptor, then use /dev/fd to designate the pipe, which works on most Unix variants:

sort a | { exec 3<&0; sort b | comm -12 /dev/fd/0 /dev/fd/3; }

This works in dash, bash, BusyBox sh, etc. but not in ksh93 and mksh. Why?

$ mksh -c 'sort a | { exec 3<&0; sort b | comm -12 /dev/fd/0 /dev/fd/3; }'
$ ksh93 -c 'sort a | { exec 3<&0; sort b | comm -12 /dev/fd/0 /dev/fd/3; }'
comm: /dev/fd/0: No such device or address

Unlike redirections on other commands, redirections on the exec builtin may be closed when the shell executes an external program. POSIX allows both behaviors. Ksh (both ATT ksh, and pdksh and mksh) close these descriptors when they execute an external utility (i.e. for a redirection on the exec builtin, after calling dup2 to perform the redirection, they set the FD_CLOEXEC flag on the new descriptor). The Bourne shell, dash, bash, zsh and BusyBox sh treat this redirection like any other redirection.

A more portable solution to the two-input-pipes problem (assuming the existence of /dev/fd) is to perform another redirection on the command that reads the input, moving the file descriptor to a new one. This extra redirection doesn't set the close-on-exec flag on the new descriptor.

sort a | { exec 3<&0; sort b | comm -12 /dev/fd/0 /dev/fd/4 4<&3; }

This works in pdksh/mksh, and in ksh93r but not in recent versions of ksh (93s+ 2008-01-31 or 93u+ 2012-08-01). I don't understand what ksh is doing there.

| improve this answer | |
  • I think the issue with the recent versions of ksh is that you are using quite a low file descriptor (3) and ksh may have already occupied it for something else. Overall, fd 3 is stdlog, IIRC :) – galaxy Jan 23 at 9:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.