3

When we run this with a POSIX shell,

$ cmd0 | cmd1

STDOUT of cmd0 is piped to STDIN of cmd1.

Q: On top of this, how can I also pipe STDOUT of cmd1 to STDIN of cmd0?

Is it mandatory to use redirect from/into a named pipe (FIFO) ? I don't like named pipes very much because they occupy some filesystem paths and I need to worry about name collisions. Or do I have to call pipe(2) via C, Python or some general purpose programming languages?

(Both cmd0 and cmd1 do some network I/O, so they won't block each other forever.)

1
  • Good to know some *nixes have "bidirectional pipes"... I wasn't imaginative enough to search across the archive with such a term.
    – nodakai
    Commented Jul 18, 2016 at 5:19

1 Answer 1

18

On systems with bi-directional pipes (not Linux), you can do:

cmd0 <&1 | cmd1 >&0

On Linux, you can do:

: | { cmd1 | cmd2; } > /dev/fd/0

That works because on Linux (and Cygwin, but generally not other systems) /dev/fd/x where x is a fd to a pipe (named or not) acts like a named pipe, that is, opening it in read mode gets you the reading end and in write mode gets you the writing end.

With the yash shell and its x>>|y pipeline redirection operator:

{ cmd0 | cmd1; } >>|0

With shells with coproc support:

  • zsh:

       coproc cmd0
       cmd1 <&p >&p
    
  • ksh

       cmd0 |&
       cmd1 <&p >&p
    
  • bash4+

       coproc cmd0
       cmd1 <&"${COPROC[0]}" >&"${COPROC[1]}"
    

Dedicated tool approaches (shamelessly copied from answers on this very similar question):

Using pipexec

pipexec [ A /path/to/cmd0 ] \
        [ B /path/to/cmd1 ] \
        '{A:1>B:0}' '{A:0>B:1}'

Or using dpipe

dpipe cmd0 = cmd1

Or using socat:

socat EXEC:cmd0 EXEC:cmd1,nofork                 # using socketpairs
socat EXEC:cmd0,commtype=pipes EXEC:cmd1,nofork  # using pipes

I don't know about python, but that's also relatively easily done in perl:

perl -e 'pipe STDOUT,STDIN; exec "cmd0 | cmd1"'

You can always also resort to named pipes, but finding unique names for them and some safe directory to create them in, restricting access to them (so other processes can't open them and interfere) and cleanup afterwards become additional problems which are hard to overcome reliably and portably.

In any case, beware of deadlocks!

6
  • Doesn't the perl -e… example buffers piped data by default?
    – Kentzo
    Commented Jan 8, 2021 at 21:18
  • 1
    @Kentzo, not sure what you mean, but in any case, perl is not involved at all in the flow of data there, it just creates the pipe and execs the shell which runs cmd0 | cmd1 in the same process, so perl is gone by the time that shell starts, and the output of cmd1 will be connected to the input of cmd0 via that pipe created by perl earlier. Commented Jan 8, 2021 at 21:23
  • Perl buffers writes to STDOUT by defaults as far as I know. Thus data output by cmd1 may be retained by perl's internals until some threshold is reached before it's piped to STDIN.
    – Kentzo
    Commented Jan 8, 2021 at 21:31
  • 2
    @Kentzo, no again, perl doesn't read nor write anything here. It is gone (has been replaced by sh) by the time cmd0 and cmd1 are started. You can check with perl -e 'pipe STDOUT,STDIN; exec "ps -f >&2"' or perl -e 'pipe STDOUT,STDIN; exec q((lsof -p "$$" +E -ad0,1 >&2); exit)' to convince yourself. Commented Jan 9, 2021 at 7:58
  • You can simplify the Linux example to : | { cmd1 | cmd2; } >/dev/fd/0
    – user313992
    Commented Feb 11, 2021 at 23:45

Not the answer you're looking for? Browse other questions tagged .