4

If I run a command like:

cat <(echo 1 | pv) | pv
cat <(echo 1 | pv) | less
cat <(echo 1 | pv) | cat

The command seems to run forever. Entering ^C (SIGINT) kills the entire shell instead of just the commands executed. Why is this the case?

Relevant output of ps xf from another shell on minimal case cat <(pv) | less:

Ss  /bin/bash
S+   \_ cat /dev/fd/XX
S    |   \_ /bin/bash
T    |       \_ pv
S+   \_ less

With these file descriptors open:

bash

0 -> /dev/pts/YY
1 -> /dev/pts/YY
2 -> /dev/pts/YY
255 -> /dev/pts/YY

cat /dev/fd/ZZ

0 -> /dev/pts/YY
1 -> pipe:[RRRRRRRR]
2 -> /dev/pts/YY
3 -> pipe:[QQQQQQQQ]
ZZ -> pipe:[QQQQQQQQ]

bash

0 -> /dev/pts/YY
1 -> pipe:[QQQQQQQQ]
2 -> /dev/pts/YY
255 -> /dev/pts/YY

pv

0 -> /dev/pts/YY
1 -> pipe:[QQQQQQQQ]
2 -> /dev/pts/YY

less

0 -> pipe:[RRRRRRRR]
1 -> /dev/pts/YY
2 -> /dev/pts/YY
3 -> /dev/tty

Using the original example, cat <(echo 1 | pv) | less (this also happens when echo is not a bash builtin but another program like dd if=/dev/zero bs=1 count=1:

Ss   /bin/bash
S+    \_ cat /dev/fd/63
S     |   \_ /bin/bash
T     |       \_ pv
S+    \_ less

bash

0 -> /dev/pts/18
1 -> /dev/pts/18
2 -> /dev/pts/18
255 -> /dev/pts/18

cat /dev/fd/63

0 -> /dev/pts/18
1 -> pipe:[36932796]
2 -> /dev/pts/18
3 -> pipe:[36929317]
63 -> pipe:[36929317]

bash

0 -> /dev/pts/18
1 -> pipe:[36929317]
2 -> /dev/pts/18
255 -> /dev/pts/18

pv

0 -> pipe:[36930391]
1 -> pipe:[36929317]
2 -> /dev/pts/18

less

0 -> pipe:[36932796]
1 -> /dev/pts/18
2 -> /dev/pts/18
3 -> /dev/tty
Improve this question
26
  • 1
    When you say “entire shell”, do you mean the terminal-emulation-window as well?
    – ctrl-alt-delor
    Dec 30 '15 at 21:23
  • 1
    @richard - thats an excellent question.
    – mikeserv
    Dec 30 '15 at 21:25
  • 1
    And here's a minimal example that exhibits the problem: cat <(pv) | cat
    – Mark Plotnick
    Jan 1 '16 at 19:03
  • 1
    @arcyqwerty In cat <(pv) | cat, pv never gets a chance to read from stdin. It gets stopped with SIGTTOU very early on. Same behavior as in the larger examples. In fact, the code in pv's main explicitly sets TOSTOP on the tty that's on fd 2, with the comment "Set terminal option TOSTOP so we get signal SIGTTOU if we try to write to the terminal while backgrounded." As Mike said, bash may not be equipped to correctly handle job control signals inside a process substitution context.
    – Mark Plotnick
    Jan 5 '16 at 15:36
  • 1
    BTW, as a workaround, cat <(echo 1 | pv 2> >(cat) ) | whatever appears to work fine.
    – Mark Plotnick
    Jan 5 '16 at 16:02
3

It happens because the <( process ) isn't properly job-controlled - it's just forked and forgot. That doesn't matter most of the time, because almost as soon as it is born that process is placed in a separate process group and backgrounded. For the instant that the shell requires to open input and output for that process, however, that is the tty's foreground process group, and, as such, vulnerable to SIGINT - unless it is otherwise trapped or ignored as your interactive shell generally does.

The thing is, though: you have a pipe deadlock there. When the parent attempts to open output for that process the pipe blocks. It never gets the chance to change the process group and all the rest because when you CTRL+C the foreground group is killed - it's sent SIGINT - and when the foreground group dies and the parent cannot resume control because it's still blocked on a pipe, the terminal sends a HUP because there's nobody home. kaboom

You need a writer first, then a reader for every pipe you open before you can open another.

Improve this answer
9
  • This sounds like it's correct but I'm not entirely comprehending. From what I understand so far, echo has input: stdin and output: pipe1, pv has input: pipe1 and output: /dev/fd/XX, cat has input: /dev/fd/XX and output: pipe2, {pv,less,cat} has input: pipe2 and output: stdout. Which one(s) is deadlocked?
    – arcyqwerty
    Dec 30 '15 at 21:44
  • no. @arcyqwerty - it doesn't happen like that. the parent shell opens two subshells and connects i/o betwixt them via a pipe. the parent cant relinquish the pipe between them until the writing subshell opens it, too. but the writing subshell is busy opening its own subshell for the <() thing. and that subshell is waiting for cat. or maybe deeper - because that one opens two more subshells for its private pipeline, and then echo, pv or whatever. it goes deep. what's wrong with echo 1 | pv | pv?
    – mikeserv
    Dec 30 '15 at 21:50
  • Is the outer pipe or the <(...) subshell evaluated first? I'm not sure what order things are happening in. Best guess (using less example for unambiguity): 1. Shell spawns echo and pv, connects them with a pipe. Spawns cat with input as output of the <(...) subshell and spawns less. Tries to connect cat and less with a pipe. OR 2. Shell spawns cat <(...) and less. cat <(...) includes a subshell so it spawns a subshell with echo and pv connected by a pipe. The output of this subshell is redirected to cat. cat and less are connected via pipe.
    – arcyqwerty
    Dec 30 '15 at 22:00
  • @arcyqwerty - well... echo "$$" "$PPID"; echo "$(echo "$$" "$PPID")"; sh -c 'echo "$$" "$PPID"'; echo "$(sh -c 'echo "$PPID"')"; echo "$(exec sh -c 'echo "$PPID")" - if you can figure out the differences between the various PIDs printed by that mess, then the following might be a helpful way to troubleshoot: cat <(exec sh -c 'echo "$PPID"') | cat <(exec sh -c 'echo "$PPID"') - - unless that deadlocks, too.
    – mikeserv
    Dec 30 '15 at 22:08
  • I've updated the question with some additional information that may shed light onto this. I'm not entirely sure how to parse the info though. My best guess is that pv is being stopped due to some job control issue maybe having to do with pipes as well
    – arcyqwerty
    Jan 5 '16 at 5:39

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.