0

I want to run a sequence of command pipelines with pv on each one. Here's an example:

for p in 1 2 3
do
  cat /dev/zero | pv -N $p | dd of=/dev/null &
done

The actual commands in the pipe don't matter (cat/dd are just an example)...

The goal being 4 concurrently running pipelines, each with their own pv output. However when I try to background the commands like this, pv stops and all I get are 4 stopped jobs. I've tried with {...|pv|...}&, bash -c "...|pv|..." & all with the same result.

How can I run multiple pv command pipelines concurrently?

1

Found that I can do this with xargs and the -P option:

josh@subdivisions:/# seq 1 10 | xargs -P 4 -I {} bash -c "dd if=/dev/zero bs=1024 count=10000000 | pv -c -N {} | dd of=/dev/null"
        3: 7.35GiB 0:00:29 [ 280MiB/s] [                                                                                         <=>                                                                 ]
        1: 7.88GiB 0:00:29 [ 312MiB/s] [                                                                                         <=>                                                                 ]
        4: 7.83GiB 0:00:29 [ 258MiB/s] [                                                                                         <=>                                                                 ]
        2: 6.55GiB 0:00:29 [ 238MiB/s] [                                                                                         <=>                                                                 ]

Send output of the array to iterate over into stdin of xargs; To run all commands simultaneously, use -P 0

1

pv cannot be started in background.

As you can see in the src/main/main.c file from pv's source code, they're setting the TOSTOP flag on the terminal with tcsetattr() (in the c.c_lflag of the termios structure). They're doing that in order to receive a SIGTTOU when trying to write to the terminal when not in foreground, catch it with a signal handler, and redirect the output to /dev/null in order not to "mess up" the terminal.

/*
 * Set terminal option TOSTOP so we get signal SIGTTOU if we try to
 * write to the terminal while backgrounded.
 *
 * Also, save the current terminal attributes for later restoration.
 */
memset(&t, 0, sizeof(t));
tcgetattr(STDERR_FILENO, &t);
t_save = t;
t.c_lflag |= TOSTOP;
tcsetattr(STDERR_FILENO, TCSANOW, &t);

This is of course gross, because it doesn't set that flag just for itself, but for all the programs using the terminal.

But that's not all. As explained in the glibc manual:

Function: int tcsetattr (int filedes, int when, const struct termios *termios-p)

If this function is called from a background process on its controlling terminal, normally all processes in the process group are sent a SIGTTOU signal, in the same way as if the process were trying to write to the terminal. The exception is if the calling process itself is ignoring or blocking SIGTTOU signals, in which case the operation is performed and no signal is sent. See Job Control.

They're not blocking or ignoring the SIGTTOU. And they're also not checking the return value of tcsetattr() (which would return -1 and set errno to EINTR if they had set the SIGTTOU signal handler before).

So the process is stopped. If it receives a SIGCONT (as from the bg command), it will stop again when trying to complete the tcsetattr().

So I guess you should consider it a feature ;-)

  • Thanks! This is really useful to understand the behavior I was seeing :) – Josh Sep 28 '18 at 21:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.