How to wait for program to finish, and pipe stdin to stdout when it's done?

Question

I'm trying to figure out how to wait for a command to complete, and then pipe stdin to stdout. I'm on a mac, but I think my question is more about how to wait for a process to complete and pipe the output then anything to do with a mac.

I've noticed that on the mac, I can run a few say commands together, and they wait for each phrase to be fully spoken before starting the next one, if I use the && operator to join them up.

$ say "stage 1" && say "stage 2"

Here's where the real use case is - I've got a bash script that I'd like to have it just pass stdin to stdout, after it finishes saying something.

$ cat /etc/passwd | say_and_pass "stage 1" | grep -v test | say_and_pass "stage 2"

So conceptually, this would say aloud, "stage 1", then immediately say "stage 2", and then dump the grepped contents of /etc/password to stdout.

My initial crack at the say_and_pass script is this:

say_and_pass

#!/usr/bin/env bash
OUT="$*"
say "$OUT" && cat 

But it doesn't seem to work ;-)

EDIT: I changed the example above to use say_and_pass "stage2" as the final command, which is needed for my solution to work...

Answer 1

Here's where the real use case is - I've got a bash script that I'd like to have it just pass stdin to stdout, after it finishes saying something.

$ cat /etc/password | say_and_pass "stage 1" | grep -v test | say "stage 2"

The problem with what you're trying to do here is that you seem to have two data streams- one for say and one for text processing (i. e. cat and grep).

You can't use a single pipeline for this, as it will just commingle the data. cat /etc/passwd | say will have your computer attempt to speak the entire contents of the file. Furthermore, say does not write anything to standard output, so nothing would go any farther down the pipeline.

If you do want to "interrupt" your data processing stream to handle separate output to other utilities like say, you need to either set yourself up with a FIFO (i. e. a "named pipe") for one or the other of your data streams, or use temporary files written to disk.

Your demonstrative use-case isn't very helpful for an example, as notwithstanding the says, you're just grepping a file, which can be done in one step with grep -v test /etc/passwd (cat file | grep pattern is a useless use of cat).

All that said, an example using a scratch file:

scratch=$(mktemp)
trap "rm -f $scratch" EXIT
cat /etc/passwd > $scratch
say "Stage one"
grep -v test $scratch
say "Stage two"

And one used a named pipe for say:

mkfifo youtalktoomuch
say youtalktoomuch &
exec 3> youtalktoomuch
do_thing_one
echo "Stage one" > youtalktoomuch
do_thing_two
echo "Stage two" > youtalktoomuch
exec 3>&-
rm youtalktoomuch

Once again, though, if you need to connect the output of `do_thing_one` to the input of `do_thing_two` without piping one into the other, you will need to use either another named pipe or a scratch file on disk to hold the data.

Answer 2

This version of say_and_pass does what OP asks for:

#!/usr/bin/env bash

# Create a temporary file to store stdin
TEMP_STDIN="$(mktemp)"

# Capture stdin and save it to temporary file
cat - > "${TEMP_STDIN}"

# say message, but prevent any output to stdout
say "$@" > /dev/null

# Dump saved stdin to stdout
cat "${TEMP_STDIN}"

# Clean up, but prevent any output to stdout
rm -f "${TEMP_STDIN}" > /dev/null

It uses a temporary file, like @DopeGhoti suggests, but keeps any of the other logic (the grep, for instance) out of the script. This makes it so that say_and_pass is more general and reusable.

My interpretation of the goal/purpose is that we want an audible way to monitor progress of a potentially long pipeline.

So when running:

cmd1 | say_and_pass "stage 1" | cmd2 | say_and_pass "stage 2"

As a user, you can audibly know that cmd1 has fully completed its execution; the trade-off is that cmd2 cannot begin processing its stdin until cmd1 is complete. But it seems that the OP is aware of this trade-off and willing to make it.

This is similar to the answer by @BradParks, but is not limited by the memory available to bash (as @agc points out), and is slightly more careful about avoiding accidental pollution of stdout.

Also note that there's no need to capture the message into a variable of its own, so we also simply forward "$@" to say.

Answer 3

Turns out all I had to do was consume all of stdin at the start of my script... I don't want it to process stdin as it's received, but when the entire previous command in the pipe chain completes.

So my say_and_pass script is simply this, which works for my use case!

#!/usr/bin/env bash

OUT=$(cat -)

MSG="$*"
say "$MSG"

echo "$OUT"

Answer 4

Try:

say "stage 1" && cat /etc/passwd  |  { say "stage 2" && grep -v test ; }

I don't actually have say on my system, but this works:

espeak "stage 1" && cat /etc/passwd  |  { espeak "stage 2" && grep -v test ; }

Answer 5

H!

I'd suggest, based on your own answer, to just dump stdin to stdout, removing the part which stores it to a variable. This should reduce memory consumption, and make the pipe chain work as pipes should (one line in, it gets processed, and the result sent to the next process).

#!/usr/bin/env bash

MSG="$*"
say "$MSG"

cat -

Answer 6

#!/usr/bin/env bash
OUT="$*"
say "$OUT" | cat -

If you use &&, its meaning , do fisrt command after do second command but fisrt command result not important for second command. I mean you have 2 different command, because you use &&. If you use |, its means fisrt command do something and give something for second command. The end, you should give a attribute for cat command. Sorry my english by the way