Capture process signal "Terminated" message output in shell pipeline

Question

When a process is ended by a signal, a message is output like Terminated or Killed depending on the signal used. Is it possible to capture that message ?

Take this example pipeline:

while true; do timeout 2 sleep 5; done

There is no output from it, however the sleep is terminated every 2 seconds. Add a cat:

while true; do timeout 2 sleep 5 | cat; done

It now outputs Terminated every 2 seconds but cat doesn't see it - if I replace cat with tee somefile then the file remaims empty.

I've tried redirecting 2>&1 in various places on the command-line but I cannot get the Terminated outputs into the file.

I think they're output by the libc linked into the terminal's shell rather than the commands in the loop.

Is it possible to get the Terminated and similar outout into the pipeline?

(Ultimately, I want to replace the sleep with a curl and the cat with an awk script and that processes both the curl output and the Terminated messages.)

Answer 1

Analysis

Yes, it's the shell what prints Terminated. Upon timeout 2 sleep 5 | cat you see Terminated because cat gets terminated. And I do mean cat.

This is explained in this another answer of mine. timeout sends SIGTERM to its entire process group and in your case the group includes sleep and cat.

The process group includes timeout as well. It appears timeout is not killed by its own signal and this is the reason sole timeout 2 sleep 5 does not make the shell print Terminated. But if you do timeout -s KILL 2 sleep 5 then you will see Killed from the shell because SIGKILL can neither be caught nor ignored. When timeout kills sleep with SIGKILL, it also kills itself and the shell reacts.

So it's cat what matters for this Terminated you observed. Now if you replace sleep with curl, cat with awk, then timeout will terminate curl and awk. This doesn't matter though, because it's the shell what prints the message. If you really want to capture the message then you need awk in a pipe after the shell.

---

Solution

If you really want to capture the message then you need awk in a pipe after the shell that prints the message. And you need a "victim" process that gets terminated, without it there will be no message. And you need a shell that runs timeout in a process group with PGID equal to its PID, because this is crucial for the mechanism (see the linked answer). E.g. awk in this pipeline should successfully process the message:

bash -ic 'timeout 2 sleep 5 | cat' 2>&1 | awk '{print "This is the message: "$0}'

-i is important; without it timeout would not kill our poor condemned cat. Strictly it's not about interactive or non-interactive Bash. It's about job control being enabled (and it is enabled by default in interactive Bash) or disabled (by default in non-interactive). So this will also work:

bash -c 'set -m; timeout 2 sleep 5 | cat' 2>&1 | awk '{print "This is the message: "$0}'

---

More elegant solution

There's a simpler way to do what you want. If ultimately you want to run this in a script, then you can take advantage of the fact a shell interpreting a script should not create separate process groups for commands. In a non-interactive script timeout … | awk will not kill awk.

Now all you need is timeout -v that will tell you what it does. The following example is designed to be run as a non-interactive script. If you want to test it in an interactive Bash, you can, but you must disable job control first (set +m). This is the script:

#/bin/sh
timeout -v 2 sleep 5 2>&1 | awk '{print "This is the message: "$0}'

Replace sleep with curl. Your awk should look for timeout: sending signal TERM to command '…' instead of trying to detect Terminated.