Script using fifos not generating output when processing from stdin

Question

I'm trying to use named pipes to process pieces of input data in parallel before pasting back the results together. I had something working until I added the possibility to take the input from stdin (following this answer).

Here I report my problem using a simplified example which just selects columns and do no further processing before pasting, but the actual script does more than that.

Example data:

$ cat data.txt
A1  A2  A3  A4  A5
B1  B2  B3  B4  B5
C1  C2  C3  C4  C5
D1  D2  D3  D4  D5
E1  E2  E3  E4  E5
F1  F2  F3  F4  F5
G1  G2  G3  G4  G5
H1  H2  H3  H4  H5
I1  I2  I3  I4  I5
J1  J2  J3  J4  J5
K1  K2  K3  K4  K5
L1  L2  L3  L4  L5

Script using normal files:

$ cat test_files.sh
#!/bin/bash

get_1()
{
    cut -f1 - > ${1}
}

get_3()
{
    cut -f3 - > ${1}
}

get_5()
{
    cut -f5 - > ${1}
}


setup()
{
    workdir=$(mktemp -d)
    col1="${workdir}/col1.txt"
    col3="${workdir}/col3.txt"
    col5="${workdir}/col5.txt"
}

setup

cleanup()
{
    rm -rf ${workdir}
}

if [ $# -ge 1 -a -f "${1}" ]
then
    cat ${1} \
        | tee >(get_1 ${col1}) \
        | tee >(get_3 ${col3}) \
        | get_5 ${col5} \
        || { echo "cat failed" && cleanup && exit 1; }
else
    cat - \
        | tee >(get_1 ${col1}) \
        | tee >(get_3 ${col3}) \
        | get_5 ${col5} \
        || { echo "cat failed" && cleanup && exit 1; }
fi

paste ${col1} ${col3} ${col5}
cleanup
exit 0

This one works well in both modes:

$ ./test_files.sh data.txt
A1  A3  A5
B1  B3  B5
C1  C3  C5
D1  D3  D5
E1  E3  E5
F1  F3  F5
G1  G3  G5
H1  H3  H5
I1  I3  I5
J1  J3  J5
K1  K3  K5
L1  L3  L5
$ cat data.txt | ./test_files.sh
A1  A3  A5
B1  B3  B5
C1  C3  C5
D1  D3  D5
E1  E3  E5
F1  F3  F5
G1  G3  G5
H1  H3  H5
I1  I3  I5
J1  J3  J5
K1  K3  K5
L1  L3  L5

Here is a version using fifos, where I perform the column extraction in the background and paste from the fifos:

$ cat test_fifos.sh
#!/bin/bash

get_1()
{
    cut -f1 - > ${1}
}

get_3()
{
    cut -f3 - > ${1}
}

get_5()
{
    cut -f5 - > ${1}
}


setup()
{
    workdir=$(mktemp -d)
    col1="${workdir}/col1.txt"
    mkfifo ${col1}
    col3="${workdir}/col3.txt"
    mkfifo ${col3}
    col5="${workdir}/col5.txt"
    mkfifo ${col5}
}

setup

cleanup()
{
    rm -rf ${workdir}
}

if [ $# -ge 1 -a -f "${1}" ]
then
    cat ${1} \
        | tee >(get_1 ${col1}) \
        | tee >(get_3 ${col3}) \
        | get_5 ${col5} \
        || { echo "cat failed" && cleanup && exit 1; } &
else
    cat - \
        | tee >(get_1 ${col1}) \
        | tee >(get_3 ${col3}) \
        | get_5 ${col5} \
        || { echo "cat failed" && cleanup && exit 1; } &
fi

paste ${col1} ${col3} ${col5}
cleanup
exit 0

It works when taking an input file as argument:

$ ./test_fifos.sh data.txt
A1  A3  A5
B1  B3  B5
C1  C3  C5
D1  D3  D5
E1  E3  E5
F1  F3  F5
G1  G3  G5
H1  H3  H5
I1  I3  I5
J1  J3  J5
K1  K3  K5
L1  L3  L5

But when taking the data from stdin, there is no output:

$ cat data.txt | ./test_fifos.sh
$ # Nothing here, no error message

Doing some experiments to generate a minimal example, I realized that the code to handle possible errors seemed to be part of the problem. Here is a version using fifos and not trying to handle errors:

$ cat test_fifos_noerr.sh
#!/bin/bash

get_1()
{
    cut -f1 - > ${1}
}

get_3()
{
    cut -f3 - > ${1}
}

get_5()
{
    cut -f5 - > ${1}
}


setup()
{
    workdir=$(mktemp -d)
    col1="${workdir}/col1.txt"
    mkfifo ${col1}
    col3="${workdir}/col3.txt"
    mkfifo ${col3}
    col5="${workdir}/col5.txt"
    mkfifo ${col5}
}

setup

cleanup()
{
    rm -rf ${workdir}
}

if [ $# -ge 1 -a -f "${1}" ]
then
    cat ${1} \
        | tee >(get_1 ${col1}) \
        | tee >(get_3 ${col3}) \
        | get_5 ${col5} &
else
    cat - \
        | tee >(get_1 ${col1}) \
        | tee >(get_3 ${col3}) \
        | get_5 ${col5} &
fi

paste ${col1} ${col3} ${col5}
cleanup
exit 0

This one works in both modes:

$ ./test_fifos_noerr.sh data.txt
A1  A3  A5
B1  B3  B5
C1  C3  C5
D1  D3  D5
E1  E3  E5
F1  F3  F5
G1  G3  G5
H1  H3  H5
I1  I3  I5
J1  J3  J5
K1  K3  K5
L1  L3  L5
$ cat data.txt | ./test_fifos_noerr.sh
A1  A3  A5
B1  B3  B5
C1  C3  C5
D1  D3  D5
E1  E3  E5
F1  F3  F5
G1  G3  G5
H1  H3  H5
I1  I3  I5
J1  J3  J5
K1  K3  K5
L1  L3  L5

Why is there no output when I both handle possible errors and use fifos while taking the data from stdin?

---

Edit: Some debugging

I added some debugging output to the failing script:

$ cat test_fifos.sh
#!/bin/bash

get_1()
{
    >&2 echo "get_1"
    cut -f1 - > ${1}
}

get_3()
{
    >&2 echo "get_3"
    cut -f3 - > ${1}
}

get_5()
{
    >&2 echo "get_5"
    cut -f5 - > ${1}
}


setup()
{
    workdir=$(mktemp -d)
    col1="${workdir}/col1.txt"
    mkfifo ${col1}
    col3="${workdir}/col3.txt"
    mkfifo ${col3}
    col5="${workdir}/col5.txt"
    mkfifo ${col5}
}

setup

cleanup()
{
    >&2 echo "cleanup"
    rm -rf ${workdir}
}

if [ $# -ge 1 -a -f "${1}" ]
then
    >&2 echo "then"
    cat ${1} \
        | tee >(get_1 ${col1}) \
        | tee >(get_3 ${col3}) \
        | get_5 ${col5} \
        || { >&2 echo "cat failed" && cleanup && exit 1; } &
else
    >&2 echo "else"
    cat - \
        | tee >(get_1 ${col1}) \
        | tee >(get_3 ${col3}) \
        | get_5 ${col5} \
        || { >&2 echo "cat failed" && cleanup && exit 1; } &
fi

>&2 echo "before paste"
paste ${col1} ${col3} ${col5}
>&2 echo "after paste"
cleanup
exit 0

Here is what happens when data is read from stdin:

$ cat data.txt | ./test_fifos.sh
else
before paste
get_3
get_5
get_1
after paste
cleanup

So this means that the else branch is executed.

Answer 1

The problem with using & to run commands in the background is that the shell will automatically close stdin for that command. So your cat - is immediately reading end-of-file. As a workaround you can duplicate file descriptor 0 to 3, for example, and use that:

...
else    exec 3<&0
        cat - <&3 \
        | tee >(get_1 ${col1}) \
...