Bash Named Pipe - Atomic Reading

Question

I have set up one script to write to a named pipe, then four scripts to read the named pipe.

For most of the entries, there are no problems, but when there is little processing to be done on the lines of input, some of the reads start to conflict with each other and the resulting text is a combination of random characters from (up to) 4 lines for the (up to) 4 processes.

The reason I can't use xargs or parallel, is that in the connection time to the database takes 1 second, which adds up as there are usually ~1300 lines that need to be processed.

Example Output:

Completed Runstats on MAIN.MK37 in 1s: DB20000I The RUNSTATS command completed successfully.

Completed Runstats on MAIN.MKAP in 0s: DB20000I The RUNSTATS command completed successfully.

Completed Runstats on MAIN.MKAL in 0s: DB20000I The RUNSTATS command completed successfully.

Completed Runstats on MAIN.MK49 in 0s: DB20000I The RUNSTATS command completed successfully.

Failed Runstats on AI.K1MAIN after 0s: SQL2306N The table or index "AI.K1MAIN" does not exist.

Failed Runstats on MNM5. after 0s: SQL0104N An unexpected token "MNM5." was found following "TBLE". Expected tokens may include: "". SQLSTATE=42601

Completed Runstats on MAIN.MK50 in 0s: DB20000I The RUNSTATS command completed successfully.

Completed Runstats on MAIN.MK52 in 0s: DB20000I The RUNSTATS command completed successfully.

Writer:

# Open pipe for writing
exec 10>"${PIPE_LOCATION}"

# Feed data
while read LINE; do
    echo "${LINE}" >&10
done <<< "${LIST}"

# Tell threads to stop
i=0
while [ "${i}" -lt 4 ]; do
    echo "stop" >&10
    (( i += 1 ))
done

# Close pipe
exec 10>&-

Reader:

# Open Pipe
exec 10<"${PIPE_LOCATION}"

while read -u 10 -r SCHEMA TABLE; do
    if [[ "${SCHEMA}" == 'stop' ]]; then
        break
    fi

    #
    # Runstats Code Here
    #

done

# Close Pipe
exec 10<&-

Is there some way to make the reads atomic?

I was thinking I could coordinate the main thread like a server, and the four readers could send it a request every time it wanted some input, which would probably solve the problem (5 pipes instead of 1, though!), but if there's an easier solution someone can suggest, I'm all ears!

Answer 1

@MarkPlotnick had the right idea. I changed the script to pad the line with spaces and make read use fixed-size records.

Note: File descriptor is now 5 instead of 10, due to @Hauke Laging's findings

I now feed the data like so:

# Feed data
while read LINE; do
    printf "%64s" "${LINE}" >&5
done <<< "${LIST}"

And read it like so:

while read -u 5 -N 64 -r LINE; do
    LINE=(${LINE})
    SCHEMA=${LINE[0]}
    TABLE=${LINE[1]}

and there are no more errors!