1

I'm trying to write some functions I can utilize in my scripts to essentially tag all output as Error vs Standard output, throw date/time stamps on it, and also include the name of the function that the output came from. I have this all working except for buffering issues. My output essentially isn't in chronological order.

I've tried combinations of unbuffer, sync, stdbuf, etc, but I can't get it to work. Any help or explanation about why specifically this is failing for me would be appreciated. Also, any tips on a simpler execution of this would be great. So far, in order to get my function name I have to reconfigure my redirection on every single function call.

#!/bin/bash

function stdOutput
{
    typeset strLogFile=$1; shift
    typeset strLogID=$1; shift
    while IFS='' read -r strInput
    do
        echo "$(eval echo ${strLogID})${strInput}" && echo "$(eval echo ${strLogID})${strInput}" >> ${strLogFile}
    done
}

function errOutput
{
    typeset strLogFile=$1; shift
    typeset strLogID=$1; shift
    while IFS='' read -r strInput
    do
        >&2 echo "$(eval echo ${strLogID})${strInput}" && echo "$(eval echo ${strLogID})${strInput}" >> ${strLogFile}
    done
}

function main
{
    stdLogID="\<STD\>!"'$'"(date +\"%Y-%m-%d!%H:%M:%S\")!"
    errLogID="\<ERR\>!"'$'"(date +\"%Y-%m-%d!%H:%M:%S\")!"
    logFile=/tmp/out.log
    > $logFile

    exec 3>&1 1> >(stdOutput ${logFile} "${stdLogID}${FUNCNAME[0]}!")
    exec 4>&2 2> >(errOutput ${logFile} "${errLogID}${FUNCNAME[0]}!")

    >&2 echo "Line1"
    echo "Line2"
    >&2 echo "Line3"
    echo "Line4"
    >&2 echo "Line5"
    echo "Line6"
    >&2 echo "Line7"
    echo "Line8"
    >&2 echo "Line9"
    echo "Line10"

    exec 1>&3 3>&-
    exec 2>&4 4>&-
}
main
sync
exit

What I would hope to get as output from this script. Every other line being ERR then STD tagged and the line numbers in order 1-10.

<ERR>!2016-08-01!14:06:15!main!Line1
<STD>!2016-08-01!14:06:15!main!Line2
<ERR>!2016-08-01!14:06:15!main!Line3
<STD>!2016-08-01!14:06:15!main!Line4
<ERR>!2016-08-01!14:06:15!main!Line5
<STD>!2016-08-01!14:06:15!main!Line6
<ERR>!2016-08-01!14:06:15!main!Line7
<STD>!2016-08-01!14:06:15!main!Line8
<ERR>!2016-08-01!14:06:15!main!Line9
<STD>!2016-08-01!14:06:15!main!Line10

Example of the type of output I usually get. Line numbers out of order due to buffering.

<ERR>!2016-08-01!14:06:15!main!Line1
<STD>!2016-08-01!14:06:15!main!Line2
<STD>!2016-08-01!14:06:15!main!Line4
<ERR>!2016-08-01!14:06:15!main!Line3
<STD>!2016-08-01!14:06:15!main!Line6
<ERR>!2016-08-01!14:06:15!main!Line5
<STD>!2016-08-01!14:06:15!main!Line8
<ERR>!2016-08-01!14:06:15!main!Line7
<STD>!2016-08-01!14:06:15!main!Line10
<ERR>!2016-08-01!14:06:15!main!Line9
Improve this question
5
  • 3
    Is there any chance that you can reduce your script to a simpler, self-contained example that demonstrate the issue, which you can then include as part of the Question? (In case the pastebin gets removed, which would make this question completely unanswerable).
    – Jeff Schaller ♦
    Commented Aug 1, 2016 at 15:42
  • The script is already simplified and self-contained, but I can remove the internal documentation and paste the script in a code block.
    – Ryan Flagler
    Commented Aug 1, 2016 at 16:19
  • If your solution (posted as an edit) works for you, please remove it from the question and submit it as an answer (you are allowed - in fact, encouraged - to answer your own question if you can). As a bonus, you'll likely score more reputation that way!
    – Toby Speight
    Commented Aug 2, 2016 at 10:00
  • Thanks, as you can likely tell, I'm new to this site. I've added my solution as an answer.
    – Ryan Flagler
    Commented Aug 2, 2016 at 14:50
  • See stackoverflow.com/a/25548995/732419 the answer with stdout.
    – ott--
    Commented Aug 2, 2016 at 17:34

1 Answer 1

Reset to default
0

My problem was with the "echo" commands OUTSIDE of my functions that handled the redirection. Every attempt I made initially was INSIDE my function. To solve this, I created a new "echo" function that unbuffers each echo call, and use "command echo" when I don't want to utilize the function. This will obviously only unbuffer echo commands. Any other executable that outputs text could cause buffering as well.

#!/bin/bash

function stdOutput
{
    typeset strLogFile=$1; shift
    typeset strLogID=$1; shift
    while IFS='' read -r strInput
    do
    command echo "$(eval command echo ${strLogID})${strInput}" && command echo "$(eval command echo ${strLogID})${strInput}" >> ${strLogFile}
    done
}

function errOutput
{
    typeset strLogFile=$1; shift
    typeset strLogID=$1; shift
    while IFS='' read -r strInput
    do
    >&2 command echo "$(eval command echo ${strLogID})${strInput}" && command echo "$(eval command echo ${strLogID})${strInput}" >> ${strLogFile}
    done
}

function echo
{
    typeset strParameters=$@
    typeset strExecutable=$(which echo)
    unbuffer ${strExecutable} $@
}

function main
{
    stdLogID="\<STD\>!"'$'"(date +\"%Y-%m-%d!%H:%M:%S\")!"
    errLogID="\<ERR\>!"'$'"(date +\"%Y-%m-%d!%H:%M:%S\")!"
    logFile=/tmp/out.log
    > $logFile

    exec 3>&1 1> >(stdOutput ${logFile} "${stdLogID}"\$"{FUNCNAME[1]}!")
    exec 4>&2 2> >(errOutput ${logFile} "${errLogID}"\$"{FUNCNAME[1]}!")

    >&2 echo "Line1"
    echo "Line2"
    >&2 echo "Line3"
    echo "Line4"
    >&2 echo "Line5"
    echo "Line6"
    >&2 echo "Line7"
    echo "Line8"
    >&2 echo "Line9"
    echo "Line10"

    exec 1>&3 3>&-
    exec 2>&4 4>&-
}
main
sync
exit
Improve this answer

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .