1

I'm trying to write some functions I can utilize in my scripts to essentially tag all output as Error vs Standard output, throw date/time stamps on it, and also include the name of the function that the output came from. I have this all working except for buffering issues. My output essentially isn't in chronological order.

I've tried combinations of unbuffer, sync, stdbuf, etc, but I can't get it to work. Any help or explanation about why specifically this is failing for me would be appreciated. Also, any tips on a simpler execution of this would be great. So far, in order to get my function name I have to reconfigure my redirection on every single function call.

#!/bin/bash

function stdOutput
{
    typeset strLogFile=$1; shift
    typeset strLogID=$1; shift
    while IFS='' read -r strInput
    do
        echo "$(eval echo ${strLogID})${strInput}" && echo "$(eval echo ${strLogID})${strInput}" >> ${strLogFile}
    done
}

function errOutput
{
    typeset strLogFile=$1; shift
    typeset strLogID=$1; shift
    while IFS='' read -r strInput
    do
        >&2 echo "$(eval echo ${strLogID})${strInput}" && echo "$(eval echo ${strLogID})${strInput}" >> ${strLogFile}
    done
}

function main
{
    stdLogID="\<STD\>!"'$'"(date +\"%Y-%m-%d!%H:%M:%S\")!"
    errLogID="\<ERR\>!"'$'"(date +\"%Y-%m-%d!%H:%M:%S\")!"
    logFile=/tmp/out.log
    > $logFile

    exec 3>&1 1> >(stdOutput ${logFile} "${stdLogID}${FUNCNAME[0]}!")
    exec 4>&2 2> >(errOutput ${logFile} "${errLogID}${FUNCNAME[0]}!")

    >&2 echo "Line1"
    echo "Line2"
    >&2 echo "Line3"
    echo "Line4"
    >&2 echo "Line5"
    echo "Line6"
    >&2 echo "Line7"
    echo "Line8"
    >&2 echo "Line9"
    echo "Line10"

    exec 1>&3 3>&-
    exec 2>&4 4>&-
}
main
sync
exit

What I would hope to get as output from this script. Every other line being ERR then STD tagged and the line numbers in order 1-10.

<ERR>!2016-08-01!14:06:15!main!Line1
<STD>!2016-08-01!14:06:15!main!Line2
<ERR>!2016-08-01!14:06:15!main!Line3
<STD>!2016-08-01!14:06:15!main!Line4
<ERR>!2016-08-01!14:06:15!main!Line5
<STD>!2016-08-01!14:06:15!main!Line6
<ERR>!2016-08-01!14:06:15!main!Line7
<STD>!2016-08-01!14:06:15!main!Line8
<ERR>!2016-08-01!14:06:15!main!Line9
<STD>!2016-08-01!14:06:15!main!Line10

Example of the type of output I usually get. Line numbers out of order due to buffering.

<ERR>!2016-08-01!14:06:15!main!Line1
<STD>!2016-08-01!14:06:15!main!Line2
<STD>!2016-08-01!14:06:15!main!Line4
<ERR>!2016-08-01!14:06:15!main!Line3
<STD>!2016-08-01!14:06:15!main!Line6
<ERR>!2016-08-01!14:06:15!main!Line5
<STD>!2016-08-01!14:06:15!main!Line8
<ERR>!2016-08-01!14:06:15!main!Line7
<STD>!2016-08-01!14:06:15!main!Line10
<ERR>!2016-08-01!14:06:15!main!Line9
  • 3
    Is there any chance that you can reduce your script to a simpler, self-contained example that demonstrate the issue, which you can then include as part of the Question? (In case the pastebin gets removed, which would make this question completely unanswerable). – Jeff Schaller Aug 1 '16 at 15:42
  • The script is already simplified and self-contained, but I can remove the internal documentation and paste the script in a code block. – Ryan Flagler Aug 1 '16 at 16:19
  • If your solution (posted as an edit) works for you, please remove it from the question and submit it as an answer (you are allowed - in fact, encouraged - to answer your own question if you can). As a bonus, you'll likely score more reputation that way! – Toby Speight Aug 2 '16 at 10:00
  • Thanks, as you can likely tell, I'm new to this site. I've added my solution as an answer. – Ryan Flagler Aug 2 '16 at 14:50
  • See stackoverflow.com/a/25548995/732419 the answer with stdout. – ott-- Aug 2 '16 at 17:34
0

My problem was with the "echo" commands OUTSIDE of my functions that handled the redirection. Every attempt I made initially was INSIDE my function. To solve this, I created a new "echo" function that unbuffers each echo call, and use "command echo" when I don't want to utilize the function. This will obviously only unbuffer echo commands. Any other executable that outputs text could cause buffering as well.

#!/bin/bash

function stdOutput
{
    typeset strLogFile=$1; shift
    typeset strLogID=$1; shift
    while IFS='' read -r strInput
    do
    command echo "$(eval command echo ${strLogID})${strInput}" && command echo "$(eval command echo ${strLogID})${strInput}" >> ${strLogFile}
    done
}

function errOutput
{
    typeset strLogFile=$1; shift
    typeset strLogID=$1; shift
    while IFS='' read -r strInput
    do
    >&2 command echo "$(eval command echo ${strLogID})${strInput}" && command echo "$(eval command echo ${strLogID})${strInput}" >> ${strLogFile}
    done
}

function echo
{
    typeset strParameters=$@
    typeset strExecutable=$(which echo)
    unbuffer ${strExecutable} $@
}

function main
{
    stdLogID="\<STD\>!"'$'"(date +\"%Y-%m-%d!%H:%M:%S\")!"
    errLogID="\<ERR\>!"'$'"(date +\"%Y-%m-%d!%H:%M:%S\")!"
    logFile=/tmp/out.log
    > $logFile

    exec 3>&1 1> >(stdOutput ${logFile} "${stdLogID}"\$"{FUNCNAME[1]}!")
    exec 4>&2 2> >(errOutput ${logFile} "${errLogID}"\$"{FUNCNAME[1]}!")

    >&2 echo "Line1"
    echo "Line2"
    >&2 echo "Line3"
    echo "Line4"
    >&2 echo "Line5"
    echo "Line6"
    >&2 echo "Line7"
    echo "Line8"
    >&2 echo "Line9"
    echo "Line10"

    exec 1>&3 3>&-
    exec 2>&4 4>&-
}
main
sync
exit

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.