How to put all function/command call output results to different corresponding vars: for stderr, for stdout, for string result and for return code?

Question

I want to extend question How to store standard error in a variable and get general solution (bash function or functions) to extract from the called function/command together with standard error also all other possible outcomes.

The function should capture and return as vars for the called function/command the following:

• stderr
• stdout
• result string out var
• return code

API for the function should be convenient to use.

---

Example of test function that produces all different output results:

func_to_test_all_outs() {
    local param="${1:?}"; shift
    local -n out_result=${1:?}; shift

    echo "test error output" >&2
    echo "test normal output"

    out_result=""
    if [[ "$param" = "return_string_result" ]]; then
        out_result="result string"
    fi

    return 3
}

More comprehensive/general example of test function that produces all different output results including spawning background processes to produce stout and stderro, interleaving stdout/stderr output, and including leading/trailing/mid white space plus potential command injection, variable expansion, and globbing chars in output streams and does redirection of FD3 in case the calling script relies on that:

func_to_test_all_outs() {
    local param="${1:?}"; shift
    local -n out_result=${1:?}; shift

    ( sleep 2; printf '\n   first  \n$RANDOM *\nerror output\n' >&2; ) &
    exec 3>&1
    printf ' first  \n$(date)\n*\nnormal output\n\n' >&3

    exec 3>&2
    printf '\nsecond $RANDOM *\nerror output\n\n\n' >&3
    ( sleep 2; printf 'second\ntest\n$(date)\n*\nnormal output\n\n' ) &

    ( printf 'third\n$(date)\n*\nnormal output\n\n' ) &

    out_result=""
    if [[ "$param" = "return_string_result" ]]; then
        printf -v out_result '\n\nresult\t\n\nstring\n\n\n'
    fi

    return 127
}

Answer 1

cat capture.sh

#!/bin/bash

get_tmp_dir() {
    local tmp_dir="/tmp"  # default dir
    local virtual_tmp_dir="/mnt/tmpfs"
    if [ -d "$virtual_tmp_dir" ]; then
        tmp_dir="$virtual_tmp_dir"  # if tmp partition in RAM exsits it will be used
    fi
    echo $tmp_dir
}

DEBUG_ENABLED=0  # 0 is disabled, 1 is enabled
LOG_TO_FILE=0  # 0 is disabled, 1 is enabled
LOG_FILE="$(get_tmp_dir)/capture.log"

script_name=$0

help="
Usage: 

$script_name ( tests | t (tests-alias) | any-test-name | any-func-name [param...] ) [ Options]

Options:

-h, --help
-lc, --log-to-console   Enabling debug logs to be printed to console
-lf, --log-to-file      Enabling logging to file. Log file: $LOG_FILE

--------------------

Examples:

$script_name -h

Running all tests:
$script_name tests

Instead of tests t alias can be used:
$script_name t

Running some specific test (or any other function inside the script with its params) with printing debug logs to console:
$script_name test_capture_stderr -lc

Enabling logging to file:
$script_name t -lf
"

log_info() {  # log version that always prints to console
    local date
    printf -v date '%(%Y-%m-%d %H:%M:%S)T' -1;
    local msg="$date - $*"

    printf '%s\n' "$msg"

    if [ "$LOG_TO_FILE" -eq 1 ]; then
        printf '%s\n' "$msg" >> "$LOG_FILE"
    fi
}

# prints to console only when debug is enabled
# prints to file when logging to file is enabled
log_debug() { 
    if [[ "$LOG_TO_FILE" -eq 0 ]] && [[ "$DEBUG_ENABLED" -eq 0 ]]; then
       return
    fi

    local params=$1; shift
    local param
    for param in "$@"; do
        params+=" '$param'"
    done

    local date
    printf -v date '%(%Y-%m-%d %H:%M:%S)T' -1;
    local msg="$date - $params"

    if [ "$LOG_TO_FILE" -eq 1 ]; then
        printf '%s\n' "$msg" >> "$LOG_FILE"
    fi
    if [ "$DEBUG_ENABLED" -eq 1 ]; then
        printf '%s\n' "$msg"
    fi
}

# For capturing only stderr
# Captures stderr from a command to a variable while maintaining stdout
# @Args:
# $1: The variable name to store the stderr output
# $2: Vararg command and arguments
# @Return:
# The Command's Return-Code or 1 if missing argument
# source: https://stackoverflow.com/a/60936303/1455694
capture_stderr() {
  local -n stderr="${1:?}"; shift
  { stderr="$({ "$@" 1>&3; } 2>&1)"; } 3>&1
}

func_to_test_all_outs() {
    local param="${1:?}"; shift
    local -n out_result=${1:?}; shift
    log_debug "func_to_test_all_outs - param: $param; out_result: $out_result"

    echo "test error output" >&2
    echo "test normal output"

    out_result=""
    if [[ "$param" = "return_string_result" ]]; then
        out_result="result string"
    fi
    log_debug "returning result - out_result: $out_result"

    return 3
}

failed_asserts_amount=0
all_asserts_amount=0

assert_equals() {
    local error_message=$1; shift
    local expected_result=$1; shift
    local actual_result=$1; shift

    if [ "$expected_result" != "$actual_result" ]; then
        log_info "$error_message; Expected: $expected_result; Actual: $actual_result"
        (( failed_asserts_amount++ ))
    fi
    (( all_asserts_amount++ ))
}

print_tests_results () {
    if [ $failed_asserts_amount -eq 0 ]; then
        log_info "All tests passed"
    else
        log_info "Number of failed asserts: $failed_asserts_amount"
    fi
    log_info "Total run asserts amount: $all_asserts_amount"
}

# For non english terminal local run with:
# LANG=C ./capture.sh test_capture_stderr
test_capture_stderr() {
    #test for capturing ls command error output
    local expected_stderr="ls: cannot access '123': No such file or directory"
    local actual_stderr
    local expected_return_code=2
    local actual_return_code

    capture_stderr actual_stderr ls "123"
    actual_return_code=$?

    assert_equals "test_capture_stderr" "$expected_stderr" "$actual_stderr"
    assert_equals "test_capture_stderr" "$expected_return_code" "$actual_return_code"

    #test for function
    local expected_stderr="test error output"
    local actual_stderr
    local expected_return_code=3
    local actual_return_code
    local expected="result string"
    local result
    capture_stderr actual_stderr func_to_test_all_outs "return_string_result" result
    actual_return_code=$?

    assert_equals "test_capture_stderr-10" "$expected_stderr" "$actual_stderr"
    assert_equals "test_capture_stderr-20" "$expected_return_code" "$actual_return_code"
}

# Calls specified as parameter command/function and captures
# stderr, stdout, string result and puts them to separate corresponding out params.
# Return code is returned for called command/function.
# See test_capture_all for usage example.
# $1 - name of the variable where the contents of stderr will be stored
# $2 - same but for stdout. But stdout with "no_stdout_redirect" value will skip using it
# $3 - called function string out parameter. "no_result" value will skip using it
# $4 - command/function to be executed
# $5, $6, ... - parameters for the command/function
capture_all() {
    local -n stderr="${1:?}"; shift
    local -n stdout="${1:?}"; shift
    local -n result="${1:?}"; shift # result is string out param, not return code
    log_debug "capture_all - stderr: $stderr; stdout: $stdout; result: $result"

    local tmp_dir="/tmp"
    local virtual_tmp_dir="/mnt/tmpfs"
    if [ -d "$virtual_tmp_dir" ]; then
        tmp_dir="$virtual_tmp_dir"
    fi
    # local tmp_file_for_stderr=$(mktemp)  # standard tmp file creation in /tmp
    local tmp_file_for_stderr=$(mktemp "${tmp_dir}/tmp_file_for_stderr.XXXXXXXXXX")
    local tmp_file_for_stdout=$(mktemp "${tmp_dir}/tmp_file_for_stdout.XXXXXXXXXX")
    log_debug "tmp_file_for_stderr: $tmp_file_for_stderr"
    log_debug "tmp_file_for_stdout: $tmp_file_for_stdout"
    
    # better to separate var declaration with local mark and $? assignment
    local -i return_code
    if [[ "$result" = "no_result" ]]; then
        if [[ "$stdout" = "no_stdout_redirect" ]]; then
            log_debug "extract only stderr"
            "$@" 2> "$tmp_file_for_stderr"
        else
            log_debug "extract both stderr and stdout"
            "$@" 2> "$tmp_file_for_stderr" > "$tmp_file_for_stdout"
        fi
    else
        if [[ "$stdout" = "no_stdout_redirect" ]]; then
            log_debug "extract result string + stderr"
            "$@" result 2> "$tmp_file_for_stderr"
        else
            log_debug "extract all: result string + stderr and stdout"
            "$@" result 2> "$tmp_file_for_stderr" > "$tmp_file_for_stdout"
        fi

    fi

    # this cammand must be the first command after calling "$@"
    return_code="$?"  # return code for called function/command will be also returned
     
    stderr="$(<"$tmp_file_for_stderr")"  # stderr is alsways returned
    if [[ "$stdout" != "no_stdout_redirect" ]]; then
        stdout="$(<"$tmp_file_for_stdout")"
    fi

    log_debug "DEBUG_ENABLED: $DEBUG_ENABLED; LOG_TO_FILE: $LOG_TO_FILE "
    # keep tmp files in case of enabled debug options
    if (( DEBUG_ENABLED != 1 )) && (( LOG_TO_FILE != 1 )); then
        rm -- "$tmp_file_for_stderr" "$tmp_file_for_stdout"
    fi
    return "$return_code"
}

# for capturing only stderr and var with output string result
capture_stderr_outres() {
    local -n stderr2="${1:?}"; shift
    local -n result2="${1:?}"; shift
    log_debug "capture_stderr_outres - stderr: $stderr2; result: $result2"
    local stdout2="no_stdout_redirect"

    capture_all stderr2 stdout2 result2 "$@"
}

test_capture_stderr_outres() {
    local expected_stderr="test error output"
    local actual_stderr
    local expected_return_code=3
    local actual_return_code
    local expected="result string"
    local actual="some previous value"  # to ensure that test will change it

    capture_stderr_outres actual_stderr actual func_to_test_all_outs "return_string_result"
    actual_return_code=$?

    assert_equals "test_capture_stderr_outres-10" "$expected_return_code" "$actual_return_code"
    assert_equals "test_capture_stderr_outres-20" "$expected_stderr" "$actual_stderr"
    assert_equals "test_capture_stderr_outres-40" "$expected" "$actual"
}

test_capture_all() {
    #test for capturing ls error output
    local expected_stderr="ls: cannot access '123': No such file or directory"
    local actual_stderr
    local expected_stdout=""
    local actual_stdout
    local expected_return_code=2
    local actual_return_code

    local actual="no_result"  # disabling string result out var, since ls command does not have out param
    capture_all actual_stderr actual_stdout actual ls "123"
    actual_return_code=$?

    assert_equals "test_capture_all-70" "$expected_stderr" "$actual_stderr"
    assert_equals "test_capture_all-80" "$expected_stdout" "$actual_stdout"
    assert_equals "test_capture_all-90" "$expected_return_code" "$actual_return_code"

    #test for returning string
    local expected_stderr="test error output"
    local actual_stderr
    local expected_stdout="test normal output"
    local actual_stdout
    local expected_return_code=3
    local actual_return_code
    local expected="result string"
    local actual="some previous value"  # to ensure that test will change it

    capture_all actual_stderr actual_stdout actual func_to_test_all_outs "return_string_result"
    actual_return_code=$?

    assert_equals "test_capture_all-10" "$expected_return_code" "$actual_return_code"
    assert_equals "test_capture_all-20" "$expected_stderr" "$actual_stderr"
    assert_equals "test_capture_all-30" "$expected_stdout" "$actual_stdout"
    assert_equals "test_capture_all-40" "$expected" "$actual"

    #test for returning empty result
    local expected=""
    local actual="some previous value"  # to ensure that test will change it
    capture_all actual_stderr actual_stdout actual func_to_test_all_outs "do_not_return_string_result"
    assert_equals "test_capture_all-50" "$expected" "$actual"
}

tests() {
    test_capture_stderr
    test_capture_stderr_outres
    test_capture_all

    print_tests_results
}

t() {
    tests
}

for param in "$@"; do
    case $param in
        -h|--help) echo "$help"; exit ;;
        -lc|--log-to-console) DEBUG_ENABLED=1; log_info "log_debug to console option applied" ;;
        -lf|--log-to-file) LOG_TO_FILE=1; log_info "log to file option applied" ;;
        -*|--*) echo "Unknown parameter passed: $1. Use -h flag for help"; exit 1 ;;
    esac
done

"$@"

• capture_stderr — captures only stderr. Example of usage in test_capture_stderr
• capture_stderr_outres — captures stderr and called function out var string. Example of usage in test_capture_stderr_outres
• capture_all — captures stderr, stdout, called function out var string. Example of usage in test_capture_all

All capture functions also return return code of called function/command.

Help for the script available with -h flag:

./capture.sh -h

Usage: 

./capture.sh ( tests | t (tests-alias) | any-test-name | any-func-name [param...] ) [ Options]

Options:

-h, --help
-lc, --log-to-console   Enabling debug logs to be printed to console
-lf, --log-to-file      Enabling logging to file. Default log file: /tmp/capture.log

--------------------

Examples:

./capture.sh -h

Running all tests:
./capture.sh tests

Instead of tests t alias can be used:
./capture.sh t

Running some specific test (or any other function inside the script with its params) with printing debug logs to console:
./capture.sh test_capture_stderr -lc

Enabling logging to file:
./capture.sh t -lf

Virtual partition in RAM can be created with this recipe: How to make a temporary partition in RAM?

If temporary partition in RAM "/mnt/tmpfs" exists tmp files will be created there, otherwise in standard "/tmp" dir. Read gurus comments about this approach here. Probably you will not like the idea with using this temporary virtual partition in RAM. It is OK. Just continue using standard "/tmp" dir. And I will continue using the virtual partition until get obvious proofs that "/tmp" dir lives in RAM and not on SSD/HDD degrading them.