5

What is the best practice for return many values from a bash function?

Example1:

Function-script:

function mysqlquery {
    local dbserver='localhost'
    local dbuser='user'
    local dbpass='pass'
    local db='mydb'
    mysql -h "$dbserver" -u "$dbuser" -p "$dbpass" --skip-column-names --raw -e "$*" "$db"
    if [ $? -ne 0 ]; then
        return 1
    fi
}

Source-script:

for XY in $(mysqlquery "select XY from ABC where DEF = 123" 2>/dev/null);do
    dosomethingwith $XY
done
if mysqlquery "select XY from ABC where DEF = 123" 2>/dev/null; then
    echo true
fi

Example2:

Function-script:

function mysqlquery {
    local dbserver='localhost'
    local dbuser='user'
    local dbpass='pass'
    local db='mydb'
    result=$(mysql -h "$dbserver" -u "$dbuser" -p "$dbpass" -e "$*" "$db" 2>/dev/null)
    if [ $? -ne 0 -o -z "$result" ]; then
        return 1
    fi
}

Source-script:

result=$(mysqlquery "select XY from ABC where DEF = 123" 2>/dev/null)
for XY in $result;do
    dosomethingwith $XY
done
if mysqlquery "select XY from ABC where DEF = 123" 2>/dev/null; then
    echo true
fi

Or are there more approaches to returning multiple pieces of information (much more then a single int value)?

migrated from unix.meta.stackexchange.com Dec 3 '17 at 13:35

This question came from our discussion, support, and feature requests site for users of Linux, FreeBSD and other Un*x-like operating systems.

5

Yes, bash's return can only return numbers, and only integers between 0 and 255.

For a shell that can return anything (lists of things), you can look at es:

$ es -c "fn f {return (a 'b c' d \$*)}; printf '%s\n' <={f x y}"
a
b c
d
x
y

Now, in Korn-like shells like bash, you can always return the data in a pre-agreed variable. And that variable can be in any type supported by the shell.

For bash, that can be scalar, sparse arrays (associative arrays with keys restricted to positive integers) or associative arrays with non-empty keys (neither key nor values can contain NUL characters).

See also zsh with normal arrays and associative arrays without those restrictions.

The equivalent of the f es function above could be done with:

f() {
  reply=(a 'b c' d "$@")
}
f
printf '%s\n' "${reply[@]}"

Now, mysql queries generally return tables, that is two-dimensional arrays. The only shell that I know that has multi-dimensional arrays is ksh93 (like bash it doesn't support NUL characters in its variables though).

ksh also supports compound variables that would be handy to return tables with their headers.

It also supports passing variables by reference.

So, there, you can do:

function f {
  typeset -n var=$1
  var=(
    (foo bar baz)
    (1 2 3)
  }
}
f reply
printf '%s\n' "${reply[0][1]}" "${reply[1][2]}"

Or:

function f {
  typeset -n var=$1
  var=(
    (firstname=John lastname=Smith)
    (firstname=Alice lastname=Doe)
  )
}

f reply
printf '%s\n' "${reply[0].lastname}"

Now, to take the output of mysql and store that in some variables, we need to parse that output which is text with columns of the table separated by TAB characters and rows separated by NL and some encoding for the values to allow them to contain both NL and TAB.

Without --raw, mysql would output a NL as \n, a TAB as \t, a backslash as \\ and a NUL as \0.

ksh93 also has read -C that can read text formatted as a variable definition (not very different from using eval though), so you can do:

function mysql_to_narray {
  awk -F '\t' -v q="'" '
    function quote(s) {
      gsub(/\\n/, "\n", s)
      gsub(/\\t/, "\t", s)
      gsub(/\\\\/, "\\", s)
      gsub(q, q "\\" q q, s)
      return q s q
    }
    BEGIN{print "("}
    {
      print "("
      for (i = 1; i <= NF; i++)
        print " " quote($i)
      print ")"
    }
    END {print ")"}'
}

function query {
  typeset -n var=$1
  typeset db=$2
  shift 2

  typeset -i n=0
  typeset IFS=' '
  typeset credentials=/path/to/file.my # not password on the command line!
  set -o pipefail

  mysql --defaults-extra-file="$credentials" --batch \
        --skip-column-names -e "$*" "$db" |
    mysql_to_narray |
    read -C var
}

To be used as

query myvar mydb 'select * from mytable' || exit
printf '%s\n' "${myvar[0][0]}"...

Or for a compound variable:

function mysql_to_array_of_compounds {
  awk -F '\t' -v q="'" '
    function quote(s) {
      gsub(/\\n/, "\n", s)
      gsub(/\\t/, "\t", s)
      gsub(/\\\\/, "\\", s)
      gsub(q, q "\\" q q, s)
      return q s q
    }
    BEGIN{print "("}
    NR == 1 {
      for (i = 1; i<= NF; i++) header[i] = $i
      next
    }
    {
      print "("
      for (i = 1; i <= NF; i++)
        print " " header[i] "=" quote($i)
      print ")"
    }
    END {print ")"}'
}

function query {
  typeset -n var=$1
  typeset db=$2
  shift 2

  typeset -i n=0
  typeset IFS=' '
  typeset credentials=/path/to/file.my # not password on the command line!
  set -o pipefail

  mysql --defaults-extra-file="$credentials" --batch \
        -e "$*" "$db" |
    mysql_to_array_of_compounds |
    read -C var
}

To be used as:

query myvar mydb 'select "First Name" as firstname, 
                         "Last Name" as lastname from mytable' || exit

printf '%s\n' "${myvar[0].firstname"

Note that the header names (firstname, lastname above) have to be valid shell identifiers.

In bash or zsh or yash (though beware array indices start at 1 in zsh and yash and only zsh can store NUL characters), you could always return one array per column, by having awk generate the code to define them:

query() {
  typeset db="$1"
  shift

  typeset IFS=' '
  typeset credentials=/path/to/file.my # not password on the command line!
  set -o pipefail

  typeset output
  output=$(
    mysql --defaults-extra-file="$credentials" --batch \
          -e "$*" "$db" |
      awk -F '\t' -v q="'" '
        function quote(s) {
          gsub(/\\n/, "\n", s)
          gsub(/\\t/, "\t", s)
          gsub(/\\\\/, "\\", s)
          gsub(q, q "\\" q q, s)
          return q s q
        }
        NR == 1 {
          for (n = 1; n<= NF; n++) column[n] = $n "=("
          next
        }
        {
          for (i = 1; i < n; i++)
            column[i] = column[i] " " quote($i)
        }
        END {
          for (i = 1; i < n; i++)
            print column[i] ") "
        }'
  ) || return
  eval "$output"
}

To be used as:

query mydb 'select "First Name" as firstname, 
                         "Last Name" as lastname from mytable' || exit

printf '%s\n' "${firstname[1]}"

Add a set -o localoptions with zsh or local - with bash4.4+ before the set -o pipefail for the setting of that option to be local to the function like with the ksh93 approach.

Note that in all the above, we're not converting back the \0s to real NULs as bash or ksh93 would choke on them. You may want to do it if using zsh to be able to work with BLOBs but note that the gsub(/\\0/, "\0", s) would not work with all awk implementations.

In any case, here, I'd use more advanced languages than a shell like perl or python to do this kind of thing.

1

Well, it depends on what kind of an output format you want/need. The easiest way is probably to just print the output from the function, that way the function behaves like any other command. Another way would be to set some variable (possibly an associative array) from inside the function. That has the benefit that different items are separated cleanly, but you may need to hard code some variables.

The function in your first example implements the former: whatever the mysql client prints from the function goes to the standard output of the function. Given that the data already comes as a stream of bytes, keeping it as such is fine.

Though here, the question becomes what to do with the output. for x in $(somecmd) ... isn't good, since the output of somecmd is split to words and processed for filename globs. It's usually better to use while read ..., see How can I read a file (data stream, variable) line-by-line (and/or field-by-field)?

To read the output from mysql line by line, you could do

mysql -h "$dbserver" etc. etc. | while read -r line ; do
    dosomethingwith "$line"
done

or with a function

mysqlquery() {
    ...
    mysql -h "$dbserver" etc. etc. 2>/dev/null
}
mysqlquery | while read -r line ; do ...

Note that you don't need the if [ $? -ne 0 ]; then return 1: the return value from the function is the same as that of the last command. Not that it's easy to look into the return value if you're using the function to feed a pipe.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.