3

I have the following bash function:

function exe {
    echo -e "Execute: $1"
    # Loops every 3s, outputting '...' until command finished executing
    LOOP=0
    while true;
    do
        if ! [ $LOOP == 0 ]; then echo -e "..."; fi;
        sleep 3;
        LOOP=$LOOP+1
    done & ERROR="$($2 2>&1)" # Execute the command and capture output to variable

    status=$?
    kill $!; trap 'kill $!' SIGTERM

    if [ $status -ne 0 ];
    then
        echo -e "✖ Error" >&2
        echo -e "$ERROR" >&2
    else
        echo -e "✔ Success"
    fi
    return $status
}

The intention is to call it as follows:

exe "Update apt indexes" \
    "sudo apt-get update"

Which outputs:

Execute: Update apt indexes
...
...
...
...
✔ Success

This works correctly except where a quoted string is used as a parameter in the passed command.

For example, the following does not work:

exe "Create self signed certificate" \
    "sudo openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout /etc/apache2/ssl/apache.key -out /etc/apache2/ssl/apache.crt -subj \"/C=GB/ST=London/L=London/O=Company Ltd/OU=IT Department/CN=dev.domain.local\""

set -x reveals the above command is transformed to the following for execution:

sudo openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout /etc/apache2/ssl/apache.key -out /etc/apache2/ssl/apache.crt -subj '"/C=GB/ST=London/L=London/O=Confetti' Celebrations Ltd/OU=IT 'Department/CN=dev.sign-in.confetti.local"'

Which appears to have picked up a number of single quotation marks and renders the command ineffective.

I'd like a version that doesn't have this limitation. Any ideas?

===============

My final code, after the suggestions and some other bug fixes is:

exe () {
    echo -e "Execute: $1"
    LOOP=0
    while true;
    do
        if ! [ $LOOP == 0 ]; then echo -e "..."; fi;
        sleep 3;
        LOOP=$((LOOP+1))
    done & ERROR=$("${@:2}" 2>&1)
    status=$?
    kill $!; trap 'kill $!' SIGTERM

    if [ $status -ne 0 ];
    then
        echo -e "✖ Error" >&2
        echo -e "$ERROR" >&2
    else
        echo -e "✔ Success"
    fi
    return $status
}

The function is intended as a 'beautifier' for a vagrant provisioning shell script and can be called as

exe "Update apt indexes" sudo apt-get update

The output appears as

Execute: Update apt indexes
...
...
...
...
✔ Success

Commands that last less than 3 seconds see no progress dots output

Unless there's an error, when you'll get an Error status back and the full output of the command.

The main intention is to rid vagrant provisioning scripts of the red lines shown when outputting messages to stderr. Many commands correctly output information to stderr as it is intended for messages that should not be piped to other commands. Vagrant print messages to stdout as This leaves a number of provisioning messages that look like errors but are not.

This function does not print to stderr unless the command executed returns a non zero status. This means that unless a command indicates a failure, you will not see red messages. When a command indicated failure with non zero message then we output the full output of the command to stderr, giving red lines.

Makes vagrant provisioning with a shell script much tidier and means we really can look out for the red messages knowing they mean something.

The full function for use with vagrant, including a little visual fluff I left out in the snippets above, can be seen here: https://gist.github.com/michaelward82/c1903f2b37a76975740e

Example output using exe function, with no errors: Example output using exe function, with  errors

Example output using exe function, with error: Example output using exe function, with error

Default output from executing commands directly, no actual errors: Default output from executing commands directly, no actual errors

  • 1
  • 1
    If you really want to go like this, try codepad.org/e2ymTBAf. Otherwise, I personally suggest passing the arguments as-is to the function like exe 'upd blah' apt-get install blah blah (so you have a normal shell list instead of a creepy string), and you can use ERROR=$("${@:1}" 2>&1) to access it. – Arthur2e5 Mar 4 '16 at 9:59
  • Interesting, thank you guys. Learning a lot more about bash shell scripting :) – michaelward82 Mar 4 '16 at 21:22
  • @Arthur2e5 Unquoting the commands and using ERROR=$("${@:2}" 2>&1) to execute has done the trick. Post it as an answer and I will mark it as correct. – michaelward82 Mar 4 '16 at 21:43
  • @michaelward82 wait, how can it be :2? That means 'starting from the third to the end'… – Arthur2e5 Mar 4 '16 at 22:11
2

You may not want to pass the whole command as a string. We have lists in shell, as the list-of-arguments, and passing a list as a list is much simpler.

Instead of writing exe blah "blahh cmd", you write the command directly like exe blah blahh cmd. Then, when you need to use the whole command directly, use the slicing expansion to get everything after the first argument: ERROR=$("${@:1}" 2>&1).

Traditionally people may use shift to shift the whole list-of-arguments 'left' (see help shift):

f(){
    local j="$1"
    shift
    echo "$j,$3"
    shift 50
    echo "$1" # guess what "$@" is now?
}

f {1..100}

But this is not necessary for bash apparantly.

Speaking of the slicing thing, you may also want to check out arrays in bash.


Ugh still.. You can use eval to run a string directly, but that's often considered a bad thing, since you are then allowing a lot more than simple commands.


And as a style hint, prefer the shorter and more (POSIX-)portable xxx() over function xxx and function xxx(). In bash they are just identical.

  • Fantastic, this has solved my issue. – michaelward82 Mar 5 '16 at 19:32
1

The core issue of your question is in "how to split an string" inside a $var.

The "evil" (because it is prone to mistakes and code execution) way is to use eval:

 eval set -- $var           ### Dangerous, not recommended, do not use.

That sets the split string in the positional arguments (an array is a bit more complex). But that the variable $var is unquoted (something to avoid at all costs unless you really know what you are doing) makes it subject to "word splitting" (which we want) but that also allow that "Pathname expansion" happen. You may try this commands (use a directory with few files)

$ var='hello * world'
$ eval set -- $var
$ echo "$@"

It is safe to execute, there are no externally set values and the expansion of * will just set the values in the positional parameters.

To avoid "Pathname expansion", a set -f is used, and in this case, is is easy to integrate in the command:

$ var='hello * world'
$ set -f
$ eval set -- $var
$ echo "$@"
hello * world

That is with a default IFS of spaceTabNew Line.

Things may get complex if IFS could have been set externally.

Several problems could be solved by using read:

$ IFS=' ' read -ra arr <<<"$var"
$ echo "${arr[@]}"
hello * world

That sets IFS for the command (avoid externally set IFS), read without processing backslash (the -r option), puts all inside an array variable (the -a option), and is using the variable quoted "$var". The only caveat is that the repeated spaces between words will be erased (because of IFS being an space). That is not a problem to an executable command line.

But trying to execute commands that need arguments with spaces will fail:

$ var='date -d "-1 day" +"%Y.%m.%d-%H:%M:%S"'
$ IFS=' ' read -ra arr <<<"$var"
$ "${arr[@]}"
date: extra operand `+"%Y.%m.%d-%H:%M:%S"'

The only real solution is that you build the array of the command correctly from the start:

$ arr=( date -d "-1 day" +"%Y.%m.%d-%H:%M:%S" )
$ "${arr[@]}"
2016.03.05-00:25:17

Think of this solution as a CSV "Comma (space) Separated Values".

This script will work:

#!/bin/bash

function exe {
    echo "Execute: $1"
    # Loops every 3s, outputting '...' until command finished executing
    LOOP=0
    while true; do
        if [ $LOOP -gt 0 ]; then echo -e "..."; fi;
            sleep 3;
            (( LOOP++ ))
    done &

    ERROR="$("${@:2}" 2>&1)" # Execute command and capture output.
    status=$?

    kill $!; trap 'kill $!' SIGTERM

    if [ $status -ne 0 ];
    then
        echo "✖ Error" >&2
        echo "$ERROR" >&2
    else
        echo "✔ Success"
    fi
    return $status
}

cmd=( date -d '-1 day' +'%Y.%m.%d-%H:%M:%S' )
exe "give me yesterday date" "${cmd[@]}" 

cmd=( sudo apt-get update )
exe "update package list" "${cmd[@]}" 
  • Wow, that's a lot of information to absorb! I'll sit down with the answer soon so I can study it in detail, I anticipate learning a few things :) – michaelward82 Mar 8 '16 at 14:04
0

In the case of quotes within parameter strings to be executed as code it is possible to reparse the parameter string into an array such as the positional parameter array $@. This can be achieved - at least for the given example - by using ... & ERROR="$( printf "%s" "$2" | xargs sh -c 'exec "$0" "$@" 2>&1' ) .... (There are cases with additional double quotes in that already quoted string that may cause xargs: unterminated quote messages).

For some futher suggestions please see: Linux/Bash: How to unquote?.

# test cases
# help :
#set -- '' "ls -ld / 'a bc'" 
set -- '' ": sudo openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout /etc/apache2/ssl/apache.key -out /etc/apache2/ssl/apache.crt -subj \"/C=GB/ST=London/L=London/O=Company Ltd/OU=IT Department/CN=dev.domain.local\""

printf "%s" "$2" | 
    xargs sh -c '
       echo "arg 0: ${0}"
       for ((i=1; i<=$#; i++)); do
          echo "arg $i: ${@:i:1}"
       done
       set -xv
       "$0" "$@"
    ' 

# output
arg 0: :
arg 1: sudo
arg 2: openssl
arg 3: req
arg 4: -x509
arg 5: -nodes
arg 6: -days
arg 7: 365
arg 8: -newkey
arg 9: rsa:2048
arg 10: -keyout
arg 11: /etc/apache2/ssl/apache.key
arg 12: -out
arg 13: /etc/apache2/ssl/apache.crt
arg 14: -subj
arg 15: /C=GB/ST=London/L=London/O=Company Ltd/OU=IT Department/CN=dev.domain.local
   "$0" "$@"
+ : sudo openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout /etc/apache2/ssl/apache.key -out /etc/apache2/ssl/apache.crt -subj '/C=GB/ST=London/L=London/O=Company Ltd/OU=IT Department/CN=dev.domain.local'

(And LOOP=$LOOP+1 in your code above should be LOOP=$((LOOP+1)) btw.)

  • Or ((LOOP++)). function xxx is bashism enough (though a not-so-beautiful one), and it's completely fine to add new ones… – Arthur2e5 Mar 4 '16 at 9:57
  • I have changed to LOOP=$((LOOP+1)) though the previous code appeared to do the job, but probably by luck and not design – michaelward82 Mar 4 '16 at 21:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.