3

I have a shell script which logs the performance of various programs passed to it as arguments, to help me choose the most performant one. The relevant snippet:

#!/usr/bin/env sh
function cal_perf () {
        real_t=0
        user_t=0
        sys_t=0
        for (( trial=1 ; trial<=$3 ; ++trial )) ; do
                shopt -s lastpipe
                /usr/bin/time -f '%e:%S:%U' $1 $2 |& IFS=":" read real sys user
                real_t=$(echo "$real + $real_t" | bc)
                user_t=$(echo "$user + $user_t" | bc)
                sys_t=$(echo "$sys + $sys_t" | bc)
        done
        real_t=$(echo "scale=2 ; $real_t / $3" | bc)
        user_t=$(echo "scale=2 ; $user_t / $3" | bc)
        sys_t=$(echo "scale=2 ; $sys_t / $3" | bc)
        printf "%s\t%d\t%.2f\t%.2f\t%.2f\n" $2 $3 $real_t $user_t $sys_t >> timings_$(date '+%Y%m%d')
}

# main
printf "program\t#trials\treal_time_am\tuser_time_am\tsys time_am\n" > timings_$(date '+%Y%m%d')
translator=$1
shift
while [ $# -gt 1 ] ; do cal_perf $translator $1 ${!#} ; shift ; done

It's supposed to be run on the command line as follows:

perf <translator_progam> <list_of_programs_to_compare> <number_of_trials>

...for instance: suppose I want to compare the performances of xip.py, foo.py, bar.py, bas.py, qux.py--the net content of the working directory--and run them each for 50 times before generating the stats; I'd invoke the script as:

perf python *py 50

I think I am missing something obvious here, but when I invoke this script as bash $HOME/bin/perf ... everything works as intended. However, the following two invocations fail (error attached):

  1. perf ...
  2. or even placing it in the working directory and invoking as ./perf ...

enter image description here

changing the shebang to /usr/bin/env bash solves this problem, but /usr/bin/sh points to /usr/bin/bash on my system.

8
  • 5
    The C-style for loop is a bashism, it doesn't exist in sh.
    – choroba
    Oct 8, 2021 at 15:27
  • True that, but should that be a concern when /usr/bin/sh is a symlink to /usr/bin/bash?
    – puwlah
    Oct 8, 2021 at 15:30
  • Are you sure /usr/bin/env sh resolves to /usr/bin/sh?
    – choroba
    Oct 8, 2021 at 15:40
  • 1
    Related (but not necessarily a duplicate) - Running a shell script when "/bin/sh" points to "/bin/bash"
    – roaima
    Oct 8, 2021 at 16:06
  • 5
    Regardless of whether sh is actually bash or not, why do you want to use sh when you know you use bash-specific functionality?
    – marcelm
    Oct 9, 2021 at 9:06

2 Answers 2

20

You are running the script with sh as the interpreter, but you are using bash features. Even assuming that sh points to bash, you still cannot do this as bash disables many of its custom features when asked to run as sh.

One solution is to declare the script correctly. You're using bash features so declare it as a bash script.

#!/bin/bash

Or, if you need continue with a dependency on env,

#!/usr/bin/env bash

Another option is to remove the bash-specific syntax.

In either case, don't forget to ensure variables are enclosed inside double-quotes when you use them. For example, if one of your unquoted variables (including script parameters) contains a space it will be split by the shell into two or more words.

4
  • @puwlah you should also revisit the /usr/bin/time ... | read real sys user construct, as it will not set any of those three variables. (Well, technically it will, but the shell will immediately discard them.) Generally you should also ensure your variables are inside double-quotes when you use them.
    – roaima
    Oct 8, 2021 at 16:03
  • (1) thanks for pointing out that bash disables many of its custom features when asked to run as sh. (2) word splitting wasn't a concern here, so quoting wasn't necessary: only place there's a risk of undesired word splitting is the expansion of $2 (files_names); which I am controlling. However, a good practice in general, agreed.... [1/2]
    – puwlah
    Oct 11, 2021 at 10:00
  • (3) The some_comand | read some variable(s) construct isn't universally useless. it won't be an issue with #!/[usr/]bin/[env ]bash because using bash as the interpreter allows for the (preceding) shopt -s lastpipe solution which executes the read to the right (of the stderr pipeline) in the current shell environment, and not a subshell. So, the variables assigned there can be accessed subsequently. [2/2].
    – puwlah
    Oct 11, 2021 at 10:00
  • @puwlah answer modified - I'd missed your shopt -s lastpipe
    – roaima
    Oct 11, 2021 at 10:32
9

The error comes when $3 expands to the empty string in

for (( trial=1 ; trial<=$3 ; ++trial ))

Bash does support for (( .. )) even in POSIX mode, so that in itself is not the issue. In this particular case, the problem is the ${!#} expansion used when calling the function. It can be parsed in two ways.

In a standard shell, it's the prefix-removal operation ${var#word} applied to the special parameter !, with an empty string to remove. Removing the empty string doesn't do anything, but for sake of an example, we could remove one character:

$ dash -c 'sleep 123 & echo $! ${!#?};'
27920 7920

But in Bash, ${!#} is taken as the indirect expansion ${!var}, using $# as the "pointer", giving the last positional parameter.

Bash does support the indirect expansion even in POSIX mode, but there, it treats that corner case in a more standard-conforming way. The documentation actually says this:

  1. While variable indirection is available, it may not be applied to the ‘#’ and ‘?’ special parameters.

Now, since $! contains the PID of the last background process, but the script doesn't start any, the standard meaning gives just the empty string. Leading to the eventual error.

You could also use "${@: -1}" to get the last positional parameter in Bash. It's not standard either, but does work in POSIX mode.

Anyway, in general, if you want to use non-POSIX features supported by Bash, like (( .. )), ${!var}, ${var:n:m}, run bash, not sh. You'll save yourself the trouble of dealing with the incompatibilities.

1
  • Note that of (( .. )), ${!var}, ${var:n:m}, only ${!var} comes from bash. The other two come from ksh. ${!var} in ksh has the almost opposite meaning. Oct 8, 2021 at 18:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.