24

This is a simple script which runs nvidia-smi command on multiple hosts and saves its output to a common file. The goal here is to make it run asynchronously.

Is & at the end of the process_host() function call sufficient? Is my script correct?

#!/bin/bash

HOSTS=(host1 host2 host3)
OUTPUT_FILE=nvidia_smi.txt

rm $OUTPUT_FILE

process_host() {
    host=$1
    echo "Processing" $host
    output=`ssh ${host} nvidia-smi`
    echo ${host} >> $OUTPUT_FILE
    echo "$output" >> $OUTPUT_FILE
}

for host in ${HOSTS[@]}; do
    process_host ${host} &
done;

wait
cat $OUTPUT_FILE

1 Answer 1

25

Yes your script is correct, but your output may be garbled or out of sequence. It's better to have the function write its output to a specific file depending on the host name and then let the main script concatenate the result (and clean up).

Also, you should double quote your variables. Copy and paste the script into ShellCheck.

Maybe something like this:

#!/bin/bash

hosts=( host1 host2 host3 )
outfile="nvidia_smi.txt"

rm -f "$outfile"

function process_host {
    local host="$1"
    local hostout="$host.out"
    printf "Processing host '%s'\n" "$host"
    echo "$host" >"$hostout"
    ssh "$host" nvidia-smi >>"$hostout"
}

for host in "${hosts[@]}"; do
    process_host "$host" &
done

wait

for host in "${hosts[@]}"; do
    hostout="$host.out"
    cat "$hostout"
    rm -f "$hostout"
done >"$outfile"

cat "$outfile"

The last loop may be replaced by

cat "${hosts[@]/%/.out}" >"$outfile"
rm -f "${hosts[@]/%/.out}"

Having another look in 2021 at my old answer from 2016, I would probably write the code like this today:

#!/bin/sh

set -- host1 host2 host3
outfile=nvidia_smi.txt

for host do
    ssh "$host" nvidia-smi >"$outfile-$host" &
done

echo 'Commands submitted, now waiting...'
wait

for host do
    cat "$outfile-$host"
    rm -f "$outfile-$host"
done >"$outfile"

echo 'Done.'

This is shorter and since we don't really need any named arrays, we can run it all with /bin/sh in place of bash. The list of hosts to connect to is instead kept in the list of positional parameters. I've also removed informational output from within the loops, as well as the shell function (which really did not do very much).

A variant that uses proper temporary files instead of making up filenames that could potentially already be taken:

#!/bin/sh

set -- host1 host2 host3
outfile=nvidia_smi.txt

for host do
    tmpfile=$(mktemp)
    ssh "$host" nvidia-smi >"$tmpfile" &
    set -- "$@" "$tmpfile"
    shift
done

echo 'Commands submitted, now waiting...'
wait

# Concatenate the output files and delete them.
cat -- "$@" >"$outfile"
rm -f -- "$@"

echo 'Done.'

This replaces the hostnames in the positional parameters by the pathnames of the temporary files that contain the output of that host.

2
  • I get exactly the output I want to get, what could go wrong here?
    – syntagma
    Commented Aug 9, 2016 at 11:10
  • 3
    @ΔλЛ If, for example, the first host is slow to respond, Processing host1 will be followed by Processing host2 and the output from host2 rather than the output from host1.
    – Kusalananda
    Commented Aug 9, 2016 at 11:14

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .