Using while loop to ssh to multiple servers

Question

I have a file servers.txt, with list of servers:

server1.mydomain.com
server2.mydomain.com
server3.mydomain.com

when I read the file line by line with while and echo each line, all works as expected. All lines are printed.

$ while read HOST ; do echo $HOST ; done < servers.txt
server1.mydomain.com
server2.mydomain.com
server3.mydomain.com

However, when I want to ssh to all servers and execute a command, suddenly my while loop stops working:

$ while read HOST ; do ssh $HOST "uname -a" ; done < servers.txt
Linux server1 2.6.30.4-1 #1 SMP Wed Aug 12 19:55:12 EDT 2009 i686 GNU/Linux

This only connects to the first server in the list, not to all of them. I don't understand what is happening here. Can somebody please explain?

This is even stranger, since using for loop works fine:

$ for HOST in $(cat servers.txt ) ; do ssh $HOST "uname -a" ; done
Linux server1 2.6.30.4-1 #1 SMP Wed Aug 12 19:55:12 EDT 2009 i686 GNU/Linux
Linux server2 2.6.30.4-1 #1 SMP Wed Aug 12 19:55:12 EDT 2009 i686 GNU/Linux
Linux server3 2.6.30.4-1 #1 SMP Wed Aug 12 19:55:12 EDT 2009 i686 GNU/Linux

It must be something specific to ssh, because other commands work fine, such as ping:

$ while read HOST ; do ping -c 1 $HOST ; done < servers.txt

Answer 1

ssh is reading the rest of your standard input.

while read HOST ; do … ; done < servers.txt

read reads from stdin. The < redirects stdin from a file.

Unfortunately, the command you're trying to run also reads stdin, so it winds up eating the rest of your file. You can see it clearly with:

$ while read HOST ; do echo start $HOST end; cat; done < servers.txt 
start server1.mydomain.com end
server2.mydomain.com
server3.mydomain.com

Notice how cat ate (and echoed) the remaining two lines. (Had read done it as expected, each line would have the "start" and "end" around the host.)

Why does for work?

Your for line doesn't redirect to stdin. (In fact, it reads the entire contents of the servers.txt file into memory before the first iteration). So ssh continues to read its stdin from the terminal (or possibly nothing, depending on how your script is called).

Solution

At least in bash, you can have read use a different file descriptor.

while read -u10 HOST ; do ssh $HOST "uname -a" ; done 10< servers.txt
#          ^^^^                                       ^^

ought to work. 10 is just an arbitrary file number I picked. 0, 1, and 2 have defined meanings, and typically opening files will start from the first available number (so 3 is next to be used). 10 is thus high enough to stay out of the way, but low enough to be under the limit in some shells. Plus its a nice round number...

Alternative Solution 1: -n

As McNisse points out in his/her answer, the OpenSSH client has an -n option that'll prevent it from reading stdin. This works well in the particular case of ssh, but of course other commands may lack this—the other solutions work regardless of which command is eating your stdin.

Alternative Solution 2: second redirect

You can apparently (as in, I tried it, it works in my version of Bash at least...) do a second redirect, which looks something like this:

while read HOST ; do ssh $HOST "uname -a" < /dev/null; done < servers.txt

You can use this with any command, but it'll be difficult if you actually want terminal input going to the command.

Answer 2

As derobert describes ssh reads your stdin.

To change this behavior you can add -n no ssh to prevent it from reading stdin.

ssh -n $HOST "uname -a"

Answer 3

You'd probably actually be better off using pssh from the parallel-ssh project.

pssh -h $hostfile -t $timeout -i $commands

-i means interactive. pssh also comes with a parallel scp and parallel rsync. What's nice is that it runs asynchronously and will run as many threads as you ask it to. The default (non -i/interactive) is to output to separate directories for stdout/stderr, which it does by $outputdir/$hostname.

Answer 4

If you find yourself doing this kind of task quite often, you should try Fabric

Install the Fabric following the instructions, most cases you just need sudo apt-get install fabric

Create a file named fabfile.py with following code:

from fabric.api import env, run

env.hosts = ['server1.mydomain.com',
             'server1.mydomain.com',
             'server1.mydomain.com']

def mytask():
    run('uname -a')

Then run fab mytask will give you the result you want.

Answer 5

Because a ssh commnand take all stream from standard input, fed by the while statement,

You can use a pipe to switch the stdin of ssh to another source:

echo "" | ssh ...

example :

while read HOST ; do echo "" | ssh $HOST "uname -a" ; done < servers.txt

stdin input of all ssh commands in a while loop must be switched to another source.

Answer 6

It's more easy using a command like this:

for f in `cat servers.txt`; do ssh $f uname -a; done

I usually do like this:

for f in `cat servers.txt`; do echo "### $f ###"; ssh $f uname -a; done

The echo is to see which server is stuck, or cannot connect to it.