4

I need to execute an application in parallel on multiple Ubuntu servers while supplying different arguments for different servers. I tried to google it, but could not get to the possible solution. I even experimented with ssh/pdsh/parallel, but without success.

To explain the scenario further, here is a non-working example (with pdsh) where script.sh should be executed on all 3 servers in parallel but with different arguments. FYI, I already have public/private ssh-key (password-free login) in place.

pdsh -w server1,server2,server3 -l username script.sh args

Where args should be 1 for server1, 2 for server2 etc.

I would appreciate if someone can help me achieve this, either using pdsh or some other tool available in Ubuntu.

  • is key based auth there ? – Rahul Patil Sep 22 '13 at 7:02
  • If you are talking about ssh public/private key auth, then yes its there. – Sachin Sep 22 '13 at 7:19
  • have you tested with parallel ? – Rahul Patil Sep 22 '13 at 9:56
3

You can use GNU version of parallel

For example, to call echo with the arguments 1 through 10, dispatching each echo command to one of server.example.com or server2.example.net:

seq 10 | parallel --sshlogin server.example.com,server2.example.net echo
  • Thank you Rahul for your suggestion, but it will not work for me. I need to provide different arguments to different servers, which is not possible in the provided example. After receiving a reply on other forum, I am planning to write a script to issue these requests serially in background. – Sachin Sep 22 '13 at 19:18
  • echo arg1 arg2 | parallel --sshlogin server.example.com,server2.example.net --colsep ' ' echo {1} {2} – Rahul Patil Sep 22 '13 at 19:50
  • also watch video youtube.com/watch?v=OpaiGYxkSuQ – Rahul Patil Sep 22 '13 at 19:51
  • @Sachin Each input line is passed as an argument to a different server (or, if there are more arguments than servers, several commands will be executed on the same server). – Gilles Sep 22 '13 at 23:16
  • Thanks Rahul and Gilles for your help. From the link of "GNU version of parallel", it seems that parallel tries to execute command on each core on remote node. Instead of this, I just want to have a single instance of application per remote node irrespective of number of cores available. For now, a script issuing jobs in a loop over servers and putting jobs in background (using pdsh) is working for me. – Sachin Sep 23 '13 at 5:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.