4

I need to run a local script on remote servers. Running script to run in parallel is important.

[ec2-user@ip-172-31-43-140 ~]$ cat hosts.txt
             ec2-user@18.218.191.143
             ec2-user@18.220.183.27
             ec2-user@18.222.199.72
             ec2-user@13.58.207.76
             ec2-user@18.191.231.120
[ec2-user@ip-172-31-43-140 ~]$ cat hosts.txt  | xargs -I {} ssh {} -T 'bash -s' < ./file.sh
ssh: Could not resolve hostname #!/bin/sh: Name or service not known
xargs: ssh: exited with status 255; aborting
[ec2-user@ip-172-31-43-140 ~]$
  • xargs -P4 -I hostname ssh -T hostname bash -s "$(< file.sh)" <hosts.txt works if the current shell of the local machine is Bash. – Fólkvangr Jul 24 '18 at 8:57
5

I like this form when doing this:

$ cat hosts.txt | xargs -n1 -P8 sh -c 'ssh -T "$1" bash -s < ./hello.bash' sh

Or with {} if you need them:

$ cat hosts.txt | xargs -n1 -P8 -I{} sh -c 'ssh -T "$1" bash -s < ./hello.bash' sh {}

Example

hello.bash
$ cat hello.bash
#!/bin/bash

echo "hi from server: $(hostname)"
ssh xargs
$ cat hosts.txt | xargs -n1 -P8 sh -c 'ssh -T "$1" bash -s < ./hello.bash' sh
hi from server: mulder.mydom.com
hi from server: skinner.mydom.com
hi from server: manny.mydom.com

Details:

  • -n1 -P8 - tells xargs to take 1 argument as input and run 8 instances of ssh
  • sh -c 'ssh -T "$1"
    • this invokes a shell and then runs the command after -c "..."
    • notice we're passing $1 in here, this is to avoid injection attacks.
    • -T disables pseudo terminal. $1 is the contents of the file being cat'ed in.
  • bash -s < ./hello.bash' - the commands that will get passed to ssh
  • sh - the trailing sh is what gets passed to xargs as the shell to invoke, argument #0 ($0).

References

  • 1
    Yeah, I think it's safer to not invoke word splitting and filename generation. If not for any other reason than that error messages may be confusing in some situations when filename generation picks up random files. – Kusalananda Jul 23 '18 at 14:38
1

hello.bash:

doit() {
  echo Define
  echo what you want done in a
  echo function
}
export -f doit

Then do:

. hello.bash
parallel --env doit --slf hosts.txt --tag --nonall doit
0

Yes, you can do this with AWS Systems manager. AWS Systems Manager Run Command allows you to remotely and securely run set of commands on EC2 as well on-premise server. Below are high-level steps to achieve this.

Attach Instance IAM role: The ec2 instance must have IAM role with policy AmazonSSMFullAccess. This role enables the instance to communicate with the Systems Manager API.

Install SSM Agent: The EC2 instance must have SSM agent installed on it. The SSM Agent process the run command requests & configure the instance as per command.

Execute command : Example usage via AWS CLI: In --instance-ids give the list of ec2 instance ids. Execute the following command to retrieve the services running on the instance. Replace Instance-ID with ec2 instance id.

aws ssm send-command --document-name "AWS-RunShellScript" --comment "listing services" --instance-ids "Instance-ID" --parameters commands="service --status-all" --region us-west-2 --output text

More information here

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.