1

How can I run a script in multiple instances in Ubuntu server?

For example, I have a long running script named scanner.sh placed in /usr/bin/, which I wish to run the script in 10 parallel instances. How can I start the parallel tasks (and pass different flags / options / parameters to them)?

and how can I stop the instances?

1
#**parallel_scaller.sh**
# Script to async run scanner.sh
# USAGE: $> bash parallel_scanner.sh "FLAGS FOR INSTANCE 1" ... "FLAGS FOR INSTANCES 10"
$1=`echo $1 | sed 's/"//g'`
$2=`echo $1 | sed 's/"//g'`
$3=`echo $1 | sed 's/"//g'`
$4=`echo $1 | sed 's/"//g'`
$5=`echo $1 | sed 's/"//g'`
$6=`echo $1 | sed 's/"//g'`
$7=`echo $1 | sed 's/"//g'`
$8=`echo $1 | sed 's/"//g'`
$9=`echo $1 | sed 's/"//g'`
$10=`echo $1 | sed 's/"//g'`

bash scanner& $1 2>&1
bash scanner& $2 2>&1
bash scanner& $3 2>&1
bash scanner& $4 2>&1 
bash scanner& $5 2>&1
bash scanner& $6 2>&1
bash scanner& $7 2>&1
bash scanner& $8 2>&1
bash scanner& $9 2>&1
bash scanner& $10 2>&1

& places task in background. 2>&1 redirects STDOUT to STDERR.

To end the processes, type:

$> ps aux | grep scanner
$> kill $PIDS
  • Thanks. But how to stop specific instance? Also, the lines below should be: bash scanner.sh $1 2>&1, right ? – Raptor Oct 3 '14 at 15:00
1

Using GNU Parallel it looks like this:

parallel scanner {} ::: "--flags --for --instance 1" "--for 2" "--for 3"

It will spawn one job per CPU. To stop simply press CTRL-C.

GNU Parallel is a general parallelizer and makes is easy to run jobs in parallel on the same machine or on multiple machines you have ssh access to. It can often replace a for loop.

If you have 32 different jobs you want to run on 4 CPUs, a straight forward way to parallelize is to run 8 jobs on each CPU:

Simple scheduling

GNU Parallel instead spawns a new process when one finishes - keeping the CPUs active and thus saving time:

GNU Parallel scheduling

Installation

If GNU Parallel is not packaged for your distribution, you can do a personal installation, which does not require root access. It can be done in 10 seconds by doing this:

(wget -O - pi.dk/3 || curl pi.dk/3/ || fetch -o - http://pi.dk/3) | bash

For other installation options see http://git.savannah.gnu.org/cgit/parallel.git/tree/README

Learn more

See more examples: http://www.gnu.org/software/parallel/man.html

Watch the intro videos: https://www.youtube.com/playlist?list=PL284C9FF2488BC6D1

Walk through the tutorial: http://www.gnu.org/software/parallel/parallel_tutorial.html

Sign up for the email list to get support: https://lists.gnu.org/mailman/listinfo/parallel

0

I'd like to propose you start using configuration management system like, preferrably Ansible.

It's written on Python, really easy-to-use for everyone, for newbies also.

Follow the docs http://docs.ansible.com/ and get started!

0

Besides the suggested answer, I use the following command:

nohup ./scanner.py -flag1 flag1value -flag2 flag2value &

to create a new instance. To view all running scanner.py instance, I use:

ps aux | grep scanner.py

To kill a specific instance, I use:

kill -9 {PID}

where {PID} is obtained from the ps aux command, e.g. kill -9 1234

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.