2

I have bash script which creates high energy collision events, if one event takes around 2 minutes then for 100000 events it takes 200000 minutes, which is too much. so if I have 50 cores in a node, I want to have one event on each core, so in this way there will be 50 events generated in 2 minutes, so time will be saved. I thought this can be done by multi-threading, or any other way by which this can be done?, so can anyone help me?

  • 2
    The bash script most likely does not create high energy collision events but starts programs which analyze/simulate such events. In any case, the script can start multiple processes in parallel and wait for their completion before starting new processes. However, a full blown CPU load scheduling program might be better suited to your problem than poor old bash. – Hans-Martin Mosner Jun 28 '16 at 10:19
  • create a kubernetes cluster on the node and divide the events to be processed on the number of containers in the cluster. – Ijaz Ahmad Khan Jun 28 '16 at 11:14
2

GNU Parallel is made for this kind of tasks.

seq 100000 | parallel do_experiment

If your experiment takes different kind of values (say, models), then you can run all experiments for all models by:

seq 100000 | parallel do_experiment --iteration {1} --model {2} :::: - ::: model1 model2

It will default to 1 process per CPU core and it will make sure the output of two parallel jobs will not be mixed. If you have multiple computers sitting unused in your lab, you can make them take part in the calculation, too.

GNU Parallel is a general parallelizer and makes is easy to run jobs in parallel on the same machine or on multiple machines you have ssh access to. It can often replace a for loop.

If you have 32 different jobs you want to run on 4 CPUs, a straight forward way to parallelize is to run 8 jobs on each CPU:

Simple scheduling

GNU Parallel instead spawns a new process when one finishes - keeping the CPUs active and thus saving time:

GNU Parallel scheduling

Installation

If GNU Parallel is not packaged for your distribution, you can do a personal installation, which does not require root access. It can be done in 10 seconds by doing this:

(wget -O - pi.dk/3 || curl pi.dk/3/ || fetch -o - http://pi.dk/3) | bash

For other installation options see http://git.savannah.gnu.org/cgit/parallel.git/tree/README

Learn more

See more examples: http://www.gnu.org/software/parallel/man.html

Watch the intro videos: https://www.youtube.com/playlist?list=PL284C9FF2488BC6D1

Walk through the tutorial: http://www.gnu.org/software/parallel/parallel_tutorial.html

Sign up for the email list to get support: https://lists.gnu.org/mailman/listinfo/parallel

0

You can start the jobs as background processes untill you reach the limit of 50 jobs in background, then wait for a job to finish before starting another one.

LIMIT=50
while collision_event_to_run
do 
    run_new_collision_event&

    while (( $(jobs | wc -l) >= LIMIT ))
    do
                sleep 1
    done

done

or better to run new events while the load average is low.

LIMIT=49
while collision_event_to_run
do 
    run_new_collision_event&
    sleep 2   # let time for the load average calculation

    while (( $(cut -d " " -f1 < /proc/loadavg) >= LIMIT ))
    do
                sleep 1
    done

done

Another option is to use the batch command to stack the jobs in the atd batch queue.

batch collision _event_1
batch collision _event_2
batch collision _event_3
...

atd service will starts the job in parallel until the server load reach a limit. That limit is a parameter of atd, it must be set in the atd service starting script, atd -l 50 for instance.

Edit:

  • To keep a little free cpu for the operating system atd -l 49.

  • atd will start a new job every minute, which is too slow, to permit the server to reach the load as jobs last 2mn. You can reduce the delay with the -b parameter. atd -b 2 -l 49 to start a job every 2 seconds will permit to reach the limit before the first jobs finishes.

  • If you use this then we get the full performance/utilization? – Nullpointer Jun 28 '16 at 13:21
  • @RaviG for full performance batch command would be better as it watches the server load, which is not the same as the number of job. If the jobs are not using 100% of a cpu, batch may run more processes than 50. Anyway that must be tuned, running processes in parallel can create other bottleneck than cpu. – Emmanuel Jun 28 '16 at 13:27
  • If script take a 100min and create 4 thread for it then script will done within 25min? – Nullpointer Jun 28 '16 at 13:59
  • @Ravi Yes it should. Beware that when a script is parallelized, its component must not access the same resources at the same time, like writing to the same file. Better made a test on a test environment first. – Emmanuel Jun 28 '16 at 14:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.