3
votes
3answers
257 views

Bash, how to let some background processes run but wait for others?

I have (yet) another wait, &, && control flow question.. Say I have a script something like this where I want to do as much work at the same time as possible: # may take some hours ...
6
votes
4answers
591 views

How to run parallel processes and combine outputs when both finished

I have a bash shell script in which I pipe some data through about 5 or 6 different programs then the final results into a tab delimited file. I then do the same again for a separate similar dataset ...
8
votes
1answer
455 views

GNU parallel vs & (I mean background) vs xargs -P

I'm confused about the difference or advantage (if any) of running a set of tasks in a .sh script using GNU parallel E.g. Ole Tange's answer: parallel ./pngout -s0 {} R{} ::: *.png rather than say ...
1
vote
4answers
120 views

How can i run a piece of code in background?

Can I run a piece of code in background instead of using another script. [sesiv@itseelm-lx4151 ~]$ cat ./testback2 #!/bin/bash start_time=$(date +%s) for i in {1..5} do ./testscript & done wait ...
6
votes
4answers
4k views

Parallel execution of a program on multiple files

I have a small script that loops through all files of a folder and executes a (usually long lasting) command. Basically it's for file in ./folder/*; do ./bin/myProgram $file > ./done/$file ...
2
votes
4answers
2k views

Parallelizing a for loop with very large number of iterations

I want to parallelize a for loop where the number of iterations in the loop can be very large such as 10^6. So ,it will be better if I can create threads rather than process. How to do it? The code is ...
9
votes
2answers
3k views

Parallelizing a for loop

I want to parallelize the for loops of the following code. How to do this? #!/bin/bash N=$1 n=$2 for (( i=1; i<=$N; i++ )); do min=100000000000000 //set min to some garbage value for (( ...