I need to run a lot of similar commands in a quickest possible amount time and using all available resources.
For example my case is processing images, when I'm using following command:
for INPUT in *.jpg do; some_command; done
the command is executed one by one and not using all the available resources.
But on the other side executing for INPUT in *.jpg do; some_command &; done
makes the machine to run out of resources in a very short time.
I know about at
's batch
command, but I'm not sure if I can use that in my case. Correct me if I am wrong.
So I was thinking about putting the commands in some kind of queue and executing just a part of them at once. I don't know how to do that in a quick way and that's the problem. I'm sure someone ran in a similar problem before.
Please advise.
parallel
command - gnu.org/software/parallel – John Nov 27 '17 at 18:26