I need to run a lot of similar commands in a quickest possible amount time and using all available resources.
For example my case is processing images, when I'm using following command:
for INPUT in *.jpg do; some_command; done the command is executed one by one and not using all the available resources.
But on the other side executing
for INPUT in *.jpg do; some_command &; done makes the machine to run out of resources in a very short time.
I know about
batch command, but I'm not sure if I can use that in my case. Correct me if I am wrong.
So I was thinking about putting the commands in some kind of queue and executing just a part of them at once. I don't know how to do that in a quick way and that's the problem. I'm sure someone ran in a similar problem before.