I'm running A file | B --params > file.txt. Since I want to accelerate the processing speed, I used the parallel -j+0< a.txt to run 20 jobs concurrently. a.txt contains all the commands:
A file1 | B --params > file1.txt
A file2 | B --params > file2.txt
A fileN | B --params > fileN.txt
Is this way safe? Will the stdout from different programs be messed when running in parallel?
Will the stdout from different programs be messed when running in parallel?
Not if they are all independent processes writing to separate files, which they appear to be -- each instance of B is distinct, and outputs to its own place.
As long as each instance of B writes to different files, you will be OK. However, the safety of this approach ultimately depends on B.
For example, I have been bitten by poorly-coded programs in the past, when attempting to do something similar to your solution. In my case, the program used srand(time(NULL)) and then wrote to a rand()-based temporary file. This broke horribly when 10 instances of the program were started at the same time, since they all wrote to the same temporary file.
As long as your program does not contain such a bug, and does not write to some fixed path, you should be OK.
The only situation where this is not safe is if A or B write to a resource that is hard coded in the program (e.g. /etc/passwd).
But for the vast majority of programs that is not the case. So it will be safe to run them in parallel.
