I'm working on a computer with 10k cores, but only allowed to access 1,000 at a time.
I have a script that could benefit from GNU parallel in multiple places. Processing level A in parallel, and within that script, do something 30x.
It will take a lot of work to re-write the entire script to use the parallel --link
and ::: A B C ::: $(seq 30)
syntax.
Is there a way that two independent calls to parallel
can communicate enough that they limit the total number of jobs to 1000 between the two of them?