GNU Parallel can generate all combinations of input, copy shell functions and run them on 3 remote servers without mixing the output, retrying the command if it fails:
#!/bin/bash
env_parallel --session
process_output() {
perl -ne '$/=undef; $|=1; @a=split//;
while(@a){
print ".",shift @a;
select($a,$a,$a,0.100);
}';
}
env_parallel --retries 4 -Sserver{1..3},: \
'echo X={1} Y={2} Z={3} | process_output' \
::: "a' \"b"{1..3} ::: "c\" 'd"{3..4} ::: "e\$ f'"{7..8}
Output:
.X.=.a.'. . .".b.1. .Y.=.c.". . .'.d.3. .Z.=.e.$. . .f.'.7.
.X.=.a.'. . .".b.1. .Y.=.c.". . .'.d.3. .Z.=.e.$. . .f.'.8.
.X.=.a.'. . .".b.1. .Y.=.c.". . .'.d.4. .Z.=.e.$. . .f.'.7.
.X.=.a.'. . .".b.1. .Y.=.c.". . .'.d.4. .Z.=.e.$. . .f.'.8.
.X.=.a.'. . .".b.2. .Y.=.c.". . .'.d.3. .Z.=.e.$. . .f.'.7.
.X.=.a.'. . .".b.2. .Y.=.c.". . .'.d.3. .Z.=.e.$. . .f.'.8.
.X.=.a.'. . .".b.2. .Y.=.c.". . .'.d.4. .Z.=.e.$. . .f.'.7.
.X.=.a.'. . .".b.2. .Y.=.c.". . .'.d.4. .Z.=.e.$. . .f.'.8.
.X.=.a.'. . .".b.3. .Y.=.c.". . .'.d.3. .Z.=.e.$. . .f.'.7.
.X.=.a.'. . .".b.3. .Y.=.c.". . .'.d.3. .Z.=.e.$. . .f.'.8.
.X.=.a.'. . .".b.3. .Y.=.c.". . .'.d.4. .Z.=.e.$. . .f.'.7.
.X.=.a.'. . .".b.3. .Y.=.c.". . .'.d.4. .Z.=.e.$. . .f.'.8.
xargs -P$(nproc) can run one job per processor.
Given
- the function
process_output, - the servers to run on (server1, server2, server3),
- the argument groups (
a' "b1, a' "b2, a' "b3;c" 'd1, c" 'd2, c" 'd3;e$ f'7, e$ f'8)
how can you emulate GNU Parallel using xargs to get the same done using process_output and the servers?
xargs -P<N>OR it should also fulfill all conditions like distributingprocess_outputcalls on 3 remote servers?