I'm trying to somehow use xargs so a program whose outputs are the lines


will cause the execution of the following command:

    ...               \
    <PREFIX1> <PREFIX2> ... ARG1 <SUFFIX1> <SUFFIX2> ... \
    <PREFIX1> <PREFIX2> ... ARG2 <SUFFIX1> <SUFFIX2> ... \
                            ...                          \
    <PREFIX1> <PREFIX2> ... ARGN <SUFFIX1> <SUFFIX2> ... \

where the prefixes and suffixes are known beforehand, but arbitrary.

Some (probably) obvious notes that I feel like I have to explicitly state anyway:

  • The parameters are not "nice". They lack newlines but might contain spaces, $, (, ), ", etc.

  • I would like to avoid converting them to a single giant string and back if at all possible.

  • Yes, I meant what I wrote. I really just only want 1 command to run: not 0, not 2, not 3, not N...

  • More generally: I don't want to spawn a new process for each argument.

  • Simpler tools are preferred, assuming correctness is given... so xargs is preferred over sed, which is preferred over awk, etc.

  • I'm trying to use standard *nix shell tools here.
    Obviously I can write a Python script, but that's not the point...


Forget xargs, just use a while loop. This assumes that printf is builtin

printf "%s\n" "command"
printf "    %s\\\n" "<GLOBAL_PREFIX_1>" "<GLOBAL_PREFIX_2>" "..."
while read -r ; do
    printf "    <PREFIX1> <PREFIX2> ... %s <SUFFIX1> <SUFFIX2> ... \\\n" "$REPLY"
printf "    %s\\\n" "<GLOBAL_SUFFIX_1>" "<GLOBAL_SUFFIX_2>"
printf "    %s\n" "..."

If the PREFIXn or SUFFIXn have interesting characters in them like % or \ then these will need to be escaped.

Of course sed would work

sed '1i\
s/.*/P1 P2 & S1 S2\\/
... '

For awk you have BEGIN and END clauses....

  • The first one spawns a new process per argument I believe, right? (Or at least it does on my system.) The sed one seems like it doesn't though? – user541686 Nov 28 '16 at 7:18
  • Use a shell which has printf as a builtin, like bash, ksh, zsh. – icarus Nov 28 '16 at 7:31
  • Oh, actually, I assumed it wasn't a builtin because I saw /usr/bin/printf... but it seems it's listed under help (for BASH) too, never mind. Thanks! +1 – user541686 Nov 28 '16 at 7:39

If the arg prefix and arg postfix does not contain space, then GNU Parallel's -X is made for you:

seq 1000 | parallel -X echo global prefix argpre{}argpost global postfix

It will distribute evenly to the number of core. If you do not like that:

seq 1000 | parallel -j1 -n10 -X echo global prefix argpre{}argpost global postfix

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.