xargs: run a command for each line and separate arguments with whitespace?

Question

I can reproduce this issue on cygwin bash and oh-my-zsh with wsl.exe. My intent is to run these two lines on the command line:

npm rm plugin-alias
npm i -D plugin-alias

Here is how I'm trying to do that:

echo -ne "rm\0i -D\0" | xargs -t -0 -IREPLACE npm REPLACE plugin-alias

The rm runs fine. -t says it is running as npm rm plugin-alias. But i -D errors:

npm 'i -D' plugin-alias
Unknown command: "i -D"

Is there a way to get xargs to pass i -D as multiple arguments so it runs npm i -D plugin-alias instead?

I've tried changing the delimiters to , and \n, but it doesn't seem to make a difference.

If I have to involve sed to remove the quotes, so be it, but I'm hoping xargs can solve the problem it appears to have introduced.

---

UPDATE: @muru provided a helpful comment:

@muru: there're no actual quotes involved

me: then why do I get the error message, Unknown command: "i -D"?

@muru: Because xargs is providing i -D as a single argument to npm.

Now that I understand, I can edit my title to be more accurate. For better searchability, I want to say the original title was, "Why does xargs quote items with spaces and how do you prevent it?"

Answer 1

To answer the:

xargs: run a command for each line and separate arguments with whitespace?

in the subject, that's exactly what the -L¹ option is for.

xargs -L1 cmd << 'EOF'
1 2 3
first 'second arg'
"a 1" b\
2
EOF

Would run cmd for each 1 Line of the input (logical line, you'll see the third line is made of two physical lines as we quoted the newline character embedded in the second argument and a trailing blanks allows breaking long lines), splitting each line into arguments on unquoted blanks.

It cannot be combined with -I nor GNU-style -0 though.

As it understands some form of quoting (beware that's with a different syntax from that of any modern shell though), it allows us to specify arbitrary arguments (with limitations depending on the implementation²) so we don't need -0.

With BSD xargs, it can be combined with -J though so you can do:

xargs -rL1 -J {} npm {} plugin-alias << 'EOF'
rm
i -D
EOF

With other implementations, you'd use sh to reorder the arguments:

xargs -L1 sh -c '[ "$#" -eq 0 ] || exec npm "$@" plugin-alias' sh << 'EOF'
rm
i -D
EOF

(here also using [ "$#" -eq 0 ] || as a substitute for the non-standard -r).

---

^{¹ GNU xargs at least has -l as a deprecated and non-standard alias for -L 1. -l can also take a number of lines like -L, but it has to be affixed to the option (-l1, not -l 1; while you can use either -L1 or -L 1).}

^{² for instance, some choke on non-text input, some have ridiculously low limits on the lengths of a command line or single argument, some understand _ as end-of-file... Also beware the list of characters recognised as delimiters vary with the implementation and the locale with some.}

Answer 2

Not that it's usually recommended to be used, you could apply the shell's word splitting for this:

printf "rm\0i -D\0" | 
   xargs -t -0 -I{} sh -f -c 'npm $1 plugin-alias' sh {}  

Here, I'm passing each NUL-delimited string as a single argument to the sh script, and the unquoted expansion of $1 causes them to be split on whitespace. The -f option (for set -f) disables globbing.

Using a script to see the distinct command line arguments clearly:

% printf "rm\0i -D\0" | 
   xargs -t -0 -I{} sh -f -c './args.sh npm $1 plugin-alias' sh {}
sh -f -c ./args.sh npm $1 plugin-alias sh rm
3 args: <npm> <rm> <plugin-alias> 
sh -f -c ./args.sh npm $1 plugin-alias sh i -D
4 args: <npm> <i> <-D> <plugin-alias> 

Alternatively, I think the -L could do what you want, if you change the NUL-separators to newlines. But it seems you still need a shell to get the args in the correct order, as -I seems to disable the splitting on whitespace:

% printf 'rm\ni -D\n' |xargs -L1 sh -c './args.sh npm "$@" plugin-alias' sh  
3 args: <npm> <rm> <plugin-alias> 
4 args: <npm> <i> <-D> <plugin-alias> 

Anyway, I'm not sure if there are differences here between the versions of xargs, but you could do the same with just a shell:

$ printf 'rm\ni -D\n' | while read -r line ; do set -f; ./args.sh npm $line plugin-alias; done
3 args: <npm> <rm> <plugin-alias> 
3 args: <npm> <i -D> <plugin-alias> 

---

args.sh here is this script:

#!/bin/sh
if [ "$#" != 0 ]; then
    printf "%d args: " "$#"
    printf "<%s> " "$@"
    echo
else
    printf "no args.\n"
fi

Answer 3

Here, maybe rather than using xargs, you could get your shell to read that list. With zsh:

while IFS=, read -rd '' -u3 -A args; do
  npm "$args[@]" plugin-alias
done 3< <(printf '%s\0' rm i,-D)

Here separating the arguments with , to make it easier to specify empty arguments.

Here it's a list of lists you want to represent. A single list of arbitrary arguments to an external command can be represented by delimiting it with NUL characters as command arguments have that limitation that they can't contain NUL characters (arguments to bash builtins and functions have that same limitation, not those of zsh), but to represent a list of lists of arbitrary elements you can't rely on a single delimiter, you'd need to use some form of encoding.

One option would be to use read's default behaviour without -r. For instance, if you drop the -r option above, you can represent 2 lists, the first with the empty string and foo,bar, the second with newline and backslash with:

printf '%s\0' ',foo\,bar' '
,\\'

You need to remember to prefix both , and \ with \ though.

Note that in zsh, an empty record is split into one empty element. In bash or ksh, into no element at all while foo, is split into 2 records: foo and an empty element. Also note that for bash, you need -a instead of -A.