I've seen Why does 'find -exec cmd {} +' need to end in '{} +'? which explains why {} +
has to be at the end of the -exec cmd
(*), and now I'd like to know why GNU xargs
reverts to one command per input argument when you use -I
(or the deprecated -i
)...or as the man page puts it
-I replace-str [...] Implies -x and -L 1.
For example:
$ seq 1 10 | xargs echo
1 2 3 4 5 6 7 8 9 10
$ seq 1 10 | xargs -I {} echo {}
1
2
3
4
5
6
7
8
9
10
The BUGS section of the xargs man page only says:
The -L option is incompatible with the -I option, but perhaps should not be.
and
When you use the -I option, each line read from the input is buffered internally. This means that there is an upper limit on the length of input line that xargs will accept when used with the -I option. To work around this limitation, you can use the -s option to increase the amount of buffer space that xargs uses, and you can also use an extra invocation of xargs to ensure that very long lines do not occur.
Neither of which explains why.
One of the main reasons to use xargs
is to reduce the number of commands executed (running one cp
or mv
with 1000 source filenames is better than running 1000 cp
s or mv
s with 1 source filename each) and this limitation prevents that in many cases....most cases since most unix commands want the source (e.g. a list of files) before the destination (e.g. a directory).
So, why does xargs
have this limitation?
Update 2015-11-05
I recently created a freebsd 10.0 VM and found that freebsd's version of xargs
has a -J
option to deal with this issue.
-J replstr
If this option is specified, xargs will use the data read from standard input to replace the first occurrence of replstr instead of appending that data after all other arguments. This option will not affect how many arguments will be read from input (-n), or the size of the command(s) xargs will generate (-s). The option just moves where those arguments will be placed in the command(s) that are executed. The replstr must show up as a distinct argument to xargs. It will not be recognized if, for instance, it is in the middle of a quoted string. Furthermore, only the first occurrence of the replstr will be replaced. For example, the following command will copy the list of files and directories which start with an uppercase letter in the current directory to destdir:
/bin/ls -1d [A-Z]* | xargs -J % cp -Rp % destdir
(*) Essentially, "because the POSIX spec says so". IMO they should have tried harder to find a solution that allowed {}
to appear anywhere in the -exec ... +
, or at least required standard tools like cp and mv to have an option like GNU's -t
option to reverse source and target.