This:
#!/bin/bash
# Run command ~100Kbytes long
/bin/echo $(perl -e 'print "x"x100000') | wc
# Run command ~54Kbytes long
# This line fails: line 7: /bin/echo: Argument list too long
/bin/echo $(perl -e 'print "x "x27000') | wc
# Same command, but run using xargs
# Run command ~100Kbytes long
perl -e 'print "x"x100000' | xargs -n 100000 /bin/echo | wc
# Run command ~54Kbytes long
# This line fails: xargs: /bin/echo: Argument list too long
perl -e 'print "x "x27000' | xargs -n 100000 /bin/echo | wc
works fine on GNU/Linux, but the 2 54Kbytes lines fail on MacOS X.
ARG_MAX
is way higher than 100KBytes, and please note the 100Kbytes lines does not fail - it is the 54KBytes line that fails.
mac$ getconf ARG_MAX
262144
mac$ uname -a
Darwin macosx 11.4.2 Darwin Kernel Version 11.4.2: Thu Aug 23 16:26:45 PDT 2012; root:xnu-1699.32.7~1/RELEASE_I386 i386
# Kusalananda suggests it may be due to the size of the environment
mac$ env | wc
27 32 956
Why do the 54Kbytes command fail?
Is there a way I can predict if the argument list is too long for MacOS X without running it?
Research
This:
#!/bin/bash
runtest() {
echo environment size:
env | wc
echo Run command ~100Kbytes long
/bin/echo $(perl -e 'print "x"x100000') | wc
echo Run command ~54Kbytes long
# This line fails: line 7: /bin/echo: Argument list too long
/bin/echo $(perl -e 'print "x "x27000') | wc
# Same command, but run using xargs
echo Run command ~100Kbytes long
perl -e 'print "x"x100000' | xargs -n 100000 /bin/echo | wc
echo Run command ~54Kbytes long
# This line fails: xargs: /bin/echo: Argument list too long
perl -e 'print "x "x27000' | xargs -n 100000 /bin/echo | wc
echo
}
# Clean environment
runtest
# Make a huge environment
for a in `seq 5000`; do eval "a$a=1" ; done
for a in `seq 5000`; do eval "a$a() { 1; }" ; done
# This works as before
runtest
# Export environment
for a in `seq 5000`; do eval export a$a ; done
for a in `seq 5000`; do eval export -f a$a ; done
# Now the 100Kbytes commands fail, too
runtest
gives this output:
environment size:
6027 6032 47849
Run command ~100Kbytes long
1 1 100001
Run command ~54Kbytes long
test: line 10: /bin/echo: Argument list too long
0 0 0
Run command ~100Kbytes long
1 1 100001
Run command ~54Kbytes long
xargs: /bin/echo: Argument list too long
0 0 0
environment size:
6027 6032 47849
Run command ~100Kbytes long
1 1 100001
Run command ~54Kbytes long
test: line 10: /bin/echo: Argument list too long
0 0 0
Run command ~100Kbytes long
1 1 100001
Run command ~54Kbytes long
xargs: /bin/echo: Argument list too long
0 0 0
environment size:
16027 26032 126742
Run command ~100Kbytes long
test: line 7: /bin/echo: Argument list too long
0 0 0
Run command ~54Kbytes long
test: line 10: /bin/echo: Argument list too long
0 0 0
Run command ~100Kbytes long
xargs: insufficient space for argument
0 0 0
Run command ~54Kbytes long
xargs: /bin/echo: Argument list too long
0 0 0
So Kusalananda is right that the exported environment can have an effect. It is unclear what the formula is to compute this: Maybe it is purely the size? Maybe the number of variables is important, too? Maybe it is only the lengths of the names? Maybe a linear combinations of those?
It still does not explain that in a given environment a 100Kbytes command works fine, but a 54Kbytes command does not.
It is as if MacOS not only has a limit on the total size but also has a limit on the number of args.
The numbers would also make sense if MacOS uses an additional 8 bytes per argument:
# One big argument
100K * "x" = 100000+2 < 262144 # Works
# 27K small arguments
27K * "x " = 27K*(8+2) > 262144 # Fails
# 26K small arguments
26K * "x " = 26K*(8+2) < 262144 # Works
But does MacOS do that?
ARG_MAX
size.xx
in place ofx
(followed by a space character) makes it work as it produces a single large argument rather than 27k small ones.