1

A line generated with command echo {,·}{,·}{,·}{,·}{,·}{,·}{,·}{,·}{,·}{,·}{,·}{,·}{,·} contains 8191 words, which is 114687 characters: 106497 dots and 8190 spaces.

Why does splitting it with | xargs -n 1 take computational ages? On my PC it's 8 seconds.

Background story.

Kidding around with bash brace expansion I've stumbled upon a fancy issue. I was checking time of bash brace expansion on the example problem: print strings of one, two and so on up to 'n' (=13) dots each in new line avoiding explicit loops and variables. I've come up with this somewhat slow solution:

$ time echo {,·}{,·}{,·}{,·}{,·}{,·}{,·}{,·}{,·}{,·}{,·}{,·}{,·} | xargs -n 1 | sort -u
·
··
···
····
·····
······
·······
········
·········
··········
···········
············
·············

real    0m8.800s
user    0m0.188s
sys     0m0.748s

I assumed it was caused by sort performance, so I've checked a solution without sorting:

$ time echo {,·}{,·}{,·}{,·}{,·}{,·}{,·}{,·}{,·}{,·}{,·}{,·}{,·} | xargs -n 1 | awk '{if (!a[$0]) {print $0} ; a[$0]=$0}'
·
··
···
····
·····
······
·······
········
·········
··········
···········
············
·············

real    0m8.250s
user    0m0.152s
sys     0m0.784s

Which was a bit faster. Expansion itself was surprisingly fast:

time echo {,·}{,·}{,·}{,·}{,·}{,·}{,·}{,·}{,·}{,·}{,·}{,·}{,·} > /dev/null

real    0m0.024s
user    0m0.020s
sys     0m0.004s

It was 0m0.250s when printing wall of dots to console, but that's the output which usually is slow compared to computation. So I checked how long does splitting to lines take:

time echo {,·}{,·}{,·}{,·}{,·}{,·}{,·}{,·}{,·}{,·}{,·}{,·}{,·} | xargs -n 1 > /dev/null 

real    0m8.551s
user    0m0.096s
sys     0m0.724s

What is xargs doing all that time?

  • xargs is running /bin/echo 2**13-1 times. Your pattern generates these 8191 combinations and then you ask xargs to echo them using a process per combination. It has nothing to do with the length of the line, just that you have 8000 things to process – icarus Dec 28 '16 at 17:30
6

xargs is slow, because it runs /bin/echo 8191 times.

Use | tr -s " " "\n" instead.

  • 2
    That's even clearly stated in its documentation, lol. This manual page documents the GNU version of xargs. xargs reads items from the standard input, delimited by blanks (which can be protected with double or single quotes or a backslash) or newlines, and executes the command (default is /bin/echo) one or more times with any initial-arguments followed by items read from standard input. Blank lines on the standard input are ignored. – Krzysztof Jabłoński Dec 28 '16 at 17:34
  • Just checked that time for x in {1..8191} ; do /bin/echo -n ; done runs for 12 seconds, while time for x in {1..8191} ; do echo -n ; done runs in no time. Again, process spawning has hit me. – Krzysztof Jabłoński Dec 29 '16 at 11:45
  • 1
    run type echo – Ipor Sircer Dec 29 '16 at 11:46
  • 1
    I did. echo is fast shell built-in, while /bin/echo is slow plain executable which runs as separate process. BTW: it seems that xargs uses vfork and execve to spawn processes, which is faster (8s) than my for loop (12s). Check this out: opensource.apple.com/source/shell_cmds/shell_cmds-116/xargs/… label exec:. – Krzysztof Jabłoński Dec 29 '16 at 17:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.