6

The /usr/bin/printf util argument list length is limited to the shell's maximum command line length, (i.e. getconf ARG_MAX, on my system that'd be 2097152); example:

# try using a list that's way too long
/usr/bin/printf '%s\n' $(seq $(( $(getconf ARG_MAX) * 2 ))) | tail -1

Output:

bash: /usr/bin/printf: Argument list too long

Today I'm informed that shell builtin printfs don't have that limit; test:

printf '%s\n' $(seq $(( $(getconf ARG_MAX) * 2 ))) | tail -1

Output:

4194304

Questions:

  1. A skim of man bash dash doesn't seem to say much about this advantage of builtin printf. Where is it documented?

  2. Do builtin printfs (e.g. bash) have an argument list maximum length in chars, and if so, what is that length?

  • nb: taking advantage of the lack of hard fixed puny limits doesn't mean that loading unpredictably long files and lines whole in the memory (as most answers to the linked question do) is either robust, scalable or nice -- there probably should be a law against it ;-) – mosvy May 19 at 20:35
14

It's not really the manual's job to advocate the use of any particular utility. It should primarily describe the available built-in utilities.

The advantages of using a built-in utility over an external one is primarily speed and the availability of extended features (printf in bash can, for example, write directly into a variable with -v varname, which no external printf could ever do).

Executing external utilities is slow in comparison to executing a built-in utility, especially if done often in e.g. a loop, and, as you have noticed, they also allow for longer argument lists (this is not something that only the built-in printf allows, but all built-in utilities).

The length of the argument list to the built-in printf utility in bash is limited by the resource restrictions on the bash process itself. On some systems, this may even mean that you could use most of the available RAM to construct its command line argument list.

The documentation from where you would find these various bits of information is

  • The bash source code, where you will see that the argument list for printf is a dynamically allocated linked list, and also that it's not using execve() to run printf (which is what limits the length of an argument list when running an external utility).

An example of a shell where printf in not a built-in utility is the ksh shell of OpenBSD. The utility can also be disabled in bash using enable -n printf.

  • Surprising that ksh doesn't have printf built-in. – Sergiy Kolodyazhnyy May 19 at 18:21
  • 2
    @SergiyKolodyazhnyy, it's builtin in ksh93. It's not in ksh88, pdksh and some of its derivatives like OpenBSD ksh or some builds of mksh. It's pretty annoying when it's not especially in ksh where echo is not generally not usable reliably. – Stéphane Chazelas May 19 at 20:51
2

The /usr/bin/printf util argument list length is limited to the shell's maximum command line length, (i.e. getconf ARG_MAX, on my system that'd be 2097152); example:

That's not a shell's limit, but a limit of the OS (of the Linux kernel), specifically of its execve(2) syscall, and caused by the antiquated way in which the command line arguments and environment variables are passed to a started program.

(Notice that that limit also includes the environment variables!).

A skim of man bash dash doesn't seem to say much about this advantage of builtin printf. Where is it documented? Do builtin printfs have an argument list length, (e.g. bash), and if so, what is it?

Since shell built-ins don't go through execve(2) they don't have to have any limit like that. Modern shell usually don't use fixed size buffers and such, so the limit is usually imposed by the amount of available memory and the layout of the virtual address space -- ie it's unlimited for all intents and purposes.

  • Does "unlimited" here mean something like RAM + swap, (so that an older system with 8M RAM and 16M swap would limit the builtin printf command line char length to 24M), or something else? – agc May 19 at 21:15
  • 2
    "unlimited" means that your program will break or crash before hitting that limit. But if your machine+OS is using virtual memory, your process can use strings or any data structures larger than both the physical memory and swap. A large file could be mmap'ed in the address space of a process, and appear just as a large chunk of memory, but its data doesn't have to be all paged in at the same time. – mosvy May 19 at 21:25
0

The POSIX standard requires the shell to be implemented without a line limit.

So this is a naturalness that shells do not document.

BTW: because of this fact, a shell script may not be a text file, as a text file is a file that does not have lines longer than LINE_MAX ;-)

  • Could you say more about how this fact relates to the execution of the printf utility (external or built-in) and the success thereof? This seems to be relating more to the input to the shell itself (i.e. the nature of a script). – Kusalananda May 20 at 14:39
  • ARG_MAX is a limit associated with exec(), which is used when calling external commands. Since running builtin commands does not call exec(), the shell needs to be able to process arbitrary sized argument lists. – schily May 20 at 14:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.