20

I am trying to understand how exactly Bash treats the following line:

$(< "$FILE")

According to the Bash man page, this is equivalent to:

$(cat "$FILE")

and I can follow the line of reasoning for this second line. Bash performs variable expansion on $FILE, enters command substitution, passes the value of $FILE to cat, cat outputs the contents of $FILE to standard output, command substitution finishes by replacing the entire line with the standard output resulting from the command inside, and Bash attempts to execute it like a simple command.

However, for the first line I mentioned above, I understand it as: Bash performs variable substitution on $FILE, Bash opens $FILE for reading on standard input, somehow standard input is copied to standard output, command substitution finishes, and Bash attempts to execute the resulting standard output.

Can someone please explain to me how the contents of $FILE goes from stdin to stdout?

4 Answers 4

24

$(<file) (also works with `<file` as well as ${<file;} in ksh93) is a special operator of the Korn shell copied by zsh and bash. It does look a lot like command substitution but it's not really.

In POSIX shells, a simple command is:

< file var1=value1 > file2 cmd 2> file3 args 3> file4

All parts are optional, you can have redirections only, command only, assignment only or combinations.

If there are redirections but no command, the redirections are performed (so a > file would open and truncate file), but then nothing happens. So

< file

Opens file for reading, but then nothing happens as there's no command. So the file is then closed and that's it. If $(< file) was a simple command substitution, then it would expand to nothing.

In the POSIX specification, in $(script), if script consists only of redirections, that produces unspecified results. That's to allow that special behaviour of the Korn shell.

In ksh (here tested with ksh93u+), if the script consists of one and only one simple command (though comments are allowed before and after) that consists only of redirections (no command, no assignment) and if the first redirection is a stdin (fd 0) input only (<, << or <<<) redirection, so:

  • $(< file)
  • $(0< file)
  • $(<&3) (also $(0>&3) actually as that's in effect the same operator)
  • $(< file > foo 2> $(whatever))

but not:

  • $(> foo < file)
  • nor $(0<> file)
  • nor $(< file; sleep 1)
  • nor $(< file; < file2)

then

  • all but the first redirection are ignored (they are parsed away)
  • and it expands to the content of the file/heredoc/herestring (or whatever can be read from the file descriptor if using things like <&3) minus the trailing newline characters.

as if using $(cat < file) except that

  • the reading is done internally by the shell and not by cat
  • no pipe nor extra process is involved
  • as a consequence of the above, since the code inside is not run in a subshell, any modification remain thereafter (as in $(<${file=foo.txt}) or $(<file$((++n))))
  • read errors (though not errors while opening files or duplicating file descriptors) are silently ignored.

In zsh, it's the same except that that special behaviour is only triggered when there's only one file input redirection (<file or 0< file, no <&3, <<<here, < a < b...)

However, except when emulating other shells, in < file that is when there's only one input redirection without commands, outside of command substitution, zsh runs the $READNULLCMD (a pager by default), and when there are more redirections or redirections other than < file (<&3, <<<text, <a <b, >file, <a >b ...) the $NULLCMD (cat by default), so even if $(<&3) is not recognized as that special operator, it will still work like in ksh by invoking cat to do it.

However while ksh's $(< a < b) would expand to the contents of a, in zsh, it expands to the content of a and b (or just b if the multios option is disabled), $(< a > b) would copy a to b and expand to nothing, etc.

bash has a similar operator but with a few differences:

  • comments are allowed before but not after:

    echo "$(
       # getting the content of file
       < file)"
    

    works but:

    echo "$(< file
       # getting the content of file
    )"
    

    expands to nothing.

  • like in zsh, only one file stdin redirection, though there's no fall back to a $READNULLCMD, so $(<&3), $(< a < b) do perform the redirections but expand to nothing.

  • for some reason, while bash does not invoke cat, it still forks a process that feeds the content of the file through a pipe making it much less of an optimisation than in other shells. It's in effect like a $(cat < file) where cat would be a builtin cat.

  • as a consequence of the above, any change made within are lost afterwards (in the $(<${file=foo.txt}), mentioned above for instance, that $file assignment is lost afterwards).

In bash, IFS= read -rd '' var < file (also works in zsh) is a more effective way to read the content of a text file into a variable. It also has the benefit of preserving the trailing newline characters. See also $mapfile[file] in zsh (in the zsh/mapfile module and only for regular files) which also works with binary files.

Note that the pdksh-based variants of ksh have a few variations compared to ksh93. Of interest, in mksh (one of those pdksh-derived shells), in

var=$(<<'EOF'
That's multi-line
test with *all* sorts of "special"
characters
EOF
)

is optimised in that the content of the here document (without the trailing newline characters) is expanded without a temporary file or pipe being used as is otherwise the case for here documents, which makes it an effective multi-line quoting syntax.

To be portable to all versions of ksh, zsh and bash, best is to limit to only $(<file) avoiding comments and bearing in mind that modifications to variables made within may or may not be preserved.

8
  • Is it correct that $(<) is an operator on filenames? Is < in $(<) a redirection operator, or not an operator on its own, and must be part of the entire operator $(<)?
    – Tim
    Aug 23, 2017 at 21:32
  • 1
    @Tim, it doesn't matter much how you want to call them. $(<file) is meant to expand to the content of file in a similar way as $(cat < file) would. How it's done varies from shell to shell which is described at length in the answer. If you like, you can say that it's a special operator that is triggered when what looks like a command substitution (syntactically) contains what looks like a single stdin redirection (syntactically), but again with caveats and variations depending on the shell as listed here. Aug 24, 2017 at 8:21
  • @StéphaneChazelas: Fascinating, as usual; I’ve bookmarked this.  So, n<&m and n>&m do the same thing? I didn’t know that, but I guess it’s not too surprising. Sep 3, 2017 at 5:45
  • @Scott, yes, they both do a dup(m, n). I can see some evidence of ksh86 using stdio and some fdopen(fd, "r" or "w"), so it might have mattered then. But using stdio in a shell makes little sense, so I don't expect you'll find any modern shell where that will make a difference. One difference is that >&n is dup(n, 1) (short for 1>&n), while <&n is dup(n, 0) (short for 0<&n). Sep 3, 2017 at 9:08
  • Right. Except, of course, the two-argument form of the file descriptor duplication call is called dup2(); dup() takes only one argument and, like open(), uses the lowest available file descriptor. (Today I learned that there is a dup3() function.) Sep 3, 2017 at 15:52
9

Because bash does it internally for you, expanded the filename and cats the file to standard output, like if you were to do $(cat < filename). It's a bash feature, maybe you need to look into the bash source code to know exactly how it works.

Here the the function to handle this feature (From bash source code, file builtins/evalstring.c):

/* Handle a $( < file ) command substitution.  This expands the filename,
   returning errors as appropriate, then just cats the file to the standard
   output. */
static int
cat_file (r)
     REDIRECT *r;
{
  char *fn;
  int fd, rval;

  if (r->instruction != r_input_direction)
    return -1;

  /* Get the filename. */
  if (posixly_correct && !interactive_shell)
    disallow_filename_globbing++;
  fn = redirection_expand (r->redirectee.filename);
  if (posixly_correct && !interactive_shell)
    disallow_filename_globbing--;

  if (fn == 0)
    {
      redirection_error (r, AMBIGUOUS_REDIRECT);
      return -1;
    }

  fd = open(fn, O_RDONLY);
  if (fd < 0)
    {
      file_error (fn);
      free (fn);
      return -1;
    }

  rval = zcatfd (fd, 1, fn);

  free (fn);
  close (fd);

  return (rval);
}

A note that $(<filename) is not exactly equivalent to $(cat filename); the latter will fail if the filename starts with a dash -.

$(<filename) was originally from ksh, and was added to bash from Bash-2.02.

1
  • 1
    cat filename will fail if the filename starts with a dash because cat accepts options. You can work around that on most modern systems with cat -- filename.
    – Adam Katz
    Jul 6, 2017 at 1:20
0

Here's a bash 3.2 snippet that shows the difference, explained:

  • Use strace to follow processes and show execve calls strace -f -e trace=execve
  • Run bash reading commands from string bash -c - with and without /bin/cat
  • Diff the output in side by side mode in 80 columns to fit here diff -y -W 80

You can see the extra execve(/bin/cat...) on the right side of the diff:

$ echo $BASH_VERSION
3.2.25(1)-release
$ echo "hi" >/tmp/f
$ strace -f -e trace=execve /bin/bash -c 'echo $(</tmp/f)'          >/tmp/no_cat 2>&1
$ strace -f -e trace=execve /bin/bash -c 'echo $(/bin/cat </tmp/f)' >/tmp/wi_cat 2>&1
$ diff -y -W 80 /tmp/no_cat /tmp/wi_cat
execve("/bin/bash", ["/bin/bash", "-c | execve("/bin/bash", ["/bin/bash", "-c
Process 24253 attached (waiting for p | Process 24256 attached (waiting for p
Process 24253 resumed (parent 24252 r | Process 24256 resumed (parent 24255 r
Process 24253 detached                | Process 24257 attached (waiting for p
                                      > Process 24257 resumed (parent 24256 r
                                      > Process 24256 suspended
                                      > [pid 24257] execve("/bin/cat", ["/bin
                                      > Process 24256 resumed
                                      > Process 24257 detached
                                      > [pid 24256] --- SIGCHLD (Child exited
                                      > Process 24256 detached
--- SIGCHLD (Child exited) @ 0 (0) --   --- SIGCHLD (Child exited) @ 0 (0) --
hi                                      hi
-4

The < isn't directly an aspect of bash command substitution. It is a redirection operator (like a pipe), which some shells allow without a command (POSIX does not specify this behavior).

Perhaps it would be more clear with more spaces:

echo $( < $FILE )

this is effectively* the same as the more POSIX-safe

echo $( cat $FILE )

... which is also effectively*

echo $( cat < $FILE )

Let's start with that last version. This runs cat with no arguments, which means it will read from standard input. $FILE is redirected into standard input due to the <, so cat puts its contents are put into standard output. The $(command) subsitution then pushes cat's output into arguments for echo.

In bash (but not in the POSIX standard), you can use < without a command. bash (and zsh and ksh but not dash) will interpret that as if cat <, though without invoking a new subprocess. As this is native to the shell, it is faster than literally running the external command cat. *This is why I say "effectively the same as."

18
  • 1
    So in the last paragraph when you say "bash will interpret that as cat filename", do you mean this behavior is specific to command substitution? Because if I run < filename by itself, bash does not cat it out. It will output nothing and return me back to a prompt.
    – Stanley Yu
    Mar 12, 2015 at 20:06
  • 1
    @StanleyYu, $(< file) is faster because it doesn't spawn a new process (cat). < is a builtin.
    – Adam Katz
    Mar 12, 2015 at 20:18
  • 1
    A pipe is a type of file. The shell operator | creates a pipe between two subprocesses (or, with some shells, from a subprocess to the shell's standard input). The shell operator $(…) creates a pipe from a subprocess to the shell itself (not to its standard input). The shell operator < does not involve a pipe, it only opens a file and moves the file descriptor to standard input. Mar 12, 2015 at 23:15
  • 7
    < file is not the same as cat < file (except in zsh where it's like $READNULLCMD < file). < file is perfectly POSIX and just opens file for reading and then does nothing (so file is close straight away). It's $(< file) or `< file` that is a special operator of ksh, zsh and bash (and the behaviour is left unspecified in POSIX). See my answer for details. Aug 23, 2017 at 15:41
  • 3
    To put @StéphaneChazelas’s comment in another light: to a first approximation, $(cmd1) $(cmd2) will typically be the same as $(cmd1; cmd2).   But look at the case where cmd2 is < file.   If we say $(cmd1; < file), the file is not read, but, with $(cmd1) $(< file), it is.   So it is incorrect to say that $(< file) is just an ordinary case of $(command) with a command of < file.   $(< …) is a special case of command substitution, and not a normal usage of redirection. Sep 3, 2017 at 5:44

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.