0

The man page for grep describes the -d ACTION option as follows:

If an input file is a directory, use ACTION to process it. By default, ACTION is read, i.e., read directories just as if they were ordinary files. [...]

Intuitively, I would expect this to mean that a directory bar is treated (for grepping purposes) as the equivalent of a text file containing something more or less along the lines of what vim displays if I type vim foo, i.e., something roughly (up to variation is what sort of explanatory information and/or metadata is at the top and bottom) like:

"============================================================================
" Netrw Directory Listing                                        (netrw v156)
"   /home/chris-henry/bar
"   Sorted by      name
"   Sort sequence: [\/]$,\<core\%(\.\d\+\)\=\>,\.h$,\.c$,\.cpp$,\~\=\*$,*,\.o$,\.obj$,\.info$,\.swp$,\.bak$,\~$
"   Quick Help: <F1>:help  -:go up dir  D:delete  R:rename  s:sort-by  x:special
" ==============================================================================
../
./
foobar/
baz/
qux

If this were the case, then grep -H foo bar would produce the output

bar: foobar/

Instead, it gives the message grep: bar: Is a directory. Why is this? And is there any (reasonably straightforward) way to get the intuitive result (not just on this simple search, but also for searches like grep foo * where * may match any or all of text files, binary files, and directories)?

ETA (2021-07-22): As suggested by the accepted answer and confirmed in the comments, grep foo bar itself actually does exactly what I'd expect it to do: It invokes the system call read (ssize_t read(int fd, void *buf, size_t count)) with the file descriptor for bar, just as it would if bar were an ordinary file. And when read, instead of filling *buf with the contents of bar, returns the error code EISDIR, grep prints an appropriate diagnostic message, then continues on to the next file - just as it would if read returned an error code (other than EINTR or, sometimes, EINVAL) and bar were an ordinary file.

The difference between my expectation and reality comes from the behavior of the Linux version (and, judging by comments, most other modern versions) of read, namely that when fd refers to a directory, it automatically returns EISDIR.

ETA2 (2021-07-23): The primary motivation for this question was not a pressing need to get the intuitive behavior described (although I was interested in that as a potential secondary benefit). The motivation was to understand why (GNU) grep seemed, based on its output, to be behaving in a manner that contradicted a statement in its man page.

The answer turned out to be that grep was actually doing just what its man page said it would, but that changes to the (typical) behavior of the system call read make the result of that, on most modern systems, substantially different from what one would infer based solely on a reading of the grep man page (without being familiar with the behavior of modern read implementations.

While it's true that I would rather, on the whole, that read didn't behave like that, I rather doubt that that behavior contradicts its man page. Given the current situation, I would like to see a line or two added to the grep man page, but it's not wrong as it is, just misleading.

0
5

Directories do not have an intrinsic representation as text. Many Unix variants allow programs to read from a directory as if it was a regular file, but this is mostly useless since the format of the content depends on the filesystem. Some modern Unix variants, including Linux, outright blocks programs from reading a directory as if it was a regular file.

For example, here's what happens on FreeBSD (an older version that still allows it — since FreeBSD 13, this is disabled by default) with a directory like your bar:

$ grep -H foo bar
Binary file bar matches
$ grep -H --text foo bar
bar:�"!
       .�
..�"!foobar�"!
              baz�"!qux

Yes, you can determine that foo is present in the directory representation, but you can't be sure that it's part of a file name. For example (still on that FreeBSD machine):

$ rmdir bar/foobar
$ grep -H --text foo bar 
bar:�"!
..�"!foobar�"!
              baz�"!foo

Deleting the directory removed it from the filesystem, but it didn't wipe the name of the deleted entry from the on-disk structure that encodes the directory.

When you ask Vim to open a directory, Vim traverses the directory (using dedicated system functions like readdir, not using the generic read function) and displays the results in a nice way.

Grep could implement something like that, but that would be a lot of work relative to the size of grep, it would deviate from grep's core purpose which is to search for the content of files, and the implementation would have to be a compromise that doesn't satisfy many people. Would the directory's representation as text only include file names or also some metadata (why doesn't grep "Jul 20" bar find files modified on July 20)? How would entries be separated (if they're separated by newlines, the representation is ambiguous since file names can contain newlines; if they're separated by null bytes, the output would only be useful for grep --null-data)?

To search in file names, there are already better tools such as shell wildcards, find and locate.

4
  • The current FreeBSD man page for read(2) even says of the EISDIR error that "Directories may only be read directly by root if the filesystem supports it and [a sysctl knob is set]", which seems to me to imply that they wouldn't allow if for regular users at all any more? Also, somehow I had the impression that some systems implemented readdir() by synthesizing the data in some standard format and having it available with read(). Apparently not FreeBSD though.
    – ilkkachu
    Jul 20 at 20:32
  • 1
    @ilkkachu I was using an older FreeBSD. The format seems to be the same for ufs and devfs, at least; I don't know if this is deliberate. Indeed implementing readdir on top of read is traditional, but these days I think the unices that still allow read on directories are more likely to provide it as a backward compatibility hack with read done as a system call that synthesizes something inside the kernel (or maybe as a library function on top of getdents or the like, but that would be less backward compatible since it wouldn't work for statically linked programs). Jul 20 at 21:09
  • So to clarify (based on both your answer and a bit of glancing at GNU's grep.c -- but not enough to be entirely confident that I understand what it's doing in this case): The literal answer to "why does grep say it will 'read directories just as if they were ordinary files', but then print an error message instead" is "grep is reading directories just as if they were ordinary files, but read is returning an error (specifically, EISDIR), which grep is then printing a textual representation of, just as it would if read returned an error on an ordinary file," yes? Jul 22 at 0:50
  • 1
    @ChrisHenry Yes. Grep calls (a library function that calls) the read system call, the system call returns EISDIR, and grep prints the name of the file that it couldn't read and the string associated to the EISDIR error in your LC_MESSAGES locale. Jul 22 at 9:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.