5

beforehand: this evolved from a general question (I thought it was the case with every folder) to this specific corner case.
as it seemed that a few other people are as curious as myself, I decided to not delete this question but rework it.


Let's say I want to list all items contained direct in my home folder, except the dot files & folders. IMHO, the following find command should do this (I explicitly wrote "dot items", because if you use a .hidden file the "hidden items" may differ):

find $HOME/ -mindepth 1 -maxdepth 1 -path "$HOME/.*" -prune -o -print

-mindepth 1 to ensure that $HOME itself is not included, -maxdepth 1 to limit to one folder level, -path "$HOME/.*" -prune to exclude everything in the home folder starting with a dot and -o -print to ensure that not the excluded elements are in the output according to the man page.

So far, so good. It kind of works (even if there are dot-folders for which you don't have the permissions), but it starts to choke on $HOME/.gvfs although it shouldn't be accessed at all due to -path ... -prune:

/home/user/file a
/home/user/folder b
...
find: '/home/user/.gvfs': Permission denied
/home/user/.gvfs
...
/home/user/file z

I know I can use -perm at find and 2> /dev/null to work around it, but what really appalled me is, that /home/user/.gvfs is included in the list (although all other dot items are excluded)!!!
and I'm pretty sure some not so careful people are tempted to get rid of the error with 2> /dev/null and move on. ...if they recognize the error at all, because at longer lists with a few 100 lines you have to explicitly check for errors with something like adding 1> /dev/null prior to working with that command.


really ignoring a directory/tree?

Why is find trying to access /home/user/.gvfs at all, despite -maxdepth 1 or at least -prune should prevent this? e.g. from the man page:

‘-prune’ action (which only prevents further descent [...])

even in an older man page (installed on my system) for -path it is stated:

To ignore a whole directory tree, use -prune rather than checking every file in the tree.

dangerous listing of an "ignored" directory itself!

If all /home/user/.* would be really ignored, perhaps the second part of my question won't be necessary, as it seems to me this is because of an internal error:
why is /home/user/.gvfs still in the list and how to prevent it? Is there a way without fiddling around with -user/-perm and 2> /dev/null for every find command (at least with a -prune action)?

future usage -> efficiency?

What is really ignored with -prune (or at least with -path ... -prune)?

If with -prune still (nearly) all files are checked, I would ditch using this complicated option and rather use ! or -not, especially as -prune is apparently much more error-prone.

3
  • "interesting"... i fired up a virgin machine and tried to reproduce it, but no luck. only when using -maxdepth 2 and without -prune i get the permission error, which is obvious. || it seems that the older machine where i tested it, is somehow f*ed up or at least its gvfs. :-( Mar 9, 2021 at 14:31
  • 1
    now i'm curious why the heck i get this permission error although nevertheless $HOME/.gvfs would match the -prune test (even if f*ed up)! but i'm afraid i'll never know... Mar 9, 2021 at 14:55
  • @StéphaneChazelas whether this works as in sfind, depends on two things: 1) the OS and the find implementation both need to support the fle type in struct dirent. 2) The callback function from the treewalker in the find implementation needs to abort the traversal early with -maxdepth.
    – schily
    Mar 9, 2021 at 20:10

1 Answer 1

5

This appears to be a bug in the GNU implementation of find. Here, the trigger seems to be the fact that the lstat() / fstatat() system call (to retrieve a file's metadata) fails with EACCESS (permission denied) on that .gvfs file.

That typically happens on Linux for files that are mount points of FUSE-based file systems for which the -o allow-other option has not being used. That can also happen with find -L / find -follow when there are symlinks that point to inaccessible areas.

I find here that -prune for those files returns false which is clearly a bug as -prune is documented (and required by POSIX) to always return true.

As a work around until that bug is fixed, you can replace -prune with '(' -prune -o -true ')' (note that -true is a GNU extension; a portable equivalent could be ! -name '' which is guaranteed to be true, though you could also do '(' -prune -o ! -prune ')')

Now, there are a few issues with your code itself. In:

find $HOME/ -mindepth 1 -maxdepth 1 -path "$HOME/.*" -prune -o -print
  • In most Bourne-like shells, that $HOME/ should be quoted as otherwise the $HOME expansion is subject to split+glob. You could also write it ~/.
  • -prune is to tell find not to descend into a directory. Here it's superfluous as maxdepth 1 will stop find from descending into any directory (other than the top-level (depth 0) one $HOME/).
  • -path takes a pattern. That means that the contents of $HOME will be taken as a pattern. If your $HOME is for instance /home/[112] me, -path '/home/[112] me/.*' would match on /home/2 me/.foo, but not on /home/[112] me/.foo. Also matching on the full path here is not necessary, you can match on the file name instead with -name '.*'.
  • * with GNU find only matches on characters, so .* matches on filenames that start with . and are also made of sequences of bytes that form valid characters. So you can only use that to match hidden files in locales where every sequence of byte form valid characters (such as the C locale).

So here, if the point is to list the non-hidden files in your HOME directory with their full paths, you'd do:

LC_ALL=C find ~/ -mindepth 1 -maxdepth 1 ! -name '.*'

Which would also avoid that GNU find bug.

Or the standard equivalent (though adds a /./ in the paths):

LC_ALL=C find ~/. ! -name . -prune ! -name '.*'

(here, the GNU find bug would be hit for files that do not start with . and are not lstat()able, but you'll see at that bug description above that whether non-lstat()able files are reported or not varies greatly with the implementation).

Here, I'd just use zsh's:

print -rC1 ~/*(ND)

Or if you don't want them sorted like with find:

print -rC1 ~/*(NDoN)
9
  • yes, in this case using -prune is superfluous, but this is just a pretty tuned down version of an other find command (trying to make it as short & easy as possible but still to have some kind of sense behind it). Mar 10, 2021 at 17:38
  • usually i quote "everything". ;-) ...no idea how this one slipped trough. - correct quoting is one of the most basic & important but IMHO also difficult topics, and sadly very few seem to care about it. Mar 10, 2021 at 17:40
  • last but not least: i try to avoid LC_ALL=C in conjunction find because it butchers language specific characters resp. replaces them with ??. Mar 10, 2021 at 17:42
  • 1
    Only when the output goes to a terminal and only with GNU find. You can always pipe the output to cat to work around it. Mar 10, 2021 at 17:45
  • awesome ideas for the workarounds!!! both are so simple and easy, but therefore genius! thank you very much @StéphaneChazelas! Mar 10, 2021 at 17:53

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .