I tried both commands and the command
find | grep 'filename' is many many times slower than the simple
find 'filename' command.
What would be a proper explanation for this behavior?
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems. It only takes a minute to sign up.Sign up to join this community
(I'm assuming GNU
would be quick, because it would just return
filename, or the names inside
filename if it's a directory, or an error if that name did not exist in the current directory. It's a very quick operation, similar to
ls filename (but recursive if
filename is a directory).
find | grep filename
find to generate a list of all names from the current directory and below, which
grep would then filter. This would obviously be a much slower operation.
I'm assuming that what was actually intended was
find . -type f -name 'filename'
This would look for
filename as the name of a regular file anywhere in the current directory or below.
This will be as quick (or comparably quick) as
find | grep filename, but the
grep solution would match
filename against the full path of each found name, similarly to what
-path '*filename*' would do with
The confusion comes from a misunderstanding of how
The utility takes a number of paths and returns all names beneath these paths.
You may then restrict the returned names using various tests that may act on the filename, the path, the timestamp, the file size, the file type, etc.
When you say
find a b c
find to list every name available under the three paths
c. If these happens to be names of regular files in the current directory, then these will be returned. If any of them happens to be the name of a directory, then it will be returned along with all further names inside that directory.
When I do
find . -type f -name 'filename'
This generates a list of all names in the current directory (
.) and below. Then it restricts the names to those of regular files, i.e. not directories etc., with
-type f. Then there is a further restriction to names that matches
-name 'filename'. The string
filename may be a filename globbing pattern, such as
*.txt (just remember to quote it!).
The following seems to "find" the file called
.profile in my home directory:
$ pwd /home/kk $ find .profile .profile
But in fact, it just returns all names at the path
.profile (there is only one name, and that is of this file).
cd up one level and try again:
$ cd .. $ pwd /home $ find .profile find: .profile: No such file or directory
find command can now not find any path called
However, if I get it to look at the current directory, and then restrict the returned names to only
.profile, it finds it from there as well:
$ pwd /home $ find . -name '.profile' ./kk/.profile
I have not understood the problem yet but can provide some more insights.
Like for Kusalananda the
find | grep call is clearly faster on my system which does not make much sense. At first I assumed some kind of buffering problem; that writing to the console slows down the time to the next syscall for reading the next file name. Writing to a pipe is very fast: about 40MiB/s even for 32-byte writes (on my rather slow system; 300 MiB/s for a block size of 1MiB). Thus I assumed that
find can read from the file system faster when writing to a pipe (or file) so that the two operations reading file paths and writing to the console could run in parallel (which
find as a single thread process cannot do on its own.
Comparing the two calls
:> time find "$HOME"/ -name '*.txt' >/dev/null real 0m0.965s user 0m0.532s sys 0m0.423s
:> time find "$HOME"/ >/dev/null real 0m0.653s user 0m0.242s sys 0m0.405s
find does something incredibly stupid (whatever that may be). It just turns out to be quite incompetent at executing
Might depend on the input / output ratio
You might think that
find -name wins if there is very little to write. But ist just gets more embarrassing for
find. It loses even if there is nothing to write at all against 200K files (13M of pipe data) for
time find /usr -name lwevhewoivhol
find can be as fast as
It turns out that
find's stupidity with
name does not extend to other tests. Use a regex instead and the problem is gone:
:> time find "$HOME"/ -regex '\.txt$' >/dev/null real 0m0.679s user 0m0.264s sys 0m0.410s
I guess this can be considered a bug. Anyone willing to file a bug report? My version is find (GNU findutils) 4.6.0
Non-Technical explanation: Looking for Jack in a crowd is faster than looking for everyone in a crowd and eliminating all from consideration except Jack.
Notice: I'll assume that you mean
find . -name filename (otherwise, you're looking for different things;
find filename actually looks into a path called filename, which might contain almost no files, hence exiting really quickly).
Suppose you have a directory holding five thousand files. On most filesystems, these files are actually stored in a tree structure, which allows to quickly locate any one given file.
So when you ask
find to locate a file whose name only requires checking,
find will ask for that file, and that file only, to the underlying filesystem, which will read very few pages from the mass storage. So if the filesystem is worth its salt, this operation will run much faster than traversing the whole tree to retrieve all entries.
When you ask for plain
find however that's exactly what you do, you traverse the whole tree, reading. Every. Single. Entry. With large directories, this might be a problem (it's exactly the reason why several softwares, needing to store lots of files on disk, will create "directory trees" two or three components deep: this way, every single leaf only needs to hold fewer files).
Lets assume the file /john/paul/george/ringo/beatles exists and the file you are searching for is called 'stones'
find / stones
find will compare 'beatles' to 'stones' and drop it when the 's' and 'b' don't match.
find / | grep stones
In this case find will pass '/john/paul/george/ringo/beatles' to grep and grep will have to work its way through the entire path before determining if its a match.
grep is therefore doing far more work which is why it takes longer