Usually when I have programs that are doing a full disk scan and going over all files in the system they take a very long time to run. Why does updatedb run so fast in comparison?
2 Answers
The answer depends on the version of locate you’re using, but there’s a fair chance it’s mlocate, whose updatedb runs quickly by avoiding doing full disk scans:
mlocate is a locate/updatedb implementation. The 'm' stands for "merging": updatedb reuses the existing database to avoid rereading most of the file system, which makes updatedb faster and does not trash the system caches as much.
(The database stores each directory’s timestamp, ctime or mtime, whichever is newer.)
Like most implementations of updatedb, mlocate’s will also skip file systems and paths which it is configured to ignore. By default there are none in mlocate’s case, but distributions typically provide a basic updatedb.conf which ignores networked file systems, virtual file systems etc. (see Debian’s configuration file for example; this is standard practice in Debian, so GNU’s updatedb is configured similarly).
answered Jan 2, 2019 at 16:20
-
Fairly good question and answer, did not even know there were "differencial" scannings. Commented Jan 2, 2019 at 16:25
-
1Thanks! I had never noticed that modifying a file also changes the ctime and mtime of all its parent directories.– hugomgCommented Jan 2, 2019 at 17:21
-
4@hugomg I don't think it actually does. It should only change the
mtimeof its immediate parent.– Kusalananda ♦Commented Jan 2, 2019 at 17:33 -
So if I understand it correctly,
mlocatecares aboutctimeandmtimewhich implies it cares only of whether list of directory entries is still the same ( no removed or added files), which means it doesn't care about actual files themselves. Is that correct ? Commented Jan 3, 2019 at 1:12 -
@Sergiy: Of course.
locateisn'tgrep -R. It does not read file content.– KevinCommented Jan 3, 2019 at 1:21
In addition to checking modification times, mlocate also ignores certain subtrees of the file system that have lots of uninteresting or potentially duplicate files, as specified in /etc/updatedb.conf (and described in man updatedb.conf):
- Bind mounts
- Some kinds of file systems (9p, afs, bdev, etc)
- VCS repository databases (.git, .hg, etc)
- Some hard-coded directories (/media, /tmp, /var/spool/cups, etc).
-
This isn’t the case by default though, so the base behaviour depends on the distribution being used. (Other
updatedbimplementations also support configured exclusions.) Commented Jan 3, 2019 at 8:17 -
find -xdevscan is very fast, try this unix.stackexchange.com/a/379827/208590 ?