readdir()
only reads 32K of directory entries at a time.
Why does it read only 32K entries in one shot? Is it dependent on buffer or any other parameters?
Can I change that value so that I can read as many directory entries as I want?
readdir()
only reads 32K of directory entries at a time.
Why does it read only 32K entries in one shot? Is it dependent on buffer or any other parameters?
Can I change that value so that I can read as many directory entries as I want?
The maximum number of directories is file system dependent:
dir_nlink
flag to increase this numberIt's only since 2006 when ext4
was ratified that max directory limits were increased above 32000 mark, so there would be no point for readdir()
to read more than that.
If you have more than 32000 subdirectories, I would consider learning about performance degradation due to large amount of subdirectoies
ext4
has dir_nlink
described as "allows more than 65000 subdirectories per directory", which hints at that. I don't think that's not the same as the max number of files per directory, and readdir()
reads files too. Also, the phrase is "32K of directory entries", not "32K directory entries", so if that means the buffer size in bytes, the actual number of entries read would be lower.
glibc's opendir()
function calculates the allocation for readdir()
as MIN(MAX(statp->st_blksize, 32KB), 1MB)
. The filesystem can recommend an I/O block size via st_blksize
. On ext4
it is:
$ stat -c '%o' .
4096
So, readdir()
will use a 32KB buffer (because this is larger than 4KB).
I don't know of a way of configuring st_blksize
on ext4
(or other filesystems).