-2

readdir() only reads 32K of directory entries at a time.

Why does it read only 32K entries in one shot? Is it dependent on buffer or any other parameters?

Can I change that value so that I can read as many directory entries as I want?

Improve this question
6
  • "Why it reads only 32K entries?" sounds like you are going to explain why that is the case, but there rest of the paragraph doesn't do that. Did you mean "Why does it read only 32K entries?" (to which the answer probably is: efficiency).
    – Anthon
    Sep 2, 2015 at 5:57
  • What I want to ask is why 32K? Whether it is fixed or it depends on system parameters such as buffer size.
    – Sushant Mane
    Sep 2, 2015 at 6:29
  • If you want to ask that, please use the normal question form "Why does it read only 32K entries". It is maybe because English is not my native language, but your phrasing is incomprehensible (at least to me).
    – Anthon
    Sep 2, 2015 at 6:34
  • I understood the question nevertheless, but motivations aren't clear. Though I guess that gwillie's answer addresses your question correctly.
    – MatthewRock
    Sep 2, 2015 at 7:38
  • Are you asking about readdir the library routine, or readdir the old system call?
    – Mark Plotnick
    Sep 2, 2015 at 9:02

2 Answers 2

Reset to default
4

The maximum number of directories is file system dependent:

  • ext2: 32768
  • ext3: 31998
  • ext4: 65000. use dir_nlink flag to increase this number

It's only since 2006 when ext4 was ratified that max directory limits were increased above 32000 mark, so there would be no point for readdir() to read more than that.

If you have more than 32000 subdirectories, I would consider learning about performance degradation due to large amount of subdirectoies

Improve this answer
1
  • Is that the maximum number of subdirectories per directory? My man page for ext4 has dir_nlink described as "allows more than 65000 subdirectories per directory", which hints at that. I don't think that's not the same as the max number of files per directory, and readdir() reads files too. Also, the phrase is "32K of directory entries", not "32K directory entries", so if that means the buffer size in bytes, the actual number of entries read would be lower.
    – ilkkachu
    Nov 23, 2019 at 12:49
1

glibc's opendir() function calculates the allocation for readdir() as MIN(MAX(statp->st_blksize, 32KB), 1MB). The filesystem can recommend an I/O block size via st_blksize. On ext4 it is:

$ stat -c '%o' .
4096

So, readdir() will use a 32KB buffer (because this is larger than 4KB).

I don't know of a way of configuring st_blksize on ext4 (or other filesystems).

Improve this answer

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .