I’d like to set the system up to use most RAM for file system metadata caching, but only a reasonably small amount for read/write caching and prefetching files. Ideally I would like to be able to browse the filesystem (as much as it fits in the RAM) without spinning up the disks until I actually open a file.
Here are the details:
I have a home-made file server. It’s got five disks in a LVM volume around 9TB, but only 4GB of RAM. Since the server doesn’t do much else then serve files, most of the RAM is used for caching. (“free” reports 3.4G out of 3.9G used for cache.)
The server lives in my bedroom, and if all disks are spinning it makes enough noise to be annoying when it’s quiet. (I don’t mean seek noise, just spinning noise. The disks are of various makes and models, and I think slight differences in rotation speeds cause interference. No disk is noisy on its own, but if some of them are spinning together there’s a slight noise with a sub-Hertz period.) So I configured the server to spin down the disks most of the time.
Of course, if the disks are spun down when I open a folder in my file manager, there’s a delay while whichever of the disks has that folder spins up. Just that is no big deal. But depending on where I look, it can happen several times in a row, if LVM happened to spread the metadata for each subfolder on the different disks.
I suspect that Linux mostly fills its cache with file contents, and possibly prefetched data. The caching isn’t very useful beyond a few MB to ensure smooth playback; if I just watched a movie I probably won’t look at it again anytime soon. Prefetching, if it happens, is also completely useless in my case, after more than a few MB.
But one would think that 4GB should be plenty to be able to cache most file-system metadata, at least those parts that were already visited, so that I could browse the files without needing to spin up the disks if it turns out they’re sleeping.
There would still be a delay when opening the file, but that’s OK. Compare “click; wait; click; wait; click; wait; play; watch” with “click; click; click; play; wait; watch”. The former is incredibly frustrating; the latter is almost expected.
Should it matter, the kernel is 3.2, the OS is Debian, the volume is lvm2, and the FS is ext4.
The only reason for the spin down is noise during the night; the server is otherwise running continuously. (I made it as low-power as reasonable.) The spin-down delay is varied depending on the time of day.
The hard disks are only for media. The OS is on a separate (small) flash drive. (Which means any spin-up delays comes from the data, not just because it needed something in
/usror whatever. I could spare a few GB on it if it would help with my problem somehow.
A reasonable impact on performance is not a big deal. The disks are faster than my network anyway.