Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

I'm looking for ways to make use of an SSD to speed up my system. In “Linux equivalent to ReadyBoost?” (and the research that triggered for me) I've learned about bcache, dm-cache and EnhanceIO. All three of these seem capable of caching read data on SSD.

However, unless I'm missing something, all three seem to store a file/block/extent/whatever in cache the first time it is read. Large sequential reads might be an exception, but otherwise it seems as if every read cache miss would cause something to get cached. I'd like the cache to cache those reads I use often. I'm worried that a search over the bodies of all my maildir files or a recursive grep in some large directory might evict large portions of stuff I read far more often.

Is there any technology to cache frequently read files, instead of recently read ones? Something which builds up some form of active set or some such? I guess adaptive replacement might be a term describing what I'm after.

Lacking that, I wonder whether it might make sense to use LVM as a bottom layer, and build up several bcache-enabled devices on top of that. The idea is that e.g. mail reads would not evict caches for /usr and the likes. Each mounted file system would get its own cache of fixed size, or none at all. Does anyone have experience with bcache on top of lvm? Is there a reason against this approach?

Any alternative suggestions are welcome as well. Note however that I'm looking for something ready for production use on Linux. I feel ZFS with its L2ARC feature doesn't fall in that category (yet), although you are welcome to argue that point if you are convinced of the opposite. The reason for LVM is that I want to be able to resize space allocated for those various file systems as needed, which is a pain using static partitioning. So proposed solutions should also provide that kind of flexibility.


Edit 1: Some clarifications.

My main concern is bootup time. I'd like to see all the files which are used for every boot readily accessible on that SSD. And I'd rather not have to worry about keeping the SSD in sync e.g. after package upgrades (which occur rather often on my Gentoo testing). If often-used data which I don't use during boot ends up in the cache as well, that's an added bonus. My current work project e.g. would be a nice candidate. But I'd guess 90% of the files I use every day will be used within the first 5 minutes after pressing the power button. One consequence of this aim is that approaches which wipe the cache after boot, like ZFS L2ARC apparently does, are not a feasible solution.

The answer by goldilocks moved the focus from cache insertion to cache eviction. But that doesn't change the fundamental nature of the problem. Unless the cache keeps track of how often or frequently an item is used, things might still drop out of the cache too soon. Particularly since I expect those files I use all the time to reside in the RAM cache from boot till shutdown, so they will be read from disk only once for every boot. The cache eviction policies I found for bcache and dm-cache, namely LRU and FIFO, both would evict those boot-time files in preference to other files read on that same working day. Thus my concern.

share|improve this question
    
Is that really not the default behaviour? I'd be surprised if it kicked out frequently used data first. –  frostschutz Dec 29 '13 at 17:01
2  
"I feel ZFS with its L2ARC feature (isn't ready for production use on Linux) (yet)" Actually, I use almost exclusively ZFS on Linux on my main workstation since a few months back, and my experience is that it's stable. I'll readily admit I haven't pushed it to its limits, but it works well enough that I have no concerns with committing the integrity of my data to ZFS. And regardless of which file system you're using, you still need to make backups. The downside of ZFS (and ZFS On Linux in particular) is that it wants 64-bit and prefers having lots of RAM. –  Michael Kjörling Dec 29 '13 at 18:27
    
@MichaelKjörling: Thanks for sharing that experience. I must confess that my last close look at ZFS was when the only Linux option was via FUSE, and shortly after I got a corrupted and unrepairable ZFS on OpenSolaris first day after setting up that system. Nevertheless, looking at the docs I read: “Persistence in the L2ARC is not needed, as the cache will be wiped on boot.” So at least to speed up booting, which is my main concern, this apparently won't help. –  MvG Dec 30 '13 at 7:54
    
Wait a minute, in the question you're asking for a way to speed up accesses to frequently read data, and now you're asking how to speed up booting. Which one will it be? If you want to speed up booting, the simple option would seem to be to put the files that are used heavily during boot onto a SSD, possibly using MD RAID1 for redundancy and possibly extra speed, and then use whatever file system and drive setup you prefer for bulk data storage. That pretty much just means separating the binaries and data storage file systems and putting each on media with appropriate characteristics. –  Michael Kjörling Dec 30 '13 at 9:04
    
One major reason why multiple file systems on different media isn't used more in Windows is its legacy of by default putting each file system on a different drive letter. Far from all programs deal well with such advanced features as NTFS junction points, which are par for the course on *nix under the name of mount points. By not focusing on how something similar is done on Windows, it's almost certainly possible to come up with a solution that works well on Linux, but of course you first have to clearly state your end goal. –  Michael Kjörling Dec 30 '13 at 9:08

1 Answer 1

However, unless I'm missing something, all three seem to store a file/block/extent/whatever in cache the first time it is read.

The other option would be to not cache anything the first time it is read, and instead keep a count of the number of times of something is needed, then use some arbitrarily number to decide when something has been "frequently used".

No one would ever implement a system this way, because it means if we say the number is 10 or 20 or 100 times, then once the number is reached, it is obvious the system failed to cache a frequently accessed item the first X number of times. Not so useful!

I'd like the cache to cache those reads I use often.

To sort of re-iterate the previous point, what's "often"? Realistically, it couldn't be a fixed number, since if the system is on long enough, many things could become "often used". It could be a number scaled to a "high score", but in that case, the scale could become very unbalanced if you have a few small items accessed a disproportionate number of times.

In short: no caching mechanism is going to use a minimum count. It's going to cache everything until the cache is full, then it will start evicting things on the basis of some priority algorithm.

Since "frequency" is a factor of "often", it makes sense that every read will cache something, even if it is the first time and the cache is full, since the last file read will be the "most often read file" if we consider a frequency of "the number of times this file was read in the past X reads", where X = 1.

I'm worried that a search over the bodies of all my maildir files or a recursive grep in some large directory might evict large portions of stuff I read far more often.

Probably not, if the cache was full to start with. Each read will be cached, but it will also be evicted sooner than previously cached stuff that has been often accessed.

I guess adaptive replacement might be a term describing what I'm after.

Notice in the "Summary" on that wikipedia page that the discussion is about different strategies (vs. LRU) for ranking things in the cache, not things that have never been in the cache. This follows the logic I've described above: everything goes in the cache, and the efficacy of the caching mechanism is determined by the algorithm for evicting things from the cache. Not putting them in.

share|improve this answer
    
Thanks! I guess I'd have considered X=2 for this “magic threshold” your thought experiment uses. So a file read once would not get cached only noted, but twice it would. This (combined with OS-level caching to RAM) should take care of most rarely read files, while at the same time minimizing the impact for really frequently used items. That said, the eviction strategies I found, namely LRU, FIFO and random, don't sound like they would be particularly aimed at keeping track of usage frequency. So even if stated in terms of eviction policy, the problem apparently remains. –  MvG Dec 29 '13 at 17:40
    
Keep in mind the normal page cache (in RAM) probably complicates all this: since the most frequently used pages are cached in RAM, the number of times the disk blocks from whence they came are accessed will be affected (it'll be very low)! Yet that stuff can be evicted from the RAM cache eventually -- how should the SSD cache mechanism compensate for this? There is a lot to consider here...I'm sure a disk block cache is intended to favor frequently used things, but exactly how may not be as straightforward as it might seem on first consideration. –  goldilocks Dec 29 '13 at 18:24

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.