4

I am looking for a stackable file system (Like UnionFS, aufs, and OverlayFS) that works similar to bcache or lvmcache but on file level - not on block level:

  • Reads will be done from top-layer. If the file is not there, read the file from bottom-layer and store it in top-layer.
  • Writes will be done to top-layer. After write is done, it will also be written to bottom-layer (like write-back/write-behind caching)
  • When the top-layer reaches a certain size, files will be erased from the top-layer (like an LRU cache). Files will only be removed if they have been written to bottom-layer.

For my use bottom-layer will be 10-1000 times bigger than top-layer - in other words: bottom-layer will not fit in top-layer (again similar to bcache).

Does such stackable file system exist? Can it be done with aufs or similar?

Background

I love file systems, and use all sorts of Fuse-based file systems. But file systems like sshfs can be rather slow, and it would be really handy to have a way of speeding that up if you know that the bottom-layer will only be changed through this system.

Using an sshfs mounted dir as the bottom-layer is just one example, so I am not looking for a specialized approach, that deals with sshfs. I am looking for a general approach where the top-layer and the bottom-layer are just treated as dirs.

-1

What you describe can be handled by the use of standard LVM cache volumes. Typically the top layer is handled by a faster device (eg. SSD). LVM cache volumes are highly configurable, for example you can specify writethrough caching (writes to the lower layer happen at the same time = no data loss) or writecache (writes to the lower layer are delayed = better performance).

lvmcache(7) describes how to convert a standard logical volume (LV) into a volume which makes use of a fast device as a caching top-layer.

| improve this answer | |
  • It's not stackable though, is it...? – roaima Mar 4 at 10:14
  • It's stackable in the sense that reads and writes go through the top layer of cache. Admittedly it's not a file-system as the caching is done on the block level but it allows a file-system to make use of caching. – John Hawthorne Mar 4 at 10:49
  • 1
    I still don't think I can stack it on top of, say, S3QL or even CIFS, though. – roaima Mar 4 at 10:53
  • lvmcache and bcache are pretty much the same thing and not what I am looking for. I have elaborated the question why a block based approach will not solve my issue. – Ole Tange Mar 4 at 14:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.