Does Linux have ways to cache IO with memory

Question

My server has a large amount of memory. However, the amount of IO (each of which is small) is very high.

Is there a way to make a virtual drive out of portion of the memory?

I realize I can also use SSD, but those are expensive.

I need both speed and size. So memory only won't work. Data not often accessed should be in hdisk.

Answer 1

Use tmpfs. It is usually mounted at /dev/shm with the default size 1/2 of total RAM. The advantage is, that the memory is available for general usage by system, until you place something there (it is reserved on the fly). You might want to tweak the default setting a bit though - I personally have something like the following in /etc/fstab:

tmpfs  /dev/shm  tmpfs  defaults,size=16m   0   0
tmpfs  /free     tmpfs  defaults,size=66%   0   0

This does two things:

1. mounts rather small (16MB) tmpfs at /dev/shm/ for applications that might want to use it. The size is limited to prevent accidental waste of memory due to bugs.

2. mounts tmpfs with size of 2/3 of available RAM at /free.

Note that the filesystem block size is equal to memory page size - should you be using architecture with larger memory page size (e.g. PowerPCs or Itanium), even empty file will occupy whole page. This overhead can be reasonably reduced by creating a large file, formatting it with a "regular" filesystem with smaller blocks (e.g. XFS can use blocks as small as 512B), and loop-mounting it.

As for SSD - they are orders of magnitude slower than RAM, will get cached anyway, and have limited number of erase cycles, so question is whether you want to use these in situations when you have enough RAM. And by the way, there even are hardware RAM drives.

Answer 2

Yes. This is called the disk cache. It happens automatically.

If you're seeing too much I/O, then either you have runaway programs, you have tuned something badly, or your system doesn't have enough resources.

You can put files in a RAM drive, but this is almost always worse than letting the kernel manage a cache of recently-used data. Every byte that you allocate to a RAM drive is that much less for applications and disk cache.

Answer 3

Check out RapidCache!

It's part of RapidDisk. It does IO Caching of Blocks vs files (as the linux cache does), making it ideal for VM's!

http://rapiddisk.org/index.php