So I have a desktop with a fast SSD and large HDD. I am trying to get a well configured large, fast zpool out of it.
I have read that I can carve separate partitions into the SSD for the ZIL and L2ARC which would seem to do what I want, except I have to manually configure how big each partition should be. What I don't like about it is that it's somewhat involved, potentially hard to reconfigure if I need to change the partitions, and it sounds like the maximum filesystem size is limited by the HDD alone since the intent is that everything on the ZIL and L2ARC has to also make it to disk, at least eventually. Also it's not clear if the L2ARC is retained after system reboot or if it has to be populated again. It also seems inefficient to have to copy data from ZIL to L2ARC if they are both on the same SSD, or even to HDD if there is currently no pressure on how much hot data I need on SSD.
Alternatively, it seems I can also just have 1 partition on SSD and 1 on HDD and add them to a zpool directly with no redundancy. I have tried this, and noticed sustained read/write speeds greater than what HDD alone can muster. But I don't know if everything is just going to the SSD for now, and everything will go to HDD later once SSD is all filled up. Ideally, I would like to have ZFS transparently shuffle the data around behind the scenes to try to always keep the hot data on the SSD similarly to what L2ARC, and have a sensible amount of empty space on SSD for new writes. The ZIL should be automatically managed to be the right size and preferably live on the SSD as much as possible. If I go the manually configured ZIL + L2ARC route, it seems like the ZIL only needs to be about (10 sec * HDD write speed) big. Doing this maximizes the size of L2ARC which is good. But what happens if I add a striped disk which effectively doubles my HDD speed (and capacity)?
Summary of questions if using SSD for ZIL + L2ARC:
- If I set up SSD for ZIL + L2ARC, how hard is it to re-set it up with different partition sizes?
- If I use SSD for L2ARC, is its capacity included in total available pool capacity, or is the pool capacity limited by HDD alone?
- Is L2ARC retained after system reboot, or does it have to be re-populated?
- Does data have to be copied from ZIL to L2ARC even if both are on same physical SSD?
- If ZIL is on SSD and there is still plenty of room for more intents to be logged, does the ZIL still automatically get flushed to SSD? If so, when/under what circumstances?
Summary of questions if using SSD + HDD in a single zpool:
- ZFS obviously notices the difference in size between SSD and HDD partitions, but does ZFS automatically recognize the relative performance of SSD and HDD partitions? In particular,
- How are writes distributed across the SSD and HDD when both are relatively empty?
- Does ZFS try to do anything smart with data shuffling once the SSD part of the zpool fill up? In particular,
- If the SSD part of zpool is filled up, does ZFS ever anticipate that I will have more writes soon and tries to move data from SSD to HDD in the background?
- If the SSD part of zpool is filled up, and I start accessing a bunch of data off HDD, and not so much off SSD, does ZFS make any effort to swap the hot data to SSD?
Finally, the most important question:
- Is it a good idea to set up SSD + HDD in same pool, or is there a better way to optimize my pair of drives for both speed and capacity?