4

I have read many controversal statements about ZFS on low memory systems on the internet, but most of the use cases was for performant data storage.

I want to use ZFS not for performance reasons, but because it supports transparent compression and deduplication (the latter may be optional) and still seems to be more mature than BTRFS. I don't want to use any RAID configuration.

I want to use it on a laptop computer, for root and home file system, and storage space and data safety (recoverability after power loss or other random inconsistencies, very low risk of corruption due to low RAM, etc.) is more important than disk performance. I want comparable safety as ext2/3/4 give.

I would like to use ext4 ontop of a ZVOL.

So, the questions are:

  • Can ZFS be configured to work reliably with "low RAM" if IO performance/ caching is not of concern, and no RAID funtionality is wanted?
  • How does the RAM needed change if I do not use ZFS as a filesystem itself, but just use ZVOLs where I put another filesystem ontop?
  • How does RAM needed change with deduplication turned on? If deduplication is turned on and RAM starts to get low, is it still safe -- can ZFS just suspend deduplication and use less RAM?
  • Is it possible to deactivate automatic deduplication, but run it from time to time manually?
  • Can ext4 ontop of a ZVOL reliably store my data even on low RAM situations, and if inconsistencies happen, success chances for repairs are high (as it is with ext2/3/4)? Does ext4 ontop of a ZVOL increase rubustness because it adds ext4's robustness, or is data as robust as the underlying ZVOL is?

System specs:

  • Linux
  • 8 GiB RAM (shared with graphics card), but most (at least 7 GiB) of it should be available for user space software,
  • about 700 GiB SSD storage to use for the ZFS,
  • maybe on another system 128 GiB of eMMC to use for ZFS.
  • Current disk usage (du -sh of the bigger directories at /) (/ is ext4, /var mounted ontop is reiserfs) (want to move that to a storage with transparent compression):

    74M     /etc
    342G    /home
    5.0G    /opt
    1.5G    /root
    261M    /tmp
    35G     /usr
    30G     /var
    

OR, just use BTRFS (have read that severe/ hard to recover data loss can occur due to "bugs", but that is all controversial ...)?

  • 1
    I would like to use ext4 ontop of a ZVOL. Why? Why not just use ZFS directly? If reliability is a concern, adding extra layers just adds more things that can break. – Andrew Henle Apr 13 '17 at 13:43
  • @AndrewHenle: Yes, no special reason. But thought ext4 is quite reliable and has low demands on the system, so I thought having the actual filesystem structure in ext4-format adds the advantages of ext4 (low ressources, reliability) ontop of the transparent compression of ZFS. Am I wrong? – Golar Ramblar Apr 13 '17 at 13:50
  • 1
    ZFS is very reliable. And if you put ext4 on top of it, you'd be reliant on both ext4 and the underlying ZFS pool. Is this added complexity worth it? Plus in your question and comments you talk about performance and resources. I can't imagine this config being optimal long-term. A file system on top of another is usually only used for testing, or to meet a requirement. – sleepyweasel Apr 14 '17 at 17:27
4

Short answer:

  • Yes, its possible to use low RAM (~ 1 GB) with ZFS successfully.
  • You should not use dedup, but RAID and compression is usually ok.
  • Once you have duplication enabled, it works for all newly written data and you cannot easily get rid of it.
  • You cannot enable dedup retroactive, because it works on online data only.
  • Your idea is needlessly complex for no good reason, so I would recommend to just use ZFS and call it a day.

Long answer:

Can ZFS be configured to work reliably with "low RAM" if IO performance/ caching is not of concern, and no RAID funtionality is wanted?

Yes, even with RAID features enabled. You need much less than people claim on the net, for example look at this guy who runs a speedy file server with FreeBSD, 2 cores and 768 MB virtualized. Or have a look at the SolarisInternals Guide (currently only available through archive.org), where 512 MB is mentioned as the bare minimum, 1 GB as minimum recommendation and 2 GB as a full recommendation.

I would stay away from dedup, though. Not because it is slow because of paging memory, but because you cannot go back to non-dedup if your system grinds to a halt. Also, its a trade between RAM and disks, and on a budget system you have neither, so you will gain not much.

How does the RAM needed change if I do not use ZFS as a filesystem itself, but just use ZVOLs where I put another filesystem ontop?

You would need additional memory for the second filesystem and for the layer above ZFS, depending on how you plan to access it (virtualization like KVM, FUSE, iSCSI etc.)

How does RAM needed change with deduplication turned on? If deduplication is turned on and RAM starts to get low, is it still safe -- can ZFS just suspend deduplication and use less RAM?

You cannot suspend deduplication, but your data is still safe. There will be a lot of memory swapping and waiting, so it might not be very usable. Deduplication is online, so to disable it, you would need to turn dedup off and write all data again (which is essentially copying all data to a new filesystem and destroying the old one).

Is it possible to deactivate automatic deduplication, but run it from time to time manually?

No, because it does not affect data at rest. If you have dedup on and want to write a block, ZFS looks if it is present in the dedup table. If yes, then the write is discarded and a reference is added to the dedup table. If no, it is written and the first reference is added. This means that your old data is not affected by dedup, and turning it on without writing any new block does nothing reagarding the used size of the old data.

Can ext4 ontop of a ZVOL reliably store my data even on low RAM situations, and if inconsistencies happen, success chances for repairs are high (as it is with ext2/3/4)? Does ext4 ontop of a ZVOL increase rubustness because it adds ext4's robustness, or is data as robust as the underlying ZVOL is?

In my eyes this is needless complexity, as you would get no new features (like in the reverse case with ext4 below and ZFS on top, e. g. snapshots), and additionally get some new responsibilities like fsck and more fdisk formatting exercises.

The only use case where I would do something like that is if had a special application that demands a specific file system's low-level features or has hard-coded assumptions (fortunately, that behavior seems to have died in recent times).

  • 1
    ext4 below and ZFS at top, in which scenario would that make sense, instead of having ZFS residing directly on the block device? – Golar Ramblar Apr 13 '17 at 16:04
  • @GolarRamblar In case your storage is out of your control (for example remote or rented storage) and you still want high-level features like snapshots, the copies property and/or file checksums (even if you cannot replace bad blocks, you still are informed that they are bad and can replace the files from older backups). – user121391 Apr 18 '17 at 7:05
1

user121391 did a great job at giving you good information, so I'm only going to touch on your memory concerns.

As far memory goes, yes, ZFS can be a memory hog. It will make use of free system memory for caching. Over the years, Sun, now Oracle has tweaked, improving performance, reliability, and improving memory usage. You can check the ZFS internals/evil tuning guide site, or ZFS blog for specifics. However, most of these tweaks and features may only be found in the Solaris 11.x version of ZFS.

If you're really worried about memory, you will not want to use dedup as it requires a fair amount of memory as dedups are live and has to hold the dedup table in memory.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.