3

I am looking for a way to compress swap on disk. I am not looking for wider discussion of alternative solutions. See discussion at the end.

I have tried...

Using compressed zfs zvol for swap is NOT WORKING. The setup of it works, swapon works, swapping does happen somewhat, so i guess one could argue it's technically a working solution, but exactly how "working" is your working solution, if it's slower than floppy disk access, and causes your system to completely freeze forever 10 times out of 10? Tried it several times -- soon as system enters memory pressure conditions, everything just freezes. I tried to use it indirectly as well, using losetup, and even tried using zfs zvol as a backing device for zram. No difference, always same results -- incredibly slow write/read rates, and system inevitably dies under pressure.

BTRFS. Only supports uncompressed swapfiles. Apparently, only supports uncompressed loop images as well, because i tried dd-ing an empty file, formatting it with regular ext2, compressing it, mounting as a loop device, and creating a swapfile inside of it. Didn't work, even when i mounted btrfs with forced compression enabled -- compsize showed the ext2 image compression ratio of exactly 1.00 .

Zswap -- it's just a buffer between ram and regular disk swap. The regular disk swap keeps on being the regular disk swap, zswap uncompresses pages before writing them on there.

Zram -- has a backing device option since it's inception as compcache, and one would think, is a perfect candidate to have had compressed disk swap for years. No such luck. While you can do writeback of compressed in-ram pages to disk at will, the pages get decompressed before they're written. Unlike zswap, doesn't write same- and zero-filled pages though, which both saves i\o, slightly improves throughput, and warrants the use of loop-mounted sparse files as backing_dev. So far, this is the best option I found for swap optimization on low-end devices, despite it still lacking disk compression.

Any ideas what else I can try? Maybe there's some compressed block device layer, that I don't know of, that can compress anything written to it, no filesystem required? Maybe there's some compressed overlay I could make use of? Not done in FUSE though, as FUSE itself is a subject to swapping, unless you know a way to prevent it from being swapped out.

Since i don't see this being explored much, you're welcome to suggest any madness you like. Please, let's throw stuff at the wall and see what sticks.

For experts -- if any of you have read, or even written, any part of linux sourse code that relates to this problem, please describe in as much detail as possible, why do you think this hasn't been implemented yet, and how do you think it could be implemented, if you have any idea. And obviously, please do implement that if you can, that'll be awesome.

Discussion

Before you mark it as a duplicate -- I'm aware there have been a few questions like that around stackexchange, but none i saw had a working answer, and few had any further feedback. So I'll attempt to describe details, sort of aggregate everything, here, in hopes that someone smarter than me can figure this out. I'm not a programmer, just a user and a script kiddie, so that should be a pretty low bar to jump over.

just buy more ram, it's cheap

get an ssd

swap is bad

compression is slow anyway, why bother

If all you have to say, are any of the above quotes -- go away. Because the argument is optimization. However cheap RAM is these days, it's not free. Swap is always needed, the fact that it's good for the system to have it, has been established for years now. And compression is nothing, even "heavy" algorithms perform stupidly fast on any processors made in the last decade. And lastly, sure, compression might actually become a bottleneck if you're using an ssd, but not everyone prioritizes speed over disk space usage, and hdd drives, which DO benefit grossly from disk compression, are still too popular and plentiful to dismiss.

Improve this question
5
  • 2
    A friendly warning... I don't think this is going to fly, don't be surprised if the question gets deleted. Open-ended opinionated discussion you're inviting isn't the name of the game in SE sites. Please check the Tour.
    – Peregrino69
    Mar 20 at 8:50
  • 3
    One significant hurdle is that the kernel skips the file system when dealing with swap, so you’d need block-level compression, not file-level.
    – Stephen Kitt
    Mar 20 at 8:51
  • 2
    I was about to read that with great interest, but after the first two paragraphs, I got the feeling that I'm being antagonized though not having said anything, and if I then wrote an answer that contradicts the author's perception in any way, it would fall on deaf ears. Deciding to just leave this question alone altogether :(
    – Marcus Müller
    Mar 20 at 9:43
  • I recognise your pain asking about swap which is somehow a religious question in the Linux world. FYI swap is not required and does not always make your system better. But it does in many cases.
    – Philip Couling
    Mar 20 at 19:44
  • 2
    Thank you for editing, yes I've seen too many times that a question gets no useful reply whatsoever, so I guess I went a bit too hard on specifying what I need. Also I agree, swap isn't absolutely necessary, I just meant that in a majority of cases it's a good idea to have one, especially for home desktop use, which is my use case
    – linuxlover69
    Mar 23 at 2:13

1 Answer 1

Reset to default
5

I don't have a concrete answer for you: something you might explore is LVM.

LVM is primarily an alternative form partitioning. However technically LVM's physical volumes can be any block device. It provides logical block devices ultimately backed by physical ones.

Since LVM logical volumes are block devices they can usually be used for swap.

LVM has a feature called VDO which provides compression:

The Virtual Data Optimizer (VDO) feature provides inline block-level deduplication, compression, and thin provisioning for storage. You can manage VDO as a type of Logical Volume Manager (LVM) Logical Volumes (LVs), similar to LVM thin-provisioned volumes.

Trouble you might run into

Any form of compression requires some memory. My main concern with any solution that was not designed to work with swap is that it may dynamically request memory from the kernel to compress pages. Since it would be compressing because the kernel needed to free up RAM, requesting RAM would be prone to cause failure.

It is of course possible that drivers are written to be aware of this and primitively request all the RAM they might need. The point is this might be a problem if the drivers were not written with swap in mind.

Improve this answer
2
  • Okay now we're moving somewhere. This worked better than zfs zvol. I did indeed run into a system deadlock immediately after running swapon directly on it, but this thing works surprisingly well, if I plug it into zram backing_dev. The writing speed is about 10 times slower though, but it's at least somewhat usable. Curiously, abstracting it with losetup and THEN plugging as a backing_dev seems to improve writing speed almost twice
    – linuxlover69
    Mar 23 at 4:07
  • Weird, I tried the reverse, making a vdo volume using zram, plugging regular swap as backing_dev and the writes to vdo volume were faster, but still incredibly slow, at least 10 times slower than regular zram. I don't get it, it's ram-to-ram, should be blazing fast despite double compression. The writes to backing_dev ( aka reads from zram ) were also noticeably slower, despite the data staying compressed due to vdo. Is it just the vdo overhead? In any case, this is a slow, but genuinely working solution, I'll wait in case anyone has a better idea, and will accept this if not. Thank you =)
    – linuxlover69
    Mar 23 at 16:00

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .