I have a real-time networking application that is extremely sensitive to any kind of "blocking" and needs to be extremely responsive most of the time with perfect latency.

The app (multithreaded) itself is unfortunately quite memory hungry, it needs about 6GB of RAM.

The system had 8GB RAM with 2GB swap. I observed some performance issues with increased app latency that I connected to swapping events (OS is monitored using grafana and node_exporter every 15 seconds), exactly in same time as bad latency happened, the OS was swapping out some pages (while still having 2GB available memory).

I understand Linux kernel VM subsystem fairly well (I read Mel Gorman's "Understanding the Linux® Virtual Memory Manager" book) so I am associating this with the OS proactively swapping out inactive pages to get more space for FS cache / buffers, which I wouldn't really mind, if it wasn't crippling the latency of this application.

I set vm.swappiness = 0 in order to get kernel to use swap only as last resort when low mem watermark is reached, but to no avail - it keeps swapping out tiny chunks of memory approx. every 30 minutes, crippling the latency of application.

Only thing that helped was disabling swap entirely - ever since I did that latency issues never reoccurred. But that is also a bit of problem, because if some unfortunate situation happened when app would temporarily need more memory approaching the 8GB of total usage, OOM killer would now kill it, instead of swapping out portion of inactive pages, which would be even worse scenario than crippled latency.

So I have 2 questions:

  • Why are these small swapout events crippling the latency of application? My understanding is that swapper is evicting inactive memory pages anyway - is the process frozen when swapper moves its memory, even if it doesn't try to access the pages that aren't in RAM or that are being moved?
  • Is there any way to enforce kernel to swap ONLY when necessary? Eg. - don't swap when > 1GB of RAM is still available?

2 Answers 2


Can't answer as to what the kernel actually does, unfortunately. Some things that might help your specific problem though:

  • Get more RAM
  • Use less RAM, optimize
  • Swap on ZRAM instead of swapping to disk, supposedly this can be ~about an order of magnitude faster. Basically when you swap out a page it just compresses it in ram instead of writing it to disk - even with the overhead of the compression algorithm, this can be a lot faster.
  • If you aren't already using it, try using the RT-linux kernel patch set. The whole point of the whole "realtime linux" project is to run applications where missing your latency target means somebody dies, so I'd assume you won't see this kind of catastrophic latency problem there. At least I hope you wouldn't, that would be Very Bad. Most distros have rt kernels available afaik.
  • The LE9 kernel patches also may be vaguely relevant, specifically they work with swap behavior in out of memory situations. The linux kernel considers pages with your executable code/libraries to be prime swapping material, it will swap those out first. Then your program goes to run that code, hits a page fault, freezes while it swaps the page in, and goes on. As I understand, LE9 patches protect your executable pages from swapping so eagerly to prevent thrashing those pages when you run out of memory, so that the system handles it gracefully instead of freezing.

While I don't know what happens when swapping out, your process is most definitely stopped whenever swapping in, that's how page faults are handled. You try to access some memory, that page isn't loaded, the kernel stops everything, loads that page, then starts your program again hopefully none the wiser that the memory wasn't there when you asked for it. I assume if something gets swapped out, it's probably also going to get swapped back in eventually, and then it will definitely stop your process. That may be what's happening.


The app (multithreaded) itself is unfortunately quite memory hungry, it needs about 6GB of RAM.
The system had 8GB RAM

Your system is short in memory; I think the only solution to your problem would be to add some RAM.

Keep in mind that pages swapped out of memory are written to disk, which offers much slower read/write access than RAM. This must be why your application is slowing down.

Linux has been observed to swap even when swappiness is set to 0. Related question: Does Linux perform "opportunistic swapping", or is it a myth?

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .