Desktop completely freezing on I/O operations

Question

In my old system, I was already having desktop freezes on some heavy I/O operations like apt-get dist-upgrade and copying 4 GB video files. But now, it has become a lot worse: any apt-get install is enough the freeze the whole desktop, and copying the videos with nice -c 3 does not help at all, I cannot do any operation for minutes. According to man ionice, the impact of an idle I/O process on normal system activity should be zero.

One weird swap problem appeared as well: I had disabled the swap partition, and twice an OOM was triggered. But instead of killing the process that trigerred the OOM (I set vm.oom_kill_allocating_task=1), the disk started spinning and totally froze the desktop for a long time, although I had run sudo swapoff -a and checked whether it worked.

The old hardware and system:

• Hard drive: 500 GB Toshiba
• System: Debian 7.9 (32 bits)
• Kernel: 3.15.10-zen-686 (Liquorix)
• Disk scheduler: BFQ
• Filesystem: ext4

The new hardware and system:

• Hard drive: 1 TB Samsung
• System: Ubuntu 15.10 (64 bits)
• Kernel: 4.2.0-30-generic
• Disk scheduler: Deadline (default)
• Filesystem: encrypted LVM with Btrfs

I have found many people who have solved similar problems by changing from the CFQ scheduler to Deadline, but Deadline is the default on the Ubuntu kernel. And I already had problems with the supposedly fast BFQ disk scheduler with no encryption.

Answer 1

Apparently, the biggest issue was a swap problem. In my old system, setting vm.swappiness=100 helped me avoiding system freezes by moving pages before the memory was exhausted. On this new system, it seems to have had the opposite effect: the system was always constantly and needlessly swapping. I have purchased more RAM and totally disabled swap, and the problem was mostly solved.

Regarding the fact that I/O operations caused desktop freezes, it seems to be because the data from the operation was being cached on RAM, and the memory used by desktop applications being thrown into the swap, which caused the system to read back from disk whenever I tried interacting with the desktop.

With enough RAM, I/O operations have mostly no effect on desktop responsiveness, but if the desktop is left unattended for some time, there may be an initial lag while interacting with the desktop. This may be due to the kernel deciding that the system is idle enough to give a higher priority to the operation, and its data may end up filling the cache completely, dropping the pages from desktop applications.

Regarding the disk scheduler, I have not done enough testing to know which would be the best option, but from my research, it seems that either deadline completely ignores ionice, or that ionice is specific to CFQ.