What can make Linux unresponsive for minutes when browsing certain websites?

Question

I'm using Linux 4.15, and this happens to me many times when the RAM usage reaches its top - The whole OS becomes unresponsive, frozen and useless. The only thing I see it to be working is the disk (main system partition), which is massively in use.

I don't know whether this issue is OS-specific, hardware-specific, or configuration-specific.

Any ideas?

Answer 1

What can make Linux so unresponsive?

Overcommitting available RAM, which causes a large amount of swapping, can definitely do this. Remember that random access I/O on your mechanical HDD requires moving a read/write head, which can only do around 100 seeks per second.

It's usual for Linux to go totally out to lunch, if you overcommit RAM "too much". I also have a spinny disk and 8GB RAM. I have had problems with a couple of pieces of software with memory leaks. I.e. their memory usage keeps growing over time and never shrinks, so the only way to control it would have been to stop the software and then restart it. Based on the experiences I had during this, I am not very surprised to hear delays over ten minutes, if you are generating 3GB+ of swap.

You won't necessarily see this in all cases where you have more than 3GB of swap. Theory says the key concept is thrashing. On the other hand, if you are trying to switch between two different working sets, and it requires swapping 3GB in and out, at 100MB/s it will take at least 60 seconds even if the I/O pattern can be perfectly optimized. In practice, the I/O pattern will be far from optimal.

After the difficulty I had with this, I reformatted my swap space to 2GB (several times smaller than before), so the system would not be able to swap as deeply. You can do this even without messing around resizing the partition, because mkswap takes an optional size parameter.

The rough balance is between running out of memory and having processes get killed, and having the system hang for so long that you give up and reboot anyway. I don't know if a 4GB swap partition is too large; it might depend what you're doing. The important thing is to watch out for when the disk starts churning, check your memory usage, and respond accordingly.

Checking memory usage of multi-process applications is difficult. To see memory usage per-process without double-counting shared memory, you can use sudo atop -R, press M and m, and look in the PSIZE column. You can also use smem. smem -t -P firefox will show PSS of all your firefox processes, followed by a line with total PSS. This is the correct approach to measure total memory usage of Firefox or Chrome based browsers. (Though there are also browser-specific features for showing memory usage, which will show individual tabs).

Answer 2

AFAIK, bloatware shouldn't make the OS unresponsive, so I wouldn't consider or even accept that the bloatware is the root cause of the problem

You're not going to like this, but I think bloatware is your problem (although I'm not sure if it's memory or disk which is the problem). Unfortunately, the Linux kernel is awful at handling high memory pressure situations, and is known to basically require a reboot once memory is exhausted. There are three things which lead me to believe your issue is resource exhaustion:

1. Your disk space on root (/) and DATA is almost full. I'm not sure what you use DATA for, but I've ran into issues before with resizing my root partition too small and my system becoming inoperable.
2. You have high-memory pressure, meaning that your RAM is almost full. When RAM starts to get full you will start to get page faults. Page faults happen when the kernel is unable to allocate enough memory for a process and must instead use some of the systems much slower swap space. This leads us to our last observation:
3. Your swap space is almost full. There's clearly some high memory pressure on your system since both RAM and swap are almost full.

Basically, put these three together and your system doesn't have enough resources available to do much of anything. As for it's unfortunate how poorly Linux handles low-memory situations (compared to, say, the NT kernel in Windows) but that seems to be how it is. You can find more discussion in this Reddit thread and its linked mailing list.

As for how to fix your situation, I would say increasing your swap size is a good idea, but since you're low on disk space that will be a problem. Unless your Minecraft server has a ton of people, I think it would be safe to reduce its memory to something around 1024m (I personally use 1024m with about 10 people and it works fine). I would also use spigot or paper for your Minecraft server since they tend to be more performant.

Good luck!

Answer 3

What's the output of free -m? The amount of RAM you have is pointless if we don't know how much you're using. That and I'm interested to know how much swap space is being used.

I do think you've answered your own question, though. Having open "many tabs" open in your browser can definitely slow down your system if you're never closing them, as they'll continue to consume memory regardless; when your system freezes, how many do you have open at a time?

It also makes sense if your system is freezing up from other memory-intensive tasks such as "generating a very large graph from a very complex UML diagram". That will absolutely slow down your system as it generates the graph, so that's hardly a surprise.

It really sounds like this is the way your system is supposed to behave. Either that or I'm missing something here.

By the way, HDD stats don't matter when it comes to your system becoming unresponsive since a lack of memory is almost always the culprit.

Answer 4

When I read the title my immediate thought was "not enough RAM", because I have experienced exactly this problem myself on Linux, 10+ minutes of frantic disk thrashing after opening too many browser tabs. I agree, it's dismal, and needs improvement. Windows handles this situation much better.

Some suggestions:

• Add a memory monitor applet to your system tray so you can keep an eye on it.
• In Firefox's preferences, set the "content process limit" to "1". As the text below the setting says: "Additional content processes can improve performance when using multiple tabs, but will also use more memory."
• Remove or replace any memory-hungry browser addons. Keep your ad blocker, as ads eat more memory than any blocker.
• Investigate and possibly remove any other memory-hungry programs.

However, the only true solution is to buy more RAM.

Not only will an abundance of RAM prevent this catastrophe from occurring, but it will allow the system to build up a large file cache in RAM, which your system currently can't ever do because it runs so close to the limit. A large file cache will take work away from the HDD and make almost every action on the system feel faster generally. It's worth it.

Answer 5

Your htop output shows that your need of RAM is higher than its capacity (total RAM+SWAP). So the obvious first consideration to make is to reduce RAM usage or increase RAM availability.

Note that modern-day firefox versions are extremely resource-hungry, due to the way that windows/tabs are given process and memory space. The idea was to avoid crashing tabs bringing the whole browser to it's knees. Is it worth the price? Who can tell... Anyway, I've had a similar problem due to the above, since my Pentium 4 mainboard only supports 2GB of RAM. To avoid possible memory exhausted crashes, I added ~800M swap space on a spare SSD, obviously with the intention to use it as little as possible. I have achieved that by changing a setting known as the swappiness, which determines how eager the kernel is to swap memory pages out. Some useful commands as follows.

Check the current swappiness: cat /proc/sys/vm/swappiness

This may well give you a result around 60, which is quite high for maximum performance on systems with a lower load. For you, obviously this works counter-productive, so you can change the setting using a command such as sysctl vm.swappiness=1 to change the setting while the system is running.

To save these changes, you'll have to look for the file /etc/sysctl.conf. In that file, change the value or add the line vm.swappiness=1.

Mind, this is not a solution in your case, but should make a usable workaround.

Credits https://askubuntu.com/questions/103915/how-do-i-configure-swappiness

source for the answer above, includes further explanation. I found that post very helpful in my case.

Answer 6

Some excellent discussion of how the problem is caused, continues and grows. I like to get ahead of problems such as you experience by throwing hardware at the initial computer’s design, and/or upgrading an existing implementation. Can you,

• add RAM (32GB works great for many setups)

• replace your hard disk drive with an SSD

• add an SSD (Solid State Drive) for swap drive

• create a swap partition in RAM (with 32 or more GB of RAM)

• get a faster HDD

• move to a system with faster processing and wider / faster bus architecture.

Some of these hardware upgrades/replacements can be well under $100US. These are not specificto Linux, nor your exact software implementations, but the hardware you are using does not seem adequate to your tasks.

Answer 7

Usually it's "just" X11 that becomes unusable. To get a keystroke from your keyboard to a program, and have it show anything on screen, code in several different processes has to run. (X server to get the keystroke from the kernel, xterm or equivalent to get the event and decide to draw something, then send a message to the X server to draw a glyph from a font.)

Just waving your mouse around over a window with a web browser showing a page with a bunch of Javascript crap can result in a bunch of messages for a bunch of processes, all of which cause those processes to wake up and touch a bunch of data. Presumably including a bunch of "cached" uncompressed bitmaps. So this is highly likely to evict more things that are soon needed.

ctrl+alt+F2 to switch to another virtual console usually makes it possible to log in and run shell commands with only a couple seconds latency when something is causing swap thrashing. It's just bash; the Linux kernel isn't swappable and it has all the VT and
keyboard<->TTY code.

It may take some time before this keypress does anything, if swap is really thrashing badly. The X server, not the kernel, has to receive that keystroke and switch away from its own VT after putting the video hardware back into the right mode. The kernel is still looking for Alt+SysRq key combos, but nothing else. But once you do get switched to a text console, there's usually at most 1 user-space process between typing and characters showing on screen.

If your X server actually locked up, then ctrl+alt+F2 may not work at all.

---

To avoid slowdown when you're not truly thrashing, reducing "swappiness" can help. e.g. I set the /proc/sys/vm/swappiness tunable to 6 on my desktop with 16GB of RAM and a 2GB swap partition on an NVMe SSD. You can read more about tuning for interactive latency (as opposed to server throughput); any guide will mention that tunable.

But if you have any swap at all, Linux will use it before invoking the OOM killer. Keep your swap partition small, just big enough for Linux to page out really stale crap that typically really doesn't get used for a long time. (e.g. memory leaks!)

I haven't had any problems with swap being full. Modern Linux deals with having limited swap space just fine. Chromium (which I use instead of firefox) does sometimes get slowish with dozens of Stack Overflow tabs open, but The Great Suspender is a nice addon for unloading tabs when you're not using them. I think that saves significant RAM for me, although it will only unload tabs where you haven't typed anything in a textbox. It might also be available for Firefox.

---

As other have suggested, 16GB of RAM is really nice for interactive use with Linux. DRAM prices are relatively low currently; after spiking about 1.5 years ago, they've mostly declined again.

Answer 8

What can make Linux unresponsive for minutes when browsing certain websites?

You are not using Linux right. Which becomes especially noticeable on a resource limited machine. You don’t need more RAM, nor a faster processor.

Background:

Almost every non-user program’s priority is 0.
Almost every user program’s priority is 20.

To ‘fix’ your issue:

Leave the non-user programs alone, but start changing the priorities (nice levels) of your user programs so they don’t cause you issues. Edit what launches your programs to include nice levels, going from usually not a problem, to the worst offender.

Real World Examples:

KMail:          nice -n 1 kmail -caption "%c" %i %m
LibreOffice:    nice -n 2 libreoffice --writer %U
Firefox:        nice -n 3 firefox %u
WorstOffender:  nice -n 9 {i'm a bad program}

Your WorstOffender will still become unresponsive for minutes, that’s literally a go buy a better box problem, but it now won’t be causing your entire OS (Linux) and everything else you have running to also become unresponsive.