This is an old problem that has been bugging me for quite some time.
Based on my experience with Linux, its OOM killer is BROKEN. It does NOT work.
Chromium browser, it doesn't kill. Other processes, it doesn't kill. This morning the process Tumblerd almost crashed my system using 2.1 GB. Even after 3 minutes wait on a painfully scary frozen system (it survived thanks to my keyboard shortcut below) Linux had NOT killed the process using 2.1 GB, which further proves that it does not work.
I'm not here to ask anyone if it works or not, so save your typing for another person. The usual bla bla bla... (how rude)
The only way I was able to workaround this issue relies on me watching my available memory with free -h all the time, otherwise I might miss the short threshold of time and the whole system becomes frozen, leading to a hard reset.
I had set a keyboard shortcut on XFCE for a script that will kill all my Chromium processes, this was sufficient enough for my particular use case. But it would be much nicer if I could find a way to "kill process which is using most memory" (which I'm currently searching at this moment due to the aforementioned freeze earlier this morning)
First, test what process you want to kill using this command:
ps aux | grep STRING
ps aux | grep PORTION_OF_NAME_WITHOUT_QUOTES_OR_SPACES
If it shows the process you want to kill, then put it in the script below inside the parenthesis.
kill -9 $(ps aux | grep type=renderer)
# the above will kill all my Chromium processes
# without having to restart my entire browser or losing progress
# incognito window will be kept alive, instead of lost
sudo killall tumblerd
Paste that into an empty text file and save it as SOMETHING.SH
Then make it executable by running chmod a+x /path/to/SOMETHING.SH
and assign it to a keyboard shortcut (it's possible in XFCE)
For the tumblerd process, the best way to avoid problems is removing it from the system, since it's only responsible for generating video thumbnails.
sudo apt remove tumbler
So for anyone that's reading this, if you have the suspicion that Linux' OOM killer is broken, you're not wrong, don't let others guilt you in thinking that you are.