I would like to setup a system with limited memory (raspberry pi) in such a way that application can't cause the desktop (x11, LXDE, openbox, mouse driver, video card driver) to be unresponsive.
When I open too many browser tab the browser use too much memory and cause Linux to swap pages to disk. Some of the pages can belong to the desktop ... which result in the keyboard or mouse input to lag significantly making it impossible to open task-manager and decide which application to kill.
If after a fresh boot, the Desktop need 200MB of memory and the machine have a total of 1GB of memory.
Is it possible to put all application started after boot in CGROUP limited to 800MB of memory such that when any memory hungry process in this CGROUP exceed 800MB it would not evict any memory pages needed for the desktop to stay responsive?
I dont want to disable SWAP completely, I think it's better for memory Hungry application to swap and become slow instead of getting killed by oom-killer. As long as the desktop itself does not become slow.