I have a system running many processes on behalf of other users. Virtual memory space with 64bit operating systems is less of a concern, but my users will still often use up all the physical memory causing performance degradation from swapping, normally just a couple of users with CPU intensive programs with many GB's of memory causing everyone elses stuff to get paged out.
Also I have found a lot of programs will allocate a GB or 2, but then I only see a few hundred MB active most of the time, so having to "allocate" the full 2GB rather than just the 500MB actively required to avoid swapping seems excessive.
What I want therefore is to reserve a certain amount of physical memory per either an entire user (preferred) or process say to 1GB, so that a users program will never get swapped out while below that, but still let them go upto a few GB total, with the "overflow" getting swapped as needed.
Currently running Debian systems with seperate user accounts, but could change this (e.g. an extra "layer" of some sorts) as long as it still appears like Debian (or maybe other Linux at a stretch) to the users.