It is due to OpenVZ. You can see the limits applied in
/proc/user_beancounters, and there is some explanation here: http://wiki.openvz.org/Privvmpages
Although I have not had memory problems in a container, I think the suggestion here:
to start by setting the stack size with
ulimit is a good one. Just note that the implication there that OpenVZ uses the same metric as you find in top's VIRT column is wrong; the "privvmpages" is I believe virtual pages marked writable and private, thus significantly less than the entire address space of a process. There is some other misinformation in there too ("Unfortunately running out of memory is an error condition that most programs fail to handle properly." -- on linux, running out of memory is by default not an error condition, so programs do not get a chance to handle it properly). But setting the stack size down is probably still worth trying.
Slm's suggestion seems like a good solution to finding the fat cats, but if your slice is like mine, there is no
/proc/bc to work with. You can, however, get the writable and private stat for a process via
pmap -d, or look at numbers in top.
1 gig might seem like enough to run a JVM, but there is a complication: since the amount of memory used and available is not a simple figure (see some discussion of why here), the total may be more than the machine can actually provide at a given point in time.