Review request for 5049299

[Michael McMahon]

Martin Buchholz wrote:
> Below is a java program and a shell extract demonstrating the problem
> on a Ubuntu-hardy-like system.  The fork() fails from within a large
> Java process (but normal commands started from an
> independent shell continue to work fine).
>
> Linux overcommit heuristics are hard to understand.
>
> The failing fork problem becomes more serious on a server system
> running only one significant large program, which is using most of
> the memory on the system, perhaps by sysctl'ing
> the vm.overcommit_ratio to a value higher than the default 50%.
>
> On systems with two large processes using most of memory,
> you may see rare failures when both try to fork() at the same time..
>
I played around with your test and I see similar behaviour on my Ubuntu 
8.04 system.
When you switch on "always overcommit" (not the default) then you get 
the expected
behaviour, ie. you can go quite close to the VM limit, and manage to 
spawn a small command
from Java.

However, in the default "heuristic" mode, it doesn't work. The comment 
on the web page
you mentioned says:

"Heuristic overcommit handling. Obvious overcommits of
address space are refused. Used for a typical system. It
ensures a seriously wild allocation fails while allowing
overcommit to reduce swap usage:

Perhaps, this specifically does not suit the Java VM situation where 
fork() of a very
large process, is considered an "obvious overcommit".

In that context, I'll look into using the clone() system call on Linux. 
It seems
to be quite versatile and configurable. It looks likely that we can 
avoid the use
of the helper command with it as well.

- Michael.