Review request for 5049299

[Florian Weimer]

* Andrew Haley:

> But those running Linux won't benefit from such a change because
> on Linux there is no transient doubling of process size: all that happens
> is that the page table entries in the new process are mapped copy on write.
> The extra pages count towards the overcommit limit, but that's wholly
> artifical.

On Linux in vm.overcommit_memory=2 mode, the whole heap (not just the
committed part) counts against the system's total memory allocation
limit.  After a fork, the heap counts twice.  Copy-on-write is just an
optimization in this mode, it does not change the physical memory
requirements of the workload (in vm.overcommit_memory=1 mode, it
does).

I guess the issue might even be worse on Linux than on Solaris because
the heap initialization code in Hotspot is wrong.  It uses
MAP_NORESERVE, which means that the VM will die with SIGSEGV when it
tries to write to a page for which the kernel cannot provide backing
storage.  This SIGSEGV is not turned into an OutOfMemoryError, leading
to hard-to-diagnose bug reports.  (It's not an OOM situation from the
kernel's point of view, so you don't get a trace from the OOM killer
in the system log, either.)

A better way seems to be to allocate the heap with PROT_NONE, and
later use mprotect with PROT_READ|PROT_WRITE (and perhaps PROT_EXEC)
to allocate chunks from the kernel.  This will fail deterministically
in the garbage collector if no physical memory is available.  The
PROT_NONE mapping is only there to reserve a continuous chunk of
address space (so that calls to malloc or dlopen do not create
mappings in the middle of the Java heap).  When I tested this some
time ago, a PROT_NONE mapping did not count towards the system's
memory allocation limit, hence the potential failure in the mprotect
call.  The main problem with this approach is that this is not a
documented way of using the kernel API; it might work accidentally now
and change behavior in the future.

Sorry for hijacking this thread, but I thought this clarification was
helpful.

-- 
Florian Weimer                <fweimer at bfk.de>
BFK edv-consulting GmbH       http://www.bfk.de/
Kriegsstraße 100              tel: +49-721-96201-1
D-76133 Karlsruhe             fax: +49-721-96201-99