no-administration heap?

[Peter B. Kessler]

Colin Walters wrote:

> On Thu, Feb 28, 2008 at 4:20 PM, Florian Weimer <fw at deneb.enyo.de> wrote:
>>  At least on Linux, the problem is that there is no documented API to
>>  reserve address space (so that no other mappings end up there) without
>>  actually allocating backing store.  MAP_NORESERVE does not work as
>>  documented.
> 
> Ah, I see.  So this problem is essentially blocked on the lack of a
> sufficient/documented Linux kernel interface?
> Do other operating systems (Windows, Solaris) provide a good enough
> interface for OpenJDK?
> 
> I'm assuming changing the collector to not require a continuous
> address space would be too invasive?


Nothing is "too invasive"!  We have a sketch of a "chunked heap";
it's just a lot of work.  Things like the card table that used to
be indexed by offsets in the heap become part of each heap chunk
and are indexed by an offset in the chunk.  A lot of the underlying
data structures have to be rewritten, but that's why they call it
software.  We would *love* to have that kind of heap.

Then there are "details" like that once you chunk the heap you
limit the maximum size of an object.  I was thinking that 64MB
chunks might be one possibility in that they would let you have
(somewhat less than) a 64MB array, which is the largest you can
have now on the client unless you use a larger -Xmx.  Few people
have individual objects larger than 64MB, but there are a few.
Of course you'd want to allow people to specify a larger chunk
size on the command line.

An interesting observation (by Andrey) is that chunking doesn't
make sense in a 64-bit JVM: there you just reserve enough contiguous
virtual memory (but not swap space) for the amount of physical
memory you have, and grow into it as you need it.  That works
because the virtual address space you have is guaranteed to be
larger than the physical memory you have, so you don't have to
worry about shutting out thread stacks, space for code, etc.
There are details in there too: as in that the JVM should not
expand the heap if it the total process footprint would exceed
the physical memory, but that's just a matter of getting the
policy right.

			... peter