-XX:+MaximizeHeapSize?

[Jon Masamitsu]

ThreadStackSize sets the stack size for limit for each thread.
The difficulty I see is that we don't know how many threads 
an application uses.  On 32bit if I reserved most of the 4g
address space for the heap (but had not used it all yet)
and then could not allocate a stack for a new thread, I would 
have to find a way to return some of the address space for the
thread stack.  I would likely have to do a full collection to
move all the objects to the bottom of the heap so that I could
free space at the top of the heap.  This is conceivable with the
UseParallelGC collector because of the layout of the generations
in the heap (perm gen at the low end, then the tenured gen
and then the young gen at the high end).  We'd have to try it to
find out where the pitfalls are.



Ted Neward wrote:

>Perhaps this flag would be used in conjunction with another, thread-specific
>allocation flag? A la
>
>Java -XX:+MaximumHeapSize -XX:+ThreadStackSize=256m ....
>
>Or, just use -ss (which IIRC is stack size, correct)?
>
>Ted Neward
>Java, .NET, XML Services
>Consulting, Teaching, Speaking, Writing
>http://www.tedneward.com
> 
>
>  
>
>>-----Original Message-----
>>From: hotspot-gc-dev-bounces at openjdk.java.net [mailto:hotspot-gc-dev-
>>bounces at openjdk.java.net] On Behalf Of Peter B. Kessler
>>Sent: Wednesday, July 11, 2007 10:46 AM
>>To: Jon Masamitsu
>>Cc: hotspot-gc-dev at openjdk.java.net
>>Subject: Re: -XX:+MaximizeHeapSize?
>>
>>Jon Masamitsu wrote:
>>
>>    
>>
>>>We've heard from users in the past that it would be nice if
>>>there was no maximum heap size limit.  Users say
>>>they don't know what the limit for their application is and
>>>have to experiment to find an adequate value.
>>>
>>>I did the following refworkload experiments.
>>>
>>>Ran "reference_client" and "startup2" (sets of client
>>>application like benchmarks) with the -client
>>>(uses the serial GC which does not have GC ergonomics) and
>>>a 1g heap.   The point of this experiment was to see if the
>>>size of the committed heap increased because of the 1g heap
>>>size limit.  There was no discernible difference in the
>>>committed size of the heap nor in the amount of GC work.
>>>
>>>I ran "reference_server" (set of server application like
>>>benchmarks)  with -server and the parallel GC (does
>>>has GC ergonomics) with -XX:DefaultMaxRAM=1g (the default)
>>>and with -XX:DefaultMaxRAM=1700m (the amount of physical
>>>memory on the platform minus some for
>>>the OS) and -XX:DefaultMaxRAMFraction=1. The point of this experiment
>>>was to see if the
>>>GC ergonomics drove the  heap to sizes larger that the 1g
>>>default limit.  As one would expect the heaps for some
>>>benchmarks grew considerably (up to the higher limits
>>>in some cases) and for some the heaps did not change.
>>>
>>>I think that these results are not unexpected.  The heap sizing
>>>policy used by the serial collector depends on the amount of
>>>live data and the live data for the client like applications
>>>fit nicely into the 64m default heap size (i.e., the larger
>>>1g heap was not needed).  On the larger server like benchmarks run
>>>with GC ergonomics, the high default throughput goal of GC
>>>ergonomics means that some benchmarks will just use as much
>>>heap as they can get in trying to reach the throughput goal.
>>>
>>>I'd like to propose an additional policy under a new command line
>>>flag.  Let me use MaximizeHeapSize for the name for now.  If
>>>MaximizeHeapSize is set, the VM will calculate
>>>the amount of physical memory on the platform and
>>>try to use that as the upper limit on the heap size.   As
>>>with the current GC ergonomics the upper limit will
>>>be ~4g on a 32bit system.   If the VM cannot actually
>>>reserve that amount of space at initialization, it will
>>>try reserving some smaller amounts until it succeeds.  There
>>>will be some minimum size (probably 64m) that it will not
>>>go below.
>>>
>>>The user would have to turn MaximizeHeapSize on
>>>explicitly.  It will be off by default until we get some
>>>experience with it.  Users who turn it on and also use
>>>GC  ergonomics will manage the heap size via
>>>a smaller or larger throughput goal (i.e., the larger
>>>the throughput goal the more heap space the VM will
>>>try to get to meet the throughput goal).
>>>For the non ergonomics collectors it should only matter
>>>if the application would otherwise have bumped into a
>>>heap limit (i.e., the application would have thrown
>>>an out-of-memory or just spent too much time doing
>>>garbage collections because the heap limit set by
>>>default or on the command line was too small).  In that
>>>case this change would allow the application to
>>>get the additional space it needs.  I expect there will
>>>be some corner cases or bugs where the heap will grow  to
>>>the limit when it shouldn't.
>>>Comments on this proposal?
>>>      
>>>
>>Don't forget to leave some address space for thread stacks,
>>other libraries, malloc space, etc.  A simple backoff policy
>>from trying to reserve all of memory might be too simplistic.
>>I don't know how to estimate beforehand how many thread stacks
>>an app will need. :-)
>>
>>			... peter
>>
>>No virus found in this incoming message.
>>Checked by AVG Free Edition.
>>Version: 7.5.476 / Virus Database: 269.10.4/896 - Release Date: 7/11/2007
>>4:09 PM
>>
>>    
>>
>
>No virus found in this outgoing message.
>Checked by AVG Free Edition. 
>Version: 7.5.476 / Virus Database: 269.10.17/915 - Release Date: 7/24/2007
>1:50 PM
> 
>
>  
>