changing the memory settings semi-dynamically vs GateKeeper and signing

[Andrew Thompson]

On Feb 22, 2012, at 4:45 PM, Jon Masamitsu wrote:


Thanks for the response Jon. I realize this isn't any easy problem to solve because of the huge variations in kinds of programs out there, not to mention reasonable people differ on what is desirable behavior.
Still, I feel the settings we have for the VM are heavily slanted towards server deployment where total amount of RAM on the host is a known quantity, and the total number of apps is usually fixed. 

Contrast the client where:

- every PC or Mac may have a different amount of RAM
- users may be starting up and shutting down programs constantly
- users expect to be able to open huge files, limited only by the amount of RAM they bought.

With that model, guessing what to put in -Xmx is borderline impossible.

More comments inline below

> 
> On 2/21/2012 9:49 PM, Srinivas Ramakrishna wrote:
>> Good points... It's a good idea to cross- post this over to hotspot-gc-dev since it's relevant to "client ergonomics" or current lack thereof....
>> 
>> Sent from my iPhone
>> 
>> On Feb 21, 2012, at 20:54, Andrew Thompson<lordpixel+openjdk at mac.com>  wrote:
>> 
>>> ...
>>> But why should this be the behavior? Apps written in Objective C don't self limit, so why not make Xmx default to physical RAM size?
> 
> Hotspot reserves the address space for the heap at initialization.  Having it automatically
> grab 1gb seems unfriendly to other users of address space within the process and may
> exclude other process from starting up (depending on how over subscription of memory
> is handled).

Is that really a concern on a 64bit VM? I know it is a huge concern on 32bit (e.g. I have never gotten WIndows XP to go beyond a 1GB Xmx reliably on 32 bit) but for 64 bit the address space is at least several GB, right? 
It seems to me that the design of G1 would one day allow discontiguous heaps where the memory need not be reserved at startup as a single piece, but isn't that already irrelevant on 64bit?


> A smaller  limit on the size of the heap also limits the pause time an application will see.
> It can also limits throughput but I believe there are many users who prefer the low pause
> more often rather than the larger pauses less often.

I can more or less control this today. I can do something like

-Xmx = 128m
-XX:GCTimeRatio=19, 
-XX:MaxGCPauseMillis=1000

Then the parallel collector will respond by not growing the heap as much... it prefers to collect over grow on more occasions.
So it is possible to keep some of the benefits of a smaller heap.

But the real issue is these 2 flags do nothing to the parallel collector, which creates the highwatermark effect

-XX:MaxHeapFreeRatio=YY
-XX:MinHeapFreeRatio=ZZ

Since mx is small the actual used amount of memory (not address space) starts low, can go up, but can't go down again unless one is running serlal gc.
This is what I think we are missing in the desktops apps. mx is hard to predict so rather than do what why not let Hotspot grab a block of address space for a potentially large heap and let it (the heap's data) grow and shrink as needed.

> Regarding shrinking the heap and giving back the memory, yes, the hotspot should do
> that.
\
So was there a fundamental flaw with last year's patch? It caused more gcs but was there a showstopper problem with it or did the developer just lose interest?