Re: RFR: 8236073: G1: Use SoftMaxHeapSize to guide GC heuristics

[Liang Mao]

Hi Stefan,

Thank you for your comments!

Based on previous discussion, the reasons are as below:
1) For the expansion after cm, I think we have the agreement that
original MinHeapFreeRatio might be too large and predicting the necessary
size from adaptive IHOP for expansion sounds reasonable and specjbb2015
have the good result.
2) About when to shrink the heap, I think it's a better spot after
 mixed collections. From my observation, the heap use is still at nearly
peak after remark for most of cases like Alibaba workloads and specjbb2015.
There could be some senario which contains a lot of humongous regions that
remark will cleanup considerable regions. But why don't we decide to shrink
the heap size when most of garbages have been cleaned after mixed GCs. We 
don't need to shrink twice in an old gc cycle. The MaxHeapFreeRatio 70 to 
keep heap capacity with 30% live objects make sence and is unified with full
 gc logic. If we only shrink the heap at remark, the maximum desired capacity
could be 100/30 times of peak heap usage which is obviously not efficient. 

Thanks,
Liang





------------------------------------------------------------------
From:Stefan Johansson <stefan.johansson at oracle.com>
Send Time:2020 Feb. 17 (Mon.) 18:10
To:"MAO, Liang" <maoliang.ml at alibaba-inc.com>
Cc:hotspot-gc-dev <hotspot-gc-dev-bounces at openjdk.java.net>; hotspot-gc-dev <hotspot-gc-dev at openjdk.java.net>
Subject:Re: RFR: 8236073: G1: Use SoftMaxHeapSize to guide GC heuristics

Hi Liang, 

I’ve started looking at this patch as well and I have a question regarding the change to not allow shrinking after concurrent mark? Before we could shrink the heap at Remark, but now we only check to expand the heap after the concurrent cycle, why is that? I get that we will be able to shrink even more after the mixed collections but if a lot of regions are freed by the concurrent cycle why not check if we can shrink here?

Also good to hear you can run SPECjvm2008, we also avoid running any problematic benchmarks. 

Thanks,
Stefan

> 17 feb. 2020 kl. 10:56 skrev Liang Mao <maoliang.ml at alibaba-inc.com>:
> 
> Hi Thomas,
> 
> I am able to run specjvm2008 by excluding the compiler subtests
> and reproduce the issue that the change commits more memory.
> The main cause is addressed that the tests have a lot of
> humongous objects which affect the evaluation of adaptive 
> IHOP. _last_unrestrained_young_size and _last_allocated_bytes
> used in G1AdaptiveIHOPControl::predict_unstrained_buffer_size are
> very large. So the expansion after concurrent mark is rather
> aggressive. I made an enhancement to restrict this uncommon
> expansion with MinHeapFreeRatio:
> http://cr.openjdk.java.net/~luchsh/8236073.webrev.4/
> 
> Thanks,
> Liang
> 
> 
> 
> 
> 
> 
> ------------------------------------------------------------------
> From:Thomas Schatzl <thomas.schatzl at oracle.com>
> Send Time:2020 Feb. 15 (Sat.) 03:51
> To:hotspot-gc-dev <hotspot-gc-dev at openjdk.java.net>
> Subject:Re: RFR: 8236073: G1: Use SoftMaxHeapSize to guide GC heuristics
> 
> Hi,
> 
> On 12.02.20 12:16, Thomas Schatzl wrote:
>> Hi Liang,
>> 
>> On 12.02.20 11:17, Liang Mao wrote:
>>> Hi Thomas,
>>> 
>>> I made a new patch for the issues we listed in JDK-8238686 and
>>> JDK-8236073:
>>> http://cr.openjdk.java.net/~luchsh/8236073.webrev.3/
>> 
>>   thanks. I only had time to quickly browse the change, and started 
>> building and testing it internally. I will run it through our perf 
>> benchmarks to look for regressions of out-of-box behavior.
>> 
>> I will need a day or two until I can get back to looking at the change 
>> in detail. There is currently something else I need to look at. Sorry.
> 
>   initial results from testing:
> 
> - gc/g1/TestPeriodicCollection.java fails consistently because the heap 
> does not shrink as expected (but probably this is a test bug as it may 
> expect that uncommit occurs at remark).
> 
> - memory usage tends to be significantly higher with the change without 
> improving scores.
> 
> E.g. I have been running specjvm2008 out-of-box with no settings on 
> different machine(s) (32gb ram min), and the build with the changes 
> almost consistently uses more heap (i.e. committed size) than without, 
> in the range of 10% without any performance increase.
> 
> Specjvm2008 benchmarks are pretty simple application in terms of 
> behavior, i.e. does the same things all the time. This also means that 
> very likely the current sizing is already way beyond the point of 
> diminishing returns (actually, this is a known issue :)); I would prefer 
> if we did not add to that. ;)
> 
> Unfortunately I lost the graphs I had generated (manually), and I do not 
> have more time available right now so can't show you right now.
> 
> I started some dacapo 2009 runs (running them for 30 iterations each).
> 
> Did not have time to look at the changes themselves any further or 
> investigate the reasons for this memory usage increase than I already 
> did earlier; will continue on Tuesday as I'm taking the day off Monday.
> 
> Thanks,
>   Thomas
>