RFR: 8236073: G1: Use SoftMaxHeapSize to guide GC heuristics

[Stefan Johansson]

Hi Liang, 

I’ve started looking at this patch as well and I have a question regarding the change to not allow shrinking after concurrent mark? Before we could shrink the heap at Remark, but now we only check to expand the heap after the concurrent cycle, why is that? I get that we will be able to shrink even more after the mixed collections but if a lot of regions are freed by the concurrent cycle why not check if we can shrink here?

Also good to hear you can run SPECjvm2008, we also avoid running any problematic benchmarks. 

Thanks,
Stefan

> 17 feb. 2020 kl. 10:56 skrev Liang Mao <maoliang.ml at alibaba-inc.com>:
> 
> Hi Thomas,
> 
> I am able to run specjvm2008 by excluding the compiler subtests
> and reproduce the issue that the change commits more memory.
> The main cause is addressed that the tests have a lot of
> humongous objects which affect the evaluation of adaptive 
> IHOP. _last_unrestrained_young_size and _last_allocated_bytes
> used in G1AdaptiveIHOPControl::predict_unstrained_buffer_size are
> very large. So the expansion after concurrent mark is rather
> aggressive. I made an enhancement to restrict this uncommon
> expansion with MinHeapFreeRatio:
> http://cr.openjdk.java.net/~luchsh/8236073.webrev.4/
> 
> Thanks,
> Liang
> 
> 
> 
> 
> 
> 
> ------------------------------------------------------------------
> From:Thomas Schatzl <thomas.schatzl at oracle.com>
> Send Time:2020 Feb. 15 (Sat.) 03:51
> To:hotspot-gc-dev <hotspot-gc-dev at openjdk.java.net>
> Subject:Re: RFR: 8236073: G1: Use SoftMaxHeapSize to guide GC heuristics
> 
> Hi,
> 
> On 12.02.20 12:16, Thomas Schatzl wrote:
>> Hi Liang,
>> 
>> On 12.02.20 11:17, Liang Mao wrote:
>>> Hi Thomas,
>>> 
>>> I made a new patch for the issues we listed in JDK-8238686 and
>>> JDK-8236073:
>>> http://cr.openjdk.java.net/~luchsh/8236073.webrev.3/
>> 
>>   thanks. I only had time to quickly browse the change, and started 
>> building and testing it internally. I will run it through our perf 
>> benchmarks to look for regressions of out-of-box behavior.
>> 
>> I will need a day or two until I can get back to looking at the change 
>> in detail. There is currently something else I need to look at. Sorry.
> 
>   initial results from testing:
> 
> - gc/g1/TestPeriodicCollection.java fails consistently because the heap 
> does not shrink as expected (but probably this is a test bug as it may 
> expect that uncommit occurs at remark).
> 
> - memory usage tends to be significantly higher with the change without 
> improving scores.
> 
> E.g. I have been running specjvm2008 out-of-box with no settings on 
> different machine(s) (32gb ram min), and the build with the changes 
> almost consistently uses more heap (i.e. committed size) than without, 
> in the range of 10% without any performance increase.
> 
> Specjvm2008 benchmarks are pretty simple application in terms of 
> behavior, i.e. does the same things all the time. This also means that 
> very likely the current sizing is already way beyond the point of 
> diminishing returns (actually, this is a known issue :)); I would prefer 
> if we did not add to that. ;)
> 
> Unfortunately I lost the graphs I had generated (manually), and I do not 
> have more time available right now so can't show you right now.
> 
> I started some dacapo 2009 runs (running them for 30 iterations each).
> 
> Did not have time to look at the changes themselves any further or 
> investigate the reasons for this memory usage increase than I already 
> did earlier; will continue on Tuesday as I'm taking the day off Monday.
> 
> Thanks,
>   Thomas
>