Status of JEP-8204088/JDK-8236073

[Thomas Schatzl]

Hi,

On 10.05.21 21:13, Jonathan Joo wrote:
> (+cc Thomas, as he is the owner/assignee of these two)
> 
> Hello,
> 
> I work on the Java Platform Team at Google, and was looking into the 
> possibility of working on something that would have similar 
> functionality to what is described in JEP-8204088/JDK-8236073, for the 
> G1 garbage collector.
> 
> I was wondering whether any progress has been made on that front, and if 
> not, if there are certain technical hurdles that made this project 
> infeasible to work on? If not, I would be happy to work on this feature 
> and even upstream it to the OpenJDK if I am able to get it working properly.
> 
> Also it would be great to collaborate with anyone familiar with this 
> space and get some insights into this work as I learn more about it :)

   as you can see the work stalled a bit. :)

The (long) story: SoftMaxHeapSize functionality does not work well 
together with current G1 heap sizing. The heuristics expand the heap too 
aggressively, typically ultimately leading to use of all heap. Even 
constant loads cause this.

Since SoftMaxHeapSize is kind of a hint overridden by that policy, it 
will be mostly ignored lots of times: i.e. SoftMaxHeapSize causing a 
concurrent cycle, releasing memory during remark, but the next GC 
immediately expanding the heap again (g1 does not shrink the heap during 
normal gc).

So I went off fixing the sizing policy (JDK-8244603 and JDK-8238687), 
ultimately allowing more heap shrinking/uncommit at every gc.

The problem has been that one of these dependencies (JDK-8244603) 
exposed two serious bugs (JDK-8249192 and JDK-82541164) that took quite 
a bit of time to subsequently fix in jdk16.

After that there were some performance regressions for some throughput 
benchmarks, caused by uncommitting during the gc pause (JDK-8238687) 
that has been fixed too by doing the uncommit work concurrently in the 
meantime (JDK-8236926).

So what's left is basically re-applying JDK-8253143 and JDK-8238687, 
doing perf tests, re-checking performance impact. I think there is just 
one dacapo benchmark or so left that shows some regression, but my 
recollection is kind of hazy after all this time. After that some 
cleanup should probably be done (JDK-8248324, maybe JDK-8247843 if 
needed) one can look into actually using SoftMaxHeapSize to guide sizing.

That latter is quite a small patch, I added what I have to JDK-8236073 
(and JDK-8248324) to gauge the approximate amount of work there :)

At some point I have been "distracted" to finally complete remembered 
set changes (JDK-8017163) which is really, actually almost ready for 
review. I was kind of planning to get back to either this 
SoftMaxHeapSize work or more remembered set related stuff after that.

Of course, I would be happy if somebody took over completion of this work.

As for patches, as you might have guessed from above write-up, there are 
(old) patches for all of the CRs mentioned (good that you asked before 
diving into this!).

I rebased my open JDK-8238687 branch to current at 
https://github.com/tschatzl/jdk/tree/8238687-investigate-memory-uncommit-during-young-gc2 
.
I noticed that I did some significant magic number tuning in the last 
few commits. Also note that the first commit in that sequence is 
JDK-8253143.

Oh, and it just compiles, did not do significant actual testing. If 
there are any questions (and I guess there are), ping me here.

Thanks,
   Thomas