Container-aware heap sizing for OpenJDK

[Thomas Schatzl]

Hi,

On 21.09.22 03:58, Jonathan Joo wrote:
> Hi all,
> 
> Apologies for the late response - I had missed the replies due to the 
> way I had set up my inbox filtering 😓. I may have also missed entire 
> emails due to this, so please feel free to re-reply to this email if I 
> have not addressed your questions.
> 
> I'll try to address all the email comments in one go, so please bear 
> with a long email ahead!
> 
> -----
> @ Thomas Schatzl
> 
> Long time no talk :) Thanks for your detailed response.
> 
>     Most of these suggestions seem to be fairly consistent with existing
>     RFEs (e.g. [1], [2], [3], ...) that have been discussed before with you
>     (e.g. in [4]) and been considered really nice to have iirc.
> 
> Agreed that there is a lot of overlap between AHS and the currently open 
> RFEs. Happy to converge on these now that I have a better understanding 
> of this area from working on it! 

Agree :)

> Notably, [1] and [2] are very similar to the two manageable flags that are
> part of AHS, so I agree that there
> is room for collaboration here. For [3], I think that is something that 
> is somewhat orthogonal (in terms of implementation) to AHS, but would be 
> very helpful for AHS, since the more frequently we can uncommit memory, 
> the closer we can get the heap to our target heap size.
> 
>     I am not convinced that having a thread inside the JVM is really the
>     best solution. Constantly querying the _environment_ for changes seems
>     to be traditionally outside of the scope of the JVM. 
> 
>     [...]
> 
>     Using some external process (however it is distributed) seems to be a
> 
>     much more flexible option (not only in customizability but also in
>     terms
>     of the release cycle for it). I would suggest to at least separate this
>     effort from improving the JVM capabilities.
> 
> Just to clarify - the AHS thread is not part of the JVM itself. It is a 
> separate thread that is kicked off in our Java launcher at the same time 
> as the JVM, but is a completely separate process. Thus the functionality 

Thanks for clearing up this misunderstanding of mine. Looking forward to 
see changes.

> that pulls from the environment and sets the manageable flags is not 
> part of hotspot/the JVM. The actual amount of changes to the JVM are not 
> actually that intrusive and actually follow somewhat similar logic to 
> https://github.com/tschatzl/jdk/tree/8238687-investigate-memory-uncommit-during-young-gc2 
> <https://github.com/tschatzl/jdk/tree/8238687-investigate-memory-uncommit-during-young-gc2>.
> 
>     some applications that exhibit a very "phased" behavior
>     results show very bad behavior.
> 
> 
> This is helpful to know - as someone who has never heard of SPECjbb2015, 
> would it be easy for me to try running it with my prototype?

SPECjbb2015 is a standard industry Java VM performance benchmark 
(https://www.spec.org/jbb2015/). Unfortunately it's commercial.

It's fairly standard to run for us at Oracle (it's actually part of our 
performance regression suite), and I've seen other companies and 
scientific papers to use it in performance comparisons (even in our bug 
tracker) as well.

Running it is very easy, at least in so-called "composite" mode which is 
sufficient to show the described behavior.

The main issue with specjbb2015 (in that benchmark mode) is that it 
tends to exhibit "long" lulls of "doing nothing" wrt to particular gc 
relevant aspect, with sudden spikes of (gc) activity.

Examples:

a) during warmup which sets some maximum parameters for the following 
measurement phase, promotion rate is practially zero for X gcs (due to 
self-tuning), and then suddenly spikes to GBs for one or two gcs.

A description of the application (object promotion) behavior can be 
found at https://bugs.openjdk.org/browse/JDK-8288966, and how the 
mis-training of predictors, in this case PLAB sizes, negatively impacts 
gc pause times (in this case specific to some HW platforms to the 
extreme showed in the CR).

However the same issue (for PLABs) exists e.g. in any application 
startup phase of the application where the PLAB sizing isn't trained yet.

b) and during measurement phase, SPECjbb2015 has a peculiar way of 
trying to find maximum sustainable load. I.e. it slowly ramps up load, 
until it "fails" to achieve goals. After such a failure there is a long 
quiescence period, after which it fairly quickly ramps up load again to 
see if the previous "failure" is consistent (or has just been some 
temporary hiccup); after X such failures it determines, yeah, that has 
really been the breaking point.

The problem with the current implementation of JDK-8238687 follows these 
phases of load and quiescence well (which is what you'd actually expect 
of that change after all). However for this benchmark it means that 
after such a quiescence period, heap size will be appropriate for the 
current load, i.e. "minimal", which means that the sudden ramp-up in 
load will almost immediately cause GCs and these "failures" again...

This means that scores are really bad with the change in out-of-box runs 
(i.e. not clamping Xms/Xmx as people usually do for best scores). Now, 
obviously, this is just a benchmark, but ...

Other, more throughput-oriented, more constant loads are only affected 
that much due to the allowed time for GC (i.e. the corresponding option 
you suggested a year ago, don't remember the name) is very generous (for 
current hardware), so tuning the defaults will help or helped last time 
I did the investigation.

I will update my 8238687 source tree to the latest with some minor fixes 
if I didn't yet (as mentioned, I was experimenting a bit with it recently).

[...]
> -----
> @ Fazil
> 
>     Any chance to be considered as a JEP in OpenJDK?
> 
> 
>   I'm not familiar with the process of turning an idea/prototype into a 
> JEP - do you have any suggestions on how to go about doing this? Is 
> there some approval process for creating a JEP, or is anyone open to 
> making one?

Any OpenJDK Committer is open to create a JEP in the bug tracker; the 
idea/process is described in 
http://cr.openjdk.java.net/~mr/jep/jep-2.0-02.html.

It's a more formal (and heavyweight) way than an RFE (or a set of) in 
the bug tracker to describe larger, more "impactful" work (to the 
public) which gives you (much) more (public) attention.

I think Man can give more detail (cc'ed, but probably you meet him 
personally frequently anyway), otherwise feel free to ask if you have 
any questions.

Hth,
   Thomas