G1-GC - Full GC [humongous allocation request failed]

[Vitaly Davidovich]

Hi Thomas,

Thanks for the clarification and insights.  A few comments below ...

On Monday, October 10, 2016, Thomas Schatzl <thomas.schatzl at oracle.com>
wrote:

> Hi all,
>
> On Fri, 2016-10-07 at 13:44 -0400, Vitaly Davidovich wrote:
> >
> > On Friday, October 7, 2016, charlie hunt <charlie.hunt at oracle.com
> <javascript:;>>
> > wrote:
> > > I think others are benefiting from your question(s) … and it’s
> > > helping refresh my memory of things too. ;-)
> > >
> > > Actually, I just looked at what we documented in Java Performance
> > > Companion for G1ReservePercent, this wording may imply a very
> > > slightly subtle different definition, “To reduce the risk of
> > > getting a promotion failure, G1 reserves some memory for
> > > promotions. This memory will not be used for the young
> > > generation.”
> > >
> > > Perhaps one of the G1 engineers can clarify this?
>   the area covered by G1ReservePercent is regular space available for
> any allocation, whether young or old or humongous.
>
> The only difference is that while the heap occupancy is beyond the
> reserve percent threshold, young gen will be minimal (like bounded by
> G1NewSizePercent). I.e. G1 will run in some kind of "degraded
> throughput" mode. "Degraded" as in young gen size is typically somehow
> correlated with allocation throughput, so if you bound young gen size,
> you also bound throughput.

Ok, so that's a quite different definition of the reserve than pretty much
all sources that I've seen :).  Your explanation makes it sound like a
"yellow zone" for G1, or a throttle/watermark for the young gen sizing.

>
> The thinking for the reserve is to cover for extraneous large
> allocations (either humongous or just a case where due to application
> behavior changes lots of young gen objects survive) while G1 is getting
> liveness information for the reclamation phase (i.e. mixed gc phase).


> The collector just can't know what is the "maximum" promotion or
> humongous object allocation rate as it is heavily application
> dependent.
> Just assuming the worst case, i.e. G1ReservePercent equals young gen,
> would be way too wasteful, and at odds with other settings actually -
> G1 can and will expand young gen to up to 70% if possible. Further,
> such a heuristic would not capture humongous allocation by the
> application anyway.
>
> Ideally G1ReservePercent and InitiatingHeapOccupancyPercent are tuned
> so that reclamation starts when occupancy reaches the G1ReservePercent
> threshold. I.e., some ASCII art:
>
>    +--------------------+  <-- heap full
> ^  |                    |
> |  | 1)G1ReservePercent |
> |  |                    |
>    +--------------------+  <-- first mixed gc
> H  |                    |
> e  | 2)Allocation into  |
> a  | old gen during     |
> p  | marking            |
>    |                    |
> o  +--------------------+ <-- InitiatingHeapOccupancyPercent
> c  |                    |
> c  . 3)"Unconstrained"  .
> u  . young gen sizing   .
> p  . operation          .
> a  .                    .
> n  .                    .
> c  .                    .
> y  .                    .
>    +--------------------+  <-- heap empty
>
> (I am probably forgetting one or the other edge case here, but that's
> the general idea; also please consider that for G1, except for
> humongous allocations, the heap does not need to )
>
> So when current young gen size + old gen occupancy is somewhere in
> areas 2)/3), G1 will expand young gen as it sees fit to meet pause
> time, i.e. is "unconstrained".
>
> If young gen size + old gen occupancy starts eating into area 1), G1
> minimizes young gen to try to keep as much memory left for these
> "extraneous allocations" that G1ReservePercent indicates, in the hope
> that the IHOP is "soon" kicking in. Until jdk9, G1 assumes that the
> user gave some sane settings according to (roughly) this model.
> With jdk9 onwards, the IHOP is determined automatically according to
> this model and so far seems to work quite nicely - at least it will
> typically give you a decent starting point for setting it on your own.

Ok, so the reserve acts like a high watermark in 9, used to adjust IHOP
dynamically.  It sounds like it's an IHOP++ setting :).

I'm also not sure winding the young gen down helps in cases where old gen
occupancy is growing.  Intuitively, that ought to make things worse
actually.  Young evacs will occur more frequently, with higher likelihood
that more objects are still live, and need to be kept alive, possibly
causing further promotion.

One way that it helps is there's more frequent feedback to G1 about heap
occupancy (since young evacs occur more frequently), and so it may notice
that things aren't looking so peachy earlier.  Is that the idea?


> As for the default value of G1ReservePercent (=10), well, consider it
> some default for the "typical" application, trying to strike some
> balance between throughput and safety to prevent running out of memory.
>
> For very large heaps, it might typically be set a bit too large as the
> young gen will most of the times be smaller than 10% of the heap due to
> pause time constraints (or e.g. G1MaxNewSizePercent) and application
> specific boundaries like "useful" allocation rates. Setting it to 40%
> seems a bit too cautious, but may be warranted in some cases. Before
> JDK9, it may be better to set InitiatingHeapOccupancyPercent properly.
>
> For very small heaps G1ReservePercent may be too small.
>
> (jdk9 specific tip: you can use G1ReservePercent to set a maximum IHOP
> value).
>
> Thanks,
>   Thomas
>
>

-- 
Sent from my phone
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.openjdk.java.net/pipermail/hotspot-gc-use/attachments/20161010/17448fed/attachment.html>