G1-GC - Full GC [humongous allocation request failed]
Vitaly Davidovich
vitalyd at gmail.com
Mon Oct 10 22:28:07 UTC 2016
On Monday, October 10, 2016, Gopal, Prasanna CWK <
prasanna.gopal at blackrock.com> wrote:
> Hi Thomas
>
>
>
> Thanks for this wonderful explanation for G1ReservePercent parameter. As
> noted by Vitaly , it is activing as watermark for young generation heap
> size. So in our case where G1ReservePercent=40, We are effectively
> asking G1 not to resize (increase) young generation , when we reach 60% of
> heap occupancy.
>
>
>
> @All -thanks for your comments. I will adjust the parameters we discussed
> and publish the outcome. I am going to tune with the following parameters
>
>
>
> 1) G1ReservePercent -To reduce this value to a reasonable value. As
> a result , we are allowing G1 to resize young generation size more which
> can reduce the object promotion rate.
>
>
>
> 2) G1MixedGCLiveThresholdPercent – To increase this percent to 85.
> This will G1 more aggressive by having more mixed GC’s
>
> Could someone please explain me how increasing it from 65 (
> which is default in JDK 7) to 85 makes G1 to collect more old regions. I
> would have thought keeping it 65 means , we asking G1 to consider
>
> regions above 65% of occupancy which will include regions with 85% as
> well. Am I missing some thing here ?
>
> This value says "what max liveness does an old region need to have to be
considered for mixed collections". In other words, a value of 65 means a
region must have liveness of 65 or less to be considered. Put another way,
if garbage is 35%+ it's a candidate.
When you set it to 85%, a region can be "more live"/less garbage (i.e. 15%+
garbage) and still be eligible.
>
>
> 3) To override MaxGCPauseMillis to a higher value , to make G1 less
> aggressive about GC pause time.
>
>
>
> 4) To move to latest version of JDK , as suggested by everyone.
>
>
>
> Thanks again for your comments. Really appreciate it.
>
>
>
> Thanks and Regards
>
> Prasanna
>
>
>
>
>
> *From:* hotspot-gc-use [mailto:hotspot-gc-use-bounces at openjdk.java.net
> <javascript:_e(%7B%7D,'cvml','hotspot-gc-use-bounces at openjdk.java.net');>]
> *On Behalf Of *Vitaly Davidovich
> *Sent:* 10 October 2016 11:42
> *To:* Thomas Schatzl <thomas.schatzl at oracle.com
> <javascript:_e(%7B%7D,'cvml','thomas.schatzl at oracle.com');>>
> *Cc:* hotspot-gc-use at openjdk.java.net
> <javascript:_e(%7B%7D,'cvml','hotspot-gc-use at openjdk.java.net');>
> *Subject:* Re: G1-GC - Full GC [humongous allocation request failed]
>
>
>
> Hi Thomas,
>
>
>
> Thanks for the clarification and insights. A few comments below ...
>
> On Monday, October 10, 2016, Thomas Schatzl <thomas.schatzl at oracle.com
> <javascript:_e(%7B%7D,'cvml','thomas.schatzl at oracle.com');>> wrote:
>
> Hi all,
>
> On Fri, 2016-10-07 at 13:44 -0400, Vitaly Davidovich wrote:
> >
> > On Friday, October 7, 2016, charlie hunt <charlie.hunt at oracle.com>
> > wrote:
> > > I think others are benefiting from your question(s) … and it’s
> > > helping refresh my memory of things too. ;-)
> > >
> > > Actually, I just looked at what we documented in Java Performance
> > > Companion for G1ReservePercent, this wording may imply a very
> > > slightly subtle different definition, “To reduce the risk of
> > > getting a promotion failure, G1 reserves some memory for
> > > promotions. This memory will not be used for the young
> > > generation.”
> > >
> > > Perhaps one of the G1 engineers can clarify this?
> the area covered by G1ReservePercent is regular space available for
> any allocation, whether young or old or humongous.
>
> The only difference is that while the heap occupancy is beyond the
> reserve percent threshold, young gen will be minimal (like bounded by
> G1NewSizePercent). I.e. G1 will run in some kind of "degraded
> throughput" mode. "Degraded" as in young gen size is typically somehow
> correlated with allocation throughput, so if you bound young gen size,
> you also bound throughput.
>
> Ok, so that's a quite different definition of the reserve than pretty much
> all sources that I've seen :). Your explanation makes it sound like a
> "yellow zone" for G1, or a throttle/watermark for the young gen sizing.
>
>
> The thinking for the reserve is to cover for extraneous large
> allocations (either humongous or just a case where due to application
> behavior changes lots of young gen objects survive) while G1 is getting
> liveness information for the reclamation phase (i.e. mixed gc phase).
>
>
> The collector just can't know what is the "maximum" promotion or
> humongous object allocation rate as it is heavily application
> dependent.
> Just assuming the worst case, i.e. G1ReservePercent equals young gen,
> would be way too wasteful, and at odds with other settings actually -
> G1 can and will expand young gen to up to 70% if possible. Further,
> such a heuristic would not capture humongous allocation by the
> application anyway.
>
> Ideally G1ReservePercent and InitiatingHeapOccupancyPercent are tuned
> so that reclamation starts when occupancy reaches the G1ReservePercent
> threshold. I.e., some ASCII art:
>
> +--------------------+ <-- heap full
> ^ | |
> | | 1)G1ReservePercent |
> | | |
> +--------------------+ <-- first mixed gc
> H | |
> e | 2)Allocation into |
> a | old gen during |
> p | marking |
> | |
> o +--------------------+ <-- InitiatingHeapOccupancyPercent
> c | |
> c . 3)"Unconstrained" .
> u . young gen sizing .
> p . operation .
> a . .
> n . .
> c . .
> y . .
> +--------------------+ <-- heap empty
>
> (I am probably forgetting one or the other edge case here, but that's
> the general idea; also please consider that for G1, except for
> humongous allocations, the heap does not need to )
>
> So when current young gen size + old gen occupancy is somewhere in
> areas 2)/3), G1 will expand young gen as it sees fit to meet pause
> time, i.e. is "unconstrained".
>
> If young gen size + old gen occupancy starts eating into area 1), G1
> minimizes young gen to try to keep as much memory left for these
> "extraneous allocations" that G1ReservePercent indicates, in the hope
> that the IHOP is "soon" kicking in. Until jdk9, G1 assumes that the
> user gave some sane settings according to (roughly) this model.
> With jdk9 onwards, the IHOP is determined automatically according to
> this model and so far seems to work quite nicely - at least it will
> typically give you a decent starting point for setting it on your own.
>
> Ok, so the reserve acts like a high watermark in 9, used to adjust IHOP
> dynamically. It sounds like it's an IHOP++ setting :).
>
>
>
> I'm also not sure winding the young gen down helps in cases where old gen
> occupancy is growing. Intuitively, that ought to make things worse
> actually. Young evacs will occur more frequently, with higher likelihood
> that more objects are still live, and need to be kept alive, possibly
> causing further promotion.
>
>
>
> One way that it helps is there's more frequent feedback to G1 about heap
> occupancy (since young evacs occur more frequently), and so it may notice
> that things aren't looking so peachy earlier. Is that the idea?
>
>
>
>
> As for the default value of G1ReservePercent (=10), well, consider it
> some default for the "typical" application, trying to strike some
> balance between throughput and safety to prevent running out of memory.
>
> For very large heaps, it might typically be set a bit too large as the
> young gen will most of the times be smaller than 10% of the heap due to
> pause time constraints (or e.g. G1MaxNewSizePercent) and application
> specific boundaries like "useful" allocation rates. Setting it to 40%
> seems a bit too cautious, but may be warranted in some cases. Before
> JDK9, it may be better to set InitiatingHeapOccupancyPercent properly.
>
> For very small heaps G1ReservePercent may be too small.
>
> (jdk9 specific tip: you can use G1ReservePercent to set a maximum IHOP
> value).
>
> Thanks,
> Thomas
>
>
>
> --
> Sent from my phone
>
>
>
> This message may contain information that is confidential or privileged.
> If you are not the intended recipient, please advise the sender immediately
> and delete this message. See http://www.blackrock.com/
> corporate/en-us/compliance/email-disclaimers for further information.
> Please refer to http://www.blackrock.com/corporate/en-us/compliance/
> privacy-policy for more information about BlackRock’s Privacy Policy.
>
> BlackRock Advisors (UK) Limited and BlackRock Investment Management (UK)
> Limited are authorised and regulated by the Financial Conduct Authority.
> Registered in England No. 796793 and No. 2020394 respectively. BlackRock
> Life Limited is authorised by the Prudential Regulation Authority and
> regulated by the Financial Conduct Authority and the Prudential Regulation
> Authority. Registered in England No. 2223202. Registered Offices: 12
> Throgmorton Avenue, London EC2N 2DL. BlackRock International Limited is
> authorised and regulated by the Financial Conduct Authority and is a
> registered investment adviser with the Securities and Exchange Commission
> (SEC). Registered in Scotland No. SC160821. Registered Office: Exchange
> Place One, 1 Semple Street, Edinburgh EH3 8BL.
>
> For a list of BlackRock's office addresses worldwide, see
> http://www.blackrock.com/corporate/en-us/about-us/contacts-locations.
>
> © 2016 BlackRock, Inc. All rights reserved.
>
>
--
Sent from my phone
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.openjdk.java.net/pipermail/hotspot-gc-use/attachments/20161010/8c42c9cf/attachment-0001.html>
More information about the hotspot-gc-use
mailing list