G1 feedback

[Todd Lipcon]

Here's a patch of all my local changes against a checkout from a couple
months back (may not completely apply against JDK7 trunk). This has the
fixes mentioned as well as a few other experiments I'd done.

I expressly grant permission to the JDK team to include this patch or parts
thereof in the JDK should they find the code useful.

-Todd

On Fri, Apr 1, 2011 at 2:07 PM, Alex Aisinzon <aaisinzon at guidewire.com>wrote:

> Todd
>
>
>
> Please share the patch. I will see if I can build it.
>
>
>
> Regards
>
>
>
> Alex Aisinzon
>
>
>
> *From:* Todd Lipcon [mailto:todd at cloudera.com]
> *Sent:* Friday, April 01, 2011 1:55 PM
>
> *To:* Alex Aisinzon
> *Cc:* hotspot-gc-use at openjdk.java.net
> *Subject:* Re: G1 feedback
>
>
>
> On Fri, Apr 1, 2011 at 1:52 PM, Alex Aisinzon <aaisinzon at guidewire.com>
> wrote:
>
> Hi Todd
>
>
>
> This is very interesting. I feel I need to read some additional material
> about G1 to fully understand your explanation. I guess the research paper on
> it would help. I will plan to read it.
>
> In any case, I am happy to give your patch a try. The challenge is that I
> am not well set to rebuild G1. Is there a way I could get the java
> binary/executable with the patch included?
>
>
>
>
>
> Sorry, I'm not well equipped to distribute a binary (and there might be
> licensing issues, I'm not even sure).
>
>
>
> -Todd
>
>
>
>
>
> *From:* Todd Lipcon [mailto:todd at cloudera.com]
> *Sent:* Friday, April 01, 2011 12:05 PM
> *To:* Alex Aisinzon
> *Cc:* hotspot-gc-use at openjdk.java.net
> *Subject:* Re: G1 feedback
>
>
>
> Hi Alex,
>
>
>
> I've had similar results - see my threads from a few months back on this
> mailing list.
>
>
>
> The summary is that, when there is a tight pause bound, some regions will
> accumulate which have estimates that are "stuck" higher than the goal. As
> these accumulate, they're never collected, which means that memory usage
> slowly grows until a full GC is required.
>
>
>
> The two situations that caused this in my experiments were:
>
> 1) The JVM got context-switched out for several scheduling quanta during
> the "other" portion of a non-young region collection. Because "other" time
> is considered constant overhead, even mostly-garbage regions were carrying
> this as part of their estimate, causing all non-young regions to be deemed
> "too expensive". I fixed this with a patch to notice when the "other time"
> estimate was greater than the pause goal and decay it back towards 0.
>
>
>
> 2) A bad region accumulates many inter-region references in its "remember
> set", overflowing into the coarse rset. Once the coarse rset has been used,
> there's no facility to "uncoarsen" the rset entries even after all the
> referring regions are dead. If the number of coarse entries is high enough
> that the time estimate is greater than the pause time goal, then again, this
> region will never be collected, and memory will fill up until a full GC. I
> wrote a patch to improve the time estimation for coarse rset entries based
> on the liveness info in the coarse regions. This helped somewhat for my
> application.
>
>
>
> Let me know if you'd like to try these patches out, I can dig them up again
> (or you might find them in the mailing list archives).
>
>
>
> -Todd
>
> On Fri, Apr 1, 2011 at 11:48 AM, Alex Aisinzon <aaisinzon at guidewire.com>
> wrote:
>
> Hi all
>
>
>
> Thoughts on this feedback about G1?
>
>
>
> Take care
>
>
>
> Alex A
>
>
>
> *From:* Alex Aisinzon
> *Sent:* Saturday, March 26, 2011 6:46 AM
> *To:* hotspot-gc-use at openjdk.java.net
> *Subject:* G1 feedback
>
>
>
> Hi all
>
>
>
> I experimented with G1 and Sun JDK 1.6 update 24 and ran two long running
> tests (8 hours) with it:
>
> With " -server -XX:+UseG1GC -XX:+UseCompressedOops -Xms24576m -Xmx24576m",
> most pauses were very short. 4 pauses were around 7 seconds and 4 at 34
> seconds.
>
> I then added the objective of keeping the longest pause around 1 second and
> used "-server -XX:+UseG1GC -XX:MaxGCPauseMillis=1000 -XX:+UseCompressedOops
> -Xms24576m -Xmx24576m". Most pauses were a little above 1 second except for
> one pause at 8 seconds and one at 57 seconds.
>
> The server is a dual X5570 (8 cores total) and has 48GB of RAM. Its average
> CPU utilization was around 60-65% so it was not over-used.
>
> Everything would be perfect if it were not for the 7, 34 and 8, 57 seconds
> pauses.
>
> What would you recommend I do to either reduce these longer pauses or give
> insights into what happened so that G1 can avoid these very rare but pretty
> long pauses in the future?
>
>
>
> Thanks in advance
>
>
>
> Alex A
>
>
> _______________________________________________
> hotspot-gc-use mailing list
> hotspot-gc-use at openjdk.java.net
> http://mail.openjdk.java.net/mailman/listinfo/hotspot-gc-use
>
>
>
>
> --
> Todd Lipcon
> Software Engineer, Cloudera
>
>
>
>
> --
> Todd Lipcon
> Software Engineer, Cloudera
>



-- 
Todd Lipcon
Software Engineer, Cloudera
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/hotspot-gc-dev/attachments/20110401/18da6b85/attachment.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: jdk7-g1-fixes.patch
Type: text/x-patch
Size: 36583 bytes
Desc: not available
URL: <https://mail.openjdk.org/pipermail/hotspot-gc-dev/attachments/20110401/18da6b85/jdk7-g1-fixes.patch>
-------------- next part --------------
_______________________________________________
hotspot-gc-use mailing list
hotspot-gc-use at openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/hotspot-gc-use