What influences young generation pause times?
Tony Printezis
tony.printezis at oracle.com
Tue Apr 20 18:11:59 UTC 2010
Osvaldo,
You misunderstand how a copying GC (which is the algorithm our young gen
GCs implement) works. It does not first mark the live objects, and then
copies them. Instead, it copies the objects as it comes across them
(i.e., at the same time it discovers they are live). So, there is no
opportunity to find big blocks of live objects and not copy them. The
end of the GC would be the only time you would be able to do that but,
by then, you've already copied all the objects anyway.
Regarding calling young GCs explicitly from an application: I can see
how, in the case of single-threaded applications, the application might
know "We are between transactions and, maybe, we have lots of garbage
and not much live in the young gen. So let's do a young GC to clean up
the young gen at maybe low overhead since we'll copy very little."
However, how will this work in the case of multi-threaded applications,
which are the vast majority of applications we see from our customers? A
thread might be between transactions, but what about the other 50, 300,
or even 2,000 threads? If a particular time is good to do a young GC for
a particular thread, it does not mean that it's also good for the rest.
Additionally, I would be willing to bet money that if we provided such
an API, library writers will abuse it thinking that "hey, the end of
this library call will be a great time to do a young GC!", without
taking into consideration that many other threads could be doing
something totally different at the same time (we've seen way too many
libraries that call System.gc() already...).
My two cents,
Tony
Osvaldo Doederlein wrote:
> If you allow some intermission... is the young-gen collector smart
> enough to avoid semispace copying in some favorable conditions? Let's
> say I am lucky and when young-GC is triggered, after marking I have
> [LLLLLLLLLLDDDD] where L=live objects, D=dead. it's stupid to copy the
> block [LLLLLLLLLL] to another space. I'd expect the collector to have
> some heuristic like: look at the top address and total size of the
> remaining live data, and if it is densely populated (say >90% live
> space - e.g. [LLLDLLLLLLDDDD]), just finish GC without any compaction
> or semispace flipping.
>
> I would expect this scenario to happen in the real world with very
> small frequency, because young-GC must be triggered at a "lucky" time,
> e.g. after some application transactions commit and before any newer
> transaction begins - but if the collector already accounts the live
> set size at the marking phase, the cost to attempt this new
> optimization is virtually zero. And we might hint the VM to make sure
> the optimal case doesn't depend on good luck. The JVM could expose an
> API that allows an application (or a container) to request a
> "lightweight GC", i.e., perform only young-GC, and only if the
> young-gen is >N% full. E.g., System.fastGC(0.8) for N=80%. A JavaEE
> application server could invoke this when it detects idle periods
> (zero running transactions / zero background processes doing anything
> important); or even after every transaction commit if the VM uses
> TLABs (in that case we only collect the TLAB; the whole thing only
> makes sense for large enough TLABs). For single-threaded processes
> (Ok, almost-single-threaded...) it's much simpler, just call the
> lightweight-GC API at special places where major activity ends and
> tons of allocated data are liberated, e.g. after the render-frame step
> of your game loop, or after importing each file in your batch ETL
> program, etc.
>
> A+
> Osvaldo
>
> 2010/4/20 Matt Khan <matt.khan at db.com <mailto:matt.khan at db.com>>
>
> Hi Tony
>
> >> Basically, the more objects survive the collection and need to be
> copied, the higher the young GC times will be.
> so when does a concurrent collector enter a STW pause?
>
> for example if I look at figure 6, p10 in the memory management white
> paper (http://java.sun.com/products/hotspot/whitepaper.html) then that
> makes it look like there is a single STW pause per young
> collection that
> is made shorter because there are n threads doing the work. Is that an
> accurate depiction of when it pauses or just a convenient
> visualisation?
>
> My reason for asking is that my app doesn't exhibit this single
> pause per
> young collection, instead I see a succession of short pauses
> between GC
> logs (example below) & I'd like to understand what causes those
> pauses.
> This app is using CMS (params used below) but there is no CMS activity
> reported at this time because v little enters the tenured
> generation and
> hence there is no collection required.
>
> Total time for which application threads were stopped: 0.0051359
> seconds
> Application time: 99.9576332 seconds
> 2010-04-13T19:14:53.185+0000: 368542.855: [GC 368542.855: [ParNew
> Desired survivor size 14450688 bytes, new threshold 1 (max 1)
> - age 1: 3377144 bytes, 3377144 total
> : 2986668K->4491K(2998976K), 0.0254753 secs]
> 3076724K->94963K(3130048K)
> icms_dc=0 , 0.0259072 secs] [Time
> s: user=0.25 sys=0.01, real=0.03 secs]
> Total time for which application threads were stopped: 0.0330759
> seconds
> Application time: 190.7387185 seconds
> Total time for which application threads were stopped: 0.0060798
> seconds
> Application time: 9.2698867 seconds
> Total time for which application threads were stopped: 0.0051861
> seconds
> Application time: 290.7195886 seconds
> Total time for which application threads were stopped: 0.0065455
> seconds
> Application time: 9.2792321 seconds
> Total time for which application threads were stopped: 0.0051541
> seconds
> Application time: 290.7292153 seconds
> Total time for which application threads were stopped: 0.0063071
> seconds
> Application time: 9.2696694 seconds
> Total time for which application threads were stopped: 0.0052036
> seconds
> Application time: 290.7093779 seconds
> Total time for which application threads were stopped: 0.0065365
> seconds
> Application time: 9.2793591 seconds
> Total time for which application threads were stopped: 0.0051265
> seconds
> Application time: 290.7301471 seconds
> Total time for which application threads were stopped: 0.0070431
> seconds
> Application time: 9.2694376 seconds
> Total time for which application threads were stopped: 0.0051428
> seconds
> Application time: 119.4074368 seconds
> Total time for which application threads were stopped: 0.0059739
> seconds
> Application time: 39.8647697 seconds
> 2010-04-13T19:40:52.550+0000: 370102.218: [GC 370102.219: [ParNew
> Desired survivor size 14450688 bytes, new threshold 1 (max 1)
> - age 1: 2911824 bytes, 2911824 total
>
> -Xms3072m
> -Xmx3072m
> -Xmn2944m
> -XX:+DisableExplicitGC
> -XX:+PrintGCDetails
> -XX:+PrintGCDateStamps
> -XX:+PrintGCApplicationStoppedTime
> -XX:+PrintGCApplicationConcurrentTime
> -XX:MaxTenuringThreshold=1
> -XX:SurvivorRatio=190
> -XX:TargetSurvivorRatio=90
> -XX:+UseConcMarkSweepGC
> -XX:+UseParNewGC
>
> Cheers
> Matt
>
> Matt Khan
> --------------------------------------------------
> GFFX Auto Trading
> Deutsche Bank, London
>
>
>
> ---
>
> This e-mail may contain confidential and/or privileged
> information. If you are not the intended recipient (or have
> received this e-mail in error) please notify the sender
> immediately and delete this e-mail. Any unauthorized copying,
> disclosure or distribution of the material in this e-mail is
> strictly forbidden.
>
> Please refer to http://www.db.com/en/content/eu_disclosures.htm
> for additional EU corporate and regulatory disclosures.
> _______________________________________________
> hotspot-gc-use mailing list
> hotspot-gc-use at openjdk.java.net
> <mailto:hotspot-gc-use at openjdk.java.net>
> http://mail.openjdk.java.net/mailman/listinfo/hotspot-gc-use
>
>
_______________________________________________
hotspot-gc-use mailing list
hotspot-gc-use at openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/hotspot-gc-use
More information about the hotspot-gc-dev
mailing list