G1 discovers same garbage again?

[Tony Printezis]

Hi Florian,

See inline.

On 11/28/2011 2:18 PM, Florian Binder wrote:
> Hi everybody,
>
> I have a java application with 20gb (large-table) memory and using the
> g1 garbage collector.

Quick clarification: I saw that you use a 20G heap from the parameters 
you showed below. Do you know what's your live data size?

> The application calculates the whole time with 10 threads some ratios
> (high cpu load). This is done without producing any garbage. About two
> times a minute a request is sent which produce a littlebit garbage.
> Since we are working with realtime data we are interested in very short
> stop-the-world pauses. Therefore we have used the CMS gc in the past
> until we have got problems with fragmentation now.

Since you don't produce much garbage how come you have fragmentation? Do 
you keep the results for all the requests you serve?

> Therefore I am trying the g1.
>
> This seemed to work very well at first. The stw-pauses were, except the
> cleanup pause,

Out of curiosity: how long are the cleanup pauses?

> very short. This yields me to my first question:
> Is this normal and are there any parameters to influence the
> cleanup-process?

I don't think there's much you can do in the app to influence the 
cleanup duration. During this pause we do some, ahem, cleanup of our 
data structures and for large heaps I have also seen the cleanup pauses 
to take longer than I thought they would take. I know this is not going 
to help you in the short term but we have plans to do the cleanup work 
concurrently (or at least mostly-concurrently) in the future.

>   I thought this phase should be short because there is
> just finished the counting, the role of the bitmaps is switched and the
> next possible garbage regions are detemined. All things, which should be
> very fast. So what is taking the time?

Most likely, the remembered set scrubbing phase...

> The second cause for my email is the crazy behaviour after a few hours:
> After the startup of the server it uses about 13.5 gb old-gen memory and
> generates very slowly eden-garbage. Since the new allocated memory is
> mostly garbage the (young) garbage collections are very fast and g1
> decides to grow up the eden space. This works 4 times until eden space
> has more than about 3.5 gb memory. After this the gc is making much more
> collections and while the collections it discovers new garbage (probably
> the old one again).

I'm not quite sure what you mean by "it discovers new garbage". For 
young GCs, G1 (and our other GCs) will reclaim any young objects that 
will discover to be dead (more accurately: that it will not discover to 
be live).

> Eden memory usage jumps between 0 and 3.5gb even
> though I am sure the java-application is not making more than before.

Well, that's not good. :-) Can you try to explicitly set the young gen 
size with -Xmn3g say, to see what happens?

Tony

>   I
> assume that it runs during a collection in the old garbage and collects
> it again. Is this possible? Or is there an overflow since eden space
> uses more than 3.5 gb?
>
> Thanks and regards,
> Flo
>
> Some useful information:
> $ java -version
> java version "1.6.0_29"
> Java(TM) SE Runtime Environment (build 1.6.0_29-b11)
> Java HotSpot(TM) 64-Bit Server VM (build 20.4-b02, mixed mode)
>
> Startup Parameters:
> -Xms20g -Xmx20g
> -verbose:gc \
> -XX:+UnlockExperimentalVMOptions \
> -XX:+UseG1GC \
> -XX:+PrintGCDetails \
> -XX:+PrintGCDateStamps \
> -XX:+UseLargePages \
> -XX:+PrintFlagsFinal \
> -XX:-TraceClassUnloading \
>
> $ cat /proc/meminfo | grep Huge
> HugePages_Total: 11264
> HugePages_Free:   1015
> HugePages_Rsvd:     32
> Hugepagesize:     2048 kB
>
> A few screen-shots of the jconsole memory-view:
> http://java4.info/g1/1h.png
> http://java4.info/g1/all.png
> http://java4.info/g1/eden_1h.png
> http://java4.info/g1/eden_all.png
> http://java4.info/g1/oldgen_all.png
>
> The sysout end syserr logfile with the gc logging and PrintFinalFlags:
> http://java4.info/g1/out_err.log.gz
> _______________________________________________
> hotspot-gc-use mailing list
> hotspot-gc-use at openjdk.java.net
> http://mail.openjdk.java.net/mailman/listinfo/hotspot-gc-use
_______________________________________________
hotspot-gc-use mailing list
hotspot-gc-use at openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/hotspot-gc-use