G1 discovers same garbage again?

Tony Printezis tony.printezis at oracle.com
Fri Dec 9 19:32:51 UTC 2011


Florian,

Inline.

On 12/05/2011 02:19 PM, Florian Binder wrote:
>> I assume you know when the data will be dropped and reinitialized,
>> right? Can you do a Full GC (with System.gc()) after you re-initialize
>> the heap. This typically helps a lot with CMS.
> Yes, this is exactly what we are trying at this time ;-)
> Either this Full-GC pauses are very long (60-300 seconds) they are much
> shorter than after a CMS-failure (when they take sometimes more than
> 4000 seconds ;-)) and we can define when they occure. Maybe this will be
> our solution. It depends on the result of my g1-experience ;-)

OK. :-) We'd better make it really good. :-)

>>>    This was the cause I am experiencing
>>> with g1 in general. I am not sure if we had fragmentation on this one.
>> G1 should not have fragmentation issues at the small / medium size
>> object level. It might only if you allocate a lot of very large arrays
>> (0.5MB+).
> Ah ok, this is good to know. So if we have a lot of 10 MB byte arrays
> g1-might have problems with them?

Let me be a bit clearer: if you just allocate such large objects once, 
then it won't be a problem. If you allocate them and drop them at a 
rapid rate it might be a problem

> May it helpt to increase the
> G1HeapRegionSize to 20MB? Or is this to large and it would be better to
> break them down into smaller arrays (0.5mb)?

First, the heap region size should be a power of 2 (so if you try to set 
it to 20MB, G1 will actually use 32MB). Also, the heap region size is 
automatically calculated for a particular heap size. If you enable 
-XX:+PrintHeapAtGC it should tell you how large it is for your 
particular app:

  garbage-first heap   total 6144K, used 3185K [0x6e800000, 0x6ee00000, 
0xae800000)
   region size 1024K, 2 young (2048K), 1 survivors (1024K)

Generally, avoiding to allocate objects that are larger than the heap 
region size is good practice as it will eliminate fragmentation issues. 
However, please also remember that each object has a header too! So if 
you allocate, say, a byte array of exactly 1MB entries, the object's 
size is going to be 1MB + the array header size (12 bytes in the 32-bit 
JVM). So you want to size arrays a bit smaller than the region size (not 
too small though, as you'll probably waste the rest of the region).

>>
>> Here's a quick description of how the Eden and the other spaces in G1
>> work. Hopefully, this will help you understand this behavior a bit
>> better.
>>
>> When new objects are allocated by the application the are placed in
>> the Eden which is a set of regions. When the Eden is full we initiate
>> a collection. During that collection we physically move all the
>> surviving objects from the Eden regions to another set of regions
>> called the Survivors. At the end of a collection the Eden regions have
>> no more live objects (we copied them somewhere else) so we can reclaim
>> them (which is why the Eden is empty after a GC). After a GC new
>> objects are allocated into Eden regions (either new ones, or ones
>> which were reclaimed before; it doesn't matter which) which make the
>> Eden grow again. And when it's full it's again collected, etc.
> So, after a (young) gc the eden space should increase only by the new
> allocated objects?

Yes.

>   Or is it possible that new non empty regions are used
> for the eden space, too?

Any free region can be used for the eden. Either once that were 
reclaimed before or new ones.

> As you can see at http://java4.info/g1/eden_all.png
> from 13:20 until 18:00 the eden space is constantly growing (with a few
> gcs)

Did you point me to the right graph? The timeline seems to be 18:06 - 18:35.

>   just by the new objects (allocated by me). But after 18:00 There
> are frequent jumps which are much more than I would ever allocate. So
> what is causing them?
>
> First of all I thought g1 is taking some old-gen-regions to the gc
> because it has enough time to do this,

We generally only do that for a few GCs after each marking phase when we 
recalculate the liveness information in the old regions and we know 
which ones are best to collect.

>   but then I saw that in this cause
> "(partial)" will be append in the out-log:
> http://java4.info/g1/out_err.log.gz
> Furthermore this should not increase the total-heap-space:
> http://java4.info/g1/all.png
>
> Or is it possible that within a young gc only a few of the young regions
> are collected and reclaimed?

Nope. We definitely collect all the young regions at every young GC.

Tony

> Thanks,
> Flo
>
>> Does this explain better the behavior you see?
>>
>>> After I have written the last email, I have seen that it has calm itself
>>> after a few hours. But it is nevertheless very curious and produces a
>>> lot of unnecessary pauses.
>> They are not really unnecessary. Each pause reclaims a lot of
>> short-lived objects.
>>
>> Tony
>>
>>> Flo
>>>
>>>> Tony
>>>>
>>>>>     I
>>>>> assume that it runs during a collection in the old garbage and
>>>>> collects
>>>>> it again. Is this possible? Or is there an overflow since eden space
>>>>> uses more than 3.5 gb?
>>>>>
>>>>> Thanks and regards,
>>>>> Flo
>>>>>
>>>>> Some useful information:
>>>>> $ java -version
>>>>> java version "1.6.0_29"
>>>>> Java(TM) SE Runtime Environment (build 1.6.0_29-b11)
>>>>> Java HotSpot(TM) 64-Bit Server VM (build 20.4-b02, mixed mode)
>>>>>
>>>>> Startup Parameters:
>>>>> -Xms20g -Xmx20g
>>>>> -verbose:gc \
>>>>> -XX:+UnlockExperimentalVMOptions \
>>>>> -XX:+UseG1GC \
>>>>> -XX:+PrintGCDetails \
>>>>> -XX:+PrintGCDateStamps \
>>>>> -XX:+UseLargePages \
>>>>> -XX:+PrintFlagsFinal \
>>>>> -XX:-TraceClassUnloading \
>>>>>
>>>>> $ cat /proc/meminfo | grep Huge
>>>>> HugePages_Total: 11264
>>>>> HugePages_Free:   1015
>>>>> HugePages_Rsvd:     32
>>>>> Hugepagesize:     2048 kB
>>>>>
>>>>> A few screen-shots of the jconsole memory-view:
>>>>> http://java4.info/g1/1h.png
>>>>> http://java4.info/g1/all.png
>>>>> http://java4.info/g1/eden_1h.png
>>>>> http://java4.info/g1/eden_all.png
>>>>> http://java4.info/g1/oldgen_all.png
>>>>>
>>>>> The sysout end syserr logfile with the gc logging and PrintFinalFlags:
>>>>> http://java4.info/g1/out_err.log.gz
>>>>> _______________________________________________
>>>>> hotspot-gc-use mailing list
>>>>> hotspot-gc-use at openjdk.java.net
>>>>> http://mail.openjdk.java.net/mailman/listinfo/hotspot-gc-use
>>> _______________________________________________
>>> hotspot-gc-use mailing list
>>> hotspot-gc-use at openjdk.java.net
>>> http://mail.openjdk.java.net/mailman/listinfo/hotspot-gc-use
> _______________________________________________
> hotspot-gc-use mailing list
> hotspot-gc-use at openjdk.java.net
> http://mail.openjdk.java.net/mailman/listinfo/hotspot-gc-use
_______________________________________________
hotspot-gc-use mailing list
hotspot-gc-use at openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/hotspot-gc-use



More information about the hotspot-gc-dev mailing list