G1 GC clean up time is too long

[YU ZHANG]

Yao,

Thanks for the logs.

I agree with Thomas' analysis.  In the gc log, there are lot of
'reclaimable percentage not over threshold'

Please try

-XX:G1HeapWastePercent=5%
It should add more old regions for mixed gc, also triggers more mixed gc after concurrent gcs.

And
-XX:InitiatingHeapOccupancyPercent=65

I think these 2 should help getting rid of to-space exhausted and full gc.  You have only 8 to-space exhausted.

If this is not good enough, you can try increasing
G1MixedGCLiveThresholdPercent (add more old regions to mixed gc)
  or decreasing G1HeapWastePercent(not much allocation spikes, so give more rooms to gc)

Thanks,
Jenny

On 12/20/2013 2:13 AM, Thomas Schatzl wrote:
> Some correction I think:
>
> On Fri, 2013-12-20 at 10:07 +0100, Thomas Schatzl wrote:
>> Hi,
>>
>> On Thu, 2013-12-19 at 14:16 -0800, yao wrote:
>>>   Hi All,
>>>
>> The other variable is G1HeapWastePercent: it indicates how much wasted
>> space G1 can leave without collecting it after considering all
>> candidates (selected above).
>>
>> That's the 10% threshold in the
>>
>>> 925756.116: [G1Ergonomics (Mixed GCs) continue mixed GCs, reason:
>>> candidate old regions available, candidate old regions: 617 regions,
>>> reclaimable: 9897360192 bytes (13.93 %), threshold: 10.00 %]
>> message.
>>
>> So in the worst case, the 10% means that G1 leaves untouched at most 10%
>> of the heap of 65% occupied regions.
>>
>> I.e. at a region size of 32M and a max heap of 66G, it leaves 6.6G/.65 =
>> 10.1G = (rounded to regions) ~320 regions.
>>
>> So in this case, your target old gen heap size is around 76G (43G/.65 +
>> (66G*0.1)/.65) - which is beyond the max heap, not even counting some
>> space for the young gen (eden and survivor) and additional safety buffer
>> for promotion spikes.
> I do not think (without looking at the code again) you need to add the
> space occupied after all mixed gcs to the maximum old gen heap size. At
> some point it seems I got that calculation wrong...
>
> Anyway, the summary of all this is that default
> G1MixedGCLiveThresholdPercent value seems to be too low for this live
> heap (assuming that it is typically 43G).
>
> I.e. the resulting old gen heap size is typically determined by the
> G1MixedGCLiveThresholdPercent only. Note that this assumes that there
> are results from marking available right when the old gen reaches this
> object distribution - so depending on when marking starts and ends, and
> the first mixed GC starts, more space may be required in reality.
>
> Thomas
>
>
> _______________________________________________
> hotspot-gc-use mailing list
> hotspot-gc-use at openjdk.java.net
> http://mail.openjdk.java.net/mailman/listinfo/hotspot-gc-use