G1:long remark pauses

Fri May 16 12:40:03 UTC 2014

 From the sound of what Thomas is describing, this might be one of those 
apps that's making heavy use of SoftReferences. Output from 
-XX:+PrintReferenceGC as Bengt suggested will show if that's the case.

If we see a large number of SoftReferences being processed per GC, we 
may get further help with tuning the SoftReference reclamation policy, 
(-XX:SoftRefLRUPolicyMSPerMB).

charlie

On 05/16/2014 07:32 AM, Bengt Rutisson wrote:
>
> Hi Thomas,
>
>
> 16 maj 2014 kl. 14:01 skrev Thomas Viessmann 
> <thomas.viessmann at oracle.com <mailto:thomas.viessmann at oracle.com>>:
>
>> Hi Bengt,
>>
>>
>> Thanks for confirming. ParallelOld had stop pauses in the range of 20 
>> to 30 seconds.
>> CMS was a disaster due to extreme fragmentation and high promotion 
>> rate even with
>> huge eden and survivors.
>
> Ok, so even with the long remark pauses G1 is performing better than 
> the other GCs?
>
>
>> There are definitely lots of references. I can find out
>> details.
>
> Thanks, it would be interesting to get this data.
>
> Thanks,
> Bengt
>
>>
>> Thanks and Regards
>>
>> Thomas
>>
>>
>>
>> On 05/16/14 13:53, Bengt Rutisson wrote:
>>>
>>> Hi again Thomas,
>>>
>>>
>>> On 2014-05-16 13:34, Thomas Viessmann wrote:
>>>> Hi Bengt,
>>>>
>>>> Sure, the application has lots of objects and references.
>>>> Downsizing the application has been tried The heap size of 24 g is
>>>> already at minimum. A  smaller heap gave OutOfmemory really quick.
>>>> My question was more whether the remark phases could be optimized
>>>> further. I assume this is not the case and we have reached the 
>>>> limitations
>>>> of G1, right?
>>>
>>>
>>> How many reference objects does the application use? Can you run it 
>>> with -XX:+PrintReferenceGC to see how many there are?
>>>
>>> If there are a lot of them I don't think there is much more that can 
>>> be done for the remark phase. But if there are not that many I guess 
>>> it means that the remark phase is inefficient.
>>>
>>> Have you tried any of the other GCs? How do they behave with this 
>>> application?
>>>
>>> Thanks,
>>> Bengt
>>>
>>>
>>>>
>>>> Thanks and Regards
>>>>
>>>> Thomas
>>>>
>>>>
>>>>
>>>> On 05/16/14 13:18, Bengt Rutisson wrote:
>>>>>
>>>>> Hi Thomas,
>>>>>
>>>>> On 2014-05-16 13:10, Thomas Viessmann wrote:
>>>>>> Hi Bengt,
>>>>>>
>>>>>> Well, that's already done and it did improve things.
>>>>>> argv[21]: -XX:+ParallelRefProcEnabled
>>>>>> argv[22]: -XX:ParallelGCThreads=48
>>>>>
>>>>> Sorry, I missed that.
>>>>>
>>>>>> before -XX:+ParallelRefProcEnabled was set the stop times were in the range of 20 to 60 seconds.
>>>>>
>>>>> OK. Glad it helped some. :)
>>>>>
>>>>>> The application is a Cacao by Oracle. So they cannot change it.
>>>>>
>>>>> Is there some way of reducing the amount of reference objects that 
>>>>> Cacao uses? Does it have cache sizes or similar that can be tuned. 
>>>>> With a JFR recording we might be able to figure out where the 
>>>>> reference objects come from.
>>>>>
>>>>> Thanks,
>>>>> Bengt
>>>>>
>>>>>> Thanks and Regards
>>>>>>
>>>>>> Thomas
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> On 05/16/14 12:58, Bengt Rutisson wrote:
>>>>>>>
>>>>>>> Hi Thomas,
>>>>>>>
>>>>>>> It looks like the application is using a lot of Reference 
>>>>>>> objects. The time spent in remark is dominated by reference 
>>>>>>> processing. See the attached graph generated from the log file 
>>>>>>> you sent.
>>>>>>>
>>>>>>> You can try to see if adding -XX:+ParallelRefProcEnabled 
>>>>>>> improves the situation.
>>>>>>>
>>>>>>> If the customer is interested in updating their application they 
>>>>>>> might want to see if they can reduce the number of 
>>>>>>> java.lang.ref.Reference objects they use.
>>>>>>>
>>>>>>> Hths,
>>>>>>> Bengt
>>>>>>>
>>>>>>>
>>>>>>> On 2014-05-16 10:26, Thomas Viessmann wrote:
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>>
>>>>>>>> I've been tuning a Java 7u51, Solaris 10, T4 system with 24G heap.
>>>>>>>> My customer is not very happy with the remark pauses of  up to 
>>>>>>>> 2 seconds.
>>>>>>>>  -XX:ParallelGCThreads=48 turned out to be the optimum. Here is 
>>>>>>>> the log file
>>>>>>>> which contains the java args at the top:
>>>>>>>>
>>>>>>>> http://aubing.de.oracle.com/gclog/gc_log_03052014.log
>>>>>>>>
>>>>>>>> Any idea to drive the remark stop times further down?
>>>>>>>>
>>>>>>>>
>>>>>>>> Thanks and Regards
>>>>>>>>
>>>>>>>> Thomas
>>>>>>>
>>>>>>
>>>>>> -- 
>>>>>> <mime-attachment.gif> <http://www.oracle.com>
>>>>>> THOMAS VIESSMANN | Senior Principal Technical Support Engineer - Java
>>>>>> Phone: +498914302496 <tel:+49814302496> | Mobile: +491743005467 
>>>>>> <tel:+491743005467>
>>>>>> Oracle Customer Technical Support - Java
>>>>>>
>>>>>> ORACLE Deutschland B.V. & Co. KG | Riesstr.25 | D-80992 Muenchen
>>>>>>
>>>>>> ORACLE Deutschland B.V. & Co. KG
>>>>>> Hauptverwaltung: Riesstr. 25, D-80992 Muenchen
>>>>>> Registergericht: Amtsgericht Muenchen, HRA 95603
>>>>>> Geschäftsführere: Juergen Kunz
>>>>>>
>>>>>> Komplementärin: ORACLE Deutschland Verwaltung B.V.
>>>>>> Hertogswetering 163/167, 3543 AS Utrecht, Niederlande
>>>>>> Handelsregister der Handelskammer Midden-Niederlande, Nr. 30143697
>>>>>> Geschäftsführer: Alexander van der Ven, Astrid Kepper, Val Maher
>>>>>>
>>>>>> ------------------------------------------------------------------------
>>>>>> ------------------------------------------------------------------------
>>>>>> <mime-attachment.gif> <http://www.oracle.com/commitment> Oracle 
>>>>>> is committed to developing practices and products that help 
>>>>>> protect the environment
>>>>>
>>>>
>>>> -- 
>>>> <mime-attachment.gif> <http://www.oracle.com>
>>>> THOMAS VIESSMANN | Senior Principal Technical Support Engineer - Java
>>>> Phone: +498914302496 <tel:+49814302496> | Mobile: +491743005467 
>>>> <tel:+491743005467>
>>>> Oracle Customer Technical Support - Java
>>>>
>>>> ORACLE Deutschland B.V. & Co. KG | Riesstr.25 | D-80992 Muenchen
>>>>
>>>> ORACLE Deutschland B.V. & Co. KG
>>>> Hauptverwaltung: Riesstr. 25, D-80992 Muenchen
>>>> Registergericht: Amtsgericht Muenchen, HRA 95603
>>>> Geschäftsführere: Juergen Kunz
>>>>
>>>> Komplementärin: ORACLE Deutschland Verwaltung B.V.
>>>> Hertogswetering 163/167, 3543 AS Utrecht, Niederlande
>>>> Handelsregister der Handelskammer Midden-Niederlande, Nr. 30143697
>>>> Geschäftsführer: Alexander van der Ven, Astrid Kepper, Val Maher
>>>>
>>>> ------------------------------------------------------------------------
>>>> ------------------------------------------------------------------------
>>>> <mime-attachment.gif> <http://www.oracle.com/commitment> Oracle is 
>>>> committed to developing practices and products that help protect 
>>>> the environment
>>>
>>
>> -- 
>> <oracle_sig_logo.gif> <http://www.oracle.com>
>> THOMAS VIESSMANN | Senior Principal Technical Support Engineer - Java
>> Phone: +498914302496 <tel:+49814302496> | Mobile: +491743005467 
>> <tel:+491743005467>
>> Oracle Customer Technical Support - Java
>>
>> ORACLE Deutschland B.V. & Co. KG | Riesstr.25 | D-80992 Muenchen
>>
>> ORACLE Deutschland B.V. & Co. KG
>> Hauptverwaltung: Riesstr. 25, D-80992 Muenchen
>> Registergericht: Amtsgericht Muenchen, HRA 95603
>> Geschäftsführere: Juergen Kunz
>>
>> Komplementärin: ORACLE Deutschland Verwaltung B.V.
>> Hertogswetering 163/167, 3543 AS Utrecht, Niederlande
>> Handelsregister der Handelskammer Midden-Niederlande, Nr. 30143697
>> Geschäftsführer: Alexander van der Ven, Astrid Kepper, Val Maher
>>
>> ------------------------------------------------------------------------
>> ------------------------------------------------------------------------
>> <green-for-email-sig_0.gif> <http://www.oracle.com/commitment> Oracle 
>> is committed to developing practices and products that help protect 
>> the environment

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/hotspot-gc-dev/attachments/20140516/59cad7a4/attachment.htm>