From ziscloud at gmail.com Tue Oct 1 01:26:22 2019 From: ziscloud at gmail.com (Tony.Wang) Date: Tue, 1 Oct 2019 09:26:22 +0800 Subject: Very long duration of Finalize marking in the Remark stage of G1GC Message-ID: Hi is this problem solved? would you please share it with me? -- Shunyun Wong ????????????????????????? *it was the best of times, it was the worst of times* -------------- next part -------------- An HTML attachment was scrubbed... URL: From thomas.schatzl at oracle.com Tue Oct 1 04:13:23 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Tue, 1 Oct 2019 06:13:23 +0200 Subject: Very long duration of Finalize marking in the Remark stage of G1GC In-Reply-To: References: Message-ID: Hi, On 01.10.19 03:26, Tony.Wang wrote: > Hi > > is this problem solved? would you please share it with me? I am not completely sure which problem or which thread you are answering here though as there is no context. There has been a thread of that name in 2015 afaict; the last log snippet here 2015-10-18T07:07:28.966-0400: 346753.917: [GC remark 2015-10-18T07:07:28.966-0400: 346753.917: [Finalize Marking, 115.4404545 secs] 2015-10-18T07:09:24.406-0400: 346869.357: [GC ref-proc2015-10-18T07:09:24.406-0400: 346869.357: [SoftReference, 54 refs, 0.0028579 secs]2015-10-18T07:09:24.409-0400: 346869.360: [WeakReference, 418 refs, 0.0020702 secs]2015-10-18T07:09:24.411-0400: 346869.362: [FinalReference, 1633 refs, 0.0025592 secs]2015-10-18T07:09:24.414-0400: 346869.365: [PhantomReference, 6 refs, 21 refs, 0.0040350 secs]2015-10-18T07:09:24.418-0400: 346869.369: [JNI Weak Reference, 0.0001088 secs], 0.0130411 secs] 2015-10-18T07:09:24.419-0400: 346869.370: [Unloading 2015-10-18T07:09:24.420-0400: 346869.371: [System Dictionary Unloading, 0.0001458 secs] 2015-10-18T07:09:24.420-0400: 346869.371: [Parallel Unloading, 0.0153302 secs] 2015-10-18T07:09:24.435-0400: 346869.386: [Deallocate Metadata, 0.0000526 secs], 0.0159394 secs], 115.5038062 secs] with the long Finalize Mark phase could be fixed for newer VM version because since that time significant improvements in that code were made (much better parallelization). If you are experiencing the same problem, it would probably be best to start over with the analysis (with new log output as a starting point) after so much time has passed. Another potential problem *could* be what is described by https://bugs.openjdk.java.net/browse/JDK-8205353; one could try lowering G1SATBBufferEnqueueingThresholdPercent to 0 - but please do not add random flags without proper analysis of potential causes and looking if the problem still occurs with a more recent jdk >= 9 first. Thanks, Thomas > > -- > Shunyun Wong > ????????????????????????? > *it was the best of times, it was the worst of times* > > _______________________________________________ > hotspot-gc-use mailing list > hotspot-gc-use at openjdk.java.net > https://mail.openjdk.java.net/mailman/listinfo/hotspot-gc-use > From java at elyograg.org Tue Oct 15 10:11:57 2019 From: java at elyograg.org (Shawn Heisey) Date: Tue, 15 Oct 2019 04:11:57 -0600 Subject: Can OOME reason be provided to -XX:OnOutOfMemoryError script? Message-ID: Can the OOME reason be provided to the script specified by -XX:OnOutOfMemoryError ? I am a committer on the Apache Lucene-Solr project. We are using -XX:OnOutOfMemoryError in our startup script. The problem we run into is that the OOME exception is frequently NOT logged by our application, because it may occur in a part of the program where exceptions are swallowed rather than being logged ... and when this happens, we have no way of knowing WHY the OOME occured. Certain options are available on -XX:OnOutOfMemoryError to provide useful information to the script, with %p for the PID being the one most often used. I would like the OOME reason (things like "Java heap memory" or "unable to create native thread") to be sent to the script as well, so we can log it. Is that currently possible? I'm aware that this particular mailing list might not be the right place for this question ... but I would prefer to limit the number of lists that I join. Thanks, Shawn From roy.sunny.zhang007 at gmail.com Thu Oct 17 06:15:35 2019 From: roy.sunny.zhang007 at gmail.com (Roy Zhang) Date: Thu, 17 Oct 2019 14:15:35 +0800 Subject: Question for MinMetaspaceFreeRatio and MaxMetaspaceFreeRatio Message-ID: Dear JVM experts, Regarding to https://gist.github.com/erikdw/e730a1249887bb4462f226658351cfa4, we can set -XX:MinMetaspaceFreeRatio=0 & -XX:MaxMetaspaceFreeRatio=100 to disable dynamic resize metaSpace (changing high-water mark) which could trigger full gc. But when i set the following metaspace related java opts(MetaspaceSize and MaxMetaspacesize is 2g), reserved MetaspaceSize is still 1G which is same as the default behavior(didn't set any metaSpace related java opts, MetaspaceSize is ~20M). Could u please kindly provide help? Thx in advance! jcmd 15958 GC.heap_info 15958: garbage-first heap total 10485760K, used 2490178K [0x0000000540000000, 0x0000000540405000, 0x00000007c0000000) region size 4096K, 552 young (2260992K), 0 survivors (0K) * Metaspace used 25746K, capacity 26634K, committed 26752K, reserved 1073152K* class space used 2227K, capacity 2358K, committed 2432K, reserved 1048576K *Metaspace java opts:* -XX:MetaspaceSize=2g -XX:MaxMetaspaceSize=2g -XX:MinMetaspaceFreeRatio=0 -XX:MaxMetaspaceFreeRatio=100 *JVM version*: openjdk version "1.8.0_201" OpenJDK Runtime Environment (build 1.8.0_201-b09) OpenJDK 64-Bit Server VM (build 25.201-b09, mixed mode) *Linux version:* 4.14.123-111.109.amzn2.x86_64 Thanks, Roy -------------- next part -------------- An HTML attachment was scrubbed... URL: From thomas.stuefe at gmail.com Thu Oct 17 07:14:33 2019 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Thu, 17 Oct 2019 09:14:33 +0200 Subject: Question for MinMetaspaceFreeRatio and MaxMetaspaceFreeRatio In-Reply-To: References: Message-ID: Hi Roy, MaxMetaspaceSize is just a soft limit capping the maximum memory size committed metaspace can take. In short it works like that (with UseCompressedClassPointers= true which is the default): Metaspace is divided into two parts, the "compressed class space" and the "non-class" metaspace. The former needs to be pre-reserved as a contiguous memory range, for technical reasons. By default that range is 1G (note, only reserved memory, not yet committed). The latter is allocated on demand. Basically, this is a chain of memory mappings which grows as new metaspace is allocated. So it is tiny at start and will grow (add new mappings as needed). This means that for the non-class metaspace part, reserved and committed should be close together. That explains the total reserved size of 1073152K which is (1G + a little bit). In reality, outside of 32bit VMs, this number is not of much concern, the committed size is more important. When the metaspace grows and new memory is committed, at periodic intervals the VM will attempt a GC to clean up remnants of unreachable classes and free Metaspace. This is to prevent a too fast growth of Metaspace. By setting XX:MaxMetaspaceFreeRatio, you basically disable this mechanism. This means metaspace committed space will grow up to MaxMetaspaceSize (if that is given) and then do a GC. You can use jcmd VM.metaspace to look at the details. See also: https://stuefe.de/posts/metaspace/analyze-metaspace-with-jcmd/ Cheers, Thomas On Thu, Oct 17, 2019 at 8:16 AM Roy Zhang wrote: > Dear JVM experts, > > Regarding to > https://gist.github.com/erikdw/e730a1249887bb4462f226658351cfa4, we can > set -XX:MinMetaspaceFreeRatio=0 & -XX:MaxMetaspaceFreeRatio=100 to disable > dynamic resize metaSpace (changing high-water mark) which could trigger > full gc. But when i set the following metaspace related java > opts(MetaspaceSize and MaxMetaspacesize is 2g), reserved MetaspaceSize is > still 1G which is same as the default behavior(didn't set any metaSpace > related java opts, MetaspaceSize is ~20M). > Could u please kindly provide help? Thx in advance! > > jcmd 15958 GC.heap_info > 15958: > garbage-first heap total 10485760K, used 2490178K [0x0000000540000000, > 0x0000000540405000, 0x00000007c0000000) > region size 4096K, 552 young (2260992K), 0 survivors (0K) > * Metaspace used 25746K, capacity 26634K, committed 26752K, reserved > 1073152K* > class space used 2227K, capacity 2358K, committed 2432K, reserved > 1048576K > > *Metaspace java opts:* > -XX:MetaspaceSize=2g > -XX:MaxMetaspaceSize=2g > -XX:MinMetaspaceFreeRatio=0 > -XX:MaxMetaspaceFreeRatio=100 > > *JVM version*: > openjdk version "1.8.0_201" > OpenJDK Runtime Environment (build 1.8.0_201-b09) > OpenJDK 64-Bit Server VM (build 25.201-b09, mixed mode) > > *Linux version:* > 4.14.123-111.109.amzn2.x86_64 > > Thanks, > Roy > _______________________________________________ > hotspot-gc-use mailing list > hotspot-gc-use at openjdk.java.net > https://mail.openjdk.java.net/mailman/listinfo/hotspot-gc-use > -------------- next part -------------- An HTML attachment was scrubbed... URL: