From torsakxx at gmail.com Sun Feb 23 21:54:51 2020 From: torsakxx at gmail.com (=?UTF-8?Q?R=C3=A9mi_p?=) Date: Sun, 23 Feb 2020 21:54:51 +0000 Subject: Troubeshooting scanned cards in remembered sets Message-ID: I am looking into reducing the young collection pause times in my Java 11 application, particularly in the Scan RS step. AFAIK the duration of this phase is linear in the sum of sizes of the remembered sets of the regions in the collection set (=old->young region references). Is there a built-in or recommended way to troubleshoot where all the old->young references are coming from? I was thinking about patching the hotspot code to log the object types of randomly sampled old->young links in order to give me some hints, but I am wondering if there is an easier/less intrusive solution. Thanks. -------------- next part -------------- An HTML attachment was scrubbed... URL: From mohdshaban000123 at gmail.com Mon Feb 24 07:45:07 2020 From: mohdshaban000123 at gmail.com (mohd shaban) Date: Mon, 24 Feb 2020 13:15:07 +0530 Subject: How to do force full garbage collection when heap usage goes beyond a certain threshold Message-ID: Hi Team, Is there a way to force GC when heap usage goes beyond a certain threshold?. I am using G1GC Algorithm. Thanks, Mohd Shaban -------------- next part -------------- An HTML attachment was scrubbed... URL: From thomas.schatzl at oracle.com Mon Feb 24 08:56:46 2020 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Mon, 24 Feb 2020 09:56:46 +0100 Subject: How to do force full garbage collection when heap usage goes beyond a certain threshold In-Reply-To: References: Message-ID: <717860d1-dd8e-c1f7-9340-b3c24e4d6186@oracle.com> Hi, On 24.02.20 08:45, mohd shaban wrote: > Hi Team, > > Is there a way to force GC when heap usage goes beyond a certain threshold?. > I am using G1GC Algorithm. > > Thanks, > Mohd Shaban > G1 fully supports the MemoryMXBean API [0] to monitor memory usage either via polling or notifications. You can issue a System.gc() in response to this information. It forces some kind of GC. Hth, Thomas [0] https://docs.oracle.com/javase/10/docs/api/java/lang/management/MemoryMXBean.html From pasthelod at gmail.com Mon Feb 24 09:08:35 2020 From: pasthelod at gmail.com (Pas) Date: Mon, 24 Feb 2020 10:08:35 +0100 Subject: How to do force full garbage collection when heap usage goes beyond a certain threshold In-Reply-To: References: Message-ID: Hello, Are you familiar with how Initiating Heap Occupancy Percent works in G1? Do you want something else? https://docs.oracle.com/javase/9/gctuning/garbage-first-garbage-collector.htm#JSGCT-GUID-572C9203-AB27-46F1-9D33-42BA4F3C6BF3 https://docs.oracle.com/en/java/javase/13/gctuning/garbage-first-garbage-collector.html#GUID-AC383806-7FA7-4698-8B92-4FD092B9F368 Pas On Mon, Feb 24, 2020, 08:45 mohd shaban wrote: > Hi Team, > > Is there a way to force GC when heap usage goes beyond a certain > threshold?. > I am using G1GC Algorithm. > > Thanks, > Mohd Shaban > _______________________________________________ > hotspot-gc-use mailing list > hotspot-gc-use at openjdk.java.net > https://mail.openjdk.java.net/mailman/listinfo/hotspot-gc-use > -------------- next part -------------- An HTML attachment was scrubbed... URL: From thomas.schatzl at oracle.com Mon Feb 24 09:57:29 2020 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Mon, 24 Feb 2020 10:57:29 +0100 Subject: Troubeshooting scanned cards in remembered sets In-Reply-To: References: Message-ID: Hi, On 23.02.20 22:54, R?mi p wrote: > I am looking into reducing the young collection pause times in my Java > 11 application, particularly in the Scan RS step. AFAIK the duration of > this phase is linear in the sum of sizes of the remembered sets of the > regions in the collection set (=old->young region references). > > Is there a built-in or recommended way to troubleshoot where all the > old->young references are coming from? I was thinking about patching the > hotspot code to log the object types of randomly sampled old->young > links in order to give me some hints, but I am wondering if there is an > easier/less intrusive solution. > At this time there is no way official way to map remembered set entries to particular objects. You can get remembered set sizes for each region with gc+remset=trace and setting G1SummarizeRSetStatsPeriod to something > 0; the reason for this convoluted process is that in jdk11 getting remembered set statistics is expensive (fixed in 14 [2]). If you want to proceed in that direction, I recommend adding the sampling in the "rebuild remembered sets" concurrent phase [3]; hook into somewhere in the path starting with G1RebuildRemSetHeapRegionClosure::scan_for_references(). What you could try in 11 to decrease remembered set memory usage (and typically Scan RS time a bit) is increasing the use of sparse PRTs by manually increasing the thresholds when using them. I.e. set G1RSetSparseRegionEntries to something much higher than the defaults (4/8/12/16/20/24 entries for 1-32M regions respectively). The JDK-8223162 change [0] in JDK13 does that for you, selecting better defaults. Another option is to increase default region size, but I assume you already maxed it out (but you did not describe your previous attempts). If you can try jdk14, we reimplemented the remembered set scan work distribution method [1], which can yield very significant pause time improvements. We've seen pause times reduced to 60% of baseline (jdk13 to jdk14) on benchmarks with very large remembered sets. Hth, Thomas [0] https://bugs.openjdk.java.net/browse/JDK-8223162 [1] https://bugs.openjdk.java.net/browse/JDK-8213108 [2] https://bugs.openjdk.java.net/browse/JDK-8233919 [3] https://bugs.openjdk.java.net/browse/JDK-8180415 From leo.korinth at oracle.com Mon Feb 24 13:42:02 2020 From: leo.korinth at oracle.com (Leo Korinth) Date: Mon, 24 Feb 2020 14:42:02 +0100 Subject: How to do force full garbage collection when heap usage goes beyond a certain threshold In-Reply-To: References: Message-ID: <376bef8d-d711-df11-4bcd-8e67ef7b830a@oracle.com> Hi Mohd, You probably do not want to force a a full gc with g1. G1 is designed so that you should be abled to do young/mixed collections and never have to do a full one. Your probably ought to be on running a full gc as few times as possible. Forcing premature full collections will probably add latency, reduce throughput and mess with valuable statistics for meeting deadlines. Also, before Java 10, full gc for g1 was single threaded. If you still have a good use case for it after this warning, you might want to weaken the explicit gc by making it a concurrent one (-XX:+ExplicitGCInvokesConcurrent). What underlying problem is it you are trying to solve? Maybe it can be solved by other means? Thanks, Leo On 24/02/2020 08:45, mohd shaban wrote: > Hi Team, > > Is there a way to force GC when heap usage goes beyond a certain threshold?. > I am using G1GC Algorithm. > > Thanks, > Mohd Shaban > > _______________________________________________ > hotspot-gc-use mailing list > hotspot-gc-use at openjdk.java.net > https://mail.openjdk.java.net/mailman/listinfo/hotspot-gc-use > From mohdshaban000123 at gmail.com Mon Feb 24 19:45:35 2020 From: mohdshaban000123 at gmail.com (mohd shaban) Date: Tue, 25 Feb 2020 01:15:35 +0530 Subject: How to do force full garbage collection when heap usage goes beyond a certain threshold In-Reply-To: References: <376bef8d-d711-df11-4bcd-8e67ef7b830a@oracle.com> Message-ID: Hi Leo , Actually we set up alerts for heap utilisation and we are getting to many alerts. So we have done the analysis and found more heap utilisation and once GC happened it backs to normal but suppose my alert scheduler runs at 8:00 and at the same heap usage reach to 90 then we get the alert, But when i monitor the heap usage , after 8:02 GC happened and it back to normal usage means 50%. So that why we want if heap usage reaches to 80-85% we trigger the GC so that we can't get the alerts. Please suggest how can we achieve this? Thanks, Mohd Shaban On Mon, Feb 24, 2020 at 11:09 PM mohd shaban wrote: > Hi Leo , > > Actually we set up alerts for heap utilisation and we are getting to many > alerts. > So we have done the analysis and found more heap utilisation and once GC > happened it backs to normal but suppose my alert scheduler runs at 8:00 and > at the same heap usage reach to 90 then we get the alert, But when i > monitor the heap usage , after 8:02 GC happened and it back to normal usage > means 50%. > > So that why we want if heap usage reaches to 80-85% we trigger the GC so > that we can't get the alerts. > > Please suggest how can we achieve this? > > Thanks, > Mohd Shaban > > On Mon, Feb 24, 2020 at 7:13 PM Leo Korinth > wrote: > >> Hi Mohd, >> >> You probably do not want to force a a full gc with g1. G1 is designed so >> that you should be abled to do young/mixed collections and never have to >> do a full one. >> >> Your probably ought to be on running a full gc as few times as possible. >> Forcing premature full collections will probably add latency, reduce >> throughput and mess with valuable statistics for meeting deadlines. >> Also, before Java 10, full gc for g1 was single threaded. >> >> If you still have a good use case for it after this warning, you might >> want to weaken the explicit gc by making it a concurrent one >> (-XX:+ExplicitGCInvokesConcurrent). >> >> What underlying problem is it you are trying to solve? Maybe it can be >> solved by other means? >> >> Thanks, >> Leo >> >> >> On 24/02/2020 08:45, mohd shaban wrote: >> > Hi Team, >> > >> > Is there a way to force GC when heap usage goes beyond a certain >> threshold?. >> > I am using G1GC Algorithm. >> > >> > Thanks, >> > Mohd Shaban >> > >> > _______________________________________________ >> > hotspot-gc-use mailing list >> > hotspot-gc-use at openjdk.java.net >> > https://mail.openjdk.java.net/mailman/listinfo/hotspot-gc-use >> > >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mohdshaban000123 at gmail.com Mon Feb 24 19:46:07 2020 From: mohdshaban000123 at gmail.com (mohd shaban) Date: Tue, 25 Feb 2020 01:16:07 +0530 Subject: How to do force full garbage collection when heap usage goes beyond a certain threshold In-Reply-To: References: Message-ID: Hi Pas , If we are not using "XX:InitiatingHeapOccupancyPercent" this JVM arguments , then still default value of that parameter is 45% so when Heap usage reaches to 45 it should trigger the GC but it is not triggering the GC. Or do we need to set this parameter then it will work?. And one more thing i want to understand , is it recommended to use "UseStringDeduplication" jvm arguments. if yes then is there any disadvantage if using this option? Thanks, Mohd Shaban On Mon, Feb 24, 2020 at 2:38 PM Pas wrote: > Hello, > > Are you familiar with how Initiating Heap Occupancy Percent works in G1? > Do you want something else? > > > > https://docs.oracle.com/javase/9/gctuning/garbage-first-garbage-collector.htm#JSGCT-GUID-572C9203-AB27-46F1-9D33-42BA4F3C6BF3 > > > https://docs.oracle.com/en/java/javase/13/gctuning/garbage-first-garbage-collector.html#GUID-AC383806-7FA7-4698-8B92-4FD092B9F368 > > > > Pas > > > > On Mon, Feb 24, 2020, 08:45 mohd shaban > wrote: > >> Hi Team, >> >> Is there a way to force GC when heap usage goes beyond a certain >> threshold?. >> I am using G1GC Algorithm. >> >> Thanks, >> Mohd Shaban >> _______________________________________________ >> hotspot-gc-use mailing list >> hotspot-gc-use at openjdk.java.net >> https://mail.openjdk.java.net/mailman/listinfo/hotspot-gc-use >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From pasthelod at gmail.com Mon Feb 24 19:49:39 2020 From: pasthelod at gmail.com (Pas) Date: Mon, 24 Feb 2020 20:49:39 +0100 Subject: How to do force full garbage collection when heap usage goes beyond a certain threshold In-Reply-To: References: Message-ID: Hello, I recommend CCing the list, as I'm just a regular JVM user, and it's not exactly unlikely that my advice is sub par. :) On Mon, 24 Feb 2020 at 18:49, mohd shaban wrote: > > Hi Pas , > > If we are not using "XX:InitiatingHeapOccupancyPercent" this JVM arguments , then still default value of that parameter is 45% so when Heap usage reaches to 45 it should trigger the GC but it is not triggering the GC. Or do we need to set this parameter then it will work?. Which version are you using exactly? Also do you have GC logging enabled? (-verbose:gc -XX:+PrintGCDetails) How big is the heap? (Do you use Xms and Xmx? Do you set them to equal value?) Furthermore, IHOP is defined in terms of "Old Gen", but there is also the "Young Gen" ("Eden" and "Survivor" regions), and Metaspace too, so 45% of "Old Gen" requires quite a few GC cycles which gradually move the surviving objects to the "Old Gen", and when "Old Gen" reaches 45% then you should see the beginning of a Marking phase for the Old Gen. (Or more likely a Mixed collection.) > > > And one more thing i want to understand , is it recommended to use "UseStringDeduplication" jvm arguments. if yes then is there any disadvantage if using this option? > If your application has a lot of identical strings, then yes, it's recommended. But it has a cost to scan all the strings (that survived a few GCs already). Of course if you have good monitoring/telemetry, then just flip it on, measure, and decide based on your measurement. > Thanks, > Mohd Shaban Pas > > On Mon, Feb 24, 2020 at 2:38 PM Pas wrote: >> >> Hello, >> >> Are you familiar with how Initiating Heap Occupancy Percent works in G1? Do you want something else? >> >> >> https://docs.oracle.com/javase/9/gctuning/garbage-first-garbage-collector.htm#JSGCT-GUID-572C9203-AB27-46F1-9D33-42BA4F3C6BF3 >> >> https://docs.oracle.com/en/java/javase/13/gctuning/garbage-first-garbage-collector.html#GUID-AC383806-7FA7-4698-8B92-4FD092B9F368 >> >> >> >> Pas >> >> >> >> On Mon, Feb 24, 2020, 08:45 mohd shaban wrote: >>> >>> Hi Team, >>> >>> Is there a way to force GC when heap usage goes beyond a certain threshold?. >>> I am using G1GC Algorithm. >>> >>> Thanks, >>> Mohd Shaban >>> _______________________________________________ >>> hotspot-gc-use mailing list >>> hotspot-gc-use at openjdk.java.net >>> https://mail.openjdk.java.net/mailman/listinfo/hotspot-gc-use From ecki at zusammenkunft.net Mon Feb 24 21:03:28 2020 From: ecki at zusammenkunft.net (Bernd Eckenfels) Date: Mon, 24 Feb 2020 21:03:28 +0000 Subject: How to do force full garbage collection when heap usage goes beyond a certain threshold In-Reply-To: References: <376bef8d-d711-df11-4bcd-8e67ef7b830a@oracle.com> , Message-ID: Hello Mohd, Would be good if you can upload sample GC logs which show the issue. If a mixed collection does not clean up your heap but a FullGC does you might need to look into soft references or direct buffers. If you have generally smallish heap one option would be to go to ParallelOld GC, it is less sensitive to fragmentation. Gruss Bernd -- http://bernd.eckenfels.net ________________________________ Von: hotspot-gc-use im Auftrag von mohd shaban Gesendet: Monday, February 24, 2020 8:45:35 PM An: Leo Korinth ; hotspot-gc-use at openjdk.java.net Betreff: Re: How to do force full garbage collection when heap usage goes beyond a certain threshold Hi Leo , Actually we set up alerts for heap utilisation and we are getting to many alerts. So we have done the analysis and found more heap utilisation and once GC happened it backs to normal but suppose my alert scheduler runs at 8:00 and at the same heap usage reach to 90 then we get the alert, But when i monitor the heap usage , after 8:02 GC happened and it back to normal usage means 50%. So that why we want if heap usage reaches to 80-85% we trigger the GC so that we can't get the alerts. Please suggest how can we achieve this? Thanks, Mohd Shaban On Mon, Feb 24, 2020 at 11:09 PM mohd shaban > wrote: Hi Leo , Actually we set up alerts for heap utilisation and we are getting to many alerts. So we have done the analysis and found more heap utilisation and once GC happened it backs to normal but suppose my alert scheduler runs at 8:00 and at the same heap usage reach to 90 then we get the alert, But when i monitor the heap usage , after 8:02 GC happened and it back to normal usage means 50%. So that why we want if heap usage reaches to 80-85% we trigger the GC so that we can't get the alerts. Please suggest how can we achieve this? Thanks, Mohd Shaban On Mon, Feb 24, 2020 at 7:13 PM Leo Korinth > wrote: Hi Mohd, You probably do not want to force a a full gc with g1. G1 is designed so that you should be abled to do young/mixed collections and never have to do a full one. Your probably ought to be on running a full gc as few times as possible. Forcing premature full collections will probably add latency, reduce throughput and mess with valuable statistics for meeting deadlines. Also, before Java 10, full gc for g1 was single threaded. If you still have a good use case for it after this warning, you might want to weaken the explicit gc by making it a concurrent one (-XX:+ExplicitGCInvokesConcurrent). What underlying problem is it you are trying to solve? Maybe it can be solved by other means? Thanks, Leo On 24/02/2020 08:45, mohd shaban wrote: > Hi Team, > > Is there a way to force GC when heap usage goes beyond a certain threshold?. > I am using G1GC Algorithm. > > Thanks, > Mohd Shaban > > _______________________________________________ > hotspot-gc-use mailing list > hotspot-gc-use at openjdk.java.net > https://mail.openjdk.java.net/mailman/listinfo/hotspot-gc-use > -------------- next part -------------- An HTML attachment was scrubbed... URL: From thomas.schatzl at oracle.com Tue Feb 25 09:54:51 2020 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Tue, 25 Feb 2020 10:54:51 +0100 Subject: How to do force full garbage collection when heap usage goes beyond a certain threshold In-Reply-To: References: <376bef8d-d711-df11-4bcd-8e67ef7b830a@oracle.com> Message-ID: <28b80aee-8d44-61aa-8e71-31788e6f40cf@oracle.com> Hi, On 24.02.20 22:03, Bernd Eckenfels wrote: > Hello Mohd, > > Would be good if you can upload sample GC logs which show the issue. If > a mixed collection does not clean up your heap but a FullGC does you > might need to look into soft references or direct buffers. > > If you have generally smallish heap one option would be to go to > ParallelOld GC, it is less sensitive to fragmentation. > > Gruss > Bernd > -- > http://bernd.eckenfels.net > ------------------------------------------------------------------------ > *Von:* hotspot-gc-use im > Auftrag von mohd shaban > *Gesendet:* Monday, February 24, 2020 8:45:35 PM > *An:* Leo Korinth ; > hotspot-gc-use at openjdk.java.net > *Betreff:* Re: How to do force full garbage collection when heap usage > goes beyond a certain threshold > Hi Leo , > > Actually we set up alerts for heap utilisation and we are getting to > many alerts. > So we have done the analysis and found more heap utilisation and once GC > happened it backs to normal but suppose my alert scheduler runs at 8:00 > and at the same heap usage reach to 90 then we get the alert, But when i > monitor the heap usage , after 8:02 GC happened and it back to normal > usage means 50%. > > So that why we want if heap usage reaches to 80-85% we trigger the GC so > that we can't get the alerts. > > Please suggest how can we achieve this? by default, in jdk9+ targets 90% heap occupancy (100 - G1ReservePercent) to be the threshold at which old generation should be reclaimed. I.e. G1 will automatically start liveness analysis in the old gen so that at (roughly) 90% of heap usage it will start cleaning out old gen. So unless you get full gcs (or evacuation failures, i.e. "to-space exhausted") this behavior is normal with default options, and could likely be fixed by increasing the G1ReservePercent value (default 10). GC logs, options used and JDK version used would help a lot to further diagnose the issue. Thanks, Thomas