From ni.ferraro at gmail.com Wed Oct 11 16:49:29 2017 From: ni.ferraro at gmail.com (Nicola Ferraro) Date: Wed, 11 Oct 2017 18:49:29 +0200 Subject: G1 pauses and number of application threads Message-ID: Hi, I've started to look at G1 garbage collector internals and I would like to know if the performance of G1 are affected by the number of threads in the application. Doing a STW means stopping all application threads, and this suggests me that the more threads you have in your application, the longer STW pauses will be. I've done some basic tests (from 1 thread to 10, 100, 1000) and I'm not able to see a clear pattern. But this may be due to my ignorance on the topic. I'm collecting data using "-XX:+PrintGCApplicationStoppedTime -XX:+PrintSafepointStatistics -XX:PrintSafepointStatisticsCount=1" on JDK 8. Is it true that the number of threads slow down G1 GC? Regards, Nicola Ferraro -------------- next part -------------- An HTML attachment was scrubbed... URL: From stefan.johansson at oracle.com Thu Oct 12 16:04:55 2017 From: stefan.johansson at oracle.com (Stefan Johansson) Date: Thu, 12 Oct 2017 18:04:55 +0200 Subject: G1 pauses and number of application threads In-Reply-To: References: Message-ID: Hi Nicola, See my comments below. On 2017-10-11 18:49, Nicola Ferraro wrote: > Hi, > I've started to look at G1 garbage collector internals and I would > like to know if the performance of G1 are affected by the number of > threads in the application. Doing a STW means stopping all application > threads, and this suggests me that the more threads you have in your > application, the longer STW pauses will be. > > I've done some basic tests (from 1 thread to 10, 100, 1000) and I'm > not able to see a clear pattern. But this may be due to my ignorance > on the topic. I'm collecting data using > "-XX:+PrintGCApplicationStoppedTime -XX:+PrintSafepointStatistics > -XX:PrintSafepointStatisticsCount=1" on JDK 8. So this is true in the sense that all your application threads have to be stopped before the STW work can be started. The time it takes to get the threads stopped is not only connected to the number of threads. Having more threads might very well take a bit longer to stop, but what the threads are doing is also a factor. So you should not really expect to find a clear pattern here that you can apply to all applications. > > Is it true that the number of threads slow down G1 GC? Not really, as you know the application threads are stopped during the STW pauses, so the number of application threads don't really affect G1 that much while doing most of the GC work. One way the application threads do affect the GC pause (true for all GCs, most of this is) is that each thread generates some work since the GC need to ensure that objects used by the thread are kept alive. But once that is done the GC runs without being affected by the application threads. Another way the application threads affect the GC is during the concurrent phase, where G1 mark through the Old regions in the heap without stopping the application threads. We usually see it as the GC affecting the application threads and not the other way around. In a scenario where a lot of application threads are running the GC threads will get less CPU time to do its work, and in that sense run slower. Hope this helps, Cheers, Stefan > > Regards, > Nicola Ferraro > > > _______________________________________________ > hotspot-gc-use mailing list > hotspot-gc-use at openjdk.java.net > http://mail.openjdk.java.net/mailman/listinfo/hotspot-gc-use -------------- next part -------------- An HTML attachment was scrubbed... URL: From amith.pawar at gmail.com Tue Oct 24 06:24:34 2017 From: amith.pawar at gmail.com (amith pawar) Date: Tue, 24 Oct 2017 11:54:34 +0530 Subject: Fwd: GC Help In-Reply-To: References: Message-ID: Hi all I have following doubt about GC phase and can anyone suggest whether thinking in this direction is right or not ?. Current observation is that whenever a new is called in JAVA, memory is allocated to that object and cleared with zero. This is similar to ?CALLOC? in C for the requested amount of size. I was testing with SPECJBB and found that millions of objects are created and GC activity is seen for every 10-15 seconds. Allocated heap size is around 28 GB. This means 24-26GB of memory is consumed within 10-15 seconds and millions objects are created in this time window. I have copied below few lines from GC log file for reference. 2017-08-08T06:30:30.243-0500: 449.366: [GC (Allocation Failure) [PSYoungGen: 26427392K->890612K(27369472K)] 26634616K->1097837K(29466624K), 0.2550044 secs] [Times: user=3.43 sys=0.00, real=0.26 secs] 2017-08-08T06:30:44.136-0500: 463.260: [GC (Allocation Failure) [PSYoungGen: 27318004K->919195K(27369472K)] 27525229K->1126419K(29466624K), 0.2509147 secs] [Times: user=3.32 sys=0.00, real=0.25 secs] 2017-08-08T06:30:55.167-0500: 474.290: [GC (Allocation Failure) [PSYoungGen: 27346587K->935445K(27369472K)] 27553811K->1142669K(29466624K), 0.2487470 secs] [Times: user=3.40 sys=0.00, real=0.25 secs] 2017-08-08T06:31:05.805-0500: 484.929: [GC (Allocation Failure) [PSYoungGen: 27362837K->920059K(27369472K)] 27570061K->1127283K(29466624K), 0.2394722 secs] [Times: user=3.23 sys=0.00, real=0.24 secs] Is it good to have bulk zeroing feature/pass in GC where once sweeping is done then do bulk zero ? If this is done then during allocation step zeroing is not required as it is done in GC itself. Not sure how much time GC will take for this. I may not be the first person to think this way and thought to ask here. Does JAVA spec allows this (such pass in GC)? if so any pointers on that. It will help me to understand more. Thanks Amit Pawar -------------- next part -------------- An HTML attachment was scrubbed... URL: From aph at redhat.com Tue Oct 24 09:34:30 2017 From: aph at redhat.com (Andrew Haley) Date: Tue, 24 Oct 2017 10:34:30 +0100 Subject: Fwd: GC Help In-Reply-To: References: Message-ID: <4c9e98cf-e9b4-7807-56a4-99b5402e8243@redhat.com> On 24/10/17 07:24, amith pawar wrote: > Is it good to have bulk zeroing feature/pass in GC where once sweeping is > done then do bulk zero ? If this is done then during allocation step > zeroing is not required as it is done in GC itself. > > Not sure how much time GC will take for this. > > I may not be the first person to think this way and thought to ask here. > Does JAVA spec allows this (such pass in GC)? if so any pointers on that. > It will help me to understand more. It's called -XX:+ZeroTLAB. See also ReduceBulkZeroing. It may or may not be effective: zeroing at allocation time is fairly cache friendly, whereas zeroing when allocating a new TLAB isn't going to be. Some architectures have cache-missing bulk zeroing, which might help. -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From hohensee at amazon.com Wed Oct 25 12:26:27 2017 From: hohensee at amazon.com (Hohensee, Paul) Date: Wed, 25 Oct 2017 12:26:27 +0000 Subject: GC Help Message-ID: Also, the server compiler makes every effort to remove zeroing fields if the constructor initializes them. Much/most zeroing ends up being elided. Paul On 10/24/17, 2:35 AM, "hotspot-gc-dev on behalf of Andrew Haley" wrote: On 24/10/17 07:24, amith pawar wrote: > Is it good to have bulk zeroing feature/pass in GC where once sweeping is > done then do bulk zero ? If this is done then during allocation step > zeroing is not required as it is done in GC itself. > > Not sure how much time GC will take for this. > > I may not be the first person to think this way and thought to ask here. > Does JAVA spec allows this (such pass in GC)? if so any pointers on that. > It will help me to understand more. It's called -XX:+ZeroTLAB. See also ReduceBulkZeroing. It may or may not be effective: zeroing at allocation time is fairly cache friendly, whereas zeroing when allocating a new TLAB isn't going to be. Some architectures have cache-missing bulk zeroing, which might help. -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From Rohit.Mohta at aexp.com Mon Oct 30 19:15:08 2017 From: Rohit.Mohta at aexp.com (Rohit Mohta) Date: Mon, 30 Oct 2017 19:15:08 +0000 Subject: Continuous Full GC - Java 8 Message-ID: Hi All, Attached GC log file for one our JVM process. Any idea why this is happening? It stopped after we restarted this process. Following is our observation * Any object allocation instruction is triggering a Full GC * As all object allocation (memory allocation) is done in Eden Space - for some reason fails continously. * We could see 452 Full GC in 2 mins. * In the attached log, we could see the below lines. What we notice is, YoungGen has 0K memory allocation, and alternatively we have 64K or so memory allocation. 2017-10-28T20:30:32.207+0000: 382100.678: [Full GC (Allocation Failure) [PSYoungGen: 0K->0K(253952K)] [ParOldGen: 146546K->146546K(348160K)] 146546K->146546K(602112K), [Metaspace: 45647K->45647K(1091584K)], 0.1865769 secs] [Times: user=0.87 sys=0.00, real=0.19 secs] 2017-10-28T20:30:32.401+0000: 382100.872: [Full GC (Allocation Failure) [PSYoungGen: 160K->0K(253952K)] [ParOldGen: 146546K->146547K(348160K)] 146706K->146547K(602112K), [Metaspace: 45647K->45647K(1091584K)], 0.1871330 secs] [Times: user=0.88 sys=0.00, real=0.18 secs] 2017-10-28T20:30:32.592+0000: 382101.064: [Full GC (Allocation Failure) [PSYoungGen: 0K->0K(253952K)] [ParOldGen: 146547K->146547K(348160K)] 146547K->146547K(602112K), [Metaspace: 45647K->45647K(1091584K)], 0.1852237 secs] [Times: user=0.88 sys=0.01, real=0.18 secs] 2017-10-28T20:30:32.786+0000: 382101.257: [Full GC (Allocation Failure) [PSYoungGen: 192K->0K(253952K)] [ParOldGen: 146547K->146558K(348160K)] 146739K->146558K(602112K), [Metaspace: 45647K->45647K(1091584K)], 0.1879624 secs] [Times: user=0.89 sys=0.00, real=0.19 secs] 2017-10-28T20:30:32.979+0000: 382101.450: [Full GC (Allocation Failure) [PSYoungGen: 0K->0K(253952K)] [ParOldGen: 146558K->146552K(348160K)] 146558K->146552K(602112K), [Metaspace: 45647K->45647K(1091584K)], 0.2331467 secs] [Times: user=1.15 sys=0.00, real=0.24 secs] Exception in std_out: java.lang.OutOfMemoryError: Java heap space at java.nio.HeapByteBuffer.(Unknown Source) ~[?:1.8.0_121] at java.nio.ByteBuffer.allocate(Unknown Source) ~[?:1.8.0_121] Similar stack trace is also observed for a plain object allocation instructions. System Details: * Java HotSpot(TM) 64-Bit Server VM (25.121-b13) for linux-amd64 JRE (1.8.0_121-b13), built on Dec 12 2016 16:36:53 by "java_re" with gcc 4.3.0 20080428 (Red Hat 4.3.0-8) * OS: Red Hat Enterprise Linux Server release 7.3 (Maipo) Thank You Rohit GAN Development Office : +1-602-537-8983 Mail To : rohit.mohta at aexp.com (Or ping in Slack) Location : American Express Desert Ridge, OB2 - 3rd Floor Near C6, 18850 N 56th Street Phoenix, AZ 85054 "if you can't explain it simply you don't understand it well enough" - Richard Feynman (maybe?) *** AXP - INTERNAL *** American Express made the following annotations ****************************************************************************** "This message and any attachments are solely for the intended recipient and may contain confidential or privileged information. If you are not the intended recipient, any disclosure, copying, use, or distribution of the information included in this message and any attachments is prohibited. If you have received this communication in error, please notify us by reply e-mail and immediately and permanently delete this message and any attachments. Thank you." American Express a ajout? le commentaire suivant le Ce courrier et toute pi?ce jointe qu'il contient sont r?serv?s au seul destinataire indiqu? et peuvent renfermer des renseignements confidentiels et privil?gi?s. Si vous n'?tes pas le destinataire pr?vu, toute divulgation, duplication, utilisation ou distribution du courrier ou de toute pi?ce jointe est interdite. Si vous avez re?u cette communication par erreur, veuillez nous en aviser par courrier et d?truire imm?diatement le courrier et les pi?ces jointes. Merci. ****************************************************************************** -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: gc-28Oct2017.log.log.zip Type: application/x-zip-compressed Size: 78285 bytes Desc: gc-28Oct2017.log.log.zip URL: