From luke.bike at gmail.com Sun Mar 17 04:54:43 2013 From: luke.bike at gmail.com (luke) Date: Sun, 17 Mar 2013 12:54:43 +0100 Subject: GC algorithms Message-ID: Hi, I have to improve performance on an application running on hotspot. At the moment I'm using Parallel Algorithm. I'dlike to try with anotherGC algorithm (ParNew or another one) but I don't know exactly how they work. thanks in advance for any suggestions luca P.S.: my gc triggers: -Xms6g -Xmx6g -XX:MaxPermSize=512m -Dsun.rmi.dgc.client.gcInterval=2100000 -Dsun.rmi.dgc.server.gcInterval=2100000 -XX:+UseParallelOldGC -XX:+UseParallelGC -XX:MaxHeapFreeRatio=70 -XX:MinHeapFreeRatio=40 -Xverify:none -XX:+BindGCTaskThreadsToCPUs -XX:NewSize=2g -XX:MaxNewSize=2g -XX:SurvivorRatio=4 -Djava.awt.headless=true GC.log report GC Event Types: PARALLEL_SCAVENGE, PARALLEL_OLD_COMPACTING Max Heap Space: 6407872K Max Heap Occupancy: 5089332K Max Perm Space: 298624K Max Perm Occupancy: 148371K Throughput: 97% Max Pause: 83966 ms Total Pause: 145054 ms First Timestamp: 5746 ms Last Timestamp: 4744888 ms -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.openjdk.java.net/pipermail/hotspot-gc-use/attachments/20130317/6df0bf4c/attachment.html From bernd.eckenfels at googlemail.com Sun Mar 17 07:04:33 2013 From: bernd.eckenfels at googlemail.com (Bernd Eckenfels) Date: Sun, 17 Mar 2013 15:04:33 +0100 Subject: GC algorithms In-Reply-To: References: Message-ID: Hello, Am 17.03.2013 um 12:54 schrieb luke : > I have to improve performance on an application running on You first need to specify what performance improvements mean (more throughput, less overhead, shorter or more consisten Pause times) then you can better chose a GC strategy. You also should device a test benchmark where you can measure if your new goals are meet. You also need to consider different aspects and phases of your application life cycle - do they all put the same pressure on the memory? And finally a Verbose GC log file would help to analyze your system. But with 97% throughout, there is a bit to improve, but certainly not a large breakthrough. How many Logical CPUs does your Server have, how many RAM, what is the memory footprint of the app - those are typical questions which help you with the decision. I think there is typically only the parallel GC, the low pause GC and G1. You can try all 3. But if you not care about pause times you might have chosen the best setting already. Bernd From ruben.badaro at gmail.com Sun Mar 17 18:54:10 2013 From: ruben.badaro at gmail.com (=?ISO-8859-1?Q?Ruben_Badar=F3?=) Date: Mon, 18 Mar 2013 01:54:10 +0000 Subject: Garbage Collection low-impact monitoring Message-ID: Hi, I'm looking at ways of monitoring garbage collection statistics. My goals are to have close to no impact to the running process and the most detail possible. For that, I would like to avoid any type of instrumentation as I want this running constantly in production. An easy way is to parse gc logs but I remember someone reading the VM structs for the garbage collection counters somewhere. Do you guys have any examples of how to do this? I looked at the Serviceability Agent but apparently it pauses the the running process while inspecting. I was also considering (because my concern here is exclusively when running Linux) to use the /proc/pid/maps memory mapped files and read them in read-only mode from my monitoring process. Does something like this already exist or is there a fundamental reason of why it wouldn't work? Thanks Ruben -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.openjdk.java.net/pipermail/hotspot-gc-use/attachments/20130318/0ac04996/attachment.html From bernd.eckenfels at googlemail.com Sun Mar 17 18:59:43 2013 From: bernd.eckenfels at googlemail.com (Bernd Eckenfels) Date: Mon, 18 Mar 2013 02:59:43 +0100 Subject: Garbage Collection low-impact monitoring In-Reply-To: References: Message-ID: Am 18.03.2013, 02:54 Uhr, schrieb Ruben Badar? : > An easy way is to parse gc logs but I remember someone reading the VM > structs for the garbage collection counters somewhere. There is a JMX bean, but I think it is not detailed enough. I think the GC log is the most open one. I am not sure about those trace/flightrecorder stuff. It seems not to be openly available. Greetings Bernd From pasthelod at gmail.com Sun Mar 17 20:35:41 2013 From: pasthelod at gmail.com (Pas) Date: Mon, 18 Mar 2013 04:35:41 +0100 Subject: Garbage Collection low-impact monitoring In-Reply-To: References: Message-ID: Hello, On Mon, Mar 18, 2013 at 2:54 AM, Ruben Badar? wrote: > Hi, > > I'm looking at ways of monitoring garbage collection statistics. My goals > are to have close to no impact to the running process and the most detail > possible. For that, I would like to avoid any type of instrumentation as I > want this running constantly in production. > > An easy way is to parse gc logs but I remember someone reading the VM > structs for the garbage collection counters somewhere. You might be thinking about the hsperfdata_ directory, which is also used by jstat in a very similar way to what you've described. (There is a descriptor file for each JVM instance, which basically is a layout map for the VM structures you're looking for, and jstat just mmap-s the relevant memory blocks and reads from them.) > Do you guys have any examples of how to do this? I looked at the > Serviceability Agent but apparently it pauses the the running process while > inspecting. Last time I've used it, I've found no pause times for simply using jstat -gc -gccause on a local JVM. (Or more precisely, no impact of a 1 ms polling interval.) > I was also considering (because my concern here is exclusively > when running Linux) to use the /proc/pid/maps memory mapped files and read > them in read-only mode from my monitoring process. > Does something like this already exist or is there a fundamental reason of > why it wouldn't work? Well, to absolutely catch all GC events you have to chose a polling interval shorter than the shortest minor GCs, which might not be efficient compared to just consuming the verbose GC log. (But, to my knowledge the GC log is written synchronously, which might be a problem if there is not enough free memory for Linux's fscache to work its magic.) > > Thanks > Ruben -- Pas From the.6th.month at gmail.com Sun Mar 17 20:45:16 2013 From: the.6th.month at gmail.com (the.6th.month at gmail.com) Date: Mon, 18 Mar 2013 11:45:16 +0800 Subject: Garbage Collection low-impact monitoring In-Reply-To: References: Message-ID: Hi, Ruben: In my prod environment, I am monitoring gc related metrics as this, I just paste some snippet here and leave out the trivial : List gcBeans = ManagementFactory .getGarbageCollectorMXBeans(); for (GarbageCollectorMXBean bean : gcBeans) { name = makeName((new StringBuilder(JVM_PREFIX)).append( *bean.getName()*).toString()); //makeName is just a convenient method to generate a metrics name following our convention *final long count = bean.getCollectionCount(); //accumulated gc count for this type of gc indicated by bean.getName since the process starting-up* * final long time = bean.getCollectionTime(); //accumulated gc time **for this type of gc indicated by bean.getName since the process starting-up* * ..................* } and then you can make a one-minute interval timer to compute how many times any type of gc happens in one minute, and draw a graph with cacti or something. All the best, Leon On 18 March 2013 09:59, Bernd Eckenfels wrote: > Am 18.03.2013, 02:54 Uhr, schrieb Ruben Badar? : > > > An easy way is to parse gc logs but I remember someone reading the VM > > structs for the garbage collection counters somewhere. > > There is a JMX bean, but I think it is not detailed enough. I think the GC > log is the most open one. I am not sure about those trace/flightrecorder > stuff. It seems not to be openly available. > > Greetings > Bernd > _______________________________________________ > hotspot-gc-use mailing list > hotspot-gc-use at openjdk.java.net > http://mail.openjdk.java.net/mailman/listinfo/hotspot-gc-use > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.openjdk.java.net/pipermail/hotspot-gc-use/attachments/20130318/c1153201/attachment.html From luke.bike at gmail.com Mon Mar 18 03:29:28 2013 From: luke.bike at gmail.com (luke) Date: Mon, 18 Mar 2013 11:29:28 +0100 Subject: GC algorithms In-Reply-To: References: Message-ID: Thanks Bernd, Ryan I'm working on webApp hosted on Jboss AS (java6), that invoke an heavy batch job. So I'm mainly interested in throughput. Application is working on multi cpu servers and when my application is running, I've only few GB of phisical memory available (3-4 GB). We started with -Xms1g -Xmx3g and performance was very bad. So we passed to -Xms6g -Xmx6g and we set Parallel algorithm for young and old generations. Using these jvm triggers application works quite well with current data. Maybe in future I've to try other improvment in order to deal with data greater than now. I'll try with low pause GC and G1 to compare performance with parallel algorihm. Thanks for your suggestions, luca 2013/3/17 Bernd Eckenfels > Hello, > > Am 17.03.2013 um 12:54 schrieb luke : > > > I have to improve performance on an application running on > > You first need to specify what performance improvements mean (more > throughput, less overhead, shorter or more consisten Pause times) then you > can better chose a GC strategy. > > You also should device a test benchmark where you can measure if your new > goals are meet. > > You also need to consider different aspects and phases of your application > life cycle - do they all put the same pressure on the memory? > > And finally a Verbose GC log file would help to analyze your system. But > with 97% throughout, there is a bit to improve, but certainly not a large > breakthrough. > > How many Logical CPUs does your Server have, how many RAM, what is the > memory footprint of the app - those are typical questions which help you > with the decision. > > I think there is typically only the parallel GC, the low pause GC and G1. > You can try all 3. But if you not care about pause times you might have > chosen the best setting already. > > Bernd -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.openjdk.java.net/pipermail/hotspot-gc-use/attachments/20130318/c122d94e/attachment.html From chunt at salesforce.com Mon Mar 18 06:57:54 2013 From: chunt at salesforce.com (Charlie Hunt) Date: Mon, 18 Mar 2013 06:57:54 -0700 Subject: GC algorithms In-Reply-To: References: Message-ID: Hi Luke, If you are ok with the worst case pause time with parallel GC, then you will get the best throughput and lowest latency with it. CMS will have slightly higher minor GC times for the same size young gen. But you may be to avoid full GCs by tuning the (mostly) concurrent old gen collector. CMS will also require a little larger Java heap than Parallel GC. G1 is a bit of a different game. With G1 you should focus on what is the worst case pause time you can deal with and set that as the G1's pause time to try to meet. G1 is adaptable so you should not size young gen. I have also seen folks trying to compare minor GC pause times of G1 minor GC pause times. IMO, that's not a good comparison since G1 uses additional data structures to support adaptiveness and your goal is to meet a worst case pause time, not comparing minor GC times for the same sized young gen. Given your interest in GCs, you might find value in getting a copy of Java Performance. It has a GC overview section in one chapter and a full chapter on tuning the JVM step by step. hths, Charlie Sent from my iPhone On Mar 18, 2013, at 5:32 AM, "luke" > wrote: Thanks Bernd, Ryan I'm working on webApp hosted on Jboss AS (java6), that invoke an heavy batch job. So I'm mainly interested in throughput. Application is working on multi cpu servers and when my application is running, I've only few GB of phisical memory available (3-4 GB). We started with -Xms1g -Xmx3g and performance was very bad. So we passed to -Xms6g -Xmx6g and we set Parallel algorithm for young and old generations. Using these jvm triggers application works quite well with current data. Maybe in future I've to try other improvment in order to deal with data greater than now. I'll try with low pause GC and G1 to compare performance with parallel algorihm. Thanks for your suggestions, luca 2013/3/17 Bernd Eckenfels > Hello, Am 17.03.2013 um 12:54 schrieb luke >: > I have to improve performance on an application running on You first need to specify what performance improvements mean (more throughput, less overhead, shorter or more consisten Pause times) then you can better chose a GC strategy. You also should device a test benchmark where you can measure if your new goals are meet. You also need to consider different aspects and phases of your application life cycle - do they all put the same pressure on the memory? And finally a Verbose GC log file would help to analyze your system. But with 97% throughout, there is a bit to improve, but certainly not a large breakthrough. How many Logical CPUs does your Server have, how many RAM, what is the memory footprint of the app - those are typical questions which help you with the decision. I think there is typically only the parallel GC, the low pause GC and G1. You can try all 3. But if you not care about pause times you might have chosen the best setting already. Bernd _______________________________________________ hotspot-gc-use mailing list hotspot-gc-use at openjdk.java.net http://mail.openjdk.java.net/mailman/listinfo/hotspot-gc-use -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.openjdk.java.net/pipermail/hotspot-gc-use/attachments/20130318/a00e228a/attachment-0001.html From aaisinzon at guidewire.com Thu Mar 28 09:37:47 2013 From: aaisinzon at guidewire.com (Alexandre Aisinzon) Date: Thu, 28 Mar 2013 16:37:47 +0000 Subject: PredictedClassLoadCount not working Message-ID: <43E49E6EC0E84F41B98C68AB6D7820C417A8BFB6@sm-ex-01-vm.guidewire.com> Hi all I read on page 15 of http://www.myexpospace.com/JavaOne2012/SessionFiles/CON7236_PDF_7236_0001.pdf that, by default, the system dictionary was sized at 1009. Therefore, I tried to run "-XX:+UnlockExperimentalVMOptions -XX:PredictedClassLoadCount=#" with a value around 50,000. The JVM does not seem to recognize the parameter. Output of ./java -XX:+UnlockExperimentalVMOptions -XX:PredictedClassLoadCount=50000 -version is: Unrecognized VM option 'PredictedClassLoadCount=50000' Error: Could not create the Java Virtual Machine. Error: A fatal exception has occurred. Program will exit. I then ran with -verbose:class and counted that 46956 classes were allocated, which further justified the interest in sizing the system dictionary. The test was conducted on JDK 1.7 u17 64 bit for Linux. Java -version output is: java version "1.7.0_17" Java(TM) SE Runtime Environment (build 1.7.0_17-b02) Java HotSpot(TM) 64-Bit Server VM (build 23.7-b01, mixed mode) Thanks for any pointers on this seemingly interesting parameter. Alex Aisinzon -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.openjdk.java.net/pipermail/hotspot-gc-use/attachments/20130328/bc014209/attachment.html From bernd.eckenfels at googlemail.com Thu Mar 28 10:11:52 2013 From: bernd.eckenfels at googlemail.com (Bernd Eckenfels) Date: Thu, 28 Mar 2013 18:11:52 +0100 Subject: PredictedClassLoadCount not working In-Reply-To: <43E49E6EC0E84F41B98C68AB6D7820C417A8BFB6@sm-ex-01-vm.guidewire.com> References: <43E49E6EC0E84F41B98C68AB6D7820C417A8BFB6@sm-ex-01-vm.guidewire.com> Message-ID: Am 28.03.2013, 17:37 Uhr, schrieb Alexandre Aisinzon : > Output of ./java -XX:+UnlockExperimentalVMOptions > -XX:PredictedClassLoadCount=50000 -version is: The name might have changed: C:\Users\eckenfel>"\Program Files\Java\jdk1.7.0_11\bin\java.exe" -XX:+PrintFlagsFinal -XX:+UnlockExperimentalVMOptions -version | findstr Predicted intx PredictedLoadedClassCount = 0 {experimental} java version "1.7.0_11" Java(TM) SE Runtime Environment (build 1.7.0_11-b21) Java HotSpot(TM) 64-Bit Server VM (build 23.6-b04, mixed mode) However I wonder if 0 means "default" or some more intelligent calculation. Greetings Bernd -- http://itblog.eckenfels.net