From martijnhoekstra at gmail.com Fri Mar 3 12:32:33 2017 From: martijnhoekstra at gmail.com (Martijn Hoekstra) Date: Fri, 3 Mar 2017 13:32:33 +0100 Subject: encoding problem in Param Message-ID: I'm using sbt-jmh, and I have a benchmark where the encoding of the source file is not the same as my platform encoding. From the results I'm getting, I have the idea that I'm not passing the correct configuration options to JMH to correctly deal with the encoding. The problem manifests here: https://github.com/martijnhoekstra/numparsers/blob/master/src/main/scala/ValidShortBenchmarks.scala#L17 That weird looking 1 is a full width 1, +uFF11. The source file is UTF8 and my platform default encoding is cp1252. The maintainer of sbt-jmh suggested I'd ask here. How should I let JMH know what encoding to use? I also asked (a variation of) this question on Stackoverflow as http://stackoverflow.com/questions/42576773/param-value-encoding From romain.quinio at gmail.com Mon Mar 6 14:57:45 2017 From: romain.quinio at gmail.com (Romain Quinio) Date: Mon, 6 Mar 2017 15:57:45 +0100 Subject: De-doubled call tree with profiler Message-ID: Hi, I've noticed using Java Mission Control profiler that for some JMH benchmarks the call tree was "de-doubled", i.e. my benchmark method (here mypackage.generated.MyBenchmark_decodeXmlToJaxb_jmhTest.decodeXmlToJaxb) appears twice, here with a ratio of 60%/40%: java.lang.Thread.run() java.util.concurrent.ThreadPoolExecutor$Worker.run() java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor$Worker) java.util.concurrent.FutureTask.run() java.util.concurrent.Executors$RunnableAdapter.call() java.util.concurrent.FutureTask.run() org.openjdk.jmh.runner.BenchmarkHandler$BenchmarkTask.call() org.openjdk.jmh.runner.BenchmarkHandler$BenchmarkTask.call() java.lang.reflect.Method.invoke(Object, Object[]) sun.reflect.DelegatingMethodAccessorImpl.invoke(Object, Object[]) sun.reflect.GeneratedMethodAccessor7.invoke(Object, Object[]) *mypackage.generated.MyBenchmark_decodeXmlToJaxb_jmhTest.decodeXmlToJaxb_AverageTime(InfraControl, ThreadParams)* sun.reflect.NativeMethodAccessorImpl.invoke(Object, Object[]) sun.reflect.NativeMethodAccessorImpl.invoke0(Method, Object, Object[]) *mypackage.generated.MyBenchmark_decodeXmlToJaxb_jmhTest.decodeXmlToJaxb_AverageTime(InfraControl, ThreadParams)* My benchmark settings are: @Fork(value = 1, jvmArgsAppend = { "-XX:+UnlockCommercialFeatures", "-XX:+FlightRecorder", "-XX:StartFlightRecording=duration=60s,filename=./profiling-decodeXmlToJaxb.jfr,name=profile,settings=profile", "-XX:FlightRecorderOptions=settings=D:/Java/jdk1.7.0_85/jre/lib/jfr/profile.jfc,samplethreads=true" }) @Benchmark @Warmup(iterations = 15, time = 1, timeUnit = TimeUnit.SECONDS) @Measurement(iterations = 20, time = 1, timeUnit = TimeUnit.SECONDS) @State(Scope.Thread) @BenchmarkMode(Mode.AverageTime) @OutputTimeUnit(TimeUnit.MILLISECONDS) I'm curious to know what could be causing this ! Thanks, Romain From shade at redhat.com Mon Mar 6 15:07:48 2017 From: shade at redhat.com (Aleksey Shipilev) Date: Mon, 6 Mar 2017 16:07:48 +0100 Subject: De-doubled call tree with profiler In-Reply-To: References: Message-ID: <9bd5b635-0be9-4936-5c49-c06f86836511@redhat.com> Hi, On 03/06/2017 03:57 PM, Romain Quinio wrote: > I'm curious to know what could be causing this ! This is Reflection inflation. See e.g. here: https://blogs.oracle.com/buck/entry/inflation_system_properties JMH does not care which way we end up in the generated code stub, because we are supposed to enter it once per iteration. > sun.reflect.DelegatingMethodAccessorImpl.invoke(Object, Object[]) > sun.reflect.GeneratedMethodAccessor7.invoke(Object, Object[]) > *mypackage.generated.MyBenchmark_decodeXmlToJaxb_jmhTest.decodeXmlToJaxb_AverageTime(InfraControl, > ThreadParams)* This is the inflated version. > sun.reflect.NativeMethodAccessorImpl.invoke(Object, Object[]) > sun.reflect.NativeMethodAccessorImpl.invoke0(Method, Object, Object[]) > *mypackage.generated.MyBenchmark_decodeXmlToJaxb_jmhTest.decodeXmlToJaxb_AverageTime(InfraControl, > ThreadParams)* This is the "native" version. Thanks, -Aleksey From henri.tremblay at gmail.com Mon Mar 6 16:17:02 2017 From: henri.tremblay at gmail.com (Henri Tremblay) Date: Mon, 6 Mar 2017 11:17:02 -0500 Subject: AuxCounters Message-ID: Hi, I want to do something basic but haven't found a way. I played with auxiliary counters but after reading the doc, I've definitely failed to understand it it seems. I want to do this: @Benchmark @Threads(Threads.MAX) public String cacheAccess() { return cache.get(key); } And have the time when the method returns null and when if returns something else. Is there a way? Thanks, Henri From shade at redhat.com Mon Mar 6 18:21:44 2017 From: shade at redhat.com (Aleksey Shipilev) Date: Mon, 6 Mar 2017 19:21:44 +0100 Subject: encoding problem in Param In-Reply-To: References: Message-ID: <4e732933-a25f-ff67-6675-a90cb43b440d@redhat.com> Hi, Martijn, On 03/03/2017 01:32 PM, Martijn Hoekstra wrote: > The problem manifests here: > https://github.com/martijnhoekstra/numparsers/blob/master/src/main/scala/ValidShortBenchmarks.scala#L17 > > That weird looking 1 is a full width 1, +uFF11. The source file is UTF8 and > my platform default encoding is cp1252. > > The maintainer of sbt-jmh suggested I'd ask here. > > How should I let JMH know what encoding to use? Reproduced in local integration test with LC_ALL=C. The problem comes from JMH internals: it uses the text file to pass benchmark configuration from benchmark generator (at compile time) to benchmark runner (at run time). That file contains param values. Writing and reading that file back in non-UTF encoding garbles param values. There is no workaround, except accepting UTF-8 as default platform encoding, or somehow put -Dfile.encoding=... all around. I was unsuccessful, though. Submitted: https://bugs.openjdk.java.net/browse/CODETOOLS-7901910 The fix I tried today seems way too intrusive. Let me see if I can make it saner. Thanks, -Aleksey From shade at redhat.com Mon Mar 6 18:27:16 2017 From: shade at redhat.com (Aleksey Shipilev) Date: Mon, 6 Mar 2017 19:27:16 +0100 Subject: AuxCounters In-Reply-To: References: Message-ID: Hi Henry, On 03/06/2017 05:17 PM, Henri Tremblay wrote: > I want to do something basic but haven't found a way. This is not as basic as you'd think. > I played with auxiliary counters but after reading the doc, I've definitely > failed to understand it it seems. > > I want to do this: > > @Benchmark > @Threads(Threads.MAX) > public String cacheAccess() { > > return cache.get(key); > > } > > And have the time when the method returns null and when if returns > something else. > > Is there a way? No, there is no way. @AuxCounters work for Throughput/AverageTime because they can record "counters" that you can increment. If you want conditional per-sample timing measurement, that is not enough. -Aleksey From t_liang1990 at qq.com Sun Mar 5 16:54:49 2017 From: t_liang1990 at qq.com (=?gb18030?B?dF9saWFuZw==?=) Date: Mon, 6 Mar 2017 00:54:49 +0800 Subject: sometime output twice, sometime output once Message-ID: # JMH 1.17.5 (released 10 days ago) # VM version: JDK 1.8.0_66, VM 25.66-b17 # VM invoker: D:\Genuitec\MyEclipse 2016\binary\com.sun.java.jdk8.win32.x86_1.8.0.u66\jre\bin\java.exe # VM options: -Xms512m -Xmx512m -Dfile.encoding=UTF-8 # Warmup: # Measurement: 1 iterations, 1 s each # Timeout: 600 s per iteration # Threads: 1 thread, will synchronize iterations # Benchmark mode: Throughput, ops/time # Benchmark: indi.liang.pref.CommJmhTest.why1 # Run progress: 0.00% complete, ETA 00:03:02 # Fork: 1 of 1 Iteration 1: why1 0.994 ops/s Result "indi.liang.pref.CommJmhTest.why1": 0.994 ops/s # JMH 1.17.5 (released 10 days ago) # VM version: JDK 1.8.0_66, VM 25.66-b17 # VM invoker: D:\Genuitec\MyEclipse 2016\binary\com.sun.java.jdk8.win32.x86_1.8.0.u66\jre\bin\java.exe # VM options: -Xms512m -Xmx512m -Dfile.encoding=UTF-8 # Warmup: # Measurement: 1 iterations, 1 s each # Timeout: 600 s per iteration # Threads: 1 thread, will synchronize iterations # Benchmark mode: Throughput, ops/time # Benchmark: indi.liang.pref.CommJmhTest.why2 # Run progress: 0.55% complete, ETA 00:04:28 # Fork: 1 of 1 Iteration 1: why2 0.998 ops/s -------------------------------------------------------- # JMH 1.17.5 (released 10 days ago) # VM version: JDK 1.8.0_66, VM 25.66-b17 # VM invoker: D:\Genuitec\MyEclipse 2016\binary\com.sun.java.jdk8.win32.x86_1.8.0.u66\jre\bin\java.exe # VM options: -Xms512m -Xmx512m -Dfile.encoding=UTF-8 # Warmup: # Measurement: 1 iterations, 1 s each # Timeout: 600 s per iteration # Threads: 1 thread, will synchronize iterations # Benchmark mode: Throughput, ops/time # Benchmark: indi.liang.pref.CommJmhTest.why1 # Run progress: 0.00% complete, ETA 00:03:02 # Fork: 1 of 1 Iteration 1: why1 why1 0.997 ops/s Result "indi.liang.pref.CommJmhTest.why1": 0.997 ops/s # JMH 1.17.5 (released 10 days ago) # VM version: JDK 1.8.0_66, VM 25.66-b17 # VM invoker: D:\Genuitec\MyEclipse 2016\binary\com.sun.java.jdk8.win32.x86_1.8.0.u66\jre\bin\java.exe # VM options: -Xms512m -Xmx512m -Dfile.encoding=UTF-8 # Warmup: # Measurement: 1 iterations, 1 s each # Timeout: 600 s per iteration # Threads: 1 thread, will synchronize iterations # Benchmark mode: Throughput, ops/time # Benchmark: indi.liang.pref.CommJmhTest.why2 # Run progress: 0.55% complete, ETA 00:07:20 # Fork: 1 of 1 Iteration 1: why2 why2 0.998 ops/s ----------------------------------------- # JMH 1.17.5 (released 10 days ago) # VM version: JDK 1.8.0_66, VM 25.66-b17 # VM invoker: D:\Genuitec\MyEclipse 2016\binary\com.sun.java.jdk8.win32.x86_1.8.0.u66\jre\bin\java.exe # VM options: -Xms512m -Xmx512m -Dfile.encoding=UTF-8 # Warmup: # Measurement: 1 iterations, 1 s each # Timeout: 600 s per iteration # Threads: 1 thread, will synchronize iterations # Benchmark mode: Throughput, ops/time # Benchmark: indi.liang.pref.CommJmhTest.why1 # Run progress: 0.00% complete, ETA 00:03:02 # Fork: 1 of 1 Iteration 1: why1 0.999 ops/s Result "indi.liang.pref.CommJmhTest.why1": 0.999 ops/s # JMH 1.17.5 (released 10 days ago) # VM version: JDK 1.8.0_66, VM 25.66-b17 # VM invoker: D:\Genuitec\MyEclipse 2016\binary\com.sun.java.jdk8.win32.x86_1.8.0.u66\jre\bin\java.exe # VM options: -Xms512m -Xmx512m -Dfile.encoding=UTF-8 # Warmup: # Measurement: 1 iterations, 1 s each # Timeout: 600 s per iteration # Threads: 1 thread, will synchronize iterations # Benchmark mode: Throughput, ops/time # Benchmark: indi.liang.pref.CommJmhTest.why2 # Run progress: 0.55% complete, ETA 00:04:20 # Fork: 1 of 1 Iteration 1: why2 why2 0.989 ops/s From bo.zh.zhang at oracle.com Mon Mar 6 20:12:22 2017 From: bo.zh.zhang at oracle.com (Bo Zhang) Date: Mon, 6 Mar 2017 14:12:22 -0600 Subject: how to regular expression match benchmark's name if the name has "$" in it Message-ID: <8ef38c9e-daca-316d-e20c-6559ddaef619@oracle.com> Hi there, I am using JMH to run benchmarks. I accidentally find a benchmark method, whose name is "findAllIn$chars$100$1k". In this case, how can I use regex to match this name in my "java -jar ./test/benchmark.jar " command? Thanks, Bo From henri.tremblay at gmail.com Mon Mar 6 21:15:09 2017 From: henri.tremblay at gmail.com (Henri Tremblay) Date: Mon, 6 Mar 2017 16:15:09 -0500 Subject: AuxCounters In-Reply-To: References: Message-ID: Ok. That was my understanding so far. Then, I don't understand how it's calculated. Lets say I use OPERATIONS and Throughput (per second). And I increment once the counter. I will get the number of time the counter was incremented per second? And then the average time is what? 1/throughput? Now, for my original question. Will it be crazy to have a way to split executions according to an enum returned by the benchmarked method? By crazy I mean impossible. Or a boolean. Thanks, Henri On 6 March 2017 at 13:27, Aleksey Shipilev wrote: > Hi Henry, > > On 03/06/2017 05:17 PM, Henri Tremblay wrote: > > I want to do something basic but haven't found a way. > > This is not as basic as you'd think. > > > I played with auxiliary counters but after reading the doc, I've > definitely > > failed to understand it it seems. > > > > I want to do this: > > > > @Benchmark > > @Threads(Threads.MAX) > > public String cacheAccess() { > > > > return cache.get(key); > > > > } > > > > And have the time when the method returns null and when if returns > > something else. > > > > Is there a way? > > No, there is no way. @AuxCounters work for Throughput/AverageTime because > they > can record "counters" that you can increment. If you want conditional > per-sample > timing measurement, that is not enough. > > -Aleksey > > From henri.tremblay at gmail.com Tue Mar 7 02:40:35 2017 From: henri.tremblay at gmail.com (Henri Tremblay) Date: Mon, 6 Mar 2017 21:40:35 -0500 Subject: how to regular expression match benchmark's name if the name has "$" in it In-Reply-To: <8ef38c9e-daca-316d-e20c-6559ddaef619@oracle.com> References: <8ef38c9e-daca-316d-e20c-6559ddaef619@oracle.com> Message-ID: The name of the method should be the name of the method you have coded. How can you find a method "accidentally"? Are we talking about the methods generated my JMH after compilation? Anyhow, those regex should match: findAllIn.* findAllIn.*100 findAllIn\$chars\$100\$1k On 6 March 2017 at 15:12, Bo Zhang wrote: > Hi there, > > I am using JMH to run benchmarks. I accidentally find a benchmark method, > whose name is "findAllIn$chars$100$1k". > > In this case, how can I use regex to match this name in my "java -jar > ./test/benchmark.jar " command? > > Thanks, > Bo > From bo.zh.zhang at oracle.com Tue Mar 7 15:48:22 2017 From: bo.zh.zhang at oracle.com (Bo Zhang) Date: Tue, 7 Mar 2017 09:48:22 -0600 Subject: how to regular expression match benchmark's name if the name has "$" in it In-Reply-To: References: <8ef38c9e-daca-316d-e20c-6559ddaef619@oracle.com> Message-ID: <0283f37c-f613-a3aa-9459-92fa48bf3764@oracle.com> Hi Henri, To clarify, by saying "accidentally", I mean I searched benchmarks from Github repository and apply them to analyze the performance of my hardware. I happened to find one repository using "$" in its name, which I believe is not a good idea and it indeed brought me trouble on regex match. I tried findAllIn\$chars\$100\$1k before I asked you. It does not work. findAllIn.*100.*1k works. Thanks a lot for help. Best regards, Bo On 3/6/2017 8:40 PM, Henri Tremblay wrote: > The name of the method should be the name of the method you have coded. > > How can you find a method "accidentally"? Are we talking about the > methods generated my JMH after compilation? > > Anyhow, those regex should match: > findAllIn.* > findAllIn.*100 > findAllIn\$chars\$100\$1k > > > On 6 March 2017 at 15:12, Bo Zhang > wrote: > > Hi there, > > I am using JMH to run benchmarks. I accidentally find a benchmark > method, whose name is "findAllIn$chars$100$1k". > > In this case, how can I use regex to match this name in my "java > -jar ./test/benchmark.jar " command? > > Thanks, > Bo > > From henri.tremblay at gmail.com Tue Mar 7 17:54:21 2017 From: henri.tremblay at gmail.com (Henri Tremblay) Date: Tue, 7 Mar 2017 12:54:21 -0500 Subject: how to regular expression match benchmark's name if the name has "$" in it In-Reply-To: <0283f37c-f613-a3aa-9459-92fa48bf3764@oracle.com> References: <8ef38c9e-daca-316d-e20c-6559ddaef619@oracle.com> <0283f37c-f613-a3aa-9459-92fa48bf3764@oracle.com> Message-ID: Fun. findAllIn\$chars\$100\$1k does work. However, you need to put it into quotes 'findAllIn\$chars\$100\$1k' on a Linux shell. I should have mentioned that. On 7 March 2017 at 10:48, Bo Zhang wrote: > Hi Henri, > > To clarify, by saying "accidentally", I mean I searched benchmarks from > Github repository and apply them to analyze the performance of my hardware. > I happened to find one repository using "$" in its name, which I believe is > not a good idea and it indeed brought me trouble on regex match. > > I tried > > findAllIn\$chars\$100\$1k > > before I asked you. It does not work. > > findAllIn.*100.*1k works. > > Thanks a lot for help. > > Best regards, > > Bo > > > > On 3/6/2017 8:40 PM, Henri Tremblay wrote: > > The name of the method should be the name of the method you have coded. > > How can you find a method "accidentally"? Are we talking about the methods > generated my JMH after compilation? > > Anyhow, those regex should match: > findAllIn.* > findAllIn.*100 > findAllIn\$chars\$100\$1k > > > On 6 March 2017 at 15:12, Bo Zhang wrote: > >> Hi there, >> >> I am using JMH to run benchmarks. I accidentally find a benchmark method, >> whose name is "findAllIn$chars$100$1k". >> >> In this case, how can I use regex to match this name in my "java -jar >> ./test/benchmark.jar " command? >> >> Thanks, >> Bo >> > > > From ecki at zusammenkunft.net Tue Mar 7 18:51:48 2017 From: ecki at zusammenkunft.net (Bernd Eckenfels) Date: Tue, 7 Mar 2017 18:51:48 +0000 (UTC) Subject: how to regular expression match benchmark's name if the name has "$" in it In-Reply-To: <0283f37c-f613-a3aa-9459-92fa48bf3764@oracle.com> References: <8ef38c9e-daca-316d-e20c-6559ddaef619@oracle.com> <0283f37c-f613-a3aa-9459-92fa48bf3764@oracle.com> Message-ID: <1931C356834729A1.3C83A39F-25F5-4E71-8722-9F221A2822AE@mail.outlook.com> You must also quote on the shell 'bla\$fasel' Gruss Bernd -- http://bernd.eckenfels.net On Tue, Mar 7, 2017 at 5:48 PM +0100, "Bo Zhang" wrote: Hi Henri, To clarify, by saying "accidentally", I mean I searched benchmarks from Github repository and apply them to analyze the performance of my hardware. I happened to find one repository using "$" in its name, which I believe is not a good idea and it indeed brought me trouble on regex match. I tried findAllIn\$chars\$100\$1k before I asked you. It does not work. findAllIn.*100.*1k works. Thanks a lot for help. Best regards, Bo On 3/6/2017 8:40 PM, Henri Tremblay wrote: > The name of the method should be the name of the method you have coded. > > How can you find a method "accidentally"? Are we talking about the > methods generated my JMH after compilation? > > Anyhow, those regex should match: > findAllIn.* > findAllIn.*100 > findAllIn\$chars\$100\$1k > > > On 6 March 2017 at 15:12, Bo Zhang > wrote: > > Hi there, > > I am using JMH to run benchmarks. I accidentally find a benchmark > method, whose name is "findAllIn$chars$100$1k". > > In this case, how can I use regex to match this name in my "java > -jar ./test/benchmark.jar " command? > > Thanks, > Bo > > From shade at redhat.com Thu Mar 9 17:07:30 2017 From: shade at redhat.com (Aleksey Shipilev) Date: Thu, 9 Mar 2017 18:07:30 +0100 Subject: sometime output twice, sometime output once In-Reply-To: References: Message-ID: Hi, On 03/05/2017 05:54 PM, t_liang wrote: > # Run progress: 0.00% complete, ETA 00:03:02 > # Fork: 1 of 1 > Iteration 1: why1 0.999 ops/s > > # Run progress: 0.55% complete, ETA 00:04:20 > # Fork: 1 of 1 > Iteration 1: why2 why2 0.989 ops/s > There are no guarantees that @Benchmark method would get executed exactly once, unless you are running in SingleShot mode. In other modes, harness may elect to run the method several times, for several reasons: a) Intrinsic race between the thread that announces the iteration finish, and the measurement threads. This is especially possible if you sleep for *just* the iteration time; b) In multi-threaded scenarios, "synchronize iterations" would run pre- and post- loops that are not measured towards performance metrics, but provide proper warmup/warmdown. Thanks, -Aleksey P.S. It would be good if you can pose the question in message body, not in subject alone. From shade at redhat.com Thu Mar 9 17:10:36 2017 From: shade at redhat.com (Aleksey Shipilev) Date: Thu, 9 Mar 2017 18:10:36 +0100 Subject: AuxCounters In-Reply-To: References: Message-ID: <296ba646-8978-83c3-1230-17313699495c@redhat.com> On 03/06/2017 10:15 PM, Henri Tremblay wrote: > Ok. That was my understanding so far. > > Then, I don't understand how it's calculated. Lets say I use OPERATIONS and > Throughput (per second). And I increment once the counter. I will get the number > of time the counter was incremented per second? And then the average time is > what? 1/throughput? Throughput is $counter / $measuredTime. Average time is $measuredTime / $counter. Think about @AuxCounter "OPERATIONS" as the _auxiliary_ operations counter, in addition to the regular JMH's one. > Now, for my original question. Will it be crazy to have a way to split > executions according to an enum returned by the benchmarked method? By crazy I > mean impossible. Or a boolean. It's not impossible technically, but the amount of work required to engineer this is probably not worth it. Non-steady state workloads are weird anyway. Partial throughput measures get even weirder. You might just want to balance the number of cache hits/misses and see how the overall performance changes with the changed mix. -Aleksey From shade at redhat.com Fri Mar 10 21:49:03 2017 From: shade at redhat.com (Aleksey Shipilev) Date: Fri, 10 Mar 2017 22:49:03 +0100 Subject: encoding problem in Param In-Reply-To: <4e732933-a25f-ff67-6675-a90cb43b440d@redhat.com> References: <4e732933-a25f-ff67-6675-a90cb43b440d@redhat.com> Message-ID: <0ebd13ba-44a8-bdc4-f0a3-a638715edb62@redhat.com> On 03/06/2017 07:21 PM, Aleksey Shipilev wrote: > Submitted: > https://bugs.openjdk.java.net/browse/CODETOOLS-7901910 > > The fix I tried today seems way too intrusive. Let me see if I can make it saner. Pushed the candidate fix into the workspace. Seems to work fine. It would be nice if you could build the bleeding edge JMH, and let sbt-jmh use 1.18-SNAPSHOT -- to test it helps your use case. Thanks, -Aleksey From ashipile at redhat.com Fri Mar 10 21:49:45 2017 From: ashipile at redhat.com (ashipile at redhat.com) Date: Fri, 10 Mar 2017 21:49:45 +0000 Subject: hg: code-tools/jmh: 7901910: Non-ASCII @Param values get garbled on non-UTF-8 platform encoding Message-ID: <201703102149.v2ALnjfV017527@aojmv0008.oracle.com> Changeset: 37ef3550476c Author: shade Date: 2017-03-10 22:47 +0100 URL: http://hg.openjdk.java.net/code-tools/jmh/rev/37ef3550476c 7901910: Non-ASCII @Param values get garbled on non-UTF-8 platform encoding ! jmh-core-ct/src/test/java/org/openjdk/jmh/ct/InMemoryGeneratorDestination.java + jmh-core-it/src/test/java/org/openjdk/jmh/it/params/UTF8ParamsTest.java ! jmh-core/src/main/java/org/openjdk/jmh/generators/core/BenchmarkGenerator.java ! jmh-core/src/main/java/org/openjdk/jmh/generators/core/CompilerControlPlugin.java ! jmh-core/src/main/java/org/openjdk/jmh/generators/core/FileSystemDestination.java ! jmh-core/src/main/java/org/openjdk/jmh/generators/core/GeneratorDestination.java ! jmh-core/src/main/java/org/openjdk/jmh/runner/AbstractResourceReader.java ! jmh-core/src/main/java/org/openjdk/jmh/runner/BenchmarkList.java + jmh-core/src/test/java/org/openjdk/jmh/runner/TestBenchmarkListEncoding.java ! jmh-generator-annprocess/src/main/java/org/openjdk/jmh/generators/annotations/APGeneratorDestinaton.java From martijnhoekstra at gmail.com Sat Mar 11 08:54:45 2017 From: martijnhoekstra at gmail.com (Martijn Hoekstra) Date: Sat, 11 Mar 2017 09:54:45 +0100 Subject: encoding problem in Param In-Reply-To: <0ebd13ba-44a8-bdc4-f0a3-a638715edb62@redhat.com> References: <4e732933-a25f-ff67-6675-a90cb43b440d@redhat.com> <0ebd13ba-44a8-bdc4-f0a3-a638715edb62@redhat.com> Message-ID: I should be able to try Sunday. Will get back to you on whether I succeed. On Mar 10, 2017 22:49, "Aleksey Shipilev" wrote: > On 03/06/2017 07:21 PM, Aleksey Shipilev wrote: > > Submitted: > > https://bugs.openjdk.java.net/browse/CODETOOLS-7901910 > > > > The fix I tried today seems way too intrusive. Let me see if I can make > it saner. > > Pushed the candidate fix into the workspace. Seems to work fine. It would > be > nice if you could build the bleeding edge JMH, and let sbt-jmh use > 1.18-SNAPSHOT > -- to test it helps your use case. > > Thanks, > -Aleksey > > That's great! I'll give it a try. It might take me a few days, as I don't really know anything about Java, Maven or Mercurial, and right now, I can't even get jmh to build. I'll get back to you once I get it figured out. Thanks, Martijn From martijnhoekstra at gmail.com Sun Mar 12 09:05:03 2017 From: martijnhoekstra at gmail.com (Martijn Hoekstra) Date: Sun, 12 Mar 2017 10:05:03 +0100 Subject: encoding problem in Param In-Reply-To: References: <4e732933-a25f-ff67-6675-a90cb43b440d@redhat.com> <0ebd13ba-44a8-bdc4-f0a3-a638715edb62@redhat.com> Message-ID: The candidate fix works for me! Thanks again! On Sat, Mar 11, 2017 at 9:54 AM, Martijn Hoekstra wrote: > I should be able to try Sunday. Will get back to you on whether I succeed. > > On Mar 10, 2017 22:49, "Aleksey Shipilev" wrote: > >> On 03/06/2017 07:21 PM, Aleksey Shipilev wrote: >> > Submitted: >> > https://bugs.openjdk.java.net/browse/CODETOOLS-7901910 >> > >> > The fix I tried today seems way too intrusive. Let me see if I can make >> it saner. >> >> Pushed the candidate fix into the workspace. Seems to work fine. It would >> be >> nice if you could build the bleeding edge JMH, and let sbt-jmh use >> 1.18-SNAPSHOT >> -- to test it helps your use case. >> >> Thanks, >> -Aleksey >> >> > That's great! I'll give it a try. It might take me a few days, as I don't > really know anything about Java, Maven or Mercurial, and right now, I can't > even get jmh to build. I'll get back to you once I get it figured out. > > Thanks, > > Martijn > From shade at redhat.com Mon Mar 13 15:46:44 2017 From: shade at redhat.com (Aleksey Shipilev) Date: Mon, 13 Mar 2017 16:46:44 +0100 Subject: JMH 1.18 Message-ID: <0fa3ff37-41a7-74f0-bc3a-6ffa2fbd75fd@redhat.com> Hi, JMH 1.18 is released with a single, but very important bugfix: The benchmark metadata was written out and read back in platform encoding. Which means, if you generate the benchmarks on one machine with platform encoding A, take the JAR to another machine, and run with platform encoding B -- it might fail if A != B. The same can happen if source/target encoding for Java is A (e.g. set in build file), but the platform encoding is B. There are code paths that use A, and some that use B. The truly sad part is that it can fail *silently*, if benchmark names are in ASCII (as they usually are), but the @Params are in non-ASCII. You will never know this until you try to print param value. Fixed: https://bugs.openjdk.java.net/browse/CODETOOLS-7901910 Thanks, -Aleksey From shade at redhat.com Mon Mar 13 15:47:38 2017 From: shade at redhat.com (Aleksey Shipilev) Date: Mon, 13 Mar 2017 16:47:38 +0100 Subject: encoding problem in Param In-Reply-To: References: <4e732933-a25f-ff67-6675-a90cb43b440d@redhat.com> <0ebd13ba-44a8-bdc4-f0a3-a638715edb62@redhat.com> Message-ID: <22be5d75-2c8d-cd61-5fea-9db985294577@redhat.com> On 03/12/2017 10:05 AM, Martijn Hoekstra wrote: > The candidate fix works for me! Thanks again! Available in 1.18. Ask Konrad to bump the sbt-jmh version. Thanks, -Aleksey From ashipile at redhat.com Mon Mar 13 15:48:21 2017 From: ashipile at redhat.com (ashipile at redhat.com) Date: Mon, 13 Mar 2017 15:48:21 +0000 Subject: hg: code-tools/jmh: 3 new changesets Message-ID: <201703131548.v2DFmLBw008503@aojmv0008.oracle.com> Changeset: 6eb89dc11810 Author: shade Date: 2017-03-13 15:26 +0100 URL: http://hg.openjdk.java.net/code-tools/jmh/rev/6eb89dc11810 JMH v1.18. ! jmh-archetypes/jmh-groovy-benchmark-archetype/pom.xml ! jmh-archetypes/jmh-java-benchmark-archetype/pom.xml ! jmh-archetypes/jmh-kotlin-benchmark-archetype/pom.xml ! jmh-archetypes/jmh-scala-benchmark-archetype/pom.xml ! jmh-archetypes/pom.xml ! jmh-core-benchmarks/pom.xml ! jmh-core-ct/pom.xml ! jmh-core-it/pom.xml ! jmh-core/pom.xml ! jmh-generator-annprocess/pom.xml ! jmh-generator-asm/pom.xml ! jmh-generator-bytecode/pom.xml ! jmh-generator-reflection/pom.xml ! jmh-samples/pom.xml ! pom.xml Changeset: 60fb040c75f4 Author: shade Date: 2017-03-13 15:26 +0100 URL: http://hg.openjdk.java.net/code-tools/jmh/rev/60fb040c75f4 Added tag 1.18 for changeset 6eb89dc11810 ! .hgtags Changeset: 113f22bd1101 Author: shade Date: 2017-03-13 15:27 +0100 URL: http://hg.openjdk.java.net/code-tools/jmh/rev/113f22bd1101 Continue in 1.19-SNAPSHOT. ! jmh-archetypes/jmh-groovy-benchmark-archetype/pom.xml ! jmh-archetypes/jmh-java-benchmark-archetype/pom.xml ! jmh-archetypes/jmh-kotlin-benchmark-archetype/pom.xml ! jmh-archetypes/jmh-scala-benchmark-archetype/pom.xml ! jmh-archetypes/pom.xml ! jmh-core-benchmarks/pom.xml ! jmh-core-ct/pom.xml ! jmh-core-it/pom.xml ! jmh-core/pom.xml ! jmh-generator-annprocess/pom.xml ! jmh-generator-asm/pom.xml ! jmh-generator-bytecode/pom.xml ! jmh-generator-reflection/pom.xml ! jmh-samples/pom.xml ! pom.xml From ashipile at redhat.com Thu Mar 16 20:07:50 2017 From: ashipile at redhat.com (ashipile at redhat.com) Date: Thu, 16 Mar 2017 20:07:50 +0000 Subject: hg: code-tools/jmh: 7901917: Update jmh-ant-sample to download artifacts Message-ID: <201703162007.v2GK7oMD006273@aojmv0008.oracle.com> Changeset: 704bfc869f77 Author: shade Date: 2017-03-16 20:59 +0100 URL: http://hg.openjdk.java.net/code-tools/jmh/rev/704bfc869f77 7901917: Update jmh-ant-sample to download artifacts ! jmh-ant-sample/build.xml From jw_list at headissue.com Thu Mar 23 09:50:17 2017 From: jw_list at headissue.com (Jens Wilke) Date: Thu, 23 Mar 2017 16:50:17 +0700 Subject: RFE: record VM and JMH version into the JSON results, maybe add Runtime.getRuntime().availableProcessors() Message-ID: <1915936.oeBHj4Q612@tapsy> Hi all, the output header of JMH now looks like: # JMH 1.18 (released 10 days ago) # VM version: JDK 1.8.0_121, VM 25.121-b13 # VM invoker: /opt//jdk1.8.0_latest/jre/bin/java # VM options: -server -Xmx10G -XX:BiasedLockingStartupDelay=0 -verbose:gc -XX: +PrintGCTimeStamps -XX:+PrintGCDetails # Warmup: 1 iterations, 10 s each # Measurement: 3 iterations, 20 s each # Timeout: 10 min per iteration # Threads: 4 threads, will synchronize iterations # Benchmark mode: Throughput, ops/time # Benchmark: org.cache2k.benchmark.jmh.suite.eviction.symmetrical.ZipfianSequenceLoadingBenchmark.operation # Parameters: (cacheFactory = org.cache2k.benchmark.Cache2kFactory, entryCount = 1000000, factor = 10) I would love to see, that the information JMH-version, VM-version, VM-invoker, VM-options are also recorded in the JSON result file which actually contains almost "everything". Another very useful piece of information would be the value of Runtime.getRuntime().availableProcessors() If this makes sense, I can do a patch. Cheers, Jens -- "Everything superfluous is wrong!" // Jens Wilke - headissue GmbH - Germany \// https://headissue.com From shade at redhat.com Thu Mar 23 09:54:23 2017 From: shade at redhat.com (Aleksey Shipilev) Date: Thu, 23 Mar 2017 10:54:23 +0100 Subject: RFE: record VM and JMH version into the JSON results, maybe add Runtime.getRuntime().availableProcessors() In-Reply-To: <1915936.oeBHj4Q612@tapsy> References: <1915936.oeBHj4Q612@tapsy> Message-ID: On 03/23/2017 10:50 AM, Jens Wilke wrote: > I would love to see, that the information JMH-version, VM-version, VM-invoker, > VM-options are also recorded in the JSON result file which actually contains > almost "everything". > > Another very useful piece of information would be the value of > Runtime.getRuntime().availableProcessors() Note that is different from the actual number of CPUs configured, especially on embedded platforms. Look around how JMH figures out the number of "hot" CPUs. > If this makes sense, I can do a patch. Sure, that makes sense. JSON output is the dumping ground anyway. :) -Aleksey From jw_list at headissue.com Thu Mar 23 10:06:15 2017 From: jw_list at headissue.com (Jens Wilke) Date: Thu, 23 Mar 2017 17:06:15 +0700 Subject: RFE: record VM and JMH version into the JSON results, maybe add Runtime.getRuntime().availableProcessors() In-Reply-To: References: <1915936.oeBHj4Q612@tapsy> Message-ID: <3242594.25IQU3q9DD@tapsy> On Donnerstag, 23. M?rz 2017 16:54:23 ICT Aleksey Shipilev wrote: > Runtime.getRuntime().availableProcessors() > > Note that is different from the actual number of CPUs configured, especially on > embedded platforms. Look around how JMH figures out the number of "hot" CPUs. Can you give some pointers? > Sure, that makes sense. JSON output is the dumping ground anyway. The only thing that bothers me is that it needs to be repeated per result since the top level structure is an array. If it happens and the format will be reconsidered hopefully it gets a struct containing version and results.... Cheers, Jens -- "Everything superfluous is wrong!" // Jens Wilke - headissue GmbH - Germany \// https://headissue.com From shade at redhat.com Thu Mar 23 10:17:02 2017 From: shade at redhat.com (Aleksey Shipilev) Date: Thu, 23 Mar 2017 11:17:02 +0100 Subject: RFE: record VM and JMH version into the JSON results, maybe add Runtime.getRuntime().availableProcessors() In-Reply-To: <3242594.25IQU3q9DD@tapsy> References: <1915936.oeBHj4Q612@tapsy> <3242594.25IQU3q9DD@tapsy> Message-ID: On 03/23/2017 11:06 AM, Jens Wilke wrote: > On Donnerstag, 23. M?rz 2017 16:54:23 ICT Aleksey Shipilev wrote: >> Runtime.getRuntime().availableProcessors() >> >> Note that is different from the actual number of CPUs configured, especially > on >> embedded platforms. Look around how JMH figures out the number of "hot" CPUs. > > Can you give some pointers? http://hg.openjdk.java.net/code-tools/jmh/file/704bfc869f77/jmh-core/src/main/java/org/openjdk/jmh/util/Utils.java#l188 >> Sure, that makes sense. JSON output is the dumping ground anyway. > > The only thing that bothers me is that it needs to be repeated per result > since the top level structure is an array. If it happens and the format will > be reconsidered hopefully it gets a struct containing version and results.... Alas. -Aleksey From ecki at zusammenkunft.net Thu Mar 23 20:33:55 2017 From: ecki at zusammenkunft.net (Bernd Eckenfels) Date: Thu, 23 Mar 2017 20:33:55 +0000 Subject: RFE: record VM and JMH version into the JSON results, maybe add Runtime.getRuntime().availableProcessors() In-Reply-To: <3242594.25IQU3q9DD@tapsy> References: <1915936.oeBHj4Q612@tapsy> ,<3242594.25IQU3q9DD@tapsy> Message-ID: The number of CPUs returned take into account the CPU mask under Windows and for other OS see for example: https://bugs.openjdk.java.net/browse/JDK-6515172 Gruss Bernd -- http://bernd.eckenfels.net ________________________________ From: jmh-dev on behalf of Jens Wilke Sent: Thursday, March 23, 2017 11:06:15 AM To: Aleksey Shipilev Cc: jmh-dev at openjdk.java.net Subject: Re: RFE: record VM and JMH version into the JSON results, maybe add Runtime.getRuntime().availableProcessors() On Donnerstag, 23. M?rz 2017 16:54:23 ICT Aleksey Shipilev wrote: > Runtime.getRuntime().availableProcessors() > > Note that is different from the actual number of CPUs configured, especially on > embedded platforms. Look around how JMH figures out the number of "hot" CPUs. Can you give some pointers? > Sure, that makes sense. JSON output is the dumping ground anyway. The only thing that bothers me is that it needs to be repeated per result since the top level structure is an array. If it happens and the format will be reconsidered hopefully it gets a struct containing version and results.... Cheers, Jens -- "Everything superfluous is wrong!" // Jens Wilke - headissue GmbH - Germany \// https://headissue.com From jw_list at headissue.com Fri Mar 24 02:03:13 2017 From: jw_list at headissue.com (Jens Wilke) Date: Fri, 24 Mar 2017 09:03:13 +0700 Subject: RFE: record VM and JMH version into the JSON results, maybe add Runtime.getRuntime().availableProcessors() In-Reply-To: References: <1915936.oeBHj4Q612@tapsy> <3242594.25IQU3q9DD@tapsy> Message-ID: <745163387.QPIUtINODT@tapsy> On Freitag, 24. M?rz 2017 03:33:55 ICT Bernd Eckenfels wrote: > The number of CPUs returned take into account the CPU mask under Windows and > for other OS see for example: > https://bugs.openjdk.java.net/browse/JDK-6515172 Hi Bernd, good pointer! Actually this is exactly the reason why I think we should have an eye on that value. So probably we should record an initial reading and a separate one taken after the warmup of each iteration. Cheers, Jens -- "Everything superfluous is wrong!" // Jens Wilke - headissue GmbH - Germany \// https://headissue.com From weijun.wang at oracle.com Tue Mar 28 06:29:10 2017 From: weijun.wang at oracle.com (Weijun Wang) Date: Tue, 28 Mar 2017 14:29:10 +0800 Subject: Set a system property before forking Message-ID: I have a system property that is used by Java in a very early stage, which means it has to be set on the command line with -Dx=y instead of inside a program with System.setProperty(). Now I'd like to write a benchmark to check how different values of this system property affect performance. I'm now able to provide the system property on the command line, run the benchmark multiple times, and compare the results. But is it possible to set it inside my benchmark with different values for different @Benchmark methods so I only need to run once? Since the @Benchmark methods run inside forked JVMs, I assume there is a way to set the system property before this forking. Thanks Max From shade at redhat.com Tue Mar 28 09:01:29 2017 From: shade at redhat.com (Aleksey Shipilev) Date: Tue, 28 Mar 2017 11:01:29 +0200 Subject: Set a system property before forking In-Reply-To: References: Message-ID: <479569b0-39dc-c0b8-bf08-ddbff2f9c3af@redhat.com> Hi, On 03/28/2017 08:29 AM, Weijun Wang wrote: > I have a system property that is used by Java in a very early stage, which means > it has to be set on the command line with -Dx=y instead of inside a program with > System.setProperty(). > > Now I'd like to write a benchmark to check how different values of this system > property affect performance. I'm now able to provide the system property on the > command line, run the benchmark multiple times, and compare the results. But is > it possible to set it inside my benchmark with different values for different > @Benchmark methods so I only need to run once? Since the @Benchmark methods run > inside forked JVMs, I assume there is a way to set the system property before > this forking. This is not possible with annotations. This is possible with Java API and adding .jvmArgs(...) to the options. -Aleksey From weijun.wang at oracle.com Tue Mar 28 14:38:27 2017 From: weijun.wang at oracle.com (Weijun Wang) Date: Tue, 28 Mar 2017 22:38:27 +0800 Subject: Set a system property before forking In-Reply-To: <479569b0-39dc-c0b8-bf08-ddbff2f9c3af@redhat.com> References: <479569b0-39dc-c0b8-bf08-ddbff2f9c3af@redhat.com> Message-ID: <3b4fc08c-8638-65a5-3f79-b38dd52960d4@oracle.com> Hi Aleksey On 03/28/2017 05:01 PM, Aleksey Shipilev wrote: > Hi, > > On 03/28/2017 08:29 AM, Weijun Wang wrote: >> I have a system property that is used by Java in a very early stage, which means >> it has to be set on the command line with -Dx=y instead of inside a program with >> System.setProperty(). >> >> Now I'd like to write a benchmark to check how different values of this system >> property affect performance. I'm now able to provide the system property on the >> command line, run the benchmark multiple times, and compare the results. But is >> it possible to set it inside my benchmark with different values for different >> @Benchmark methods so I only need to run once? Since the @Benchmark methods run >> inside forked JVMs, I assume there is a way to set the system property before >> this forking. > > This is not possible with annotations. This is possible with Java API and adding > .jvmArgs(...) to the options. Great, I'm trying it now. So I created 2 OptionsBuilder, the 1st one setting the system property to value 1 and include(method1), the 2nd one setting value2 and include(method2). However, it looks each option still runs both method1 and method2. The "Forking 1 times using command" line does not show the include() argument at all. Maybe I misunderstood include()? How to filter @Benchmark methods in an option? Thanks Max > > -Aleksey > From weijun.wang at oracle.com Tue Mar 28 15:38:34 2017 From: weijun.wang at oracle.com (Weijun Wang) Date: Tue, 28 Mar 2017 23:38:34 +0800 Subject: Set a system property before forking In-Reply-To: <3b4fc08c-8638-65a5-3f79-b38dd52960d4@oracle.com> References: <479569b0-39dc-c0b8-bf08-ddbff2f9c3af@redhat.com> <3b4fc08c-8638-65a5-3f79-b38dd52960d4@oracle.com> Message-ID: <5685b617-8581-7c45-9027-9569d4c183f8@oracle.com> On 03/28/2017 10:38 PM, Weijun Wang wrote: > Hi Aleksey > > On 03/28/2017 05:01 PM, Aleksey Shipilev wrote: >> Hi, >> >> On 03/28/2017 08:29 AM, Weijun Wang wrote: >>> I have a system property that is used by Java in a very early >>> stage, which means it has to be set on the command line with >>> -Dx=y instead of inside a program with System.setProperty(). >>> >>> Now I'd like to write a benchmark to check how different values >>> of this system property affect performance. I'm now able to >>> provide the system property on the command line, run the >>> benchmark multiple times, and compare the results. But is it >>> possible to set it inside my benchmark with different values for >>> different @Benchmark methods so I only need to run once? Since >>> the @Benchmark methods run inside forked JVMs, I assume there is >>> a way to set the system property before this forking. >> >> This is not possible with annotations. This is possible with Java >> API and adding .jvmArgs(...) to the options. > > Great, I'm trying it now. > > So I created 2 OptionsBuilder, the 1st one setting the system > property to value 1 and include(method1), the 2nd one setting value2 > and include(method2). However, it looks each option still runs both > method1 and method2. The "Forking 1 times using command" line does > not show the include() argument at all. > > Maybe I misunderstood include()? How to filter @Benchmark methods in > an option? Oops, my fault. I still keep the include(classname) call (copied from line 96 of org.openjdk.jmh.samples.JMHSample_25_API_GA) and it matches everything. Thanks again, I think I'm on the right road now. --Max > > Thanks Max > >> >> -Aleksey >> From ashipile at redhat.com Tue Mar 28 16:35:41 2017 From: ashipile at redhat.com (ashipile at redhat.com) Date: Tue, 28 Mar 2017 16:35:41 +0000 Subject: hg: code-tools/jmh: 7901930: MultisetStatistics replies +Inf, -Inf when empty Message-ID: <201703281635.v2SGZf0w026505@aojmv0008.oracle.com> Changeset: 36a2ee9a075e Author: shade Date: 2017-03-28 18:34 +0200 URL: http://hg.openjdk.java.net/code-tools/jmh/rev/36a2ee9a075e 7901930: MultisetStatistics replies +Inf, -Inf when empty ! jmh-core/src/main/java/org/openjdk/jmh/util/MultisetStatistics.java ! jmh-core/src/test/java/org/openjdk/jmh/util/TestMultisetStatistics.java From jw_list at headissue.com Wed Mar 29 04:11:53 2017 From: jw_list at headissue.com (Jens Wilke) Date: Wed, 29 Mar 2017 11:11:53 +0700 Subject: Tracking Memory Consumption Metrics With JMH What&How Message-ID: <1704672.C8g3UXIKRC@tapsy> Good Morning Everybody, quite a (long) while ago I promised to do some work on tracking memory consumption metrics and add profiler code to JMH. To see and discuss what makes sense it needs some relevant benchmark scenarios, experimentations and some graphs. Took me a while. Everything that I found out so far is compiled into a blog post: https://cruftex.net/2017/03/28/The-6-Memory-Metrics-You-Should-Track-in-Your-Java-Benchmarks.html?pk_campaign=jmh Feedback is highly appreciated! Depending on the feedback I will start and compile some patches to get this integrated into JMH. Cheers, Jens -- "Everything superfluous is wrong!" // Jens Wilke - headissue GmbH - Germany \// https://headissue.com From kirk at kodewerk.com Wed Mar 29 06:42:06 2017 From: kirk at kodewerk.com (Kirk Pepperdine) Date: Wed, 29 Mar 2017 08:42:06 +0200 Subject: Tracking Memory Consumption Metrics With JMH What&How In-Reply-To: <1704672.C8g3UXIKRC@tapsy> References: <1704672.C8g3UXIKRC@tapsy> Message-ID: Hi Jens, Interesting?.. I find a much less intrusive way of looking at memory is to simply use the data in the GC logs. Yes, they also suffer from inaccuracies but over a long enough benchmark the errors settle out in the dust. We also look at about a half a dozen additional metrics but you can?t get to them directly from SA counters. The one metric that is missing and can have an impact on GC performance is the rate of mutations. You should be able to get a handle on this value by looking at the refinement queue in G1 or card table updates with the generational collectors. This also will not be accurate but it is a measure. Unfortunately you will have to instrument the JVM to get these values but for benching it?s an acceptable option. Another cheap way to estimate allocation is to count the number of collections and multiply that by the size of Eden. Yes, it doesn?t take into consideration of waste or allocations directly in tenured but the waste should be 2% or less and in a bench you should be able to control or at least understand the number of allocations in tenured. You can contact me offline is you?d like a copy of our GC log analysis tooling. Kind regards, Kirk > quite a (long) while ago I promised to do some work on tracking memory > consumption metrics and add profiler code to JMH. > > To see and discuss what makes sense it needs some relevant benchmark > scenarios, experimentations and some graphs. Took me a while. Everything that > I found out so far is compiled into a blog post: > > https://cruftex.net/2017/03/28/The-6-Memory-Metrics-You-Should-Track-in-Your-Java-Benchmarks.html?pk_campaign=jmh > > Feedback is highly appreciated! > > Depending on the feedback I will start and compile some patches to get this > integrated into JMH. > > Cheers, > > Jens > > -- > "Everything superfluous is wrong!" > > // Jens Wilke - headissue GmbH - Germany > \// https://headissue.com