From claes.redestad at oracle.com Sun Mar 1 17:53:05 2015 From: claes.redestad at oracle.com (Claes Redestad) Date: Sun, 01 Mar 2015 18:53:05 +0100 Subject: Cleanup lingering mention of GMB Message-ID: <54F35201.9050105@oracle.com> Hi, I spotted some remnants of the @GenerateMicroBenchmark/@GMB annotation in one of the JMH samples, which might be confusing. I took a stab at cleaning this up and remove all mention of @GMB/gmb from the sources while I was at it: http://cr.openjdk.java.net/~redestad/jmh/gmb.webrev Feel free to take this patch into consideration. Thanks! /Claes From aleksey.shipilev at oracle.com Mon Mar 2 22:55:34 2015 From: aleksey.shipilev at oracle.com (aleksey.shipilev at oracle.com) Date: Mon, 02 Mar 2015 22:55:34 +0000 Subject: hg: code-tools/jmh: 7901318: Cleanup lingering mentions of GenerateMicroBenchmark Message-ID: <201503022255.t22MtYv0016159@aojmv0008> Changeset: fdf8427d5f6c Author: shade Date: 2015-03-03 01:55 +0300 URL: http://hg.openjdk.java.net/code-tools/jmh/rev/fdf8427d5f6c 7901318: Cleanup lingering mentions of GenerateMicroBenchmark Contributed-by: Claes Redestad + jmh-core-ct/src/test/java/org/openjdk/jmh/ct/benchmark/BenchmarkTest.java + jmh-core-ct/src/test/java/org/openjdk/jmh/ct/benchmark/PublicAbstractBenchmarkTest.java + jmh-core-ct/src/test/java/org/openjdk/jmh/ct/benchmark/PublicBenchmarkTest.java + jmh-core-ct/src/test/java/org/openjdk/jmh/ct/benchmark/PublicStaticBenchmarkTest.java + jmh-core-ct/src/test/java/org/openjdk/jmh/ct/benchmark/PublicSynchronizedBenchmarkTest.java + jmh-core-ct/src/test/java/org/openjdk/jmh/ct/benchmark/PublicSynchronizedStateBenchmarkTest.java + jmh-core-ct/src/test/java/org/openjdk/jmh/ct/benchmark/PublicSynchronizedStaticBenchmarkStateBenchmarkTest.java + jmh-core-ct/src/test/java/org/openjdk/jmh/ct/benchmark/PublicSynchronizedStaticBenchmarkTest.java + jmh-core-ct/src/test/java/org/openjdk/jmh/ct/benchmark/PublicSynchronizedStaticThreadStateBenchmarkTest.java + jmh-core-ct/src/test/java/org/openjdk/jmh/ct/benchmark/args/BenchmarkParamsTest.java + jmh-core-ct/src/test/java/org/openjdk/jmh/ct/benchmark/args/BlackholeTest.java + jmh-core-ct/src/test/java/org/openjdk/jmh/ct/benchmark/args/ControlTest.java + jmh-core-ct/src/test/java/org/openjdk/jmh/ct/benchmark/args/IterationParamsTest.java + jmh-core-ct/src/test/java/org/openjdk/jmh/ct/benchmark/args/NoStateTest.java + jmh-core-ct/src/test/java/org/openjdk/jmh/ct/benchmark/args/ObjectArrayTest.java + jmh-core-ct/src/test/java/org/openjdk/jmh/ct/benchmark/args/ObjectTest.java + jmh-core-ct/src/test/java/org/openjdk/jmh/ct/benchmark/args/StateTest.java + jmh-core-ct/src/test/java/org/openjdk/jmh/ct/benchmark/args/StringArrayTest.java + jmh-core-ct/src/test/java/org/openjdk/jmh/ct/benchmark/args/StringTest.java + jmh-core-ct/src/test/java/org/openjdk/jmh/ct/benchmark/args/prims/PrimBooleanArrayTest.java + jmh-core-ct/src/test/java/org/openjdk/jmh/ct/benchmark/args/prims/PrimBooleanTest.java + jmh-core-ct/src/test/java/org/openjdk/jmh/ct/benchmark/args/prims/PrimByteArrayTest.java + jmh-core-ct/src/test/java/org/openjdk/jmh/ct/benchmark/args/prims/PrimByteTest.java + jmh-core-ct/src/test/java/org/openjdk/jmh/ct/benchmark/args/prims/PrimCharArrayTest.java + jmh-core-ct/src/test/java/org/openjdk/jmh/ct/benchmark/args/prims/PrimCharTest.java + jmh-core-ct/src/test/java/org/openjdk/jmh/ct/benchmark/args/prims/PrimDoubleArrayTest.java + jmh-core-ct/src/test/java/org/openjdk/jmh/ct/benchmark/args/prims/PrimDoubleTest.java + jmh-core-ct/src/test/java/org/openjdk/jmh/ct/benchmark/args/prims/PrimFloatArrayTest.java + jmh-core-ct/src/test/java/org/openjdk/jmh/ct/benchmark/args/prims/PrimFloatTest.java + jmh-core-ct/src/test/java/org/openjdk/jmh/ct/benchmark/args/prims/PrimIntArrayTest.java + jmh-core-ct/src/test/java/org/openjdk/jmh/ct/benchmark/args/prims/PrimIntTest.java + jmh-core-ct/src/test/java/org/openjdk/jmh/ct/benchmark/args/prims/PrimLongArrayTest.java + jmh-core-ct/src/test/java/org/openjdk/jmh/ct/benchmark/args/prims/PrimLongTest.java + jmh-core-ct/src/test/java/org/openjdk/jmh/ct/benchmark/args/prims/PrimShortArrayTest.java + jmh-core-ct/src/test/java/org/openjdk/jmh/ct/benchmark/args/prims/PrimShortTest.java - jmh-core-ct/src/test/java/org/openjdk/jmh/ct/gmb/GMBTest.java - jmh-core-ct/src/test/java/org/openjdk/jmh/ct/gmb/PublicAbstractGMBTest.java - jmh-core-ct/src/test/java/org/openjdk/jmh/ct/gmb/PublicGMBTest.java - jmh-core-ct/src/test/java/org/openjdk/jmh/ct/gmb/PublicStaticGMBTest.java - jmh-core-ct/src/test/java/org/openjdk/jmh/ct/gmb/PublicSynchronizedGMBTest.java - jmh-core-ct/src/test/java/org/openjdk/jmh/ct/gmb/PublicSynchronizedStateGMBTest.java - jmh-core-ct/src/test/java/org/openjdk/jmh/ct/gmb/PublicSynchronizedStaticBenchmarkStateGMBTest.java - jmh-core-ct/src/test/java/org/openjdk/jmh/ct/gmb/PublicSynchronizedStaticGMBTest.java - jmh-core-ct/src/test/java/org/openjdk/jmh/ct/gmb/PublicSynchronizedStaticThreadStateGMBTest.java - jmh-core-ct/src/test/java/org/openjdk/jmh/ct/gmb/args/BenchmarkParamsTest.java - jmh-core-ct/src/test/java/org/openjdk/jmh/ct/gmb/args/BlackholeTest.java - jmh-core-ct/src/test/java/org/openjdk/jmh/ct/gmb/args/ControlTest.java - jmh-core-ct/src/test/java/org/openjdk/jmh/ct/gmb/args/IterationParamsTest.java - jmh-core-ct/src/test/java/org/openjdk/jmh/ct/gmb/args/NoStateTest.java - jmh-core-ct/src/test/java/org/openjdk/jmh/ct/gmb/args/ObjectArrayTest.java - jmh-core-ct/src/test/java/org/openjdk/jmh/ct/gmb/args/ObjectTest.java - jmh-core-ct/src/test/java/org/openjdk/jmh/ct/gmb/args/StateTest.java - jmh-core-ct/src/test/java/org/openjdk/jmh/ct/gmb/args/StringArrayTest.java - jmh-core-ct/src/test/java/org/openjdk/jmh/ct/gmb/args/StringTest.java - jmh-core-ct/src/test/java/org/openjdk/jmh/ct/gmb/args/prims/PrimBooleanArrayTest.java - jmh-core-ct/src/test/java/org/openjdk/jmh/ct/gmb/args/prims/PrimBooleanTest.java - jmh-core-ct/src/test/java/org/openjdk/jmh/ct/gmb/args/prims/PrimByteArrayTest.java - jmh-core-ct/src/test/java/org/openjdk/jmh/ct/gmb/args/prims/PrimByteTest.java - jmh-core-ct/src/test/java/org/openjdk/jmh/ct/gmb/args/prims/PrimCharArrayTest.java - jmh-core-ct/src/test/java/org/openjdk/jmh/ct/gmb/args/prims/PrimCharTest.java - jmh-core-ct/src/test/java/org/openjdk/jmh/ct/gmb/args/prims/PrimDoubleArrayTest.java - jmh-core-ct/src/test/java/org/openjdk/jmh/ct/gmb/args/prims/PrimDoubleTest.java - jmh-core-ct/src/test/java/org/openjdk/jmh/ct/gmb/args/prims/PrimFloatArrayTest.java - jmh-core-ct/src/test/java/org/openjdk/jmh/ct/gmb/args/prims/PrimFloatTest.java - jmh-core-ct/src/test/java/org/openjdk/jmh/ct/gmb/args/prims/PrimIntArrayTest.java - jmh-core-ct/src/test/java/org/openjdk/jmh/ct/gmb/args/prims/PrimIntTest.java - jmh-core-ct/src/test/java/org/openjdk/jmh/ct/gmb/args/prims/PrimLongArrayTest.java - jmh-core-ct/src/test/java/org/openjdk/jmh/ct/gmb/args/prims/PrimLongTest.java - jmh-core-ct/src/test/java/org/openjdk/jmh/ct/gmb/args/prims/PrimShortArrayTest.java - jmh-core-ct/src/test/java/org/openjdk/jmh/ct/gmb/args/prims/PrimShortTest.java ! jmh-core-it/src/test/java/org/openjdk/jmh/it/ccontrol/CompilerControlDontInlineActualTest.java ! jmh-core-it/src/test/java/org/openjdk/jmh/it/ccontrol/CompilerControlDontInlineTest.java ! jmh-core-it/src/test/java/org/openjdk/jmh/it/ccontrol/CompilerControlExcludeActualTest.java ! jmh-core-it/src/test/java/org/openjdk/jmh/it/ccontrol/CompilerControlExcludeTest.java ! jmh-core-it/src/test/java/org/openjdk/jmh/it/ccontrol/CompilerControlInlineActualTest.java ! jmh-core-it/src/test/java/org/openjdk/jmh/it/ccontrol/CompilerControlInlineTest.java ! jmh-core-it/src/test/java/org/openjdk/jmh/it/footprint/ForkedFootprintTest.java ! jmh-core/src/main/java/org/openjdk/jmh/generators/core/BenchmarkGenerator.java ! jmh-core/src/main/java/org/openjdk/jmh/generators/core/StateObjectHandler.java ! jmh-generator-annprocess/src/main/java/org/openjdk/jmh/generators/annotations/APGeneratorSource.java ! jmh-samples/src/main/java/org/openjdk/jmh/samples/JMHSample_24_Inheritance.java ! jmh-samples/src/main/java/org/openjdk/jmh/samples/JMHSample_28_BlackholeHelpers.java From aleksey.shipilev at oracle.com Mon Mar 2 22:56:49 2015 From: aleksey.shipilev at oracle.com (Aleksey Shipilev) Date: Tue, 03 Mar 2015 01:56:49 +0300 Subject: Cleanup lingering mention of GMB In-Reply-To: <54F35201.9050105@oracle.com> References: <54F35201.9050105@oracle.com> Message-ID: <54F4EAB1.3070607@oracle.com> On 03/01/2015 08:53 PM, Claes Redestad wrote: > http://cr.openjdk.java.net/~redestad/jmh/gmb.webrev > > Feel free to take this patch into consideration. Thanks, pushed: https://bugs.openjdk.java.net/browse/CODETOOLS-7901318 http://hg.openjdk.java.net/code-tools/jmh/rev/fdf8427d5f6c -Aleksey. From aleksey.shipilev at oracle.com Tue Mar 3 10:43:08 2015 From: aleksey.shipilev at oracle.com (aleksey.shipilev at oracle.com) Date: Tue, 03 Mar 2015 10:43:08 +0000 Subject: hg: code-tools/jmh: 7901319: Increase update reminder to 90-180 days Message-ID: <201503031043.t23Ah8QV007376@aojmv0008> Changeset: 4c15defe2406 Author: shade Date: 2015-03-03 13:42 +0300 URL: http://hg.openjdk.java.net/code-tools/jmh/rev/4c15defe2406 7901319: Increase update reminder to 90-180 days ! jmh-core/src/main/java/org/openjdk/jmh/util/Version.java From aleksey.shipilev at oracle.com Tue Mar 3 18:31:34 2015 From: aleksey.shipilev at oracle.com (aleksey.shipilev at oracle.com) Date: Tue, 03 Mar 2015 18:31:34 +0000 Subject: hg: code-tools/jmh: jmh-core-ct: separate the test source from the error messages, otherwise assert text always matches Message-ID: <201503031831.t23IVZiA021483@aojmv0008> Changeset: 0de334030a48 Author: shade Date: 2015-03-03 21:24 +0300 URL: http://hg.openjdk.java.net/code-tools/jmh/rev/0de334030a48 jmh-core-ct: separate the test source from the error messages, otherwise assert text always matches ! jmh-core-ct/src/test/java/org/openjdk/jmh/ct/CompileTest.java From aleksey.shipilev at oracle.com Tue Mar 3 18:31:42 2015 From: aleksey.shipilev at oracle.com (aleksey.shipilev at oracle.com) Date: Tue, 03 Mar 2015 18:31:42 +0000 Subject: hg: code-tools/jmh: 7901320: JMH should check for final @Param fields Message-ID: <201503031831.t23IVgaj021538@aojmv0008> Changeset: 71611fa284a5 Author: shade Date: 2015-03-03 21:31 +0300 URL: http://hg.openjdk.java.net/code-tools/jmh/rev/71611fa284a5 7901320: JMH should check for final @Param fields + jmh-core-ct/src/test/java/org/openjdk/jmh/ct/params/invalid/finals/BooleanTest.java + jmh-core-ct/src/test/java/org/openjdk/jmh/ct/params/invalid/finals/ByteTest.java + jmh-core-ct/src/test/java/org/openjdk/jmh/ct/params/invalid/finals/CharTest.java + jmh-core-ct/src/test/java/org/openjdk/jmh/ct/params/invalid/finals/DoubleTest.java + jmh-core-ct/src/test/java/org/openjdk/jmh/ct/params/invalid/finals/EnumTest.java + jmh-core-ct/src/test/java/org/openjdk/jmh/ct/params/invalid/finals/FloatTest.java + jmh-core-ct/src/test/java/org/openjdk/jmh/ct/params/invalid/finals/LongTest.java + jmh-core-ct/src/test/java/org/openjdk/jmh/ct/params/invalid/finals/Short1Test.java ! jmh-core/src/main/java/org/openjdk/jmh/generators/core/FieldInfo.java ! jmh-core/src/main/java/org/openjdk/jmh/generators/core/StateObjectHandler.java ! jmh-generator-annprocess/src/main/java/org/openjdk/jmh/generators/annotations/APFieldInfo.java ! jmh-generator-asm/src/main/java/org/openjdk/jmh/generators/asm/ASMFieldInfo.java ! jmh-generator-reflection/src/main/java/org/openjdk/jmh/generators/reflection/RFFieldInfo.java From aleksey.shipilev at oracle.com Tue Mar 3 19:17:43 2015 From: aleksey.shipilev at oracle.com (aleksey.shipilev at oracle.com) Date: Tue, 03 Mar 2015 19:17:43 +0000 Subject: hg: code-tools/jmh: 7901321: JMH should support static @Param fields Message-ID: <201503031917.t23JHiQM001746@aojmv0008> Changeset: c00aaa531496 Author: shade Date: 2015-03-03 22:15 +0300 URL: http://hg.openjdk.java.net/code-tools/jmh/rev/c00aaa531496 7901321: JMH should support static @Param fields + jmh-core-ct/src/test/java/org/openjdk/jmh/ct/params/valid/statics/BooleanTest.java + jmh-core-ct/src/test/java/org/openjdk/jmh/ct/params/valid/statics/ByteTest.java + jmh-core-ct/src/test/java/org/openjdk/jmh/ct/params/valid/statics/CharTest.java + jmh-core-ct/src/test/java/org/openjdk/jmh/ct/params/valid/statics/DoubleTest.java + jmh-core-ct/src/test/java/org/openjdk/jmh/ct/params/valid/statics/EnumTest.java + jmh-core-ct/src/test/java/org/openjdk/jmh/ct/params/valid/statics/FloatTest.java + jmh-core-ct/src/test/java/org/openjdk/jmh/ct/params/valid/statics/IntTest.java + jmh-core-ct/src/test/java/org/openjdk/jmh/ct/params/valid/statics/LongTest.java + jmh-core-ct/src/test/java/org/openjdk/jmh/ct/params/valid/statics/ShortTest.java + jmh-core-ct/src/test/java/org/openjdk/jmh/ct/params/valid/statics/StringTest.java + jmh-core-it/src/test/java/org/openjdk/jmh/it/params/StaticParamSanityTest.java ! jmh-core/src/main/java/org/openjdk/jmh/annotations/Param.java ! jmh-core/src/main/java/org/openjdk/jmh/generators/core/StateObjectHandler.java From aleksey.shipilev at oracle.com Tue Mar 3 20:20:46 2015 From: aleksey.shipilev at oracle.com (aleksey.shipilev at oracle.com) Date: Tue, 03 Mar 2015 20:20:46 +0000 Subject: hg: code-tools/jmh: jmh-core-it: ease off CompilerControl tests that push a lot over the binary link. Message-ID: <201503032020.t23KKkXQ016358@aojmv0008> Changeset: a344ddc5d659 Author: shade Date: 2015-03-03 23:20 +0300 URL: http://hg.openjdk.java.net/code-tools/jmh/rev/a344ddc5d659 jmh-core-it: ease off CompilerControl tests that push a lot over the binary link. ! jmh-core-it/src/test/java/org/openjdk/jmh/it/ccontrol/CompilerControlDontInlineActualTest.java ! jmh-core-it/src/test/java/org/openjdk/jmh/it/ccontrol/CompilerControlExcludeActualTest.java ! jmh-core-it/src/test/java/org/openjdk/jmh/it/ccontrol/CompilerControlInlineActualTest.java From aleksey.shipilev at oracle.com Wed Mar 4 09:50:54 2015 From: aleksey.shipilev at oracle.com (aleksey.shipilev at oracle.com) Date: Wed, 04 Mar 2015 09:50:54 +0000 Subject: hg: code-tools/jmh: jmh-core-it: simpler test mask selector. Message-ID: <201503040950.t249osUg025493@aojmv0008> Changeset: 573f1de172ed Author: shade Date: 2015-03-03 23:24 +0300 URL: http://hg.openjdk.java.net/code-tools/jmh/rev/573f1de172ed jmh-core-it: simpler test mask selector. ! jmh-core-it/src/test/java/org/openjdk/jmh/it/Fixtures.java From aleksey.shipilev at oracle.com Wed Mar 4 14:00:15 2015 From: aleksey.shipilev at oracle.com (Aleksey Shipilev) Date: Wed, 04 Mar 2015 17:00:15 +0300 Subject: JMH 1.6.2 Message-ID: <54F70FEF.6020208@oracle.com> Hi, JMH 1.6.2 is released and available at Maven Central (thanks to Evgeny, as usual). This is a maintenance release. If you are migrating to 1.6.x, read the release notes for 1.6 and 1.6.1 as well: http://mail.openjdk.java.net/pipermail/jmh-dev/2015-February/001702.html http://mail.openjdk.java.net/pipermail/jmh-dev/2015-February/001713.html Summary of changes in 1.6.2: * Cleaned up the remained mentions of @GenerateMicroBenchmark in code, javadocs and samples. Contributed by Claes Redestad: https://bugs.openjdk.java.net/browse/CODETOOLS-7901318 * As per Javadoc, @Params are not acceptable on final fields, because they can be folded in the generated code, and setting them via reflection would not change the behavior appropriately. However, JMH did not check it. JMH will fail to compile @Param over final fields now: https://bugs.openjdk.java.net/browse/CODETOOLS-7901320 * @Params were also rejected on static fields. While the best practice is still using @Params over the instance fields, in order to properly initialize @State, in some corner cases it makes sense to allow @Params over static fields. One example is accessing the parameters from the nested (static) classes within the benchmark. Implemented: https://bugs.openjdk.java.net/browse/CODETOOLS-7901321 * Update reminder is set at 90 days now: https://bugs.openjdk.java.net/browse/CODETOOLS-7901319 * Polishing and fixups in internal tests. Enjoy! -Aleksey. From aleksey.shipilev at oracle.com Wed Mar 4 14:00:50 2015 From: aleksey.shipilev at oracle.com (aleksey.shipilev at oracle.com) Date: Wed, 04 Mar 2015 14:00:50 +0000 Subject: hg: code-tools/jmh: 3 new changesets Message-ID: <201503041400.t24E0oEW016165@aojmv0008> Changeset: 43167efb0a90 Author: shade Date: 2015-03-04 12:50 +0300 URL: http://hg.openjdk.java.net/code-tools/jmh/rev/43167efb0a90 JMH v1.6.2. ! jmh-archetypes/jmh-groovy-benchmark-archetype/pom.xml ! jmh-archetypes/jmh-java-benchmark-archetype/pom.xml ! jmh-archetypes/jmh-kotlin-benchmark-archetype/pom.xml ! jmh-archetypes/jmh-scala-benchmark-archetype/pom.xml ! jmh-archetypes/pom.xml ! jmh-core-benchmarks/pom.xml ! jmh-core-ct/pom.xml ! jmh-core-it/pom.xml ! jmh-core/pom.xml ! jmh-generator-annprocess/pom.xml ! jmh-generator-asm/pom.xml ! jmh-generator-bytecode/pom.xml ! jmh-generator-reflection/pom.xml ! jmh-samples/pom.xml ! pom.xml Changeset: 96fd13ed0be8 Author: shade Date: 2015-03-04 12:50 +0300 URL: http://hg.openjdk.java.net/code-tools/jmh/rev/96fd13ed0be8 Added tag 1.6.2 for changeset 43167efb0a90 ! .hgtags Changeset: bc8c05e9feb1 Author: shade Date: 2015-03-04 12:50 +0300 URL: http://hg.openjdk.java.net/code-tools/jmh/rev/bc8c05e9feb1 Continue in 1.7-SNAPSHOT. ! jmh-archetypes/jmh-groovy-benchmark-archetype/pom.xml ! jmh-archetypes/jmh-java-benchmark-archetype/pom.xml ! jmh-archetypes/jmh-kotlin-benchmark-archetype/pom.xml ! jmh-archetypes/jmh-scala-benchmark-archetype/pom.xml ! jmh-archetypes/pom.xml ! jmh-core-benchmarks/pom.xml ! jmh-core-ct/pom.xml ! jmh-core-it/pom.xml ! jmh-core/pom.xml ! jmh-generator-annprocess/pom.xml ! jmh-generator-asm/pom.xml ! jmh-generator-bytecode/pom.xml ! jmh-generator-reflection/pom.xml ! jmh-samples/pom.xml ! pom.xml From ychuang_cn at hotmail.com Wed Mar 4 10:24:04 2015 From: ychuang_cn at hotmail.com (YC Huang) Date: Wed, 4 Mar 2015 10:24:04 +0000 Subject: benchmark unaligned memory access Message-ID: Hi, I'm trying to benchmark what's the cost between aligned/unaligned memory access,but my test-case doesn't show there's obvious performance difference when running on my laptop(Pentium 6200). Am I doing something wrong in the Test? Thank you... BR,YC Huang following is the test code: @State(Scope.Thread)public class MemoryAccessTest { private long alignedAddr, unalignedAddr; @Param({ "24" }) public int structSize; public static Unsafe unsafe; static { try { Field field = Unsafe.class.getDeclaredField("theUnsafe"); field.setAccessible(true); unsafe = (Unsafe) field.get(null); } catch (Exception e) { } } @Setup public void init() { alignedAddr = unsafe.allocateMemory(structSize); assert alignedAddr % 8 == 0; unalignedAddr = alignedAddr + 9;// } @Benchmark public long alignedMemAccess() { return unsafe.getLong(alignedAddr); } @Benchmark public long unalignedMemAccess() { return unsafe.getLong(unalignedAddr); }} From aleksey.shipilev at oracle.com Wed Mar 4 17:40:32 2015 From: aleksey.shipilev at oracle.com (Aleksey Shipilev) Date: Wed, 04 Mar 2015 20:40:32 +0300 Subject: benchmark unaligned memory access In-Reply-To: References: Message-ID: <54F74390.8080406@oracle.com> Hi YC, These questions belong to StackOverflow, please ask it there. This list is for JMH development, not benchmark reviews. On 04.03.2015 13:24, YC Huang wrote: > Hi, I'm trying to benchmark what's the cost between aligned/unaligned > memory access,but my test-case doesn't show there's obvious > performance difference when running on my laptop(Pentium 6200). Did you consider the performance difference is not "obvious", if you can't replicate it easily? ;) All in all, you are trying to follow up on a very thin phenomena, and probably the infrastructure costs dominate the read performance. > Am I doing something wrong in the Test? The test looks OK, except for Unsafe instance might reside in "static *final*" field. -Aleksey. From nitsanw at yahoo.com Thu Mar 5 11:32:36 2015 From: nitsanw at yahoo.com (Nitsan Wakart) Date: Thu, 5 Mar 2015 11:32:36 +0000 (UTC) Subject: benchmark unaligned memory access In-Reply-To: References: Message-ID: <279800588.4914768.1425555156103.JavaMail.yahoo@mail.yahoo.com> This isn't really a JMH issue.Unaligned memory access is not necessarily more expensive, depends on CPU model. The cost difference on recent intel is pretty small unless you read/write on the cache line boundary.I also suspect the cost of the load is eclipsed by the cost of the implicit Blackhole.consume() in your particular benchmark, you can try and find other ways to avoid DCE. On Thursday, March 5, 2015 1:08 PM, YC Huang wrote: Hi, I'm trying to benchmark what's the cost between aligned/unaligned memory access,but my test-case doesn't show there's obvious performance difference when running on my laptop(Pentium 6200). Am I doing something wrong in the Test? Thank you... BR,YC Huang following is the test code: @State(Scope.Thread)public class MemoryAccessTest {??? private long alignedAddr, unalignedAddr; ??? @Param({ "24" })??? public int structSize; ??? public static Unsafe unsafe;??? static {??? ??? try {??? ??? ??? Field field = Unsafe.class.getDeclaredField("theUnsafe");??? ??? ??? field.setAccessible(true);??? ??? ??? unsafe = (Unsafe) field.get(null);??? ??? } catch (Exception e) {??? ??? }??? } ??? @Setup??? public void init() {??? ??? alignedAddr = unsafe.allocateMemory(structSize);??? ??? assert alignedAddr % 8 == 0; ??? ??? unalignedAddr = alignedAddr + 9;//??? } ??? @Benchmark??? public long alignedMemAccess() {??? ??? return unsafe.getLong(alignedAddr);??? } ??? @Benchmark??? public long unalignedMemAccess() {??? ??? return unsafe.getLong(unalignedAddr);??? }} ??? ??? ??? ? ??? ??? ? From ychuang_cn at hotmail.com Thu Mar 5 02:38:30 2015 From: ychuang_cn at hotmail.com (YC Huang) Date: Thu, 5 Mar 2015 02:38:30 +0000 Subject: benchmark unaligned memory access In-Reply-To: <54F74390.8080406@oracle.com> References: , <54F74390.8080406@oracle.com> Message-ID: Aleksey, Thank you for your reply. I'm developing some off-heap serializing lib, and I'm wondering if we do not padding to aligning, it could save us a lot of memory. I guess memory prefetch might have impact on this, so I change the benchmark to iterate over a large memory with different step size; the result do so show obvious difference between aligned/unaligned access when step size increasing, but the cost of aligned access is steady and seems doesn't affected by the step size, which still confused me. Anyway, I'll turn to StackOverflow for this issue... During running those benchmark with "-prof perf", sometimes I got following perf stats which I'm not sure if it's abnormal: most cache/tlb loads are 0. I'm not sure if this is related to JMH, just post here for information. Perf stats:-------------------------------------------------- 9551.219858 task-clock (msec) # 0.312 CPUs utilized 1,465 context-switches # 0.153 K/sec 350 cpu-migrations # 0.037 K/sec 88 page-faults # 0.009 K/sec 20,005,142,254 cycles # 2.095 GHz [22.25%] 10,067,151,576 stalled-cycles-frontend # 50.32% frontend cycles idle [22.43%] 123,390,551 stalled-cycles-backend # 0.62% backend cycles idle [22.53%] 37,104,168,166 instructions # 1.85 insns per cycle # 0.27 stalled cycles per insn [28.10%] 7,152,762,320 branches # 748.885 M/sec [28.17%] 820,260 branch-misses # 0.01% of all branches [28.09%] 0 L1-dcache-loads # 0.000 K/sec [ 0.00%]4,375,650,713,294,279 L1-dcache-load-misses # 0.00% of all L1-dcache hits [ 0.00%]1,061,862,286,476,262 LLC-loads # 111175567.337 M/sec [ 0.00%]319,981,527,223,167 LLC-load-misses # 30.13% of all LL-cache hits [ 0.00%] 0 L1-icache-loads # 0.000 K/sec [ 0.00%]8,224,297,943,167,724 L1-icache-load-misses # 0.00% of all L1-icache hits [ 0.00%] 0 dTLB-loads # 0.000 K/sec [ 0.00%]207,075,866,954,057 dTLB-load-misses # 0.00% of all dTLB cache hits [ 0.00%] 0 iTLB-loads # 0.000 K/sec [ 0.00%]34,929,601,635,080 iTLB-load-misses # 0.00% of all iTLB cache hits [ 0.00%]135,034,208,792,378 L1-dcache-prefetches # 14137901.839 M/sec [ 0.00%]173,581,104,967,126 L1-dcache-prefetch-misses # 18173710.536 M/sec [ 0.00%] 30.588279372 seconds time elapsed > Date: Wed, 4 Mar 2015 20:40:32 +0300 > From: aleksey.shipilev at oracle.com > To: ychuang_cn at hotmail.com; jmh-dev at openjdk.java.net > Subject: Re: benchmark unaligned memory access > > Hi YC, > > These questions belong to StackOverflow, please ask it there. This list > is for JMH development, not benchmark reviews. > > On 04.03.2015 13:24, YC Huang wrote: > > Hi, I'm trying to benchmark what's the cost between aligned/unaligned > > memory access,but my test-case doesn't show there's obvious > > performance difference when running on my laptop(Pentium 6200). > > Did you consider the performance difference is not "obvious", if you > can't replicate it easily? ;) All in all, you are trying to follow up on > a very thin phenomena, and probably the infrastructure costs dominate > the read performance. > > > Am I doing something wrong in the Test? > > The test looks OK, except for Unsafe instance might reside in "static > *final*" field. > > -Aleksey. > From ychuang_cn at hotmail.com Fri Mar 6 02:18:12 2015 From: ychuang_cn at hotmail.com (YC Huang) Date: Fri, 6 Mar 2015 02:18:12 +0000 Subject: benchmark unaligned memory access In-Reply-To: <279800588.4914768.1425555156103.JavaMail.yahoo@mail.yahoo.com> References: , <279800588.4914768.1425555156103.JavaMail.yahoo@mail.yahoo.com> Message-ID: Thank you Nitsanw, running with "prof perfasm" shows there do have implicit Blackhole.consume(): ....[Hottest Methods (after inlining)].............................................................. 45.12% 23.97% org.sample.MemoryAccessTest::alignedMemAccess 29.23% 39.38% org.sample.generated.MemoryAccessTest_alignedMemAccess::alignedMemAccess_avgt_jmhStub 16.80% 30.44% org.openjdk.jmh.infra.Blackhole::consume I change the method to void Instead of return something, and assign the value to a class field,which seems could eliminate the implicit Blackhole.consume() and still avoid DCE... From aleksey.shipilev at oracle.com Fri Mar 6 06:52:25 2015 From: aleksey.shipilev at oracle.com (Aleksey Shipilev) Date: Fri, 06 Mar 2015 09:52:25 +0300 Subject: benchmark unaligned memory access In-Reply-To: References: , <279800588.4914768.1425555156103.JavaMail.yahoo@mail.yahoo.com> Message-ID: <54F94EA9.1020808@oracle.com> On 03/06/2015 05:18 AM, YC Huang wrote: > I change the method to void Instead of return something, and assign > the value to a class field,which seems could eliminate the implicit > Blackhole.consume() and still avoid DCE... This is how you do it: http://cr.openjdk.java.net/~shade/scratch/Misaligned.java -Aleksey. From ychuang_cn at hotmail.com Fri Mar 6 11:03:36 2015 From: ychuang_cn at hotmail.com (YC Huang) Date: Fri, 6 Mar 2015 11:03:36 +0000 Subject: benchmark unaligned memory access In-Reply-To: <54F94EA9.1020808@oracle.com> References: , <279800588.4914768.1425555156103.JavaMail.yahoo@mail.yahoo.com> , <54F94EA9.1020808@oracle.com> Message-ID: Aleksey, Great! That's just what I want...Thank you very much. ? -YC ---------------------------------------- > Date: Fri, 6 Mar 2015 09:52:25 +0300 > From: aleksey.shipilev at oracle.com > To: ychuang_cn at hotmail.com; nitsanw at yahoo.com; jmh-dev at openjdk.java.net > Subject: Re: benchmark unaligned memory access > > On 03/06/2015 05:18 AM, YC Huang wrote: >> I change the method to void Instead of return something, and assign >> the value to a class field,which seems could eliminate the implicit >> Blackhole.consume() and still avoid DCE... > > This is how you do it: > http://cr.openjdk.java.net/~shade/scratch/Misaligned.java > > -Aleksey. > > From aleksey.shipilev at oracle.com Tue Mar 10 22:25:17 2015 From: aleksey.shipilev at oracle.com (aleksey.shipilev at oracle.com) Date: Tue, 10 Mar 2015 22:25:17 +0000 Subject: hg: code-tools/jmh: 7901332: Main JMH process runs out of memory due to accumulated Result-related garbage Message-ID: <201503102225.t2AMPHTW013786@aojmv0008> Changeset: 6b290f26e637 Author: shade Date: 2015-03-11 01:25 +0300 URL: http://hg.openjdk.java.net/code-tools/jmh/rev/6b290f26e637 7901332: Main JMH process runs out of memory due to accumulated Result-related garbage Summary: Tune up a few places showing up in heap dumps. ! jmh-core/src/main/java/org/openjdk/jmh/results/IterationResult.java ! jmh-core/src/main/java/org/openjdk/jmh/results/Result.java ! jmh-core/src/main/java/org/openjdk/jmh/runner/link/BinaryLinkClient.java ! jmh-core/src/main/java/org/openjdk/jmh/runner/link/ClassConventions.java + jmh-core/src/main/java/org/openjdk/jmh/util/SingletonStatistics.java + jmh-core/src/test/java/org/openjdk/jmh/util/TestSingletonStatistics.java From aleksey.shipilev at oracle.com Tue Mar 10 23:02:33 2015 From: aleksey.shipilev at oracle.com (aleksey.shipilev at oracle.com) Date: Tue, 10 Mar 2015 23:02:33 +0000 Subject: hg: code-tools/jmh: jmh-core-it: update stress tests for 7901332 (host VM footprint) Message-ID: <201503102302.t2AN2X8k020196@aojmv0008> Changeset: 7f5b4d3188ec Author: shade Date: 2015-03-11 02:02 +0300 URL: http://hg.openjdk.java.net/code-tools/jmh/rev/7f5b4d3188ec jmh-core-it: update stress tests for 7901332 (host VM footprint) ! jmh-core-it/pom.xml ! jmh-core-it/src/test/java/org/openjdk/jmh/it/footprint/ForkedFootprintTest.java From aleksey.shipilev at oracle.com Thu Mar 12 21:14:25 2015 From: aleksey.shipilev at oracle.com (Aleksey Shipilev) Date: Fri, 13 Mar 2015 00:14:25 +0300 Subject: JMH 1.6.3 Message-ID: <550201B1.9000605@oracle.com> Hi, JMH 1.6.3 is released and available at Maven Central (thanks to Evgeny, as usual). This is a maintenance release, and it contains a single user-visible fix: * Forked VM is resetting binary link state after each frame sent to host VM in order to maintain the low memory footprint. However, this has a nasty side effect for host VM, as it will accumulate garbage, basically duplicating all the objects received over the link in the host VM heap. If you require tens of thousands of iterations, the footprint becomes significant. Fixed with increasing the period between flushes, and also improving the memory footprint for auxiliary data structures: https://bugs.openjdk.java.net/browse/CODETOOLS-7901332 * Polishing and fixups in internal tests. Enjoy! -Aleksey. From aleksey.shipilev at oracle.com Thu Mar 12 21:20:59 2015 From: aleksey.shipilev at oracle.com (aleksey.shipilev at oracle.com) Date: Thu, 12 Mar 2015 21:20:59 +0000 Subject: hg: code-tools/jmh: 3 new changesets Message-ID: <201503122120.t2CLKxYu005654@aojmv0008> Changeset: 68c86695196a Author: shade Date: 2015-03-12 23:09 +0300 URL: http://hg.openjdk.java.net/code-tools/jmh/rev/68c86695196a JMH v1.6.3. ! jmh-archetypes/jmh-groovy-benchmark-archetype/pom.xml ! jmh-archetypes/jmh-java-benchmark-archetype/pom.xml ! jmh-archetypes/jmh-kotlin-benchmark-archetype/pom.xml ! jmh-archetypes/jmh-scala-benchmark-archetype/pom.xml ! jmh-archetypes/pom.xml ! jmh-core-benchmarks/pom.xml ! jmh-core-ct/pom.xml ! jmh-core-it/pom.xml ! jmh-core/pom.xml ! jmh-generator-annprocess/pom.xml ! jmh-generator-asm/pom.xml ! jmh-generator-bytecode/pom.xml ! jmh-generator-reflection/pom.xml ! jmh-samples/pom.xml ! pom.xml Changeset: 8db4c86ebe28 Author: shade Date: 2015-03-12 23:10 +0300 URL: http://hg.openjdk.java.net/code-tools/jmh/rev/8db4c86ebe28 Added tag 1.6.3 for changeset 68c86695196a ! .hgtags Changeset: 06c8e8aa576b Author: shade Date: 2015-03-12 23:10 +0300 URL: http://hg.openjdk.java.net/code-tools/jmh/rev/06c8e8aa576b Continue in 1.7-SNAPSHOT. ! jmh-archetypes/jmh-groovy-benchmark-archetype/pom.xml ! jmh-archetypes/jmh-java-benchmark-archetype/pom.xml ! jmh-archetypes/jmh-kotlin-benchmark-archetype/pom.xml ! jmh-archetypes/jmh-scala-benchmark-archetype/pom.xml ! jmh-archetypes/pom.xml ! jmh-core-benchmarks/pom.xml ! jmh-core-ct/pom.xml ! jmh-core-it/pom.xml ! jmh-core/pom.xml ! jmh-generator-annprocess/pom.xml ! jmh-generator-asm/pom.xml ! jmh-generator-bytecode/pom.xml ! jmh-generator-reflection/pom.xml ! jmh-samples/pom.xml ! pom.xml From jozefhartinger at gmail.com Tue Mar 17 08:40:50 2015 From: jozefhartinger at gmail.com (Jozef Hartinger) Date: Tue, 17 Mar 2015 09:40:50 +0100 Subject: Overriding benchmark method metadata does not work Message-ID: <5507E892.5000306@gmail.com> Hi all, I have a simple benchmark like this: public class Benchmark1 { private static final int[] ARRAY = new int[1 << 20]; @Benchmark @BenchmarkMode(Mode.Throughput) @Measurement(batchSize = 1 << 2, iterations = 5) public boolean run() { Arrays.sort(getData()); return true; } protected int[] getData() { return ARRAY; } } This is a base class. Subclasses exist which simply override getData() and provide different data sets. For a certain subclass I also need to change the batch size. I started with: public class Benchmark2 extends Benchmark1 { @Measurement(batchSize = 1 << 5, iterations = 5) @Override public boolean run() { return super.run(); } @Override protected int[] getData() { return super.getData(); } } That is overriding the run method and adding @Measurement with a different value. That does not seem to have any effect. Then I tried: public class Benchmark2 extends Benchmark1 { @Benchmark @BenchmarkMode(Mode.Throughput) @Measurement(batchSize = 1 << 5, iterations = 5) @Override public boolean run() { return super.run(); } @Override protected int[] getData() { return super.getData(); } } but then I get: "Benchmark1.java:[33,20] Internal error: multiple methods per @Group, but not all methods have @Group" I think that if a method is overriden, the metadata present on the overriding method should override the original metadata. WDYT? Jozef From aleksey.shipilev at oracle.com Tue Mar 17 20:37:20 2015 From: aleksey.shipilev at oracle.com (Aleksey Shipilev) Date: Tue, 17 Mar 2015 23:37:20 +0300 Subject: Overriding benchmark method metadata does not work In-Reply-To: <5507E892.5000306@gmail.com> References: <5507E892.5000306@gmail.com> Message-ID: <55089080.8060402@oracle.com> Hi Josef, On 17.03.2015 11:40, Jozef Hartinger wrote: > but then I get: > "Benchmark1.java:[33,20] Internal error: multiple methods per @Group, > but not all methods have @Group" I think this is similar to: https://bugs.openjdk.java.net/browse/CODETOOLS-7901009 Aggelos is experimenting with the fixes for that. > I think that if a method is overriden, the metadata present on the > overriding method should override the original metadata. WDYT? So, the metadata on overriding methods is probably get FUBARed because of the issue above. The workaround for your case would be to put the metadata on the class level. In that case, the metadata on the subclass will take precedence over the superclass metadata. Thanks, -Aleksey From jozefhartinger at gmail.com Wed Mar 18 10:01:06 2015 From: jozefhartinger at gmail.com (Jozef Hartinger) Date: Wed, 18 Mar 2015 11:01:06 +0100 Subject: Overriding benchmark method metadata does not work In-Reply-To: <55089080.8060402@oracle.com> References: <5507E892.5000306@gmail.com> <55089080.8060402@oracle.com> Message-ID: <55094CE2.2090001@gmail.com> Class-level overrides work fine, thanks! On 03/17/2015 09:37 PM, Aleksey Shipilev wrote: > Hi Josef, > > On 17.03.2015 11:40, Jozef Hartinger wrote: >> but then I get: >> "Benchmark1.java:[33,20] Internal error: multiple methods per @Group, >> but not all methods have @Group" > I think this is similar to: > https://bugs.openjdk.java.net/browse/CODETOOLS-7901009 > > Aggelos is experimenting with the fixes for that. > >> I think that if a method is overriden, the metadata present on the >> overriding method should override the original metadata. WDYT? > So, the metadata on overriding methods is probably get FUBARed because > of the issue above. The workaround for your case would be to put the > metadata on the class level. In that case, the metadata on the subclass > will take precedence over the superclass metadata. > > Thanks, > -Aleksey > > From biboudis at gmail.com Wed Mar 18 17:51:04 2015 From: biboudis at gmail.com (Aggelos Biboudis) Date: Wed, 18 Mar 2015 19:51:04 +0200 Subject: Overriding benchmark method metadata does not work In-Reply-To: <55089080.8060402@oracle.com> References: <5507E892.5000306@gmail.com> <55089080.8060402@oracle.com> Message-ID: Hi all, Yeap. It is definitely related to 7901009. My patch is partially complete ( http://cgi.di.uoa.gr/~biboudis/cr/jmh/CODETOOLS-7901009/) I wanted to check some corner cases. I will finalize it asap. Cheers, Aggelos. On Tue, Mar 17, 2015 at 10:37 PM, Aleksey Shipilev < aleksey.shipilev at oracle.com> wrote: > Hi Josef, > > On 17.03.2015 11:40, Jozef Hartinger wrote: > > but then I get: > > "Benchmark1.java:[33,20] Internal error: multiple methods per @Group, > > but not all methods have @Group" > > I think this is similar to: > https://bugs.openjdk.java.net/browse/CODETOOLS-7901009 > > Aggelos is experimenting with the fixes for that. > > > I think that if a method is overriden, the metadata present on the > > overriding method should override the original metadata. WDYT? > > So, the metadata on overriding methods is probably get FUBARed because > of the issue above. The workaround for your case would be to put the > metadata on the class level. In that case, the metadata on the subclass > will take precedence over the superclass metadata. > > Thanks, > -Aleksey > > > From aleksey.shipilev at oracle.com Thu Mar 19 09:28:11 2015 From: aleksey.shipilev at oracle.com (aleksey.shipilev at oracle.com) Date: Thu, 19 Mar 2015 09:28:11 +0000 Subject: hg: code-tools/jmh: jmh-core: merge LoopBenchmarkHandler into BenchmarkHandler, as it's the only implementation anyway Message-ID: <201503190928.t2J9SBxa027831@aojmv0008> Changeset: 964023d59b57 Author: shade Date: 2015-03-19 12:27 +0300 URL: http://hg.openjdk.java.net/code-tools/jmh/rev/964023d59b57 jmh-core: merge LoopBenchmarkHandler into BenchmarkHandler, as it's the only implementation anyway - jmh-core/src/main/java/org/openjdk/jmh/runner/BaseBenchmarkHandler.java ! jmh-core/src/main/java/org/openjdk/jmh/runner/BaseRunner.java ! jmh-core/src/main/java/org/openjdk/jmh/runner/BenchmarkHandler.java - jmh-core/src/main/java/org/openjdk/jmh/runner/BenchmarkHandlers.java - jmh-core/src/main/java/org/openjdk/jmh/runner/LoopBenchmarkHandler.java ! jmh-core/src/test/java/org/openjdk/jmh/runner/DistributeGroupsTest.java From aleksey.shipilev at oracle.com Thu Mar 19 13:44:46 2015 From: aleksey.shipilev at oracle.com (aleksey.shipilev at oracle.com) Date: Thu, 19 Mar 2015 13:44:46 +0000 Subject: hg: code-tools/jmh: 7901345: Refactor batchSize/opsPerInv handling in the generated code and counting the operations Message-ID: <201503191344.t2JDika7024788@aojmv0008> Changeset: c6413955f9d8 Author: shade Date: 2015-03-19 16:44 +0300 URL: http://hg.openjdk.java.net/code-tools/jmh/rev/c6413955f9d8 7901345: Refactor batchSize/opsPerInv handling in the generated code and counting the operations + jmh-core-it/src/test/java/org/openjdk/jmh/it/batchsize/BatchSizeSanityTest.java + jmh-core-it/src/test/java/org/openjdk/jmh/it/batchsize/OpsPerInvSanityTest.java ! jmh-core/src/main/java/org/openjdk/jmh/generators/core/BenchmarkGenerator.java + jmh-core/src/main/java/org/openjdk/jmh/results/BenchmarkTaskResult.java ! jmh-core/src/main/java/org/openjdk/jmh/results/RawResults.java ! jmh-core/src/main/java/org/openjdk/jmh/runner/BenchmarkHandler.java From aleksey.shipilev at oracle.com Thu Mar 19 14:12:00 2015 From: aleksey.shipilev at oracle.com (aleksey.shipilev at oracle.com) Date: Thu, 19 Mar 2015 14:12:00 +0000 Subject: hg: code-tools/jmh: Fix the buildd failure after 7901345. Message-ID: <201503191412.t2JEC07V000732@aojmv0008> Changeset: 02fedaf27367 Author: shade Date: 2015-03-19 16:51 +0300 URL: http://hg.openjdk.java.net/code-tools/jmh/rev/02fedaf27367 Fix the buildd failure after 7901345. ! jmh-core/src/main/java/org/openjdk/jmh/generators/core/BenchmarkGenerator.java From aleksey.shipilev at oracle.com Thu Mar 19 18:26:40 2015 From: aleksey.shipilev at oracle.com (aleksey.shipilev at oracle.com) Date: Thu, 19 Mar 2015 18:26:40 +0000 Subject: hg: code-tools/jmh: 7901346: Profilers should be able to know the actual warmup/measurement durations Message-ID: <201503191826.t2JIQeb5004676@aojmv0008> Changeset: ea66b1770352 Author: shade Date: 2015-03-19 18:54 +0300 URL: http://hg.openjdk.java.net/code-tools/jmh/rev/ea66b1770352 7901346: Profilers should be able to know the actual warmup/measurement durations ! jmh-core-it/src/test/java/org/openjdk/jmh/it/ccontrol/LogConsumeProfiler.java ! jmh-core-it/src/test/java/org/openjdk/jmh/it/profilers/ItExternalProfiler.java ! jmh-core-it/src/test/java/org/openjdk/jmh/it/profilers/order/AbstractExternalProfiler.java ! jmh-core/src/main/java/org/openjdk/jmh/profile/AbstractPerfAsmProfiler.java ! jmh-core/src/main/java/org/openjdk/jmh/profile/ExternalProfiler.java ! jmh-core/src/main/java/org/openjdk/jmh/profile/LinuxPerfProfiler.java ! jmh-core/src/main/java/org/openjdk/jmh/profile/WinPerfAsmProfiler.java ! jmh-core/src/main/java/org/openjdk/jmh/results/BenchmarkResult.java + jmh-core/src/main/java/org/openjdk/jmh/results/BenchmarkResultMetaData.java ! jmh-core/src/main/java/org/openjdk/jmh/results/BenchmarkTaskResult.java + jmh-core/src/main/java/org/openjdk/jmh/results/HandlerResult.java ! jmh-core/src/main/java/org/openjdk/jmh/runner/BaseRunner.java ! jmh-core/src/main/java/org/openjdk/jmh/runner/BenchmarkHandler.java ! jmh-core/src/main/java/org/openjdk/jmh/runner/ForkedRunner.java ! jmh-core/src/main/java/org/openjdk/jmh/runner/IterationResultAcceptor.java ! jmh-core/src/main/java/org/openjdk/jmh/runner/Runner.java ! jmh-core/src/main/java/org/openjdk/jmh/runner/link/BinaryLinkClient.java ! jmh-core/src/main/java/org/openjdk/jmh/runner/link/BinaryLinkServer.java + jmh-core/src/main/java/org/openjdk/jmh/runner/link/ResultMetadataFrame.java From aleksey.shipilev at oracle.com Thu Mar 19 19:29:14 2015 From: aleksey.shipilev at oracle.com (aleksey.shipilev at oracle.com) Date: Thu, 19 Mar 2015 19:29:14 +0000 Subject: hg: code-tools/jmh: 7901347: Text report improvements: don't print "NaN" in errors, etc Message-ID: <201503191929.t2JJTETK017225@aojmv0008> Changeset: 9d9c35e9cf04 Author: shade Date: 2015-03-19 22:28 +0300 URL: http://hg.openjdk.java.net/code-tools/jmh/rev/9d9c35e9cf04 7901347: Text report improvements: don't print "NaN" in errors, etc ! jmh-core/src/main/java/org/openjdk/jmh/results/format/TextResultFormat.java ! jmh-core/src/main/java/org/openjdk/jmh/runner/format/TextReportFormat.java ! jmh-core/src/test/resources/org/openjdk/jmh/results/format/output-golden.text.root ! jmh-core/src/test/resources/org/openjdk/jmh/results/format/output-golden.text.ru ! jmh-core/src/test/resources/org/openjdk/jmh/results/format/output-golden.text.us From aleksey.shipilev at oracle.com Thu Mar 19 19:31:59 2015 From: aleksey.shipilev at oracle.com (aleksey.shipilev at oracle.com) Date: Thu, 19 Mar 2015 19:31:59 +0000 Subject: hg: code-tools/jmh: link: unconditionally overwrite the received metadata. Message-ID: <201503191931.t2JJVxK1017663@aojmv0008> Changeset: 0ea217986c8e Author: shade Date: 2015-03-19 22:31 +0300 URL: http://hg.openjdk.java.net/code-tools/jmh/rev/0ea217986c8e link: unconditionally overwrite the received metadata. ! jmh-core/src/main/java/org/openjdk/jmh/runner/link/BinaryLinkServer.java From aleksey.shipilev at oracle.com Thu Mar 19 19:59:17 2015 From: aleksey.shipilev at oracle.com (aleksey.shipilev at oracle.com) Date: Thu, 19 Mar 2015 19:59:17 +0000 Subject: hg: code-tools/jmh: 7901348: "perfnorm": perf-based profiler that normalizes counters by operation count Message-ID: <201503191959.t2JJxHEm022588@aojmv0008> Changeset: 5bb27724a701 Author: shade Date: 2015-03-19 22:57 +0300 URL: http://hg.openjdk.java.net/code-tools/jmh/rev/5bb27724a701 7901348: "perfnorm": perf-based profiler that normalizes counters by operation count ! jmh-core/src/main/java/org/openjdk/jmh/profile/LinuxPerfAsmProfiler.java + jmh-core/src/main/java/org/openjdk/jmh/profile/LinuxPerfNormProfiler.java ! jmh-core/src/main/java/org/openjdk/jmh/profile/LinuxPerfProfiler.java - jmh-core/src/main/java/org/openjdk/jmh/profile/LinuxPerfUtil.java ! jmh-core/src/main/java/org/openjdk/jmh/profile/ProfilerFactory.java From aleksey.shipilev at oracle.com Fri Mar 20 08:20:22 2015 From: aleksey.shipilev at oracle.com (aleksey.shipilev at oracle.com) Date: Fri, 20 Mar 2015 08:20:22 +0000 Subject: hg: code-tools/jmh: perfnorm: a little polish in warning messages. Message-ID: <201503200820.t2K8KM3M020853@aojmv0008> Changeset: 5fa3679cc741 Author: shade Date: 2015-03-20 11:20 +0300 URL: http://hg.openjdk.java.net/code-tools/jmh/rev/5fa3679cc741 perfnorm: a little polish in warning messages. ! jmh-core/src/main/java/org/openjdk/jmh/profile/LinuxPerfNormProfiler.java From aleksey.shipilev at oracle.com Fri Mar 20 10:38:04 2015 From: aleksey.shipilev at oracle.com (aleksey.shipilev at oracle.com) Date: Fri, 20 Mar 2015 10:38:04 +0000 Subject: hg: code-tools/jmh: perfnorm: fix the accidental debug printing. Message-ID: <201503201038.t2KAc5rH024582@aojmv0008> Changeset: 4ba1df6f80e3 Author: shade Date: 2015-03-20 13:37 +0300 URL: http://hg.openjdk.java.net/code-tools/jmh/rev/4ba1df6f80e3 perfnorm: fix the accidental debug printing. ! jmh-core/src/main/java/org/openjdk/jmh/profile/LinuxPerfNormProfiler.java From aleksey.shipilev at oracle.com Fri Mar 20 12:12:39 2015 From: aleksey.shipilev at oracle.com (Aleksey Shipilev) Date: Fri, 20 Mar 2015 15:12:39 +0300 Subject: JMH 1.7 Message-ID: <550C0EB7.5050303@oracle.com> Hi, JMH 1.7 is released and available at Maven Central (thanks to Evgeny Mandrikov). This is the feature release, and it contains improvements to generated code, profilers, and reports. Summary of changes: * "-prof perfnorm" is now available! It looks (almost) like the "-prof perf" profiler, but with one critical improvement: it parses the "perf stat" output, and normalizes the counters by operation count. That way, perfnorm can estimate the number of uArch events per benchmark op, which is a usable piece of data in nano-benchmarks. Changes: https://bugs.openjdk.java.net/browse/CODETOOLS-7901348 The run sample: http://cr.openjdk.java.net/~shade/jmh/perfnorm-sample.log Please try to run "-prof perfnorm" on your cases! As other "perf"-based profilers, it requires Linux perf to be functional. A modern kernel with a modern linux-tools is preferred to make the profiling more accurate. The profilers would run on the old perfs as well, but not that reliably. Feedbacks, bug reports, improvements (even better, patches) are welcome. Note that you might want to run with multiple forks to get the error estimates for the counters. In many cases, when workload is not steady and/or the benchmarking environment is flaky, the counters per op may swing wildly. You better to be catching these swings with multiple forks. * Text report was cleaned up a little bit: we do not print NaN's when errors are not defined, etc. This helps to reduce the noise when e.g. the profilers are enabled. Changes: https://bugs.openjdk.java.net/browse/CODETOOLS-7901347 * "External" profilers are now able to figure out when the actual warmup have ended, instead of guessing from the benchmark configuration. That makes the profiled results more reliable, and lets to correlate the measurement ops count with the actual counters. This required rework in binary link, and it can (potentially) cause issues in some complex cases, take care. This also, unfortunately, breaks the profiler API (BenchmarkParams -> BenchmarkResult is changed in one signature). Changes: https://bugs.openjdk.java.net/browse/CODETOOLS-7901346 * The internal mechanics for "batch sizes" and "operations per invocation" was reworked and covered with more tests. Chances are there are (unlikely) regressions in the cases not covered by tests, take care. Changes: https://bugs.openjdk.java.net/browse/CODETOOLS-7901345 * Other user-invisible touchups and polishings. Thanks, -Aleksey. From aleksey.shipilev at oracle.com Fri Mar 20 12:12:57 2015 From: aleksey.shipilev at oracle.com (aleksey.shipilev at oracle.com) Date: Fri, 20 Mar 2015 12:12:57 +0000 Subject: hg: code-tools/jmh: 3 new changesets Message-ID: <201503201212.t2KCCvk8015571@aojmv0008> Changeset: 96a2892f41a9 Author: shade Date: 2015-03-20 13:45 +0300 URL: http://hg.openjdk.java.net/code-tools/jmh/rev/96a2892f41a9 JMH v1.7. ! jmh-archetypes/jmh-groovy-benchmark-archetype/pom.xml ! jmh-archetypes/jmh-java-benchmark-archetype/pom.xml ! jmh-archetypes/jmh-kotlin-benchmark-archetype/pom.xml ! jmh-archetypes/jmh-scala-benchmark-archetype/pom.xml ! jmh-archetypes/pom.xml ! jmh-core-benchmarks/pom.xml ! jmh-core-ct/pom.xml ! jmh-core-it/pom.xml ! jmh-core/pom.xml ! jmh-generator-annprocess/pom.xml ! jmh-generator-asm/pom.xml ! jmh-generator-bytecode/pom.xml ! jmh-generator-reflection/pom.xml ! jmh-samples/pom.xml ! pom.xml Changeset: a5e35417e3f4 Author: shade Date: 2015-03-20 13:45 +0300 URL: http://hg.openjdk.java.net/code-tools/jmh/rev/a5e35417e3f4 Added tag 1.7 for changeset 96a2892f41a9 ! .hgtags Changeset: 7e1c1c40872d Author: shade Date: 2015-03-20 13:45 +0300 URL: http://hg.openjdk.java.net/code-tools/jmh/rev/7e1c1c40872d Continue in 1.8-SNAPSHOT. ! jmh-archetypes/jmh-groovy-benchmark-archetype/pom.xml ! jmh-archetypes/jmh-java-benchmark-archetype/pom.xml ! jmh-archetypes/jmh-kotlin-benchmark-archetype/pom.xml ! jmh-archetypes/jmh-scala-benchmark-archetype/pom.xml ! jmh-archetypes/pom.xml ! jmh-core-benchmarks/pom.xml ! jmh-core-ct/pom.xml ! jmh-core-it/pom.xml ! jmh-core/pom.xml ! jmh-generator-annprocess/pom.xml ! jmh-generator-asm/pom.xml ! jmh-generator-bytecode/pom.xml ! jmh-generator-reflection/pom.xml ! jmh-samples/pom.xml ! pom.xml From trask.stalnaker at gmail.com Tue Mar 24 03:42:43 2015 From: trask.stalnaker at gmail.com (Trask Stalnaker) Date: Tue, 24 Mar 2015 03:42:43 +0000 Subject: Passing jvmArgs with spaces? Message-ID: Strangely this is working for me on windows but not on linux: java -jar benchmarks.jar -jvmArgs "-Dx=12 \"-Dy=one two\"" On Windows: # JMH 1.7 (released 3 days ago) # VM invoker: C:\dev\java\jre1.8.0_40\bin\java.exe # VM options: -Dx=12 "-Dy=one two" and the benchmark succeeds. While on Linux: # JMH 1.7 (released 3 days ago) # VM invoker: /usr/lib/jvm/java-8-oracle/jre/bin/java # VM options: -Dx=12 "-Dy=one two" and the benchmark fails with Error: Could not find or load main class "-Dy=one Thanks, Trask From aleksey.shipilev at oracle.com Mon Mar 30 15:57:20 2015 From: aleksey.shipilev at oracle.com (aleksey.shipilev at oracle.com) Date: Mon, 30 Mar 2015 15:57:20 +0000 Subject: hg: code-tools/jmh: 7901361: perfnorm should also report CPI Message-ID: <201503301557.t2UFvK3B003912@aojmv0008> Changeset: f4fdf1969ee9 Author: shade Date: 2015-03-30 18:57 +0300 URL: http://hg.openjdk.java.net/code-tools/jmh/rev/f4fdf1969ee9 7901361: perfnorm should also report CPI ! jmh-core/src/main/java/org/openjdk/jmh/profile/LinuxPerfNormProfiler.java From aleksey.shipilev at oracle.com Tue Mar 31 08:17:07 2015 From: aleksey.shipilev at oracle.com (aleksey.shipilev at oracle.com) Date: Tue, 31 Mar 2015 08:17:07 +0000 Subject: hg: code-tools/jmh: 7901330: Fork annotation value on class ignored in presence of Fork annotation on Benchmark Message-ID: <201503310817.t2V8H7Ed003607@aojmv0008> Changeset: b33187f91f94 Author: shade Date: 2015-03-31 11:14 +0300 URL: http://hg.openjdk.java.net/code-tools/jmh/rev/b33187f91f94 7901330: Fork annotation value on class ignored in presence of Fork annotation on Benchmark Summary: Implement per-parameter annotation merging. + jmh-core-it/src/test/java/org/openjdk/jmh/it/fork/ForkMergeTest.java ! jmh-core-it/src/test/java/org/openjdk/jmh/it/fork/ForkedJvmAppendPrependArgs2_Test.java ! jmh-core-it/src/test/java/org/openjdk/jmh/it/fork/ForkedJvmArgs2_Test.java ! jmh-core/src/main/java/org/openjdk/jmh/generators/core/BenchmarkGeneratorUtils.java ! jmh-core/src/main/java/org/openjdk/jmh/generators/core/MethodGroup.java From aleksey.shipilev at oracle.com Tue Mar 31 11:07:28 2015 From: aleksey.shipilev at oracle.com (aleksey.shipilev at oracle.com) Date: Tue, 31 Mar 2015 11:07:28 +0000 Subject: hg: code-tools/jmh: 7901362: JMH handling of quoted jvmArgs is broken Message-ID: <201503311107.t2VB7SJg015024@aojmv0008> Changeset: a695e0c1c6b1 Author: shade Date: 2015-03-31 14:07 +0300 URL: http://hg.openjdk.java.net/code-tools/jmh/rev/a695e0c1c6b1 7901362: JMH handling of quoted jvmArgs is broken Summary: Revisit the way we deal with quoted arguments. ! jmh-core/src/main/java/org/openjdk/jmh/runner/options/CommandLineOptions.java ! jmh-core/src/main/java/org/openjdk/jmh/util/Utils.java ! jmh-core/src/test/java/org/openjdk/jmh/util/TestUtil.java From aleksey.shipilev at oracle.com Tue Mar 31 11:07:34 2015 From: aleksey.shipilev at oracle.com (Aleksey Shipilev) Date: Tue, 31 Mar 2015 14:07:34 +0300 Subject: Passing jvmArgs with spaces? In-Reply-To: References: Message-ID: <551A7FF6.8070300@oracle.com> Hi Trask, On 03/24/2015 06:42 AM, Trask Stalnaker wrote: > While on Linux: > > # JMH 1.7 (released 3 days ago) > # VM invoker: /usr/lib/jvm/java-8-oracle/jre/bin/java > # VM options: -Dx=12 "-Dy=one two" > > and the benchmark fails with > > Error: Could not find or load main class "-Dy=one The real cause for this issue is that JMH splits the single jvmArg argument into multiple arguments with naive split("[ ]+"), that also splits by space between "one" and "two". There are two ways to address this: a) Accept multiple -jvmArgs, each for a single argument. This seems to be a proper fix, that is also coherent with .jvmArgs() in API. The drawback for users: requires a tedious listing when multiple options are required. b) Make -jvmArgs splitter to be resilient for quotes. The ambigous part about this: how to handle nested quotes. Trivially, we may just ignore the issue and treat the quotes as non-nested. With: https://bugs.openjdk.java.net/browse/CODETOOLS-7901362 ...we have implemented both. Therefore, two proper ways emerged: $ java -jar benchmarks.jar -jvmArgs "-Dx=12 -Dy=\"one two\"" $ java -jar benchmarks.jar -jvmArgs "-Dx=12" -jvmArgs "-Dy=one two" Both work fine on Linux and Windows. Please confirm. Thanks, -Aleksey. P.S. I believe it worked "fine" on Windows, because Java launcher there intervened and "fixed" some quirks. From aleksey.shipilev at oracle.com Tue Mar 31 14:33:20 2015 From: aleksey.shipilev at oracle.com (aleksey.shipilev at oracle.com) Date: Tue, 31 Mar 2015 14:33:20 +0000 Subject: hg: code-tools/jmh: JMHSample_34_SafeLooping sample. Message-ID: <201503311433.t2VEXK89004183@aojmv0008> Changeset: fceec061059d Author: shade Date: 2015-03-31 17:33 +0300 URL: http://hg.openjdk.java.net/code-tools/jmh/rev/fceec061059d JMHSample_34_SafeLooping sample. + jmh-samples/src/main/java/org/openjdk/jmh/samples/JMHSample_34_SafeLooping.java From aleksey.shipilev at oracle.com Tue Mar 31 15:12:36 2015 From: aleksey.shipilev at oracle.com (aleksey.shipilev at oracle.com) Date: Tue, 31 Mar 2015 15:12:36 +0000 Subject: hg: code-tools/jmh: 7901363: @Benchmark methods with the same name, but different signatures should yield a proper error message Message-ID: <201503311512.t2VFCaTd014500@aojmv0008> Changeset: adcd0af0a03b Author: shade Date: 2015-03-31 18:05 +0300 URL: http://hg.openjdk.java.net/code-tools/jmh/rev/adcd0af0a03b 7901363: @Benchmark methods with the same name, but different signatures should yield a proper error message + jmh-core-ct/src/test/java/org/openjdk/jmh/ct/benchmark/args/ArgumentListAmbiguityTest.java ! jmh-core/src/main/java/org/openjdk/jmh/generators/core/MethodGroup.java ! jmh-core/src/main/java/org/openjdk/jmh/generators/core/MethodInvocation.java