From lev at serebryakov.spb.ru Thu Mar 21 13:16:06 2019 From: lev at serebryakov.spb.ru (Lev Serebryakov) Date: Thu, 21 Mar 2019 16:16:06 +0300 Subject: JMH and JDK 13 Message-ID: <2c6acd16-ab2e-c0b9-b49b-fae52c03722d@serebryakov.spb.ru> I'm trying to run benchmarks of Panama code (or, to be more precise, my code, which uses new Panama featrues) on JDK 13 on Windows. And my results are wildely depends on output! First of all, JDK complains: WARNING: An illegal reflective access operation has occurred WARNING: Illegal reflective access by org.openjdk.jmh.util.Utils (file:/C:/home/lev/Projects/panama-benchmarks/vector/build/libs/vector-benc hmark.jar) to field java.io.PrintStream.charOut WARNING: Please consider reporting this to the maintainers of org.openjdk.jmh.util.Utils WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations WARNING: All illegal access operations will be denied in a future release But it is not main problem. Main problem is, that there is my benchmark with simple "java -jar benchmark.jar: # Warmup: 5 iterations, 10 s each # Measurement: 10 iterations, 2 s each # Timeout: 10 min per iteration # Threads: 1 thread, will synchronize iterations # Benchmark mode: Throughput, ops/time # Benchmark: vector.VectorBenchmarks.VOVec_cs_div_cv # Parameters: (callSize = 16, startOffset = 0) # Run progress: 0,00% complete, ETA 00:14:00 # Fork: 1 of 2 # Warmup Iteration 1: 7448,058 ops/s # Warmup Iteration 2: 7671,674 ops/s # Warmup Iteration 3: 7632,154 ops/s # Warmup Iteration 4: 7591,418 ops/s # Warmup Iteration 5: 7645,616 ops/s Iteration 1: 7629,146 ops/s Iteration 2: 7644,162 ops/s Iteration 3: 7640,662 ops/s Iteration 4:^C But I if I add "-XX:+PrintCompilation" and then filter-out all this diagnostic output (which is not simple) I see: # Warmup Iteration 1: 467 # Warmup Iteration 2: 11584 # Warmup Iteration 3: 21533 # Warmup Iteration 4: 31547 # Warmup Iteration 5: 41582 Iteration 1: 50571 Iteration 2: 54600 Iteration 4: 58630 Iteration 5: 59635 Iteration 6: 61649 Iteration 7: 65679 Iteration 8: 67694 Iteration 9: 67694 Iteration 10: 69710 I'm not sure, is it JMH problem or JDK problem, but I can not good way to check. BTW, adding `-o' option to benchmark run doesn't change result. -- // Black Lion AKA Lev Serebryakov From lev at serebryakov.spb.ru Thu Mar 21 16:45:08 2019 From: lev at serebryakov.spb.ru (Lev Serebryakov) Date: Thu, 21 Mar 2019 19:45:08 +0300 Subject: JMH and JDK 13 In-Reply-To: <2c6acd16-ab2e-c0b9-b49b-fae52c03722d@serebryakov.spb.ru> References: <2c6acd16-ab2e-c0b9-b49b-fae52c03722d@serebryakov.spb.ru> Message-ID: On 21.03.2019 16:16, Lev Serebryakov wrote: > But I if I add "-XX:+PrintCompilation" and then filter-out all this > diagnostic output (which is not simple) I see: But not always. Now after several runs it stopped to help :-( So, behavior is very erratic. -- // Black Lion AKA Lev Serebryakov