make test TEST="micro:" got java.lang.UnsupportedClassVersionError

[Claes Redestad]

FWIW I just integrated the change to remove --enable-preview from
the micros build, so the UnsupportedClassVersionError should be
gone if you sync up. No --enable-preview needed.

/Claes

On 2020-10-28 11:09, Liu, Xin wrote:
> Hi, Claes,
> 
> Thank you for the great guideline of micro benchmarking.  now I have a clearer picture for it.
> I'm happy to learn that 5-10 minutes for each Benchmark!  Let me polish mine.
> 
> Thanks,
> --lx
> 
> On 10/27/20, 3:59 AM, "Claes Redestad" <claes.redestad at oracle.com> wrote:
> 
>      CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.
> 
> 
> 
>      Hi lx,
> 
>      the need to add --enable-preview is known, and it's a bug introduced a
>      couple of months ago thanks to a suggestion I did (without realizing
>      that compiling one micro with --enable-preview would poison all of
>      them..). This issue is tracked by a few bugs, see
>      https://bugs.openjdk.java.net/browse/JDK-8250669 and
>      https://bugs.openjdk.java.net/browse/JDK-8253828
> 
>      The fix will likely be to remove the --enable-preview from the micros
>      build after Records goes out of preview and then think more carefully on
>      how to add support for benchmarking preview features.
> 
>      As to your other question: When adding a microbenchmark it's appreciated
>      if the default configuration makes it as quick to run as possible
>      without sacrificing quality. Often the default number of forks,
>      iteration runtime et.c. can be tuned down by using custom settings in
>      @Fork et.c. Custom parameters used can be limited to a select handful.
> 
>      Still it's normal for a "good" microbenchmark to require
>      at least five, usually ten minutes per @Benchmark to produce reasonably
>      trustworthy results.
> 
>      I can't speak for what others do, but we (Oracle) select what to run
>      regularly from the available microbenchmarks at our discretion, though.
>      That means adding a microbenchmark that takes a prohibitively long time
>      won't be disruptive to our nightly testing, but for obvious (mostly
>      economic) reasons we'd be unlikely to add an extremely long-running
>      micro to regular coverage.
> 
>      /Claes
> 
>      On 2020-10-27 10:06, Liu, Xin wrote:
>      > Hi,
>      >
>      > I follow the instruction on https://urldefense.com/v3/__https://htmlpreview.github.io/?https:**Agithub.com*openjdk*jdk*blob*master*doc*testing.html*microbenchmarks__;Ly8vLy8vLy8j!!GqivPVa7Brio!ND3mbOTCkCdkICq5NKxlPCWh7dWYhtMdQdg0GabOEKe0FlDTJDkj-lfNyWujH7AmDts$  to run a new microbenchmark.
>      > I got error message as follows.
>      > java.lang.UnsupportedClassVersionError: Preview features are not enabled for org/openjdk/bench/java/lang/reflect/proxy/jmh_generated/ProxyBench_newProxyInstance1i_jmhTest (class file version 60.65535). Try running with '--enable-preview'
>      >          at java.base/java.lang.ClassLoader.defineClass1(Native Method)
>      >          at java.base/java.lang.ClassLoader.defineClass(ClassLoader.java:1010)
>      >          at java.base/java.security.SecureClassLoader.defineClass(SecureClassLoader.java:150)
>      >          at java.base/jdk.internal.loader.BuiltinClassLoader.defineClass(BuiltinClassLoader.java:855)
>      >          at java.base/jdk.internal.loader.BuiltinClassLoader.findClassOnClassPathOrNull(BuiltinClassLoader.java:753)
>      >          at java.base/jdk.internal.loader.BuiltinClassLoader.loadClassOrNull(BuiltinClassLoader.java:676)
>      >          at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:634)
>      >          at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:182)
>      >          at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:519)
>      >          at java.base/java.lang.Class.forName0(Native Method)
>      >          at java.base/java.lang.Class.forName(Class.java:377)
>      >          at org.openjdk.jmh.util.ClassUtils.loadClass(ClassUtils.java:72)
>      >          at org.openjdk.jmh.runner.BenchmarkHandler.<init>(BenchmarkHandler.java:68)
>      >          at org.openjdk.jmh.runner.BaseRunner.runBenchmark(BaseRunner.java:232)
>      >          at org.openjdk.jmh.runner.BaseRunner.doSingle(BaseRunner.java:138)
>      >          at org.openjdk.jmh.runner.BaseRunner.runBenchmarksForked(BaseRunner.java:75)
>      >          at org.openjdk.jmh.runner.ForkedRunner.run(ForkedRunner.java:72)
>      >          at org.openjdk.jmh.runner.ForkedMain.main(ForkedMain.java:84)
>      >
>      >
>      > If I add an extra flag “—enable-preview” , the micro bench will work.
>      > make test TEST="micro:java.lang.reflect" MICRO="VM_OPTIONS=--enable-preview"
>      >
>      > I am using jmh from make/devkit/createJMHBundle.sh, which gives me jmh-core-1.26.jar
>      > Is that jmh-core too old or we should adjust RunTest.gmk a little bit to have that flag?
>      >
>      > Does Openjdk CI run benchmarks to detect performance regression?  If so,  could you point me to a guideline of microbenmark?
>      > I lack of the basic sense of a “good” microbenchmark.  My concern is the new micro I added is too long to break the established test run.
>      >
>      > Thanks,
>      > --lx
>      >
>      >
>