make test TEST="micro:" got java.lang.UnsupportedClassVersionError

[Liu, Xin]

Hi, Claes, 

Thank you for fixing that. 
--lx


On 10/28/20, 4:07 PM, "Claes Redestad" <claes.redestad at oracle.com> wrote:

    CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.



    FWIW I just integrated the change to remove --enable-preview from
    the micros build, so the UnsupportedClassVersionError should be
    gone if you sync up. No --enable-preview needed.

    /Claes

    On 2020-10-28 11:09, Liu, Xin wrote:
    > Hi, Claes,
    >
    > Thank you for the great guideline of micro benchmarking.  now I have a clearer picture for it.
    > I'm happy to learn that 5-10 minutes for each Benchmark!  Let me polish mine.
    >
    > Thanks,
    > --lx
    >
    > On 10/27/20, 3:59 AM, "Claes Redestad" <claes.redestad at oracle.com> wrote:
    >
    >      CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.
    >
    >
    >
    >      Hi lx,
    >
    >      the need to add --enable-preview is known, and it's a bug introduced a
    >      couple of months ago thanks to a suggestion I did (without realizing
    >      that compiling one micro with --enable-preview would poison all of
    >      them..). This issue is tracked by a few bugs, see
    >      https://bugs.openjdk.java.net/browse/JDK-8250669 and
    >      https://bugs.openjdk.java.net/browse/JDK-8253828
    >
    >      The fix will likely be to remove the --enable-preview from the micros
    >      build after Records goes out of preview and then think more carefully on
    >      how to add support for benchmarking preview features.
    >
    >      As to your other question: When adding a microbenchmark it's appreciated
    >      if the default configuration makes it as quick to run as possible
    >      without sacrificing quality. Often the default number of forks,
    >      iteration runtime et.c. can be tuned down by using custom settings in
    >      @Fork et.c. Custom parameters used can be limited to a select handful.
    >
    >      Still it's normal for a "good" microbenchmark to require
    >      at least five, usually ten minutes per @Benchmark to produce reasonably
    >      trustworthy results.
    >
    >      I can't speak for what others do, but we (Oracle) select what to run
    >      regularly from the available microbenchmarks at our discretion, though.
    >      That means adding a microbenchmark that takes a prohibitively long time
    >      won't be disruptive to our nightly testing, but for obvious (mostly
    >      economic) reasons we'd be unlikely to add an extremely long-running
    >      micro to regular coverage.
    >
    >      /Claes
    >
    >      On 2020-10-27 10:06, Liu, Xin wrote:
    >      > Hi,
    >      >
    >      > I follow the instruction on https://urldefense.com/v3/__https://htmlpreview.github.io/?https:**Agithub.com*openjdk*jdk*blob*master*doc*testing.html*microbenchmarks__;Ly8vLy8vLy8j!!GqivPVa7Brio!ND3mbOTCkCdkICq5NKxlPCWh7dWYhtMdQdg0GabOEKe0FlDTJDkj-lfNyWujH7AmDts$  to run a new microbenchmark.
    >      > I got error message as follows.
    >      > java.lang.UnsupportedClassVersionError: Preview features are not enabled for org/openjdk/bench/java/lang/reflect/proxy/jmh_generated/ProxyBench_newProxyInstance1i_jmhTest (class file version 60.65535). Try running with '--enable-preview'
    >      >          at java.base/java.lang.ClassLoader.defineClass1(Native Method)
    >      >          at java.base/java.lang.ClassLoader.defineClass(ClassLoader.java:1010)
    >      >          at java.base/java.security.SecureClassLoader.defineClass(SecureClassLoader.java:150)
    >      >          at java.base/jdk.internal.loader.BuiltinClassLoader.defineClass(BuiltinClassLoader.java:855)
    >      >          at java.base/jdk.internal.loader.BuiltinClassLoader.findClassOnClassPathOrNull(BuiltinClassLoader.java:753)
    >      >          at java.base/jdk.internal.loader.BuiltinClassLoader.loadClassOrNull(BuiltinClassLoader.java:676)
    >      >          at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:634)
    >      >          at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:182)
    >      >          at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:519)
    >      >          at java.base/java.lang.Class.forName0(Native Method)
    >      >          at java.base/java.lang.Class.forName(Class.java:377)
    >      >          at org.openjdk.jmh.util.ClassUtils.loadClass(ClassUtils.java:72)
    >      >          at org.openjdk.jmh.runner.BenchmarkHandler.<init>(BenchmarkHandler.java:68)
    >      >          at org.openjdk.jmh.runner.BaseRunner.runBenchmark(BaseRunner.java:232)
    >      >          at org.openjdk.jmh.runner.BaseRunner.doSingle(BaseRunner.java:138)
    >      >          at org.openjdk.jmh.runner.BaseRunner.runBenchmarksForked(BaseRunner.java:75)
    >      >          at org.openjdk.jmh.runner.ForkedRunner.run(ForkedRunner.java:72)
    >      >          at org.openjdk.jmh.runner.ForkedMain.main(ForkedMain.java:84)
    >      >
    >      >
    >      > If I add an extra flag “—enable-preview” , the micro bench will work.
    >      > make test TEST="micro:java.lang.reflect" MICRO="VM_OPTIONS=--enable-preview"
    >      >
    >      > I am using jmh from make/devkit/createJMHBundle.sh, which gives me jmh-core-1.26.jar
    >      > Is that jmh-core too old or we should adjust RunTest.gmk a little bit to have that flag?
    >      >
    >      > Does Openjdk CI run benchmarks to detect performance regression?  If so,  could you point me to a guideline of microbenmark?
    >      > I lack of the basic sense of a “good” microbenchmark.  My concern is the new micro I added is too long to break the established test run.
    >      >
    >      > Thanks,
    >      > --lx
    >      >
    >      >
    >