RFR: 7903596: jmh-java-benchmark-archetype generated benchmark fails at runtime - ERROR: Unable to find the resource: /META-INF/BenchmarkList [v2]

[Andy Boothe]

On Tue, 8 Oct 2024 07:29:31 GMT, Jaikiran Pai <jpai at openjdk.org> wrote:

>> Can I please get a review of this change which proposes to address the issue noted in https://bugs.openjdk.org/browse/CODETOOLS-7903596?
>> 
>> As noted in that issue, when a benchmark generated using Java 22 is run, at runtime it fails with:
>> 
>> 
>> java -jar target/benchmarks.jar
>> Exception in thread "main" java.lang.RuntimeException: ERROR: Unable to find the resource: /META-INF/BenchmarkList
>> 	at org.openjdk.jmh.runner.AbstractResourceReader.getReaders(AbstractResourceReader.java:98)
>> 	at org.openjdk.jmh.runner.BenchmarkList.find(BenchmarkList.java:124)
>> 	at org.openjdk.jmh.runner.Runner.internalRun(Runner.java:252)
>> 	at org.openjdk.jmh.runner.Runner.run(Runner.java:208)
>> 	at org.openjdk.jmh.Main.main(Main.java:71)
>> 
>> 
>> This is due to annotation processors being disabled by default during compilation starting Java 22. The commit in this PR, explicitly enables the `org.openjdk.jmh.generators.BenchmarkProcessor` annotation processor to allow for it to generate the necessary benchmark resources.
>> 
>> I've tested this change locally, with Java 8, Java 21 as well as Java 22 and the newly generated benchmark is now functional on all these versions.
>
> Jaikiran Pai has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains two additional commits since the last revision:
> 
>  - merge latest from master branch
>  - 7903596: jmh-java-benchmark-archetype generated benchmark fails at runtime - ERROR: Unable to find the resource: /META-INF/BenchmarkList

I just got bit by this issue and lost a couple of hours. Having this archetype up-to-date would have helped. Instead, my developer experience looked like this:

1. Try to add JMH benchmarks to a new project
2. Consistently get above error about /META-INF/BenchmarkList
3. Try to run existing JMH benchmarks on existing projects
4. Those fail with /META-INF/BenchmarkList
5. Generate a fresh project from the archetype
6. Try to run JMH benchmarks against the archetype
7. Those fail with /META-INF/BenchmarkList
8. Bang head against wall
9. Check JMH PRs and find this issue
10. Retry builds with Java 17, and everything works

First, I could have saved some time if this archetype had been up-to-date, since it would have given me the thread of the relevant difference in the POM to pull on. Second, that I'm only discovering this through a PR I had to track down semi-manually means that there are probably a lot of folks who are having this issue but don't find the cause.

In my opinion, for whatever that's worth, I think merging this PR would be very helpful, since it would at least put this PR in the main branch to help people find it. I also think adding a "gotcha" to the README would help, too.

-------------

PR Comment: https://git.openjdk.org/jmh/pull/125#issuecomment-2453764992