Proposal: Java Microbenchmark Harness (JMH)

[Jonathan Gibbons]

In the absence of any discussion, this proposal is accepted.

Welcome to Code Tools.

-- Jon

On 03/11/2013 02:09 PM, Jonathan Gibbons wrote:
> This proposal was posted over two weeks ago.
>
> If anyone has any comments, please post them by the
> end of Friday, March 15.
>
> -- Jon
>
>
> On 02/21/2013 06:40 AM, Aleksey Shipilev wrote:
>> Hi,
>>
>> I'd like to propose the new tool for inclusion to Codetools Project
>> within the OpenJDK. Please find the proposal below:
>>
>> ----8<---------------------------------------------------------------
>>
>> Tool Name:
>> Java Microbenchmark Harness (JMH)
>>
>> Tool Purpose:
>> Java harness for building, running, and analysing the nano/micro/macro
>> benchmarks, written in Java and other languages targeting the JVM.
>>
>> Proposed By:
>> Aleksey Shipilev, Oracle, Java SE Performance team
>>
>> Rationale:
>> Measuring the performance is the fine art, and measuring the performance
>> on microbenchmarks is double so. There are multiple caveats one should
>> take a great care of while designing the experiment involving
>> microbenchmarks. The advanced experience with VM technology is required,
>> the exposure with particular nits on exact VM is also a plus.
>>
>> JMH is designed to aid the development of reliable microbenchmarks. Our
>> experience shows there is the clear need for the user-friendly way to
>> build ad-hoc benchmarks for JDK development and performance research.
>> None of the tools in the wild are meeting all the needs required for
>> this kind of work.
>>
>> Including JMH in the Code Tools project allows us to use HotSpot- and
>> OpenJDK-specific functionality available only on this platform to get
>> better support for benchmarking. (Examples are: sun.misc.*, compiler
>> whitebox testing API, etc.)
>>
>> It is also important to have the harness to keep up with the latest
>> API/ABI changes, available ahead of GA for a new release of JDK. It had
>> already been proved useful when dealing with JSR166, JSR292, Lambda
>> performance assessments, and other bleeding edge platform improvements.
>>
>> In addition, JMH samples will guide the developers through the general
>> benchmarking pitfalls, as well as OpenJDK-specific nits, which will
>> greatly improve the quality of the ad-hoc performance testing in 
>> OpenJDK.
>>
>> ----8<---------------------------------------------------------------
>>
>> Thanks,
>> Aleksey.
>