RFR: 8316814: NMT: A separate script or Java program is needed to analyze and make useful reports ouf of JMH benchmarks outputs. [v2]

[Afshin Zafari]

On Mon, 23 Oct 2023 08:58:54 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

>> Afshin Zafari has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   empty 'finally' block is removed.
>
> I agree having the JMH comparison tool is important, I just don't think that:
>  a) JDK microbenchmark suite is the proper home for it; looks like we need to support this natively in JMH instead (maybe as separate tool as well);
>  b) Parsing the human-readable result is a proper way to do this, as it is not guaranteed to be stable; JMH has machine-readable JSON output for external tools;

@shipilev  and @dholmes-ora 
I think since this tool (or even any other tool) is completely related/dependent to a specific benchmark, it should be put next to the benchmark. If we put it somewhere else, we should mention somehow which benchmark it is related to. There is a high dependency between the benchmark and the tool. Whenever the benchmark changes, the tool should be changed accordingly, since it relies on the names and formats used in the benchmark.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/16070#issuecomment-1794456841