RFR: 8316814: NMT: A separate script or Java program is needed to analyze and make useful reports ouf of JMH benchmarks outputs. [v2]

[David Holmes]

On Mon, 6 Nov 2023 09:55:10 GMT, Afshin Zafari <azafari at openjdk.org> wrote:

>> I agree having the JMH comparison tool is important, I just don't think that:
>>  a) JDK microbenchmark suite is the proper home for it; looks like we need to support this natively in JMH instead (maybe as separate tool as well);
>>  b) Parsing the human-readable result is a proper way to do this, as it is not guaranteed to be stable; JMH has machine-readable JSON output for external tools;
>
> @shipilev  and @dholmes-ora 
> I think since this tool (or even any other tool) is completely related/dependent to a specific benchmark, it should be put next to the benchmark. If we put it somewhere else, we should mention somehow which benchmark it is related to. There is a high dependency between the benchmark and the tool. Whenever the benchmark changes, the tool should be changed accordingly, since it relies on the names and formats used in the benchmark.

Sorry @afshin-zafari  I'm really not sure what to do about this one, but it seems a JMH issue.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/16070#issuecomment-1821948407