RFR: [DRAFT] Support Java literals in @Param annotation

[Aleksey Shipilev]

On Thu, 9 Sep 2021 16:27:44 GMT, Vsevolod Tolstopyatov <duke at openjdk.java.net> wrote:

> This is a follow-up to https://mail.openjdk.java.net/pipermail/jmh-dev/2020-October/003039.html
> 
> I've prototyped it and found few corners which I'd like to highlight and discuss.
> 
> The initial purpose of the change is to be able to use underscores for the sake of better readability in both code and text reports:
> 
> `@Param("1_000", "10_000", "100_000")` 
> 
> and
> 
> 
> Benchmark        (listSize)   Mode  Cnt      Score   Error  Units
> ShowOff.consume       1_000  thrpt       42294.795          ops/s
> ShowOff.consume      10_000  thrpt        4057.322          ops/s
> ShowOff.consume     100_000  thrpt        2866.069          ops/s
> 
> 
> 
> While doing so, I've decided that it would be nice to support the strict subset of actual Java literals (at least to have a reference to point at) which has a lot of pitfalls:
> 
> * Now we have (?) to support octal, binary and hexadecimal literals
> * The actual grammar is quite wild, so literals like `0_12___3_4` are also supported
> * `l` and `L` at the end of `long` literals are not supported (it seems a bit weird to do so)
> * `NumericLiteralsParser` leaks to public API, so the generated benchmarks can invoke it (not sure what's the API policy here, but it's worth pointing it out)
> * Underscores in floating points literals are not supported, mostly because then I have to match hexadecimal literals with underscores as well. No particular reason apart from "it's extra work without clear benefits". I can implement it if it's required for the PR though.
> 
> Most importantly, support of such literals will break existing benchmarks that prefixed their numeric parameters with zero -- now they are interpreted as octal literals (and either rejected, e.g. `09` or converted to decimal, breaking the original authors intent). Previously, `X.parseX` functions (e.g. `Integer.parseInt`) were used and they are okay with leading zeroes.
> 
> Breaking change seems to be the blocker here, and I'd like to hear your opinion of the acceptable approach aligned with JMH API
> 
> Potential alternatives:
> 
> * Do nothing, don't support underscores
> * Support only decimal literals
> * Support any subset of literals (e.g. decimal and hexadecimal only)
> * Just trim all the underscores and parse it as before via `Integer.parseInt` and so on

Hi, sorry for extremely late reply. This is a tough one compatibility-wise. I think it would be safer to:

- Drop octal literals support, leave only binary, decimal, hexadecimal; they seem to be unambiguous when applied over integral types;
- Move `NumericLiteralsParser` into generated benchmark code to avoid exposure to external code; yes, that would make unit testing much less convenient, you would have to do real JMH tests in `jmh-core-it` to reach the code; and put negative tests in `jmh-core-ct`, so we can check that known-bad values are rejected at compile time;
- Need additional test to verify that `@Param` over `String` does not accidentally converts the values;
- Figure out how hard would it be to support floating-point values, I'd think this one is useful in floating-point benchmarks `HexadecimalFloatingPointLiteral HexSignificand BinaryExponent`

Don't invest much time into this, as we might run into some unknown blocker going forward, which would derail all this.

-------------

PR: https://git.openjdk.java.net/jmh/pull/50