RFR: 8259074: regex benchmarks and tests

[Claes Redestad]

On Tue, 2 Feb 2021 02:58:30 GMT, Martin Buchholz <martin at openjdk.org> wrote:

>>> A manual exploration of a new set of micros would naturally start with
>>> the default config, so if such a config runs forever, that would be poor
>>> ergonomics IMHO. I don't think such configurations should be checked in
>>> in an active state.
>> 
>> We're actually in agreement.  There's no actual O(2^N) operation here, and jmh is similar to jtreg in having timeouts indicating failure.
>> 
>> Although I've been using shell loops as included in the class comments, I'll make sure running the tests using all the defaults gives sensible results.
>> 
>> I'm surprised to see jmh use the same number (5) of warmup and measurement iterations.  Unless you're looking for very small effects, one warmup run should be sufficient.
>
> I added annotations for sensible (faster) default, including @Warmup(iterations = 1)

> I added annotations for sensible (faster) default, including `@WarmUp(iterations = 1)`

Perhaps just a personal preference.. if you instead tune down time per iteration (`
@Warmup(iterations = 5, time = 2)`) you would get the same warmup time budget but also some immediate feedback that the code is stabilizing well before switching over to measurement iterations.

-------------

PR: https://git.openjdk.java.net/jdk/pull/1940