[vector] RFR: JMH benchmarks improvements

[Vladimir Ivanov]

Thanks, Jp! Pushed.

I did additional round of refactorings/bug fixes and introduced scalar 
variants to simplify performance comparisons.

I haven't done any extensive performance validation testing, just 
ensured the benchmarks & tests work.

Best regards,
Vladimir Ivanov

On 30/11/2018 10:54, Halimi, Jean-Philippe wrote:
> Hi Vladimir,
> 
> I think the changes look great! Thanks a lot for continuing this work. Great work!
> And as a side note, thanks for fixing the Cygwin error message.
> 
> -Jp
> 
> -----Original Message-----
> From: panama-dev [mailto:panama-dev-bounces at openjdk.java.net] On Behalf Of Vladimir Ivanov
> Sent: Thursday, November 29, 2018 4:52 PM
> To: panama-dev at openjdk.java.net
> Subject: [vector] RFR: JMH benchmarks improvements
> 
> Hi,
> 
> I've spent some time improving microbenchmark suite which is derived from unit tests:
> 
>   
> http://cr.openjdk.java.net/~vlivanov/panama/vector/benchmarks/jmh_project.shared/
> 
>     * Introduce JMH project. It enables stand-alone build (bundled as
> uber-jar) of microbenchmarks equipped with JMH launcher.
> 
>     * Some cleanups in scripts & templates.
> 
> Generated benchmarks:
>   
> http://cr.openjdk.java.net/~vlivanov/panama/vector/benchmarks/jmh_project.generated/
> 
> 
> Also, some refactorings on top of it:
>   
> http://cr.openjdk.java.net/~vlivanov/panama/vector/benchmarks/improvements.shared/
> 
>     * Get rid of outer loop in benchmarks: INVOC_COUNT=1 and reduce data initialization overhead;
> 
> 
> Changes in generated files:
>   
> http://cr.openjdk.java.net/~vlivanov/panama/vector/benchmarks/improvements.generated/
> 
> 
> Some examples:
> 
> - List all available benchmarks:
> 
>     $ java -jar vector-benchmarks.jar -l '.*'
> 
> 
> - Run a subset of benchmarks (defined as a regexp):
> 
>     $ java -jar vector-benchmarks.jar -l '.*Int.*add$'
>     $ java -jar vector-benchmarks.jar    '.*Int.*add$'
> 
> 
> - Control execution configuration (2 runs x 5 warmup/measurement
> iterations x 10 each):
> 
>     $ java -jar vector-benchmarks.jar -f 2 -i 5 -wi 5 -r 10 -w 10 '.*'
> 
> 
> - Run with different data set sizes:
> 
>     $ java -jar vector-benchmarks.jar -p 16,1024,65536 '.*'
> 
> Best regards,
> Vladimir Ivanov
>