[vector] RFR: JMH benchmarks improvements

[Halimi, Jean-Philippe]

Hi Vladimir,

I think the changes look great! Thanks a lot for continuing this work. Great work!
And as a side note, thanks for fixing the Cygwin error message.

-Jp

-----Original Message-----
From: panama-dev [mailto:panama-dev-bounces at openjdk.java.net] On Behalf Of Vladimir Ivanov
Sent: Thursday, November 29, 2018 4:52 PM
To: panama-dev at openjdk.java.net
Subject: [vector] RFR: JMH benchmarks improvements

Hi,

I've spent some time improving microbenchmark suite which is derived from unit tests:

 
http://cr.openjdk.java.net/~vlivanov/panama/vector/benchmarks/jmh_project.shared/

   * Introduce JMH project. It enables stand-alone build (bundled as
uber-jar) of microbenchmarks equipped with JMH launcher.

   * Some cleanups in scripts & templates.

Generated benchmarks:
 
http://cr.openjdk.java.net/~vlivanov/panama/vector/benchmarks/jmh_project.generated/


Also, some refactorings on top of it:
 
http://cr.openjdk.java.net/~vlivanov/panama/vector/benchmarks/improvements.shared/

   * Get rid of outer loop in benchmarks: INVOC_COUNT=1 and reduce data initialization overhead;


Changes in generated files:
 
http://cr.openjdk.java.net/~vlivanov/panama/vector/benchmarks/improvements.generated/


Some examples:

- List all available benchmarks:

   $ java -jar vector-benchmarks.jar -l '.*'


- Run a subset of benchmarks (defined as a regexp):

   $ java -jar vector-benchmarks.jar -l '.*Int.*add$'
   $ java -jar vector-benchmarks.jar    '.*Int.*add$'


- Control execution configuration (2 runs x 5 warmup/measurement 
iterations x 10 each):

   $ java -jar vector-benchmarks.jar -f 2 -i 5 -wi 5 -r 10 -w 10 '.*'


- Run with different data set sizes:

   $ java -jar vector-benchmarks.jar -p 16,1024,65536 '.*'

Best regards,
Vladimir Ivanov