VectorAPI: JMH performance testing infrastructure review
Vladimir Ivanov
vladimir.x.ivanov at oracle.com
Sat Aug 18 06:25:57 UTC 2018
Hi Jp,
Nice work!
Some comments:
The webrev doesn't include benchmark sources.
======
public static void main(String[] args) throws RunnerException {
Options opt = new OptionsBuilder()
.include(Byte128VectorTestsPerf.class.getSimpleName())
.forks(1)
.build();
new Runner(opt).run();
}
I'd prefer to see main() replaced with annotations and standard JMH
launcher to be used. In my experience, it's much more convenient.
Having a Maven project pre-generated would be a big plus as well.
======
static final int INVOC_COUNT =
Integer.getInteger("jdk.incubator.vector.test.loop-iterations", 100);
There's @Param("100") available. If JMH launcher is used, default value
can be overriden with "-p INVOC_COUNT=value".
======
@Benchmark
static void addByte128VectorTests() {
byte[] a = fa.apply(SPECIES.length());
byte[] b = fb.apply(SPECIES.length());
byte[] r = new byte[a.length];
Source/destination arrays allocation/initialization is part of the
benchmark. Why not move it to setup step (method marked with
@Setup(Level.Iteration)), cache arrays into instance fields, and reuse
them in the benchmarks?
======
static final IntFunction<byte[]> fa = (length) -> {
byte[] arr = new byte[length];
...
static void addByte128VectorTests() {
byte[] a = fa.apply(SPECIES.length());
...
for (int ic = 0; ic < INVOC_COUNT; ic++) {
for (int i = 0; i < a.length; i += SPECIES.length()) {
ByteVector<Shapes.S128Bit> av = SPECIES.fromArray(a, i);
ByteVector<Shapes.S128Bit> bv = SPECIES.fromArray(b, i);
av.add(bv).intoArray(r, i);
}
}
Arrays are precisely the size of vector. So, inner loop has only 1
iteration. I'd prefer to see INVOC_COUNT replaced with a parameter
controlling the size of the arrays.
Also, it's important to control data size to be able to exclude L1/2/3
cache effects from consideration.
Best regards,
Vladimir Ivanov
On 13/08/2018 18:41, Halimi, Jean-Philippe wrote:
> Dear all,
>
> I have uploaded a webrev allowing automatic generation of VectorAPI JMH performance test source files along with the regular unit tests that are already generated by the VectorAPI testing infrastructure.
>
> I had to modify the template files and the core scripting that handles the source generation. Please let me know what your thoughts are! :)
>
> You can find the webrev here: http://cr.openjdk.java.net/~jphalimi/webrev_jmh_tests_v1.0/webrev/
>
> The JMH performance test files have not been tested, but should work with Maven.
>
> Thank you,
>
> Jp
>
More information about the panama-dev
mailing list