Vector API testing infrastructure

[Vladimir Ivanov]

> 
> It might even be possible make the test methods generic over the shape, alas generic enums are problematic, but the enums can implement a generic interface, and an instance of that interface is passed to the test method as an argument.

You can push it even further and end up without any template generation 
at all, only with abstract code shapes. For example, operations can be 
represented as MethodHandles and applied using MH.invoke(), arrays 
erased to Objects.

> 
> A downside of this approach is the actual intrinsic is hidden behind a function. I suspect this is not an issue with regards to actual generation of a vector hardware instruction but would like some confirmation, Razvan/Vladimir?

Intrinsification will reliably happen with generic intrinsics, but for 
original implementation it depends on what is passed in as 
Int512BinaryOp. If it's a lambda which refers to concrete types, then 
intrinsification will happen as well. But no inlining in both cases.

> 
> If not it still might be an issue with regards to the effectiveness of vector box eliding and loop unrolling etc. But, i argue for such tests it does not matter as these are primarily functional tests not performance tests, which should be something separate that focus on using JMH.

If Vector API were a JDK-only feature, I'd agree with you here. But we 
have substantial portion of code in JVM and we need to cover it with 
tests as well. Benchmarks have completely different purpose than unit 
tests and having 2 (mostly duplicating) sets of tests for JDK & JVM 
parts looks like an overkill.

So, I'd prefer to see the tests to be both JDK- & JVM-friendly: checking 
correctness with code shapes amenable to optimizations.

Best regards,
Vladimir Ivanov

> If we need to generate specific operation tests i would still suggest using an enum for the scalar operations and test assertion, thus it can be shared across shapes. I like the generated code to be as if i would have written it by hand :-)
> 
> Further, i wonder if we could then use the enum values to drive test method generation, perhaps something to consider further on (pushing more of such logic into Java code rather than bash).
> 
> Paul.
> 
> [1] https://ci.adoptopenjdk.net/view/Dependencies/job/jtreg/
> 
> 
>> On Feb 8, 2018, at 2:06 PM, Halimi, Jean-Philippe <jean-philippe.halimi at intel.com> wrote:
>>
>> Thanks a lot for your feedback.
>>
>> I agree with all of your points so far. I will work on that in the coming days.
>>
>> Thanks to Razvan and Vivek's help, I published a new follow-up patch adding generated tests.
>>
>> http://cr.openjdk.java.net/~rlupusoru/panama/webrev_vector_api_test_infra_01/
>>
>> Cheers,
>>
>> Jp
>>
>> -----Original Message-----
>> From: Vladimir Ivanov [mailto:vladimir.x.ivanov at oracle.com]
>> Sent: Thursday, February 8, 2018 1:38 PM
>> To: Halimi, Jean-Philippe <jean-philippe.halimi at intel.com>; panama-dev at openjdk.java.net
>> Cc: Viswanathan, Sandhya <sandhya.viswanathan at intel.com>
>> Subject: Re: Vector API testing infrastructure
>>
>> Nice work, Jp! Looks very good!
>>
>>> It will be helpful to include the generated test cases as well.
>>
>> I second that. Please, add generated test cases as well.
>>
>> Some other comments:
>>
>>    * testng in test path is useless
>>
>> Do you see any problems with putting files right into test/jdk/jdk/incubator/vector/?
>>
>>    * In the longer term, there should be only 1 script left: the one which generates test sources from templates.
>>
>> Best regards,
>> Vladimir Ivanov
>>
>>> -----Original Message-----
>>> From: panama-dev [mailto:panama-dev-bounces at openjdk.java.net] On
>>> Behalf Of Halimi, Jean-Philippe
>>> Sent: Thursday, February 08, 2018 10:52 AM
>>> To: panama-dev at openjdk.java.net
>>> Subject: Vector API testing infrastructure
>>>
>>> Hi,
>>>
>>> I am a new contributor to this list. I am contributing Vector API testing framework. Please have a look at the webrev below to learn more about it. It features a README file in case you want to generate, build and run the tests.
>>> The generated source code is not part of the initial commit, but they can be added if need be.
>>>
>>> http://cr.openjdk.java.net/~rlupusoru/panama/webrev_vector_api_test_in
>>> fra_00/
>>>
>>> Cheers,
>>>
>>> Jp
>>>
>