Vector API testing infrastructure

[Paul Sandoz]

> On Feb 12, 2018, at 1:14 PM, Halimi, Jean-Philippe <jean-philippe.halimi at intel.com> wrote:
> 
> Hi all,
>  
> Thanks a lot for your feedback. It's great to see the enthusiasm around this project! J I have already addressed some of your concerns regarding the Testing API. I’m trying here to summarize where we are and what is next.
>  
> Right now, the situation is that we have:
> ·         A bunch of features we want to add
> ·         A bunch of design modifications we want to perform
> ·         A bunch of tests to be added/written by various people
>  
> Here is a summary of the feedback I have so far. Feel free to complete/edit if I missed anything. J
> 1.       Jtreg integration

And the interim documentation can be update to reference testng from jtreg too.


> 2.       Java/javac environment-defined
> 3.       Abstract data generation in a user-defined into a data provider
> 4.       Command-line controlled vector size
>  
> I have already taken care of the 2 first items and it seems to work as is with jtreg. Here are now my questions:
> 1.       Do you think it is reasonable to merge the code as is (as soon as I get my current webrev ready), so that people can get familiar with the interface, and then proceed with the rest?

Yes.


> 2.       Now that we can run the tests with jtreg, do you think we should keep the run_tests.sh script or just let it to the user to use his favorite jtreg command?

I think it ok to leave it there for now for others who are less comfortable with jtreg, but at some point i anticipate it should go away. (We had a similar strategy with Stream API tests.)

I suggest that you separate compilation of the tests from generation of the tests, perhaps move it into the run script?

BTW when you run with jtreg with the verbose option it will output text that is a script to run a test explicitly, so one can copy and paste it into a file and run. I often use that approach for more intensive debugging either of the JDK or HotSpot. The only down side i recall is it may not be possible to select a particular test method to run, but we could modify jtreg to support that. 

Paul.