Vector API testing infrastructure

[Halimi, Jean-Philippe]

Dear both,

Thanks for your feedback. I will provide Razvan with the latest webrev today so that he can upload it.

@Paul, as far as the scripts go, I have already revised it to dissociate generation and compilation. There's one optional script left to run if you want to re-generate the scripts.
@Razvan, I will leave the compilation / run script here for your convenience. I am totally fine with keeping both. :-)

Jp

-----Original Message-----
From: Paul Sandoz [mailto:paul.sandoz at oracle.com] 
Sent: Tuesday, February 13, 2018 9:04 AM
To: Halimi, Jean-Philippe <jean-philippe.halimi at intel.com>
Cc: Vladimir Ivanov <vladimir.x.ivanov at oracle.com>; panama-dev at openjdk.java.net
Subject: Re: Vector API testing infrastructure



> On Feb 12, 2018, at 1:14 PM, Halimi, Jean-Philippe <jean-philippe.halimi at intel.com> wrote:
> 
> Hi all,
>  
> Thanks a lot for your feedback. It's great to see the enthusiasm around this project! J I have already addressed some of your concerns regarding the Testing API. I’m trying here to summarize where we are and what is next.
>  
> Right now, the situation is that we have:
> ·         A bunch of features we want to add
> ·         A bunch of design modifications we want to perform
> ·         A bunch of tests to be added/written by various people
>  
> Here is a summary of the feedback I have so far. Feel free to complete/edit if I missed anything. J
> 1.       Jtreg integration

And the interim documentation can be update to reference testng from jtreg too.


> 2.       Java/javac environment-defined
> 3.       Abstract data generation in a user-defined into a data provider
> 4.       Command-line controlled vector size
>  
> I have already taken care of the 2 first items and it seems to work as is with jtreg. Here are now my questions:
> 1.       Do you think it is reasonable to merge the code as is (as soon as I get my current webrev ready), so that people can get familiar with the interface, and then proceed with the rest?

Yes.


> 2.       Now that we can run the tests with jtreg, do you think we should keep the run_tests.sh script or just let it to the user to use his favorite jtreg command?

I think it ok to leave it there for now for others who are less comfortable with jtreg, but at some point i anticipate it should go away. (We had a similar strategy with Stream API tests.)

I suggest that you separate compilation of the tests from generation of the tests, perhaps move it into the run script?

BTW when you run with jtreg with the verbose option it will output text that is a script to run a test explicitly, so one can copy and paste it into a file and run. I often use that approach for more intensive debugging either of the JDK or HotSpot. The only down side i recall is it may not be possible to select a particular test method to run, but we could modify jtreg to support that. 

Paul.