RFR: 8064799: [TESTBUG] JT-Reg Serviceability tests to be run as part of JPRT submit job

[Alan Bateman]

On 14/11/2014 08:40, Staffan Larsen wrote:
> :
>
> So the goal here has been to increase the test coverage of hotspot 
> jprt push jobs, but with a limited impact on execution time. This is 
> all to make sure hotspot changes do no break serviceability features. 
> While it would be great to run all tests at all times, we don’t have 
> time for that. Mikael has been doing code coverage analysis to find 
> the subset of test that provides the biggest bang for the buck. 
> Starting with JDI is as good as any place to start.
>
> I agree that listing individual tests is not particularly appealing, 
> but I don’t see many other options. We could possibly use @key tags to 
> select the tests but there isn’t much support in makefiles and jprt 
> for that if I recall. We could use sub-folders, but that quickly gets 
> out of hand.
>
> We could move the _sanity lists to one place in the file to make it 
> easier to see the rest.
>

My main concern is keeping the test group hierarchy easy to understand 
and maintain. It has to be easy to identify any tests that aren't run by 
any of the top-level groups for example. For serviceability then the 
original idea was to have the jdk_svc group that ran all of the tests, 
this in turn consisted of 5 sub-groups to cover the various areas. This 
sub-groups will execute concurrently on different JPRT clients and works 
reasonably well, albeit with some imbalance in the execution time.

The "real" definitions of the groups end just after the section with the 
comment "Client area groups" so if you can move the new sanity groups 
down to below that with a good comment to explain what they are then it 
should okay.

At some point we need to look at removing completely the "Profile based 
..." groups at the end. They are completely unmaintainable and there are 
much better ways of doing this now (with @requires).

-Alan


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.openjdk.java.net/pipermail/serviceability-dev/attachments/20141114/cd49bd76/attachment.html>