RFR: 8006164 [TESTBUG] compact profile hotspot test issues

[Daniel D. Daugherty]

 > http://cr.openjdk.java.net/~dholmes/8006164/webrev/

test/TEST.ROOT
     No comments.

test/TEST.groups
     Just to be clear: The addition of any new test will require this file
     to updated? If so, then we'll have to be careful during reviews. What
     happens if a test isn't listed? Is it by default in the 'jdk' group?

Thumbs up.

Dan


On 8/27/13 2:55 AM, Vladimir Kozlov wrote:
> > Thanks. I assume you don't mind being the Reviewer for this?
>
> Yes, it is reviewed.
>
> Thanks,
> Vladimir
>
> On 8/26/13 8:36 PM, David Holmes wrote:
>> On 27/08/2013 1:14 PM, Vladimir Kozlov wrote:
>>> On 8/26/13 7:13 PM, David Holmes wrote:
>>>> Hi Vladimir,
>>>>
>>>> On 27/08/2013 3:14 AM, Vladimir Kozlov wrote:
>>>>> Thank you, David
>>>>>
>>>>> For me it looks odd when you specify group as not other flags, like:
>>>>>
>>>>> -group:compact1
>>>>>
>>>>> Why it was designed this way?
>>>>
>>>> Are you referring to the jtreg usage? ie
>>>>
>>>> jtreg -jdk:xxx -v -r reportdir :compact1
>>>
>>> Yes, I asked about that. From the examples in TEST.groups it was not
>>> clear how to use them.
>>>
>>>>
>>>> The group is not an option it is a test designator similar to doing:
>>>
>>> It is not destination from my understanding - it is named list of 
>>> tests.
>>
>> Right, a group defines an explicit named list of tests, while gc/ 
>> "names" a list of tests implicitly by their location
>> in the directory structure. Both form the "here are the tests to run" 
>> arguments to jtreg.
>>
>>> So I don't see why we can't use it as flag. I understand that I am late
>>> and the feature already implemented in jtreg and it is only a matter 
>>> how
>>> jtreg parses command line. But it is unusual to have command parameter
>>> starting with :.
>>
>> jtreg needs to be able to distinguish between the name of a test and 
>> the name of a group hence : is a group name
>> delimiter. Actually the complete syntax allows another designator in 
>> front of the :
>>
>> testdir:group
>>
>> which I believe supports the fact that any testdir can contain a 
>> TEST.ROOT file which among other things can define the
>> group file in which the group definition can be found.
>>
>> But this is all jtreg stuff :)
>>
>>>>
>>>> jtreg -jdk:xxx -v -r reportdir runtime/NMT gc/
>>>
>>> Can you run subsets of tests from a group?:
>>>
>>>   jtreg -jdk:xxx -v -r reportdir :needs_jdk gc
>>
>> Not at present. A group defines a set of tests, so the above would be 
>> a union of the tests in group :needs_jdk and the
>> tests under directory gc.
>>
>> I am currently arguing for a way to exclude a group on the command 
>> line so that you can "subset" to some degree eg:
>>
>> jtreg gc -:needs_jdk
>>
>> would run all gc tests except those listed in the needs_jdk group. 
>> This avoids the need to define composite groups for
>> all permutations of tests that you might want to run.
>>
>> BTW the way jtreg produces the set of tests in a group is to first do 
>> the union of all included tests and groups, then
>> do a union of the excluded tests and groups, and then subtract the 
>> second set from the first. And each group is
>> evaluated in isolation before being used in another group (as opposed 
>> to logically being inlined and then the result
>> evaluated).
>>
>>>>
>>>>> Can ou add jprt group too (as template for now)? We want for long 
>>>>> time
>>>>> to run in JPRT small subset of our jtreg tests which will take less
>>>>> 15min. I know, it is separate problem but can you add it as template
>>>>> which we can use and extend later?
>>>>
>>>> Adding this is trivial. The "template" would be:
>>>>
>>>> jprt =
>>>>
>>>> There doesn't seem much point in me adding that at this time though.
>>>> Better to leave it until we know how it needs to be
>>>> defined.
>>>
>>> Okay.
>>
>> Thanks. I assume you don't mind being the Reviewer for this?
>>
>> It would be nice to get a second reviewer too. :)
>>
>> David
>>
>>> Thanks,
>>> Vladimir
>>>
>>>>
>>>> Thanks,
>>>> David
>>>>
>>>>> Thanks,
>>>>> Vladimir
>>>>>
>>>>> On 8/25/13 6:45 PM, David Holmes wrote:
>>>>>> This change introduces the TEST.groups file to allow jtreg to run
>>>>>> regression tests by groups - where the groups are defined to support
>>>>>> testing of compact profiles and the minimal VM.
>>>>>>
>>>>>> webrev: http://cr.openjdk.java.net/~dholmes/8006164/webrev/
>>>>>>
>>>>>> The primary groups are:
>>>>>>   - jdk
>>>>>>   - jre
>>>>>>   - compact3
>>>>>>   - compact2
>>>>>>   - compact2_minimal
>>>>>>   - compact1
>>>>>>   - compact1_minimal
>>>>>>
>>>>>> The minimal VM is only supported on compact1 and compact2.
>>>>>>
>>>>>> To select a group of tests you use :<group-name>
>>>>>>
>>>>>> Eg to run only those tests that can run on compact1 use:
>>>>>>
>>>>>> jtreg :compact1
>>>>>>
>>>>>> Of course you still need to point jtreg at the right kind of runtime
>>>>>> image (and give it a full JDK as the compile-jdk!); and if 
>>>>>> testing the
>>>>>> minimal VM you need to tell jtreg to select it using
>>>>>> -javaoptions:-minimal
>>>>>>
>>>>>> The full jtreg group facility is only available in the most recent
>>>>>> jtreg
>>>>>> builds, so you will need to grab the latest nightly build, or latest
>>>>>> sources.
>>>>>>
>>>>>> It is expected that these group definitions will need some 
>>>>>> tweaking. So
>>>>>> far testing has been limited to linux, but we want to get this in 
>>>>>> ASAP
>>>>>> to extend the testing.
>>>>>>
>>>>>> Note: once this is in place, anyone writing regression tests will 
>>>>>> need
>>>>>> to be aware of whether that test is limited to certain profiles and
>>>>>> update the group file accordingly. Sometimes it is not the item 
>>>>>> being
>>>>>> tested that determines the minimum needed profile, but the test
>>>>>> infrastructure eg if it uses XML.
>>>>>>
>>>>>> The changeset will be pushed via hotspot-embas that is what is 
>>>>>> used for
>>>>>> profile testing.
>>>>>>
>>>>>> Thanks,
>>>>>> David
>
>