RFR: 8006164 [TESTBUG] compact profile hotspot test issues

[Vladimir Kozlov]

 > Thanks. I assume you don't mind being the Reviewer for this?

Yes, it is reviewed.

Thanks,
Vladimir

On 8/26/13 8:36 PM, David Holmes wrote:
> On 27/08/2013 1:14 PM, Vladimir Kozlov wrote:
>> On 8/26/13 7:13 PM, David Holmes wrote:
>>> Hi Vladimir,
>>>
>>> On 27/08/2013 3:14 AM, Vladimir Kozlov wrote:
>>>> Thank you, David
>>>>
>>>> For me it looks odd when you specify group as not other flags, like:
>>>>
>>>> -group:compact1
>>>>
>>>> Why it was designed this way?
>>>
>>> Are you referring to the jtreg usage? ie
>>>
>>> jtreg -jdk:xxx -v -r reportdir :compact1
>>
>> Yes, I asked about that. From the examples in TEST.groups it was not
>> clear how to use them.
>>
>>>
>>> The group is not an option it is a test designator similar to doing:
>>
>> It is not destination from my understanding - it is named list of tests.
>
> Right, a group defines an explicit named list of tests, while gc/ "names" a list of tests implicitly by their location
> in the directory structure. Both form the "here are the tests to run" arguments to jtreg.
>
>> So I don't see why we can't use it as flag. I understand that I am late
>> and the feature already implemented in jtreg and it is only a matter how
>> jtreg parses command line. But it is unusual to have command parameter
>> starting with :.
>
> jtreg needs to be able to distinguish between the name of a test and the name of a group hence : is a group name
> delimiter. Actually the complete syntax allows another designator in front of the :
>
> testdir:group
>
> which I believe supports the fact that any testdir can contain a TEST.ROOT file which among other things can define the
> group file in which the group definition can be found.
>
> But this is all jtreg stuff :)
>
>>>
>>> jtreg -jdk:xxx -v -r reportdir runtime/NMT gc/
>>
>> Can you run subsets of tests from a group?:
>>
>>   jtreg -jdk:xxx -v -r reportdir :needs_jdk gc
>
> Not at present. A group defines a set of tests, so the above would be a union of the tests in group :needs_jdk and the
> tests under directory gc.
>
> I am currently arguing for a way to exclude a group on the command line so that you can "subset" to some degree eg:
>
> jtreg gc -:needs_jdk
>
> would run all gc tests except those listed in the needs_jdk group. This avoids the need to define composite groups for
> all permutations of tests that you might want to run.
>
> BTW the way jtreg produces the set of tests in a group is to first do the union of all included tests and groups, then
> do a union of the excluded tests and groups, and then subtract the second set from the first. And each group is
> evaluated in isolation before being used in another group (as opposed to logically being inlined and then the result
> evaluated).
>
>>>
>>>> Can ou add jprt group too (as template for now)? We want for long time
>>>> to run in JPRT small subset of our jtreg tests which will take less
>>>> 15min. I know, it is separate problem but can you add it as template
>>>> which we can use and extend later?
>>>
>>> Adding this is trivial. The "template" would be:
>>>
>>> jprt =
>>>
>>> There doesn't seem much point in me adding that at this time though.
>>> Better to leave it until we know how it needs to be
>>> defined.
>>
>> Okay.
>
> Thanks. I assume you don't mind being the Reviewer for this?
>
> It would be nice to get a second reviewer too. :)
>
> David
>
>> Thanks,
>> Vladimir
>>
>>>
>>> Thanks,
>>> David
>>>
>>>> Thanks,
>>>> Vladimir
>>>>
>>>> On 8/25/13 6:45 PM, David Holmes wrote:
>>>>> This change introduces the TEST.groups file to allow jtreg to run
>>>>> regression tests by groups - where the groups are defined to support
>>>>> testing of compact profiles and the minimal VM.
>>>>>
>>>>> webrev: http://cr.openjdk.java.net/~dholmes/8006164/webrev/
>>>>>
>>>>> The primary groups are:
>>>>>   - jdk
>>>>>   - jre
>>>>>   - compact3
>>>>>   - compact2
>>>>>   - compact2_minimal
>>>>>   - compact1
>>>>>   - compact1_minimal
>>>>>
>>>>> The minimal VM is only supported on compact1 and compact2.
>>>>>
>>>>> To select a group of tests you use :<group-name>
>>>>>
>>>>> Eg to run only those tests that can run on compact1 use:
>>>>>
>>>>> jtreg :compact1
>>>>>
>>>>> Of course you still need to point jtreg at the right kind of runtime
>>>>> image (and give it a full JDK as the compile-jdk!); and if testing the
>>>>> minimal VM you need to tell jtreg to select it using
>>>>> -javaoptions:-minimal
>>>>>
>>>>> The full jtreg group facility is only available in the most recent
>>>>> jtreg
>>>>> builds, so you will need to grab the latest nightly build, or latest
>>>>> sources.
>>>>>
>>>>> It is expected that these group definitions will need some tweaking. So
>>>>> far testing has been limited to linux, but we want to get this in ASAP
>>>>> to extend the testing.
>>>>>
>>>>> Note: once this is in place, anyone writing regression tests will need
>>>>> to be aware of whether that test is limited to certain profiles and
>>>>> update the group file accordingly. Sometimes it is not the item being
>>>>> tested that determines the minimum needed profile, but the test
>>>>> infrastructure eg if it uses XML.
>>>>>
>>>>> The changeset will be pushed via hotspot-embas that is what is used for
>>>>> profile testing.
>>>>>
>>>>> Thanks,
>>>>> David