RFR(T): 8222299: [TESTBUG] Docker tests should be excluded from hotspot_runtime group

[Igor Ignatyev]

> Igor,  do you have any input or opinion on this alternative?

I'm fine w/ 'test/hotspot/jtreg/(misc)/containers/' given we create :hotspot_containers/:hotspot_misc to include these tests and your changes don't change our current tier definitions.

-- Igor

> On Apr 10, 2019, at 7:38 PM, mikhailo.seledtsov at oracle.com wrote:
> 
> Hi David, 
> 
>   Thank you for your input.
> 
> 
> On 4/10/19 6:58 PM, David Holmes wrote:
>> Hi Misha, 
>> 
>> On 11/04/2019 11:44 am, Igor Ignatyev wrote: 
>>> Hi Misha, 
>>> 
>>> although it is not formally stated anywhere, :hotspot_runtime is expected to include _all_ jtreg-jtreg runtime related tests, that's to say I'd expect to have docker tests included into this group, regardless of a host's ability/inability to run them. hence I'd prefer us to use Skipped exception in these tests. 
>> 
>> I agree with Igor - :hotspot_runtime is supposed to include everything. We exclude specific tests when we run more specific test groups like tier1_runtime. (Personally I no longer use groups for local testing but just directories.) 
>> 
>> If @requires is infeasible (and I can easily see it is) then using "skipped" exception is one approach - however I'm also concerned about wasting time failing to run these tests. 
>> 
>> Another option perhaps is to move "containers" to be a top-level test group alongside runtime, rather than within it? 
> I agree. This is another good alternative. Would you recommend moving these tests to test/hotspot/jtreg/containers/..., or should they move under another "top-level" directory, such as test/hotspot/jtreg/misc/containers  or similar ?
> 
> Igor,  do you have any input or opinion on this alternative?
> 
> 
> Thank you,
> Misha
>> 
>> Thanks, 
>> David 
>> 
>>> Thanks, 
>>> -- Igor 
>>> 
>>>> On Apr 10, 2019, at 6:39 PM, mikhailo.seledtsov at oracle.com <mailto:mikhailo.seledtsov at oracle.com> wrote: 
>>>> 
>>>> Hi Igor, 
>>>> 
>>>> Thank you for taking a look. 
>>>> 
>>>> 
>>>> On 4/10/19 6:13 PM, Igor Ignatev wrote: 
>>>>> Should these test be filtered out using @requires? 
>>>> Checking all the conditions for this via @requires will require building a test docker image (or at least downloading the base/FROM) image in evaluation of at-requires, which will be unacceptably long, especially given that @requires is evaluated each time one runs jtreg command for any test in hotspot. 
>>>> 
>>>> Alternatively, if the current approach is undesirable, we can throw jtreg.Skipped exception if docker base image fails to download. Let me know if this is your preference. 
>>>> 
>>>> Misha 
>>>>> 
>>>>> — Igor 
>>>>> 
>>>>>> On Apr 10, 2019, at 6:06 PM, mikhailo.seledtsov at oracle.com <mailto:mikhailo.seledtsov at oracle.com> wrote: 
>>>>>> 
>>>>>> Please review this small (trivial) change that excludes docker tests from hotspot_runtime. The rational for this change is that docker tests require specially configured environment (docker engine installed, test user being member of docker group, docker proxy or docker mirror repo configured). This may lead to unexpected errors when docker tests are ran as part of hotspot_runtime group in general environment(s). 
>>>>>> 
>>>>>> 
>>>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8222299 <https://bugs.openjdk.java.net/browse/JDK-8222299> 
>>>>>> 
>>>>>> Change: 
>>>>>> 
>>>>>> diff --git a/test/hotspot/jtreg/TEST.groups b/test/hotspot/jtreg/TEST.groups 
>>>>>> --- a/test/hotspot/jtreg/TEST.groups 
>>>>>> +++ b/test/hotspot/jtreg/TEST.groups 
>>>>>> @@ -44,7 +44,8 @@ 
>>>>>>     -gc/nvdimm 
>>>>>> 
>>>>>>   hotspot_runtime = \ 
>>>>>> -  runtime 
>>>>>> +  runtime \ 
>>>>>> +  -runtime/containers/docker 
>>>>>> 
>>>>>>   hotspot_handshake = \ 
>>>>>>     runtime/handshake 
>>>>>> 
>>>>>> Testing: 
>>>>>> 
>>>>>>    jtreg -l /ws/hg/jdk/jdk/work01/open/test/hotspot/jtreg/:hotspot_runtime 
>>>>>> 
>>>>>>    jtreg -l /ws/hg/jdk/jdk/work01/open/test/hotspot/jtreg/:hotspot_runtime | grep docker 
>>>>>> 
>>>>>> 
>>>>>> Thank you, 
>>>>>> 
>>>>>> Misha 
>>>>>> 
>>>> 
>>> 
>