RFR(T): 8222299: [TESTBUG] Docker tests should be excluded from hotspot_runtime group

[David Holmes]

On 11/04/2019 12:38 pm, mikhailo.seledtsov at oracle.com wrote:
> Hi David,
> 
>    Thank you for your input.
> 
> 
> On 4/10/19 6:58 PM, David Holmes wrote:
>> Hi Misha,
>>
>> On 11/04/2019 11:44 am, Igor Ignatyev wrote:
>>> Hi Misha,
>>>
>>> although it is not formally stated anywhere, :hotspot_runtime is 
>>> expected to include _all_ jtreg-jtreg runtime related tests, that's 
>>> to say I'd expect to have docker tests included into this group, 
>>> regardless of a host's ability/inability to run them. hence I'd 
>>> prefer us to use Skipped exception in these tests.
>>
>> I agree with Igor - :hotspot_runtime is supposed to include 
>> everything. We exclude specific tests when we run more specific test 
>> groups like tier1_runtime. (Personally I no longer use groups for 
>> local testing but just directories.)
>>
>> If @requires is infeasible (and I can easily see it is) then using 
>> "skipped" exception is one approach - however I'm also concerned about 
>> wasting time failing to run these tests.
>>
>> Another option perhaps is to move "containers" to be a top-level test 
>> group alongside runtime, rather than within it?
> I agree. This is another good alternative. Would you recommend moving 
> these tests to test/hotspot/jtreg/containers/..., 

Yes that is what I was thinking.

Thanks,
David

> or should they move 
> under another "top-level" directory, such as 
> test/hotspot/jtreg/*misc*/containers  or similar ?
> 
> Igor,  do you have any input or opinion on this alternative?
> 
> 
> Thank you,
> Misha
>>
>> Thanks,
>> David
>>
>>> Thanks,
>>> -- Igor
>>>
>>>> On Apr 10, 2019, at 6:39 PM, mikhailo.seledtsov at oracle.com wrote:
>>>>
>>>> Hi Igor,
>>>>
>>>> Thank you for taking a look.
>>>>
>>>>
>>>> On 4/10/19 6:13 PM, Igor Ignatev wrote:
>>>>> Should these test be filtered out using @requires?
>>>> Checking all the conditions for this via @requires will require 
>>>> building a test docker image (or at least downloading the base/FROM) 
>>>> image in evaluation of at-requires, which will be unacceptably long, 
>>>> especially given that @requires is evaluated each time one runs 
>>>> jtreg command for any test in hotspot.
>>>>
>>>> Alternatively, if the current approach is undesirable, we can throw 
>>>> jtreg.Skipped exception if docker base image fails to download. Let 
>>>> me know if this is your preference.
>>>>
>>>> Misha
>>>>>
>>>>> — Igor
>>>>>
>>>>>> On Apr 10, 2019, at 6:06 PM, mikhailo.seledtsov at oracle.com wrote:
>>>>>>
>>>>>> Please review this small (trivial) change that excludes docker 
>>>>>> tests from hotspot_runtime. The rational for this change is that 
>>>>>> docker tests require specially configured environment (docker 
>>>>>> engine installed, test user being member of docker group, docker 
>>>>>> proxy or docker mirror repo configured). This may lead to 
>>>>>> unexpected errors when docker tests are ran as part of 
>>>>>> hotspot_runtime group in general environment(s).
>>>>>>
>>>>>>
>>>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8222299
>>>>>>
>>>>>> Change:
>>>>>>
>>>>>> diff --git a/test/hotspot/jtreg/TEST.groups 
>>>>>> b/test/hotspot/jtreg/TEST.groups
>>>>>> --- a/test/hotspot/jtreg/TEST.groups
>>>>>> +++ b/test/hotspot/jtreg/TEST.groups
>>>>>> @@ -44,7 +44,8 @@
>>>>>>     -gc/nvdimm
>>>>>>
>>>>>>   hotspot_runtime = \
>>>>>> -  runtime
>>>>>> +  runtime \
>>>>>> +  -runtime/containers/docker
>>>>>>
>>>>>>   hotspot_handshake = \
>>>>>>     runtime/handshake
>>>>>>
>>>>>> Testing:
>>>>>>
>>>>>>    jtreg -l 
>>>>>> /ws/hg/jdk/jdk/work01/open/test/hotspot/jtreg/:hotspot_runtime
>>>>>>
>>>>>>    jtreg -l 
>>>>>> /ws/hg/jdk/jdk/work01/open/test/hotspot/jtreg/:hotspot_runtime | 
>>>>>> grep docker
>>>>>>
>>>>>>
>>>>>> Thank you,
>>>>>>
>>>>>> Misha
>>>>>>
>>>>
>>>
>