RFR(T): 8222299: [TESTBUG] Docker tests should be excluded from hotspot_runtime group

[mikhailo.seledtsov at oracle.com]

David, Igor,

   Thank you for looking at my change, and providing your advice.

I will then proceed with the following:

   - hg move the container tests from 
test/hotspot/jtreg/runtime/containers to test/hotspot/jtreg/containers

   - will create :hotspot_containers group

   - update test definitions accordingly (no changes except to use 
hotspot_containers group instead of the directory path)

I will change the JBS issue description, but keep using the same issue. 
I will post the updated webrev shortly, with the updated subject.

I am closing this thread.


Thank you,

Misha


On 4/10/19 11:13 PM, Igor Ignatyev wrote:
>> Igor,  do you have any input or opinion on this alternative?
>
> I'm fine w/ 'test/hotspot/jtreg/(misc)/containers/' given we create 
> :hotspot_containers/:hotspot_misc to include these tests and your 
> changes don't change our current tier definitions.
>
> -- Igor
>
>> On Apr 10, 2019, at 7:38 PM, mikhailo.seledtsov at oracle.com 
>> <mailto:mikhailo.seledtsov at oracle.com> wrote:
>>
>> Hi David,
>>
>>   Thank you for your input.
>>
>>
>> On 4/10/19 6:58 PM, David Holmes wrote:
>>> Hi Misha,
>>>
>>> On 11/04/2019 11:44 am, Igor Ignatyev wrote:
>>>> Hi Misha,
>>>>
>>>> although it is not formally stated anywhere, :hotspot_runtime is 
>>>> expected to include _all_ jtreg-jtreg runtime related tests, that's 
>>>> to say I'd expect to have docker tests included into this group, 
>>>> regardless of a host's ability/inability to run them. hence I'd 
>>>> prefer us to use Skipped exception in these tests.
>>>
>>> I agree with Igor - :hotspot_runtime is supposed to include 
>>> everything. We exclude specific tests when we run more specific test 
>>> groups like tier1_runtime. (Personally I no longer use groups for 
>>> local testing but just directories.)
>>>
>>> If @requires is infeasible (and I can easily see it is) then using 
>>> "skipped" exception is one approach - however I'm also concerned 
>>> about wasting time failing to run these tests.
>>>
>>> Another option perhaps is to move "containers" to be a top-level 
>>> test group alongside runtime, rather than within it?
>> I agree. This is another good alternative. Would you recommend moving 
>> these tests to test/hotspot/jtreg/containers/..., or should they move 
>> under another "top-level" directory, such as 
>> test/hotspot/jtreg/*misc*/containers  or similar ?
>>
>> Igor,  do you have any input or opinion on this alternative?
>>
>>
>> Thank you,
>> Misha
>>>
>>> Thanks,
>>> David
>>>
>>>> Thanks,
>>>> -- Igor
>>>>
>>>>> On Apr 10, 2019, at 6:39 PM, mikhailo.seledtsov at oracle.com wrote:
>>>>>
>>>>> Hi Igor,
>>>>>
>>>>> Thank you for taking a look.
>>>>>
>>>>>
>>>>> On 4/10/19 6:13 PM, Igor Ignatev wrote:
>>>>>> Should these test be filtered out using @requires?
>>>>> Checking all the conditions for this via @requires will require 
>>>>> building a test docker image (or at least downloading the 
>>>>> base/FROM) image in evaluation of at-requires, which will be 
>>>>> unacceptably long, especially given that @requires is evaluated 
>>>>> each time one runs jtreg command for any test in hotspot.
>>>>>
>>>>> Alternatively, if the current approach is undesirable, we can 
>>>>> throw jtreg.Skipped exception if docker base image fails to 
>>>>> download. Let me know if this is your preference.
>>>>>
>>>>> Misha
>>>>>>
>>>>>> — Igor
>>>>>>
>>>>>>> On Apr 10, 2019, at 6:06 PM, mikhailo.seledtsov at oracle.com wrote:
>>>>>>>
>>>>>>> Please review this small (trivial) change that excludes docker 
>>>>>>> tests from hotspot_runtime. The rational for this change is that 
>>>>>>> docker tests require specially configured environment (docker 
>>>>>>> engine installed, test user being member of docker group, docker 
>>>>>>> proxy or docker mirror repo configured). This may lead to 
>>>>>>> unexpected errors when docker tests are ran as part of 
>>>>>>> hotspot_runtime group in general environment(s).
>>>>>>>
>>>>>>>
>>>>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8222299
>>>>>>>
>>>>>>> Change:
>>>>>>>
>>>>>>> diff --git a/test/hotspot/jtreg/TEST.groups 
>>>>>>> b/test/hotspot/jtreg/TEST.groups
>>>>>>> --- a/test/hotspot/jtreg/TEST.groups
>>>>>>> +++ b/test/hotspot/jtreg/TEST.groups
>>>>>>> @@ -44,7 +44,8 @@
>>>>>>>     -gc/nvdimm
>>>>>>>
>>>>>>>   hotspot_runtime = \
>>>>>>> -  runtime
>>>>>>> +  runtime \
>>>>>>> +  -runtime/containers/docker
>>>>>>>
>>>>>>>   hotspot_handshake = \
>>>>>>>     runtime/handshake
>>>>>>>
>>>>>>> Testing:
>>>>>>>
>>>>>>>    jtreg -l 
>>>>>>> /ws/hg/jdk/jdk/work01/open/test/hotspot/jtreg/:hotspot_runtime
>>>>>>>
>>>>>>>    jtreg -l 
>>>>>>> /ws/hg/jdk/jdk/work01/open/test/hotspot/jtreg/:hotspot_runtime | 
>>>>>>> grep docker
>>>>>>>
>>>>>>>
>>>>>>> Thank you,
>>>>>>>
>>>>>>> Misha
>>>>>>>
>>>>>
>>>>
>>
>