RFR(T): 8222299: [TESTBUG] Docker tests should be excluded from hotspot_runtime group

[mikhailo.seledtsov at oracle.com]

Hi David,

   Thank you for your input.


On 4/10/19 6:58 PM, David Holmes wrote:
> Hi Misha,
>
> On 11/04/2019 11:44 am, Igor Ignatyev wrote:
>> Hi Misha,
>>
>> although it is not formally stated anywhere, :hotspot_runtime is 
>> expected to include _all_ jtreg-jtreg runtime related tests, that's 
>> to say I'd expect to have docker tests included into this group, 
>> regardless of a host's ability/inability to run them. hence I'd 
>> prefer us to use Skipped exception in these tests.
>
> I agree with Igor - :hotspot_runtime is supposed to include 
> everything. We exclude specific tests when we run more specific test 
> groups like tier1_runtime. (Personally I no longer use groups for 
> local testing but just directories.)
>
> If @requires is infeasible (and I can easily see it is) then using 
> "skipped" exception is one approach - however I'm also concerned about 
> wasting time failing to run these tests.
>
> Another option perhaps is to move "containers" to be a top-level test 
> group alongside runtime, rather than within it?
I agree. This is another good alternative. Would you recommend moving 
these tests to test/hotspot/jtreg/containers/..., or should they move 
under another "top-level" directory, such as 
test/hotspot/jtreg/*misc*/containers  or similar ?

Igor,  do you have any input or opinion on this alternative?


Thank you,
Misha
>
> Thanks,
> David
>
>> Thanks,
>> -- Igor
>>
>>> On Apr 10, 2019, at 6:39 PM, mikhailo.seledtsov at oracle.com wrote:
>>>
>>> Hi Igor,
>>>
>>> Thank you for taking a look.
>>>
>>>
>>> On 4/10/19 6:13 PM, Igor Ignatev wrote:
>>>> Should these test be filtered out using @requires?
>>> Checking all the conditions for this via @requires will require 
>>> building a test docker image (or at least downloading the base/FROM) 
>>> image in evaluation of at-requires, which will be unacceptably long, 
>>> especially given that @requires is evaluated each time one runs 
>>> jtreg command for any test in hotspot.
>>>
>>> Alternatively, if the current approach is undesirable, we can throw 
>>> jtreg.Skipped exception if docker base image fails to download. Let 
>>> me know if this is your preference.
>>>
>>> Misha
>>>>
>>>> — Igor
>>>>
>>>>> On Apr 10, 2019, at 6:06 PM, mikhailo.seledtsov at oracle.com wrote:
>>>>>
>>>>> Please review this small (trivial) change that excludes docker 
>>>>> tests from hotspot_runtime. The rational for this change is that 
>>>>> docker tests require specially configured environment (docker 
>>>>> engine installed, test user being member of docker group, docker 
>>>>> proxy or docker mirror repo configured). This may lead to 
>>>>> unexpected errors when docker tests are ran as part of 
>>>>> hotspot_runtime group in general environment(s).
>>>>>
>>>>>
>>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8222299
>>>>>
>>>>> Change:
>>>>>
>>>>> diff --git a/test/hotspot/jtreg/TEST.groups 
>>>>> b/test/hotspot/jtreg/TEST.groups
>>>>> --- a/test/hotspot/jtreg/TEST.groups
>>>>> +++ b/test/hotspot/jtreg/TEST.groups
>>>>> @@ -44,7 +44,8 @@
>>>>>     -gc/nvdimm
>>>>>
>>>>>   hotspot_runtime = \
>>>>> -  runtime
>>>>> +  runtime \
>>>>> +  -runtime/containers/docker
>>>>>
>>>>>   hotspot_handshake = \
>>>>>     runtime/handshake
>>>>>
>>>>> Testing:
>>>>>
>>>>>    jtreg -l 
>>>>> /ws/hg/jdk/jdk/work01/open/test/hotspot/jtreg/:hotspot_runtime
>>>>>
>>>>>    jtreg -l 
>>>>> /ws/hg/jdk/jdk/work01/open/test/hotspot/jtreg/:hotspot_runtime | 
>>>>> grep docker
>>>>>
>>>>>
>>>>> Thank you,
>>>>>
>>>>> Misha
>>>>>
>>>
>>