RFR: 8221221: [testbug] make test TEST="jtreg:test/hotspot/jtreg/runtime/containers" failed on Ubuntu 18.04

Sat Mar 23 01:15:53 UTC 2019

As discussed, I have implemented a change to generate the Dockerfile 
generation parameterized with base image name and version. The change is 
"8221342: [TESTBUG] Generate Dockerfile for docker testing", currently 
in review. It preserves current behavior and configuration as far as 
existing platforms are concerned as well as providing an ability to 
specify the base image+version at test execution time.

E.g.: jtreg -Djdk.test.docker.image.name=ubuntu 
-Djdk.test.docker.image.version=latest 
test/hotspot/jtreg/runtime/containers/docker/DockerBasicTest.java

Once the 8221342 is integrated, it will hopefully solve this problem.

Best regards,

Misha

On 3/22/19 10:51 AM, mikhailo.seledtsov at oracle.com wrote:
>
>
> On 3/22/19 10:44 AM, Severin Gehwolf wrote:
>> On Fri, 2019-03-22 at 10:06 -0700, mikhailo.seledtsov at oracle.com wrote:
>>>> This doesn't seem to work for me as ARG before FROM seems to require a
>>>> docker version which is likely too new (v17.06) for most users to be
>>>> prohibitive.
>>> I am using v18.03. The v17.06 is from 2017, but I can imagine that some
>>> systems are constrained using specific OS and possibly specific older
>>> versions of docker engine.
>>> If this is a common situation then we should consider alternatives, as
>>> you suggested below - generating the Dockerfile as part of 
>>> DockerTestUtils.
>>> Or, we can require use of certain reasonably up-to-date docker 
>>> version -
>>> but this may be not a feasible requirement ?
>> One data point:
>>
>> $ rpm -q docker
>> docker-1.13.1-65.git1185cfd.fc29.x86_64
>>
>> Not sure how that translates to v17.06. Perhaps this forked at some
>> point. I'd rather be conservative.
>>
>> Given that the basic Dockerfile is 4 lines in total (omitting
>> deprecated MAINTAINER) currently it would be reasonable to generate it
>> with something like this:
>>
>> String dockerFileTmpl = "FROM %s:%s\n" +
>>                          "COPY /jdk /jdk\n" +
>>                          "ENV JAVA_HOME=/jdk\n" +
>>                          "CMD [\"/bin/bash\"]\n";
>> String dockerFileStr = String.format(dockerFileTmpl, baseImage, 
>> baseImageVers);
>> Files.writeString(dockerfile, dockerFileStr);
>>
>> This would also remove some duplication we already have with
>> Dockerfile-BasicTest{,-aarch64,-ppc64le} which merely change the base
>> image as well.
>>
>> Thoughts?
> I agree, this makes sense. It will take care of older versions of 
> docker and reduce duplication, and the changes are rather minor.
>
> Thank you,
> Misha
>
>
>> Thanks,
>> Severin
>>
>>> Thank you,
>>> Misha
>>>>    But since the docker tests more or less preprocess the
>>>> Dockerfile already we might be able to work around it by generating
>>>> the
>>>> Dockerfile on the fly and get rid of Dockerfile-BasicTest-<OS-ARCH>
>>>> versions. The idea with the properties would be the same.
>>>>
>>>> Thanks,
>>>> Severin
>>>>
>>>>>    One comment:
>>>>>
>>>>> Some more files will need similar treatment:
>>>>>
>>>>> $ find test -type f -name Dockerfile-BasicTest\*
>>>>> test/hotspot/jtreg/runtime/containers/docker/Dockerfile-
>>>>> BasicTest-ppc64le
>>>>> test/hotspot/jtreg/runtime/containers/docker/Dockerfile-
>>>>> BasicTest-s390x
>>>>> test/hotspot/jtreg/runtime/containers/docker/Dockerfile-
>>>>> BasicTest-aarch64
>>>>> test/hotspot/jtreg/runtime/containers/docker/Dockerfile-BasicTest
>>>>> test/jdk/jdk/internal/platform/docker/Dockerfile-BasicTest-
>>>>> ppc64le
>>>>> test/jdk/jdk/internal/platform/docker/Dockerfile-BasicTest-s390x
>>>>> test/jdk/jdk/internal/platform/docker/Dockerfile-BasicTest-
>>>>> aarch64
>>>>> test/jdk/jdk/internal/platform/docker/Dockerfile-BasicTest
>>>>>
>>>>> Thanks,
>>>>> Severin
>>>>>
>>>>>
>>>>>> Best regards,
>>>>>> Misha
>>>>>>
>>>>>>
>>>>>> On 3/21/19 2:50 AM, Jie Fu wrote:
>>>>>>> Good suggestions!
>>>>>>>
>>>>>>> Thanks Severin.
>>>>>>>
>>>>>>>
>>>>>>> On 2019/3/21 下午4:57, Severin Gehwolf wrote:
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>> On Thu, 2019-03-21 at 16:23 +0800, Jie Fu wrote:
>>>>>>>>> Hi all,
>>>>>>>>>
>>>>>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8221221
>>>>>>>>>
>>>>>>>>> make test
>>>>>>>>> TEST="jtreg:test/hotspot/jtreg/runtime/containers" failed
>>>>>>>>> on
>>>>>>>>> Ubuntu 18.04.
>>>>>>>>>
>>>>>>>>> The issue can be fixed by
>>>>>>>>> -----------------------------------------------
>>>>>>>>> diff -r ce78fac1f6d5
>>>>>>>>> test/hotspot/jtreg/runtime/containers/docker/Dockerfile-
>>>>>>>>> BasicTest
>>>>>>>>> ---
>>>>>>>>> a/test/hotspot/jtreg/runtime/containers/docker/Dockerfile
>>>>>>>>> -BasicTest
>>>>>>>>> Thu Mar 21 01:49:27 2019 +0100
>>>>>>>>> +++
>>>>>>>>> b/test/hotspot/jtreg/runtime/containers/docker/Dockerfile
>>>>>>>>> -BasicTest
>>>>>>>>> Thu Mar 21 15:44:39 2019 +0800
>>>>>>>>> @@ -1,4 +1,4 @@
>>>>>>>>> -FROM oraclelinux:7.6
>>>>>>>>> +FROM fedora:29
>>>>>>>>>      MAINTAINER mikhailo.seledtsov at oracle.com
>>>>>>>>>
>>>>>>>>>      COPY /jdk /jdk
>>>>>>>>> -----------------------------------------------
>>>>>>>>>
>>>>>>>>> The change has been tested with
>>>>>>>>> -----------------------------------------------
>>>>>>>>> make test TEST="test/jdk/jdk/modules/scenarios/container
>>>>>>>>> test/hotspot/jtreg/vmTestbase/vm/gc/containers
>>>>>>>>> test/hotspot/jtreg/runtime/containers"
>>>>>>>>> -----------------------------------------------
>>>>>>>>> on Linux/x86-64 (Ubuntu 18.04).
>>>>>>>>>
>>>>>>>>> Could you please review it?
>>>>>>>>> Thanks a lot.
>>>>>>>>>
>>>>>>>>> Best regards,
>>>>>>>>> Jie
>>>>>>>> This fix will be inadequate as the expectation is to have
>>>>>>>> the
>>>>>>>> (reasonably?) same base image as the build platform. If you
>>>>>>>> had built
>>>>>>>> the JDK on say, centos 7, then the test would likely work.
>>>>>>>>
>>>>>>>> So if anything, I agree with David who commented on the
>>>>>>>> bug, there
>>>>>>>> should be a way to set the to-be-used base image via some
>>>>>>>> option in the
>>>>>>>> test using the current value as a default.
>>>>>>>>
>>>>>>>> Here is what AdoptOpenJDK does:
>>>>>>>>
>>>>>>>> ARG IMAGE_NAME=adoptopenjdk/openjdk8
>>>>>>>> ARG IMAGE_VERSION=latest
>>>>>>>>
>>>>>>>> FROM $IMAGE_NAME:$IMAGE_VERSION
>>>>>>>>
>>>>>>>> Example usage:
>>>>>>>> docker build --build-arg IMAGE_NAME=fedora --build-arg
>>>>>>>> IMAGE_VERSION=29 [...]
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> Severin
>>>>>>>>
>