RFR: 8227642: [TESTBUG] Make docker tests podman compatible

Tue Jul 16 21:01:00 UTC 2019

Hi Igor,

I understand the concern and I guess I could remove it and locally
install a wrapper in /bin or /usr/bin for podman which adds /usr/sbin
to the path. On the other hand...

This seems to be an issue of code being run through jtreg. Looking at
the jtr files I see this:

----------rerun:(21/1545)*----------
cd /disk/openjdk/upstream-sources/openjdk-head/JTwork/scratch && \\
DISPLAY=:0 \\
HOME=/home/sgehwolf \\
LANG=en_US.UTF-8 \\
PATH=/bin:/usr/bin \\
XMODIFIERS=@im=ibus \\
[...]

So jtreg reduces the host's PATH to /bin:/usr/bin, which is
insufficient for the podman case. The tag-spec docs[1] for jtreg
mention for "shell" tests that it sets the PATH to the above settings.
This affects Java tests as well it seems.

ProcessBuilder outside jtreg has a sensible PATH as set up by the
system, FWIW.

So while my system is properly set up, jtreg interferes and renders
this necessary. Any suggestions as to how to convince jtreg to use the
host's PATH setting?

Thanks,
Severin

[1] http://openjdk.java.net/jtreg/tag-spec.html

On Tue, 2019-07-16 at 13:32 -0700, Igor Ignatyev wrote:
> Hi Misha,
> 
> I understand that it doesn't alter the host system. my concern is
> that we move problem of host-configuration into tests. let's say
> tomorrow a new container engine will require something from
> /usr/local/sbin, or /usr/local/Cellar/docker/bin on another OS, or,
> god forbid, C:\Program Files(x86)\podman\bin. it has nothing to do w/
> tests, it's a question of configuring a host, as I said, should be
> handled at another level by "scripts" run (once) prior test
> execution.
> 
> -- Igor
> 
> > On Jul 16, 2019, at 1:23 PM, mikhailo.seledtsov at oracle.com wrote:
> > 
> > Hi Igor,
> > 
> >    In both cases the environment variable is set for the
> > Docker/Podman container process, not the host system. This will not
> > affect the host system in any way. The docker process has its own
> > namespace for environment variables. Does this alleviate your
> > concerns?
> > 
> > 
> > Thank you,
> > 
> > Misha
> > 
> > On 7/16/19 11:49 AM, Igor Ignatyev wrote:
> > > Hi Severin,
> > > 
> > > I don't think that tests (or test libraries for that matter)
> > > should be responsible for setting correct PATH value, it should
> > > be a part of host configuration procedure (tests can/should check
> > > that all required bins are available though). in other words, I'd
> > > prefer if you remove 'env.put("PATH", ...)' lines from both
> > > DockerTestUtils and TestJFREvents. the rest looks good to me.
> > > 
> > > Thanks,
> > > -- Igor
> > > 
> > > > On Jul 16, 2019, at 5:36 AM, Severin Gehwolf <
> > > > sgehwolf at redhat.com> wrote:
> > > > 
> > > > Hi,
> > > > 
> > > > I believe I still need a *R*eviewer for this. Any takers?
> > > > 
> > > > Thanks,
> > > > Severin
> > > > 
> > > > On Fri, 2019-07-12 at 15:19 -0700, 
> > > > mikhailo.seledtsov at oracle.com wrote:
> > > > > Hi Severin,
> > > > > 
> > > > >   The change looks good to me. Thank you for adding support
> > > > > for Podman
> > > > > container technology.
> > > > > 
> > > > > Testing: I ran both HotSpot and JDK container tests with your
> > > > > patch;
> > > > > tests executed on Oracle Linux 7.6 using default container
> > > > > engine (Docker):
> > > > > 
> > > > >     test/hotspot/jtreg/containers/   AND
> > > > > test/jdk/jdk/internal/platform/docker/
> > > > > 
> > > > > All PASS
> > > > > 
> > > > > 
> > > > > Thanks,
> > > > > 
> > > > > Misha
> > > > > 
> > > > > 
> > > > > On 7/12/19 11:08 AM, Severin Gehwolf wrote:
> > > > > > Hi,
> > > > > > 
> > > > > > There is an alternative container engine which is being
> > > > > > used by Fedora
> > > > > > and RHEL 8, called podman[1]. It's mostly compatible with
> > > > > > docker. It
> > > > > > looks like OpenJDK docker tests can be made podman
> > > > > > compatible with a
> > > > > > few little tweaks. One "interesting" one is to not assert
> > > > > > "Successfully
> > > > > > built" in the build output but only rely on the exit code,
> > > > > > which seems
> > > > > > to be OK for my testing. Interestingly the test would be
> > > > > > skipped in
> > > > > > that case.
> > > > > > 
> > > > > > Bug: https://bugs.openjdk.java.net/browse/JDK-8227642
> > > > > > webrev: 
> > > > > > http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8227642/01/webrev/
> > > > > > 
> > > > > > Adjustments I've done:
> > > > > >  * Don't assert "Successfully built" in image build
> > > > > > output[2].
> > > > > >  * Add /usr/sbin to PATH as the podman binary relies on
> > > > > > iptables for it
> > > > > >    to work which is in /usr/sbin on Fedora
> > > > > >  * Allow for Metrics.getCpuSystemUsage() and
> > > > > > Metrics.getCpuUserUsage()
> > > > > >    to be equal to the previous value. I've found those
> > > > > > counters to be
> > > > > >    slowly increasing, which made the tests unreliable.
> > > > > > 
> > > > > > Testing:
> > > > > > 
> > > > > > Running docker tests with docker as engine. Did the same
> > > > > > with podman as
> > > > > > engine via -Djdk.test.docker.command=podman on Linux
> > > > > > x86_64. Both
> > > > > > passed (non-trivially).
> > > > > > 
> > > > > > Thoughts?
> > > > > > 
> > > > > > Thanks,
> > > > > > Severin
> > > > > > 
> > > > > > [1] https://podman.io/
> > > > > > [2] Image builds with podman look
> > > > > >     like ("COMMIT" over "Successfully built"):
> > > > > > STEP 1: FROM fedora:29
> > > > > > STEP 2: RUN dnf install -y java-11-openjdk-devel &&     dnf
> > > > > > clean all
> > > > > > --> Using cache
> > > > > > 96f8b1a0dfe7dba581a64fc67a27002ddf52e032af55f9ddc765182a690
> > > > > > afd9d
> > > > > > STEP 3: COPY TestMetrics.class  TestMetrics.java /opt/
> > > > > > 269042160f7a4e6a06789cd19640ea658a8f941bc53de0fd40a574dc3bd
> > > > > > b49a8
> > > > > > STEP 4: CMD /usr/lib/jvm/java-11-openjdk/bin/java -cp /opt
> > > > > > --add-modules java.base --add-exports
> > > > > > java.base/jdk.internal.platform=ALL-UNNAMED TestMetrics
> > > > > > STEP 5: COMMIT fedora-metrics-11
> > > > > > d749088d6ce4510f212820ad4eca55a9b05e5c5c245f2372b6cfe91926e
> > > > > > 8cd7e
> > > > > > 