RFR: 8236617: jtreg test containers/docker/TestMemoryAwareness.java fails after 8226575

[Baesken, Matthias]

Hi Christoph, thanks for the review !

Bob are you fine with the latest version ?

Best regards, Matthias


> Hi Matthias,
> 
> this change looks good to me.
> 
> Best regards
> Christoph
> 
> > -----Original Message-----
> > From: hotspot-dev <hotspot-dev-bounces at openjdk.java.net> On Behalf
> Of
> > Baesken, Matthias
> > Sent: Freitag, 3. Januar 2020 11:15
> > To: Bob Vandette <bob.vandette at oracle.com>
> > Cc: hotspot-dev at openjdk.java.net
> > Subject: RE: RFR: 8236617: jtreg test
> > containers/docker/TestMemoryAwareness.java fails after 8226575
> >
> > HI Bob,
> > Looking at the docker sources,  the message seems to come  from here :
> >
> > daemon_unix.go  :
> >
> > // verifyPlatformContainerResources performs platform-specific validation
> of
> > the container's resource-configuration
> > func verifyPlatformContainerResources(resources
> > *containertypes.Resources, sysInfo *sysinfo.SysInfo, update bool)
> (warnings
> > []string, err error) {
> >   .....
> >     //
> >     // means resources have positive Memory limit,  memory+swap is not
> > unlimited  AND  SwapLimit (memory.memsw.limit_in_bytes ?) is not
> enabled
> > [comment added by me)
> > 	if resources.Memory > 0 && resources.MemorySwap != -1 &&
> > !sysInfo.SwapLimit {
> > 		warnings = append(warnings, "Your kernel does not support
> > swap limit capabilities or the cgroup is not mounted. Memory limited
> without
> > swap.")
> > 		resources.MemorySwap = -1
> > 	}
> >
> > with Resources  from hostconfig.go  :
> >
> > // Resources contains container's resources (cgroups config, ulimits...)
> > type Resources struct {
> > 	Memory    int64 // Memory limit (in bytes)
> >  ...
> > 	MemorySwap           int64           // Total memory usage (memory +
> > swap); set `-1` to enable unlimited swap
> >
> >
> >
> > So I think  your suggestion  to return  0  in that special case in function
> > getTotalSwapSpaceSize   sounds reasonable to me ( at least better than
> > return a large negative value ).
> > New webrev :
> >
> >
> > http://cr.openjdk.java.net/~mbaesken/webrevs/8236617.2/
> >
> >
> >
> > Thanks, Matthias
> >
> >
> >
> >
> > >
> > > Matthias,
> > >
> > > I really don’t like testing for some Docker message that could possibly
> > change
> > > or go away in the future.
> > > There may be other reasons that the getTotalSwapSpaceSize function will
> > fail
> > > and return 0.
> > >
> > > The real problem here is that the
> > > OperatingSystemMXBean.getTotalSwapSpaceSize is returning a
> > > negative value when there is no swap available.
> > >
> > > I’d prefer that we fix this problem by correcting the
> getTotalSwapSpaceSize
> > > function to properly return 0
> > > under these conditions and allow 0 to be a valid expected result in the
> test.
> > >
> > >  if (limit >= 0 && memLimit >= 0) {
> > >    return (limit < memLimit) ? 0 : limit - memLimit;
> > >  }
> > >
> > > Note:  My suggestion assumes that there is no swap available when the
> > > kernel swap limit capability is not enabled.
> > >            I have not verified this.  The message does claim that this is the case
> > > "Memory limited without swap”.
> > >
> > > Bob.
> > >
> > >
> > > > On Jan 2, 2020, at 8:26 AM, Baesken, Matthias
> > > <matthias.baesken at sap.com> wrote:
> > > >
> > > > Hello, please review this small adjustment to  jtreg test
> > > containers/docker/TestMemoryAwareness.java .
> > > >
> > > > After change   "8226575: OperatingSystemMXBean should be made
> > > container aware"    has been pushed,
> > > > we observe failures on linux s390x / ppc64le in the docker related jtreg
> > > tests  .
> > > >
> > > >
> > > > The test runs into  the following error :
> > > > java.lang.RuntimeException:
> > > 'OperatingSystemMXBean.getTotalSwapSpaceSize: 52428800' missing
> from
> > > stdout/stderr
> > > >
> > > > at
> > >
> >
> jdk.test.lib.process.OutputAnalyzer.shouldContain(OutputAnalyzer.java:187)
> > > > at
> > >
> >
> TestMemoryAwareness.testOperatingSystemMXBeanAwareness(TestMem
> > > oryAwareness.java:154)
> > > > at TestMemoryAwareness.main(TestMemoryAwareness.java:65)
> > > > at
> > >
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
> > > Method)
> > > > at
> > >
> >
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMet
> > > hodAccessorImpl.java:62)
> > > > at
> > >
> >
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Delega
> > > tingMethodAccessorImpl.java:43)
> > > > at java.base/java.lang.reflect.Method.invoke(Method.java:564)
> > > > at
> > >
> >
> com.sun.javatest.regtest.agent.MainWrapper$MainThread.run(MainWrapp
> > > er.java:127)
> > > > at java.base/java.lang.Thread.run(Thread.java:832)
> > > >
> > > >
> > > > The reason is that the value found is instead
> > > OperatingSystemMXBean.getTotalSwapSpaceSize: -104857600    .
> > > > When looking into the getTotalSwapSpaceSize() function, we get values
> > of
> > > 0 for "limit" and 104857600 for "memLimit" :
> > > >
> > > > 57 long limit = containerMetrics.getMemoryAndSwapLimit();
> > > > ....
> > > > 62 long memLimit = containerMetrics.getMemoryLimit();
> > > > 63 if (limit >= 0 && memLimit >= 0) {
> > > > 64 return limit - memLimit;
> > > > 65 }
> > > >
> > > > That explains the value "-104857600" . We see messages    "Your kernel
> > > does not support swap limit capabilities or the cgroup is not mounted.
> > > Memory limited without swap" , this most likely
> > > > causes the unexpected limit == 0  value  .
> > > >
> > > >
> > > > Bug/webrev :
> > > >
> > > > https://bugs.openjdk.java.net/browse/JDK-8236617
> > > >
> > > > http://cr.openjdk.java.net/~mbaesken/webrevs/8236617.0/
> > > >
> > > >
> > > > Thanks, Matthias