RFR(XS): 8194232: Container memory not properly recognized.
Lindenmaier, Goetz
goetz.lindenmaier at sap.com
Thu Jan 4 10:59:40 UTC 2018
THanks Martin!
Best regards,
Goetz.
> -----Original Message-----
> From: Doerr, Martin
> Sent: Donnerstag, 4. Januar 2018 11:54
> To: Lindenmaier, Goetz <goetz.lindenmaier at sap.com>; Bob Vandette
> <bob.vandette at oracle.com>; hotspot-runtime-dev at openjdk.java.net
> Subject: RE: RFR(XS): 8194232: Container memory not properly recognized.
>
> Hi Götz,
>
> thanks for the updated webrev. Looks good.
>
> Best regards,
> Martin
>
>
> -----Original Message-----
> From: Lindenmaier, Goetz
> Sent: Donnerstag, 4. Januar 2018 10:10
> To: Bob Vandette <bob.vandette at oracle.com>; Doerr, Martin
> <martin.doerr at sap.com>; hotspot-runtime-dev at openjdk.java.net
> Subject: RE: RFR(XS): 8194232: Container memory not properly recognized.
>
> Hi Bob,
>
> A new webrev using your version:
> http://cr.openjdk.java.net/~goetz/wr17/8194232-
> ppcle_unlimited/webrev.02/
>
> It fixes the issue I saw, and traces the output you describe below.
> Actually, in my case gc/arguments/TestAgressiveHeap.java failed
> and this also passes now. The machine has cgroups but no docker
> which is why the test failed.
> The five tests below don't run on ppc because docker.support is
> false.
>
> Best regards,
> Goetz
>
>
>
>
> > -----Original Message-----
> > From: Bob Vandette [mailto:bob.vandette at oracle.com]
> > Sent: Mittwoch, 3. Januar 2018 20:12
> > To: Lindenmaier, Goetz <goetz.lindenmaier at sap.com>; Doerr, Martin
> > <martin.doerr at sap.com>; hotspot-runtime-dev at openjdk.java.net
> > Subject: Re: RFR(XS): 8194232: Container memory not properly recognized.
> >
> > I built and ran a VM with the changes below (with jong -> jlong type fixed)
> > and it passed
> > the jtreg container tests.
> >
> > Passed: runtime/containers/docker/DockerBasicTest.java
> > Passed: runtime/containers/docker/TestCPUAwareness.java
> > Passed: runtime/containers/docker/TestCPUSets.java
> > Passed: runtime/containers/docker/TestMemoryAwareness.java
> > Passed: runtime/containers/docker/TestMisc.java
> >
> > If you run this command on your OS and look in the logs, you should see
> the
> > Unlimited message:
> >
> > ./java -Xlog:os+containers=trace -version
> >
> > …..
> > [0.001s][trace][os,container] Memory Limit is: 9223372036854771712 <<—
> —
> > This number will be different for you.
> > [0.001s][trace][os,container] Memory Limit is: Unlimited
> > [0.001s][debug][os,container] container memory unlimited, using host
> value
> > …..
> >
> > Bob.
> >
> >
> > > On Jan 3, 2018, at 1:54 PM, Bob Vandette <bob.vandette at oracle.com>
> > wrote:
> > >
> > >
> > >> On Jan 3, 2018, at 2:43 AM, Lindenmaier, Goetz
> > <goetz.lindenmaier at sap.com> wrote:
> > >>
> > >> Hi Bob,
> > >>
> > >> The value is read from the file system as jlong.
> > >> I think making the reading of the value as jlong or julong
> > >> depend on the kernel version is not a good idea.
> > >
> > > I agree that we shouldn't make the solution based on the running kernel
> > version.
> > >
> > >>
> > >> So should I test for
> > >> memlimit < 0 || memlimit > LONG_MAX / os::vm_page_size() *
> > os::vm_page_size()
> > >> and get rid of the constant altogether?
> > >
> > > I was thinking that we could change the type read from the file system to
> a
> > julong and then just
> > > use memlimit > LONG_MAX / os::vm_page_size() * os::vm_page_size().
> > You’ll need to
> > > add a (jlong) cast but since we’re checking for > LONG_MAX, there will be
> > no precision
> > > lost.
> > >
> > > Something like this:
> > >
> > > --- a/src/hotspot/os/linux/osContainer_linux.cpp
> > > +++ b/src/hotspot/os/linux/osContainer_linux.cpp
> > > @@ -31,16 +31,12 @@
> > > #include "logging/log.hpp"
> > > #include "osContainer_linux.hpp"
> > >
> > > -/*
> > > - * Warning: Some linux distros use 0x7FFFFFFFFFFFF000
> > > - * and others use 0x7FFFFFFFFFFFFFFF for unlimited.
> > > - */
> > > -#define UNLIMITED_MEM CONST64(0x7FFFFFFFFFFFF000)
> > >
> > > #define PER_CPU_SHARES 1024
> > >
> > > bool OSContainer::_is_initialized = false;
> > > bool OSContainer::_is_containerized = false;
> > > +julong _unlimited_memory;
> > >
> > > class CgroupSubsystem: CHeapObj<mtInternal> {
> > > friend class OSContainer;
> > > @@ -217,6 +213,8 @@
> > > _is_initialized = true;
> > > _is_containerized = false;
> > >
> > > + _unlimited_memory = (LONG_MAX / os::vm_page_size()) *
> > os::vm_page_size();
> > > +
> > > log_trace(os, container)("OSContainer::init: Initializing Container
> > Support");
> > > if (!UseContainerSupport) {
> > > log_trace(os, container)("Container Support not enabled");
> > > @@ -419,37 +417,37 @@
> > > * OSCONTAINER_ERROR for not supported
> > > */
> > > jlong OSContainer::memory_limit_in_bytes() {
> > > - GET_CONTAINER_INFO(jlong, memory, "/memory.limit_in_bytes",
> > > - "Memory Limit is: " JLONG_FORMAT, JLONG_FORMAT,
> > memlimit);
> > > + GET_CONTAINER_INFO(julong, memory, "/memory.limit_in_bytes",
> > > + "Memory Limit is: " JULONG_FORMAT, JULONG_FORMAT,
> > memlimit);
> > >
> > > - if (memlimit >= UNLIMITED_MEM) {
> > > + if (memlimit >= _unlimited_memory) {
> > > log_trace(os, container)("Memory Limit is: Unlimited");
> > > return (jlong)-1;
> > > }
> > > else {
> > > - return memlimit;
> > > + return (jong)memlimit;
> > > }
> > > }
> > >
> > > jlong OSContainer::memory_and_swap_limit_in_bytes() {
> > > - GET_CONTAINER_INFO(jlong, memory,
> > "/memory.memsw.limit_in_bytes",
> > > - "Memory and Swap Limit is: " JLONG_FORMAT,
> > JLONG_FORMAT, memswlimit);
> > > - if (memswlimit >= UNLIMITED_MEM) {
> > > + GET_CONTAINER_INFO(julong, memory,
> > "/memory.memsw.limit_in_bytes",
> > > + "Memory and Swap Limit is: " JULONG_FORMAT,
> > JULONG_FORMAT, memswlimit);
> > > + if (memswlimit >= _unlimited_memory) {
> > > log_trace(os, container)("Memory and Swap Limit is: Unlimited");
> > > return (jlong)-1;
> > > } else {
> > > - return memswlimit;
> > > + return (jlong)memswlimit;
> > > }
> > > }
> > >
> > > jlong OSContainer::memory_soft_limit_in_bytes() {
> > > - GET_CONTAINER_INFO(jlong, memory,
> "/memory.soft_limit_in_bytes",
> > > - "Memory Soft Limit is: " JLONG_FORMAT, JLONG_FORMAT,
> > memsoftlimit);
> > > - if (memsoftlimit >= UNLIMITED_MEM) {
> > > + GET_CONTAINER_INFO(julong, memory,
> > "/memory.soft_limit_in_bytes",
> > > + "Memory Soft Limit is: " JULONG_FORMAT,
> JULONG_FORMAT,
> > memsoftlimit);
> > > + if (memsoftlimit >= _unlimited_memory) {
> > > log_trace(os, container)("Memory Soft Limit is: Unlimited");
> > > return (jlong)-1;
> > > } else {
> > > - return memsoftlimit;
> > > + return (jlong)memsoftlimit;
> > > }
> > > }
> > > Bob.
> > >
> > >> Instead, I would implement a method is_unlimited_memory(memlimit).
> > >>
> > >> I don't want to make the change too big, as I need to get it fixed in 10.
> > >>
> > >> Best regards,
> > >> Goetz.
> > >>
> > >>> -----Original Message-----
> > >>> From: Bob Vandette [mailto:bob.vandette at oracle.com]
> > >>> Sent: Dienstag, 2. Januar 2018 17:10
> > >>> To: Lindenmaier, Goetz <goetz.lindenmaier at sap.com>
> > >>> Cc: Doerr, Martin <martin.doerr at sap.com>; hotspot-runtime-
> > >>> dev at openjdk.java.net
> > >>> Subject: Re: RFR(XS): 8194232: Container memory not properly
> > recognized.
> > >>>
> > >>> I just read that unlimited may be one of these possible values
> depending
> > on
> > >>> Kernel version.
> > >>>
> > >>> LONG_MAX (Linux Kernel Version < 3.1.2)
> > >>> ULONG_MAX (3.12 <= Linux Kernel Version < 3.19)
> > >>> LONG_MAX / pagesize * pagesize (Linux Kernel Version >= 3.19)
> > >>>
> > >>> Assuming os::page_size() is initialized before the container init, we
> > should
> > >>> calculate
> > >>> the unlimited value based on this and take the ULONG_MAX situation
> > into
> > >>> consideration.
> > >>>
> > >>> Bob.
> > >>>
> > >>>
> > >>>> On Dec 27, 2017, at 6:24 AM, Lindenmaier, Goetz
> > >>> <goetz.lindenmaier at sap.com> wrote:
> > >>>>
> > >>>> Hi Martin,
> > >>>>
> > >>>> Yes, I also figured that the constant is one system page less than max
> > long.
> > >>>> If the system pages are > 64K, this constant will be too small, again.
> > >>>> But I'm not sure whether adding 8 zeros is right thing to do in ramp
> > down
> > >>> phase.
> > >>>>
> > >>>> Best regards,
> > >>>> Goetz.
> > >>>>
> > >>>>
> > >>>>> -----Original Message-----
> > >>>>> From: Doerr, Martin
> > >>>>> Sent: Mittwoch, 27. Dezember 2017 12:10
> > >>>>> To: Lindenmaier, Goetz <goetz.lindenmaier at sap.com>; hotspot-
> > runtime-
> > >>>>> dev at openjdk.java.net
> > >>>>> Subject: RE: RFR(XS): 8194232: Container memory not properly
> > >>> recognized.
> > >>>>>
> > >>>>> Hi Götz,
> > >>>>>
> > >>>>> thanks for fixing it.
> > >>>>>
> > >>>>> I wonder if 4 zeroes will be sufficient on all linux distros in the long
> > run.
> > >>> Does
> > >>>>> anything speak against using e.g. 8 zeroes?
> > >>>>>
> > >>>>> Best regards,
> > >>>>> Martin
> > >>>>>
> > >>>>>
> > >>>>> -----Original Message-----
> > >>>>> From: hotspot-runtime-dev [mailto:hotspot-runtime-dev-
> > >>>>> bounces at openjdk.java.net] On Behalf Of Lindenmaier, Goetz
> > >>>>> Sent: Mittwoch, 27. Dezember 2017 11:38
> > >>>>> To: hotspot-runtime-dev at openjdk.java.net
> > >>>>> Subject: RFR(XS): 8194232: Container memory not properly
> > recognized.
> > >>>>>
> > >>>>> Hi
> > >>>>>
> > >>>>> Please review and sponsor this tiny fix. It needs to go to jdk10.
> > >>>>> http://cr.openjdk.java.net/~goetz/wr17/8194232-
> > >>>>> ppcle_unlimited/webrev.01/
> > >>>>>
> > >>>>> TestAggressiveHeap.java fails because the container recognition
> > >>>>> misinterprets the available memory size. On SLES 12.1 ppc64le,
> > >>>>> GET_CONTAINER_INFO() sets memlimit to 0x7FFFFFFFFFFF0000.
> This
> > >>>>> is compared to UNLIMITED_MEM == 0x7FFFFFFFFFFFF000, making
> > >>>>> the VM believe memory is _not_ unlimited.
> > >>>>>
> > >>>>> Best regards,
> > >>>>> Goetz.
> > >>>>
> > >>
> > >
More information about the hotspot-runtime-dev
mailing list