OperatingSystemMXBean unaware of container memory limits

[Bob Vandette]

> On Jun 21, 2019, at 4:22 AM, Severin Gehwolf <sgehwolf at redhat.com> wrote:
> 
> Hi Bob,
> 
> On Thu, 2019-06-20 at 10:16 -0400, Bob Vandette wrote:
>> Hi Andrew,
>> 
>> I am aware of the limitations of the OperatingSystemMXBean and was
>> hoping to address these limitations during the implementation of
>> https://bugs.openjdk.java.net/browse/JDK-8199944.
>> 
>> It would be helpful if you feel this is important to add some votes
>> to this issue.
> 
> It seems strange that the getAvailableProcessors() returns the
> container limit, while the memory limits are for the physical host. If
> anything, shouldn't they agree (both physical host or both container
> limits)?
> 
> When I briefly looked into it initially it seems to be a side-effect of
> what is being used by the JDK code implementation-wise. IIRC
> getAvailableProcessors() uses a runtime call. Memory reporting has it's
> own logic[1], thus not reporting the container limit.
> 
> This seems weird from a consistency perspective. Thoughts?
> 
> If I understand you correctly, you are arguing that container reporting
> should go into its own MX bean. On the other hand, CPU reporting for
> the OS MX bean is container aware already. That makes me believe we
> should rather make this consistent before evaluating a new MX bean.

You make a good point.  I’ll split the enhancement and add a bug to fix the
current MX Bean since this is pretty easy to do.

Bob.

> 
> Thanks,
> Severin
> 
> [1] http://hg.openjdk.java.net/jdk/jdk/file/1c242c2d037f/src/jdk.management/unix/native/libmanagement_ext/OperatingSystemImpl.c#l365
> 
> 
>> Bob.
>> 
>> 
>>> On Jun 20, 2019, at 9:43 AM, Andrew Azores <aazores at redhat.com>
>>> wrote:
>>> 
>>> Hi all,
>>> 
>>> Apologies if this is not the most appropriate list, in which case
>>> please direct me where to go.
>>> 
>>> I've noticed a surprising result from the
>>> com.sun.management.OperatingSystemMXBean implementation when
>>> running in
>>> a containerized (specifically, using Docker on Linux) environment.
>>> The
>>> bean appears to be container-aware for processors, in that running
>>> with
>>> Docker option `--cpus 1.0` for example, on a multicore system, will
>>> cause both java.lang.Runtime#availableProcessors and
>>> java.lang.management.OperatingSystemMXBean#getAvailableProcessors /
>>> com.sun.management.OperatingSystemMXBean#getAvailableProcessors to
>>> return 1. However, the Docker option `--memory 100M` (or any other
>>> limit value) is not reflected in the value returned by
>>> com.sun.management.OperatingSystemMXBeam#getTotalPhysicalMemorySize
>>> ,
>>> and instead the returned value is still the total physical memory
>>> of
>>> the host machine - of which only a small portion may actually be
>>> available to the "Operating System" of the JVM. Similarly for the
>>> methods regarding free physical memory, total swap, and free swap.
>>> 
>>> I have attached a patch which adds a small reproducer to the
>>> existing
>>> MemoryAwareness test.
>>> 
>>> This seems like a bug to me, since if the imposed container limit
>>> on
>>> processors as a resource is included as part of the "Operating
>>> System"
>>> resource reporting, then surely memory resources should be reported
>>> the
>>> same way. As I said, I found the current behaviour quite
>>> surprising.
>>> 
>>> -- 
>>> Andrew Azores
>>> Software Engineer, OpenJDK Team
>>> Red Hat
>>> <jdk-osmxbean-container-memory-test-01.patch>
>> 
>> 
>