RFR: 8292083: Detected container memory limit may exceed physical machine memory [v13]

[Severin Gehwolf]

On Tue, 23 Aug 2022 08:32:26 GMT, Jonathan Dowland <jdowland at openjdk.org> wrote:

>> We discovered some systems configured with cgroups v1 which report a bogus container memory limit value which is above the physical memory of the host. OpenJDK then calculates flags such as InitialHeapSize based on this invalid value; this can be larger than the available memory which can result in the OS terminating the process due to OOM.
>> 
>> hotspot's container awareness attempts to sanity check the limit value by ensuring it's below `_unlimited_memory = (LONG_MAX / os::vm_page_size()) * os::vm_page_size()`, but that still leaves a large range of potential invalid values between physical RAM and that ceiling value.
>> 
>> Cgroups V1 in particular returns an uninitialised value for the memory limit when one has not been explicitly set. Cgroups v2 does not suffer the same problem: however, it's possible for any value to be set for the max memory, including values exceeding the available physical memory, in either v1 or v2.
>> 
>> This fixes the problem in two places. Further work may be required in the area of Java metrics / MXBeans. I'd also look again at whether the existing ceiling value `_unlimited_memory` serves any useful purpose. I personally don't feel those improvements should hold up this fix.
>
> Jonathan Dowland has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Avoid memory_usage_in_bytes when unconstrained
>   
>   Thanks to Severin for the suggestion. When there is no memory limit
>   in place, avoid calling OSContainer::memory_usage_in_bytes, instead
>   preferring the non-container sysinfo approach.

src/hotspot/os/linux/os_linux.cpp line 199:

> 197:     jlong mem_limit, mem_usage, host_mem;
> 198:     host_mem = Linux::physical_memory();
> 199:     mem_limit = OSContainer::memory_limit_in_bytes();

With your latest changes in `cgroupV1Subsystem_linux.cpp` `mem_limit` would only exceed `host_mem` on cg2 with a memory limit specified `> host_mem` (very unlikely). Now we only do the gymnastics here to correct this situation. Instead, we could do it for both v1 and v2 in [CgroupSubsystem::memory_limit_in_bytes()](https://github.com/openjdk/jdk/blob/cf0067741249cc3260b1d220769dac408b614f22/src/hotspot/os/linux/cgroupSubsystem_linux.cpp#L525).

I'm not a big fan of this hybrid approach: correct cg1 values in `cgroupV1Subsystem_linux.cpp` and cg2 values in `os_linux`. My strong preference would be to do this in `CgroupSubsystem` code and keep `os_linux` code simpler.

-------------

PR: https://git.openjdk.org/jdk/pull/9880