[RFC 8285277] - How should the JVM handle container memory limits

[Severin Gehwolf]

On Wed, 2022-04-20 at 11:09 -0700, Ioi Lam wrote:
> I would like to have a discussion on how (or whether) the JVM should 
> handle container memory limits -- see JDK-8285277 [1]

It does handle container memory limits. os::physical_memory() returns
the container limit instead of the host value if a container limit is
in place. That was one big aspect of JDK-8146115 when it was done in
JDK 10 timeframe. See:

https://github.com/openjdk/jdk/blob/fa04d1f832ff201248f935939fa255988053a1d0/src/hotspot/os/linux/os_linux.cpp
#L217..L232

What am I missing?

> The JVM may be terminated by the Linux OOM killer if it tries to use 
> more memory than the memory limit specified for a container. JDK-8284900 
> [2] tries to avoid OOM by checking InitialHeapSize against the memory
> limit. However, this is incomplete because the JVM can use memory in 
> other ways:
> 
> - If -Xmx is larger than -Xms, the heap may expand
> - malloc memory
> - code cache
> - thread stacks

JDK-8284900 is somewhat a special case which, IIUIC, tries to avoid
user-errors detectable at JVM startup. For example setting -Xms to a
value >= physical memory (including swap). In this case the JVM would
be at risk to get killed by the OOM killer and doing this check on
start-up would catch it before it comes to it.

On the other hand, I don't see any handling of such a case on a
*physical* machine. Consider a machine with 8GB of RAM and somebody
spawning a JVM with -Xms8g on it. It'll happily oblige? In one of my
VMs with 8GB I see this:

$ ./jdk19-jdk/bin/java -Xms8g -XX:+PrintFlagsFinal -version 2>&1 | grep InitialHeapSize
   size_t InitialHeapSize                          = 8589934592                                {product} {command line}
$ free --giga
               total        used        free      shared  buff/cache   available
Mem:               8           0           7           0           0           7
Swap:              8           0           8

> We have several choices:
> 
> (a) Following the direction of JDK-8284900, check for the initial memory 
> usage  of other types of memory as well. The problem with this is that 
> checking the "initial" usage some type of memory usage (malloc or stack) 
> is difficult or impossible.
> 
> (b) Avoid committing more memory when the total memory usage is close to 
> the limit. E.g., avoid expanding the Java heap, or return NULL from 
> os::malloc(). The problem is that some operations in the VM cannot 
> handle malloc failure and will terminate the VM anyway. Also, trying to 
> obey the memory limit with concurrent allocator/deallocator threads is 
> probably difficult or impossible.
> 
> (c) Do not consider the memory limit and let the VM be killed by the OOM 
> killer. This may be what the user wants -- the user knows that the app 
> will normally run within the memory limit, so if the heap expands too
> much the app probably has a memory leak and should be killed, to be 
> automatically restarted by an external mechanism (such as Kubernetes).

Neither of the above?

It would perhaps make sense to do some sanitiy checks on JVM start-up
(somewhat akin to JDK-8284900). But this should probably be done with
the main operating system API. Not using OSContainer API as proposed in
JDK-8284900. For example, print a warning if MaxHeapSize,
InitialHeapSize and MinHeapSize are larger than the reported physical
memory (and/or cap it at a lower size if so).

Isn't (b) being handled by setting os::pysical_memory() and
os::available_memory() to the container limits (which we currently do)?
It would behave similarly to a physical machine running out of memory,
no?

Thanks,
Severin


> [1] https://bugs.openjdk.java.net/browse/JDK-8285277
> [2] https://bugs.openjdk.java.net/browse/JDK-8284900
>