[RFC 8285277] - How should the JVM handle container memory limits
Ioi Lam
ioi.lam at oracle.com
Wed Apr 27 20:24:49 UTC 2022
On 4/27/2022 6:01 AM, Severin Gehwolf wrote:
> On Tue, 2022-04-26 at 11:12 -0700, Ioi Lam wrote:
>> On 4/21/2022 2:49 AM, Severin Gehwolf wrote:
>>> On Wed, 2022-04-20 at 11:09 -0700, Ioi Lam wrote:
>>>> I would like to have a discussion on how (or whether) the JVM should
>>>> handle container memory limits -- see JDK-8285277 [1]
>>> It does handle container memory limits. os::physical_memory() returns
>>> the container limit instead of the host value if a container limit is
>>> in place. That was one big aspect of JDK-8146115 when it was done in
>>> JDK 10 timeframe. See:
>>>
>>> https://github.com/openjdk/jdk/blob/fa04d1f832ff201248f935939fa255988053a1d0/src/hotspot/os/linux/os_linux.cpp#L217..L232
>>>
>>> What am I missing?
>> Yes, the container support code does expose the memory limit through
>> APIs like os::physical_memory(), and we use the same ergonomic logic as
>> if we are running on a physical machine with a small amount of RAM.
>>
>> However, I am wondering if we need to reconsider our logic if the
>> "physical memory" is unusually small.
> Not just for small containers. Some tunings are needed for larger
> containers too (considering that it's usually 1 process per container).
>
>> Currently, the ergonomic code will set the heap size to a "sensible"
>> fraction of os::physical_memory(). This usually works fine on modern day
>> machines with lots of RAM and swap. Even the below-$100 windows laptops
>> have at least 4GB.
> For some definition of "sensible". For a physical machine a value of
> MaxRAMPercentage=25 makes sense. For a container where the typical use
> case is some cloud orchestration framework spawning them and in 99% of
> the cases the container runs a single process with some memory limit
> imposed a value of MaxRAMPercentage=25 doesn't make as much sense. Say
> a container limit is 500MB, then your heap limit (MaxHeapSize) without
> any further tuning elsewhere will be 500/4 (126MB), which in that case
> most of the time won't be what the user wants. Note that this means
> that the JVM will throw OutOfMemory errors as soon as the MaxHeapSize
> is being reached, even though there probably is unused container memory
> still available (most likely 3*500/4 - N, where N the amount of native
> JVM memory).
>
>> On my 64GB box, max heap is set to 16GB by default. This should be
>> enough for most apps, and will practically never be killed by OOM unless
>> the system is really overloaded.
>>
>> ================
>> $ java -XX:+PrintFlagsFinal --version | egrep
>> '((MaxHeapSize)|(InitialHeapSize))'
>> size_t InitialHeapSize =
>> 1056964608 {product} {ergonomic}
>> size_t MaxHeapSize =
>> 16869490688 {product} {ergonomic}
>> ================
>>
>> But inside a container that has only a 128MB limit (which might be
>> "typical" for micro-services??)
>>
>> ================
>> $ java -XX:+PrintFlagsFinal --version | egrep
>> '((MaxHeapSize)|(InitialHeapSize))'
>> size_t InitialHeapSize =
>> 8388608 {product} {ergonomic}
>> size_t MaxHeapSize =
>> 67108864 {product} {ergonomic}
>> ================
>>
>> The default MaxHeapSize seems wasteful. We are paying for 128MB but
>> really just use 64MB of heap. I am able to start with a 108MB heap size
>> without getting killed.
> Yes. While I got confused a little what you meant by "wasteful" it
> sounds like you are saying it should set MaxHeapSize to a higher level.
> Is that correct? If so that's basically what I was trying to say with
> the (wrong) default of MaxRAMPercentage when run in a container. My
> experimentations with this didn't go anywhere, though, as we cannot
> reliably detect if we are on a plain Linux system or in a container.
> See JDK-8261242.
>
>> ======================
>> $ java -XshowSettings:system -Xms108m -XX:+AlwaysPreTouch --version
>> Operating System Metrics:
>> Provider: cgroupv1
>> ....
>> Memory Limit: 128.00M
>> Memory Soft Limit: Unlimited
>> Memory & Swap Limit: 128.00M
>>
>> java 19-internal 2022-09-20
>> Java(TM) SE Runtime Environment (build 19-internal-adhoc.iklam)
>> Java HotSpot(TM) 64-Bit Server VM (build 19-internal-adhoc.iklam, mixed
>> mode, sharing)
>> ==========================
>>
>> The problem is
>>
>> - If we want to efficiently use all the memory we paid for in a
>> container, we need to tune the VM parameters manually
>>
>> - But tuning is very difficult and error prone. With the 100mb heap
>> setting, my app could easily be killed if it uses too much memory in
>> malloc, native buffers, or even the JIT. Currently we don't fail early
>> (no check of -Xms, -Xmx, etc, against available memory), and don't fail
>> predictably.
> +1 to doing some more sanity checks on JVM startup when some of those
> settings exceed physical memory.
I think this makes sense. We need to decide what the behavior should be:
- what flags to check
- print warnings?
- abort the VM?
- override the settings to conform to available memory?
- always do this, or only when running inside a container? If the
latter, we will have the problem of JDK-8261242
Also, we will need a CSR if we decide to abort the VM or override the
user-specified settings.
Thanks
- Ioi
>> So, I am wondering if there's anything we can do to make Java both safe
>> and efficient when running under an small memory budget.
> I sure hope that we can improve this. One low hanging fruit would be to
> adjust MaxRAMPercentage (in containers). The question is how to do this
> without causing issues for JVMs running on physical machines on Linux.
>
> Thanks,
> Severin
>
>> Thanks
>> - Ioi
>>
>>
>>>> The JVM may be terminated by the Linux OOM killer if it tries to use
>>>> more memory than the memory limit specified for a container. JDK-8284900
>>>> [2] tries to avoid OOM by checking InitialHeapSize against the memory
>>>> limit. However, this is incomplete because the JVM can use memory in
>>>> other ways:
>>>>
>>>> - If -Xmx is larger than -Xms, the heap may expand
>>>> - malloc memory
>>>> - code cache
>>>> - thread stacks
>>> JDK-8284900 is somewhat a special case which, IIUIC, tries to avoid
>>> user-errors detectable at JVM startup. For example setting -Xms to a
>>> value >= physical memory (including swap). In this case the JVM would
>>> be at risk to get killed by the OOM killer and doing this check on
>>> start-up would catch it before it comes to it.
>>>
>>> On the other hand, I don't see any handling of such a case on a
>>> *physical* machine. Consider a machine with 8GB of RAM and somebody
>>> spawning a JVM with -Xms8g on it. It'll happily oblige? In one of my
>>> VMs with 8GB I see this:
>>>
>>> $ ./jdk19-jdk/bin/java -Xms8g -XX:+PrintFlagsFinal -version 2>&1 | grep InitialHeapSize
>>> size_t InitialHeapSize = 8589934592 {product} {command line}
>>> $ free --giga
>>> total used free shared buff/cache available
>>> Mem: 8 0 7 0 0 7
>>> Swap: 8 0 8
>>>
>>>> We have several choices:
>>>>
>>>> (a) Following the direction of JDK-8284900, check for the initial memory
>>>> usage of other types of memory as well. The problem with this is that
>>>> checking the "initial" usage some type of memory usage (malloc or stack)
>>>> is difficult or impossible.
>>>>
>>>> (b) Avoid committing more memory when the total memory usage is close to
>>>> the limit. E.g., avoid expanding the Java heap, or return NULL from
>>>> os::malloc(). The problem is that some operations in the VM cannot
>>>> handle malloc failure and will terminate the VM anyway. Also, trying to
>>>> obey the memory limit with concurrent allocator/deallocator threads is
>>>> probably difficult or impossible.
>>>>
>>>> (c) Do not consider the memory limit and let the VM be killed by the OOM
>>>> killer. This may be what the user wants -- the user knows that the app
>>>> will normally run within the memory limit, so if the heap expands too
>>>> much the app probably has a memory leak and should be killed, to be
>>>> automatically restarted by an external mechanism (such as Kubernetes).
>>> Neither of the above?
>>>
>>> It would perhaps make sense to do some sanitiy checks on JVM start-up
>>> (somewhat akin to JDK-8284900). But this should probably be done with
>>> the main operating system API. Not using OSContainer API as proposed in
>>> JDK-8284900. For example, print a warning if MaxHeapSize,
>>> InitialHeapSize and MinHeapSize are larger than the reported physical
>>> memory (and/or cap it at a lower size if so).
>>>
>>> Isn't (b) being handled by setting os::pysical_memory() and
>>> os::available_memory() to the container limits (which we currently do)?
>>> It would behave similarly to a physical machine running out of memory,
>>> no?
>>>
>>> Thanks,
>>> Severin
>>>
>>>
>>>> [1] https://bugs.openjdk.java.net/browse/JDK-8285277
>>>> [2] https://bugs.openjdk.java.net/browse/JDK-8284900
>>>>
More information about the hotspot-dev
mailing list