[Proposal] Better systemd slice memory limit support for OpenJDK

[Bob Vandette]

Do you happen to know if this issue impacts Ubuntu 19.04?

Looking at our official OS support list, we’re only supporting Ubuntu 18.04 with JDK 12 which 
is using kernel version 4.15.  Even if this gets bumped to 18.10, this is still only using 4.18.

Go ahead and file a bug while we look into this issue a bit more.  I’d like to know the exact
kernel cutoff and which distros are impacted.  If it’s wide-spread on distros getting released 
this year, we should fix it in JDK 13.

Bob.


> On Jan 17, 2019, at 11:16 AM, Severin Gehwolf <sgehwolf at redhat.com> wrote:
> 
> Hi Bob,
> 
> On Thu, 2019-01-17 at 10:59 -0500, Bob Vandette wrote:
>> I checked a few systems I have access to and they all have use_hierarchy enabled.  When I set a memory
>> limit the memory.stat hierarchical_memory_limit is identical to the memory.limit_in_bytes contents.
> 
> This seems to suggest they're running on older kernels. I know F29, F28
> are affected.
> 
>> Do you have any idea why the kernel behavior changed?
> 
> No, not really. It could be with the "unified control group hierarchy"
> work done in 3.16, but that's speculation:
> https://lwn.net/Articles/601840/
> 
> Some more discussion with systemd/kernel folk is here:
> https://bugzilla.redhat.com/show_bug.cgi?id=1599387
> 
>>  Is this documented behavior?
> 
> Yes. Though, in the self-proclaimed very outdated document:
> https://www.kernel.org/doc/Documentation/cgroup-v1/memory.txt
> 
> Section 6, "Hierarchy support".
> 
>> I wouldn’t want to add a work-around for a transient kernel bug.
> 
> FWIW, cgroup-v1 is a legacy interface, so I doubt anything in that code
> has a great chance of getting "fixed". So far there is no sign of
> evidence what the expected behaviour should be.
> 
> Thanks,
> Severin
> 
>> Bob.
>> 
>> 
>>> On Jan 17, 2019, at 9:57 AM, Severin Gehwolf <sgehwolf at redhat.com> wrote:
>>> 
>>> Hi,
>>> 
>>> Current container awareness for OpenJDK seems to work for systemd
>>> slices too, on some systems. To be precise, this works for older
>>> Kernels e.g. 3.10. However, we've noticed that this breaks for newer
>>> Kernel versions[1] such as the one in F28, currently 4.19.14-200. If
>>> the container support would also look at hierarchical memory limits
>>> exposed via memory.stat in the cgroup file system, it would again
>>> work[2]. A proof of concept implementation is here:
>>> 
>>> http://cr.openjdk.java.net/~sgehwolf/webrevs/container-systemd-slice-01/webrev/
>>> 
>>> This enhancement wouldn't change any existing container memory limit
>>> detection as it only kicks in when all other look-ups determined that
>>> there is no limit in place. I've verified this by running Docker
>>> container tests. The idea is to look for hierarchical_memory_limit and
>>> hierarchical_memsw_limit lines in the memory.stat file of the cgroup
>>> tree.
>>> 
>>> Would it be possible to consider such an enhancement upstream? If so,
>>> I'll file a bug and propose it for review.
>>> 
>>> This issue has been originally raised here:
>>> https://bugzilla.redhat.com/show_bug.cgi?id=1509371
>>> 
>>> Thanks,
>>> Severin
>>> 
>>> [1] Java process gets killed by oom killer.
>>>   See: http://cr.openjdk.java.net/~sgehwolf/webrevs/container-systemd-slice-01/before.txt
>>> [2] Java process throws OutOfMemoryError as expected.
>>>   See: http://cr.openjdk.java.net/~sgehwolf/webrevs/container-systemd-slice-01/after.txt
>>> 
>> 
>> 
>