RFR: 8170888: [linux] Experimental support for cgroup memory limits in container (ie Docker) environments

[Karen Kinnear]

David,

Looks good. Thank you for printing the sample output.
And I like the scoping of the experimental flag name.

thanks,
Karen

> On Dec 8, 2016, at 9:59 PM, David Holmes <david.holmes at oracle.com> wrote:
> 
> Bug: https://bugs.openjdk.java.net/browse/JDK-8170888
> 
> webrev: http://cr.openjdk.java.net/~dholmes/8170888/webrev/
> 
> Adapted from the original submission here:
> 
> http://mail.openjdk.java.net/pipermail/hotspot-dev/2016-November/025404.html
> 
> and continuing:
> 
> http://mail.openjdk.java.net/pipermail/hotspot-dev/2016-December/025406.html
> 
> Christine: who should get original contribution credit - you or Andrew or both?
> 
> The change adds an experimental flag, -XX:+UseCGroupMemoryLimitForHeap, to opt-in to using the cgroup memory limit value as the value to use for "physical memory" when doing heap sizing. This is determined solely by the presence of, and then content of, the file /sys/fs/cgroup/memory/memory.limit_in_bytes. More extensive support will be investigated for JDK 10.
> 
> As the flag is opt-in, and we don't expect people to use it unless they expect there to be active cgroups, we warn if the file can not be found, or the memory limit value can not be parsed.
> 
> I considered making the flag linux-only and ifdef'ing the logic, but thought it might be useful to be able to pretend that a memory limit exists on other platforms just by creating this marker file (even though you'd need the right perms to create it).
> 
> Testing: manual - see below for output
> 
> Thanks,
> David
> -----
> 
> Here's an example run in a container with memory limited to 256M. First without using the flag:
> 
> root at 2f2664c92737:/# ./jdk-9/fastdebug/bin/java -XX:+PrintFlagsFinal -version | grep MaxHeap
>   uintx MaxHeapFreeRatio = 70 {manageable} {default}
>   size_t MaxHeapSize = 16873684992 {product} {ergonomic}
> java version "9-internal"
> Java(TM) SE Runtime Environment (fastdebug build 9-internal+0-2016-12-07-202859.daholme.jdk9-hs-dev2)
> Java HotSpot(TM) 64-Bit Server VM (fastdebug build 9-internal+0-2016-12-07-202859.daholme.jdk9-hs-dev2, mixed mode)
> 
> And then with:
> 
> root at 2f2664c92737:/# ./jdk-9/fastdebug/bin/java -XX:+PrintFlagsFinal -XX:+UnlockExperimentalVMOptions -XX:+UseCGroupMemoryLimitForHeap -version | grep MaxHeap
>    uintx MaxHeapFreeRatio = 70 {manageable} {default}
>   size_t MaxHeapSize = 132120576 {product} {ergonomic}
> java version "9-internal"
> Java(TM) SE Runtime Environment (fastdebug build 9-internal+0-2016-12-07-202859.daholme.jdk9-hs-dev2)
> Java HotSpot(TM) 64-Bit Server VM (fastdebug build 9-internal+0-2016-12-07-202859.daholme.jdk9-hs-dev2, mixed mode)
> 
> so we can see the affect on the heap size.
> 
> Example of logging output:
> 
> root at e3f0f811cd91:/# ./jdk-9/fastdebug/bin/java -XX:+UnlockExperimentalVMOptions -XX:+UseCGroupMemoryLimitForHeap -Xlog:gc+heap -version
> [0.003s][info][gc,heap] Setting phys_mem to the min of cgroup limit (128MB) and initial phys_mem (64353MB)
> [0.016s][info][gc,heap] Heap region size: 1M
> java version "9-internal"
> Java(TM) SE Runtime Environment (fastdebug build 9-internal+0-2016-12-07-202859.daholme.jdk9-hs-dev2)
> Java HotSpot(TM) 64-Bit Server VM (fastdebug build 9-internal+0-2016-12-07-202859.daholme.jdk9-hs-dev2, mixed mode)
> 
> Example of enabling the flag on a system with no cgroups:
> 
> > ./build/linux-x64-debug/images/jdk/bin/java -XX:+UnlockExperimentalVMOptions -XX:+UseCGroupMemoryLimitForHeap -version
> Java HotSpot(TM) 64-Bit Server VM warning: Unable to open cgroup memory limit file /sys/fs/cgroup/memory/memory.limit_in_bytes (No such file or directory)
> java version "9-internal"
> Java(TM) SE Runtime Environment (fastdebug build 9-internal+0-2016-12-07-202859.daholme.jdk9-hs-dev2)
> Java HotSpot(TM) 64-Bit Server VM (fastdebug build 9-internal+0-2016-12-07-202859.daholme.jdk9-hs-dev2, mixed mode)
> 
> Example logging output when running unlimited:
> 
> ./jdk-9/fastdebug/bin/java -XX:+UnlockExperimentalVMOptions -XX:+UseCGroupMemoryLimitForHeap -Xlog:gc+heap -version
> [0.002s][info][gc,heap] Setting phys_mem to the min of cgroup limit (8796093022207MB) and initial phys_mem (64353MB)
> [0.016s][info][gc,heap] Heap region size: 4M
> java version "9-internal"
> Java(TM) SE Runtime Environment (fastdebug build 9-internal+0-2016-12-07-202859.daholme.jdk9-hs-dev2)
> Java HotSpot(TM) 64-Bit Server VM (fastdebug build 9-internal+0-2016-12-07-202859.daholme.jdk9-hs-dev2, mixed mode)