RFR (8u): JDK-8146115: Improve docker container detection and resource configuration usage

Fri May 18 13:30:38 UTC 2018

Hello Bob,

Thanks a lot for reviewing the changes!

On 5/17/18 11:12 AM, Bob Vandette wrote:
> The backport of my changes look pretty good.
>
> If the new PrintContainerInfo option is only referenced on Linux platforms, you might
> want to move it to globals_linux.hpp.
Yes, currently it is being used only on Linux platforms, but I think it 
is a general option and might be used on other platforms at a later 
date.  So I think we can leave it in globals.hpp.
>
> Is there a reason PrintActiveCpus is a diagnostic flag but PrintContainerInfo is not?
Yes, PrintContainerInfo should also be a diagnostic option. I have 
changed it.
Updated webrev: http://cr.openjdk.java.net/~poonam/8146115/webrev.01/

>
> Is it acceptable to add these new VM flags in a backport that won’t be supported in the latest release?
Since JDK 9 and later releases use Unified JVM logging, and we don't 
have that in JDK8, a new option is required in 8 to log the information 
which is being logged under 'container' tag in 10 and 11. We will need 
to have a CSR request approved for the new JVM options added as part of 
this backport.

Thanks,
Poonam

>
> Bob.
>
>
>> On May 15, 2018, at 4:46 PM, Poonam Parhar <poonam.bajaj at oracle.com> wrote:
>>
>> Hello,
>>
>> Please review the docker container support changes backported to JDK 8u. These changes include the backport of the following enhancement and the follow-on bug fixes done on top of that in jdk 10 and 11.
>>
>> Webrev: http://cr.openjdk.java.net/~poonam/8146115/webrev.00/
>>
>> Enhancement:JDK-8146115: Improve docker container detection and resource configuration usage <https://bugs.openjdk.java.net/browse/JDK-8146115>
>>
>> The changes also include the fixes for the following two bugs:
>> Bug JDK-8186248: Allow more flexibility in selecting Heap % of available RAM <https://bugs.openjdk.java.net/browse/JDK-8186248><https://bugs.openjdk.java.net/browse/JDK-8190283>
>> BugJDK-8190283: <https://bugs.openjdk.java.net/browse/JDK-8190283> Default heap sizing options select a MaxHeapSize larger than available physical memory in some cases <https://bugs.openjdk.java.net/browse/JDK-8190283>
>>
>> These changes add a new JVM option 'PrintContainerInfo' for tracing container related information which is specific to jdk 8.
>>
>> Testing results (with -XX:+UnlockDiagnosticVMOptions -XX:+PrintContainerInfo -XX:+PrintActiveCpus JVM options):
>> --------------
>> poonam at poonam-VirtualBox:~/docker-image$ java TestCPUsMemory
>> Number of Processors: 3
>> Max Memory: 921174016
>> poonam at poonam-VirtualBox:~/docker-image$ sudo docker run --cpus 1 -m1024m  --rm myimage
>> [sudo] password for poonam:
>> WARNING: Your kernel does not support swap limit capabilities or the cgroup is not mounted. Memory limited without swap.
>> OSContainer::init: Initializing Container Support
>> Path to /memory.limit_in_bytes is /sys/fs/cgroup/memory/memory.limit_in_bytes
>> Memory Limit is: 1073741824
>> active_processor_count: sched_getaffinity processor count: 3
>> Path to /cpu.cfs_quota_us is /sys/fs/cgroup/cpu,cpuacct/cpu.cfs_quota_us
>> CPU Quota is: 100000
>> Path to /cpu.cfs_period_us is /sys/fs/cgroup/cpu,cpuacct/cpu.cfs_period_us
>> CPU Period is: 100000
>> Path to /cpu.shares is /sys/fs/cgroup/cpu,cpuacct/cpu.shares
>> CPU Shares is: 1024
>> CPU Quota count based on quota/period: 1
>> OSContainer::active_processor_count: 1
>> active_processor_count: determined by OSContainer: 1
>> active_processor_count: sched_getaffinity processor count: 3
>> Path to /cpu.cfs_quota_us is /sys/fs/cgroup/cpu,cpuacct/cpu.cfs_quota_us
>> CPU Quota is: 100000
>> Path to /cpu.cfs_period_us is /sys/fs/cgroup/cpu,cpuacct/cpu.cfs_period_us
>> CPU Period is: 100000
>> Path to /cpu.shares is /sys/fs/cgroup/cpu,cpuacct/cpu.shares
>> CPU Shares is: 1024
>> CPU Quota count based on quota/period: 1
>> OSContainer::active_processor_count: 1
>> active_processor_count: determined by OSContainer: 1
>> Path to /memory.limit_in_bytes is /sys/fs/cgroup/memory/memory.limit_in_bytes
>> Memory Limit is: 1073741824
>> total container memory: 1073741824
>> active_processor_count: sched_getaffinity processor count: 3
>> Path to /cpu.cfs_quota_us is /sys/fs/cgroup/cpu,cpuacct/cpu.cfs_quota_us
>> CPU Quota is: 100000
>> Path to /cpu.cfs_period_us is /sys/fs/cgroup/cpu,cpuacct/cpu.cfs_period_us
>> CPU Period is: 100000
>> Path to /cpu.shares is /sys/fs/cgroup/cpu,cpuacct/cpu.shares
>> CPU Shares is: 1024
>> CPU Quota count based on quota/period: 1
>> OSContainer::active_processor_count: 1
>> active_processor_count: determined by OSContainer: 1
>> active_processor_count: sched_getaffinity processor count: 3
>> Path to /cpu.cfs_quota_us is /sys/fs/cgroup/cpu,cpuacct/cpu.cfs_quota_us
>> CPU Quota is: 100000
>> Path to /cpu.cfs_period_us is /sys/fs/cgroup/cpu,cpuacct/cpu.cfs_period_us
>> CPU Period is: 100000
>> Path to /cpu.shares is /sys/fs/cgroup/cpu,cpuacct/cpu.shares
>> CPU Shares is: 1024
>> CPU Quota count based on quota/period: 1
>> OSContainer::active_processor_count: 1
>> active_processor_count: determined by OSContainer: 1
>>
>> active_processor_count: sched_getaffinity processor count: 3
>> Path to /cpu.cfs_quota_us is /sys/fs/cgroup/cpu,cpuacct/cpu.cfs_quota_us
>> CPU Quota is: 100000
>> Path to /cpu.cfs_period_us is /sys/fs/cgroup/cpu,cpuacct/cpu.cfs_period_us
>> CPU Period is: 100000
>> Path to /cpu.shares is /sys/fs/cgroup/cpu,cpuacct/cpu.shares
>> CPU Shares is: 1024
>> CPU Quota count based on quota/period: 1
>> OSContainer::active_processor_count: 1
>> active_processor_count: determined by OSContainer: 1
>> Number of Processors: 1
>> Max Memory: 259522560
>>
>> poonam at poonam-VirtualBox:~/docker-image$ sudo docker run --cpu-shares 2048 -m1024m  --rm myimage
>> WARNING: Your kernel does not support swap limit capabilities or the cgroup is not mounted. Memory limited without swap.
>> OSContainer::init: Initializing Container Support
>> Path to /memory.limit_in_bytes is /sys/fs/cgroup/memory/memory.limit_in_bytes
>> Memory Limit is: 1073741824
>> active_processor_count: sched_getaffinity processor count: 3
>> Path to /cpu.cfs_quota_us is /sys/fs/cgroup/cpu,cpuacct/cpu.cfs_quota_us
>> CPU Quota is: -1
>> Path to /cpu.cfs_period_us is /sys/fs/cgroup/cpu,cpuacct/cpu.cfs_period_us
>> CPU Period is: 100000
>> Path to /cpu.shares is /sys/fs/cgroup/cpu,cpuacct/cpu.shares
>> CPU Shares is: 2048
>> CPU Share count based on shares: 2
>> OSContainer::active_processor_count: 2
>> active_processor_count: determined by OSContainer: 2
>> active_processor_count: sched_getaffinity processor count: 3
>> Path to /cpu.cfs_quota_us is /sys/fs/cgroup/cpu,cpuacct/cpu.cfs_quota_us
>> CPU Quota is: -1
>> Path to /cpu.cfs_period_us is /sys/fs/cgroup/cpu,cpuacct/cpu.cfs_period_us
>> CPU Period is: 100000
>> Path to /cpu.shares is /sys/fs/cgroup/cpu,cpuacct/cpu.shares
>> CPU Shares is: 2048
>> CPU Share count based on shares: 2
>> OSContainer::active_processor_count: 2
>> active_processor_count: determined by OSContainer: 2
>> Path to /memory.limit_in_bytes is /sys/fs/cgroup/memory/memory.limit_in_bytes
>> Memory Limit is: 1073741824
>> total container memory: 1073741824
>> Path to /memory.limit_in_bytes is /sys/fs/cgroup/memory/memory.limit_in_bytes
>> Memory Limit is: 1073741824
>> total container memory: 1073741824
>> active_processor_count: sched_getaffinity processor count: 3
>> Path to /cpu.cfs_quota_us is /sys/fs/cgroup/cpu,cpuacct/cpu.cfs_quota_us
>> CPU Quota is: -1
>> Path to /cpu.cfs_period_us is /sys/fs/cgroup/cpu,cpuacct/cpu.cfs_period_us
>> CPU Period is: 100000
>> Path to /cpu.shares is /sys/fs/cgroup/cpu,cpuacct/cpu.shares
>> CPU Shares is: 2048
>> CPU Share count based on shares: 2
>> OSContainer::active_processor_count: 2
>> active_processor_count: determined by OSContainer: 2
>> active_processor_count: sched_getaffinity processor count: 3
>> Path to /cpu.cfs_quota_us is /sys/fs/cgroup/cpu,cpuacct/cpu.cfs_quota_us
>> CPU Quota is: -1
>> Path to /cpu.cfs_period_us is /sys/fs/cgroup/cpu,cpuacct/cpu.cfs_period_us
>> CPU Period is: 100000
>> Path to /cpu.shares is /sys/fs/cgroup/cpu,cpuacct/cpu.shares
>> CPU Shares is: 2048
>> CPU Share count based on shares: 2
>> OSContainer::active_processor_count: 2
>> active_processor_count: determined by OSContainer: 2
>>
>> active_processor_count: sched_getaffinity processor count: 3
>> Path to /cpu.cfs_quota_us is /sys/fs/cgroup/cpu,cpuacct/cpu.cfs_quota_us
>> CPU Quota is: -1
>> Path to /cpu.cfs_period_us is /sys/fs/cgroup/cpu,cpuacct/cpu.cfs_period_us
>> CPU Period is: 100000
>> Path to /cpu.shares is /sys/fs/cgroup/cpu,cpuacct/cpu.shares
>> CPU Shares is: 2048
>> CPU Share count based on shares: 2
>> OSContainer::active_processor_count: 2
>> active_processor_count: determined by OSContainer: 2
>> Number of Processors: 2
>> Max Memory: 259522560
>>
>> poonam at poonam-VirtualBox:~/docker-image$ sudo docker run --cpuset-cpus 0-1 -m1024m  --rm myimage
>> WARNING: Your kernel does not support swap limit capabilities or the cgroup is not mounted. Memory limited without swap.
>> OSContainer::init: Initializing Container Support
>> Path to /memory.limit_in_bytes is /sys/fs/cgroup/memory/memory.limit_in_bytes
>> Memory Limit is: 1073741824
>> active_processor_count: sched_getaffinity processor count: 2
>> Path to /cpu.cfs_quota_us is /sys/fs/cgroup/cpu,cpuacct/cpu.cfs_quota_us
>> CPU Quota is: -1
>> Path to /cpu.cfs_period_us is /sys/fs/cgroup/cpu,cpuacct/cpu.cfs_period_us
>> CPU Period is: 100000
>> Path to /cpu.shares is /sys/fs/cgroup/cpu,cpuacct/cpu.shares
>> CPU Shares is: 1024
>> OSContainer::active_processor_count: 2
>> active_processor_count: determined by OSContainer: 2
>> active_processor_count: sched_getaffinity processor count: 2
>> Path to /cpu.cfs_quota_us is /sys/fs/cgroup/cpu,cpuacct/cpu.cfs_quota_us
>> CPU Quota is: -1
>> Path to /cpu.cfs_period_us is /sys/fs/cgroup/cpu,cpuacct/cpu.cfs_period_us
>> CPU Period is: 100000
>> Path to /cpu.shares is /sys/fs/cgroup/cpu,cpuacct/cpu.shares
>> CPU Shares is: 1024
>> OSContainer::active_processor_count: 2
>> active_processor_count: determined by OSContainer: 2
>> Path to /memory.limit_in_bytes is /sys/fs/cgroup/memory/memory.limit_in_bytes
>> Memory Limit is: 1073741824
>> total container memory: 1073741824
>> Path to /memory.limit_in_bytes is /sys/fs/cgroup/memory/memory.limit_in_bytes
>> Memory Limit is: 1073741824
>> total container memory: 1073741824
>> active_processor_count: sched_getaffinity processor count: 2
>> Path to /cpu.cfs_quota_us is /sys/fs/cgroup/cpu,cpuacct/cpu.cfs_quota_us
>> CPU Quota is: -1
>> Path to /cpu.cfs_period_us is /sys/fs/cgroup/cpu,cpuacct/cpu.cfs_period_us
>> CPU Period is: 100000
>> Path to /cpu.shares is /sys/fs/cgroup/cpu,cpuacct/cpu.shares
>> CPU Shares is: 1024
>> OSContainer::active_processor_count: 2
>> active_processor_count: determined by OSContainer: 2
>> active_processor_count: sched_getaffinity processor count: 2
>> Path to /cpu.cfs_quota_us is /sys/fs/cgroup/cpu,cpuacct/cpu.cfs_quota_us
>> CPU Quota is: -1
>> Path to /cpu.cfs_period_us is /sys/fs/cgroup/cpu,cpuacct/cpu.cfs_period_us
>> CPU Period is: 100000
>> Path to /cpu.shares is /sys/fs/cgroup/cpu,cpuacct/cpu.shares
>> CPU Shares is: 1024
>> OSContainer::active_processor_count: 2
>> active_processor_count: determined by OSContainer: 2
>>
>> active_processor_count: sched_getaffinity processor count: 2
>> Path to /cpu.cfs_quota_us is /sys/fs/cgroup/cpu,cpuacct/cpu.cfs_quota_us
>> CPU Quota is: -1
>> Path to /cpu.cfs_period_us is /sys/fs/cgroup/cpu,cpuacct/cpu.cfs_period_us
>> CPU Period is: 100000
>> Path to /cpu.shares is /sys/fs/cgroup/cpu,cpuacct/cpu.shares
>> CPU Shares is: 1024
>> OSContainer::active_processor_count: 2
>> active_processor_count: determined by OSContainer: 2
>> Number of Processors: 2
>> Max Memory: 259522560
>> ------------------
>>
>> Thanks,
>> Poonam
>>