RFR: JDK-8293472: Incorrect container resource limit detection if manual cgroup fs mounts present [v9]

[Ioi Lam]

On Thu, 15 Sep 2022 02:58:41 GMT, 王超 <duke at openjdk.org> wrote:

>> test/hotspot/jtreg/containers/docker/DockerBasicTest.java line 94:
>> 
>>> 92:         DockerTestUtils.dockerRunJava(opts)
>>> 93:             .shouldHaveExitValue(0)
>>> 94:             .shouldNotContain("[os,container]");
>> 
>> I think it's worth commenting what you are testing for here. Maybe something like
>> 
>> 
>> Duplicated cgroup mounts should be handled by the container detection
>> code and should not cause any error/warning output.
>> 
>> 
>> The `[os,container]` may not match exactly. Some extra spaces may be inserted before the close bracket if longer log tags have been printed. It should be changed to 
>> 
>> 
>> shouldNotMatch("[os,container *]");
>> 
>> 
>> (Please run the test on a build without your fix to verified that it would indeed fail).
>
> Thank you for the review suggestions.
> 
> The comment has been added. When turn on `-Xlog:os+container=trace `, the output is like the following
> 
> [0.000s][trace][os,container] OSContainer::init: Initializing Container Support
> [0.000s][debug][os,container] Detected optional pids controller entry in /proc/cgroups
> [0.000s][debug][os,container] Duplicate pids controllers detected. Picking /sys/fs/cgroup/pids, skipping /cgroups-in/pids.
> [0.000s][debug][os,container] Duplicate cpuset controllers detected. Picking /sys/fs/cgroup/cpuset, skipping /cgroups-in/cpuset.
> [0.000s][debug][os,container] Duplicate cpu controllers detected. Picking /sys/fs/cgroup/cpu,cpuacct, skipping /cgroups-in/cpu,cpuacct.
> [0.001s][debug][os,container] Duplicate cpuacct controllers detected. Picking /sys/fs/cgroup/cpu,cpuacct, skipping /cgroups-in/cpu,cpuacct.
> [0.001s][debug][os,container] Duplicate memory controllers detected. Picking /sys/fs/cgroup/memory, skipping /cgroups-in/memory.
> [0.001s][debug][os,container] Detected cgroups hybrid or legacy hierarchy, using cgroups v1 controllers
> [0.001s][trace][os,container] Path to /memory.use_hierarchy is /sys/fs/cgroup/memory/memory.use_hierarchy
> [0.001s][trace][os,container] Use Hierarchy is: 1
> [0.001s][trace][os,container] Path to /cpu.cfs_quota_us is /sys/fs/cgroup/cpu,cpuacct/cpu.cfs_quota_us
> [0.001s][trace][os,container] CPU Quota is: -1
> [0.001s][trace][os,container] Path to /cpu.cfs_period_us is /sys/fs/cgroup/cpu,cpuacct/cpu.cfs_period_us
> [0.001s][trace][os,container] CPU Period is: 100000
> [0.001s][trace][os,container] OSContainer::active_processor_count: 16
> [0.001s][trace][os,container] CgroupSubsystem::active_processor_count (cached): 16
> [0.001s][trace][os,container] total physical memory: 33396502528
> [0.001s][trace][os,container] Path to /memory.limit_in_bytes is /sys/fs/cgroup/memory/memory.limit_in_bytes
> [0.001s][trace][os,container] Memory Limit is: 9223372036854771712
> [0.001s][trace][os,container] Non-Hierarchical Memory Limit is: Unlimited
> [0.001s][trace][os,container] Path to /memory.stat is /sys/fs/cgroup/memory/memory.stat
> [0.001s][trace][os,container] Hierarchical Memory Limit is: 9223372036854771712
> [0.001s][trace][os,container] Hierarchical Memory Limit is: Unlimited
> [0.001s][debug][os,container] container memory limit unlimited: -1, using host value 33396502528
> [0.003s][trace][os,container] CgroupSubsystem::active_processor_count (cached): 16
> [0.045s][trace][os,container] Path to /cpu.cfs_quota_us is /sys/fs/cgroup/cpu,cpuacct/cpu.cfs_quota_us
> [0.045s][trace][os,container] CPU Quota is: -1
> [0.045s][trace][os,container] Path to /cpu.cfs_period_us is /sys/fs/cgroup/cpu,cpuacct/cpu.cfs_period_us
> [0.045s][trace][os,container] CPU Period is: 100000
> [0.045s][trace][os,container] OSContainer::active_processor_count: 16
> [0.052s][trace][os,container] total physical memory: 33396502528
> [0.052s][trace][os,container] Path to /memory.limit_in_bytes is /sys/fs/cgroup/memory/memory.limit_in_bytes
> [0.052s][trace][os,container] Memory Limit is: 9223372036854771712
> [0.052s][trace][os,container] Non-Hierarchical Memory Limit is: Unlimited
> [0.052s][trace][os,container] Path to /memory.stat is /sys/fs/cgroup/memory/memory.stat
> [0.052s][trace][os,container] Hierarchical Memory Limit is: 9223372036854771712
> [0.052s][trace][os,container] Hierarchical Memory Limit is: Unlimited
> [0.052s][debug][os,container] container memory limit unlimited: -1, using host value 33396502528
> 
> 
> so `"[os,container *]"` will not match the output, and add an extra `*` will make failed test pass.

I am sorry, I meant to use a regular expression to account for any possible space characters that comes after the `os,container`. Example:


$ java -Xlog:all -Xlog:os+container=debug --version | grep -i container | tail -2
[0.033s][debug][os,container   ] container memory limit unlimited: -1, using host value
[0.033s][debug][os,container   ] container memory limit unlimited: -1, using host value


Instead of `shouldNotContain`, which takes a plain string, you should use `shouldNotMatch`, which takes a regular expression:


.shouldNotMatch("[os,container *]");

-------------

PR: https://git.openjdk.org/jdk/pull/10193