RFR: 8292083: Detected container memory limit may exceed physical machine memory [v4]

[Jonathan Dowland]

On Thu, 18 Aug 2022 09:47:43 GMT, Thomas Stuefe <stuefe at openjdk.org> wrote:

>> Looking at the [original changeset](https://hg.openjdk.java.net/jdk/jdk/rev/7f22774a5f42#l11.30) for this it appears the only reason for it is to account for `-XX:+AggressiveHeap` and setting related flags. As to why the hook in `os::physical_memory()` isn't sufficient I'm at a loss. The original review here https://mail.openjdk.org/pipermail/hotspot-dev/2017-October/028635.html suggests that there was supposed to be some support for interleaving os initialization and arguments processing, but I don't see it. To be safe, I'd just leave this as it is for this patch and potentially clean this up as a separate bug. Thoughts?
>
> Okay. Maybe something for a follow-up RFE.

I've tried to address this in 4d8a85732c2ebbe31ae1dfc5c756ac14d4989755. My approach was to bubble the physical-ram-versus-cgroup-limit comparison right up to `os::physical_memory`. Now, the cgroups code simply reads and reports what is in cgroups (even if it's nonsense) and we no longer overwrite `Linux::_physical_memory` with the cgroup value (which felt misleading to me, and also, meant the physical host RAM value is lost to the JVM after the first cgroup read: if the JVM launched under a low cgroup memory limit that was later lifted, that higher number would never be reported)

Unfortunately, the logging is quite a lot noisier:


# baseline
$ /usr/lib/jvm/java-17-openjdk-amd64/bin/java -Xlog:os+container=trace -XX:InitialRAMPercentage=25.0 -XX:+PrintFlagsFinal -version 2>&1 | wc -l
724
$ /usr/lib/jvm/java-17-openjdk-amd64/bin/java -Xlog:os+container=trace -XX:InitialRAMPercentage=25.0 -XX:+PrintFlagsFinal -version 2>&1 | awk '{ $1=""; print }' | sort | uniq -c | sort -nk1 | tail
      3  Raw value for memory limit is: 137438953472
      4  Memory Usage is: 3886501888
      4  Open of file /sys/fs/cgroup/user.slice/user-1000.slice/user at 1000.service/app.slice/app-org.gnome.Terminal.slice/vte-spawn-cdfa23a8-08e3-4781-b456-573c833609e9.scope/cpu.max failed, No such file or directory
      4  Path to /cpu.max is /sys/fs/cgroup/user.slice/user-1000.slice/user at 1000.service/app.slice/app-org.gnome.Terminal.slice/vte-spawn-cdfa23a8-08e3-4781-b456-573c833609e9.scope/cpu.max
      6  Memory Usage is: 3887812608
      7  Memory Usage is: 3887157248
      8  Memory Usage is: 3888467968
      9  Memory Usage is: 3886239744
      9  Memory Usage is: 3888599040
     58  Path to /memory.current is /sys/fs/cgroup/user.slice/user-1000.slice/user at 1000.service/app.slice/app-org.gnome.Terminal.slice/vte-spawn-cdfa23a8-08e3-4781-b456-573c833609e9.scope/memory.current
# this PR
$ $JAVA_HOME/bin/java -Xlog:os+container=trace -XX:InitialRAMPercentage=25.0 -XX:+PrintFlagsFinal -version 2>&1 | wc -l
1418
$ ;$JAVA_HOME/bin/java -Xlog:os+container=trace -XX:InitialRAMPercentage=25.0 -XX:+PrintFlagsFinal -version 2>&1 | awk '{ $1=""; print }' | sort | uniq -c | sort -nk1 | tail
     10  Memory Usage is: 3833311232
     11  Memory Usage is: 3826597888
     11  Memory Usage is: 3827486720
     12  Memory Usage is: 3828273152
     13  Memory Usage is: 3826860032
     14  Memory Usage is: 3827118080
     15  Memory Usage is: 3827748864
     17  Memory Usage is: 3825704960
     20  Memory Usage is: 3826339840
    404  Path to /memory.current is /sys/fs/cgroup/user.slice/user-1000.slice/user at 1000.service/app.slice/app-org.gnome.Terminal.slice/vte-spawn-cdfa23a8-08e3-4781-b456-573c833609e9.scope/memory.current

-------------

PR: https://git.openjdk.org/jdk/pull/9880