RFR: 8305770: os::Linux::available_memory() should refer MemAvailable in /proc/meminfo

[Roberto Castañeda Lozano]

On Tue, 25 Apr 2023 04:45:56 GMT, Yasumasa Suenaga <ysuenaga at openjdk.org> wrote:

>> `os::Linux::available_memory()` returns available memory from cgroups or sysinfo(2). In case of the process which run on out of container, that value is based on `freeram` from sysinfo(2).
>> 
>> `freeram` is equivalent to `MemFree` in `/proc/meminfo` [1]. However it means just a free RAM. We should use `MemAvailable` when we want to know how much memory is available for the process [2]. `MemAvailable` is available in modern Linux kernel, and it has been backported some older kernels (e.g. RHEL). In `sar` from sysstat, it refers that value and shows it as `kbavail` [3].
>> 
>> AFAIK PhysicalMemory event in JFR depends on `os::Linux::available_memory()`, and it is used in automated analysis in JMC. So the JFR/JMC user could misunderstand physical memory was exhausted even if the memory was available enough.
>> 
>> [1] https://github.com/torvalds/linux/blob/c9c3395d5e3dcc6daee66c6908354d47bf98cb0c/fs/proc/meminfo.c#L59
>> [2] https://docs.kernel.org/filesystems/proc.html?highlight=memavailable
>> [3] https://github.com/sysstat/sysstat/blob/ac1df71ca252c158e8d418ded93e5ed52f5e8765/rd_stats.c#L325-L328
>
> Hi hotspot-compiler folks,
> 
> I'd like to change `os::Linux::available_memory()` to refer `MemAvailable` in /proc/meminfo on Linux. One of user of this function is JIT compiler. It is used for determine number of compiler threads. After this change, more compiler threads would be started because `MemAvailable` includes not only free memory but also some caches - it means `MemAvailable` is bigger than `MemFree`.
> 
> I think this change is not a problem because number of compiler threads is limited by `CICompilerCount`. Do you have any concerns in compiler perspective?

Hi @YaSuenag, just to confirm that this change would not lead to excessive creation/deletion of compiler threads (which can have a significant cost in terms of memory usage, see e.g. [JDK-8302264](https://bugs.openjdk.org/browse/JDK-8302264)), it would be useful to see some measurements about number of created compiler threads over the execution of some applications in a environment configured so that `available_memory / (200*M)` becomes the limiting factor in https://github.com/openjdk/jdk/blob/f968da97a5a5c68c28ad29d13fdfbe3a4adf5ef7/src/hotspot/share/compiler/compileBroker.cpp#L1024-L1027, before and after the change. This could be easily measured e.g. using `-XX:+TraceCompilerThreads`. Do you have (or could produce) such measurements?

-------------

PR Comment: https://git.openjdk.org/jdk/pull/13398#issuecomment-1521385514