RFR: 8365306: Provide OS Process Size and Libc statistic metrics to JFR

[Erik Gahlin]

On Wed, 13 Aug 2025 09:42:57 GMT, Thomas Stuefe <stuefe at openjdk.org> wrote:

> This provides the following new metrics:
> - `ProcessSize` event (new, periodic) 
>   - vsize (for analyzing address-space fragmentation issues)
>   - RSS including subtypes (subtypes are useful for excluding atypical issues, e.g. kernel problems that cause large file buffer bloat)
>   - peak RSS 
>   - process swap (if we swap we cannot trust the RSS values, plus it indicates bad sizing)
>   - pte size (to quickly see if we run with a super-large working set but an unsuitably small page size)
> - `LibcStatistics` (new, periodic)
>   - outstanding malloc size (important counterpoint to whatever NMT tries to tell me, which alone is often misleading)
>   - retained malloc size (super-important for the same reason)
>   - number of libc trims the hotspot executed (needed to gauge the usefulness of the retain counter, and to see if a customer employs native heap auto trimming (`-XX:TrimNativeHeapInterval`)
> - `NativeHeapTrim` (new, event-driven) (for both manual and automatic trims)
>    - RSS before and RSS after
>    - RSS recovered by this trim
>    - whether it was an automatic or manual trim
>    - duration
> - `JavaThreadStatistic`
>   - os thread counter (new field) (useful to understand the behavior of third-party code in our process if threads are created that bypass the JVM. E.g. some custom launchers do that.)
>   - nonJava thread counter (new field) (needed to interprete the os thread counter)
> 
> Notes:
> - we already have `ResidentSetSize` event, and the new `ProcessSize` event is a superset of that. I don't know how these cases are handled. I'd prefer to throw the old event out, but JMC has a hard-coded chart for RSS, so I kept it in unless someone tells me to remove it.
> 
> - Obviously, the libc events are very platform-specific. Still, I argue that these metrics are highly useful. We want people to use JFR and JMC; people include developers that are dealing with performance problems that require platform-specific knowledge to understand. See my comment in the JBS issue.
> 
> I provided implementations, as far as possible, to Linux, MacOS and Windows.
> 
> Testing:
> - ran the new tests manually and as part of GHAs

Events can be viewed as tables in a relational database. Duplicating information or designing tables based on the user interface is not a good idea. Correlating or grouping data from different events is not a new problem. The way we have solved it in the past is by using an explicit ID, similar to a foreign key (gcId, compilerId, safepointId etc), or by using the same timestamp (ActiveSetting, NativeMemory* etc.). In this case, I think a timestamp makes more sense. Tools can place the value on a timeline and have memory on the y-axis.

An alternative design is to do something similar to the NativeMemoryUsage event. That is, to split the data per type

    <Event name="ProcessMemoryUsage">
      <Field type="MemoryType" name="type" label="Type" />
      <Field type="ulong" contentType="bytes" name="amount" label="Amount"/>
    </Event>

and then emit what is supported on a specific platform and have all those event with the same timestamp.

I'm not sure which of the following memory types make sense to include, there is also a maintenance aspect, or if it the list is correct or complete, but the data would be normalized, even if it differs depending on the platform.

    Memory Type                 Linux    macOS   Windows Description
    -------------------------- ------ -------- --------- ----------------------------------------------------------'
    Resident Set Size             Yes      Yes         -  Resident memory currently in physical RAM
    Private Bytes                   -        -       Yes  Privately allocated memory
    Shared Memory                 Yes      Yes       Yes  Memory shared among processes
    Virtual Memory Size           Yes      Yes       Yes  Total reserved address space
    Commit Charge                   -        -       Yes  Promised/guaranteed memory for process
    Mapped File Memory            Yes      Yes       Yes  Memory mapped from files
    Page Table Size               Yes        -         -  Memory used for page tables
    Swap / Swapped                Yes      Yes       Yes  Memory swapped out to disk
    Wired Memory                    -      Yes         -  Locked memory that can't be paged out
    Compressed Memory               -      Yes         -  Memory compressed by the system
    Paged Pool                      -        -       Yes  Pool for pageable kernel allocations
    Non-paged Pool                  -        -       Yes  Pool for non-pageable kernel allocations
    Working Set                     -        -       Yes  Memory pages currently in RAM for the process
    Stack                         Yes      Yes       Yes  Memory used by call stacks
    Heap / Malloc                 Yes      Yes       Yes  Dynamically allocated memory (heap)
    Image                         Yes      Yes         -  Memory used for executables and loaded libraries

-------------

PR Comment: https://git.openjdk.org/jdk/pull/26756#issuecomment-3224085095