Proposal: On Linux, add option to periodically trim the glibc heap

[Remi Forax]

----- Original Message -----
> From: "Kim Barrett" <kim.barrett at oracle.com>
> To: "Thomas Stüfe" <thomas.stuefe at gmail.com>
> Cc: "hotspot-dev" <hotspot-dev at openjdk.java.net>
> Sent: Vendredi 27 Août 2021 13:06:40
> Subject: Re: Proposal: On Linux, add option to periodically trim the glibc heap

>> On Aug 27, 2021, at 5:05 AM, Thomas Stüfe <thomas.stuefe at gmail.com> wrote:
>> 
>> Hi all,
>> 
>> I would like to gauge opinions on the following idea.
>> 
>> The glibc can be very reluctant to return released C-heap memory to the OS.
>> Memory returned to it via free(3) is typically cached within the glibc and
>> still counts toward the process RSS. It may be reused for
>> future allocations, but for now it is idling around. This manifests in the
>> process RSS not recovering from C-heap usage spikes, e.g. after intense
>> compilations. This effect is most pronounced with many small allocations
>> which have actually be touched - large allocations are mapped and unmapped
>> correctly on free(3).
>> 
>> I do not know of any other libc which has this problem to this extend. AIX
>> libc is somewhat bad too - in our proprietary VM we use manual mmap as a
>> backing for Arenas for that reason - but glibc is worse.
>> 
>> However, the glibc offers a very simple API to shake the memory loose and
>> return it to the OS: `malloc_trim(3)` [1].
> 
> Not general purpose, but this seems like something that could/should be added
> to G1 periodic GCs (https://openjdk.java.net/jeps/346). If the application is
> considered inactive, so that returning Java heap space to the OS is desirable,
> then similarly for C heap space.
> 
> Obviously that doesn't cover all the cases you are describing, but does cover
> what seems like an important one.  And this seems like it should be fairly easy.

Yes, and Shenandoah and ZGC are also able to uncommit unused Memory.

Rémi