Proposal: On Linux, add option to periodically trim the glibc heap

[Kim Barrett]

> On Aug 27, 2021, at 5:05 AM, Thomas Stüfe <thomas.stuefe at gmail.com> wrote:
> 
> Hi all,
> 
> I would like to gauge opinions on the following idea.
> 
> The glibc can be very reluctant to return released C-heap memory to the OS.
> Memory returned to it via free(3) is typically cached within the glibc and
> still counts toward the process RSS. It may be reused for
> future allocations, but for now it is idling around. This manifests in the
> process RSS not recovering from C-heap usage spikes, e.g. after intense
> compilations. This effect is most pronounced with many small allocations
> which have actually be touched - large allocations are mapped and unmapped
> correctly on free(3).
> 
> I do not know of any other libc which has this problem to this extend. AIX
> libc is somewhat bad too - in our proprietary VM we use manual mmap as a
> backing for Arenas for that reason - but glibc is worse.
> 
> However, the glibc offers a very simple API to shake the memory loose and
> return it to the OS: `malloc_trim(3)` [1].

Not general purpose, but this seems like something that could/should be added
to G1 periodic GCs (https://openjdk.java.net/jeps/346). If the application is
considered inactive, so that returning Java heap space to the OS is desirable,
then similarly for C heap space.

Obviously that doesn't cover all the cases you are describing, but does cover
what seems like an important one.  And this seems like it should be fairly easy.