RFR: JDK-8293114: GC should trim the native heap [v2]

[Aleksey Shipilev]

On Thu, 1 Sep 2022 06:47:27 GMT, Thomas Stuefe <stuefe at openjdk.org> wrote:

>> (*Updated 2023-07-05 to reflect the current state of the patch*)
>> 
>> This RFE adds the option to auto-trim the Glibc heap as part of the GC cycle. If the VM process suffered high temporary malloc spikes (regardless of whether from JVM- or user code), this could recover significant amounts of memory.
>> 
>> We discussed this a year ago [1], but the item got pushed to the bottom of my work pile, therefore, it took longer than I thought.
>> 
>> ### Motivation
>> 
>> The Glibc is reluctant to return memory to the OS, more so than other allocators. Temporary malloc spikes often carry over as permanent RSS increase. Note that C-heap retention is difficult to observe. Since it is freed memory, it won't appear in NMT; it is just a part of RSS.
>> 
>> This is, effectively, caching, and a performance tradeoff by the glibc. It makes a lot of sense with applications that cause high traffic on the C-heap (the typical native application). The JVM, however, clusters allocations and for a lot of use cases rolls its own memory management via mmap. And app's malloc load can fluctuate wildly, with temporary spikes and long idle periods.
>> 
>> To help, Glibc exports an API to trim the C-heap: `malloc_trim(3)`. With JDK 18 [2], SAP contributed a new jcmd command to *manually* trim the C-heap on Linux. This RFE adds a complementary way to trim automatically.
>> 
>> #### Is this even a problem?
>> 
>> Yes. 
>> 
>> The JVM clusters most native memory allocations and satisfies them with mmap. But there are enough C-heap allocations left to cause malloc spikes that are subject of memory retention. Note that one example are hotspot arenas themselves.
>> 
>> But many cases of high memory retention in Glibc I have seen in third-party JNI code. Libraries allocate large buffers via malloc as temporary buffers. In fact, since we introduced the jcmd "System.trim_native_heap", some of our customers started to call this command periodically in scripts to counter these issues.
>> 
>> ### How trimming works
>> 
>> Trimming is done via `malloc_trim(2)`. `malloc_trim` will iterate over all arenas and trim each one subsequently. While doing that, it will lock the arena, which may cause some (but not all) subsequent actions on the same arenas to block. glibc also trims automatically on free, but that is very limited (see https://github.com/openjdk/jdk/pull/10085#issuecomment-1619638641 for details).
>> 
>> `malloc_trim` offers almost no way to control its behavior; in particular, no way to limit its runtime. Its run...
>
> Thomas Stuefe has updated the pull request incrementally with two additional commits since the last revision:
> 
>  - reduce test runtime on slow hardware
>  - make tests more stable on slow hardware

src/hotspot/os/aix/os_aix.cpp line 2990:

> 2988: void os::print_memory_mappings(char* addr, size_t bytes, outputStream* st) {}
> 2989: 
> 2990: // stubbed-out trim-native support

I think these comments should be more succinct. Example: "Native heap trimming is not implemented yet." (this tells it can be implemented in future)

src/hotspot/os/linux/os_linux.cpp line 190:

> 188:   int keepcost;
> 189: };
> 190: typedef struct glibc_mallinfo (*mallinfo_func_t)(void);

Should be `os::Linux::glibc_mallinfo` for consistency?

src/hotspot/os/linux/os_linux.cpp line 5440:

> 5438:     out->uordblks = (int) mi.uordblks;
> 5439:     out->fordblks = (int) mi.fordblks;
> 5440:     out->keepcost = (int) mi.keepcost;

Style: please indent it so that `=` are in the same column?

src/hotspot/os/linux/os_linux.cpp line 5452:

> 5450:   return true;
> 5451: #else
> 5452:   return false; // musl

Let's avoid comments like `// musl` -- it might mislead if we ever go for e.g. `uClibc` and friends?

src/hotspot/os/linux/trimCHeapDCmd.cpp line 36:

> 34: 
> 35: void TrimCLibcHeapDCmd::execute(DCmdSource source, TRAPS) {
> 36:   if (os::can_trim_native_heap()) {

So, this dcmd command basically trims "asynchronously", in `GCTrimNative`-speak. This seems to be a bit at odds with `GCTrimNative` configured in "sync mode". Should it just notify the trimming thread in async mode, and leave it to GC in sync mode?

src/hotspot/share/gc/shared/gcTrimNativeHeap.cpp line 52:

> 50: // The mode is set as argument to GCTrimNative::initialize().
> 51: 
> 52: class NativeTrimmer : public ConcurrentGCThread {

It is a bit ugly it pretends to be `ConcurrentGCThread`. Can it be just `NamedThread`?

src/hotspot/share/gc/shared/gcTrimNativeHeap.cpp line 67:

> 65: 
> 66:     // Note: GCTrimNativeHeapInterval=0 -> zero wait time -> indefinite waits, disabling periodic trim
> 67:     const int64_t delay_ms = GCTrimNativeHeapInterval * 1000;

I wonder if `run_service` should just exit if no periodic trims are configured, instead of wasting a thread here.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/10085#discussion_r980059530
PR Review Comment: https://git.openjdk.org/jdk/pull/10085#discussion_r980062314
PR Review Comment: https://git.openjdk.org/jdk/pull/10085#discussion_r980064163
PR Review Comment: https://git.openjdk.org/jdk/pull/10085#discussion_r980066226
PR Review Comment: https://git.openjdk.org/jdk/pull/10085#discussion_r980101247
PR Review Comment: https://git.openjdk.org/jdk/pull/10085#discussion_r980094065
PR Review Comment: https://git.openjdk.org/jdk/pull/10085#discussion_r980095999