Integrating Native Memory Tracking (NMT) with core libraries

[Johan Sjolen]

Hi,

I'd like to present a new proposal for integration of NMT with OpenJDK's native/core libraries. This is the third time that such a proposal appears on the mailing lists. The first two proposals were initiated by Thomas Stüfe, so thank you to him for taking this on first and opening the discussion.
If you wonder why we want to have NMT in the native/core libraries, then I think that Thomas did a great case for why in [1], so please read that! Thomas also has a good explanation of the costs and risks for his proposal, the same costs and risks also apply to this proposal.

Thomas's ideas were as follows:
1. Adding an interposition library via LD_PRELOAD which captures all relevant allocation/free calls and accounts for these in NMT.
   Unfortunately, the tagging granularity is quite coarse for an interposition library, as no code is changed to provide more details.
2. Directly exporting NMT's current API to the native libraries via jvm.h [1]
   The idea of adding new memory categories for core libraries by changing source code in Hotspot was deemed as undesirable.
   Instead, it was requested that the core libraries can create new memory categories for themselves.

So, the goal for a good proposal is to have sufficient category granularity, and that core libraries can be free in specifying their own categories.
I believe that my proposal fulfills both of these goals, and so I'd like to open up discussion for this third approach. We will still have an interface exposed via jvm.h but with the possibility of creating new memory tags (categories) dynamically at runtime.  This solves both the granularity issue, and let's us avoid changing `memTags.hpp` for the addition of new ones. I have written a small API and implemented a basic prototype of it[0]. NMT is a fairly large system, with multiple modes of operation (summary & detail) and multiple trackers (malloc, virtual, and memory file). For an initial changeset I intend to only implement malloc tracking in summary mode, with the intention of implementing the remaining NMT features in future RFEs. I believe that this gives you "80% of the value for 20% of the cost".

Tags are created by calling `JVM_MakeNamedAllocator` which in turn returns a handle to a `named allocator`.  A named allocator is uniquely identified by its name, a string provided during creation. The handles are then used similarly to `MemTag`s in HotSpot today, in that they are passed to allocation functions in order to provide NMT with information on accounting.
I chose `NamedAllocator` because it's unique and can be read out loud. I appreciate a bit of bike shedding when it comes to naming, so please suggest better names if you've got them.

The API is as follows:
```c
typedef struct {
  int32_t allocator_info_handle;
} named_allocator_t;

/*
  named_allocator_t x = JVM_MakeNamedAllocator("MyName");
  named_allocaotr_t x2 = JVM_MakeNamedAllocator("MyName");
  assert(x.allocator_info_handle == x2.allocator_info_handle); // Same name returns same allocator_info_handle

  Making named allocators is serialized across threads, that is: A lock is taken.
  The remainder of the API is lock-free as it pertains to NMT. Of course, no guarantees are given to the underlying malloc-implementation.
*/
JNIEXPORT named_allocator_t JVM_MakeNamedAllocator(const char *name);
JNIEXPORT void * JVM_NamedAllocatorAlloc(size_t size, named_allocator_t a);
JNIEXPORT void * JVM_NamedAllocatorRealloc(void *p, size_t size, named_allocator_t a);
JNIEXPORT void * JVM_NamedAllocatorCalloc(size_t numelems, size_t elemsize, named_allocator_t a);
JNIEXPORT void   JVM_NamedAllocatorFree(void* ptr);
```

Using this API is fairly easy: Create an allocator and save its handle, then change your malloc/free calls to the new ones. I did a small conversion of libzip to use this API, which was mundane work. Let's go through some of the changes I made.
First, we need access to an allocator. I can see two practical ways of doing this
We can have a 'memoized' accessor, creating the allocator the first time it's called and otherwise accessing the already created allocator. This is used in `zip_util.c`:
```c
/* Zip allocator */
named_allocator_t zip_allocator = {.allocator_info_handle = -1 };
const char* allocator_name = "java.util.zip";
named_allocator_t allocator() {
  if (zip_allocator.allocator_info_handle == -1) {
    zip_allocator = JVM_MakeNamedAllocator(allocator_name);
  }
  return zip_allocator;
}
```

A second variant is to have a `init` function that is called before all other functions, this is used in `Deflater.c` and looks like this:
```c
named_allocator_t deflate_allocator;

Java_java_util_zip_Deflater_init(JNIEnv *env, jclass cls, jint level,
                                 jint strategy, jboolean nowrap)
{
    deflate_allocator = JVM_MakeNamedAllocator("java.util.zip.Deflater");
    // ...
```
The rest of the work is simply replacing your calls to malloc and friends to the appropriate function, passing in the `named_allocator_t` handle to each of the functions.

That's my proposal in a nutshell. For those who are familiar with NMT's internals, I intend to expand the memory tagging mechanism to store 4 bytes of tagging info per allocation, allowing us to have an essentially infinite amount of memory tags.

Regarding a future with more FFM and less C libraries: This proposal fits neatly into the Arena concept of FFM. One could imagine a future where Arenas can be given names, and these names are then used for accounting with a NamedAllocator in NMT We're not there yet, but the point is: Hey, this proposal doesn't rule out such a future and that's nice.

Thanks for reading, and I look forward to your responses.

All the best,
Johan Sjölén

[0]
The full prototype source code is available here: https://github.com/jdksjolen/jdk/tree/native-libs-nmt-take2
It does not integrate with the currently existing NMT and is not MT-safe. I wanted to see what the ergonomics of this API was, which is why I wrote it this way.
Note: This prototype refers to the new API allocators as `arena`s, I've chosen a different name in this text as to not overload the name with the Hotspot concept.

[1]
https://mail.openjdk.org/pipermail/core-libs-dev/2022-November/096197.html