Extend Native Memory Tracking over the JDK ? (was: Proposal: track zlib native memory usage with NMT)

Thu Dec 1 18:14:52 UTC 2022

Are there any opinions about whether or not to extend NMT across the JDK?

This blocks https://bugs.openjdk.org/browse/JDK-8296360, and I had a PR
prepared as https://github.com/openjdk/jdk/pull/10988. Originally I was
hoping to get this into JDK 20, but I don't think that is realistic
anymore. I am fine with postponing my work in favor of a baseline
discussion, but so far there is very little discussion about this topic.

How should I proceed?

Thanks, Thomas

On Wed, Nov 9, 2022 at 8:12 AM Thomas Stüfe <thomas.stuefe at gmail.com> wrote:

> Hi Alan,
>
> (replaced hotspot-runtime-dev with hotspot-dev, since its more of a
> general topic)
>
> thank you for your time!
>
> I am very happy to talk this through. I think native memory observability
> in the JDK (and customer code!) is sorely lacking. Witness the countless
> "where did my native memory go" blog articles. At SAP we have been
> struggling with this topic for a long time and have come up with a mixture
> of solutions. The aforementioned tracker was one, which extended our
> version of NMT across the JDK. Our SapMachine MallocTracer, which allows us
> to trace uninstrumented customer code, another. We even experimented with
> exchanging the allocator (using jemalloc) to gain insights. But that is a
> whole different topic with deep logistical implications, I don't want to
> touch it here. Exchanging the allocator does not help to observe virtual
> memory or the brk segment, of course.
>
> And to make the picture complete, another insight we currently lack is the
> implicit allocator overhead, which can be very significant and is hidden by
> the libc. We also have observability for that in the SapMachine, and I miss
> it in OpenJDK.
>
> As you noticed, my original intent was just to instrument Zlib and
> possibly improve tracking for DBBs. Although, thinking beyond that, another
> attractive instrumentation target would be mapped NIO buffers at least.
>
> So I think native memory observability is important. Arguably we could
> even extend observability to cover other OS resources, e.g. file handles.
> If we shift code around, to java/Panama: data that move the java heap does
> not need to be tracked, but other memory will always come from one of the
> basic system APIs, regardless of who allocates it and where in the stack
> allocation happens. Be it native JDK code, Panama, or even customer JNI
> code.
>
> If we agree on the importance of native memory observability, then I
> believe NMT is the right tool for it. It is a good tool. The machinery is
> already there. It covers both C-heap and virtual memory APIs, as well as
> thread stacks, and could easily be extended to cover sbrk if needed. And I
> assume that whatever shape OpenJDK takes on in the future, there always
> will be a libjvm.so at its core, so we will always have it. But even if
> not, NMT could be separated from libjvm.so quite easily, since it has no
> deep ties with the JVM.
>
> About coupling JVM with outside code: We don't have to directly link
> against libjvm.so. We can keep things loose if the intent is to be runnable
> without a JVM, or be JVM-version-agnostic. That could take the form of a
> function-pointer interface like JVMTI. Or outside code could dynamically
> dlsym the JVM allocation hooks. In any case gracefully falling back to
> system allocation routines when necessary.
>
> And I agree, polluting the NMT tag space with outside meaning is ugly. I
> only did it because I planned to go no further than instrumenting Zlib and
> possibly DBBs. But if we take this further, my preferred solution would be
> a reserved tag range or -ranges for outside use, whose inner meaning would
> be opaque to the JVM. Kind of like SIGRTMIN+SIGRTMAX. Then, outside code
> could register tags and their meta information with the JVM, or we find a
> different way to convey the tag meaning to NMT (config files, or
> callbacks). That could even be opened up for customer use.
>
> This also touches on another question, that of NMT tag space. NMT tags are
> very useful since they allow cheap tracking without capturing call stacks.
> However, tags are underused and show growing pains since they are too
> one-dimensional and restrictive. We had competing interests in the past
> about tag granularity. It is all over the place. We have coarse-grained
> tags like "mtThread", and very fine-grained ones like "mtObjectMonitor".
> There are several ways we could improve, e.g., by making them combinable
> like UL does, or allowing for a hierarchy of them - either a hard-wired
> limited one like "domain"+"tag", or an unlimited tree-like one. Technically
> interesting since whatever the new encoding is, they still must fit into a
> malloc header. I opened https://bugs.openjdk.org/browse/JDK-8281819 to
> track ideas like these.
>
> Instrumenting Panama allocations, including the ability to tag
> allocations, would be a very good idea. For instance, if we ever remove the
> native Zlib layer and convert it to java using Panama, we can do the same
> with Panama I do now natively - use the Zlib zalloc interface to hook in
> JVM memory allocation functions. The result could be completely identical,
> and the end user looking at the NMT output need never know that anything
> changed.
>
> And that goes for all instrumentation - if today we add it to JNI code,
> and that code gets removed tomorrow, we can add it to Panama code too.
> Unless data structures move to the heap, in which case there is no need to
> track them.
>
> You mentioned that NMT was more of an in-house support tool. Our
> experience is different. Even though it was positioned as a tool for JVM
> developers, and we never cared for the backward compatibility or
> consistency, it gets used a *lot* by our customers. We have to explain its
> output frequently. Also, many blog articles exist documenting its use. So,
> maybe it would be okay to elevate it to a user-facing tool since it seems
> to occupy that role anyway. We may also open up consumption of NMT results
> via java APIs, or expose its results via MXBeans.
>
> If this is to be a JEP, okay, but I'm afraid it would stall things a bit.
> I am interested in getting a simpler and quicker solution for older support
> releases at least, possibly based on my PR. I know that would be
> unconventional though.
>
> Thank you,
>
> Thomas
>
>
> On Sun, Nov 6, 2022 at 9:31 AM Alan Bateman <Alan.Bateman at oracle.com>
> wrote:
>
>> On 04/11/2022 16:54, Thomas Stüfe wrote:
>> > Hi all,
>> >
>> > I am currently working on https://bugs.openjdk.org/browse/JDK-8296360;
>> > I was preparing the final PR [1], but then Alan did ask me to discuss
>> > this on core-libs first.
>> >
>> > Backstory:
>> >
>> > NMT tracks hotspot native allocations but does not cover the JDK
>> > libraries (small exception: Unsafe.AllocateMemory). However, the
>> > native memory footprint of JDK libraries can be significant. We have
>> > no in-VM tracker for these and need tools like valgrind or our
>> > SapMachine MallocTracer [2] to observe them.
>>
>> Thanks for starting a discussion on this as this is a topic that
>> requires agreement from several areas. If this is the start of something
>> bigger, where you want to have all allocation sites in the libraries
>> using NMT, then I think it needs a write-up, maybe a JEP.
>>
>> For starters, I think it needs some agreement on using NMT for memory
>> allocated outside of libjvm. You mentioned Unsafe as an exception but
>> that is implemented in the VM so you get tracking for free, albeit I
>> think all allocations are in the "mtOther" category.
>>
>> A general concern is that it creates more coupling between the VM code
>> and the libraries code. As you probably know, we've removed most of the
>> dependences on JVM_* functions from non-core areas over many years. So I
>> think that needs consideration as I assume we don't want
>> memory/allocation.hpp declaring a dozen catagories for allocations done
>> in say java.desktop module for example. Maybe your proposal will be
>> strictly limited to java.base but even then, do we really want the VM
>> even knowing about categories that are specific to zip compression or
>> decompression?
>>
>> There are probably longer term trends that should be part of the
>> discussion too. One general trend is that "run time" is becoming more
>> and more a hybrid of code in libvm and the Java libraries. Lambdas,
>> module system, virtual threads implementations are a few examples in the
>> last few release. This comes with many "Java on Java" challenges,
>> including serviceability where users of the platform will expect tools
>> to just work and won't care where the code is. NMT is probably more for
>> support teams and not something that most developers will ever use but I
>> think is part of the challenge of having serviceability solutions "just
>> work".
>>
>> In addition to having more of the Java runtime written in Java, there
>> will likely be less JNI code in the future. It's very possible that the
>> JNI code (including the JNI methods in libzip) will be replaced with
>> code that uses Panama memory and linker APIs once they are become
>> permanent. The effect of that would to have a lot of the memory
>> allocations be tracked in the mtOther category again. Maybe integration
>> with memory tracking should be looked at in conjunction with these APIs
>> and this migration. I could imagine the proposed "Arena" API
>> (MemorySession in Java 19) having some integration with NMT and it might
>> be interesting to look into that.
>>
>> So yes, this topic does need broader discussion and it might be a bit
>> premature to start with a PR for libzip without talking about the bigger
>> picture first.
>>
>> -Alan
>>
>>
>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/core-libs-dev/attachments/20221201/11472e41/attachment.htm>