Possible memory leak in invoker_completeInvokeRequest()?

[Roman Kennke]

A user is reporting a memory leak starting from 
invoker_completeInvokeRequest(). Consider the trace below (relative to 
JDK11):

It looks like invoker_completeInvokeRequest() calls 
outStream_writeValue(), which calls outStream_writeObjectRef(), which 
then ends up calling commonRef_refToID(). There a new node is created 
via newCommonRef().

I am not sure about the intended lifecycle of the RefNode object in the 
hashtable. Is it cleaned when the backing Java object gets collected? Or 
does it have to be released explicitely via commonRef_release()?

(There is another leak in the same block of code in 
invoker_completeInvokeRequest(): the PacketOutputStream gets initialized 
via outStream_initReply(&out, id), but it seems like it's missing a 
corresponding outStream_destroy(&out). I'm working on a fix already:

https://github.com/rkennke/jdk11u-dev/tree/memleak-invoker2

I will do it vs jdk19, but I want to understand the other leak first, 
maybe I can group the fixes.)


[115200 bytes leaked]
je_prof_backtrace (/home/sandreev/git/misc/jemalloc/src/prof.c:636 
(discriminator 2))
je_malloc_default (/home/sandreev/git/misc/jemalloc/src/jemalloc.c:2289)
os::malloc (/tmp/jdk11u-dev/./src/hotspot/share/runtime/os.cpp:682)
JvmtiEnv::Allocate 
(/tmp/jdk11u-dev/./src/hotspot/share/prims/jvmtiEnv.cpp:127)
jvmti_Allocate 
(/tmp/jdk11u-dev/./build/linux-x86_64-normal-server-release/hotspot/variant-server/gensrc/jvmtifiles/jvmtiEnter.cpp:343)
jvmtiAllocate 
(/tmp/jdk11u-dev/./src/jdk.jdwp.agent/share/native/libjdwp/util.c:1808)
commonRef_refToID 
(/tmp/jdk11u-dev/./src/jdk.jdwp.agent/share/native/libjdwp/commonRef.c:428)
outStream_writeObjectRef.part.2 
(/tmp/jdk11u-dev/./src/jdk.jdwp.agent/share/native/libjdwp/outStream.c:219)
outStream_writeValue 
(/tmp/jdk11u-dev/./src/jdk.jdwp.agent/share/native/libjdwp/outStream.c:344)
invoker_completeInvokeRequest 
(/tmp/jdk11u-dev/./src/jdk.jdwp.agent/share/native/libjdwp/invoker.c:839)
commandLoop 
(/tmp/jdk11u-dev/./src/jdk.jdwp.agent/share/native/libjdwp/eventHelper.c:713)
JvmtiAgentThread::start_function_wrapper 
(/tmp/jdk11u-dev/./src/hotspot/share/prims/jvmtiImpl.cpp:80)
JavaThread::thread_main_inner 
(/tmp/jdk11u-dev/./src/hotspot/share/runtime/thread.cpp:1869)
Thread::call_run 
(/tmp/jdk11u-dev/./src/hotspot/share/runtime/thread.cpp:387)
thread_native_entry 
(/tmp/jdk11u-dev/./src/hotspot/os/linux/os_linux.cpp:799)
start_thread (/usr/src/debug/glibc-2.17-c758a686/nptl/pthread_create.c:307)
?? 
(/usr/src/debug////////glibc-2.17-c758a686/misc/../sysdeps/unix/sysv/linux/x86_64/clone.S:111)