RFR: 8276990: Memory leak in invoker.c fillInvokeRequest() during JDI operations [v4]

[Chris Plummer]

On Wed, 2 Feb 2022 14:11:45 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>>> In `invoker_completeInvokeRequest()` this appears to be the last reference:
>>> 
>>> ` jbyte returnType = methodSignature_returnTag(request->methodSignature);`
>>> 
>>> I would suggest freeing outside of the `if (!detached)` block and setting it to `NULL`. You might want to add an assert for `NULL` where you are currently freeing the pointer.
>> 
>> Alright, that seems sensible. Thank you!
>> I am not 100% if methodSignature can always be expected to be != NULL there, as I asserted. WDYT?
>
> We need to deallocate the methodSignature after deleteGlobalArgumentRefs() because that method accesses it. Or better yet, deallocate it there, because the only point of deleteGlobalArgumentRefs() seems to reset the methodSignature anyway. And that method seems to assume methodSignature != NULL, so we can do the same. WDYT? The change passes tier1 tests (including com/sun/jdi which exercises this code).

Are you still hitting the assert in fillInvokeRequest()? I'm not sure why it would ever not be NULL there. It's probably worth investigating some more. Otherwise you need to remove the assert, and possibly still have a leak.

-------------

PR: https://git.openjdk.java.net/jdk/pull/7306