Metaspace leak with instrumentation.retransform

[David Holmes]

Hi,

On 24/05/2023 4:45 pm, Jean-Philippe Bempel wrote:
> Hi David,
> 
> On Wed, May 24, 2023 at 7:02 AM David Holmes <david.holmes at oracle.com> wrote:
>>
>>> class MyClass {
>>>       private static void writeFile() {
>>
>> I don't see this ever getting called. Is some code missing?
> 
> No code is missing, code is not executed, The whole class is
> re-transformed and just by the
> presence of this method with this code in the classfile trigger the issue

Ah now I get it. So basically you're filling up metaspace faster than it 
can be reclaimed. Other than class unloading I'm not sure what the 
reclamation mechanism is, so I'll defer to Thomas at this point. We 
definitely need to see more information such as logging or the jcmd 
output Thomas suggested.

Cheers,
David
-----


>>
>> If commenting this removes the problem then it seems likely the actual
>> Throwable being thrown as var3 is holding onto some references which
>> keeps the old class alive. But as close() does nothing I can't see how
>> it can throw anything. ??
>>
> 
> Again, no code executed :)
> 
>>>               if (classToRetransform != null) {
>>>                   try {
>>>                       instrumentation.retransformClasses(classToRetransform);
>>
>> What ClassfileTransformer is actually in use here?
> 
> None :)
> 
>>
>> It sounds to me like this isn't a leak as such but rather the exception
>> keeps things alive. But we need the missing details.
> 
> I think Thomas get into something, but the only presence of the
> bytecode in classfile (no execution)
> is enough to fill metaspace while re-transforming the class even
> without ClassFileTransformer registered.
> 
> 
> Jean-Philippe Bempel