Metaspace leak with instrumentation.retransform

[David Holmes]

On 24/05/2023 5:08 pm, David Holmes wrote:
> Hi,
> 
> On 24/05/2023 4:45 pm, Jean-Philippe Bempel wrote:
>> Hi David,
>>
>> On Wed, May 24, 2023 at 7:02 AM David Holmes <david.holmes at oracle.com> 
>> wrote:
>>>
>>>> class MyClass {
>>>>       private static void writeFile() {
>>>
>>> I don't see this ever getting called. Is some code missing?
>>
>> No code is missing, code is not executed, The whole class is
>> re-transformed and just by the
>> presence of this method with this code in the classfile trigger the issue
> 
> Ah now I get it. So basically you're filling up metaspace faster than it 
> can be reclaimed. Other than class unloading I'm not sure what the 
> reclamation mechanism is, so I'll defer to Thomas at this point. We 
> definitely need to see more information such as logging or the jcmd 
> output Thomas suggested.

That said, I'm at a loss to understand what affect the addSuppressed 
could possibly have here.

David

> Cheers,
> David
> -----
> 
> 
>>>
>>> If commenting this removes the problem then it seems likely the actual
>>> Throwable being thrown as var3 is holding onto some references which
>>> keeps the old class alive. But as close() does nothing I can't see how
>>> it can throw anything. ??
>>>
>>
>> Again, no code executed :)
>>
>>>>               if (classToRetransform != null) {
>>>>                   try {
>>>>                       
>>>> instrumentation.retransformClasses(classToRetransform);
>>>
>>> What ClassfileTransformer is actually in use here?
>>
>> None :)
>>
>>>
>>> It sounds to me like this isn't a leak as such but rather the exception
>>> keeps things alive. But we need the missing details.
>>
>> I think Thomas get into something, but the only presence of the
>> bytecode in classfile (no execution)
>> is enough to fill metaspace while re-transforming the class even
>> without ClassFileTransformer registered.
>>
>>
>> Jean-Philippe Bempel