Metaspace leak with instrumentation.retransform

[Jean-Philippe Bempel]

Hi David,

On Wed, May 24, 2023 at 7:02 AM David Holmes <david.holmes at oracle.com> wrote:
>
> > class MyClass {
> >      private static void writeFile() {
>
> I don't see this ever getting called. Is some code missing?

No code is missing, code is not executed, The whole class is
re-transformed and just by the
presence of this method with this code in the classfile trigger the issue

>
> If commenting this removes the problem then it seems likely the actual
> Throwable being thrown as var3 is holding onto some references which
> keeps the old class alive. But as close() does nothing I can't see how
> it can throw anything. ??
>

Again, no code executed :)

> >              if (classToRetransform != null) {
> >                  try {
> >                      instrumentation.retransformClasses(classToRetransform);
>
> What ClassfileTransformer is actually in use here?

None :)

>
> It sounds to me like this isn't a leak as such but rather the exception
> keeps things alive. But we need the missing details.

I think Thomas get into something, but the only presence of the
bytecode in classfile (no execution)
is enough to fill metaspace while re-transforming the class even
without ClassFileTransformer registered.


Jean-Philippe Bempel