Metaspace leak with instrumentation.retransform

[Thomas Stüfe]

On Wed, May 24, 2023 at 9:08 AM David Holmes <david.holmes at oracle.com>
wrote:

> Hi,
>
> On 24/05/2023 4:45 pm, Jean-Philippe Bempel wrote:
> > Hi David,
> >
> > On Wed, May 24, 2023 at 7:02 AM David Holmes <david.holmes at oracle.com>
> wrote:
> >>
> >>> class MyClass {
> >>>       private static void writeFile() {
> >>
> >> I don't see this ever getting called. Is some code missing?
> >
> > No code is missing, code is not executed, The whole class is
> > re-transformed and just by the
> > presence of this method with this code in the classfile trigger the issue
>
> Ah now I get it. So basically you're filling up metaspace faster than it
> can be reclaimed. Other than class unloading I'm not sure what the
> reclamation mechanism is, so I'll defer to Thomas at this point. We
> definitely need to see more information such as logging or the jcmd
> output Thomas suggested.
>
> Cheers,
> David
> -----
>

The space for the old bytecode gets added to an arena-internal structure,
and we attempt to reuse those pieces of memory for allocations within the
same arena.

If all classes are loaded, and all we do is transform them, the only chance
of re-using those pieces are the next redefinitions - space allocated for
new bytecode. So, when retransforming class B we allocate space for
bytecode class B version 2 and will attempt to reuse space from class A
version 1 that we retransformed before.

But if the new byte codes are consistently larger than the old ones, this
may not often work. This may just be an obnoxious usage pattern.

I don't know if, at redefinition time, we are smart enough to reuse space
for bytecodes if the new space requirement is <= old space size. That would
be a simple first improvement.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/hotspot-dev/attachments/20230524/62a4ca85/attachment.htm>