[8u] RFR backport of JDK-8198423: Improve metaspace chunk allocation

[Thomas Stüfe]

So, you already backported JDK-8198423 inhouse and got it working?

--

About your scenario, I remember that Zhengyu Gu added a small patch for JDK
10 reducing the chunk size for anonymous classes from 4K to 1K (
https://bugs.openjdk.java.net/browse/JDK-8190729).

That one would be easy to downport to jdk8 and might alleviate the pain a
little.

Cheers, Thomas


On Tue, Sep 24, 2019 at 3:39 PM Denghui Dong <denghui.ddh at alibaba-inc.com>
wrote:

> Hi,
>    In fact, i have found some application in our production environment
> occurred the problem mentioned in my first email,
> What they have in common is that they use a lot of small groovy or
> scripts., with a lot of classes loading and unloading, there are
> many 4k(Small for no-klass) chunks in used or in freelist. So we backport
> JDK-8198423 to our in-house jdk, and it produced some good results.
>   As Andrew said, we can increase the MaxMetaspaceSize or upgrad to
> jdk11u, but it is difficult to do these changes for some reasons.
>   Finally, i quite agree with your point of view.
>
> Thanks,
> Denghui Dong
> ------------------------------------------------------------------
> From:Thomas Stüfe <thomas.stuefe at gmail.com>
> Send Time:2019年9月24日(星期二) 20:40
> To:"Langer, Christoph" <christoph.langer at sap.com>
> Cc:Andrew Dinn <adinn at redhat.com>; 董登辉(卓昂) <denghui.ddh at alibaba-inc.com>;
> jdk8u-dev at openjdk.java.net <jdk8u-dev at openjdk.java.net>; Andrew Haley <
> aph at redhat.com>; Andrew John Hughes <gnu.andrew at redhat.com>
> Subject:Re: [8u] RFR backport of JDK-8198423: Improve metaspace chunk
> allocation
>
> I agree with Andrew that backporting JDK-8198423 is a large effort and a
> bit risky. If that has not been clear from my first mail I recommend
> against it.
>
> As for JDK-8221173, lets see how that pans out. I plan to bring it at
> least to SapMachine11, upstream 11 if it makes sense. But this is still in
> the future.
>
> Cheers, Thomas
>
>
> On Tue, Sep 24, 2019 at 2:22 PM Langer, Christoph <
> christoph.langer at sap.com> wrote:
> Hi,
>
>  from a technical perspective, Thomas Stuefe is probably the right contact
> for this backport. And I see, he has already commented on it.
>
>  Whether it is appropriate for OpenJDK 8u and can make it there, you'll
> probably have to convince Andrew Haley and Andrew Hughes. I'm not a
> maintainer for 8u.
>
>  Best regards
>  Christoph
>
>  > -----Original Message-----
>  > From: Andrew Dinn <adinn at redhat.com>
>  > Sent: Dienstag, 24. September 2019 11:56
>  > To: Denghui Dong <denghui.ddh at alibaba-inc.com>; jdk8u-
>  > dev at openjdk.java.net; Andrew Haley <aph at redhat.com>; Langer,
>  > Christoph <christoph.langer at sap.com>
>  > Subject: Re: [8u] RFR backport of JDK-8198423: Improve metaspace chunk
>  > allocation
>  >
>  > On 23/09/2019 10:49, Denghui Dong wrote:
>  > > Hi all,
>  > >   I'd like to request a backport of JDK-8198423.
>  > >   I found some long-time running application will oom caused by
> metaspace
>  > if
>  > > the application loads / unloads classes (by different classloaders)
>  > frequently,
>  > > even if there are many free chunks in freelist.
>  > > . . .
>  > >   This patch doesn't apply cleanly to 8u-dev, if you think this
> backport is
>  > reasonable,
>  > > i will provide my webrev.
>  > >   What's your comments?
>  > I think backporting this patch is probably a bad idea. It is a large and
>  > complex change to a critical part of the JVM runtime so the risk
>  > involved is quite high.
>  >
>  > Likewise I would expect that backporting whatever eventually gets done
>  > under JDK-8221173 is going to be equally risky.
>  >
>  > It might help if you could give an idea of how much of a problem this
>  > is. I can see how very long-running apps might suffer from fragmentation
>  > of metadata space without the patch. Is this something you are seeing
>  > regularly or intermittently?
>  >
>  > I would expect that allocating more memory to the JVM can be used as a
>  > workaround to delay the onset/lower the frequency of this problem. Is
>  > there a reason why that approach is not available to you?
>  >
>  > And, of course, there is always the option of upgrading to jdk11u.
>  > Eventually, that is going to be needed to resolve more and more of the
>  > problems remaining in jdk8u.
>  >
>  > Perhaps Andrew Haley or Christoph Langer might be able to comment on
>  > this?
>  >
>  > regards,
>  >
>  >
>  > Andrew Dinn
>  > -----------
>  > Senior Principal Software Engineer
>  > Red Hat UK Ltd
>  > Registered in England and Wales under Company Registration No. 03798903
>  > Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander
>