Reducing Classloader's parallelLockMap memory consumption proposal

[Dmytro Ukhlov]

Hello Alan,
Thank you for the useful information.

Jenkins's class loader topology is hierarchical. I believe Jenkins can
make its internal class loader implementations non-parallel capable.
However, the problem persists because Jenkins pipelines are built on
top of Groovy scripts. Running a pipeline requires the compilation and
execution of Groovy scripts at runtime.

When loading these dynamically generated classes (with unique names),
the GroovyClassLoader first tries to load them from the parent class
loader. As a result, a large number of lock objects are created
throughout the class loader chain, all the way down to the
PlatformClassLoader, which is parallel capable and cannot be modified.

Additionally, Groovy performs class lookups via class loading. For
example, if we have MyObject.name in a script, Groovy will attempt to
load MyObject, java.lang.MyObject, java.util.MyObject,
groovy.lang.MyObject, and so on.

To summarize:
The existing JDK java.lang.ClassLoader works well for traditional Java
applications, but it causes memory leaks when using dynamic class
loaders that load short-lived classes. It seems that we need a way to
determine whether a class can be loaded from the parent class loader
chain, and avoid calling parent.loadClass() if unnecessary - so as not
to accumulate entries in parallelLockMap.

If anyone has experience solving such issues, I would greatly
appreciate it if you could share your insights.

>
> On 20/05/2025 03:42, Dmytro Ukhlov wrote:
> > Hello!
> >
> > I created PR is scope of jenkins-core project:
> > https://github.com/jenkinsci/jenkins/pull/10659
> > Jira ticket: https://issues.jenkins.io/browse/JENKINS-75675
> >
> > In this PR i propped to override
> > protected Object getClassLoadingLock(String className)
> > method and use weak references for lock objects
> >
> > Jenkins is a plugable platform, each plugin has its own class loader
> > and its parallelLockMap may consume 10mb of RAM. As a result it might
> > have 2gb overhead if ~200 plugins installed.
> >
> > Jenkins maintainers ask me to consider making this improvement in base
>
> This is an issue for "parallel capable" class loaders, esp. those that
> don't directly delegate. I don't know what topology is used in the
> Jenkins plugin class loaders and whether these class loader need to
> parallel capable.
>
> Expunging entries isn't really feasible, at least not without changing
> the spec.
>
> JDK-8005233 [1] has a write-up from David Holmes on this topic and a
> prototype for "fully concurrent" class loaders. This work has been
> parked for several years. Maybe some day it will get dusted off and a
> JEP drafted (due to the significance).
>
> -Alan
>
> [1] https://bugs.openjdk.org/browse/JDK-8005233