ClassLoaderStats VM operation can be quadratic in some cases

[Jean-Philippe BEMPEL]

Hello all,

We have a customer complaining about long VM operations especially
ClassLoaderStats using JFR. This operation can last several seconds!

Looking into the recording of this application, we realize that there is
40K+ ClassLoaders, which is unusual, maybe a classloader leak, but this is
not the point, as without JFR the application is running fine.

I can reproduce the issue with this simple class:

package com.bempel.sandbox;

import java.net.URL;
import java.net.URLClassLoader;
import java.util.ArrayList;
import java.util.List;

public class JFRClassLoaderStatsIssue {

    private static List<ClassLoader> classLoaderList = new ArrayList<>();

    public static void main(String[] args) throws Exception {
        URL location =
JFRClassLoaderStatsIssue.class.getProtectionDomain().getCodeSource().getLocation();
        for (int i = 0; i < 40_000; i++) {
            URLClassLoader classLoader = new URLClassLoader(new
URL[]{location});
            Class.forName(JFRClassLoaderStatsIssue.class.getCanonicalName(),
true, classLoader);
            classLoaderList.add(classLoader);
        }
    }
}

Running with on JDK11 with the following command line:

java -Xlog:safepoint
-XX:StartFlightRecording=dumponexit=true,filename=issue.jfr
-XX:-UseBiasedLocking -cp build/classes/java/main
com.bempel.sandbox.JFRClassLoaderStatsIssue

output:
[...]
[0.861s][info][safepoint] Entering safepoint region:
ClassLoaderStatsOperation
[6.593s][info][safepoint] Leaving safepoint region
[6.593s][info][safepoint] Total time for which application threads were
stopped: 5.7324145 seconds, Stopping threads took: 0.0000127 seconds
[...]

Looking into ClassLoaderStatsClosure::do_cld method [1] we can see that
each classloader is inserted into a hashtable for the duration of the VM
operation.
but this hashtable is fixed in size to 256 buckets. At some point, lookups
and insertions become linear (too many collisions). Those lookups and
insertions are done for each Classloader => O(n^2)

With a small number of classloaders it's fine, but with 40K it's not ok :)
My thought about this issue would be to cap the number of classloaders
inserted into the hashmap based on _total_loaders field counter (which BTW
seems not used at all).

Thanks for looking into it.

Kind Regards,
Jean-Philippe Bempel
[1]
https://github.com/openjdk/jdk/blob/master/src/hotspot/share/classfile/classLoaderStats.cpp#L48