Performance regression in BuiltinClassLoader?

[Eirik Bjørsnøs]

For reference, here's the EnterpriseClassloader benchmark I made.

It assumes you have the jar files from Spring Petclinic in BOOT-INF/lib
(but any large collection of jars should work).

public class ZIPBenchmark {

    public static void main(String[] args) {

        final File libDir = new File("BOOT-INF/lib/");

        final Strategy strategy = Strategy.valueOf(args[0]);

        EnterpriseClassLoader classLoader =
EnterpriseClassLoader.create(libDir, strategy);

        final List<String> names = classLoader.getNames();


        long gcTimeBefore = 0;
        long gcCountBefore = 0;
        for (GarbageCollectorMXBean bean :
ManagementFactory.getGarbageCollectorMXBeans()) {
            gcCountBefore += bean.getCollectionCount();
            gcTimeBefore += bean.getCollectionTime();
        }


        long before = System.nanoTime();



        for (String name : names) {
            classLoader.getResource(name);
        }

        long gcTimeAfter  = 0;
        long gcCountAfter = 0;
        for (GarbageCollectorMXBean bean :
ManagementFactory.getGarbageCollectorMXBeans()) {
            gcCountAfter += bean.getCollectionCount();
            gcTimeAfter += bean.getCollectionTime();
        }

        final long time = System.nanoTime() - before;

        System.out.printf("%s: took %s ms\n", strategy,
TimeUnit.NANOSECONDS.toMillis(time));
        System.out.printf("Collections: %s\n", gcCountAfter-gcCountBefore);
        System.out.printf("Collection time: %s ms\n",
gcTimeAfter-gcTimeBefore);

    }



    public enum Strategy {
        SELF_ONLY, BOOTSTRAP, PLATFORM, SYSTEM, PARENT_DELEGATION
    }


    private static class EnterpriseClassLoader extends URLClassLoader {
        private final List<String> names;

        private final Strategy lookupStrategy;

        public EnterpriseClassLoader(URL[] urls, List<String> names,
Strategy strategy) {
            super(urls, strategy == Strategy.BOOTSTRAP ? null :
ClassLoader.getSystemClassLoader());
            this.names = names;
            this.lookupStrategy = strategy;
        }

        public static EnterpriseClassLoader create(File libDir, Strategy
strategy) {
            final File[] jarFiles = libDir.listFiles(f ->
f.getName().endsWith(".jar"));

            final URL[] urls = Stream.of(jarFiles)
                    .map(File::toURI).map(EnterpriseClassLoader::toURL)
                    .toArray(URL[]::new);

            List<String> names = Stream.of(jarFiles)
                    .flatMap(EnterpriseClassLoader::entryNameStream)
                    .collect(Collectors.toList());

            return new EnterpriseClassLoader(urls, names, strategy);
        }

        @Override
        public URL getResource(String name) {
            if(lookupStrategy == Strategy.BOOTSTRAP) {
                return super.getResource(name);
            } else if (lookupStrategy == Strategy.PLATFORM) {
                return
ClassLoader.getPlatformClassLoader().getResource(name);
            } else if (lookupStrategy == Strategy.SYSTEM) {
                return ClassLoader.getSystemClassLoader().getResource(name);
            } else if(lookupStrategy == Strategy.PARENT_DELEGATION) {
                return super.getResource(name);
            } else if(lookupStrategy == Strategy.SELF_ONLY) {
                return super.findResource(name);
            } else {
                throw new IllegalStateException("Unknown strategy");
            }
        }

        @Override
        public URL findResource(String name) {
            if(lookupStrategy == Strategy.BOOTSTRAP) {
                return null;
            } else {
                return super.findResource(name);
            }
        }

        private static Stream<String> entryNameStream(File url) {
            List<String> names;
            try (JarFile file = new JarFile(url)) {

                names = file.stream()
                        .map(ZipEntry::getName)
                        .filter(name -> name.endsWith(".class"))
                        .collect(Collectors.toList());

            } catch (IOException e) {
                throw new RuntimeException(e);
            }
            return names.stream();
        }

        private static URL toURL(URI uri) {
            try {
                return uri.toURL();
            } catch (MalformedURLException e) {
                throw new RuntimeException(e);
            }
        }
        public List<String> getNames() {
            return names;
        }

    }


}

On Mon, Jan 18, 2021 at 7:34 PM Eirik Bjørsnøs <eirbjo at gmail.com> wrote:

>
> Hello,
>
> I've been looking into ZipFile again with the aim to speed up typical
> enterprise classloading / resource lookups.
>
> To test the performance impact of my changes, I made a benchmark which
> creates an EnterpriseClassLoader (subclass of URLClassLoader) with the jar
> files in Spring Petclinic (89 jars with ~30k entries total).
>
> (This is intentionally implemented using a plain main method with no JMH
> or warmup since I want to measure a representative cold application startup)
>
> The benchmark measures the time needed to call getResource 30K times, once
> for each entry name.
>
> Using 15.0.1, this completes in ~1700 ms
> Using 17 master, it completes in 12851 ms
>
> If I run the benchmark from the classpath instead of as a module, the
> result is even worse:
>
> 15: 1724 ms
> 17: 21885 ms
>
> If also measure GC collections and times, and get the following:
>
> 15:
> Collections: 4
> Collection time: 12 ms
>
> 17:
> Collections: 24
> Collection time: 44 ms
>
> (However, a JHM with -prof gc indicates that both 17 and 15 allocate
> ~5.6Kb/getResource after warmup, so not convinced this needs to be
> allocation related)
>
> The EnterpriseClassLoader is a bit clever in that it can perform lookups
> using different strategies: Regular parent-first, self-only, or use the
> bootstrap / platform system classloaders directly.
>
> Here are the results on 17 using each strategy (benchmark on class path):
>
> PARENT_DELEGATION: 23111 ms
> SELF_ONLY: 428 ms
> BOOTSTRAP: 8131 ms
> PLATFORM: 15149 ms
> SYSTEM: 20628 ms
>
> Here are the same results on 15:
>
> PARENT_DELEGATION: 1691 ms
> SELF_ONLY: 393 ms
> BOOTSTRAP: 579 ms
> PLATFORM: 908 ms
> SYSTEM: 1368 ms
>
> Note that the delegation chain is Enterprise -> System -> Platform ->
> Bootstrap.
>
> Interesting to note that URLClassLoader itself does not seem to regress
> much when only looking up its own resources, while delegating up the chain
> does.
>
> Has there been any significant work in class loading / resource lookup
> since 15.0.1 that could help explain this regression?
>
> Eirik.
>
>
>