Creating snapshots with dynamically loaded JARs

[Nikola Grcevski]

Thanks Dan, I’ll give that a shot. We’ll need the classloader after restore, but I can recreate it.
________________________________
From: Dan Heidinga <heidinga at redhat.com>
Sent: Friday, December 2, 2022 2:53:20 PM
To: Nikola Grcevski <grcevski at gmail.com>
Cc: crac-dev at openjdk.org <crac-dev at openjdk.org>
Subject: Re: Creating snapshots with dynamically loaded JARs

Do you need the classloader to be able to load new classes after the checkpoint is restored?  If not, there's a "close()" method on URLClassLoader that should close the jar and file URLs and "makes a best effort attempt to close all opened files".

Obviously not a complete solution but it may work around your issue for now.

--Dan

On Fri, Dec 2, 2022 at 2:30 PM Nikola Grcevski <grcevski at gmail.com<mailto:grcevski at gmail.com>> wrote:
Hello crac-dev,

While experimenting more with CRaC and Elasticsearch I encountered one
more issue and I thought I should bring it up for discussion.

We load some JARs dynamically and these don't get marked as 'persistent'.

For example:

final ClassLoader classLoader = buildClassLoader();
try {
   final Class<? extends Supplier<?>> cls = (Class<? extends
Supplier<?>>)classLoader.loadClass("org.elasticsearch.io.ansi.AnsiConsoleLoader");
...
} catch (ReflectiveOperationException e) {
   throw new RuntimeException("Failed to load ANSI console", e);
}

private static ClassLoader buildClassLoader() {
   final Path libDir = Path.of(".").resolve("lib").resolve("ansi-console");

   try {
      final URL[] urls = Files.list(libDir)
                                  .filter(each ->
each.getFileName().toString().endsWith(".jar"))
                                   .map(SimpleModule::pathToURL)
                                   .toArray(URL[]::new);

       return URLClassLoader.newInstance(urls,
SimpleModule.class.getClassLoader());
    } catch (IOException e) {
       throw new RuntimeException("Failed to list jars in [" + libDir
+ "]: " + e.getMessage(), e);
    }
}

I think typically we'd say that these should be closed and recreated
on snapshot and restore,
but it's not easy to do with ClassLoaders. I think we'd need to lose
the reference and have
GC collect the class loader for all the files to close.

I was able to work around this issue by putting all the jars we might
load dynamically
on the class path with -cp.

I'm wondering if the work around is the correct 'fix' or is there something
else that should be done? If the URLs are files, I suppose we can mark those as
persistent?

I made a GitHub repo with all the code that shows the issue here:
https://github.com/grcevski/openjdk_CRaC_simplemodule

Thanks,
Nikola

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/crac-dev/attachments/20221202/28f0c779/attachment-0001.htm>