Creating snapshots with dynamically loaded JARs

[Dan Heidinga]

Do you need the classloader to be able to load new classes after the
checkpoint is restored?  If not, there's a "close()" method on
URLClassLoader that should close the jar and file URLs and "makes a best
effort attempt to close all opened files".

Obviously not a complete solution but it may work around your issue for now.

--Dan

On Fri, Dec 2, 2022 at 2:30 PM Nikola Grcevski <grcevski at gmail.com> wrote:

> Hello crac-dev,
>
> While experimenting more with CRaC and Elasticsearch I encountered one
> more issue and I thought I should bring it up for discussion.
>
> We load some JARs dynamically and these don't get marked as 'persistent'.
>
> For example:
>
> final ClassLoader classLoader = buildClassLoader();
> try {
>    final Class<? extends Supplier<?>> cls = (Class<? extends
>
> Supplier<?>>)classLoader.loadClass("org.elasticsearch.io.ansi.AnsiConsoleLoader");
> ...
> } catch (ReflectiveOperationException e) {
>    throw new RuntimeException("Failed to load ANSI console", e);
> }
>
> private static ClassLoader buildClassLoader() {
>    final Path libDir = Path.of(".").resolve("lib").resolve("ansi-console");
>
>    try {
>       final URL[] urls = Files.list(libDir)
>                                   .filter(each ->
> each.getFileName().toString().endsWith(".jar"))
>                                    .map(SimpleModule::pathToURL)
>                                    .toArray(URL[]::new);
>
>        return URLClassLoader.newInstance(urls,
> SimpleModule.class.getClassLoader());
>     } catch (IOException e) {
>        throw new RuntimeException("Failed to list jars in [" + libDir
> + "]: " + e.getMessage(), e);
>     }
> }
>
> I think typically we'd say that these should be closed and recreated
> on snapshot and restore,
> but it's not easy to do with ClassLoaders. I think we'd need to lose
> the reference and have
> GC collect the class loader for all the files to close.
>
> I was able to work around this issue by putting all the jars we might
> load dynamically
> on the class path with -cp.
>
> I'm wondering if the work around is the correct 'fix' or is there something
> else that should be done? If the URLs are files, I suppose we can mark
> those as
> persistent?
>
> I made a GitHub repo with all the code that shows the issue here:
> https://github.com/grcevski/openjdk_CRaC_simplemodule
>
> Thanks,
> Nikola
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/crac-dev/attachments/20221202/207784f7/attachment.htm>