error writing the shared archive file with large number of loaded classes

[Olivier Bourgain]

Hello,

I have been experimenting with Leyden for a pretty large Spring Boot
application (>1M loc, fat jar is about 240MB) and I think I hit a
limitation of the current implementation of  Leyden.

I got the following error message at the end of the training run:

[31,832s][error  ][cds,heap   ] [ 0] {0x000000060080be70}
jdk.internal.loader.ArchivedClassLoaders::appLoader (offset = 16)

[31,832s][error  ][cds,heap   ] [ 1] {0x0000000601400000}
jdk.internal.loader.ClassLoaders$AppClassLoader::parallelLockMap (offset =
36)

[31,832s][error  ][cds,heap   ] [ 2] {0x000000060095c5a8}
java.util.concurrent.ConcurrentHashMap

[31,832s][error  ][cds,heap   ] Cannot archive the sub-graph referenced
from [Ljava.util.concurrent.ConcurrentHashMap$Node; object
(0x000000060354bc88) size 262160, skipped.

[31,832s][error  ][cds    ] An error has occurred while writing the shared
archive file.

So it looks like some classloader has a very large ConcurrentHashMap that
ends up with a backing array of length 65536.

Tracing back to Leyden’s sources, I found that the the archived objects
size is limited

   -


   https://github.com/openjdk/leyden/blob/master/src/hotspot/share/cds/heapShared.cpp#L295
   -


   https://github.com/openjdk/leyden/blob/master/src/hotspot/share/cds/archiveHeapWriter.cpp#L118
   <https://github.com/openjdk/leyden/blob/master/src/hotspot/share/cds/archiveHeapWriter.cpp#L121>
   -


   https://github.com/openjdk/leyden/blob/83c7d3bbe860656ee403fa29df6a9e0aae962839/src/hotspot/share/cds/archiveHeapWriter.hpp#L118



By adding a --add-opens java.base/java.util=ALL-UNNAMED to the command
line, it disables the optimized module handling

[0.001s][info][cds] optimized module handling: disabled due to incompatible
property: jdk.module.addopens=java.base/java.util=ALL-UNNAMED

And the error goes away.

I tried to make my case work without adding some add-opens that are not
required by the app. As the limit in object size comes from
MIN_GC_REGION_ALIGNMENT, which is limited by Shenandoah’s region size, and
I don’t use that GC, I simply modified is_too_large_to_archive() to always
return false. Then the archiving with optimized module handling succeeds.

I created a Github repo to reproduce the issue, see the readme there:
https://github.com/obourgain/leyden-classloader

With regard to the improvements in startup time (on a M1 mac), here is what
is get:

   -

   base line, exploded Spring Boot jar = 11s
   -

   spring AOT = 10.5s
   -

   leyden with optimized module handling disabled = 7.8s
   -

   leyden with optimized module handling enabled = 7.1s
   -

   leyden with optimized module handling disabled and Spring AOT = 7.3s
   -

   leyden with optimized module handling enabled and Spring AOT = 6.3s


Overall, I think that’s pretty impressive improvements for this app’s
startup which was pretty efficient already. Moreover, I did that on a
project that started 12 or 13 years ago, so it shows that Leyden is
applicable for that kind of project with substantial gains.

I plan to try the premain branch soon.

Let me know if I can help with more information.

(this message was first submitted before I subscribed to the list. A copy
of it may be pending review by the admin. Feel free to delete the pending
copy)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/leyden-dev/attachments/20250416/f5c3023e/attachment.htm>