error writing the shared archive file with large number of loaded classes

[Olivier Bourgain]

Hello,

I can confirm that this patch fixes the issue, thanks!

Le jeu. 17 avr. 2025 à 07:54, <ioi.lam at oracle.com> a écrit :

> Hi Olivier,
>
> Thanks for reporting the problem and sharing your benchmarking results.
> It's good to see that you can apply Leyden on such big projects and see
> substantial start-up benefits.
>
> We already have a bug opened for this issue:
>
> https://bugs.openjdk.org/browse/JDK-8342283
>
> I opened a PR a while back but got side tracked and the PR got closed:
>
> https://github.com/openjdk/jdk/pull/21797
>
> Could you try applying this PR and see if it works for your test case?
>
> I will revive the PR and hopefully be able to fix it in JDK 25, where the
> bug is currently targeted to.
>
> Thanks
>
> - Ioi
>
>
> On 4/16/25 1:56 AM, Olivier Bourgain wrote:
>
> Hello,
>
> I have been experimenting with Leyden for a pretty large Spring Boot
> application (>1M loc, fat jar is about 240MB) and I think I hit a
> limitation of the current implementation of  Leyden.
>
> I got the following error message at the end of the training run:
>
> [31,832s][error  ][cds,heap   ] [ 0] {0x000000060080be70}
> jdk.internal.loader.ArchivedClassLoaders::appLoader (offset = 16)
>
> [31,832s][error  ][cds,heap   ] [ 1] {0x0000000601400000}
> jdk.internal.loader.ClassLoaders$AppClassLoader::parallelLockMap (offset =
> 36)
>
> [31,832s][error  ][cds,heap   ] [ 2] {0x000000060095c5a8}
> java.util.concurrent.ConcurrentHashMap
>
> [31,832s][error  ][cds,heap   ] Cannot archive the sub-graph referenced
> from [Ljava.util.concurrent.ConcurrentHashMap$Node; object
> (0x000000060354bc88) size 262160, skipped.
>
> [31,832s][error  ][cds    ] An error has occurred while writing the
> shared archive file.
>
> So it looks like some classloader has a very large ConcurrentHashMap that
> ends up with a backing array of length 65536.
>
> Tracing back to Leyden’s sources, I found that the the archived objects
> size is limited
>
>    -
>
>
>    https://github.com/openjdk/leyden/blob/master/src/hotspot/share/cds/heapShared.cpp#L295
>    -
>
>
>    https://github.com/openjdk/leyden/blob/master/src/hotspot/share/cds/archiveHeapWriter.cpp#L118
>    <https://github.com/openjdk/leyden/blob/master/src/hotspot/share/cds/archiveHeapWriter.cpp#L121>
>    -
>
>
>    https://github.com/openjdk/leyden/blob/83c7d3bbe860656ee403fa29df6a9e0aae962839/src/hotspot/share/cds/archiveHeapWriter.hpp#L118
>
>
>
> By adding a --add-opens java.base/java.util=ALL-UNNAMED to the command
> line, it disables the optimized module handling
>
> [0.001s][info][cds] optimized module handling: disabled due to
> incompatible property: jdk.module.addopens=java.base/java.util=ALL-UNNAMED
>
> And the error goes away.
>
> I tried to make my case work without adding some add-opens that are not
> required by the app. As the limit in object size comes from
> MIN_GC_REGION_ALIGNMENT, which is limited by Shenandoah’s region size, and
> I don’t use that GC, I simply modified is_too_large_to_archive() to always
> return false. Then the archiving with optimized module handling succeeds.
>
> I created a Github repo to reproduce the issue, see the readme there:
> https://github.com/obourgain/leyden-classloader
>
> With regard to the improvements in startup time (on a M1 mac), here is
> what is get:
>
>    -
>
>    base line, exploded Spring Boot jar = 11s
>    -
>
>    spring AOT = 10.5s
>    -
>
>    leyden with optimized module handling disabled = 7.8s
>    -
>
>    leyden with optimized module handling enabled = 7.1s
>    -
>
>    leyden with optimized module handling disabled and Spring AOT = 7.3s
>    -
>
>    leyden with optimized module handling enabled and Spring AOT = 6.3s
>
>
> Overall, I think that’s pretty impressive improvements for this app’s
> startup which was pretty efficient already. Moreover, I did that on a
> project that started 12 or 13 years ago, so it shows that Leyden is
> applicable for that kind of project with substantial gains.
>
> I plan to try the premain branch soon.
>
> Let me know if I can help with more information.
>
> (this message was first submitted before I subscribed to the list. A copy
> of it may be pending review by the admin. Feel free to delete the pending
> copy)
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/leyden-dev/attachments/20250418/44290591/attachment.htm>