error writing the shared archive file with large number of loaded classes

[ioi.lam at oracle.com]

Hi Olivier,

Thanks for reporting the problem and sharing your benchmarking results. 
It's good to see that you can apply Leyden on such big projects and see 
substantial start-up benefits.

We already have a bug opened for this issue:

https://bugs.openjdk.org/browse/JDK-8342283

I opened a PR a while back but got side tracked and the PR got closed:

https://github.com/openjdk/jdk/pull/21797

Could you try applying this PR and see if it works for your test case?

I will revive the PR and hopefully be able to fix it in JDK 25, where 
the bug is currently targeted to.

Thanks

- Ioi


On 4/16/25 1:56 AM, Olivier Bourgain wrote:
>
> Hello,
>
>
> I have been experimenting with Leyden for a pretty large Spring Boot 
> application (>1M loc, fat jar is about 240MB) and I think I hit a 
> limitation of the current implementation of  Leyden.
>
>
> I got the following error message at the end of the training run:
>
> [31,832s][error  ][cds,heap   ] [ 0] {0x000000060080be70} 
> jdk.internal.loader.ArchivedClassLoaders::appLoader (offset = 16)
>
> [31,832s][error  ][cds,heap   ] [ 1] {0x0000000601400000} 
> jdk.internal.loader.ClassLoaders$AppClassLoader::parallelLockMap 
> (offset = 36)
>
> [31,832s][error  ][cds,heap   ] [ 2] {0x000000060095c5a8} 
> java.util.concurrent.ConcurrentHashMap
>
> [31,832s][error  ][cds,heap   ] Cannot archive the sub-graph 
> referenced from [Ljava.util.concurrent.ConcurrentHashMap$Node; object 
> (0x000000060354bc88) size 262160, skipped.
>
> [31,832s][error  ][cds ] An error has occurred while writing the 
> shared archive file.
>
>
> So it looks like some classloader has a very large ConcurrentHashMap 
> that ends up with a backing array of length 65536.
>
> Tracing back to Leyden’s sources, I found that the the archived 
> objects size is limited
>
>  *
>
>     https://github.com/openjdk/leyden/blob/master/src/hotspot/share/cds/heapShared.cpp#L295
>     <https://github.com/openjdk/leyden/blob/master/src/hotspot/share/cds/heapShared.cpp#L295>
>
>  *
>
>     https://github.com/openjdk/leyden/blob/master/src/hotspot/share/cds/archiveHeapWriter.cpp#L118
>     <https://github.com/openjdk/leyden/blob/master/src/hotspot/share/cds/archiveHeapWriter.cpp#L121>
>
>  *
>
>     https://github.com/openjdk/leyden/blob/83c7d3bbe860656ee403fa29df6a9e0aae962839/src/hotspot/share/cds/archiveHeapWriter.hpp#L118
>     <https://github.com/openjdk/leyden/blob/83c7d3bbe860656ee403fa29df6a9e0aae962839/src/hotspot/share/cds/archiveHeapWriter.hpp#L118>
>
>
>
> By adding a --add-opens java.base/java.util=ALL-UNNAMED to the command 
> line, it disables the optimized module handling
>
> [0.001s][info][cds] optimized module handling: disabled due to 
> incompatible property: jdk.module.addopens=java.base/java.util=ALL-UNNAMED
>
> And the error goes away.
>
>
> I tried to make my case work without adding some add-opens that are 
> not required by the app. As the limit in object size comes from 
> MIN_GC_REGION_ALIGNMENT, which is limited by Shenandoah’s region size, 
> and I don’t use that GC, I simply modified is_too_large_to_archive() 
> to always return false. Then the archiving with optimized module 
> handling succeeds.
>
>
> I created a Github repo to reproduce the issue, see the readme there: 
> https://github.com/obourgain/leyden-classloader 
> <https://github.com/obourgain/leyden-classloader>
>
>
> With regard to the improvements in startup time (on a M1 mac), here is 
> what is get:
>
>  *
>
>     base line, exploded Spring Boot jar = 11s
>
>  *
>
>     spring AOT = 10.5s
>
>  *
>
>     leyden with optimized module handling disabled = 7.8s
>
>  *
>
>     leyden with optimized module handling enabled = 7.1s
>
>  *
>
>     leyden with optimized module handling disabled and Spring AOT = 7.3s
>
>  *
>
>     leyden with optimized module handling enabled and Spring AOT = 6.3s
>
>
> Overall, I think that’s pretty impressive improvements for this app’s 
> startup which was pretty efficient already. Moreover, I did that on a 
> project that started 12 or 13 years ago, so it shows that Leyden is 
> applicable for that kind of project with substantial gains.
>
>
> I plan to try the premain branch soon.
>
> Let me know if I can help with more information.
>
>
> (this message was first submitted before I subscribed to the list. A 
> copy of it may be pending review by the admin. Feel free to delete the 
> pending copy)
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/leyden-dev/attachments/20250416/26dd52c6/attachment.htm>