issue and crash when testing premain on a real world large app

Tue Apr 22 09:49:22 UTC 2025

Hello,

I tested the premain branch on a pretty large Spring Boot application (>1M
loc, fat jar is about 240MB) and had two issues.

First issue is similar to my previous issue
https://mail.openjdk.org/pipermail/leyden-dev/2025-April/001875.html, I had
to set -XX:-ArchiveLoaderLookupCache to avoid Leyden failing due to a large
ConcurrentHashMap's backing array.

Second issue is way worse, I had a segfault on application start.
I tracked down the issue to:
- use the Bouncy Castle security provider
- run as a Spring Boot app.

I created a reproducer there https://github.com/obourgain/leyden-crash
I couldn't reproduce as a simple main, but it should be possible to reduce
the size of the reproducer even more.

Some observations:

- During the aotconf creation, bouncy castle classes are excluded
because they are from a signed jar like:
Skipping org/bouncycastle/jcajce/provider/asymmetric/util/BaseKeyFactorySpi:
Signed JAR

- During the AOT cache creation there are a lot of warnings , it may
be related to the crash:
[1,064s][warning][cds,heap] Archive heap points to a static field that
may hold a different value at runtime:
[1,064s][warning][cds,heap] Field:
sun/security/x509/AlgorithmId::SHA3_512withRSA_oid
[1,064s][warning][cds,heap] Value: sun.security.util.ObjectIdentifier
[1,064s][warning][cds,heap] {0x00000006000453b8} - klass:
'sun/security/util/ObjectIdentifier' - flags:
[1,064s][warning][cds,heap]
[1,064s][warning][cds,heap]  - ---- fields (total size 4 words):
[1,064s][warning][cds,heap]  - private 'componentLen' 'I' @12  -1 (0xffffffff)
[1,064s][warning][cds,heap]  - private transient
'componentsCalculated' 'Z' @16  false (0x00)
[1,064s][warning][cds,heap]  - private 'encoding' '[B' @20
[B{0x00000006000453d8} (0xc0008a7b)
[1,064s][warning][cds,heap]  - private volatile transient 'stringForm'
'Ljava/lang/String;' @24
"2.16.840.1.101.3.4.3.16"{0x0000000600045378} (0xc0008a6f)
[1,064s][warning][cds,heap]  - private 'components'
'Ljava/lang/Object;' @28  null (0x00000000)
[1,064s][warning][cds,heap] --- trace begin ---
[1,064s][warning][cds,heap] [ 0] {0x0000000600043330}
java.util.concurrent.ConcurrentHashMap::table (offset = 20)
[1,064s][warning][cds,heap] [ 1] {0x0000000600043370}
[Ljava.util.concurrent.ConcurrentHashMap$Node; @[87]
[1,064s][warning][cds,heap] [ 2] {0x0000000600045358}
java.util.concurrent.ConcurrentHashMap$Node::val (offset = 20)
[1,064s][warning][cds,heap] [ 3] {0x00000006000453b8}
sun.security.util.ObjectIdentifier
[1,064s][warning][cds,heap] --- trace end ---

- Sample Backtrace from the segfault:
💣 Program crashed: Bad pointer dereference at 0x0000000000000000

Thread 2 crashed:

 0 0x0000000000000000
 1 0x00000001132b5230
 2 0x00000001132b0154
 3 JavaCalls::call_helper(JavaValue*, methodHandle const&,
JavaCallArguments*, JavaThread*) + 988 in libjvm.dylib at
make/hotspot/src/hotspot/share/runtime/javaCalls.cpp:415:7
 4 InstanceKlass::call_class_initializer(JavaThread*) + 764 in
libjvm.dylib at
make/hotspot/src/hotspot/share/oops/instanceKlass.cpp:1777:5
 5 InstanceKlass::initialize_impl(JavaThread*) + 2748 in libjvm.dylib
at make/hotspot/src/hotspot/share/oops/instanceKlass.cpp:1331:7
 6 InstanceKlass::initialize_impl(JavaThread*) + 1540 in libjvm.dylib
at make/hotspot/src/hotspot/share/oops/instanceKlass.cpp:1292:20
 7 HeapShared::resolve_or_init_classes_for_subgraph_of(Klass*, bool,
JavaThread*) + 900 in libjvm.dylib at
make/hotspot/src/hotspot/share/cds/heapShared.cpp:1491:9
 8 HeapShared::initialize_from_archived_subgraph(JavaThread*, Klass*)
+ 336 in libjvm.dylib at
make/hotspot/src/hotspot/share/cds/heapShared.cpp:1415:5
 9 JVM_InitializeFromArchive + 560 in libjvm.dylib at
make/hotspot/src/hotspot/share/prims/jvm.cpp:3369:3
10 0x00000001132b8e80
11 0x00000001132b4a90
12 0x00000001132b0154
13 JavaCalls::call_helper(JavaValue*, methodHandle const&,
JavaCallArguments*, JavaThread*) + 988 in libjvm.dylib at
make/hotspot/src/hotspot/share/runtime/javaCalls.cpp:415:7
14 InstanceKlass::call_class_initializer(JavaThread*) + 764 in
libjvm.dylib at
make/hotspot/src/hotspot/share/oops/instanceKlass.cpp:1777:5
15 InstanceKlass::initialize_impl(JavaThread*) + 2748 in libjvm.dylib
at make/hotspot/src/hotspot/share/oops/instanceKlass.cpp:1331:7
16 LinkResolver::resolve_static_call(CallInfo&, LinkInfo const&, bool,
JavaThread*) + 152 in libjvm.dylib at
make/hotspot/src/hotspot/share/interpreter/linkResolver.cpp:1116:21
17 LinkResolver::resolve_invokestatic(CallInfo&, constantPoolHandle
const&, int, JavaThread*) + 52 in libjvm.dylib at
make/hotspot/src/hotspot/share/interpreter/linkResolver.cpp:1749:3
18 LinkResolver::resolve_invoke(CallInfo&, Handle, constantPoolHandle
const&, int, Bytecodes::Code, JavaThread*) + 116 in libjvm.dylib at
make/hotspot/src/hotspot/share/interpreter/linkResolver.cpp:1708:39
19 InterpreterRuntime::resolve_invoke(JavaThread*, Bytecodes::Code) +
748 in libjvm.dylib at
make/hotspot/src/hotspot/share/interpreter/interpreterRuntime.cpp:988:5
20 InterpreterRuntime::resolve_invokestatic(JavaThread*) + 420 in
libjvm.dylib at
make/hotspot/src/hotspot/share/interpreter/interpreterRuntime.cpp:952:3
21 InterpreterRuntime::resolve_from_cache(JavaThread*,
Bytecodes::Code) + 2644 in libjvm.dylib at
make/hotspot/src/hotspot/share/interpreter/interpreterRuntime.cpp:1167:37
22 0x00000001132c48c4
23 0x00000001132b0154
24 JavaCalls::call_helper(JavaValue*, methodHandle const&,
JavaCallArguments*, JavaThread*) + 988 in libjvm.dylib at
make/hotspot/src/hotspot/share/runtime/javaCalls.cpp:415:7
25 InstanceKlass::call_class_initializer(JavaThread*) + 764 in
libjvm.dylib at
make/hotspot/src/hotspot/share/oops/instanceKlass.cpp:1777:5
26 InstanceKlass::initialize_impl(JavaThread*) + 2748 in libjvm.dylib
at make/hotspot/src/hotspot/share/oops/instanceKlass.cpp:1331:7
27 InstanceKlass::initialize_impl(JavaThread*) + 1540 in libjvm.dylib
at make/hotspot/src/hotspot/share/oops/instanceKlass.cpp:1292:20
28 HeapShared::init_classes_for_special_subgraph(Handle, JavaThread*)
+ 356 in libjvm.dylib at
make/hotspot/src/hotspot/share/cds/heapShared.cpp:1387:13
29 Threads::create_vm(JavaVMInitArgs*, bool*) + 1504 in libjvm.dylib
at make/hotspot/src/hotspot/share/runtime/threads.cpp:808:5
30 JNI_CreateJavaVM_inner(JavaVM_**, void**, void*) + 80 in
libjvm.dylib at make/hotspot/src/hotspot/share/prims/jni.cpp:3587:12
31 JNI_CreateJavaVM + 116 in libjvm.dylib at
make/hotspot/src/hotspot/share/prims/jni.cpp:3678:14
32 InitializeJVM + 184 in libjli.dylib at
make/src/java.base/share/native/libjli/java.c:1510:9
33 JavaMain + 256 in libjli.dylib at
make/src/java.base/share/native/libjli/java.c:494:10
34 ThreadJavaMain + 12 in libjli.dylib at
make/src/java.base/macosx/native/libjli/java_md_macosx.m:679:29
35 0x0000000197d71c0c _pthread_start + 136 in libsystem_pthread.dylib

With regards to performance, with disabling the loading of
BouncyCastleProvider in the application, it starts way faster and with
lower resources, impressive work.

- baseline (extracted Spring Boot jar)
  Time (mean ± σ):     11.510 s ±  0.432 s    [User: 32.354 s, System:
1.751 s]

- Leyden main branch
  Time (mean ± σ):      6.031 s ±  0.090 s    [User: 12.591 s, System:
0.513 s]
aotcache file = 250 MB

- Leyden premain branch
  Time (mean ± σ):      4.801 s ±  0.022 s    [User: 6.662 s, System: 0.390
s]
aotcache file = 300 MB

Regards.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/leyden-dev/attachments/20250422/16dd4123/attachment.htm>