<!DOCTYPE html><html><head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body>
<p><br>
</p>
<div class="moz-cite-prefix">On 9/19/24 7:44 AM, Ashutosh Mehra
wrote:<br>
</div>
<blockquote type="cite" cite="mid:CAKt0pyRj8_mDiH58v10sqXNHA4iHja6-7D9N_S2VW9rhESPcTw@mail.gmail.com">
<div dir="ltr">
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">As
I am cleaning up the code for upstreaming to mainline, I am
going add an equivalent check in the C code to filter out
these indy call sites, so they won't be resolved at all during
the assembly phase. Otherwise, I will run into problems
described in <a href="https://bugs.openjdk.org/browse/JDK-8290417" target="_blank" moz-do-not-send="true" class="moz-txt-link-freetext">https://bugs.openjdk.org/browse/JDK-8290417</a></blockquote>
<div><br>
</div>
<div>Thanks for the link to the bug. The scenario described in
that bug is exactly the same as the Infinispan case.</div>
<div>So if we filter out such cases during indy resolution then
it should resolve the Infinispan issue as well.</div>
<div><br>
</div>
<div>A basic question: why can't CDS handle the lambda proxy
class generated in <span style="color:rgb(31,35,40)"><font face="arial, sans-serif">useImplMethodHandle</font></span><span style="color:rgb(31,35,40);font-family:-apple-system,BlinkMacSystemFont,"Segoe UI","Noto Sans",Helvetica,Arial,sans-serif,"Apple Color Emoji","Segoe UI Emoji""> mode?</span><br>
</div>
<div><span style="color:rgb(31,35,40);font-family:-apple-system,BlinkMacSystemFont,"Segoe UI","Noto Sans",Helvetica,Arial,sans-serif,"Apple Color Emoji","Segoe UI Emoji""><br>
</span></div>
</div>
</blockquote>
<p>If I remember correctly, it has to do with the shape of
dynamically generated bytecode:</p>
<p> public void accept(java.lang.Object);<br>
Code:<br>
0: ldc #28 // Dynamic #0:_:Ljava/lang/invoke/MethodHandle;<br>
2: aload_0<br>
3: getfield #15 // Field arg$1:LTester;<br>
6: aload_1<br>
7: checkcast #30 // class java/lang/String<br>
10: invokevirtual #36 // Method
java/lang/invoke/MethodHandle.invokeExact:(LTester;Ljava/lang/String;)V<br>
13: return<br>
</p>
<p>The result of the "ldc" was not symbolically encoded in the
generated class (as the generated class has no permission to
access that method). So the MethodHandle is stored as a binary
object in the mirror of this generated class (with the
java.lang.Class::classData field).</p>
<p>Plain CDS doesn't archive class mirrors, so the classData will be
lost.</p>
<p>With JEP 483, we should be able to preserve the classData, so I
am not sure why the useImplMethodHandle case is still failing. My
plan is to filter these out for now, but I will get back to it
later when I have more time.</p>
<p>Thanks</p>
<p>- Ioi<br>
</p>
<br>
<p><br>
</p>
<p><br>
</p>
<blockquote type="cite" cite="mid:CAKt0pyRj8_mDiH58v10sqXNHA4iHja6-7D9N_S2VW9rhESPcTw@mail.gmail.com">
<div dir="ltr">
<div><span style="color:rgb(31,35,40);font-family:-apple-system,BlinkMacSystemFont,"Segoe UI","Noto Sans",Helvetica,Arial,sans-serif,"Apple Color Emoji","Segoe UI Emoji"">Thanks,</span></div>
<div>
<div dir="ltr" class="gmail_signature" data-smartmail="gmail_signature">
<div dir="ltr">- Ashutosh Mehra</div>
</div>
</div>
<br>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On Thu, Sep 19, 2024 at
1:45 AM <<a href="mailto:ioi.lam@oracle.com" moz-do-not-send="true" class="moz-txt-link-freetext">ioi.lam@oracle.com</a>>
wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div>
<p>Hi Ashutosh,<br>
</p>
<p>I have some update:</p>
<p>The original crash was caused by the
"useImplMethodHandle" code in
InnerClassLambdaMetafactory.java:<br>
</p>
<p> // If the target class invokes a protected method
inherited from a<br>
// superclass in a different package, or does
'invokespecial', the<br>
// lambda class has no access to the resolved
method, or does<br>
// 'invokestatic' on a hidden class which cannot
be resolved by name.<br>
// Instead, we need to pass the live
implementation method handle to<br>
// the proxy class to invoke directly. (javac
prefers to avoid this<br>
// situation by generating bridges in the target
class)<br>
useImplMethodHandle =
(Modifier.isProtected(implInfo.getModifiers()) &&<br>
!VerifyAccess.isSamePackage(targetClass,
implInfo.getDeclaringClass())) ||<br>
implKind ==
MethodHandleInfo.REF_invokeSpecial ||<br>
implKind ==
MethodHandleInfo.REF_invokeStatic &&
implClass.isHidden();<br>
</p>
<p>As I am cleaning up the code for upstreaming to mainline,
I am going add an equivalent check in the C code to filter
out these indy call sites, so they won't be resolved at
all during the assembly phase. Otherwise, I will run into
problems described in <a href="https://bugs.openjdk.org/browse/JDK-8290417" target="_blank" moz-do-not-send="true" class="moz-txt-link-freetext">https://bugs.openjdk.org/browse/JDK-8290417</a><br>
</p>
<p>Once I get the filtering code working, I will integrate
it back to premain.<br>
</p>
<p>>I am wondering if we can workaround class circularity
issues by recording class initialization order<br>
>during training run and use that to guide the
initialization during assembly phase.<br>
</p>
<p>In the production run we take different paths than the
training run<br>
</p>
<p>(1) some classes are aot-initialized (especially the
enums)<br>
(2) some classes make special CDS calls<br>
</p>
<p>so I am not sure if it's possible to get the same
initialization order as in the training run (or assembly
phase).</p>
<p>(more below)<br>
</p>
<div>On 9/18/24 9:10 AM, Ashutosh Mehra wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">Hi Ioi,
<div><br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">I
was having a similar circularity issue (but in
production time) and I just added enough classes to
make the NPE go away.<br>
</blockquote>
<div> </div>
<div>
<div>I am wondering if you have a test case that
reproduces the NPE which prompted you to add these
classes:</div>
<div><br>
</div>
<div><a href="https://urldefense.com/v3/__https://github.com/openjdk/leyden/blob/7781109154bf2af89854c7e13aa3e160bb82608e/src/hotspot/share/cds/aotClassInitializer.cpp*L65-L78__;Iw!!ACWV5N9M2RV99hQ!NYvqLbb_Ib2kSdm2fcYVLGby-AiB2TG7KtC1HQHVKLiM-_kRb5mOt8krV5IZfReaYZ0MPadDfa08hQ$" target="_blank" moz-do-not-send="true">https://github.com/openjdk/leyden/blob/7781109154bf2af89854c7e13aa3e160bb82608e/src/hotspot/share/cds/aotClassInitializer.cpp#L65-L78</a></div>
<div><br>
</div>
<div>I commented this code and ran the tests under
premain but didn't hit the NPE.</div>
<div><br>
</div>
</div>
</div>
</blockquote>
<p>I forgot what the problem was, but it happened for very
simple cases.<br>
</p>
<p>Thanks</p>
<p>- Ioi<br>
</p>
<p><br>
</p>
<blockquote type="cite">
<div dir="ltr">
<div>
<div>Thanks,</div>
<div>
<div>
<div dir="ltr" class="gmail_signature">
<div dir="ltr">- Ashutosh Mehra</div>
</div>
</div>
<br>
</div>
</div>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On Tue, Sep 17, 2024
at 1:11 AM <<a href="mailto:ioi.lam@oracle.com" target="_blank" moz-do-not-send="true" class="moz-txt-link-freetext">ioi.lam@oracle.com</a>>
wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div>
<p>Hi Ashutosh,<br>
</p>
<p>So this looks like a potential bug (or feature)
in the core lib code. When CDS forcefully
initializes a class in an unexpected (or untested)
order, the "initialization soup" fails.</p>
<p>Perhaps a work-around would be to make some
harmless calls at the place where CDS was calling
into MethodType.createArchivedObjects(). E.g., do
something like this:</p>
<p><font face="monospace">+ if
(CDSConfig::is_dumping_invokedynamic()) {<br>
+ // call into Java:
jdk.internal.misc::warmupInvokeDynamic();<br>
+ }<br>
// Rewrite and link classes<br>
log_info(cds)("Rewriting and linking classes
...");<br>
<br>
</font>Maybe you can add a new Lambda expressions,
MethodHandle invocations, etc, that would
hopefully cause PrimitiveClassDescImpl and friends
to be initialized in their natural order.<br>
</p>
<p>Or call
class.forName("java.lang.constant.ConstantDescs")
??</p>
<p>BTW, you can see my comments in
AOTClassInitializer::is_forced_preinit_class():<br>
<br>
// TODO:<br>
// This is needed since JDK-8338532. Without
this, when<br>
// archived heap objects are used, the class
init order is not<br>
// expected by the jdk/internal/constant
bootstrap code and we<br>
// will get a null pointer exception.<br>
//<br>
// When bootstraping has intricated/fragile
order, it's probably<br>
// better to archive all related classes in an
initialized state<br>
// (i.e., take a snapshot). The existing
approach in<br>
//
heapShared::resolve_or_init_classes_for_subgraph_of()
won't work.<br>
"jdk/internal/constant/PrimitiveClassDescImpl",<br>
"jdk/internal/constant/ReferenceClassDescImpl",<br>
"java/lang/constant/ConstantDescs",<br>
"sun/invoke/util/Wrapper",<br>
<br>
I was having a similar circularity issue (but in
production time) and I just added enough classes
to make the NPE go away. For your test case, if
you manage to fix in in the assembly run but run
into NPE in production run, you might need to add
more classes to this list. Yes, it's a hack :-(<br>
</p>
<p><br>
</p>
<p>Thanks</p>
<p>- Ioi<br>
</p>
<div>On 9/13/24 7:05 PM, Ashutosh Mehra wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">This is turning out to be a real
example of class initialization soup!
<div>As mentioned during the meeting, I am
getting NPE in the assembly phase when testing
the patch [0] that I proposed in my earlier
mail </div>
<div>using a test case inspired by the
Infinispan code.</div>
<div>NPE occurs when running the class
initializer for <font face="monospace">PrimitiveClassDescImpl </font></div>
<div>
<div>Interestingly, <span style="font-family:monospace">PrimitiveClassDescImpl</span> is
"forcefully" initialized by <font face="monospace">MetaspaceShared::link_shared_classes()</font>.</div>
<div><br>
</div>
<div>
<div>
<div dir="ltr" class="gmail_signature">
<div dir="ltr">I couldn't get a stack
trace so I relied on exception logs
(using -Xlog:exceptions=trace) to find
the cause which indicate following
frames on the stack:</div>
<div dir="ltr"><br>
</div>
<div dir="ltr"><font face="monospace">[0]
jdk/internal/constant/MethodTypeDescImpl::validateArgument(Ljava/lang/constant/ClassDesc;)Ljava/lang/constant/ClassDesc; @
bci 1<br>
</font></div>
<div dir="ltr"><font face="monospace">[1]
jdk/internal/constant/MethodTypeDescImpl::ofTrusted(Ljava/lang/constant/ClassDesc;[Ljava/lang/constant/ClassDesc;)Ljdk/internal/constant/MethodTypeDescImpl; @
bci 27<br>
</font></div>
<div dir="ltr"><font face="monospace">[2]
java/lang/constant/ConstantDescs::ofConstantBootstrap(Ljava/lang/constant/ClassDesc;Ljava/lang/String;Ljava/lang/constant/ClassDesc;[Ljava/lang/constant/ClassDesc;)Ljava/lang/constant/DirectMethodHandleDesc; @
bci 47</font></div>
<div dir="ltr"><font face="monospace">[3]
java/lang/constant/ConstantDescs::<clinit> @ bci 664<br>
</font></div>
<div dir="ltr"><font face="monospace">[4]
jdk/internal/constant/PrimitiveClassDescImpl::<init>(Ljava/lang/String;)V @
bci 1<br>
</font></div>
<div dir="ltr"><font face="monospace">[5]
jdk/internal/constant/PrimitiveClassDescImpl::<clinit>(Ljava/lang/String;)V @
bci 6</font><span style="font-family:monospace"><br>
</span></div>
<div dir="ltr"><span style="font-family:monospace"><br>
</span></div>
<div dir="ltr">Notice that invocation
of <font face="monospace">PrimitiveClassDescImpl::<clinit>
</font>results in initialization of <font face="monospace">ConstantDescs</font> class
(see frame 3).<br>
</div>
<div><font face="monospace">ConstantDescs::<clinit> @
664 </font><font face="arial, sans-serif">corresponds
to following java code:</font></div>
<div><br>
</div>
<div><font face="monospace"> public
static final DirectMethodHandleDesc
BSM_CLASS_DATA_AT<br>
</font></div>
<div><font face="monospace">
=
ofConstantBootstrap(CD_MethodHandles,
"classDataAt",<br>
CD_Object, CD_int);<br>
</font></div>
<div><br>
</div>
<div>The last parameter CD_int is
initialized as:</div>
<div><br>
</div>
<div><font face="monospace"> public
static final ClassDesc CD_int =
PrimitiveClassDescImpl.CD_int;<br>
</font></div>
<div><br>
</div>
<div>So, its value is obtained from <font face="monospace">PrimitiveClassDescImpl.CD_int</font> which
hasn't been initialized properly yet.
As a result <font face="monospace">ConstantDescs::CD_int
</font><font face="arial, sans-serif">is
assigned</font> null which results
in <font face="monospace">MethodTypeDescImpl::validateArgument</font>
throwing NPE later.</div>
<div>There is a clear class
initialization circularity involving <font face="monospace">PrimitiveClassDescImpl</font>
and <font face="monospace">ConstantDescs</font>,
and the result depends on which class
gets initialized first.</div>
<div><br>
</div>
<div>Without my patch this issue is not
seen because <font face="monospace">PrimitiveClassDescImpl</font>
has already been initialized by the
time <font face="monospace">MetaspaceShared::link_shared_classes()</font><font face="arial, sans-serif"> is called.</font></div>
<div><font face="arial, sans-serif">Its
initialization is triggered by the
call to </font><font face="monospace">MethodType::createArchivedObjects(). </font></div>
<div><font face="arial, sans-serif">It
also explains why my patch
introduced this issue because it
effectively moved the call to </font><span style="font-family:monospace">MethodType::createArchivedObjects()
</span><font face="arial, sans-serif">after </font><span style="font-family:monospace">MetaspaceShared::link_shared_classes().</span></div>
<div><span style="font-family:monospace"><br>
</span></div>
<div><font face="arial, sans-serif">[0] <a href="https://urldefense.com/v3/__https://github.com/ashu-mehra/leyden/commit/d8f99cce67df1c7b0f7ef8562676df438633a66e__;!!ACWV5N9M2RV99hQ!MnS3LSY2PXCCXbHvdyMfR6Nj57XH7Ey7aZuOLmQ__hhE_RhUrKnkzZ0uP8AwLZYcl8lAhhZK3GwoiA$" target="_blank" moz-do-not-send="true">https://github.com/ashu-mehra/leyden/commit/d8f99cce67df1c7b0f7ef8562676df438633a66e</a></font></div>
<div><font face="arial, sans-serif"><br>
</font></div>
<div><font face="arial, sans-serif">Thanks,</font></div>
<div><font face="arial, sans-serif">-
Ashutosh Mehra</font></div>
</div>
</div>
<br>
</div>
</div>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On Wed, Sep
11, 2024 at 3:12 PM Ashutosh Mehra <<a href="mailto:asmehra@redhat.com" target="_blank" moz-do-not-send="true" class="moz-txt-link-freetext">asmehra@redhat.com</a>>
wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div dir="ltr">Regarding the
WrongMethodTypeException that I mentioned in
my previous email (see pt 3),
<div>this exception happens when lambda
proxy class attempts to invoke the
MethodHandle it obtained from the
classData:</div>
<div><br>
</div>
<div><font face="monospace"> public void
accept(java.lang.Object);<br>
descriptor: (Ljava/lang/Object;)V<br>
flags: (0x0001) ACC_PUBLIC<br>
Code:<br>
stack=3, locals=2, args_size=2<br>
0: ldc #26
// Dynamic
#0:_:Ljava/lang/invoke/MethodHandle;<br>
2: aload_0<br>
3: getfield #13
// Field
arg$1:Lorg/wildfly/security/WildFlyElytronBaseProvider;<br>
6: aload_1<br>
7: checkcast #28
// class
java/security/Provider$Service<br>
10: invokevirtual #34
// Method
java/lang/invoke/MethodHandle.invokeExact:(Lorg/wildfly/security/WildFlyElytronBaseProvider;Ljava/security/Provider$Service;)V<br>
13: return</font><br>
</div>
<div><br>
</div>
<div>
<div>The scenario is during the assembly
phase as part of the indy resolution the
MethodHandle for which the exception is
thrown gets created.</div>
<div>Normally MethodHandle's type gets
added in <font face="monospace">MethodType::internTable</font>
but by the time indy resolution happens,
JVM has already taken </div>
<div>snapshot of the <font face="monospace">MethodType::internTable</font>
through an upcall to <font face="monospace">MethodType::createArchivedObjects()</font>.</div>
<div>As a result the AOTCache ends up with
the MethodType object which is not in <font face="monospace">AOTHolder.archivedMethodTypes</font>.</div>
<div><br>
</div>
<div>During the production run, when the
jvm invokes the MethodHandle, it
searches for the MethodType
corresponding to the signature passed at
the callsite.</div>
<div>As expected, it fails to find it in
the <font face="monospace">AOTHolder.archivedMethodTypes</font>,
so it creates a new instance of the
MethodType.</div>
<div>But <font face="monospace">Invokers.checkExactType()</font>
relies on the MethodHandle's type to be
the same object as the MethodType object
passed as parameter.</div>
<div><br>
</div>
<div><font face="monospace"> static
void checkExactType(M</font><span style="font-family:monospace">ethodHandle
mh</span><span style="font-family:monospace">M,
MethodType expected) {</span></div>
<div><font face="monospace">
MethodType targetType = mh.type();<br>
if (targetType != expected)<br>
throw
newWrongMethodTypeException(targetType,
expected);<br>
}</font><br>
</div>
<div><br>
</div>
<div>Hence, it throws <span style="font-family:monospace">WrongMethodTypeException</span><font face="arial, sans-serif"> though the
two MT objects have the same
signature.</font></div>
<div><font face="arial, sans-serif"><br>
</font></div>
<div><font face="arial, sans-serif">To
handle this scenario, I changed the
order of indy resolution and </font>upcall
to <font face="monospace">MethodType::createArchivedObjects()</font><font face="arial, sans-serif"> as:</font></div>
<div><font face="arial, sans-serif"><br>
</font></div>
<div><font face="monospace">diff --git
a/src/hotspot/share/cds/metaspaceShared.cpp
b/src/hotspot/share/cds/metaspaceShared.cpp<br>
index df4bcadefa3..457716cac5b 100644<br>
---
a/src/hotspot/share/cds/metaspaceShared.cpp<br>
+++
b/src/hotspot/share/cds/metaspaceShared.cpp<br>
@@ -751,6 +751,20 @@ void
MetaspaceShared::link_shared_classes(bool
jcmd_request, TRAPS) {<br>
if
(CDSConfig::is_dumping_final_static_archive())
{<br>
FinalImageRecipes::apply_recipes(CHECK);<br>
}<br>
+<br>
+#if INCLUDE_CDS_JAVA_HEAP<br>
+ if
(CDSConfig::is_dumping_invokedynamic())
{<br>
+ // This makes sure that the
MethodType and MethodTypeForm tables
won't be updated<br>
+ // concurrently when we are
saving their contents into a side
table.<br>
+
assert(CDSConfig::allow_only_single_java_thread(),
"Required");<br>
+<br>
+ JavaValue result(T_VOID);<br>
+
JavaCalls::call_static(&result,
vmClasses::MethodType_klass(),<br>
+
vmSymbols::createArchivedObjects(),<br>
+
vmSymbols::void_method_signature(),<br>
+ CHECK);<br>
+ }<br>
+#endif<br>
}</font><br>
</div>
<div>Note that indy resolution happens as
part of <span style="font-family:monospace">FinalImageRecipes::apply_recipes(CHECK)
</span><font face="arial, sans-serif">which
is now invoked before the upcall to </font><span style="font-family:monospace">createArchivedObjects().</span></div>
<div><font face="arial, sans-serif">With
this change I am able to run the
application without any exceptions.</font></div>
<div>My complete patch can be seen here: <a href="https://urldefense.com/v3/__https://github.com/ashu-mehra/leyden/commit/d8f99cce67df1c7b0f7ef8562676df438633a66e__;!!ACWV5N9M2RV99hQ!MnS3LSY2PXCCXbHvdyMfR6Nj57XH7Ey7aZuOLmQ__hhE_RhUrKnkzZ0uP8AwLZYcl8lAhhZK3GwoiA$" target="_blank" moz-do-not-send="true">https://github.com/ashu-mehra/leyden/commit/d8f99cce67df1c7b0f7ef8562676df438633a66e</a></div>
<div>I will do more testing with this
patch.</div>
<div><br>
</div>
<div><a class="gmail_plusreply" id="m_5385423560619785655m_-6081196208708063630m_-8326742188757259955m_-669397441264475055m_4538222356731470316m_-1862706623606286828plusReplyChip-0" href="mailto:ioi.lam@oracle.com" target="_blank" moz-do-not-send="true">@Ioi
Lam</a> do you have any feedback on
this patch.<br>
</div>
<div><br>
</div>
<div>Thanks,<br clear="all">
<div>
<div dir="ltr" class="gmail_signature">
<div dir="ltr">- Ashutosh Mehra</div>
</div>
</div>
<br>
</div>
<div><br>
</div>
</div>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On Wed,
Sep 11, 2024 at 10:14 AM Ashutosh Mehra
<<a href="mailto:asmehra@redhat.com" target="_blank" moz-do-not-send="true" class="moz-txt-link-freetext">asmehra@redhat.com</a>>
wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div dir="ltr">Hi Andrew,
<div>Thanks for sharing the initial
investigation.
<div>I have been looking into this and
have a few of things to add to your
analysis:</div>
<div><br>
</div>
<div>1. As you mentioned the
classData for the lambda
class WildFlyElytronBaseProvider$$Lambda
is null.</div>
<div>The classData is stored in the
mirror object of the InstanceKlass
when the class is defined
through JVM_LookupDefineClass.</div>
<div>However, when we create the
scratch mirror object (which get
stored in the AOT cache) the
classData is not populated.</div>
<div>See <a href="https://urldefense.com/v3/__https://github.com/openjdk/leyden/blob/d23b9f2d5e3523cc547337da59327ed86a6057a3/src/hotspot/share/classfile/javaClasses.cpp*L1128-L1131__;Iw!!ACWV5N9M2RV99hQ!MnS3LSY2PXCCXbHvdyMfR6Nj57XH7Ey7aZuOLmQ__hhE_RhUrKnkzZ0uP8AwLZYcl8lAhhbKjzG3gw$" target="_blank" moz-do-not-send="true">https://github.com/openjdk/leyden/blob/d23b9f2d5e3523cc547337da59327ed86a6057a3/src/hotspot/share/classfile/javaClasses.cpp#L1128-L1131</a></div>
<div><br>
</div>
<div><font face="monospace"> Handle
classData; // set to null. Will be
reinitialized at runtime<br>
Handle mirror;<br>
Handle comp_mirror;<br>
allocate_mirror(k,
/*is_scratch=*/true,
protection_domain, classData,
mirror, comp_mirror, CHECK);</font><br>
</div>
<div><br>
</div>
<div>So this explains why the call to
<font face="monospace">classData(caller.lookupClass())</font><font face="arial, sans-serif"> returned
null.</font></div>
<div><br>
</div>
<div>2. In the mainline there is a
check
in InnerClassLambdaMetafactory.java
for the particular code pattern used
by the application.</div>
<div>If this code pattern is found
then the lambda proxy class is not
included in the CDS archive.</div>
<div>See <a href="https://urldefense.com/v3/__https://github.com/openjdk/leyden/blob/d23b9f2d5e3523cc547337da59327ed86a6057a3/src/java.base/share/classes/java/lang/invoke/InnerClassLambdaMetafactory.java*L163-L170__;Iw!!ACWV5N9M2RV99hQ!MnS3LSY2PXCCXbHvdyMfR6Nj57XH7Ey7aZuOLmQ__hhE_RhUrKnkzZ0uP8AwLZYcl8lAhhZl2S9tQg$" target="_blank" moz-do-not-send="true">https://github.com/openjdk/leyden/blob/d23b9f2d5e3523cc547337da59327ed86a6057a3/src/java.base/share/classes/java/lang/invoke/InnerClassLambdaMetafactory.java#L163-L170</a></div>
<div>and <a href="https://urldefense.com/v3/__https://github.com/openjdk/leyden/blob/d23b9f2d5e3523cc547337da59327ed86a6057a3/src/java.base/share/classes/java/lang/invoke/InnerClassLambdaMetafactory.java*L246__;Iw!!ACWV5N9M2RV99hQ!MnS3LSY2PXCCXbHvdyMfR6Nj57XH7Ey7aZuOLmQ__hhE_RhUrKnkzZ0uP8AwLZYcl8lAhhYsI3Xx0g$" target="_blank" moz-do-not-send="true">https://github.com/openjdk/leyden/blob/d23b9f2d5e3523cc547337da59327ed86a6057a3/src/java.base/share/classes/java/lang/invoke/InnerClassLambdaMetafactory.java#L246</a></div>
<div><br>
</div>
<div><font face="monospace"> //
If the target class invokes a
protected method inherited from a<br>
// superclass in a
different package, or does
'invokespecial', the<br>
// lambda class has no
access to the resolved method, or
does<br>
// 'invokestatic' on a
hidden class which cannot be
resolved by name.<br>
// Instead, we need to
pass the live implementation
method handle to<br>
// the proxy class to
invoke directly. (javac prefers to
avoid this<br>
// situation by generating
bridges in the target class)<br>
useImplMethodHandle =
(Modifier.isProtected(implInfo.getModifiers())
&&<br>
!VerifyAccess.isSamePackage(targetClass,
implInfo.getDeclaringClass())) ||<br>
implKind ==
MethodHandleInfo.REF_invokeSpecial
||<br>
implKind ==
MethodHandleInfo.REF_invokeStatic
&& implClass.isHidden();</font><br>
</div>
<div><br>
</div>
<div>In premain lambda proxy classes
get included in the AOT cache as a
result of indy resolution and that
mechanism doesn't have this kind of
check.</div>
<div><br>
</div>
<div>3. For the null classData problem
mentioned above, I tried to fix it
by storing classData in the scratch
mirror using the following patch:<br>
<br>
</div>
<div><font face="monospace">diff --git
a/src/hotspot/share/classfile/javaClasses.cpp
b/src/hotspot/share/classfile/javaClasses.cpp<br>
index bd8141adbcc..41766e98093
100644<br>
---
a/src/hotspot/share/classfile/javaClasses.cpp<br>
+++
b/src/hotspot/share/classfile/javaClasses.cpp<br>
@@ -1094,9 +1094,9 @@ void
java_lang_Class::create_mirror(Klass*
k, Handle class_loader,<br>
}<br>
if
(CDSConfig::is_dumping_heap()) {<br>
if
(CDSConfig::is_dumping_protection_domains())
{<br>
- create_scratch_mirror(k,
protection_domain, CHECK);<br>
+ create_scratch_mirror(k,
protection_domain, classData,
CHECK);<br>
} else {<br>
- create_scratch_mirror(k,
Handle() /* null
protection_domain*/, CHECK);<br>
+ create_scratch_mirror(k,
Handle() /* null
protection_domain*/, classData,
CHECK);<br>
}<br>
}<br>
} else {<br>
@@ -1117,7 +1117,7 @@ void
java_lang_Class::create_mirror(Klass*
k, Handle class_loader,<br>
// Note: we archive the "scratch
mirror" instead of
k->java_mirror(), because the<br>
// latter may contain
dumptime-specific information that
cannot be archived<br>
// (e.g., ClassLoaderData*, or
static fields that are modified by
Java code execution).<br>
-void
java_lang_Class::create_scratch_mirror(Klass*
k, Handle protection_domain,
TRAPS) {<br>
+void
java_lang_Class::create_scratch_mirror(Klass*
k, Handle protection_domain,
Handle classData, TRAPS) {<br>
if (k->class_loader() !=
nullptr &&<br>
k->class_loader() !=
SystemDictionary::java_platform_loader()
&&<br>
k->class_loader() !=
SystemDictionary::java_system_loader())
{<br>
@@ -1125,9 +1125,11 @@ void
java_lang_Class::create_scratch_mirror(Klass*
k, Handle protection_domain,<br>
return;<br>
}<br>
<br>
- Handle classData; // set to
null. Will be reinitialized at
runtime<br>
+ //Handle classData; // set to
null. Will be reinitialized at
runtime<br>
Handle mirror;<br>
Handle comp_mirror;<br>
allocate_mirror(k,
/*is_scratch=*/true,
protection_domain, classData,
mirror, comp_mirror, CHECK);<br>
<br>
if (comp_mirror() != nullptr) {<br>
diff --git
a/src/hotspot/share/classfile/javaClasses.hpp
b/src/hotspot/share/classfile/javaClasses.hpp<br>
index bc49a0861a7..7ec2a2556dd
100644<br>
---
a/src/hotspot/share/classfile/javaClasses.hpp<br>
+++
b/src/hotspot/share/classfile/javaClasses.hpp<br>
@@ -263,7 +263,7 @@ class
java_lang_Class : AllStatic {<br>
<br>
// Archiving<br>
static void
serialize_offsets(SerializeClosure*
f) NOT_CDS_RETURN;<br>
- static void
create_scratch_mirror(Klass* k,
Handle protection_domain, TRAPS)
NOT_CDS_JAVA_HEAP_RETURN;<br>
+ static void
create_scratch_mirror(Klass* k,
Handle protection_domain, Handle
classData, TRAPS)
NOT_CDS_JAVA_HEAP_RETURN;<br>
static bool
restore_archived_mirror(Klass *k,
Handle class_loader, Handle
module,<br>
Handle protection_domain,<br>
TRAPS)
NOT_CDS_JAVA_HEAP_RETURN_(false);</font><br>
</div>
<div><br>
</div>
<div>But this resulted in a different
exception:</div>
<div><br>
</div>
<div><font face="monospace">Exception
in thread "main"
java.lang.ExceptionInInitializerError<br>
at
com.redhat.leyden.Main.main(Main.java:7)<br>
Caused by:
java.lang.invoke.WrongMethodTypeException:
handle's method type
(WildFlyElytronBaseProvider,Service)void
but found
(WildFlyElytronBaseProvider,Service)void<br>
at
java.base/java.lang.invoke.Invokers.newWrongMethodTypeException(Invokers.java:521)<br>
at
java.base/java.lang.invoke.Invokers.checkExactType(Invokers.java:530)<br>
at
java.base/java.lang.invoke.Invokers$Holder.invokeExact_MT(Invokers$Holder)<br>
at
org.wildfly.security.WildFlyElytronBaseProvider$$Lambda/0x80000000c.accept(Unknown
Source)<br>
at
org.wildfly.security.WildFlyElytronBaseProvider.putMakedPasswordImplementations(WildFlyElytronBaseProvider.java:112)<br>
at
org.wildfly.security.WildFlyElytronBaseProvider.putPasswordImplementations(WildFlyElytronBaseProvider.java:107)<br>
at
org.wildfly.security.password.WildFlyElytronPasswordProvider.<init>(WildFlyElytronPasswordProvider.java:43)<br>
at
org.wildfly.security.password.WildFlyElytronPasswordProvider.<clinit>(WildFlyElytronPasswordProvider.java:36)<br>
... 1 more</font><br>
</div>
<div><font face="monospace"><br>
</font></div>
<div>The exception message is strange
because the handle's method type and
the expected type are both
symbolically the same.</div>
<div>I am debugging this exception at
the moment.</div>
<div> </div>
<div>Thanks,</div>
<div>
<div>
<div dir="ltr" class="gmail_signature">
<div dir="ltr">- Ashutosh Mehra</div>
</div>
</div>
<br>
</div>
</div>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On
Wed, Sep 11, 2024 at 6:03 AM Andrew
Dinn <<a href="mailto:adinn@redhat.com" target="_blank" moz-do-not-send="true" class="moz-txt-link-freetext">adinn@redhat.com</a>>
wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Oops,
sorry, I debugged this a few days ago!
Correction to a few details:<br>
<br>
On 11/09/2024 10:39, Andrew Dinn
wrote:<br>
> A crash due to an NPE was
observed in the Infinispan (Data Grid)
server <br>
> app when deployed using the
Leyden EA. The crash still manifests
with <br>
> the latest premain code. The
crash happens below an application
call <br>
> which employs a method reference
as argument<br>
> <br>
>
putMakedPasswordImplementations(this::putService,
this);<br>
<br>
The called method in turn calls
consumer.accept<br>
<br>
consumer.accept(new
Service(provider, <br>
PASSWORD_FACTORY_TYPE, algorithm, <br>
"org.wildfly.security.password.impl.PasswordFactorySpiImpl", emptyList,
<br>
emptyMap));<br>
<br>
which enters enters
MethodHandleNative::linkDynamicConstant()<br>
<br>
> Debugging shows that the call to
linkDynamicConstant() returns null.<br>
> <br>
> A simple reproducer for the
problem is available as a maven
project on <br>
> github:<br>
> <br>
> <a href="https://urldefense.com/v3/__https://github.com/tristantarrant/elytron-leyden__;!!ACWV5N9M2RV99hQ!MnS3LSY2PXCCXbHvdyMfR6Nj57XH7Ey7aZuOLmQ__hhE_RhUrKnkzZ0uP8AwLZYcl8lAhhZaZr1akQ$" rel="noreferrer" target="_blank" moz-do-not-send="true">https://github.com/tristantarrant/elytron-leyden</a><br>
> <br>
> The ReadMe provides an
explanation of how to reproduce the
problem. I <br>
> did so and the debugged to find
out some of the details of what is <br>
> happening (see below) but did not
fully clarify the problem. Help from <br>
> someone more conversant with the
ins and outs of method handle <br>
> bootstraps in premain would be
welcome. Details follow.<br>
> <br>
> regards,<br>
> <br>
> <br>
> Andrew Dinn<br>
> -----------<br>
> <br>
> I downloaded the git repo and
attached the Java sources using Maven
command<br>
> <br>
> $ mvn dependency:sources<br>
> <br>
> Having manifested the crash by
following the instructions in the
README <br>
> I reran the leyden JVM under gdb
using the following commands to enable
<br>
> Java debugging<br>
> <br>
> $ gdb ${LEYDEN_HOME}/bin/java<br>
> (gdb) cd /path/to/mvn/project<br>
> (gdb) run <br>
>
-agentlib:jdwp=transport=dt_socket,server=y,suspend=y,address=5005
<br>
> -classpath <br>
>
/home/adinn/redhat/openjdk/infinispan/elytron-leyden/base/target/elytron-leyden-base-0.0.1-SNAPSHOT.jar:/home/adinn/.m2/repository/org/wildfly/security/wildfly-elytron-credential/<a href="https://urldefense.com/v3/__http://2.5.1.__;!!ACWV5N9M2RV99hQ!MnS3LSY2PXCCXbHvdyMfR6Nj57XH7Ey7aZuOLmQ__hhE_RhUrKnkzZ0uP8AwLZYcl8lAhhauJ-RjcQ$" target="_blank" moz-do-not-send="true">2.5.1.</a>Final/wildfly-elytron-credential-2.5.1.Final.jar:/home/adinn/.m2/repository/org/wildfly/security/wildfly-elytron-base/2.5.1.Final/wildfly-elytron-base-2.5.1.Final.jar
-XX:CacheDataStore=elytron.aot
com.redhat.leyden.Main<br>
> <br>
> The problem manifests at
WildflyElytronBaseProvider.java:112 in
method <br>
>
WildflyElytronBaseProvider::putMakedPasswordImplementations<br>
<br>
static void
putMakedPasswordImplementations(Consumer<Service>
<br>
consumer, Provider provider) {<br>
for (String algorithm :
MASKED_ALGORITHMS) {<br>
consumer.accept(new
Service(provider, <br>
PASSWORD_FACTORY_TYPE, algorithm, <br>
"org.wildfly.security.password.impl.PasswordFactorySpiImpl", emptyList,
<br>
emptyMap)); <== NPE under this call<br>
}<br>
<br>
<br>
> The source code for this method
can be found in the following source
jar<br>
> <br>
> <br>
>
${M2_REPO}/org/wildfly/security/wildfly-elytron-base/2.5.1.Final/wildfly-elytron-base-2.5.1.Final-sources.jar<br>
> <br>
> (where M2_REPO will normally be
~/.m2/repository)<br>
> <br>
> Stepping into accept eventually
enters
MethodHandleNative::linkDynamicConstant
<br>
> which in turn enters into
ConstantBootstraps.makeConstant(). The
caller <br>
> Class at this point is a lambda
class which prints as <br>
>
org.wildfly.security.WildflyElytronBaseProvider$$Lambda/0x800000000c<br>
> <br>
> Several steps further the code
enters BootstrapMethodInvoker::invoke
<br>
> (below the app method call but
via 3 hidden frames) with
bootstrapMethod <br>
> bound to a DirectMethodHandle.
After several more steps this enters <br>
>
DirectMethodHandle$Holder.invokeStatic
which in turn calls <br>
>
MethodHandles::classData(Lookup,String,Class).<br>
> <br>
> At this point caller is a
MethodHandleLookup for the lambda
class <br>
> Lambda/0x800000000c mentioned
above. The following call<br>
> <br>
> Object classdata =
classData(caller.lookupClass());<br>
> <br>
> returns null to
DirectMethodHandle$Holder.invokeStatic
which pops the <br>
> same result back out to
BootstrapMethodInvoker::invoke at line
90<br>
> <br>
> if (type
instanceof Class<?> c) {<br>
> result =
bootstrapMethod.invoke(caller, name,
c); <br>
> <== null<br>
> <br>
> This null result pops back out as
the value for the call to <br>
> BootstrapMethodInvoker.invoke(),
ConstantBootstraps.makeConstant() and
<br>
>
MethodHandleNative::linkDynamicConstant().<br>
> <br>
<br>
</blockquote>
</div>
</blockquote>
</div>
</blockquote>
</div>
</blockquote>
</div>
</blockquote>
</div>
</blockquote>
</div>
</blockquote>
</div>
</blockquote>
</body>
</html>