Java 11 vs 14 MethodHandle behavior change
Hi, in Jetty 10, we use method handles in the WebSocket implementation. We need to scan WebSocket EndPoint application classes (MyEndPoint) for annotated methods whose signature may vary and may take application classes, for example `@OnMessage void onMessage(Session s, MyString m)` where @OnMessage and Session are JSR 356 classes and MyEndPoint and MyString are application classes. It may be possible that the same web application is deployed twice, perhaps with different configurations. Therefore we have: * class "Impl" is the Jetty implementation class that creates MethodHandle.Lookup, loaded by serverClassLoader and in a named JPMS module * class MyEndPoint and MyString are in webapp1's WEB-INF/classes, loaded by webClassLoader1 in an unnamed JPMS module * class MyEndPoint and MyString are also in webapp2's WEB-INF/classes, loaded by webClassLoader2 in another unnamed JPMS module Class Impl does this: MethodHandles.publicLookup().in(endPointClass).findVirtual(...), where "endPointClass" is class MyEndPoint loaded by the web class loader (either webClassLoader1 or webClassLoader2). We must call .in() to separate the 2 web applications (failing to call .in() results in linkage errors). Calling publicLookup().in() works in Java 11-13. In Java 14, it does not work anymore. The reason seems to stem from the fixes for https://bugs.openjdk.java.net/browse/JDK-8173978. In particular, in Java 14 publicLookup() returns a Lookup with mode=UNCONDITIONAL (but not PUBLIC? Is this intended?) and Lookup.lookupClassOrNull() returns Object.class rather than the class passed to .in() - like it was doing in Java 11. We have debugged and manually changed the return value of lookupClassOrNull() from Object.class to the endPoint class, and it works again. We have written a simple reproducer and been able to workaround the issue (we need to test it better in Jetty to confirm that the workaround for the simple reproducer also works in Jetty) by changing the Lookup from: publicLookup().in() to: lookup().dropLookupMode(<all apart PUBLIC>).in() and it also works. Note that we really want to have a Lookup object with the minimum set of "powers" to lookup public methods in public classes of the web application - otherwise the JPMS configuration becomes more complicated. We do have a simple reproducer that works in Java 11 and fails in 14. I am a JDK author so I can open an issue about this. Thanks! -- Simone Bordet --- Finally, no matter how good the architecture and design are, to deliver bug-free software with optimal performance and reliability, the implementation technique must be flawless. Victoria Livschitz
Hi Simone, On 4/28/20 5:42 AM, Simone Bordet wrote:
Hi,
in Jetty 10, we use method handles in the WebSocket implementation.
We need to scan WebSocket EndPoint application classes (MyEndPoint) for annotated methods whose signature may vary and may take application classes, for example `@OnMessage void onMessage(Session s, MyString m)` where @OnMessage and Session are JSR 356 classes and MyEndPoint and MyString are application classes.
It may be possible that the same web application is deployed twice, perhaps with different configurations.
Therefore we have: * class "Impl" is the Jetty implementation class that creates MethodHandle.Lookup, loaded by serverClassLoader and in a named JPMS module * class MyEndPoint and MyString are in webapp1's WEB-INF/classes, loaded by webClassLoader1 in an unnamed JPMS module * class MyEndPoint and MyString are also in webapp2's WEB-INF/classes, loaded by webClassLoader2 in another unnamed JPMS module
Class Impl does this: MethodHandles.publicLookup().in(endPointClass).findVirtual(...), where "endPointClass" is class MyEndPoint loaded by the web class loader (either webClassLoader1 or webClassLoader2).
endPointClass is in unnamed module and so it's unconditionally exported. The public lookup should be able to find public members from it. One thing to double check if endPointClass is publicly accessible?
We must call .in() to separate the 2 web applications (failing to call .in() results in linkage errors).
Calling publicLookup().in() works in Java 11-13.
In Java 14, it does not work anymore. The reason seems to stem from the fixes for https://bugs.openjdk.java.net/browse/JDK-8173978.
Did you get any exception in 14? Is it from findVirtual or from in?
In particular, in Java 14 publicLookup() returns a Lookup with mode=UNCONDITIONAL (but not PUBLIC? Is this intended?)
Yes this is intentional. The new public lookup returned has endPointClass as the lookup class (which should not have behavioral change). The access modes are summarized in the javadoc: https://download.java.net/java/early_access/jdk15/docs/api/java.base/java/la... However, there is no behavioral change in public lookup. Any lookup object with UNCONDITIONAL access mode assume readability and also finds all public accessible members as in Java 9. See also CSR JDK-8226916.
and Lookup.lookupClassOrNull() returns Object.class rather than the class passed to .in() - like it was doing in Java 11.
We have debugged and manually changed the return value of lookupClassOrNull() from Object.class to the endPoint class, and it works again.
The current implementation already does that. private Class<?> lookupClassOrNull() { if (allowedModes == TRUSTED) { return null; } if (allowedModes == UNCONDITIONAL) { // use Object as the caller to pass to VM doing resolution return Object.class; } return lookupClass; } What exactly have you changed?
We have written a simple reproducer and been able to workaround the issue (we need to test it better in Jetty to confirm that the workaround for the simple reproducer also works in Jetty) by changing the Lookup from: publicLookup().in() to: lookup().dropLookupMode(<all apart PUBLIC>).in() and it also works.
Note that we really want to have a Lookup object with the minimum set of "powers" to lookup public methods in public classes of the web application - otherwise the JPMS configuration becomes more complicated.
We do have a simple reproducer that works in Java 11 and fails in 14. I am a JDK author so I can open an issue about this.
Yes, please file a JBS issue and I will look into it. If the requested target class to be accessed through Lookup::in is exported, it should work because the set of classes that public lookups can access should not change. thanks Mandy
Hi, On Tue, Apr 28, 2020 at 7:34 PM Mandy Chung <mandy.chung@oracle.com> wrote:
endPointClass is in unnamed module and so it's unconditionally exported. The public lookup should be able to find public members from it. One thing to double check if endPointClass is publicly accessible?
It is.
Did you get any exception in 14? Is it from findVirtual or from in?
From findVirtual():
Exception in thread "main" java.lang.IllegalAccessException: no such method: org.module1.MyEndPoint.onMessage(MyString)void/invokeVirtual at java.base/java.lang.invoke.MemberName.makeAccessException(MemberName.java:971) at java.base/java.lang.invoke.MemberName$Factory.resolveOrFail(MemberName.java:1114) at java.base/java.lang.invoke.MethodHandles$Lookup.resolveOrFail(MethodHandles.java:2785) at java.base/java.lang.invoke.MethodHandles$Lookup.findVirtual(MethodHandles.java:1883) at org.module2.Main.main(Main.java:28) Caused by: java.lang.LinkageError: loader constraint violation: when resolving method 'void org.module1.MyEndPoint.onMessage(org.module1.MyString)' the class loader 'bootstrap' of the current class, java/lang/Object, and the class loader java.net.URLClassLoader @7291c18f for the method's defining class, org/module1/MyEndPoint, have different Class objects for the type org/module1/MyString used in the signature (java.lang.Object is in module java.base of loader 'bootstrap'; org.module1.MyEndPoint is in unnamed module of loader java.net.URLClassLoader @7291c18f, parent loader 'app') at java.base/java.lang.invoke.MethodHandleNatives.resolve(Native Method) at java.base/java.lang.invoke.MemberName$Factory.resolve(MemberName.java:1084) at java.base/java.lang.invoke.MemberName$Factory.resolveOrFail(MemberName.java:1111) ... 3 more
The current implementation already does that.
private Class<?> lookupClassOrNull() { if (allowedModes == TRUSTED) { return null; } if (allowedModes == UNCONDITIONAL) { // use Object as the caller to pass to VM doing resolution return Object.class; } return lookupClass; }
What exactly have you changed?
if (allowedModes == UNCONDITIONAL) { return lookupClass; }
Yes, please file a JBS issue and I will look into it. If the requested target class to be accessed through Lookup::in is exported, it should work because the set of classes that public lookups can access should not change.
https://bugs.openjdk.java.net/browse/JDK-8244090 Thanks! -- Simone Bordet --- Finally, no matter how good the architecture and design are, to deliver bug-free software with optimal performance and reliability, the implementation technique must be flawless. Victoria Livschitz
On 4/29/20 2:07 AM, Simone Bordet wrote:
Did you get any exception in 14? Is it from findVirtual or from in? From findVirtual():
Exception in thread "main" java.lang.IllegalAccessException: no such method: org.module1.MyEndPoint.onMessage(MyString)void/invokeVirtual at java.base/java.lang.invoke.MemberName.makeAccessException(MemberName.java:971) at java.base/java.lang.invoke.MemberName$Factory.resolveOrFail(MemberName.java:1114) at java.base/java.lang.invoke.MethodHandles$Lookup.resolveOrFail(MethodHandles.java:2785) at java.base/java.lang.invoke.MethodHandles$Lookup.findVirtual(MethodHandles.java:1883) at org.module2.Main.main(Main.java:28) Caused by: java.lang.LinkageError: loader constraint violation: when resolving method 'void org.module1.MyEndPoint.onMessage(org.module1.MyString)' the class loader 'bootstrap' of the current class, java/lang/Object, and the class loader java.net.URLClassLoader @7291c18f for the method's defining class, org/module1/MyEndPoint, have different Class objects for the type org/module1/MyString used in the signature (java.lang.Object is in module java.base of loader 'bootstrap'; org.module1.MyEndPoint is in unnamed module of loader java.net.URLClassLoader @7291c18f, parent loader 'app') at java.base/java.lang.invoke.MethodHandleNatives.resolve(Native Method) at java.base/java.lang.invoke.MemberName$Factory.resolve(MemberName.java:1084) at java.base/java.lang.invoke.MemberName$Factory.resolveOrFail(MemberName.java:1111) ... 3 more
Thanks for the reproducer. `publicLookup().in(endPointClass1)` in JDK 13 produces a Lookup object on endPointClass1 with PUBLIC access only. UNCONDITIONAL bit was lost. The resulting Lookup can access classes unconditionally exported from module M that the module of endPointClass1 can read. The resulting Lookup is no longer a public lookup, i.e. the lookup context is the lookup class (i.e. endPointClass1) and the lookup mode (i.e. PUBLIC). `publicLookup().in(endPointClass1)` in JDK 14 produces a Lookup object on endPointClass1 with UNCONDITIONAL bit due to the spec change for JDK-8173978. The resulting Lookup remains to be a public lookup which can access any unconditionally exported classes from any module. Although the lookup class of the teleported public Lookup is endPointClass1, it should not affect the lookup context. Unfortunately, the workaround for JDK-8228671 causes this bug. The reproducer shows that the new public Lookup uses Object as the lookup clsas that violates the loader constraint. lookup1 finds "org.module1.MyEndPoint" loaded by CL1 and it adds a loader constraint with the boot loader (rather than CL1) due to the workaround fixed by JDK-8228671 that uses Object as the lookup class. When lookup2 finds "org.module1.MyEndPoint" loaded by CL2, the lookup fails with loader constraint violation.
The current implementation already does that.
private Class<?> lookupClassOrNull() { if (allowedModes == TRUSTED) { return null; } if (allowedModes == UNCONDITIONAL) { // use Object as the caller to pass to VM doing resolution return Object.class; } return lookupClass; }
What exactly have you changed? if (allowedModes == UNCONDITIONAL) { return lookupClass; }
This patch will hit the assertion that JDK-8228671 ran into. We need a long-term fix (perhaps to look into JDK-8173977)
Thanks. I will look into it. Mandy
participants (2)
-
Mandy Chung
-
Simone Bordet