Unsafe and JPMS

[Jeremy Manson]

On Fri, Jan 19, 2018 at 5:05 AM, Alan Bateman <Alan.Bateman at oracle.com>
wrote:

> Moving this thread to jigsaw-dev as that is where this topic has been
> discussed ad nauseam.


Sorry to beat this to death.  The engineering time we're spending on the
Java 9 transition is very costly.  I have a responsibility to make sure I'm
doing what I can to mitigate future costs without concomitant benefits.

On 19/01/2018 01:28, Jeremy Manson wrote:
>
>> Hey folks,
>>
>> I know this has come up before, but I wanted to revive the question.  As
>> we
>> play more and more with JPMS and Java 9, we're finding more and more
>> places
>> where Unsafe is now the only feasible way to achieve what we want to
>> achieve.
>>
>> The big use case is, of course, access to the internals of modularized
>> libraries in ways that would have used standard reflection in Java 8 and
>> earlier, but for which we now fail with an exception.
>>
> Are they really failing with exceptions? I ask because the standard and
> JDK modules open most of their packages in JDK 9 and JDK 10 for so-called
> "deep reflection". If there are libraries on the class path hacking into
> JDK internals then they have a good chance that they will continue to work,
> assuming the internals haven't changed. There may be a warning of course
> but that is to help identify code that may break in the future.



1)
testStaticsDiscovery(com.google.common.profile.HeapInspectorTest)java.lang.reflect.InaccessibleObjectException:
Unable to make field final jdk.internal.loader.URLClassPath
jdk.internal.loader.ClassLoaders$AppClassLoader.ucp accessible: module
java.base does not "opens jdk.internal.loader" to unnamed module @3e11f9e9;
did you mean --add-opens=java.base/jdk.internal.loader=ALL-UNNAMED
at
java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:345)
at
java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:281)
at java.base/java.lang.reflect.Field.checkCanSetAccessible(Field.java:176)
at java.base/java.lang.reflect.Field.setAccessible(Field.java:170)
at
com.google.common.profile.HeapInspector.getClassFields(HeapInspector.java:1234)


> The answer of adding command line flags doesn't really work for general
>> purpose libraries.  If I want to do this in (for example) Guava, asking
>> every Guava user to add add-opens on the command line is a complete
>> non-starter.  Many people have no idea what libraries they are using.
>>
>> The answer of creating an agent is also a non-starter for similar reasons.
>>
>> The answer of using privateLookupIn assumes that you have access to or
>> control over the target class, which is often not the case, especially if
>> you are introspecting into JDK internals.
>>
>> As people within Google try Java 9, this is coming up in quite a bit of
>> our
>> infrastructure, especially in diagnostic code.  The latest place I've
>> found
>> it was a tool that printed out / counted the number of objects reachable
>> from a root object; it died when it got into a JDK library.
>>
> JVM TI is the API for walking the heap of a running VM. That should work
> as before.
>
> Diagnostic tools doing heap walking in Java feels like something for a
> java agent rather than a general purpose library on the class path
> (ignoring serialization for now as that's a discussion in itself). If the
> tooling is using reflection to potentially access every field of every
> object on the heap then it may need changes (maybe now if it finds itself
> walking references to objects of classes in packages that are new in JDK
> 9). The Instrumentation API has the power to do that of course as it can
> open any package to anyone. This means it can get full-power Lookup to any
> class in the JDK with privateLookupIn. Agents need to guard this capability
> closely of course.

As regards starting java agents then one addition in JDK 9 to look at is
> the Launcher-Agent-Class attribute that executable JAR files including an
> agent can use.
>

If Launcher-Agent-Class is for executable JAR files, then that's basically
a non-starter for library classes, too.

Also, it's worth pointing out that even if it feels wrong to you, it has
worked in the platform for many years - the cost I'm talking about was
written for Java 4, and hasn't needed to be changed since then.

I also think that diagnostic code is a red herring here.  The OSS use cases
I pointed to are not in diagnostic code.

So... OpenJDK has this commitment not to replace Unsafe without providing
>> supported replacements for its functionality and a transition period.
>> Does
>> that include the functionality that Unsafe can break module encapsulation?
>>
> I'm not aware of any current proposals to remove Unsafe. I'm sure
> degrading Unsafe will come up once Panama and maybe Valhalla are further
> along.
>

This was a goal until JEP 260 came along.  My read on this was that it was
simply a matter of time before it came up again.

For now, I think the right thing is to continue the effort to identify
> candidate APIs (where it makes sense) so that code hacking into the JDK
> today can migrate in the future. This was the motivation for many new APIs
> in JDK 9 and I've no doubt there will be more before the JDK modules are
> fully encapsulated.


I generally agree with the principle of making real APIs to do what we need
to do with Unsafe and migrating.  My concern is not the removal of Unsafe,
it's that the replacement won't be equivalent if it doesn't let you cross
module boundaries.

Jeremy