Unsafe and JPMS

[Alan Bateman]

Moving this thread to jigsaw-dev as that is where this topic has been 
discussed ad nauseam.

On 19/01/2018 01:28, Jeremy Manson wrote:
> Hey folks,
>
> I know this has come up before, but I wanted to revive the question.  As we
> play more and more with JPMS and Java 9, we're finding more and more places
> where Unsafe is now the only feasible way to achieve what we want to
> achieve.
>
> The big use case is, of course, access to the internals of modularized
> libraries in ways that would have used standard reflection in Java 8 and
> earlier, but for which we now fail with an exception.
Are they really failing with exceptions? I ask because the standard and 
JDK modules open most of their packages in JDK 9 and JDK 10 for 
so-called "deep reflection". If there are libraries on the class path 
hacking into JDK internals then they have a good chance that they will 
continue to work, assuming the internals haven't changed. There may be a 
warning of course but that is to help identify code that may break in 
the future.

>
> The answer of adding command line flags doesn't really work for general
> purpose libraries.  If I want to do this in (for example) Guava, asking
> every Guava user to add add-opens on the command line is a complete
> non-starter.  Many people have no idea what libraries they are using.
>
> The answer of creating an agent is also a non-starter for similar reasons.
>
> The answer of using privateLookupIn assumes that you have access to or
> control over the target class, which is often not the case, especially if
> you are introspecting into JDK internals.
>
> As people within Google try Java 9, this is coming up in quite a bit of our
> infrastructure, especially in diagnostic code.  The latest place I've found
> it was a tool that printed out / counted the number of objects reachable
> from a root object; it died when it got into a JDK library.
JVM TI is the API for walking the heap of a running VM. That should work 
as before.

Diagnostic tools doing heap walking in Java feels like something for a 
java agent rather than a general purpose library on the class path 
(ignoring serialization for now as that's a discussion in itself). If 
the tooling is using reflection to potentially access every field of 
every object on the heap then it may need changes (maybe now if it finds 
itself walking references to objects of classes in packages that are new 
in JDK 9). The Instrumentation API has the power to do that of course as 
it can open any package to anyone. This means it can get full-power 
Lookup to any class in the JDK with privateLookupIn. Agents need to 
guard this capability closely of course.

As regards starting java agents then one addition in JDK 9 to look at is 
the Launcher-Agent-Class attribute that executable JAR files including 
an agent can use.

> :
>
> So... OpenJDK has this commitment not to replace Unsafe without providing
> supported replacements for its functionality and a transition period.  Does
> that include the functionality that Unsafe can break module encapsulation?
I'm not aware of any current proposals to remove Unsafe. I'm sure 
degrading Unsafe will come up once Panama and maybe Valhalla are further 
along.

For now, I think the right thing is to continue the effort to identify 
candidate APIs (where it makes sense) so that code hacking into the JDK 
today can migrate in the future. This was the motivation for many new 
APIs in JDK 9 and I've no doubt there will be more before the JDK 
modules are fully encapsulated.

-Alan.