Clarification on what the JNI spec means by nonmovable

[Krystal Mok]

Hi Jon,

On Thu, Sep 8, 2016 at 9:30 AM, Jon V. <sybersnake at gmail.com> wrote:

> Hello everyone!
>
> I’m trying to document the exact behavior of direct pointer access of
> passed arguments into JNI in HotSpot.  I have a huge post on StackOverflow
> for posterity purposes as it seems to be a commonly misunderstood behavior.
>
> http://stackoverflow.com/questions/39381339/understanding-safe-access-of-
> jni-arguments
>
> from
> https://docs.oracle.com/javase/7/docs/technotes/
> guides/jni/spec/design.html#wp16789
>
> The JNI spec says, “To implement local references, the Java VM creates a
> registry for each transition of control from Java to a native method. A
> registry maps nonmovable local references to Java objects, and keeps the
> objects from being garbage collected.”
>
> "Nonmovable" refers to the "local JNI reference", not the Java object
instance itself. What that gurantees is that, given a jobject (which is
typedef'd as an opaque pointer), you can safely say that as long as this
jobject is still in scope, the value of the jobject won't change. But that
doesn't imply anything related whether or not the Java object it's
referring to is pinned.

The "registry map" here is just an abstract notion that doesn't necessarily
materialize into any real data structures in some certain JVM
implementations.


> I’m trying to understand if “nonmovable” actually means that the objects in
> the map WILL NOT be compacted/moved (heap memory) or if it just means they
> won’t be garbage-collected.  The answer dictates if the use of critical
> sections are actually necessary.
>
> Rule of thumb: don't try to bypass the JNI abstractions. It'll bite back
hard if you switch between different JVM implementations.

Specifically in HotSpot, jobject and friends are referred to as "JNI
handles". That's because they're implemented as handles, i.e.
double-indirection pointers. The underlying type that implements jobject is
"oop*", where an "oop" is typedef'd from "oopDesc*", and oopDesc is the
root type of garbage collected objects.

jobject    JNIHandle   Java object
[ oop* ] -> [ oop ] -> [ oopDesc ]

Following the spec, what it guarantees is that for a given jobject, the
JNIHandle representing the referent object will not be moved, but the
actual referent object is free to move.

Don't try to access raw oops from outside the VM by accessing the
underlying handle internals. It's very likely to get dangling pointers
after GCs.

What are the effects for each garbage collector?
>
> All garbage collectors in HotSpot, except for CMS (mostly concurrent
mark-sweep), are moving collectors. As such, after each completed
collection, Java objects will move. If you're holding raw oops that the GC
isn't aware of, they won't be updated during GC, and will be left dangling
afterwards.

JNI critical sections are implemented in HotSpot via temporarily disabling
the GCs, and as soon as all JNI critical sections are completed, a full GC
(by default) will be triggered to perform the delayed collection if needed.
This mechanism is called the "GCLocker" in HotSpot.
(GCs may be able to expand the heap to fulfill allocation requests, if the
current capacity of the GC heap is smaller than the maximum capacity
configured. So disabling the GC isn't always as bad as it sounds.)


> I found an email on the archives that says that JNI critical sections do
> not effect operation of CMS and I’m interested in understanding the
> behavior for G1 as well.
>
> CMS is a mark-sweep collector, so it doesn't move objects. Thus, it is
possible to perform CMS old gen collections even when the GCLocker is
active (meaning there's at least one Java thread in a JNI critical section).

G1, on the other hand, is a moving collector (an incremental copying
collector with optional concurrent global marking; sometimes simply
referred to as a concurrent mark-compact collector, but it's not the usual
mark-compact notion). G1 GC cannot be performed when the GCLocker is active.


> This all boils down to the ability to use unsafe raw direct pointers in JNI
> without critical sections in HotSpot.
>
> Nope. Don't try that on HotSpot.
There are certain code patterns that might be able to access raw oops
safely without going into JNI critical sections. But that involves very
heavy VM internals knowledge, which is also subject to change without
notice, so practically don't try that.

Hope that helps,
Kris (OpenJDK username: kmo)


> Thank you,
> J
>