Clarification on what the JNI spec means by nonmovable

[Krystal Mok]

Hi Peter,

Thanks for the follow-up!

I did mean to say "CMS the old gen collector" though. I should have made it
clear that when "CMS the old gen collector" is chosen, ParNew is
automatically chosen to be the young gen collector, and a mark-compact
collector ("Serial Old" in current OpenJDK HotSpot, parallelized in some
other variants of HotSpot) is chosen as the fallback full GC collector. All
of the above, except for "CMS the old gen collector", are moving collectors.

It certainly doesn't change anything as to how JNI handles should be used
by external code. To paraphrase Peter's words: there's no reliable way from
the outside to control whether a young/minor GC, a concurrent old gen GC,
or a fallback full GC is going to run.

Thanks,
Kris

On Thu, Sep 8, 2016 at 11:27 AM, Peter B. Kessler <
Peter.B.Kessler at oracle.com> wrote:

> A CMS young generation collection will move objects.  I don't think that
> changes the way you should use JNI handles.  But I didn't want people
> switching to CMS because they thought it was a non-moving collector and
> they could cheat.
>
>                         ... peter
>
>
> On 09/ 8/16 10:30 AM, Krystal Mok wrote:
>
>> Hi Jon,
>>
>> On Thu, Sep 8, 2016 at 9:30 AM, Jon V. <sybersnake at gmail.com> wrote:
>>
>> Hello everyone!
>>>
>>> I’m trying to document the exact behavior of direct pointer access of
>>> passed arguments into JNI in HotSpot.  I have a huge post on
>>> StackOverflow
>>> for posterity purposes as it seems to be a commonly misunderstood
>>> behavior.
>>>
>>> http://stackoverflow.com/questions/39381339/understanding-
>>> safe-access-of-
>>> jni-arguments
>>>
>>> from
>>> https://docs.oracle.com/javase/7/docs/technotes/
>>> guides/jni/spec/design.html#wp16789
>>>
>>> The JNI spec says, “To implement local references, the Java VM creates a
>>> registry for each transition of control from Java to a native method. A
>>> registry maps nonmovable local references to Java objects, and keeps the
>>> objects from being garbage collected.”
>>>
>>> "Nonmovable" refers to the "local JNI reference", not the Java object
>>>
>> instance itself. What that gurantees is that, given a jobject (which is
>> typedef'd as an opaque pointer), you can safely say that as long as this
>> jobject is still in scope, the value of the jobject won't change. But that
>> doesn't imply anything related whether or not the Java object it's
>> referring to is pinned.
>>
>> The "registry map" here is just an abstract notion that doesn't
>> necessarily
>> materialize into any real data structures in some certain JVM
>> implementations.
>>
>>
>> I’m trying to understand if “nonmovable” actually means that the objects
>>> in
>>> the map WILL NOT be compacted/moved (heap memory) or if it just means
>>> they
>>> won’t be garbage-collected.  The answer dictates if the use of critical
>>> sections are actually necessary.
>>>
>>> Rule of thumb: don't try to bypass the JNI abstractions. It'll bite back
>>>
>> hard if you switch between different JVM implementations.
>>
>> Specifically in HotSpot, jobject and friends are referred to as "JNI
>> handles". That's because they're implemented as handles, i.e.
>> double-indirection pointers. The underlying type that implements jobject
>> is
>> "oop*", where an "oop" is typedef'd from "oopDesc*", and oopDesc is the
>> root type of garbage collected objects.
>>
>> jobject    JNIHandle   Java object
>> [ oop* ] -> [ oop ] -> [ oopDesc ]
>>
>> Following the spec, what it guarantees is that for a given jobject, the
>> JNIHandle representing the referent object will not be moved, but the
>> actual referent object is free to move.
>>
>> Don't try to access raw oops from outside the VM by accessing the
>> underlying handle internals. It's very likely to get dangling pointers
>> after GCs.
>>
>> What are the effects for each garbage collector?
>>
>>>
>>> All garbage collectors in HotSpot, except for CMS (mostly concurrent
>>>
>> mark-sweep), are moving collectors. As such, after each completed
>> collection, Java objects will move. If you're holding raw oops that the GC
>> isn't aware of, they won't be updated during GC, and will be left dangling
>> afterwards.
>>
>> JNI critical sections are implemented in HotSpot via temporarily disabling
>> the GCs, and as soon as all JNI critical sections are completed, a full GC
>> (by default) will be triggered to perform the delayed collection if
>> needed.
>> This mechanism is called the "GCLocker" in HotSpot.
>> (GCs may be able to expand the heap to fulfill allocation requests, if the
>> current capacity of the GC heap is smaller than the maximum capacity
>> configured. So disabling the GC isn't always as bad as it sounds.)
>>
>>
>> I found an email on the archives that says that JNI critical sections do
>>> not effect operation of CMS and I’m interested in understanding the
>>> behavior for G1 as well.
>>>
>>> CMS is a mark-sweep collector, so it doesn't move objects. Thus, it is
>>>
>> possible to perform CMS old gen collections even when the GCLocker is
>> active (meaning there's at least one Java thread in a JNI critical
>> section).
>>
>> G1, on the other hand, is a moving collector (an incremental copying
>> collector with optional concurrent global marking; sometimes simply
>> referred to as a concurrent mark-compact collector, but it's not the usual
>> mark-compact notion). G1 GC cannot be performed when the GCLocker is
>> active.
>>
>>
>> This all boils down to the ability to use unsafe raw direct pointers in
>>> JNI
>>> without critical sections in HotSpot.
>>>
>>> Nope. Don't try that on HotSpot.
>>>
>> There are certain code patterns that might be able to access raw oops
>> safely without going into JNI critical sections. But that involves very
>> heavy VM internals knowledge, which is also subject to change without
>> notice, so practically don't try that.
>>
>> Hope that helps,
>> Kris (OpenJDK username: kmo)
>>
>>
>> Thank you,
>>> J
>>>
>>>