RFR: Make object pinning safepoint aware

[Roman Kennke]

Am 15.10.2017 um 04:28 schrieb Zhengyu Gu:
> Thread running native code does not usually participate safepoints, 
> but the write barrier in object pinning code can potential interfere 
> with safepoints and mutate the heap, so we need to make it safepoint 
> aware.
>
> Webrev: 
> http://cr.openjdk.java.net/~zgu/shenandoah/obj_pin_safepoint/webrev.00/index.html
>
> Test:
>   hotspot_gc_shenandoah fastdebug + release on x64 and aarch64.
>   + new stress test.
>
> Thanks,
>
> -Zhengyu

This doesn't seem right.

Native code (i.e. *outside* a JNI call) does not participate in 
safepoints. But as soon as we are *inside* a JNI call, i.e. 
GetPrimitiveArrayCritical, the thread is in VM state. This means it does 
participate in safepoint in that it cannot reach a safepoint unless we 
explicitely tell it to. This should be good for us: we will not get any 
nasty surprises (e.g. change of evac-in-progress flag on the fly) in our 
write barrier for example.

Now you introduce a VM->java->VM state transition inside the JNI 
function. This seems wrong. And what does it help? The thread can now 
reach a safepoint in the middle of GetPrimitiveArrayCritical: ugh. I 
doubt very much that this is the right fix. (And besides, there are 
scoped objects to do that...)

Can you describe in more detail what the problem is, e.g. you must have 
seen a crash or wrong behaviour in a testcase or real program? How does 
it manifest? Then we can look for a solution.

Roman