RFR: 8375209: Strengthen Xcheck:jni for JNI critical regions [v3]

[Aleksey Shipilev]

> [JDK-8375188](https://bugs.openjdk.org/browse/JDK-8375188) shows that misuse for JNI critical is able to deadlock the GC. Therefore, I think we really want to have `-Xcheck:jni` to preempt these cases and allow us to diagnose the issues in the field.
> 
> G1 and Shenandoah are not exactly affected by this deadlock. ZGC never had this check, AFAICS, and is affected by the deadlock. We used to have some capability in Serial/Parallel prior to [JDK-8192647](https://bugs.openjdk.org/browse/JDK-8192647), AFAICS: https://github.com/openjdk/jdk/commit/a9c9f7f0cbb2f2395fef08348bf867ffa8875d73#diff-d27fc793db1bf9314b322d494cd1c3269629fe27a605b4441de08d543d020fc3L341-L344
> 
> Additional testing:
>  - [x] New test, 100x repetitions
>  - [ ] Linux x86_64 server fastdebug, `all` with `-XX:+UseParallelGC -Xcheck:jni`

Aleksey Shipilev has updated the pull request incrementally with one additional commit since the last revision:

  Capture bad state at the end of JNI transition, check criticality on JNI function enters
  Move the check to native->Java transition

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/29206/files
  - new: https://git.openjdk.org/jdk/pull/29206/files/4eeb4a04..765ef9d0

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=29206&range=02
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=29206&range=01-02

  Stats: 46 lines in 8 files changed: 26 ins; 9 del; 11 mod
  Patch: https://git.openjdk.org/jdk/pull/29206.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/29206/head:pull/29206

PR: https://git.openjdk.org/jdk/pull/29206