RFR(S) 8238585: Use handshake for JvmtiEventControllerPrivate::enter_interp_only_mode() and don't make compiled methods on stack not_entrant

[Vladimir Ivanov]

> Webrev: http://cr.openjdk.java.net/~rrich/webrevs/8238585/webrev.0/

Not an expert in JVMTI code base, so can't comment on the actual changes.

 From JIT-compilers perspective it looks good.

Best regards,
Vladimir Ivanov

> Bug:    https://bugs.openjdk.java.net/browse/JDK-8238585
> 
> The change avoids making all compiled methods on stack not_entrant when switching a java thread to
> interpreter only execution for jvmti purposes. It is sufficient to deoptimize the compiled frames on stack.
> 
> Additionally a handshake is used instead of a vm operation to walk the stack and do the deoptimizations.
> 
> Testing: JCK and JTREG tests, also in Xcomp mode with fastdebug and release builds on all platforms.
> 
> Thanks, Richard.
> 
> See also my question if anyone knows a reason for making the compiled methods not_entrant:
> http://mail.openjdk.java.net/pipermail/serviceability-dev/2020-January/030339.html
>