RFR: 8247992: [JVMCI] HotSpotNmethod.executeVarargs can try execute a zombie nmethod

[Doug Simon]

Hi, can I please get some reviews for this patch that fixes a race described in JDK-8247992 <https://bugs.openjdk.java.net/browse/JDK-8247992>.

The fix in the patch is to pass the HotSpotNmethod mirror object (in a handle) through the problematic VM-to-Java transition and only after the transition will the verified entry point be extracted from the mirror. This will prevent a zombie nmethod from being executed.

It’s still possible for the call to not reach the “alternative target” due to sweeping heuristics. This is an acceptable limitation of having an alternative call target in JavaCallWrapper. It could be fixed by altering CPU specific i2c trampoline code but the added complexity is not worth it for this test-only mechanism. Tests using this API can detect whether the alternative target was actually called if they really care about it.

Thanks to Erik Osterlund for the in-depth analysis and suggested solutions.

https://dougxc.github.io/webrevs/8247992_16/index.html
https://bugs.openjdk.java.net/browse/JDK-8247992

Testing: hs-tier1,hs-tier2,hs-tier3-graal

-Doug