RFR: 8258032: Reconsider LEAF entry restrictions

[Coleen Phillimore]

On Thu, 7 Jan 2021 21:43:54 GMT, Daniel D. Daugherty <dcubed at openjdk.org> wrote:

>> Please review the smaller version of this enhancement.  JRT_LEAF can allow Handles.  Fixed places where a ResetNoHandleMark was added to workaround the restriction and remove some obsolete comments in deoptimization (the function is a JRT_BLOCK_ENTRY that has a HandleMarkCleaner which cleans up the HandleMark to the one before call_stub() in JavaCalls. A HandleMarkCleaner also allows HandleMark and Handles.
>> 
>> Tested with tier1-3, 4-6 with the larger change and 7-8 in progress.
>
> Like the original changes, the most interesting stuff is in
> interfaceSupport.inline.hpp. Just to keep it clear in my head:
> - remove NoHandleMark helper from VM_LEAF_BASE; it gets added to JNI_LEAF and JVM_LEAF which need it; JRT_LEAF doesn't need it.
> - remove ResetNoHandleMark helper from VM_ENTRY_BASE_FROM_LEAF since ThreadInVMfromNative will have one with your fix for JDK-8259374.
> 
> Please see my comment in JVM_ENTRY_FROM_LEAF.

JRT_LEAF still has a NoSafepointVerifier if that helps.  There's a comment about the things it's not allowed to do in interfaceSupport.inline.hpp.  The things it's not supposed to do are all checked by the NSV.
One of the things it can do now (especially if it's not going to safepoint) is to have a Handle with an enclosing HandleMark because GC won't touch the oops anyway.  This is one of the things @pron ran into with loom (see issue) and @fisk and I discussed.
I originally took the bug because I was going to close it as WNF, because I hate removing restrictions.
I do think there is a bug lurking here.  If you have a Handle without an enclosing HandleMark in JRT_LEAF, it will not be cleaned up when the leaf function returns.  This might need a HandleMarkCleaner.

-------------

PR: https://git.openjdk.java.net/jdk/pull/1990