VM_EnterInterpOnlyMode: why make compiled methods on stack not_entrant?

[Vladimir Ivanov]

Hi Richard,

> I noticed that VM_EnterInterpOnlyMode makes all compiled methods on stack not_entrant. To me this
> seems unnecessary. I think it would be sufficient to patch the return pc of compiled frames and
> let them return to their deopt handler. Or what would be the reason to make the compiled methods
> not_entrant?
> 
> VM_EnterInterpOnlyMode::doit(): all compiled methods on stack that are not native methods get
> marked.
> 
> Deoptimization::deoptimize_all_marked(): all marked methods are made not_entrant.
> 
> Maybe this is for historical reasons? If I remember correctly, many years ago, deoptimizing a
> compiled frame used to require making the corresponding nmethod not_entrant. And making a nmethod
> not_entrant meant overriding its code with a slide of nops that led to the deopt handler.
> 
> I'd like to change this and just apply Deoptimization::deoptimize() on every compiled frame on
> stack. I've done this locally and tested the change without issues.

I'm not a JVMTI expert, but considering VM_EnterInterpOnlyMode is 
performed on per-thread basis [1], it looks like per-frame 
deoptimization (instead of per-nmethod) should be enough to do the job.

So, from JIT-compiler perspective, your suggestion to use 
Deoptimization::deoptimize() instead of 
CompiledMethod::mark_for_deoptimization()/Deoptimization::deoptimize_all_marked() 
sounds good.

PS: also, it looks like handshakes become a perfect fit for 
VM_EnterInterpOnlyMode: once you don't need to deoptimize on per-nmethod 
basis, there's no need to trigger global safepoint.

Best regards,
Vladimir Ivanov

[1] 
http://hg.openjdk.java.net/jdk/jdk/file/tip/src/hotspot/share/prims/jvmtiEventController.cpp#l227

     // If running in fullspeed mode, single stepping is implemented
     // as follows: first, the interpreter does not dispatch to
     // compiled code for threads that have single stepping enabled;
     // second, we deoptimize all methods on the thread's stack when
     // interpreted-only mode is enabled the first time for a given
     // thread (nothing to do if no Java frames yet).