RFR: 8359064: Expose reason for marking nmethod non-entrant to JVMCI client [v6]

[Tom Rodriguez]

On Thu, 12 Jun 2025 20:18:10 GMT, Cesar Soares Lucas <cslucas at openjdk.org> wrote:

>> We recently introduced a way to set the reason why a nmethod was being marked as `not entrant`, see [here](https://github.com/openjdk/jdk/pull/23980) and [here](https://github.com/openjdk/jdk/pull/25338).
>> 
>> This PR is to expose in the JVMCI interface the reason why the nmethod was flagged as `not entrant`. This will allow JVMCI-based compilers to implement heuristics to handle re-compilations differently based on what happened to earlier versions of a method, for instance, this will likely be used to address this [RFE in Truffle](https://github.com/oracle/graal/issues/11045
>> ). 
>> 
>> Tested on Linux x86_64, ARM with JTREG tier 1-3.
>
> Cesar Soares Lucas has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Fix some remaining renames.

src/hotspot/share/jvmci/jvmciCompilerToVM.cpp line 1409:

> 1407: C2V_VMENTRY(void, invalidateHotSpotNmethod, (JNIEnv* env, jobject, jobject hs_nmethod, jboolean deoptimize, jint invalidation_reason))
> 1408:   JVMCIObject nmethod_mirror = JVMCIENV->wrap(hs_nmethod);
> 1409:   JVMCIENV->invalidate_nmethod_mirror(nmethod_mirror, deoptimize, static_cast<nmethod::InvalidationReason>(invalidation_reason), JVMCI_CHECK);

There should probably be a check here that `invalidation_reason` is in the range of `nmethod::InvalidationReason`.

src/hotspot/share/jvmci/jvmciRuntime.cpp line 818:

> 816:       HotSpotJVMCI::InstalledCode::set_address(jvmciEnv, nmethod_mirror, 0);
> 817:       HotSpotJVMCI::InstalledCode::set_entryPoint(jvmciEnv, nmethod_mirror, 0);
> 818:       HotSpotJVMCI::HotSpotNmethod::set_invalidationReason(jvmciEnv, nmethod_mirror, static_cast<int>(invalidation_reason));

I think you're in danger of overwriting the invalidationReason here.  `invalidate_nmethod_mirror` can be called for unloaded nmethods after they have been made not entrant.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/25706#discussion_r2145899316
PR Review Comment: https://git.openjdk.org/jdk/pull/25706#discussion_r2145896972