RFR: 8371046: Segfault in compiler/whitebox/StressNMethodRelocation.java with -XX:+UseZGC [v2]

[duke]

On Tue, 2 Dec 2025 22:06:18 GMT, Chad Rakoczy <duke at openjdk.org> wrote:

>> [JDK-8371046](https://bugs.openjdk.org/browse/JDK-8371046)
>> 
>> This pull request fixes two crashes (see below) and adds `InvalidationReason::RELOCATED` to better describe why an nmethod is marked not entrant during relocation.
>> 
>> ---
>> 
>> #### 1. Test Bug
>> 
>> It’s possible for an `nmethod` to be unloaded without its `_state` being explicitly set to `not_entrant`. Checking only `is_in_use()` isn’t sufficient, since the `nmethod` may already be in the process of unloading and therefore may not have a lock (as with ZGC, where `nmethods` are locked individually).
>> 
>> The fix adds an additional `is_unloading()` check in WhiteBox before acquiring the lock.
>> 
>> This issue was reproducible fairly consistently (every few runs) by executing `compiler/whitebox/StressNMethodRelocation.java` with  `-XX:+UseZGC -XX:ReservedCodeCacheSize=32m`
>> 
>> 
>> After applying this patch, the original crash stopped occurring, though a more infrequent crash was still observed.
>> 
>> ---
>> 
>> #### 2. Implementation Bug
>> 
>> `nmethod::relocate` works by copying the instructions of an `nmethod` and then adjusting the call sites to account for new PC-relative offsets.
>> 
>> Previously, this fix-up happened *after* calling `post_init()`, which registers the `nmethod` and makes it visible to the GC. This introduced a race condition where the GC might attempt to resolve a call site before it had been fixed.
>> 
>> The fix ensures that all call sites are patched **before** the `nmethod` is registered.
>> 
>> In testing, the crash previously occurred roughly 60 times in 5,000 runs (~1.2%). With this patch, no crashes were observed in the same number of runs.
>
> Chad Rakoczy has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains six additional commits since the last revision:
> 
>  - Merge remote-tracking branch 'origin/master' into JDK-8371046
>  - Add comment
>  - Add guarantee
>  - Clear inline caches before calling post_init
>  - Fix relocations before registering nmethod
>  - Add is_unloading() check before aquiring ic lock

@chadrako 
Your change (at version 7160f835b4f70f3626f2a7f32a021ace8d266200) is now ready to be sponsored by a Committer.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/28241#issuecomment-3608924399