Integrated: 8334482: Shenandoah: Deadlock when safepoint is pending during nmethods iteration

[Aleksey Shipilev]

On Wed, 24 Jul 2024 09:10:35 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

> There is a problem in interaction between STS taken in `ShenandoahConcurrentWeakRootsEvacUpdateTask::work` and nmethod list iteration. If we start the nmethod iteration in `ShCWREUT` constructor, proceed to STS and block, any waiter that waits for the end of iteration would block until safepoint is over. If that waiter waits without a safepoint check, like anyone waiting on `CodeCache_lock`, e.g. Sweeper (not really in mainline, but we seen a case in JDK 17), or a JIT compiler patching code, then we would have a deadlock.
> 
> A solution is to initialize iteration only immediately before the use of iterator. This will pass any pending STSes that we have seen could deadlock. As long as nothing in nmethod handling safepoints, the deadlock is resolved. I think `NoSafepointVerifier` would give us a warning if something safepoints while iteration is on.
> 
> Unfortunately, it is really hard to reproduce in mainline, since the absence of Sweeper makes this deadlock exceedingly rare.
> 
> Additional testing:
>  - [x] Reproducer on JDK 17u with this patch applied now does not deadlock
>  - [x] Linux AArch64 server fastdebug, `all` with `-XX:+UseShenandoahGC`
>  - [x] Linux x86_64 server fastdebug, `all` with `-XX:+UseShenandoahGC`

This pull request has now been integrated.

Changeset: 2aeb12ec
Author:    Aleksey Shipilev <shade at openjdk.org>
URL:       https://git.openjdk.org/jdk/commit/2aeb12ec03944c777d617d0be48982fd225b16e7
Stats:     72 lines in 4 files changed: 25 ins; 33 del; 14 mod

8334482: Shenandoah: Deadlock when safepoint is pending during nmethods iteration

Reviewed-by: wkemper, xpeng, rkennke

-------------

PR: https://git.openjdk.org/jdk/pull/20309