RFR: 8325587: Shenandoah: ShenandoahLock should allow blocking in VM

[Robbin Ehn]

On Mon, 12 Feb 2024 17:40:00 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

> `ShenandoahLock` is a spinlock that is supposed to guard heap state. That lock is normally only lightly contended, as threads normally only allocate large TLABs/GCLABs. So we just summarily delegated to `Thread::{SpinAcquire,SpinRelease}`, which spins a bit, and then starts going to sleep/yield dance on contention.
> 
> This does not work well when there are lots of threads near the OOM conditions. Then, most of these threads would fail to allocate the TLAB, go for out-of-TLAB alloc, start lock acquisition, and spend _a lot_ of time trying to acquire the lock. The handshake/safepoint would think those threads are running, even when they are actually yielding or sleeping. As seen in bug report, a handshake operation over many such threads could then take hundreds of seconds.
> 
> The solution is to notify VM that we are blocking before going for `sleep` or `yield`. This is similar to what other VM code does near such yields. This involves state transitions, so it is only cheap to do near the actual blocking. Protecting the whole lock with VM transition would be very slow.
> 
> I also de-uglified bits of adjacent code.
> 
> Additional testing:
>  - [x] Original Extremem reproducer does not have outliers anymore
>  - [x] Linux x86_64 server fastdebug, `hotspot_gc_shenandoah`
>  - [x] Linux x86_64 server fastdebug, `all` passes with `-XX:+UseShenandoahGC` (some old failures, but quite a few "timeouts" also disappear)

Think about if you can add missing pieces to: https://github.com/openjdk/jdk/blob/master/src/hotspot/share/utilities/spinYield.hpp
And utilize that, as it is would be nice to have one? configurable back-off strategy in the future.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/17813#issuecomment-1939319623