RFR: 8313202: MutexLocker should disallow null Mutexes

[Aleksey Shipilev]

On Wed, 26 Jul 2023 17:06:02 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

> As seen in [JDK-8313081](https://bugs.openjdk.org/browse/JDK-8313081), it is fairly easy to pass nullptr `Mutex` to `MutexLocker` by accident, which would just silently avoid the lock.
> 
> There are a few places in Hotspot where we pass `nullptr` to simulate re-entrancy and/or conditionally take the lock. Those places can be more explicit, and the default `MutexLocker` can disallow nullptrs for extra safety.
> 
> Open for some bikeshedding on the names of the new `MutexLockers`. Particularly `ReentrantMutexLocker` might lull readers into believing it does safepoint checks on re-entrant "lock", which it actually does not do.
> 
> More thorough testing with different GC/JIT combinations is running now, we might find more issues there. Meanwhile, please comment on the approach.
> 
> Additional testing:
>  - [x] `grep -R "MutexLocker " src/hotspot | grep -i null`, no hits
>  - [x] `grep -R "MutexLocker " src/hotspot | grep -i ?`, no hits
>  - [x] Linux AArch64 fastdebug, `tier1 tier2 tier3` (re-run in progress)

There is a lateral thing we can do to mitigate future problems like JDK-8313081: make all named Mutexes/Monitors unconditionally initialized, regardless of runtime options and/or defines. It feels unsufficient, as it still does not protect from misuse of private Mutexes/Monitors, but at least one could be reasonably sure the named Mutex/Monitor from `mutexLocker.hpp` is guaranteed to be initialized during the normal VM operation.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/15043#issuecomment-1653170061