RFR: 8375125: assert(false) failed: "Attempting to acquire lock NativeHeapTrimmer_lock/nosafepoint out of order with lock ConcurrentHashTableResize_lock/nosafepoint-2 -- possible deadlock" when using native heap trimmer
Guanqiang Han
ghan at openjdk.org
Wed Jan 14 00:13:31 UTC 2026
Please review this change. Thanks!
**Description:**
When -XX:TrimNativeHeapInterval=non-zero is enabled, running Test6892265 can trigger a VM abort due to a detected deadlock (or lock-order issue) during SymbolTable cleanup.
SymbolTable::clean_dead_entries() calls BulkDeleteTask::prepare(), which may take ConcurrentHashTableResize_lock (nosafepoint-2).
https://github.com/openjdk/jdk/blob/0d19d91b44e5232dbd99d34dcdf6500f892e3048/src/hotspot/share/classfile/symbolTable.cpp#L765-L769
https://github.com/openjdk/jdk/blob/0d19d91b44e5232dbd99d34dcdf6500f892e3048/src/hotspot/share/utilities/concurrentHashTableTasks.inline.hpp#L159-L160
https://github.com/openjdk/jdk/blob/0d19d91b44e5232dbd99d34dcdf6500f892e3048/src/hotspot/share/utilities/concurrentHashTable.inline.hpp#L307-L310
It then constructs a NativeHeapTrimmer::SuspendMark, which may take NativeHeapTrimmer_lock (nosafepoint). This can invert the lock order with the trimmer thread.
https://github.com/openjdk/jdk/blob/0d19d91b44e5232dbd99d34dcdf6500f892e3048/src/hotspot/share/classfile/symbolTable.cpp#L773
https://github.com/openjdk/jdk/blob/0d19d91b44e5232dbd99d34dcdf6500f892e3048/src/hotspot/share/runtime/trimNativeHeap.hpp#L56-L59
https://github.com/openjdk/jdk/blob/0d19d91b44e5232dbd99d34dcdf6500f892e3048/src/hotspot/share/runtime/trimNativeHeap.cpp#L177-L181
**Fix:**
Move NativeHeapTrimmer::SuspendMark to the beginning of SymbolTable::clean_dead_entries() before BulkDeleteTask::prepare() to enforce a consistent lock order
**Test:**
GHA
-------------
Commit messages:
- fix 8375125
Changes: https://git.openjdk.org/jdk/pull/29212/files
Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=29212&range=00
Issue: https://bugs.openjdk.org/browse/JDK-8375125
Stats: 9 lines in 2 files changed: 5 ins; 1 del; 3 mod
Patch: https://git.openjdk.org/jdk/pull/29212.diff
Fetch: git fetch https://git.openjdk.org/jdk.git pull/29212/head:pull/29212
PR: https://git.openjdk.org/jdk/pull/29212
More information about the hotspot-dev
mailing list