RFR: 8374862: assert(false) failed: Attempting to acquire lock MDOExtraData_lock/nosafepoint-1 out of order with lock tty_lock/tty -- possible deadlock (running with -XX:+Verbose -XX:+WizardMode -XX:+PrintDeoptimizationDetails) [v2]

[Dean Long]

On Wed, 14 Jan 2026 02:32:59 GMT, Guanqiang Han <ghan at openjdk.org> wrote:

>> Please review this change. Thanks!
>> 
>> **Description:**
>> 
>> When -XX:+PrintDeoptimizationDetails is enabled, vframeArray.cpp prints interpreted frames under ttyLocker. 
>> https://github.com/openjdk/jdk/blob/578204f8c49f06be8b9c4855359ca61c9e107678/src/hotspot/share/runtime/vframeArray.cpp#L493-L503
>> In the WizardMode && Verbose branch it calls Method::print_codes() / print_codes_on(). The bytecode printing path may acquire MDOExtraData_lock (rank nosafepoint-1). 
>> https://github.com/openjdk/jdk/blob/578204f8c49f06be8b9c4855359ca61c9e107678/src/hotspot/share/interpreter/bytecodeTracer.cpp#L597-L602
>> Calling it while holding tty_lock violates the global lock ranking order and triggers a lock rank inversion assertion.
>> 
>> **Fix:**
>> 
>> Generate the bytecode dump before taking tty_lock, and print it afterwards under ttyLocker to keep the output coherent while preserving correct lock acquisition order. 
>> To avoid double buffering when the target stream is already a thread-local stringStream, extend BytecodeTracer::print_method_codes with a buffered flag . The new call site uses buffered=false when dumping into the temporary stringStream.
>> 
>> **Test:** 
>> 
>> GHA
>
> Guanqiang Han has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains four additional commits since the last revision:
> 
>  - change variable name
>  - Merge remote-tracking branch 'upstream/master' into 8374862
>  - fix a compile  error
>  - fix 8374862

I think adding `virtual bool is_buffered() `to outputStream might be a cleaner solution.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/29186#issuecomment-3747452892