RFR: JDK-8277056: Combining several C2 Print* flags asserts in xmlStream::pop_tag

[Dean Long]

On Tue, 12 Apr 2022 13:15:28 GMT, Tobias Holenstein <duke at openjdk.java.net> wrote:

> Sometimes the writing to xmlStream is mixed from several threads, and therefore the `xmlStream` tag stack can end up in a bad state. When this occurs, the VM crashes in `xmlStream::pop_tag` with `assert(false) failed: bad tag in log`.
> 
> The logging between `xtty->head` and `xtty->tail` is guarded by a ttyLocker which also locks `VMThread::should_terminate()` (ttyLocker is needed here to serializes the output info about the termination of the VMThread). 
> 
> The problem is that `print_metadata` and `dump_asm` may block for safepoint which then releases the ttyLocker (`break_tty_lock_for_safepoint`). When the `print_metadata` or `dump_asm` continues after the safepoint other threads could have taken the ttyLocker and may be busy printing and interfere with the tag stack of `xmlStream`.
> 
> The solution is to call `print_metadata` and `dump_asm` first and let them write to a local stringStream. Then acquire the ttyLocker to do the printing (using the local stringStream) and use a separate ttyLocker for `VMThread::should_terminate()`
> 
> The same issue was already targeted in https://bugs.openjdk.java.net/browse/JDK-8153527 but the fix was incomplete.

Is the compiler thread running in VM mode or native mode at this point?  If it's in native mode, couldn't the tty lock get broken at any time?  It seems like we should collect the output in stringStreams, enter VM mode, lock tty, then perform output.

-------------

PR: https://git.openjdk.java.net/jdk/pull/8203