RFR: 8364735: [asan] heap-use-after-free error detected in defaultStream::writer during VM shutdown

[David Holmes]

On Tue, 19 Aug 2025 00:11:38 GMT, David Holmes <dholmes at openjdk.org> wrote:

> `ostream_exit` was deleting the stream underlying the `xtty` prior to nulling the `xtty` global variable, resulting in a use-after-free-error. Due to races during VM shutdown we cannot make use of `xtty` perfectly safe, but we can certainly narrow the window during which use-after-free is possible.
> 
> Testing:
>  - tiers 1-3 sanity
> 
> Thanks

@kimbarrett and @tstuefe I looked at trying to not actually delete the stream but this is far more complex than it may appear. If you look at what happens via the `defaultStream` destructor there is a lot of cleanup and deletion of other stream objects (xmlStream, fileStream). It is far from clear what exactly a shutdown "without closing or deleting" would actually look like - nor what potential bugs it may introduce.

What I have proposed here is a simple fix to the use-after-free problem. I don't think a redesign of stream handling at shutdown is a reasonable request to address this.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/26832#issuecomment-3235376630