RFR: 8364735: [asan] heap-use-after-free error detected in defaultStream::writer during VM shutdown

[Kim Barrett]

On Tue, 19 Aug 2025 00:11:38 GMT, David Holmes <dholmes at openjdk.org> wrote:

> `ostream_exit` was deleting the stream underlying the `xtty` prior to nulling the `xtty` global variable, resulting in a use-after-free-error. Due to races during VM shutdown we cannot make use of `xtty` perfectly safe, but we can certainly narrow the window during which use-after-free is possible.
> 
> Testing:
>  - tiers 1-3 sanity
> 
> Thanks

src/hotspot/share/utilities/ostream.cpp line 1001:

> 999:   xtty = nullptr;
> 1000:   OrderAccess::fence(); // force visibility to concurrently executing threads
> 1001:   delete ds;

There are lots of places in the VM that don't even try to clean up on VM exit. I think we even talk
about non-cleanup on exit in the style guide. So why are we doing this cleanup, in such a potentially
sensitive/fragile area?

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/26832#discussion_r2284607261