RFR: 8371944: AOT configuration is corrupted when app closes System.out

[Ioi Lam]

On Sat, 15 Nov 2025 05:34:32 GMT, Vladimir Kozlov <kvn at openjdk.org> wrote:

>> During an AOT training run, some application may close `System.out` (usually inadvertently by naive code like the following):
>> 
>> 
>> try (var err = new PrintWriter(System.err);
>>       var out = new PrintWriter(System.out)) {
>>      out.println("Hello Confused World");
>> }
>> 
>> 
>> This has the side effect of closing the JVM's STDOUT.
>> 
>> When the JVM is about to exit, we will open the AOT configuration file for writing. On Windows we may get back a file HANDLE (which is just an integer) that's identical to the now closed STDOUT.
>> 
>> If the JVM writes to STDOUT (usually with UL logging), it will corrupt the contents of the AOT configuration file.
>> 
>> This doesn't happen on Posix as `System.out.close()` will keep file descriptions 1 and 2 open, preventing them from being reused by files that are opened in the future.
>> 
>> The fix is to open the AOT configuration file early, before any application code is executed. That way we can guarantee that we will have a file HANDLE (or file descriptor) that's different than STDOUT.
>> 
>> The test failed failed about 20% of the time on Windows before the fix. After the fix, it ran 100 times without failure.
>
> The fix seems fine for this case.
> 
> VM itself may open a lot of files for different logs (JFR, JIT logs, UL, etc). May VM close some before you open AOT file which could cause the same issue? Is it possible to track life time of such files to make sure we don't write into them after they are closed?

Thanks @vnkozlov @veresov for the review

-------------

PR Comment: https://git.openjdk.org/jdk/pull/28335#issuecomment-3554604701