RFR: 8366736: Closed System.out causes child process to hang on Windows

[Ioi Lam]

On Tue, 13 Jan 2026 16:52:36 GMT, Ioi Lam <iklam at openjdk.org> wrote:

> The bug is here on line 121:
> 
> https://github.com/openjdk/jdk/blob/586846b84a38d285c5905437e903cfc57f609410/src/java.base/windows/classes/java/lang/ProcessImpl.java#L118-L121
> 
> If `System.out` has been closed, `fdAccess.getHandle()` will return -1. This causes `stdHandles[1]` to have the same value as if the child process's stdout was redirected with `Redirect.PIPE`. This will cause a Pipe to be created here for the child process's STDOUT on line 168:
> 
> https://github.com/openjdk/jdk/blob/586846b84a38d285c5905437e903cfc57f609410/src/java.base/windows/native/libjava/ProcessImpl_md.c#L158-L184
> 
> However, the caller of the `ProcessBuilder` is not aware of this and will not drain this pipe. This causes the child process to get stuck when writing to its stdout when the pipe 's buffer is filled up.
> 
> The fix is to treat the redirection as `Redirect.DISCARD` when `System.out` and/or `System.err` have been closed.

> The bug is here on line 121:
> 
> https://github.com/openjdk/jdk/blob/586846b84a38d285c5905437e903cfc57f609410/src/java.base/windows/classes/java/lang/ProcessImpl.java#L118-L121
> 
> If `System.out` has been closed, `fdAccess.getHandle()` will return -1. This causes `stdHandles[1]` to have the same value as if the child process's stdout was redirected with `Redirect.PIPE`. This will cause a Pipe to be created here for the child process's STDOUT on line 168:
> 
> https://github.com/openjdk/jdk/blob/586846b84a38d285c5905437e903cfc57f609410/src/java.base/windows/native/libjava/ProcessImpl_md.c#L158-L184
> 
> However, the caller of the `ProcessBuilder` is not aware of this and will not drain this pipe. This causes the child process to get stuck when writing to its stdout after the pipe 's buffer is filled up.
> 
> The fix is to treat the redirection as `Redirect.DISCARD` when `System.out` and/or `System.err` have been closed.
> 
> ### Progress
> * [ ]  Change must be properly reviewed (1 review required, with at least 1 [Reviewer](https://openjdk.org/bylaws#reviewer))
> * [x]  Change must not contain extraneous whitespace
> * [x]  Commit message must refer to an issue
> 
> ### Issue
> * [JDK-8366736](https://bugs.openjdk.org/browse/JDK-8366736): Closed System.out causes child process to hang on Windows (**Bug** - P4)
> 
> ### Reviewing
> Using `git`
> Checkout this PR locally: `$ git fetch https://git.openjdk.org/jdk.git pull/29198/head:pull/29198` `$ git checkout pull/29198`
> 
> Update a local copy of the PR: `$ git checkout pull/29198` `$ git pull https://git.openjdk.org/jdk.git pull/29198/head`
> 
> Using Skara CLI tools
> Checkout this PR locally: `$ git pr checkout 29198`
> 
> View PR using the GUI difftool: `$ git pr show -t 29198`
> 
> Using diff file

-------------

PR Comment: https://git.openjdk.org/jdk/pull/29198#issuecomment-3745764159