Suggestion needed to port the fix to JDK17 and JDK11

[Shruthi .]

We discussed the issue with AIX OS team to get their input as this issue is happening only in AIX platform. Update from AIX OS dev team: A thread gets stuck in dup2() while duplicating a file descriptor that is already being used by two other threads (one for reading, one for writing). The writer thread does not write or close the FIFO, causing the reader to wait indefinitely, which blocks dup2(). Also he is saying that we cannot change OS side as it may break or change its existing behavior which other existing applications may be depending on.

Can we apply the previously proposed fix at java level(reordering the preClose() method) while adding an AIX-specific check to ensure it doesn’t affect other platforms? Or do you have any alternative suggestions?

            if (!tryClose()) {
                long th = thread;
                if (th != 0) {
                    if (!AIX.isAIX)
                        nd.preClose(fd);
                    NativeThread.signal(th);
                    if (AIX.isAIX)
                        nd.preClose(fd);
                }
            }
Restarting the previous discussion: https://mail.openjdk.org/pipermail/net-dev/2023-August/021859.html

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/net-dev/attachments/20250212/e6d8ddc8/attachment.htm>