Suggestion needed to port the fix to JDK17 and JDK11

[Shruthi Shruthi1]

Problem Description :
Customer has reported an issue , hang in preClose() method.

Analysis & Observation :
The preClose() method internally calls the dup2() system call. If there is a dup2() call on a file descriptor on a thread while another thread is blocked in a read/write operation on the same file descriptor, then the dup2() call is known to hang. Currently, preClose() experiences a hang because we call dup2() before killing the reader/writer thread(s).

Suggested Solution:
The solution is to first kill the reader/writer thread(s) and then do the preClose() sequence. This reordering in the different channel implementations resolves the customer problem as per the testing.

// wait for any outstanding read to complete
        if (blocking) {
            synchronized (stateLock) {
                assert state == ST_CLOSING;
                long th = thread;
                if (th != 0) {
                    nd.preClose(fd);
                    NativeThread.signal(th);

The above code is changed to

 // wait for any outstanding read to complete
        if (blocking) {
            synchronized (stateLock) {
                assert state == ST_CLOSING;
                long th = thread;
                if (th != 0) {
                    NativeThread.signal(th);
                    nd.preClose(fd);

Releases:
OpenJDK11 & OpenJDK17 the issue is seen. In OpenJDK21 the issue is not seen.

Next steps:
We want to take the above mentioned fix to OpenJDK but unfortunately we are seeing the below issue.
* The customer has confirmed that the issue is not reproducible in OpenJDK dev branch
* We have not been successful in creating a java reproducer even after spending a considerable amount of time.

Please review and suggest if the above understanding is right and we shall take the fix to OpenJDK17 and backport to OpenJDK11.


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/net-dev/attachments/20230821/4ebd31ba/attachment-0001.htm>