RFR 8182757: JDWP: Socket Transport handshake hangs on Solaris

[Gerald Thornbrugh]

Hi Serguei,

Thanks!

Jerry
> On Aug 2, 2017, at 9:58 AM, serguei.spitsyn at oracle.com wrote:
> 
> Hi Gerald,
> 
> It looks good.
> 
> Thanks,
> Serguei
> 
> 
> On 8/2/17 08:16, Gerald Thornbrugh wrote:
>> Hi,
>> 
>> Please review this JDK-10 fix for JDK-8182757. 
>> 
>> The bug: https://bugs.openjdk.java.net/browse/JDK-8182757 <https://bugs.openjdk.java.net/browse/JDK-8182757>
>> 
>> The webrev: http://cr.openjdk.java.net/~gthornbr/8182757/webrev.00 <http://cr.openjdk.java.net/%7Egthornbr/8182757/webrev.00>
>> 
>> If a socket is being setup without a fixed port using the SO_REUSEADDR flag it can lead to other
>> processes interfering with the poll/receive process of a debugger/debuggee configuring a socket
>> for communication. When SO_REUSEADDR is used other processes can attempt a listen() on
>> the same port and receive a connect from the debuggee. This causes the debugger to stay in
>> poll() waiting for a connect and the debuggee stays in recv() waiting to receive data from the
>> "rogue" process that will never send it.
>> 
>> This can also lead to connections being terminated early on the debuggee side when the “rogue”
>> process terminates the connection because it does not receive what it expected from the client
>> process (i.e. the debuggee).
>> 
>> The fix is to not use the SO_REUSEADDR flag for non-fixed port sockets. This keeps “rogue”
>> processes from reusing the port address and from stealing the connects sent by the debuggee.
>> 
>> The changes to src/jdk.jdi/share/classes/com/sun/tools/jdi/SocketTransportService.java addresses
>> when JDI (the debugger side) is acting in “server mode”.
>> 
>> The changes to src/jdk.jdwp.agent/share/native/libdt_socket/socketTransport.c addresses when
>> JDWP (the debuggee side) is acting in “server mode”.
>> 
>> I have run the JDK JDI tests and the internal Oracle VM/NSK JPDA tests on all platforms without errors.
>> We were able to reproduce the failures with a specific load on a test machine while running JDI
>> related tests.  After we applied the fix to this system we were not able to reproduce the failures.
>> 
>> Thanks,
>> 
>> Gerald
>