adding rsockets support into JDK

[Lu, Yingqi]

Hi Alan/Chris,

I was able to confirm that connecting on non-blocking socket causes issues. It happens when connect/accept occurs in the same thread or different threads in the same process. 

Then, I did a small tweak in Chris's sample application by spawning a thread doing rpoll on the connection_fd. Now, the connect/accept works in both of the cases above. Please let me know if this is a valid workaround for the issue. 

Performance wise, this workaround should not impact send/receive at all. It might only add a small overhead to the connection setup phase only with non-blocking RDMA socket. 

The modified app code is available at

For connect/accept occur in the same thread: https://cr.openjdk.java.net/~ylu/testNonBlocking_raccept_modified.c

For connect/accept occur in two different threads: https://cr.openjdk.java.net/~ylu/testNonBlocking_raccept_modified_2threads.c

Thanks,
Lucy

>-----Original Message-----
>From: Alan Bateman [mailto:Alan.Bateman at oracle.com]
>Sent: Saturday, December 8, 2018 8:10 AM
>To: Chris Hegarty <chris.hegarty at oracle.com>; Lu, Yingqi
><yingqi.lu at intel.com>; nio-dev at openjdk.java.net
>Cc: Viswanathan, Sandhya <sandhya.viswanathan at intel.com>; Aundhe,
>Shirish <shirish.aundhe at intel.com>; Kaczmarek, Eric
><eric.kaczmarek at intel.com>
>Subject: Re: adding rsockets support into JDK
>
>On 08/12/2018 09:39, Chris Hegarty wrote:
>> :
>>
>> - It has become apparent that mixing blocking and non-blocking
>>   connect/accept operations, in the same thread, may cause issues. For
>>   example, attempting to setup a connected-socket on the same host by
>>   issuing a non-blocking connect followed by a blocking accept, will
>>   just hang and not make progress [3]. Upon further enquiries it
>> appears
>>   that the programming model for rsocket is a subtly different than
>> that
>>   of the regular Berkeley sockets ( at least for the connection
>>   handshake ). It is not immediately clear how to reasonably
>> workaround
>>   this issue ( it's not a bug in rdma-core, but more a fundamental
>> part
>>   of its thread-less operation ).
>>
>Would it be possible to expand on this to say whether the same issues arises
>when the non-blocking connect is initiated on a different thread, or in a
>different process, or even a different machine on the fabric.
>That is, if the socket is non-blocking and I do a rconnect and then delay before
>doing anything else on the socket then will the peer doing accept be
>blocked/hung in the mean-time?
>
>-Alan