RFR: 8274524: SSLSocket.close() hangs if it is called during the ssl handshake [v4]

[Alexey Bakhtin]

On Wed, 9 Feb 2022 22:09:40 GMT, Xue-Lei Andrew Fan <xuelei at openjdk.org> wrote:

>> Alexey Bakhtin has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   Fix jcheck issues
>
> With the patch, the readLock is placed in handshakeLock, and the handshake may require lock write lock.  These nested lock is pretty risk and will run into deadlocks.  That's the underlying reason why readHandshakeRecord() is not placed within read lock and write lock.
> 
> It looks like the problem is cause by close() method.  Maybe, improving the close() implementation could be an alternative solution.  For example, give up the skip operation if no lock can be granted.

Hi @XueleiFan thank you for explanation and suggestions.
We already have code to give up the skip operation if appInput.readLock can no be granted. Unfortunately, read operation during handshake is not protected by appInput.readLock. So it does not help in this case.
I just found, that the issue happens in case of zero SO_TIMEOUT only. I suggest to force non-zero SO_TIMEOUT during the skip(). New pull request is submitted: https://github.com/openjdk/jdk/pull/7432

-------------

PR: https://git.openjdk.java.net/jdk/pull/5760