Unable to use custom SSLEngine with default TrustManagerFactory after updating to ea20 (and later)

[Xuelei Fan]

It is now tracked in JBS:
   https://bugs.openjdk.java.net/browse/JDK-8207029

Thanks,
Xuelei

On 7/10/2018 11:21 PM, Norman Maurer wrote:
> Submitted it via https://bugreport.java.com.
> 
> Please let me know once it “transferred” to https://bugs.openjdk.java.net
> 
> 
> Bye
> Norman
> 
> 
> 
>> On 10. Jul 2018, at 20:26, Norman Maurer <norman.maurer at googlemail.com 
>> <mailto:norman.maurer at googlemail.com>> wrote:
>>
>> Will do tomorrow latest.
>>
>> Thanks for the quick reply.
>>
>> Bye
>> Norman
>>
>>> Am 10.07.2018 um 18:53 schrieb Xuelei Fan <xuelei.fan at oracle.com 
>>> <mailto:xuelei.fan at oracle.com>>:
>>>
>>> Hi Norman,
>>>
>>> It's an interesting user case of the TrustManagerFactory.  Please 
>>> file a bug.
>>>
>>> Thanks,
>>> Xuelei
>>>
>>>> On 7/10/2018 9:57 AM, Alan Bateman wrote:
>>>> Forwarding to security-dev.
>>>>> On 10/07/2018 17:47, Norman Maurer wrote:
>>>>> Hi all,
>>>>>
>>>>> I just tried to run netty[1] testsuite with the latest jdk11 EA 
>>>>> release (21) and saw some class-cast-exception with our custom 
>>>>> SSLEngine implementation
>>>>>
>>>>>
>>>>> Caused by: java.lang.ClassCastException: class 
>>>>> io.netty.handler.ssl.OpenSslEngine cannot be cast to class 
>>>>> sun.security.ssl.SSLEngineImpl (io.netty.handler.ssl.OpenSslEngine 
>>>>> is in unnamed module of loader 'app'; 
>>>>> sun.security.ssl.SSLEngineImpl is in module java.base of loader 
>>>>> 'bootstrap')
>>>>> at 
>>>>> java.base/sun.security.ssl.SSLAlgorithmConstraints.<init>(SSLAlgorithmConstraints.java:93)
>>>>> at 
>>>>> java.base/sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:270)
>>>>> at 
>>>>> java.base/sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:141)
>>>>> at 
>>>>> io.netty.handler.ssl.ReferenceCountedOpenSslClientContext$ExtendedTrustManagerVerifyCallback.verify(ReferenceCountedOpenSslClientContext.java:237)
>>>>> at 
>>>>> io.netty.handler.ssl.ReferenceCountedOpenSslContext$AbstractCertificateVerifier.verify(ReferenceCountedOpenSslContext.java:621)
>>>>> ... 27 more
>>>>>
>>>>>
>>>>> This change seems to be related to:
>>>>> http://hg.openjdk.java.net/jdk/jdk11/rev/68fa3d4026ea
>>>>>
>>>>> I think you miss an instanceof check here in 
>>>>> SSLAlgorithmConstraints before try to cast to SSLEngineImpl, as 
>>>>> otherwise it will be impossible to use custom implementations of 
>>>>> SSLEngine (which we have in netty) with the default 
>>>>> TrustManagerFactory.
>>>>>
>>>>> Does this sound correct ? Should I open a bug-report ?
>>>>>
>>>>> Bye
>>>>> Norman
>>>>>
>>>>>
>>>>>
>