Unable to use custom SSLEngine with default TrustManagerFactory after updating to ea20 (and later)

[Norman Maurer]

thanks… used the wrong mailing list again :(



> On 10. Jul 2018, at 17:57, Alan Bateman <Alan.Bateman at oracle.com> wrote:
> 
> Forwarding to security-dev.
> 
> On 10/07/2018 17:47, Norman Maurer wrote:
>> Hi all,
>> 
>> I just tried to run netty[1] testsuite with the latest jdk11 EA release (21) and saw some class-cast-exception with our custom SSLEngine implementation
>> 
>> 
>> Caused by: java.lang.ClassCastException: class io.netty.handler.ssl.OpenSslEngine cannot be cast to class sun.security.ssl.SSLEngineImpl (io.netty.handler.ssl.OpenSslEngine is in unnamed module of loader 'app'; sun.security.ssl.SSLEngineImpl is in module java.base of loader 'bootstrap')
>> 	at java.base/sun.security.ssl.SSLAlgorithmConstraints.<init>(SSLAlgorithmConstraints.java:93)
>> 	at java.base/sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:270)
>> 	at java.base/sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:141)
>> 	at io.netty.handler.ssl.ReferenceCountedOpenSslClientContext$ExtendedTrustManagerVerifyCallback.verify(ReferenceCountedOpenSslClientContext.java:237)
>> 	at io.netty.handler.ssl.ReferenceCountedOpenSslContext$AbstractCertificateVerifier.verify(ReferenceCountedOpenSslContext.java:621)
>> 	... 27 more
>> 
>> 
>> This change seems to be related to:
>> http://hg.openjdk.java.net/jdk/jdk11/rev/68fa3d4026ea <http://hg.openjdk.java.net/jdk/jdk11/rev/68fa3d4026ea>
>> 
>> I think you miss an instanceof check here in SSLAlgorithmConstraints before try to cast to SSLEngineImpl, as otherwise it will be impossible to use custom implementations of SSLEngine (which we have in netty) with the default TrustManagerFactory.
>> 
>> Does this sound correct ? Should I open a bug-report ?
>> 
>> Bye
>> Norman
>> 
>> 
>> 
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/security-dev/attachments/20180710/534b7841/attachment.htm>