[8u] TLSv1.3 RFR: 8251478: Backport TLSv1.3 regression tests to JDK8u

Martin Balao mbalao at redhat.com
Thu Aug 27 20:53:23 UTC 2020 

Hi,

On 8/27/20 5:33 PM, Alexey Bakhtin wrote:
> You are right. Thank you.
> test/sun/security/pkcs11/fips/ClientJSSEServerJSSE.java test can be enabled now. Fixed by removing @ignore annotation.
> 
> New Webrev : http://cr.openjdk.java.net/~abakhtin/tls1.3/8245466/8251478/webrev.v4/
> 

Step 15 - 8251478 has a set of changes required for tests from JDK
11.0.7 to pass in 8u and jtreg '@modules' tags removed (which are N/A
for 8u).

 * test/javax/net/ssl/FixingJavadocs/ComURLNulls.java (modified)
  * Ok. 8074531 is not in 8u. This change reverts to the previous test
getServerCertificateChain use and removes the modules jtreg header.
 * test/javax/net/ssl/FixingJavadocs/SSLSessionNulls.java (modified)
  * Ok
 * test/javax/net/ssl/SSLSession/RenegotiateTLS13.java (modified)
  * Ok
 * test/javax/net/ssl/SSLSession/ResumeTLS13withSNI.java (modified)
  * Ok
 * test/javax/net/ssl/SSLSocket/InputStreamClosure.java (modified)
  * Ok
 * test/javax/net/ssl/SSLSocket/OutputStreamClosure.java (modified)
  * Ok
 * test/javax/net/ssl/SSLSocket/Tls13PacketSize.java (modified)
  * Ok
 * test/javax/net/ssl/Stapling/HttpsUrlConnClient.java (modified)
  * Ok
 * test/javax/net/ssl/Stapling/SSLEngineWithStapling.java (modified)
  * Ok
 * test/javax/net/ssl/Stapling/SSLSocketWithStapling.java (modified)
  * Ok
 * test/javax/net/ssl/Stapling/StapleEnableProps.java (modified)
  * Ok
 * test/javax/net/ssl/TLS/TLSClientPropertyTest.java (modified)
  * Ok, TLSv1.3 is disabled by default for clients
 * test/javax/net/ssl/TLS/TLSDataExchangeTest.java (modified)
  * Ok
 * test/javax/net/ssl/TLS/TLSEnginesClosureTest.java (modified)
  * Ok
 * test/javax/net/ssl/TLS/TLSHandshakeTest.java (modified)
  * Ok
 * test/javax/net/ssl/TLS/TLSMFLNTest.java (modified)
  * Ok
 * test/javax/net/ssl/TLS/TLSNotEnabledRC4Test.java (modified)
  * Ok
 * test/javax/net/ssl/TLS/TLSRehandshakeTest.java (modified)
  * Ok
 * test/javax/net/ssl/TLS/TLSRehandshakeWithCipherChangeTest.java (modified)
  * Ok
 * test/javax/net/ssl/TLS/TLSRehandshakeWithDataExTest.java (modified)
  * Ok
 * test/javax/net/ssl/TLS/TLSUnsupportedCiphersTest.java (modified)
  * Ok
 * test/javax/net/ssl/TLS/TestJSSEClientDefaultProtocol.java (modified)
  * Ok
 * test/javax/net/ssl/TLS/TestJSSEClientProtocol.java (modified)
  * Ok
 * test/javax/net/ssl/TLS/TestJSSENoCommonProtocols.java (modified)
  * Ok
 * test/javax/net/ssl/TLS/TestJSSEServerProtocol.java (modified)
  * Ok
 * test/javax/net/ssl/TLSCommon/Protocol.java (modified)
  * Ok
 * test/javax/net/ssl/TLSCommon/RehandshakeWithCipherChangeTest.java
(modified)
  * Ok
 * test/javax/net/ssl/TLSCommon/RehandshakeWithDataExTest.java (modified)
  * Ok
 * test/javax/net/ssl/TLSCommon/SSLEngineTestCase.java (modified)
  * Ok
 * test/javax/net/ssl/TLSv1/TLSDataExchangeTest.java (modified)
  * Ok
 * test/javax/net/ssl/TLSv1/TLSEnginesClosureTest.java (modified)
  * Ok
 * test/javax/net/ssl/TLSv1/TLSHandshakeTest.java (modified)
  * Ok
 * test/javax/net/ssl/TLSv1/TLSMFLNTest.java (modified)
  * Ok
 * test/javax/net/ssl/TLSv1/TLSNotEnabledRC4Test.java (modified)
  * Ok
 * test/javax/net/ssl/TLSv1/TLSRehandshakeTest.java (modified)
  * Ok
 * test/javax/net/ssl/TLSv1/TLSRehandshakeWithCipherChangeTest.java
(modified)
  * Ok
 * test/javax/net/ssl/TLSv1/TLSRehandshakeWithDataExTest.java (modified)
  * Ok
 * test/javax/net/ssl/TLSv1/TLSUnsupportedCiphersTest.java (modified)
  * Ok
 * test/javax/net/ssl/TLSv11/ExportableBlockCipher.java (modified)
  * Ok
 * test/javax/net/ssl/TLSv11/ExportableStreamCipher.java (modified)
  * Ok
 * test/javax/net/ssl/TLSv11/TLSDataExchangeTest.java (modified)
  * Ok
 * test/javax/net/ssl/TLSv11/TLSEnginesClosureTest.java (modified)
  * Ok
 * test/javax/net/ssl/TLSv11/TLSHandshakeTest.java (modified)
  * Ok
 * test/javax/net/ssl/TLSv11/TLSMFLNTest.java (modified)
  * Ok
 * test/javax/net/ssl/TLSv11/TLSNotEnabledRC4Test.java (modified)
  * Ok
 * test/javax/net/ssl/TLSv11/TLSRehandshakeTest.java (modified)
  * Ok
 * test/javax/net/ssl/TLSv11/TLSRehandshakeWithCipherChangeTest.java
(modified)
  * Ok
 * test/javax/net/ssl/TLSv11/TLSRehandshakeWithDataExTest.java (modified)
  * Ok
 * test/javax/net/ssl/TLSv11/TLSUnsupportedCiphersTest.java (modified)
  * Ok
 * test/javax/net/ssl/TLSv12/TLSEnginesClosureTest.java (modified)
  * Ok
 * test/javax/net/ssl/ciphersuites/DisabledAlgorithms.java (modified)
  * Ok
 * test/javax/net/ssl/ciphersuites/ECCurvesconstraints.java (modified)
  * Ok
 * test/javax/net/ssl/interop/ClientHelloBufferUnderflowException.java
(modified)
  * Ok
 * test/javax/net/ssl/interop/ClientHelloChromeInterOp.java (modified)
  * Ok
 * test/javax/net/ssl/sanity/ciphersuites/CheckCipherSuites.java (modified)
  * Ok
 * test/javax/net/ssl/templates/SSLContextTemplate.java (modified)
  * Ok, only default methods allowed in 8u interface
 * test/javax/net/ssl/templates/SSLSocketTemplate.java (modified)
  * Ok
 * test/sun/net/www/protocol/https/HttpsClient/ProxyAuthTest.java (modified)
  * Ok
 * test/sun/net/www/protocol/https/HttpsURLConnection/B6216082.java
(modified)
  * Ok
 * test/sun/net/www/protocol/https/NewImpl/ComHTTPSConnection.java
(modified)
  * Ok, same as before in 8u
 * test/sun/net/www/protocol/https/NewImpl/ComHostnameVerifier.java
(modified)
  * Ok
 * test/sun/security/pkcs11/sslecc/ClientJSSEServerJSSE.java (modified)
  * Ok
 * test/sun/security/ssl/CertPathRestrictions/TLSRestrictions.java
(modified)
  * Ok
 * test/sun/security/ssl/ClientHandshaker/LengthCheckTest.java (modified)
  * Ok
 * test/sun/security/ssl/DHKeyExchange/UseStrongDHSizes.java (modified)
  * Ok
 *
test/sun/security/ssl/SSLContextImpl/CustomizedServerDefaultProtocols.java
(modified)
  * Ok, TLSv1.3 is not by default enabled for clients
 * test/sun/security/ssl/SSLContextImpl/DefaultEnabledProtocols.java
(modified)
  * Ok, TLSv1.3 is not by default enabled for clients
 * test/sun/security/ssl/SSLEngineImpl/SSLEngineKeyLimit.java (modified)
  * Ok
 * test/sun/security/ssl/SSLEngineImpl/TLS13BeginHandshake.java (modified)
  * Ok
 * test/sun/security/ssl/SSLSessionImpl/ResumeChecksClient.java (modified)
  * Ok
 * test/sun/security/ssl/SSLSessionImpl/ResumeChecksServer.java (modified)
  * Ok
 * test/sun/security/ssl/SSLSocketImpl/SSLSocketKeyLimit.java (modified)
  * Ok
 *
test/sun/security/ssl/ServerHandshaker/AnonCipherWithWantClientAuth.java
(modified)
  * Ok
 * test/sun/security/ssl/Stapling/StatusResponseManager.java (deleted)
  * Ok, in 8u we build the JAR with a script
 * test/sun/security/ssl/Stapling/StatusResponseManager.sh (new)
  * Ok, in 8u we build the JAR with a script
 * test/sun/security/ssl/X509TrustManagerImpl/ClientServer.java (modified)
  * Ok
 * test/sun/security/ssl/X509TrustManagerImpl/Symantec/Distrust.java
(modified)
  * Ok
 * test/sun/security/ssl/internal/TestRun.java (deleted)
  * Ok, in 8u we build the JAR with a script
 * test/sun/security/ssl/internal/TestRun.sh (new)
  * Ok, in 8u we build the JAR with a script
 *
test/sun/security/provider/certpath/Extensions/OCSPNonceExtensionTests.java
(modified)
  * Ok
 * test/sun/security/provider/certpath/ResponderId/ResponderIdTests.java
(modified)
  * Ok

In addition to the static analysis, I ran the tests of the following
categories (SSL-related):

 * sun/security/ssl
 * javax/net/ssl
 * sun/net/www/protocol/https
 * sun/security/pkcs11
 * sun/security/provider/certpath
 * com/sun/net/ssl

Per-category analysis of tests:

 * sun/security/ssl
  * Test results: passed: 94; failed: 1; error: 3
  * Failures/errors:
   * sun/security/ssl/SSLSocketImpl/ClientTimeout.java
    * Fails in 11.0.7 so this is not a regression - passing in
jdk8u272-b01 (old SunJSSE engine)
   * sun/security/ssl/SSLSocketImpl/NonAutoClose.java
    * Fails in 11.0.7 so this is not a regression - passing in
jdk8u272-b01 (old SunJSSE engine)
   * sun/security/ssl/SSLSocketImpl/SetClientMode.java
    * Fails in 11.0.7 so this is not a regression - passing in
jdk8u272-b01 (old SunJSSE engine)
   * sun/security/ssl/SSLSocketImpl/SSLSocketKeyLimit.java
    * Ok, failed in Webrev.02 but passed in Webrev.03

 * javax/net/ssl
  * Test results: passed: 120; failed: 1; error: 9
  * Failures/errors:
   * javax/net/ssl/compatibility/Compatibility.java
    * Fails in 11.0.7 so this is not a regression - not available in
jdk8u272-b01 (old SunJSSE engine)
   * javax/net/ssl/sanity/ciphersuites/CipherSuitesInOrder.java
    * Fails in 11.0.7 so this is not a regression - passing in
jdk8u272-b01 (old SunJSSE engine)
   * javax/net/ssl/SSLEngine/Basics.java
    * Fails in 11.0.7 and in jdk8u272-b01 (old SunJSSE engine) so this
is not a regression
   * javax/net/ssl/SSLEngine/CheckStatus.java
    * Fails in 11.0.7 so this is not a regression - passing in
jdk8u272-b01 (old SunJSSE engine)
   * javax/net/ssl/SSLEngine/ConnectionTest.java
    * Fails in 11.0.7 so this is not a regression - passing in
jdk8u272-b01 (old SunJSSE engine)
   * javax/net/ssl/SSLEngine/EngineCloseOnAlert.java
    * Fails in 11.0.7 so this is not a regression - not available in
jdk8u272-b01 (old SunJSSE engine)
   * javax/net/ssl/SSLEngine/IllegalHandshakeMessage.java
    * Fails in 11.0.7 so this is not a regression - not available in
jdk8u272-b01 (old SunJSSE engine)
   * javax/net/ssl/SSLEngine/IllegalRecordVersion.java
    * Fails in 11.0.7 so this is not a regression - passing in
jdk8u272-b01 (old SunJSSE engine)
   * javax/net/ssl/SSLEngine/TestAllSuites.java
    * Fails in 11.0.7 so this is not a regression - not available in
jdk8u272-b01 (old SunJSSE engine)
   * javax/net/ssl/SSLSession/CheckMyTrustedKeystore.java
    * Fails in 11.0.7 so this is not a regression - not available in
jdk8u272-b01 (old SunJSSE engine)

 * sun/net/www/protocol/https
  * Test results: passed: 27; error: 1
  * Failures/errors:
   * sun/net/www/protocol/https/HttpsURLConnection/CloseKeepAliveCached.java
    * Fails in 11.0.7 and in jdk8u272-b01 (old SunJSSE engine) so this
is not a regression

 * sun/security/pkcs11
  * Test results: passed: 65; failed: 5; error: 1
  * Failures/errors:
   * sun/security/pkcs11/ec/TestKeyFactory.java
    * Fails in 11.0.7 and in jdk8u272-b01 (old SunJSSE engine) so this
is not a regression
   * sun/security/pkcs11/fips/ClientJSSEServerJSSE.java
    * Ok, 'ignore' label removed in Webrev.04 and test passes as in 11.0.7
   * sun/security/pkcs11/KeyStore/SecretKeysBasic.sh
    * Fails in 11.0.7 and in jdk8u272-b01 (old SunJSSE engine) so this
is not a regression
   * sun/security/pkcs11/Secmod/GetPrivateKey.java
    * Failed in my initial batch run and passed when run standalone.
Does not seem to be a TLS issue anyways. Ok.
   * sun/security/pkcs11/Secmod/JksSetPrivateKey.java
    * Failed in my initial batch run and passed when run standalone.
Does not seem to be a TLS issue anyways. Ok.
   * sun/security/pkcs11/tls/TestKeyMaterial.java
    * Fails in 11.0.7 and in jdk8u272-b01 (old SunJSSE engine) so this
is not a regression

 * sun/security/provider/certpath
  * Test results: passed: 11

 * com/sun/net/ssl
  * Test results: passed: 2

I ran all SSL-related tests and observed no regressions.

The following comments are part of this review: [1] [2] [3].

With that said, Webrev.04 looks good to me.

Thanks,
Martin.-

--
[1] -
https://mail.openjdk.java.net/pipermail/jdk8u-dev/2020-August/012523.html
[2] -
https://mail.openjdk.java.net/pipermail/jdk8u-dev/2020-August/012550.html
[3] -
https://mail.openjdk.java.net/pipermail/jdk8u-dev/2020-August/012565.html

More information about the jdk8u-dev mailing list