RFR[10] 8186057: TLS interoperability testing between different Java versions

[sha.jiang at oracle.com]

Hi Artem,
Thanks for your comments!
Please see inline.

On 07/09/2017 14:32, Artem Smotrakov wrote:
> Hi John,
>
> Thanks for starting working on this. I believe this tool is going to 
> be very helpful.
>
> Let me skip some coding comments for a while, I have a couple of 
> comments about the design. The main idea is that it should cover as 
> many cases as it can. Even if it might look a bit redundant, I believe 
> it is actually not redundant because we basically never know what's 
> going to break down.
>
> 1. At the moment, you define cases in Compatibility.java
>
>   59     private static final String[] CASE_TYPES = new String[] {
>   60             Utils.CASE_HANDSHAKE,
>   61             Utils.CASE_DATA_EXCHANGE,
>   62             Utils.CASE_ALPN };
>
> I believe there are much more many case which we can implement later. 
> But this approach doesn't looks flexible. For example, let's consider 
> that we'd like to use a couple of extensions in the same time. Let's 
> say we'd like to use SNI and ALPN. It seems like we have to create a 
> separate case for that, and add it manually to the array. Would it be 
> possible to generate cases automatically? For example, you can define 
> parameters of TLS connection, for example:
> - handshake type: full, session-resumption
> - use ALPN: true, false
> - use SNI: true, false
> - use client auth: true, false,
> etc
>
> Then you can build a Cartesian product of all parameters. Client and 
> sever should take parameters, and say if they support all of the or 
> not (for example, JDK 6 might not support all features that JDK 9 
> does). If it does, client and server can configure themselves with 
> those parameters, and start handshaking.
Frankly, what case combinations would be tested is a problem. Currently, 
I don't expand the combinations too much.
For example, I don't make combinations for data exchange and ALPN, and 
client auth always be enabled.
That's why I don't use complex data structures.
If the test requirements or case combinations could be confirmed, the 
data structures would be enriched.

> 2. With the approach above, it might be possible to avoid those nested 
> loops:
>
>   93       for (String caseType : CASE_TYPES) {
>   94             for (String protocol : PROTOCOLS) {
>   95                 for (String cipherSuite : CIPHER_SUITES) {
>   96                     for (JdkInfo serverJdk : jdkInfos) {
But, how to build the above Cartesian product automatically (not by manual)?

> 3. I believe we should cover all supported cipher suites. Test run is 
> going to take more time, but it think it's okay. Another option is to 
> introduce two modes:
> - light: run the test with a couple of cipher suites
> - heavy: run the test with all supported cipher suites
I hesitate to do that.
Different JDK builds and TLS versions supports different cipher suites.
And many cipher suites use the same algorithms on key exchange, 
signature, encryption/decryption and authentication, then is it 
necessary to check each of them?
Although we don't know "what's going to break down", I'm not sure a 
single test could check every point on TLS communication.
> A couple of comments about the code:
> - What's the reason of using disabled SHA1 and RSA with 1024-bit keys? 
> You might use stronger algorithms and key sizes, and avoid modifying 
> java.security file
Different JDK builds possibly support different algorithms and have 
different restrictions on such algorithms and key sizes. Weaker or 
stronger, that is relative. And this point should not be a focus of the 
test.
So, I just pick the ones that are common in history, and provides an 
alternative java.security file to avoid possible broken.

> - Please use try-with-resources if possible (files, sockets, etc)
The test uses only JDK 6-supported language features, but 
try-with-resources is introduced by JDK 7.

Best regards,
John Jiang
>
> Artem
>
> On 09/07/2017 08:52 AM, sha.jiang at oracle.com wrote:
>> Hi,
>> Please review this test for checking the interop compatibility on 
>> JSSE among different JDK releases (from 6 to 10).
>> It covers the cases, like handshake, data exchange, client 
>> authentication and APLN, on all TLS versions (if possible).
>> And the selected TLS cipher suites are: TLS_RSA_WITH_AES_128_CBC_SHA, 
>> TLS_DHE_DSS_WITH_AES_128_CBC_SHA and 
>> TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA.
>> For more details, please look though the README.
>>
>> Webrev: http://cr.openjdk.java.net/~jjiang/8186057/webrev.00
>> Issue: https://bugs.openjdk.java.net/browse/JDK-8186057
>>
>> Best regards,
>> John Jiang
>>
>
>