RFR[10] 8186057: TLS interoperability testing between different Java versions

[Artem Smotrakov]

Hi John,

Please see inline.


On 09/07/2017 10:52 AM, sha.jiang at oracle.com wrote:
>>
>> Then you can build a Cartesian product of all parameters. Client and 
>> sever should take parameters, and say if they support all of the or 
>> not (for example, JDK 6 might not support all features that JDK 9 
>> does). If it does, client and server can configure themselves with 
>> those parameters, and start handshaking.
> Frankly, what case combinations would be tested is a problem. 
> Currently, I don't expand the combinations too much.
> For example, I don't make combinations for data exchange and ALPN, and 
> client auth always be enabled.
> That's why I don't use complex data structures.
> If the test requirements or case combinations could be confirmed, the 
> data structures would be enriched.
That's fine to start with limited amount of parameters, but I believe we 
can design the test to be able to extend it easily in future. That's my 
main point here.
>
>> 2. With the approach above, it might be possible to avoid those 
>> nested loops:
>>
>>   93       for (String caseType : CASE_TYPES) {
>>   94             for (String protocol : PROTOCOLS) {
>>   95                 for (String cipherSuite : CIPHER_SUITES) {
>>   96                     for (JdkInfo serverJdk : jdkInfos) {
> But, how to build the above Cartesian product automatically (not by 
> manual)?
"automatically" means writing some code of course, but I believe we 
don't have to define such an array of cases manually.
>
>> 3. I believe we should cover all supported cipher suites. Test run is 
>> going to take more time, but it think it's okay. Another option is to 
>> introduce two modes:
>> - light: run the test with a couple of cipher suites
>> - heavy: run the test with all supported cipher suites
> I hesitate to do that.
> Different JDK builds and TLS versions supports different cipher suites.
That's fine. If you test JDK 6 against 9, you can use only cipher suites 
which are supported by 6.
> And many cipher suites use the same algorithms on key exchange, 
> signature, encryption/decryption and authentication, then is it 
> necessary to check each of them?
I think it would be worth to be able to do that at least. Again, we 
never know what can go wrong.
> Although we don't know "what's going to break down", I'm not sure a 
> single test could check every point on TLS communication.
Right, we can't test everything, but we can to our best. If we know that 
the test can be easily enhanced in this way, it's worth to do that.
>> A couple of comments about the code:
>> - What's the reason of using disabled SHA1 and RSA with 1024-bit 
>> keys? You might use stronger algorithms and key sizes, and avoid 
>> modifying java.security file
> Different JDK builds possibly support different algorithms and have 
> different restrictions on such algorithms and key sizes. Weaker or 
> stronger, that is relative. And this point should not be a focus of 
> the test.
> So, I just pick the ones that are common in history, and provides an 
> alternative java.security file to avoid possible broken.
That makes sense.
>
>> - Please use try-with-resources if possible (files, sockets, etc)
> The test uses only JDK 6-supported language features, but 
> try-with-resources is introduced by JDK 7.
Here we come to another issue. I believe that it would be good to use 
write clients for all JDK versions. If you use the same code for all 
Java versions, that means that you use only JDK 6 API. Let's consider 
the following:
- we want to test 8 vs 9
- 8 and 9 may have some API (or just functionality) which is not 
supported by 6 (for example, ALPN, SNI)

If you write code which is compatible with 6, then you can use API from 
newer versions, but we still may want to test it.

Artem
>
> Best regards,
> John Jiang
>>
>> Artem
>>
>> On 09/07/2017 08:52 AM, sha.jiang at oracle.com wrote:
>>> Hi,
>>> Please review this test for checking the interop compatibility on 
>>> JSSE among different JDK releases (from 6 to 10).
>>> It covers the cases, like handshake, data exchange, client 
>>> authentication and APLN, on all TLS versions (if possible).
>>> And the selected TLS cipher suites are: 
>>> TLS_RSA_WITH_AES_128_CBC_SHA, TLS_DHE_DSS_WITH_AES_128_CBC_SHA and 
>>> TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA.
>>> For more details, please look though the README.
>>>
>>> Webrev: http://cr.openjdk.java.net/~jjiang/8186057/webrev.00
>>> Issue: https://bugs.openjdk.java.net/browse/JDK-8186057
>>>
>>> Best regards,
>>> John Jiang
>>>
>>
>>
>