RFR: 8311596: Add separate system properties for TLS server and client for maximum chain length [v8]

[Xue-Lei Andrew Fan]

On Thu, 9 Nov 2023 19:03:27 GMT, Sean Mullan <mullan at openjdk.org> wrote:

>> I'm not sure if the number 8 or 10 really make a good difference in practice.  No matter 8 or 10,  if customers need lower value, they can always consider adjusting it.  My concern is mainly about compatibility issues.  If you want to keep the behavior changes, as there is potential compatibility issue, please feel free to describe the behavior change in release note and CSR if you would like.
>
> Good point - the CSR and RN could have been a bit more specific about the compatibility effect of changing the default from 10 to 8, so we will update that. Note that the CertPathBuilder default max path length [is 5 non-self-issued intermediate CA certificates](https://docs.oracle.com/en/java/javase/21/docs/api/java.base/java/security/cert/PKIXBuilderParameters.html#setMaxPathLength(int)), so even a lower value of 8 should have low risk as rigid TLS cert chains greater than 6 certs (where the 6th cert is the end entity cert which is not affected by the CertPathBuilder limit) on the wire will already be rejected. The additional certs permitted in the wire format is more for including a few more additional certs that might help build a valid path when there is more than one possible chain that a server or client might accept, which sometimes happens.

Good point about the setMaxPathLength() limitation.  It looks like there might be an issue when the interactivities of setMaxPathLength(and its default value) and the properties defined here are not considered.  For example, what if the property is set to 8, while the setMaxPathLength is of value 5?  For the "PKIX" trust manager, per your description, if the property is set to 8, but 5 is the limit actually.  It looks like a weird behavior to me.  If I remember correctly, the "SunX509" trust manager does not use PKIXBuilderParameters, while the "PKIX" trust manager does.  It might be not the behavior we'd like to have that property 8 work for "SunX509" but not for "PKIX" trust manager.

Anyway, it might be better to look into the interactive behaviors among the properties and setMaxPathLength/default value.

> Good point - the CSR and RN could have been a bit more specific about the compatibility effect of changing the default from 10 to 8, so we will update that. Note that the CertPathBuilder default max path length [is 5 non-self-issued intermediate CA certificates](https://docs.oracle.com/en/java/javase/21/docs/api/java.base/java/security/cert/PKIXBuilderParameters.html#setMaxPathLength(int)), so even a lower value of 8 should have low risk as rigid TLS cert chains greater than 6 certs (where the 6th cert is the end entity cert which is not affected by the CertPathBuilder limit) on the wire will already be rejected. The additional certs permitted in the wire format is more for including a few more additional certs that might help build a valid path when there is more than one possible chain that a server or client might accept, which sometimes happens.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/15163#discussion_r1388988092