RFR: 8350807: Certificates using MD5 algorithm that are disabled by default are incorrectly allowed in TLSv1.3 when re-enabled [v3]

[Sean Mullan]

On Mon, 14 Apr 2025 15:19:18 GMT, Artur Barashev <abarashev at openjdk.org> wrote:

>> MD5 algorithm is prohibited by TLSv1.3 RFC to be used in certificates:
>> 
>> 
>> Any endpoint receiving any certificate which it would need to
>> validate using any signature algorithm using an MD5 hash MUST abort
>> the handshake with a "bad_certificate" alert.
>> 
>> 
>> 
>> The bug manifests itself when older versions of protocol are supported besides TLSv1.3, such as TLSv1.2. When multiple protocol versions are supported, both client and server calculate their respective SSLSessions's "localSupportedSignAlgs" based on supported signature algorithms for all active protocols and don't update it when negotiated protocol is established. Then "localSupportedSignAlgs" list is used to validate certificate's algorithm.
>> 
>> While we disable "MD5withRSA" in java.security config, MD5 algorithm should not be allowed in TLSv1.3 regardless of optional configuration.
>> 
>> The underlying issue we are fixing here is not MD5-specific: when multiple TLS versions are supported, we compute local supported algorithms for ALL supported TLS versions. Thus MD5 and other algorithms that are supported in TLSv1.2 are being used when actually TLSv1.3 ends up being the negotiated protocol version.
>
> Artur Barashev has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Update Copyright

test/jdk/sun/security/ssl/SignatureScheme/MD5NotAllowedInTLS13CertificateSignature.java line 57:

> 55:     // Certificates and keys used in the test.
> 56:     // Certificates are signed with signature using MD5WithRSA algorithm.
> 57:     static String trusedCertStr =

We try to avoid hard-coding certificates in tests - can you create these certs as part of a test setup using keytool instead?

test/jdk/sun/security/ssl/SignatureScheme/MD5NotAllowedInTLS13CertificateSignature.java line 270:

> 268:         // MD5 is disabled by default in java.security config file.
> 269:         Security.setProperty("jdk.certpath.disabledAlgorithms", "");
> 270:         Security.setProperty("jdk.tls.disabledAlgorithms", "");

Use `SecurityUtils.removeFromDisabledAlgs` and only remove MD5 from these properties.

test/jdk/sun/security/ssl/SignatureScheme/MD5NotAllowedInTLS13CertificateSignature.java line 304:

> 302:     }
> 303: 
> 304:     private static SSLContext getSSLContext(String trusedCertStr,

Typo: s/trusedCertStr/trustedCertStr/

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/24425#discussion_r2044749198
PR Review Comment: https://git.openjdk.org/jdk/pull/24425#discussion_r2044759691
PR Review Comment: https://git.openjdk.org/jdk/pull/24425#discussion_r2044772389