RFR: 8365820: Apply certificate scope constraints to algorithms in "signature_algorithms" extension when "signature_algorithms_cert" extension is not being sent [v6]

[Hai-May Chao]

On Tue, 16 Sep 2025 19:24:05 GMT, Artur Barashev <abarashev at openjdk.org> wrote:

>> [JDK-8349583](https://bugs.openjdk.org/browse/JDK-8349583) implementation assumes that OpenJDK client always sends "signature_algorithms_cert" extension together with "signature_algorithms" extension. But we didn't account for `jdk.tls.client.disableExtensions` and `jdk.tls.server.disableExtensions` system properties which can disable producing "signature_algorithms_cert" extension. This is an issue similar to [JDK-8355779](https://bugs.openjdk.org/browse/JDK-8355779) but on the extension producing side.
>> 
>> Per TLSv1.3 RFC:
>> 
>>>  If no "signature_algorithms_cert" extension is
>>>    present, then the "signature_algorithms" extension also applies to
>>>    signatures appearing in certificates.
>> 
>> Also making a few cosmetic changes to the existing code.
>
> Artur Barashev has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains eight additional commits since the last revision:
> 
>  - Add a ticket number to unit tests
>  - Merge branch 'master' into JDK-8365820
>  - Add a server-side unit test. Rename existing tests.
>  - Update tests
>  - Revert "Include RSASSA-PKCS1-v1_5 and Legacy algorithms in signature_algorithms for TLSv1.3"
>    
>    This reverts commit adc236be4bcac11614e2741c99545aa593f6af5b.
>  - Merge branch 'master' into JDK-8365820
>  - Include RSASSA-PKCS1-v1_5 and Legacy algorithms in signature_algorithms for TLSv1.3
>  - 8365820: Apply certificate scope constraints to algorithms in "signature_algorithms" extension when "signature_algorithms_cert" extension is not being sent

test/jdk/sun/security/ssl/SignatureScheme/DisableCertSignAlgsExtForServerTLS13.java line 131:

> 129:                                     // instead, depends on network setup.
> 130:                                     || ex instanceof SocketException));
> 131:                         }

Here for TLS 1.3, handshake always fails because SHA256withRSA is not allowed for client certificates. Would you consider adding a positive test for TLS 1.3 with a client certificate signed with RSASSA-PSS so we could test handshake will succeed as the client complies?

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/26887#discussion_r2367086905