RFR: 8360539: DTLS handshakes fails due to improper cookie validation logic

[Hai-May Chao]

On Thu, 26 Jun 2025 21:36:34 GMT, Artur Barashev <abarashev at openjdk.org> wrote:

> DESCRIPTION OF THE PROBLEM:
> There appears to be a bug in the isCookieValid method of HelloCookieManager that can lead to improper cookie validation results after a new cookieSecret is generated. The cookie version (cookieVersion) is initialized with a random number and increments with each cookie, regenerating the secret every 0xFFFFFF cookies.
> 
> When cookieVersion is negative, the expression ((cookieVersion >> 24) & 0xFF) results in an integer that does not match the signed byte value in cookie[0], causing the else clause to be executed incorrectly. This leads to the use of legacySecret even for valid cookies.

Marked as reviewed by hchao (Reviewer).

-------------

PR Review: https://git.openjdk.org/jdk/pull/26006#pullrequestreview-2964816343