RFR: 8329623: NegativeArraySizeException encoding large String to UTF-8

[Raffaello Giulietti]

On Fri, 5 Apr 2024 17:44:38 GMT, Roger Riggs <rriggs at openjdk.org> wrote:

> When encoding a vary large string in String.getBytes(StandardCharset.UTF_8) computation of the buffer size may exceed the range of a positive 32-bit Integer.
> If the estimated size for the result byte array is too large, pre-compute the exact buffer size. 
> If that exceeds the range, then throw OutOfMemoryError.

Code fix looks good, but see the comment in the test.

test/jdk/java/lang/String/CompactString/MaxSizeUTF16String.java line 143:

> 141:         // Strings of size min+1...min+2, throw OOME
> 142:         // The resulting byte array would exceed implementation limits
> 143:         for (int count = min + 1; count < max; count++) {

The case `min + 1` cannot lead to a `NegativeArraySizeException` in the current code, since `3 * (min + 1) <= MAX_VALUE`. In theory, it should succeed by returning the encoded `byte[]`, although It throws `OOME` for exceeding VM limits. That is, this case does not trigger the invocation of `computeSizeUTF8_UTF16()` in the proposed fix.

Only `min + 2` throws `NegativeArraySizeException` in the current code, and thus the invocation of `computeSizeUTF8_UTF16()` in the proposed fix.

-------------

PR Review: https://git.openjdk.org/jdk/pull/18663#pullrequestreview-1985830221
PR Review Comment: https://git.openjdk.org/jdk/pull/18663#discussion_r1555466467