RFR: 8296496: Overzealous check in sizecalc.h prevents large memory allocation [v3]

[Sergey Bylokhov]

On Mon, 14 Nov 2022 19:20:25 GMT, Sergey Bylokhov <serb at openjdk.org> wrote:

>> Alexander Zuev has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   Fixed casting in SAFE_SIZE_NEW_ARRAY2
>
> src/java.desktop/share/native/common/awt/utility/sizecalc.h line 94:
> 
>> 92: 
>> 93: #define SAFE_SIZE_NEW_ARRAY2(type, n, m) \
>> 94:     (IS_SAFE_SIZE_MUL((m), (n)) && IS_SAFE_SIZE_MUL(sizeof(type), (n) * (m)) ? \
> 
> Why we do not cast it here: (n) * (m)?

Filed https://bugs.openjdk.org/browse/JDK-8296957

-------------

PR: https://git.openjdk.org/jdk/pull/11030