RFR: 8296496: Overzealous check in sizecalc.h prevents large memory allocation [v3]

[Alexey Ivanov]

On Mon, 14 Nov 2022 19:27:44 GMT, Sergey Bylokhov <serb at openjdk.org> wrote:

>> src/java.desktop/share/native/common/awt/utility/sizecalc.h line 94:
>> 
>>> 92: 
>>> 93: #define SAFE_SIZE_NEW_ARRAY2(type, n, m) \
>>> 94:     (IS_SAFE_SIZE_MUL((m), (n)) && IS_SAFE_SIZE_MUL(sizeof(type), (n) * (m)) ? \
>> 
>> Why we do not cast it here: (n) * (m)?
>
> Filed https://bugs.openjdk.org/browse/JDK-8296957

You're right, it must be cast here too, I have missed it.

Would you mind submitting a bug?

-------------

PR: https://git.openjdk.org/jdk/pull/11030