RFR: 8277299: STACK_OVERFLOW in Java_sun_awt_shell_Win32ShellFolder2_getIconBits

[Sergey Bylokhov]

On Fri, 19 Nov 2021 06:34:30 GMT, Alexander Zuev <kizune at openjdk.org> wrote:

> Made colorBits and maskBits arrays dynamic so they are allocated on heap instead of stack.
> Added regression test.

src/java.desktop/windows/native/libawt/windows/ShellFolder2.cpp line 1060:

> 1058:             int nBits = iconSize * iconSize;
> 1059:             long * colorBits;
> 1060:             colorBits = (long*)safe_Malloc(MAX_ICON_SIZE * MAX_ICON_SIZE * sizeof(long));

I am not sure that the bad_alloc will be properly handled in this Java_sun_awt_shell_Win32ShellFolder2_getIconBits method.
+Probably it will be better to merge assigning into one line.

-------------

PR: https://git.openjdk.java.net/jdk/pull/6473