RFR: 8277299: STACK_OVERFLOW in Java_sun_awt_shell_Win32ShellFolder2_getIconBits [v3]

[Alexander Zuev]

On Fri, 3 Dec 2021 13:26:57 GMT, Alexey Ivanov <aivanov at openjdk.org> wrote:

>> Alexander Zuev has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   Eliminate potential resource leak.
>
> src/java.desktop/windows/native/libawt/windows/ShellFolder2.cpp line 1068:
> 
>> 1066:             }
>> 1067: 
>> 1068:             if (colorBits != NULL) {
> 
> I think this way defeats the use of exception to handle allocation error. You could use:
> ```c++
> long *colorBits = (long*)malloc(MAX_ICON_SIZE * MAX_ICON_SIZE * sizeof(long));
> if (colorBits != NULL) {
> 
> to achieve the same effect, which is shorter and clearer.

Yes, but then handle_bad_alloc() will not be called which will not raise the OutOfMemoryError on the java side.

> src/java.desktop/windows/native/libawt/windows/ShellFolder2.cpp line 1102:
> 
>> 1100:                 }
>> 1101:                 // Release DC
>> 1102:                 ReleaseDC(NULL, dc);
> 
> The DC has to be released even if `colorBits` allocation failed, so this needs to be after `if (colorBits != NULL)`.

Fixed.

-------------

PR: https://git.openjdk.java.net/jdk/pull/6473