RFR: 8293862: javax/swing/JFileChooser/8046391/bug8046391.java failed with 'Cannot invoke "java.awt.Image.getWidth(java.awt.image.ImageObserver)" because "retVal" is null' [v4]

[Alexey Ivanov]

On Thu, 12 Jan 2023 02:07:06 GMT, Sergey Bylokhov <serb at openjdk.org> wrote:

>>> > Not only _Downloads_ folder returns null, as I was mentioning it was varying for multiple test runs.
>>> 
>>> I remember you were saying that only this one was affected. I may have missed that other folders were affected as well.
>>> 
>>> > Then submitting a new bug for ImageIcon failing to load Image in headless systems right?
>>> 
>>> Yes, for this particular problem: a folder icon cannot be fetched from Windows on a headless system.
>> 
>> Created - [JDK-8299893](https://bugs.openjdk.org/browse/JDK-8299893). Shall I integrate this PR by removing the test from problemList?
>
>> Created - [JDK-8299893](https://bugs.openjdk.org/browse/JDK-8299893). Shall I integrate this PR by removing the test from problemList?
> 
> Did we have a chance to reproduce it at least once other than accidentally in mach5? If not that not necessarily is a root cause, I doubt that the new bug will not be fixed any time soon.

@mrserb was absolutely right! Marking the test `@headful` will not resolve the problem, it will make us forget about the problem, and apparently it may manifest itself on headful systems too.

I added my own traces and I've found three issues with the code.

**`E_PENDING`**
https://github.com/openjdk/jdk/blob/8cb25d3de494c6d9357a1c199e1a9dbff9be9948/src/java.desktop/windows/classes/sun/awt/shell/Win32ShellFolder2.java#L1157-L1158

An icon handle could be negative: `hIcon = 0xffffffffa25400f7` is perfectly valid handle but it's rejected by our current code.

The same problem exists two lines below:
https://github.com/openjdk/jdk/blob/8cb25d3de494c6d9357a1c199e1a9dbff9be9948/src/java.desktop/windows/classes/sun/awt/shell/Win32ShellFolder2.java#L1161

**Result of `Extract`**
https://github.com/openjdk/jdk/blob/8cb25d3de494c6d9357a1c199e1a9dbff9be9948/src/java.desktop/windows/native/libawt/windows/ShellFolder2.cpp#L997

The result of calling `Extract` isn't verified.

This second error led to returning an invalid handle to the *(default)* folder icon. I saw it only for 16×16 icons, in all these cases something that looks like handle is returned, that is `hIcon` isn't null. Yet when the icon bits are extracted, `fn_GetIconInfo` returns an error code 0x57a - _Invalid cursor handle_.

The two errors combined lead to quite a few *random* failures. No particular host reproduced the problem more often. Sometimes 5 of 10 tests failed, sometimes only 1 of 10. Yet none of the test runs I submitted succeeded before I resolved the above issues.

I'll re-assign the bug to myself and submit a new PR after I do additional tests.

-------------

PR: https://git.openjdk.org/jdk/pull/11104