<AWT Dev> RFR: 8182043 Access to Windows Large Icons

[Alexey Ivanov]

I've been playing with a build that contains the proposed fix on 
different devices for a while.

I see no benefit in fetching several sizes of the icon from Windows API, 
at this time at least. Rather the opposite: at 150% the rendered icons 
look pixelated; however, in the build without the fix all the icons look 
crisp. More details on this below.

On 17/06/2020 18:59, Philip Race wrote:
> Icon has API that returns size.
> I was wondering how problematic this is for returning an Icon backed 
> by an MRI
> But this being a Swing API I suspect the reason for it is so Swing can 
> lay out.
> So it does not want to know the real icon size in "device" pixels.
> That would actually be a problem not a benefit. It wants to know the 
> user coordinate size for layout.
> I think it also means that the size can't change when you drag between 
> display devices with
> different UI scales.
>
> So maybe we can do this properly with an MRI backed Icon and just 
> return whatever is the base image size.
> Also it means that maybe we don't need a new API at all.
> I've not understood all of the reasoning behind many of the 
> suggestions (going back years)
> very well but if this can be done without API it would be very helpful 
> in general.
> No problem backporting and apps will get the benefit without re-coding.

I believe it is how it works now. I mean the application requests the 
logical size, if Windows Shell API returns the icon of the requested 
size, the BufferedImage with icon pixels is returned; if Windows Shell 
API returns the icon of different size, the BufferedImage is wrapped 
into MultiResolutionIconImage. The latter case handles High DPI 
environments.

The current Swing API provides access to two sizes: 16×16 and 32×32, 
small and large correspondingly.

As I understand the RFE, access to other possible icon sizes is desired. 
If a programmer wants to display a larger icon, say 48×48, in their own 
control, there's no way to get such an icon without writing code to 
extract this icon. Before Java 9, the programmer could access the 
internal Swing API to retrieve this icon; with module encapsulation it 
has become impossible as explained in JDK-8156183 [1].

Thus, the purpose of the new API is not for High DPI support.

[1] https://bugs.openjdk.java.net/browse/JDK-8156183



As for High DPI support, there's still room for improvement. The icons 
in Windows are essentially a multi resolution image but we're not using 
this fact: there's always only one icon size stored. Alexander 
implemented it now: the icon contains several other sizes in addition to 
the requested one. But something does not work as expected.

I have my main monitor and system scaling set to 100%, a regular 
environment. I've been using SwingSet2 for my testing.

With Alexander's proposed fix, File Chooser requests two sizes 16 and 
32. This scenario works fine.

If I launch it with uiScale=2.0, the UI is scaled; File Chooser requests 
two size 16 and 32, just like before. But the icons in the file list are 
scaled versions of 16×16 instead of 32×32 as I would expect. This is 
clearly seen because folder icon is flat in 16×16 but it's 3D-like in 
32×32. (I cannot explain why this does not work.)

If I set my laptop screen scaling to 150%, my main monitor and system 
scaling remains at 100%, and start SwingSet2 without setting uiScale, 
then it works as expected on the main monitor. If I move JFileChooser to 
my laptop screen, the UI scales up, but the icons are still scaled up 
versions of the base 16×16 icon. Windows Explorer starts to display 
24×24 icon in this case (folder icon is 3D-like in 24×24 too).


I also tested it with system scaling set to 150% and 200%. This scenario 
works fine without the fix: the displayed icons are crisp. As I 
mentioned above, some of the icons look pixelated in 150% setting with 
the proposed fix. In 200% scaling, File Chooser UI got weird: file list 
had a mix of 64×64 and 32×32 icons displayed for files; text on the left 
side panel was clipped.
http://cr.openjdk.java.net/~aivanov/8182043/webrev.01-JFileChooser-200.png

This is what happens, to the best of my understanding. To get the icon 
pixels, a DC is used; this DC is created with GetDC(NULL). Swing 
requests 16×16 icon, but the API actually returns 24×24 and 32×32 icons, 
the scaling being 150% and 200% correspondingly. This is why sometimes 
BufferedImage is wrapped in MultiResolutionIconImage, see JDK-8151385 [2].

And at 200% scaling, the rendering code gets confused: the 
MultiResolutionIconImage returned for the base size of 16 now contains 
32×32 image, and 64×64 image for the base size of 32. Since the initial 
size is 16, when scaling is set to 200%, the 32px-sized image should be 
used for rendering but there's larger image actually.

This problem also exists at 150%, the images for 16 and 32 come from 
native as 24 and 48.

I didn't test the High DPI scenario where another monitor with 100% 
scaling exists in the system. I expect that icon sizes do not adjust 
correctly, I mean that a larger icon will be down-scaled just like a 
smaller icon is up-scaled in the case where main display is at 100%.


[2] https://bugs.openjdk.java.net/browse/JDK-8151385



To address JDK-8182043, this RFE, I would suggest extending the current 
API to allow for arbitrary size. The requested size is in Swing logical 
pixels, it's passed to native code. An application developer would 
choose the size, JFileChooser will continue to use 16 and 32 as the icon 
size like it does now. The result will contain either BufferedImage or 
MultiResolutionIconImage to accommodate High DPI environments.

I would suggest submitting a separate bug to address High DPI 
environment issues and to fetch a more suitable icon size from Windows. 
The icons displayed in JOptionPane are also affected.


Regards,
Alexey

>
>
> If this is not possible can someone summarise why it is absolutely 
> impossible.
>
> Going back the submitter needed to dig out the hi-res icon because we 
> didn't do it for them but
> that doesn't mean it is what he wants to do, it is what he had to do. 
> He'll have to re-code
> regardless because of using internal APIs.
>
> So if we re-phrase the problem as "automatically pick the best image 
> at *display* time
> without requiring applications to re-code", what would this look like 
> and what is lost ?
>
> Would it look like no API change, and the only thing that is lost is 
> what does not actually
> exist (API for the application to dig in to internals) ?
>
> -phil.
>
> On 6/15/20, 9:40 PM, Sergey Bylokhov wrote:
>> Hi, Alexander.
>>
>> Some generic questions about the general approach:
>>  - Did you experiment using LookupIconIdFromDirectoryEx? This method 
>> was referred in the previous email.
>>  - Did you check how the performance will be affected if we always 
>> load a few icons, all icons, or if we try to load them on
>>    the fly when the user request some size from the MRI?
>>  - As far as I understand to access the list of icons the user will 
>> need to use instanceof MRI, can we improve it somehow?
>>    Probably add example to the javadoc, and to the test?
>>  - What about returning the array of icons, will that solution be 
>> simpler/faster then using MRI or not?
>>  - Did you try to merge implementation of 
>> getSystemIcon(File)/getSystemIcon(File, size)?
>>  - I think we need to improve the spec for the new method:
>>    * "width and height of the icon in pixels to be scaled" - what 
>> does it mean "to be scaled", is
>>      the returned icon will always have this size in pixels?
>>    * "Whenever possible the icon returned will be a multi-resolution 
>> icon image
>>       with maximum quality icon provided by the system and the size 
>> stored
>>       reflecting the size requested by the user. That will allow nice 
>> scaling
>>       with differen magnification factors." Does our implementation 
>> follow this spec?
>>
>> On 6/9/20 2:24 pm, Alexander Zuev wrote:
>>>    while generally agree i'm not sure that we need to populate ALL 
>>> of the resolutions for every sing> icon we request. So my idea that 
>>> you can see in the new webrev is to create the true MRI which will
>>> hold all standatd resolutions within the range of half of requested 
>>> icon size and double of it. That would
>>> cover magnification factors from 50% to 200%. I tested it and the 
>>> icons in the standard JFioleChooser looks
>>> pretty good on display with different magnification factors.
>>
>>  - Looks like in the fix we always use the "size" passed by the user 
>> when we call getShell32Icon() so we
>> always fetch the same icons from the native?
>>  - The change in the makeIcon reverts back the fix for JDK-8151385, 
>> can you please confirm that those fix
>>    still works as expected?
>>  - The new parameter "size" in the makeIcon(long hIcon, int size) is 
>> unused.
>>