Windows support for HiDPI displays, call for feedback and testing

[Dr. Michael Paus]

Hi Jim

let me answer your last questions first.

 > Just to be sure I understood that last paragraph correctly
 > - adding a ColorAdjust to a group of ImageViews makes it slow down on 
8u60, but not on 8u40.

Yes, but I was not able to reproduce that behavior in a simple example. 
Otherwise I would have written an official bug report already.

 > Having the same group without the ColorAdjust does not affect 
performance between 8u40 and 8u60.

Yes.

 > Setting the cache hints on the group slows it down on 8u60, but not 8u40.

No. Setting group.setCache(true) also fixes the problem. So I currently 
have too options on the Mac.
Either eliminate the ColorAdjust effect OR setCache(true). The first one 
is no real option because
I need the functionality and the second seemed to be a waste of memory 
to me and was not needed
until now.

Now to your other questions. There was typo in my mail. Of course I 
wanted to say "Screen.getRenderScale"
and not "System...". (I know this is private but read on ...)

If I understand you correctly you are saying that all this 
HiDPI-handling is done under the covers and should
be fully transparent to the programmer, whereas I think that this is not 
possible for non-trivial cases.
(Is this foo.<suffix> versus foo at 2x.<suffix> handling documented anywhere?)

Just a few questions:

How do you determine which images to download from a server or database 
if you don't know the render scale?

How do you handle cases where your images are read from a stream where 
you do not have a @2x name extension?
(Same for other non-named image sources.)

How do you draw thin (single pixel wide) lines if you don't know the 
render scale?

How do you display an image at exactly 100% scale (like the option in 
Photoshop for example) if you don't know the render scale?

How do you take a snapshot of a node at full resolution? For example a 
500x500 canvas will be internally backed by a 1000x1000
image which looks crisp on your Retina screen. When you now take a 
snapshot of this canvas it will be a blurry 500x500 image.
(At least that was the result when I last tried it.)

There are workarounds for all these issues but only if you know the 
render scale.

Michael


Am 10.06.15 um 00:19 schrieb Jim Graham:
> Hi Michael,
>
> I'm not sure I understand the problem you were having with 
> getRenderScale.
>
> What is "System.getRenderScale()"?  Where are you getting that from?
>
> What is the tile cache and why was it affected by the hidden scale 
> that we have been intending to not affect applications in any way 
> (other than what the user can see visually on the screen)?  Is this 
> something you implement in your code?
>
> Since we apply the render scale under the covers it shouldn't matter 
> what size your images are - they should be displayed scaled by the 
> render scale.  The only "scaling" that affects images is if you 
> request foo.<suffix> and we find a foo at 2x.<suffix> then we will load 
> it and internally downscale it by 2x when we render it, but that 
> should be the same on both Mac and Windows.
>
> Just to be sure I understood that last paragraph correctly - adding a 
> ColorAdjust to a group of ImageViews makes it slow down on 8u60, but 
> not on 8u40.  Having the same group without the ColorAdjust does not 
> affect performance between 8u40 and 8u60. Setting the cache hints on 
> the group slows it down on 8u60, but not 8u40.  Did I get all of that 
> correctly?
>
>             ...jim
>
> On 6/9/2015 12:45 PM, Dr. Michael Paus wrote:
>> Hi
>> I finally managed to set up a non-retina Windows 7 box and
>> get my software running on it. I tried it with JDK 8u45 and 8u60b18.
>>
>> It worked with 8u45 out of the box and the performance was ok.
>>
>> With 8u60 the whole application seemed to be broken because all
>> tiles where displayed at only half their normal size. The reason for 
>> this
>> was that the system suddenly behaves like a retina device and
>> System.getRenderScale returns 2.0 instead of 1.0 as before.
>> This problem was easy to fix though because I just had to erase
>> the tile cache manually. The decision to download retina tiles
>> (512x512) or normal tiles (256x256) is based on this factor.
>> I did not expect such a change between runs of the software.
>>
>> After fixing this issue the application behaved in the same way
>> as with 8u45. So a semi-retina :-) Windows box does not seem to
>> suffer from the performance drop.
>>
>> But there is something else I observed on the Mac. I found out
>> that the performance drop seems to be triggered by an effect
>> that I have added to the group containing all the ImageViews.
>> Its a ColorAdjust effect which I used to control the brightness
>> of the map. As soon as I remove it from the group the performance
>> is ok again. Something else that has the same effect is to activate
>> the caching of the group but I wonder why that was not necessary
>> before.
>>
>> But again I failed to create some simple example to show this
>> effect. It drives me nuts.
>>
>> Michael
>>
>> PS: I also tried the options proposed by Kevin but they did not change
>> anything
>> and the output suggest that I am not using too much vram.
>>
>>
>> Am 08.06.15 um 20:33 schrieb Jim Graham:
>>> The only thing that might affect this was the change in the default
>>> size of the vram pool, but that happened in 8u40 which predates both
>>> of the builds involved in the experiment.  I'd be curious as well to
>>> hear how the application is affected on non-retina screens, and
>>> Windows/D3D for that matter...
>>>
>>>             ...jim
>>>
>>> On 6/8/15 5:42 AM, Kevin Rushforth wrote:
>>>> Does the same slowdown happen on a non-Retina system?
>>>>
>>>> How large is your image? How large are the individual tiles? One thing
>>>> that occurs to me is that your application might be thrashing texture
>>>> memory (although I can't think of anything that changed in 8u60 that
>>>> would affect this). You might try the following debug flag:
>>>>
>>>> java -Dprism.poolstats=true
>>>>
>>>> and see if there is a lot of thrashing of texture memory. Also, you 
>>>> can
>>>> increase the memory that will be used for textures to, say, 1Gbyte
>>>> (default is 512Mbytes) by:
>>>>
>>>> java -Dprism.maxvram=1g
>>>>
>>>> and see if that makes a difference. Jim might have additional
>>>> suggestions.
>>>>
>>>> -- Kevin
>>>>
>>>>
>>>> Dr. Michael Paus wrote:
>>>>> Many thanks for the clarification.
>>>>>
>>>>> I'd also like to be more specific on the performance drop I observed
>>>>> but I have to
>>>>> admit that I don't have any idea what is going on there. The only
>>>>> thing I can say
>>>>> for sure is that I can consistently reproduce this drop in 
>>>>> performance
>>>>> by switching
>>>>> between 8u45 and 8u60 (tested with b08 and b15).
>>>>>
>>>>> My application contains a map view (something similar to Google Maps)
>>>>> which
>>>>> displays an area of retina map tiles. By dragging with the mouse I 
>>>>> can
>>>>> move this
>>>>> map around within the map pane. This works smoothly with 8u45 but is
>>>>> completely
>>>>> shaky with 8u60. I even experimented with various techniques to
>>>>> implement such
>>>>> a map view (single large image, single large canvas, individual
>>>>> images/canvases for
>>>>> each map tile) and they all show the same behavior, so it cannot be
>>>>> something which
>>>>> is specific to a particular implementation approach. During my test
>>>>> all tiles where already
>>>>> cached in memory, so I can exclude any file or network IO related
>>>>> issues.
>>>>>
>>>>> I tried to implement a little example where I just move around a 
>>>>> large
>>>>> image but in
>>>>> this simple scenario the problem does not show up.
>>>>>
>>>>> So in order to do a bit more precise measuring I added an
>>>>> AnimationTimer to my map pane
>>>>> which I use to compute a new position of the map for each frame so
>>>>> that a point on the map
>>>>> moves along a circle. This helps to show the performance drop more
>>>>> precisely than moving
>>>>> around the map by hand. This is a smooth movement with 8u45 but shaky
>>>>> with 8u60.
>>>>>
>>>>> But one other observation puzzles me completely. The handle-method of
>>>>> the timer is called with
>>>>> a frame rate of 60 Hz even though the graphics output is shaky and 
>>>>> has
>>>>> a visual frame rate
>>>>> of something between 2 - 5 Hz. It is new to me that something like
>>>>> this is technically possible.
>>>>> It looks as if the system is running at full speed but skips
>>>>> displaying most of the generated
>>>>> frames for some reason. Is such a behavior possible at all?
>>>>>
>>>>> Am 05.06.15 um 22:49 schrieb Jim Graham:
>>>>>> I actually use a retina MBP for testing - dual booting Windows and
>>>>>> MacOS.
>>>>>>
>>>>>> There should be no changes at all on other platforms, 
>>>>>> particularly in
>>>>>> regard to rendering speed.  There was one small regression on Linux
>>>>>> resulting in a bad first paint due to asynchronous initialization of
>>>>>> the Window size, but no performance issues.  Can you be more 
>>>>>> specific
>>>>>> about where you see the performance drop? What kind of scene
>>>>>> graph/animation?
>>>>>>
>>>>>> The system properties are all Windows specific, thus the "glass.win"
>>>>>> prefix.  In 9 we might generalize all of the HiDPI support and offer
>>>>>> something more centralized, but those are just there for testing the
>>>>>> new Windows support for now.  We'll also look at how to advertise 
>>>>>> the
>>>>>> pixel scaling to applications at that time.
>>>>>>
>>>>>> One thing to note is that the way that Mac retina support is 
>>>>>> done, if
>>>>>> the user ever sets their control panel to anything other than "just
>>>>>> pick the best conditions for this display" then there is
>>>>>> pixel-scaling going on behind the scenes in the OS, so any attempts
>>>>>> to try to line up with pixels will be muddied by the virtual DPI 
>>>>>> they
>>>>>> are emulating at the system level.  We'll also be looking at ways to
>>>>>> bypass their built-in "virtualized" HiDPI support in a later release
>>>>>> so that we can actually talk directly to display pixels 
>>>>>> regardless of
>>>>>> CP settings.
>>>>>>
>>>>>> This virtualized scaling is mitigated by the fact that this HiDPI
>>>>>> pixel stretching is happening on a HiDPI display and so any linear
>>>>>> interpolation is hidden by tiny pixel sizes.
>>>>>>
>>>>>> To that end, we are doing something similar with Windows scaling.
>>>>>> There are a number of places in the FX code that assume that it can
>>>>>> predict an integer translation from the FX code and so we always
>>>>>> render at integer scales so that integers in FX Scene coordinates 
>>>>>> map
>>>>>> to integers in rendered pixel coordinates.  We'll try to fix 
>>>>>> those in
>>>>>> 9 so that we can do non-integer scaling, but until then there 
>>>>>> will be
>>>>>> the same disconnect between trying to line up with display pixels 
>>>>>> and
>>>>>> the actions of HiDPI as Mac OS already has...
>>>>>>
>>>>>>             ...jim
>>>>>>
>>>>>> On 6/5/15 3:03 AM, Dr. Michael Paus wrote:
>>>>>>> I cannot provide any Windows specific feedback on HiDPI displays
>>>>>>> because
>>>>>>> I don't own one but I am working on a MacBook Pro with Retina 
>>>>>>> display
>>>>>>> and thus would like to ask a few questions in this context.
>>>>>>>
>>>>>>> How are other platforms (Mac, Linux) affected by the changes you 
>>>>>>> made
>>>>>>> for the Windows support? (On my Mac I noticed a severe performance
>>>>>>> drop
>>>>>>> when using 8u60 which may possibly be related to these changes.)
>>>>>>>
>>>>>>> Are any of the special system properties you mentioned also 
>>>>>>> usable on
>>>>>>> other platforms?
>>>>>>>
>>>>>>> Is there any way now to find out what the current pixel scale 
>>>>>>> factor
>>>>>>> is?
>>>>>>> Without that knowledge it is impossible to do proper graphics on 
>>>>>>> any
>>>>>>> HiDPI screen. (E.g., you can't position graphic elements 
>>>>>>> correctly in
>>>>>>> order to get crisp lines or you can't get a crisp snapshot of a
>>>>>>> node if
>>>>>>> you don't take special actions based on the pixel scale.)
>>>>>
>>