Question/feedback regarding Windows Hi DPI support and how it will affect applications

[Jim Graham]

I'm currently investigating what changes we need to make to get Windows 
HiDPI support up and running.  As it stands, anyone with a Hi DPI 
Windows machine will see all Java and JavaFX programs run very tiny 
since the Java executables have declared that we are "DPI Aware" in the 
program manifest for a few releases now.  That setting was necessary to 
prevent the platform from pixel-scaling us back at a time when the 
actual DPIs of screens were not that drastically different from the norm 
and pixel scaling looked remarkably ugly compared to the amount of size 
difference it bought you.

But, things are changing rapidly and there are a number of screens out 
there now upon which Windows recommends that you should scale by 1.5x up 
to 3x.  A retina iMac or MacBook Pro (220 DPI) running Windows 7+, for 
example, will request us to scale by 2x, we'll implicitly indicate that 
we've heard that request by the fact that the Java manifest says we are 
"System DPI Aware" and they'll trust our sizes in pixels even though we 
calculated them based on a regular DPI screen - and we end up being half 
the size we want and nearly unreadable.  There are also Windows laptops 
such as the Lenovo Yoga 2 Pro (released Oct 2013) and Yoga 3 Pro that 
have nearly 300DPI screens where the default pixel scale requested is 
3x.  Java and FX windows are even tinier and harder to read on those 
screens.

(Side note: In FX, as it turns out, our "em" scaling in css is actually 
DPI aware so if you sized everything using CSS and the "em", then you 
might actually look OK.  In practice, though, there are a ton of pixel 
dimensions in the default L&F CSS files so you'd have to do quite a bit 
of customization to make that work.)

Adding a scale factor to the Windows platform code is not hard, and I 
have that code up and running.  But, we run into the following issues 
that didn't come up when we did the retina port:

- non-integer coordinates in events

Our Mac code truncates and filters events to integers, but Windows may 
specify non-integer scaling so this gets trickier now.  Certainly we can 
round or truncate, and filter for integers, but we end up in situations 
where the user can see the mouse move, but the program does not react 
because that event was filtered out.  Note that a test program on the 
Mac does show that we sometimes repeat Mouse coordinates on retina 
screens which shows that even with the 2x scaling on Mac we are seeing 
repeated coordinates - they are somewhat rare, though, since I think the 
Mac internally only processes mouse moves in a virtual space.  It will 
be much more common with Windows more flexible scaling, though, since 
the mouse cursor moves based on pixel distances, not based on filtered 
event distances.

- screen dimensions

For a single screen we can simply divide by the pixel scale, but with 3x 
scaling the division produces non-integers.  This would also show up if 
we embrace the 125% and 150% scales that Windows documents/APIs 
recommend we support.

But, a bigger issue happens with multi-monitor setups.  The Mac lays out 
the screens (in the Control Panel UI, and in the bounds they report) in 
virtual coordinates based on the pixel scale.  That, combined with the 
fact that they use 2x scaling only and screens are even numbers of 
pixels in size, means we can pass those numbers on and they don't look 
odd.  But, Windows allows a wider range of scales (125%, 150%, 200%, 
300%) and it also lays out screens in pixels which raises a couple of 
issues - non-integer screen sizes for some combinations of scale factors 
and pixel sizes, and also screens are supposed to lie against each 
others' edges so the user can line them up perfectly in the Control 
Panel and if we try to represent that same layout in "virtual scaled 
coordinates" the monitors may not line up correctly.  I'll get into more 
detail on that below.

So, my questions to developers are:

- Would seeing non-integer event coordinates break your code?
- Would seeing non-integer window sizes and positions break your code?
- Would non-integer screen sizes break your code?
- If we try to translate Windows Screen pixel sizes into virtual 
coordinates and the layout doesn't quite represent what they see in the 
Control Panel, how much of a problem is that?
- If we try to translate Windows Screen pixel sizes into virtual 
coordinates and, due to a complex layout, our algorithm cannot get all 
of the edges to match each other as they do in pixel space and we need 
to punt, how much might it break code if there are gaps or overlaps in 
the reported Screen bounds (i.e. this is worse than "the layout doesn't 
quite look the same" I asked about above)?
- If we provide a flag or scale factor on the screens to indicate that 
the dimensions provided are in pixels and you must convert your desired 
window sizes by the indicated scale factor in order to know how big to 
make your window - how much impact would that have?
- As a followup to "screens in pixel sizes", we could alternately 
arrange so that specifying how big your Scene is and/or using the 
standard setWidth()/setHeight() methods on stage and relying on default 
window positioning or using the default "centerOnScreen()" types of 
positioning methods could be taught to work regardless of what we decide 
here.  It would only be if you wanted to correlate all of the screens 
and come up with your own x,y,w,h - that would be where you might need 
to be aware if the screen dimensions are not in the same space as scene 
dimensions.

To get an idea of the kinds of problems we might encounter when trying 
to translate the screens into "virtual coordinates", consider that the 
default layout of linearly placing all screens next to each other is 
easy to translate (though if one screen doesn't "divide out" evenly then 
some of the locations might be non-integers).  But, consider a 3x3 grid 
of displays (i.e. 9 monitors in a square grid layout).  If they are 
identical then the layout would be a perfectly aligned grid.  If the 
middle screen has double (or half) the resolution of the others then we 
have to either have it float in the middle of a "virtual size" space 
that is too large for its dimensions, or have it overlap the 8 screens 
around the outside, or push the 8 screens around the outside away from 
each other so that they no longer touch.  Since the user laid the 
screens out in pixels in the Control Panel UI, they believe that they 
may have created a perfect grid, but we will have to present an 
imperfect grid to the JavaFX program if we want to maintain the illusion 
that FX Screen object sizes are scaled.  Even worse, the adjustments we 
have to make would be heuristic and depending on what we choose it might 
break what they wanted to achieve, or it might not matter, it depends on 
what they were thinking and what we independently decided as to how to 
best represent it.

So, we could:

- Expose the "screen dimensions are pixels which may not match the 
coordinates in your scenes" property of Windows screens.  This may cause 
problems for programs that examine the screens and attempt to do their 
own custom positioning, but should not matter to programs that trust the 
default positioning, or centerOnScreen() method(s?).

- Expose "screen dimensions are pixels", but make common APIs work in 
Virtual coordinates - scene/stage.setWidth/Height() could be in virtual 
coordinates for instance.  Screen might have multiple getBounds() 
methods depending on if you want virtual coordinates or pixel 
coordinates and only relying on the pixel coordinate versions will work 
in Windows in all cases.  The issue here is that sometimes the base APIs 
might not correlate well with each other, but they will make sense if 
you know the trick.

- Try to convert Windows screens into logical coordinates that make 
sense.  It will work for 99% of cases probably most of which are single 
screen or default linear layouts.  In some number of corner cases, 
though, we may get it wrong and the values you find in screen bounds 
could have gaps or overlaps or bear no resemblance to how the user 
perceives his workspace.

- Any other ideas?

			...jim