FXML and high dpi screens

[Tom Eugelink]

Aren't we mixing up things?

Responsive webdesign is primarily focused at how to show the same information on less screen space, using the same elements in the same resolution. The questions here are: what to hide or add, what to place where.

High DPI screens foremost involves keeping things the same visual size for the user, the actual layout does not change as with responsive webdesign, but the attempt is to have it look the same on all DPIs. The questions here are: how do I keep things readable for the user, how do I keep things sharp (because scaling causes blurry borders, especially when scaling up).

Most approaches at least use a non resolution dependent unit; being it dips or old fashioned inch or cm. If you state that you want the font do be 0.4cm, the device (knowing its dpi) can easily calculate the number of pixels. The blurryness and visual misses a.o. come from rounding; 0.4cm may not be exactly 2x the pixels from 0.8cm. The biggest pains are the graphical elements, because they have to scaled and often get all kinds of minor artifacts that make the whole seem off. Vectors initially seem like a good approach to catch this, but as Kirill has shown (http://www.pushing-pixels.org/2011/11/04/about-those-vector-icons.html), this only works for a limited range of resolution.

So maybe the best approach is a combination of vector and what Android does:
- use a device independent unit and scale accordingly
- optionally split DPI into ranges and provide graphical elements for each range, preferably vector but you can't forbid bitmap, preferably scaling down.

Optionally because not all graphical element need several DPI related versions; a vector circle can be be scaled all the way.

And after solving this DPI thing, then there is the question of available screen size. So you can have (http://en.wikipedia.org/wiki/List_of_displays_by_pixel_density):
- a 3 inch, 200 DPI screen
- a 6 inch, 100 DPI screen

Both have 600 pixels horizontally, but the user only has half (or twice) the visual screen. This means responsive layout. This is something that is solved in code/FXML (layout) and CSS (hide/show).

To keep things understandable I would try to keep those two really separate. So two topics:
- DPI related
- visual sized based layout


On 2013-05-20 23:58, Daniel Zwolenski wrote:
> Agree that this would have to be done currently at the FXML or code level currently, it's not just size, it's also layouts and visibility, alignment, etc.
>
> The 'responsive' web stuff usually divides it up by a few key screen resolutions. It doesn't just change sizes but actual layouts as well.
>
> So anything over 1200 wide uses a fixed width layout with absolute coordinates, centered with white space or a pattern outside of the centered view.
>
> Anything between 900 and 1200 uses a smaller fixed width (ie 900). Resizing the screen from 1201 to 1199 causes it to jump from the larger fixed width to the smaller.
>
> Anything between 900 and iPhone size uses a fluid layout (ie the ui shrinks and grows to fill the space - more like a jfx layout manager approach). We also start to see menus get hidden at this size and things might layout differently.
>
> Anything below that changes to a compact view where a menu bar suddenly becomes a compressed drop down menu, a search box changes into a search button that goes to a separate search page, and horizontal rows get stacked vertically, etc.
>
> It's all totally customizable using media selector rules but those 3 or 4 cut off points seem to cover the 90% useful cases.
>
>
>
>
>
>
> On 21/05/2013, at 7:20 AM, Danno Ferrin <danno.ferrin at shemnon.com> wrote:
>
>> The JavaFX CSS as it stands in JDK 7 does not support sizing and positioning in the CSS (there may have been some work in 8, can't remember).
>>
>> But since N is small (1.0, 1.25, 1.5, and 2.0) multiple FXMLs may be the way we solve it, possibly with some mechanical translation.
>>
>> On Mon, May 20, 2013 at 3:15 PM, Daniel Zwolenski <zonski at gmail.com> wrote:
>> Is it worth also looking at web land for strategies here? HTML seems to cater for all screen sizes and resolutions from little androids to massive iMacs.
>>
>> The Bootstrap strategy is to provide alternate CSS rules for different media types and resolutions.
>>
>> http://www.onextrapixel.com/2012/11/12/how-to-use-twitter-bootstrap-to-create-a-responsive-website-design/
>>
>> Just something to throw in the mix.
>>
>>
>>
>> On 21/05/2013, at 6:54 AM, John Moxley <jack at moxley.co.uk> wrote:
>>
>>> To be fair its been about a year since I have shipped any apps, I am currently working on treasury systems in a bank, so not quite as relevant. :-)
>>>
>>> When porting for very small devices, the main rule of thumb was to allow the screen to adapt to larger screens programatically. Lets hypothetically say I was designing an application that displayed a graph of, for arguments sake, sales figures. On a small device I would want a single graph to be displayed, with maybe a button to show the legend in a pop up. As the screen got larger I would want that legend to appear on the graph itself, as it got larger I might want to add options or even more graphs. I certainly wouldn't want the line on the graph to be pixel stretched. I also wouldn't want a massive graph with a button for the legend.
>>>
>>> With high resolution screens, apples solution was to basically add 2X to the name of all its images designed for the retina display, this works if all you are doing is doubling your res, but in the android world pixel density is more in flux, and the costs associated with generating so many images is going to be high. Being able to dynamically determine when and where you apply such tactics is invaluable.
>>>
>>> Can I suggest reading up on best practices at http://mobiforge.com/ It arose from the .mobi domain back in the day when we were dealing with a mix of wap and html4, it might give an insight into the practice of mobile UI designers.
>>>
>>> I think my main point is it has to be adaptable. Android does have too much ui configuration for my liking, but it is powerful. Javafx has got a bigger problem them iOS or Android in that its got to support even more resolutions and devices.
>>>
>>> I don't have a proposal, but I will think on it.
>>>
>>> On 20 May 2013, at 16:53, Richard Bair <richard.bair at oracle.com> wrote:
>>>
>>>> Hi Jack,
>>>>
>>>> Our experience is that the Apple solution is much cleaner, however you're experience is very interesting and (since you are actually shipping apps on it instead of living in the land of theory) relevant. Do you have a proposal?
>>>>
>>>> One thing to consider is that any scale factor that doesn't end up pixel aligned will result in fuzzy UI. Our "snap to pixel" layout snaps to whole numbers for positions -- but based on the scale factor this may or may not end up pixel aligned (otherwise when animating a UI control it would snap from pixel to pixel instead of smoothly animating through them).
>>>>
>>>> Can you give some examples of the type of control you need, and why?
>>>>
>>>> Thanks!
>>>> Richard
>>>>
>>>> On May 18, 2013, at 12:20 AM, Jack Moxley <jack at moxley.co.uk> wrote:
>>>>
>>>>> You have to be careful, apples support for pixel densities and screen sizes is very lack luster, it came late in the day and they only ever needed to support a small number of densities and screen sizes.
>>>>>
>>>>> Googles approach on android, or preferably the standard html5 approach using media queries is going to cause less issues in the long run. The requirement to have the the layout change because of orientation, screen size or density is prevalent in every mobile application I have ever had to write or port.
>>>>>
>>>>> As such having it done for me, does not appeal, unless I have a way of overriding it.
>>>>>
>>>>> Sent from my iPhone
>>>>>
>>>>> On 18 May 2013, at 00:01, Jim Graham <james.graham at oracle.com> wrote:
>>>>>
>>>>>> The way we deal with platform "stretching" for retina displays is to tell the platform that we are HiDPI aware and do the "pixel stretching" ourselves, but this is using only the MacOS APIs for DPI communication.
>>>>>>
>>>>>> We have an outstanding Jira issue to do the same thing for Windows 8, but we haven't investigated what that involves yet, primarily because there hadn't been a major hardware shipment of Win8 HiDPI screens at the time we implemented Retina support.  It sounds like the new Surface hardware might be opening that box on the Windows side now.
>>>>>>
>>>>>> If we'd implemented retina support entirely by suggesting a different EM to the CSS code then only code that used CSS for sizing would look good.  The solution we used for Retina scales all content appropriately including, implicitly, the CSS EM sizing...
>>>>>>
>>>>>>          ...jim
>>>>>>
>>>>>> On 5/16/13 11:51 AM, Danno Ferrin wrote:
>>>>>>> Executive summary: Is there any way in FXML to specify the sizes of the
>>>>>>> components in units other than pixels?  Either now or in the future?
>>>>>>>
>>>>>>> So I am running a Java Swing Application on a Surface tablet with multiple
>>>>>>> embedded FXPanels (mostly to deal with cross toolkit dialog modality... an
>>>>>>> issue for another thread).  In order to not get the horrible blurred view
>>>>>>> (since surface does 150% font scaling) I turn off the dpi fixing for the
>>>>>>> java app, so the swing app is pixel to pixel for the most part, with some
>>>>>>> dramatic sizing issues with the XP theming (an issue I don't expect ever to
>>>>>>> be fixed).
>>>>>>>
>>>>>>> So I have an FXML panel that is supposed to be 560x400, and I literally get
>>>>>>> 560px by 400px.  But the widgets are all sized in EM, which respects the
>>>>>>> native DPI scale.  I don't want to turn that off, because in this case I
>>>>>>> like it.  But the widgets scale up, so it sizes like a 373.333x266.666
>>>>>>> panel.  What I do want to be able to do is specify the FXML sizes in EM, so
>>>>>>> I would call it 35em x 25em.  Is there a way in FXML to specify these?  Or
>>>>>>> will I have to do multiple FXMLs mechanically scaling the sizes up.  Or is
>>>>>>> there some escape sequence or auxiliary data I can add to say "scale pixel
>>>>>>> sizes by 150%"?
>>>>>>>
>>>>