How to reach retina support

[Tobias Bley]

Is there anything we could do out now with JavaFX 2.2 to reach Retina support?


Am 05.12.2012 um 01:00 schrieb Richard Bair <richard.bair at oracle.com>:

> In fact, we've implemented retina once already for the iOS prototype. I'm hoping adding the support to Mac desktop isn't terribly hard. Good question about level of support though -- so far we have been thinking only of the 2x retina case. One reason for this is that aligning to pixel boundaries for crisp controls etc is a PAIN if we don't simply double, but start doing fractional scaling. You position at 100 thinking it will be crisp but it ends up at some fractional position instead.
> 
> On Dec 4, 2012, at 3:07 PM, John Smith <John_Smith at symantec.com> wrote:
> 
>> What is desirable is likely:
>> 1. A glass implementation which takes advantages of OS API level optimizations for Retina.
>> 2. An ability to support apps spanning retina and non-retina windows.
>> 3. Image loaders which are aware of whether to use retina or non-retina images
>> 4. Ensuring that retina aware apps work when embedded in Swing and Browsers
>> 5. Identifying to the OS that the app is retina aware, so that the OS doesn't try to double up the applications pixels.
>> 6. A conditional feature which identifies to the app writer that they are running on a retina so they can take special action if they want to.
>> 7. High quality text support in retina mode with correct font sizing.
>> 8. The ability, perhaps at a node level, to indicate that a 1x or a 4x backing store should be used.
>> 9. The ability to specify co-ordinates in a device resolution independent way (e.g. measurement facilities like font ems and percentage values).
>> 
>> -----------------------------------
>> 
>> If interested, there is some further background info below:
>> 
>> There are some comments by Mike Swingler on developing Retina support for Swing.  They won't all apply to JavaFX, and I don't understand them all either ;-)  But they are very interesting to read.  See: http://prod.lists.apple.com/archives/java-dev/2012/Oct/msg00144.html
>> 
>> 1. History of retina in Swing. 
>> 
>> It seems that Apple got retina support working in Apple JDK6 by hacking the graphics pipeline pretty severely and that that work can't be directly ported to OpenJDK, so getting Retina support for Swing is an open item and I don't know if it is scheduled at any time.  If Swing for the JDK7+ does not get Retina support, then the impact that has on embedding JFXPanels in Swing will need to be assessed.
>> 
>> Regarding potential OpenJDK AWT support for Retina, Mike comments: "In order to render at a Retina resolution, the OpenGL pipeline will have to create a 4x sized backing store for the pixels, and apply a 2x affine transform to all graphics operations going from the API layer to the graphics context layer. Just that easy (I am, of course joking, the work is actually very hard and is littered with corner cases)."
>> 
>> Perhaps this is also relevant for supporting Retina in JavaFX.
>> 
>> 2. What to do if your app window spans multiple screens, one Retina and one not. 
>> 
>> Mike comments: "handling Retina *well* will also include tearing down and recreating the backing store dynamically as window is dragged across displays of varying resolution, and getting the app to redraw a the right times to make this transition smooth."
>> 
>> 3. Apple's OS allows embedding some retina scaled panels inside some panels which are not retina scaled.  There is a very good overview of the user facing issues of software running on Retina here: http://www.anandtech.com/show/6023/the-nextgen-macbook-pro-with-retina-display-review/6.  From the linked article about Retina support in Aperture:
>> 
>> "Here Apple is scaling the UI elements like the menus and widgets on the screen (backing scale factor = 2.0), but displaying the open image unscaled (backing scale factor = 1.0). As a result we can fit almost an entire 2880 x 1800 image on the screen without zooming out. Remember the backing scale factor isn't global, individual elements on the screen can be scaled independently depending on their purpose."
>> 
>> 4. There is a whole range of things which apple recommends when developing apps for Retina:  https://developer.apple.com/library/mac/#documentation/GraphicsAnimation/Conceptual/HighResolutionOSX/Explained/Explained.html#//apple_ref/doc/uid/TP40012302-CH4-SW4
>> Providing @2x graphics in addition to "standard" resolution images and adopting an @2x graphics naming convention.
>> Including high resolution icons.
>> Packaging your app with meta data which indicates to the OS that it is retina capable.
>> Loading images using methods which seamlessly lookup the appropriate retina or non-retina images.
>> Use higher point fonts for retina displays.
>> Make use of advanced apple APIs for high performance rendering of pixels and queries of pixel data.
>> 
>> Retina support for Apple devices is a probably easier than hi-dpi support in non-apple devices.  For instance, Apple really only target a limited set of screen resolutions, whereby retina seems to mean double the "standard" resolution.  There are currently a bunch of non-apple devices which have high resolution displays (e.g. 1920x1080 13 inch laptops) that are near retina, but not quite and the pixels on these devices become so small that it is hard to make stuff out and use the devices unless your eyes are really young.  What might be optimal for JavaFX is display resolution and zoom level independent support.  e.g. image lookup routines which choose from bitmapped graphics at @0.75x @1x @1.5x @2x, rather than just @1x and @2x.  Such resolution independent apps is useful for accessibility applications.  Perhaps there is a link between the Hi-DPI discussion and support for high quality display of resolution and zoom level independent applications.  
>> 
>> Hopefully good support for display resolution and zoom level independent JavaFX will not prove difficult.   JavaFX has a thin native porting layer in Glass.  Most of the JavaFX system is vector based.  JavaFX already includes scaling and zooming primitives and a pipeline which supports these at independent levels for different Scene Graph nodes.  JavaFX also already features resizable controls supported by sophisticated layout managers.
>> 
>> -----Original Message-----
>> From: openjfx-dev-bounces at openjdk.java.net [mailto:openjfx-dev-bounces at openjdk.java.net] On Behalf Of Randahl Fink Isaksen
>> Sent: Tuesday, December 04, 2012 1:11 PM
>> To: Tobi
>> Cc: Mailing List openjfx-dev at openjdk.java.net
>> Subject: Re: How to reach retina support
>> 
>> Hi Tobi
>> 
>> I am not sure what you mean by "Scale all javaFX rendering by factor 2.0". Since JavaFX is primarily vector based, I cannot see why you would do that.
>> 
>> As long as JavaFX prevents Mac OS from switching a JavaFX app to the magnifying mode, I can ensure that my app looks great, as long as I bring fine enough images along with it. ImageView is already able to scale my images, so why do you need more than bullet 1?
>> 
>> Randahl
>> 
>> 
>> 
>> On Dec 4, 2012, at 21:50 , Tobi <tobi at ultramixer.com> wrote:
>> 
>>> Hi,
>>> 
>>> I would like to start the discussion about the technical requirements for retina support in javaFX. Java6 and flash supports already retina Displays (aka HiDPI)
>>> 
>>> There seams to be two Important Points:
>>> 
>>> 1. prevent MacOSX from switching The JavaFX app to the magnifying mode so that not all rendering is scaled by 2.0
>>> 
>>> 2. Scale all javaFX rendering by factor 2.0
>>> 
>>> What do we have to do besides these points?
>>> 
>>> Best regard,
>>> Tobi
>>