JavaFX graphics performance and suitability for advanced animations

[Hervé Girod]

Hmm, there's no point to talk politics about platforms here I think. 

And I don't think that we will go anywhere if we only say that Oracle does not do enough, or that the performance is not enough, without being more specific.

Again we use swing (yes swing...) for complex graphic cockpit soft real time simulations with a lot of animated elements (including purely graphic widgets), and we found the performance really sufficient for our use. But we heard that swing was not adapted for this type of use cases...

We think that we will have a better performance with JavaFX, even with 2.2. 

I'm sure that a lot of things can and must be improved for JavaFX apps performance, but saying that in its current state it's unusable is a bit over stretched.

Concerning other frameworks, I know a bit about QML, but its performance is IMO far below JavaFX in its current state, but regardless it seems to be used a lot, even with its shortcomings.

Herve

Sent from my iPhone

On 30 mai 2013, at 14:56, "John C. Turnbull" <ozemale at ozemail.com.au> wrote:

> Hi Richard,
> 
> Thanks for the comprehensive reply.  Responses inline.
> 
>>> 1.       Can someone from Oracle please outline the full range of
>>> applications for which JavaFX is or will be suitable for?
>> 
>> That's a pretty broad question. Lots of stuff? At a minimum everything
> Swing and SWT were used for, as well as mobile and > embedded UIs, rich
> media, graphics, etc. I don't expect somebody to write Halo 5 with it.
> 
> OK, I get that JavaFX can be used to "replace" Swing in many circumstances
> but to say "JavaFX is suitable for mobile" is a bit of a stretch considering
> Oracle's attitude to investing in developing an official release of JavaFX
> for mobile platforms.  Recent work with RoboVM has indeed shown that there
> is potential in this area but if Oracle management (and I stress
> *management*, not the JavaFX development team) still doesn't get the
> significance of tablets and mobiles and having JavaFX support on those
> devices then I don't really see JavaFX as being suitable for serious
> application development on those platforms.
> 
> Rich media?  Yes, in principle but the range of supported media formats is
> very limited.  As is the range of platforms supported by JavaFX as a whole.
> 
> Graphics?  Yes, to a point.  But my post was really about graphics and the
> issues related to performance.  Again, unless those issues are resolved then
> it's not appropriate to state that JavaFX is suitable for "graphics".  Being
> able to render simple vector graphics is nowhere near enough to encourage
> developers to adopt JavaFX for "graphics" applications.
> 
> Then you mention Halo 5.  I have to say the subtext here troubles me
> greatly.  If I read you correctly then you are saying that JavaFX is not
> really suitable for games (at least anything beyond the demands of something
> like Solitaire).  As someone else pointed out, what is point of developing
> 3D support in JavaFX if it is not really suitable for games?  To say it is
> not suitable for games implies that it is not really suitable for *any*
> application that requires performant animations and visualisations.  What
> use then is the 3D API?
> 
>>> 2.       Is there something inherent in the JavaFX architecture (such as
>>> CPU/GPU interaction, the performance of the JVM or the Java language 
>>> itself) that limits its suitability and thus effectiveness in advanced 
>>> animations/visualisations?
>> 
>> Absolutely not, in fact, quite the opposite. The basic architecture
> (threading model, GPU usage model, etc) is designed for high
>> concurrency and throughput. Some of the features in Controls though (like
> CSS lookup, color derivation, etc) put a tax on
>> performance. When it wasn't exposed in the API, every design decision is
> made with performance as a for most thought.
>> When it comes to API usability is considered primarily but performance is
> also always considered (along with security).
>> And for every feature that adds weight, there is a way to avoid it when
> absolutely necessary.
> 
> What you say here about the focus of the development work is encouraging but
> (obviously) it needs to translate to actual results in the field.
> 
>>> 3.       Is this choppiness and lack of smoothness I have experienced
>>> typical of JavaFX performance or is it some issue with my 
>>> environment/drivers etc.?
>> 
>> Hard to say. I saw a couple weeks ago a machine where scrolling the table
> view was 14fps whereas it was 320fps for me.
>> The difference was the other system was falling back to the software
> pipeline. To determine what you're seeing,
>> we need to know whether what you're running is producing consistently slow
> results or erratic results.
>> Also, we need to know whether you are render bound or compute bound.
> What's taking so long to complete?
>> 
>> We have seen situations where we are preparing a frame but never rendering
> it, which might also be contributing to the choppiness.
> 
> The symptoms I have observed are certainly not consistent in that the degree
> of choppiness varies considerably from "noticeable but only mildly
> distracting" to "this app is broken".
> 
> I have tried JavaFX on only 3 machines which were all Windows 7 machines but
> had very different specs and GPUs.  On all those machines I observed
> choppiness in all of the animations/transitions demos in Ensemble and in
> every application or sample code I have ever downloaded and run for myself.
> I am yet to see any JavaFX application which uses animations that has
> performed in what I would describe as an acceptable manner.
> 
> What I find most interesting is that there is this significant variance in
> the degree of choppiness.  On the main machine I work on (which has a very
> advanced and modern GPU), the degree of choppiness varies markedly very
> often.  One minute when I run an application I observe the choppiness in the
> lower range and then, a minute later and with apparently no change in the
> either the environment, configuration or in the apps running at the time,
> the choppiness in same JavaFX application can degrade so badly that it is
> severely broken.  Importantly, this can happen even if the application in
> question has not been restarted during that time and is still running.  At
> the same time other applications (typically those written in C++ and OpenGL
> or Direct3D) maintain their performance characteristics and continue to
> function well so it is clearly not a fault with the graphics hardware or
> drivers.
> 
> I also want it noted that I have never observed the kind of choppiness I
> frequently observe in JavaFX applications in Flash, Silverlight or in any
> games or visualisations I have run on the same machine.
> 
>>> 4.       Is JavaFX more targeted at form-based UIs rather than high
>>> performance graphics?
>> 
>> No.
> 
> Again, good to know but where are all the high-performance JavaFX apps?  So
> far I have seen nothing but form-based apps.  Wouldn't it be prudent for
> Oracle to include a "showcase" app or game that shows us the full
> capabilities of JavaFX with high-performance graphics?
> 
> I am sure I am not alone in feeling that the animation examples in Ensemble
> do very little to promote the graphics capabilities of JavaFX.
> 
>>> 5.       Do you have any other comments on JavaFX and its suitability for
>>> advanced animations and visualizations?
>> 
>> The biggest issue at present architecturally is that we don't expose any
> way for you to *really* draw without the scene graph.
>> The Canvas gets you partway there, but ultimately that guy is still just
> buffering up drawing commands and issuing them later
>> against a texture, rather than allowing you to go directly down to OpenGL.
> So that's a feature that is missing that is going to impact some people.
> 
> I am so glad you referred to this.  For me, this is a *HUGE* limitation of
> JavaFX i.e. not being able to access the graphics API directly.  As you
> point out, there is only so much you can do by fiddling with a texture.
> 
> For me, the real power in something like OpenGL is shaders.  I don't have
> much experience with Direct3D but I am sure the same applies.  Without being
> able to use custom shaders in JavaFX there will always be large impediments
> to the development of animations or visualisations of any reasonable
> complexity.
> 
> Having said that, I am well aware of the inherent complexities in support
> custom shaders in something like JavaFX especially when there are two
> disparate underlying graphics APIs that must be supported.  Even so, I do
> see support for shaders as the single most important feature that must be
> added to JavaFX for it to gain any kind of market traction or adoption for
> anything other than form-based UIs.  And I do not believe that it is
> impossible or too complex/expensive to get working and there are
> technologies such as OSL which may help.
> 
> What I do know for sure is that there are many applications I plan to
> develop that simply cannot be implemented using JavaFX until custom shader
> support becomes a reality.  This may force me to look at other products,
> other languages and other platforms entirely which for someone who has
> devoted years to mastering the Java language is daunting.
> 
>> 
>> Instead, you have to do everything with the scene graph which in more
> advanced scenarios means a huge scene graph and tons of memory.
>> 
>> We're still making a lot of progress on the raw performance side. We had
> an embedded hack-fest a couple weeks ago in which
>> performance on desktop went from 320-800+fps on table view scrolling,
> which in large measure came down to reducing the number of state
>> switches on the graphics card (and the resulting decrease in the number of
> OpenGL calls).
> 
> I am very pleased that you are doing so much work in the background to
> improve the performance of JavaFX.
> 
>> However choppiness is often the culprit in perceived performance rather
> than actual fps.
> 
> Hmm, not sure where you are going with this.  I see that there have been
> other discussions about FPS in this thread and I tend to agree with much of
> the sentiment expressed there.
> 
> For me, FPS is just a number that often doesn't mean very much (especially
> to the end user).  Really, if your app is regularly achieving 25-30FPS then
> anything above that is barely noticeable (if at all).  Therefore to claim
> that you app is actually achieving 60FPS or more does not translate into any
> perceivable benefit to the end user.
> 
> Also, for me at least, FPS is far less important than reduction or
> elimination of choppiness.  Choppiness is *very* noticeable and *very*
> annoying.  If the frame rate degrades then it is also noticeable but (in my
> opinion) far less aggravating to the user.  Choppiness reflects poorly on
> the app in question and gives it a kind of amateurish impression.
> 
>> One thing you can try is to run your application with
> -Djavafx.pulseLogger=true and analyze the output. This records the amount of
>> time spent in various phases of the pulse, the number of dirty nodes
> processed per frame, etc. One thing I saw a couple weeks back,
>> for instance, was that if more than 15 nodes are dirty (or is it 12?) then
> we punt on determining the dirty region and accumulate the
>> entire parent. This is a heuristic used to trade off figuring out how big
> the dirty area is against just drawing it -- 
>> sometimes it is cheaper to do the former, sometimes the latter.
>> 
>> Also each individual dirty region probably comes with some overhead in
> terms of setup for each render pass (each unique dirty area 
>> ends up getting its own render pass), and this fixed cost has not been
> analyzed and perhaps needs to be factored in to our determination of the
> number of dirty regions we support, or the heuristic in any case.
> 
> When I get a bit more time I will do a more thorough analysis of the kind of
> behaviours I am reporting including use of the parameters you describe.  One
> thing I can say for certain is that it is definitely not a case of the SW
> pipeline kicking in.
> 
>> Are your slow examples reproducible? If so we need the test case. Is there
> an issue filed? We can't fix things we can't reproduce.
>> We spend a *considerable* amount of time and energy on performance and for
> the things we're measuring we're doing well.
>> As the saying goes "what's measured, improves". After the switch to gradle
> and the new project layout, one thing I'm going to
>> look at is using JMH[2] in OpenJFX so we can write micro benchmarks and
> have them easy for everybody to run and 
>> contribute to. Our current set of micro benchmarks are based on the
> predecessor of JMH which was the
>> JRockit benchmark suite and was proprietary (hence we cannot just open
> source our existing benchmarks without doing some rewrite).
> 
> All good points.
> 
> However, I am not sure that having me preparing "reproducible" test cases
> will actually help.  In my experience, the Ensemble app already serves this
> purpose.  The choppiness I describe is *always* prevalent when I run the
> animations and transitions in Ensemble (including Ensemble 8).  The only
> variation is in the degree of that choppiness.
> 
> Thanks,
> 
> -jct
>