JavaFX graphics performance and suitability for advanced animations

Tue May 28 19:31:28 PDT 2013

No need to reply to this email (or this thread could rapidly get very long ;-)

The email is just a summary of some thoughts I had on performance.
Perhaps more wiki material than anything else.

-----------

> The biggest thing when doing performance work is identifying the benchmarks. Once we know what we're measuring, it *will* get faster.

This is one of the most difficult things I found about trying to code performance sensitive stuff for JavaFX.  It's the not knowing part of it.  

JavaFX features high level features such as effects, css and animation and it's hard to know where performance bottlenecks will be without trial and error.  

For instance, you can draw hundreds of thousands of lines really quick, but if you try to draw a path with more than 10000 elements, things start rendering slow - so you can speed your rendering up by using lines rather than paths, but unless you know that or try it, you might get stuck. 

Or another instance is selecting the wrong pixel format for a WritableImage can kill the performance of trying to animate a video by twiddling the image's pixels because the frame-rate drops an order of magnitude without the right pixel format.

I also find it hard to know the impact of something like effects or CSS on the GPU or battery life, because it is pretty difficult for me to objectively measure those kind of things.  

Should I set up lots of parallel animations, or am I better off having a central pulse style system which does everything on a tick?  Without knowing how animations are implemented, e.g. if they use their own thread or if they incur a bunch of other overheads, it's hard to make an objective decision about that.  

If your application uses WebView with intensive JavaScript, then you are better off using a 32 bit jvm on windows rather than a 64 bit one, because one will use a JIT JavaScript compiler and the other won't.

The type of effects used make a large performance difference.  For example, boxblur is a whole lot quicker than a gaussianblur - as it's supposed to be I guess from reading the wiki pages on what the algorithms incur.

Buffering of canvas commands, and a subsequent pause while initially rendering a canvas with lots of commands can introduce pauses to the application that most api users aren't going to know about until it starts occurring.

I am sure there are many more similar performance impacting tradeoffs which could have been listed here that I don't really know about or understand (such as whether I should rely on dirty region heuristics or node cache hints to optimize rendering performance or should I just snapshot the nodes myself and use the snapshot rather than relying on a platform optimization).

I think a lot of the above is just the nature of JavaFX and goes with the territory - it's a relatively high level library which abstracts you from some of the low level implementation details so that the true cost of some of your api usage choices are hidden from you.  

> Did we try turning cache to true and cache hint to SPEED?

A simple game I wrote used some basic animation of about 50 nodes with effects applied to them (translucent, blur, sectioned viewports into a large Image) and without caching ran painfully slowly.  Setting caching to true and using cache hints (just on the animated nodes) made a massive performance difference (on a macbook air), it was the difference between a game which was playable and a game which was not (i.e. framerate did not drop to single digits and the air's fan didn't spin up).  

Turning on caching for select nodes was the easiest and single biggest performance improvement I got for the game.

> We had an embedded hack-fest a couple weeks ago in which performance on desktop went from 320-800+fps on table view scrolling, which in large measure came down to reducing the number of state switches on the graphics card (and the resulting decrease in the number of OpenGL calls).

I realize the above statement is to do with internal optimizations, but should I, as a user of the JavaFX API ever have to worry that the way in which I write my user code may result in something like an increased number of state switches on the graphics card?  

Or should I just be able to ignore that kind of stuff as an implementation detail, kind of like when I drive my car, I press the accelerator and it goes and I don't really need to worry much about how that happened? 

The difficulty for me here is that I don't know what a state switch on a graphics card is and have no way of knowing whether a particular code path is triggering a lot of switches.

It seems like an aim for JavaFX is to not require the developer be a low-level mechanic to make things work.

> We should never require you to have to follow a 15 point performance plan just to get acceptable performance, or to avoid choppiness

Nevertheless, an official performance guide would be useful (like Android's: http://developer.android.com/training/best-performance.html).

I put together a short (and necessarily incomplete) guide as part of an answer to a stackoverflow question on obtaining good performance with JavaFX: 
http://stackoverflow.com/questions/14467719/what-could-be-the-best-way-to-view-millions-of-images-with-java

2013/5/29 Richard Bair <richard.bair at oracle.com>

> Hi John,
>
> > 1.       Can someone from Oracle please outline the full range of
> > applications for which JavaFX is or will be suitable for?
>
> That's a pretty broad question. Lots of stuff? At a minimum everything 
> Swing and SWT were used for, as well as mobile and embedded UIs, rich 
> media, graphics, etc. I don't expect somebody to write Halo 5 with it.
>
> > 2.       Is there something inherent in the JavaFX architecture (such as
> > CPU/GPU interaction, the performance of the JVM or the Java language
> itself)
> > that limits its suitability and thus effectiveness in advanced 
> > animations/visualisations?
>
> Absolutely not, in fact, quite the opposite. The basic architecture 
> (threading model, GPU usage model, etc) is designed for high 
> concurrency and throughput. Some of the features in Controls though 
> (like CSS lookup, color derivation, etc) put a tax on performance. 
> When it wasn't exposed in the API, every design decision is made with 
> performance as a for most thought. When it comes to API usability is 
> considered primarily but performance is also always considered (along 
> with security). And for every feature that adds weight, there is a way 
> to avoid it when absolutely necessary.
>
> > 3.       Is this choppiness and lack of smoothness I have experienced
> > typical of JavaFX performance or is it some issue with my 
> > environment/drivers etc.?
>
> Hard to say. I saw a couple weeks ago a machine where scrolling the 
> table view was 14fps whereas it was 320fps for me. The difference was 
> the other system was falling back to the software pipeline. To 
> determine what you're seeing, we need to know whether what you're 
> running is producing consistently slow results or erratic results. 
> Also, we need to know whether you are render bound or compute bound. What's taking so long to complete?
>
> We have seen situations where we are preparing a frame but never 
> rendering it, which might also be contributing to the choppiness.
>
> > 4.       Is JavaFX more targeted at form-based UIs rather than high
> > performance graphics?
>
> No.
>
> > 5.       Do you have any other comments on JavaFX and its suitability for
> > advanced animations and visualizations?
>
> The biggest issue at present architecturally is that we don't expose 
> any way for you to *really* draw without the scene graph. The Canvas 
> gets you partway there, but ultimately that guy is still just 
> buffering up drawing commands and issuing them later against a 
> texture, rather than allowing you to go directly down to OpenGL. So 
> that's a feature that is missing that is going to impact some people.
>
> Instead, you have to do everything with the scene graph which in more 
> advanced scenarios means a huge scene graph and tons of memory.
>
> We're still making a lot of progress on the raw performance side. We 
> had an embedded hack-fest a couple weeks ago in which performance on 
> desktop went from 320-800+fps on table view scrolling, which in large 
> measure came down to reducing the number of state switches on the 
> graphics card (and the resulting decrease in the number of OpenGL calls).
>
> However choppiness is often the culprit in perceived performance 
> rather than actual fps.
>
> One thing you can try is to run your application with 
> -Djavafx.pulseLogger=true and analyze the output. This records the 
> amount of time spent in various phases of the pulse, the number of 
> dirty nodes processed per frame, etc. One thing I saw a couple weeks 
> back, for instance, was that if more than 15 nodes are dirty (or is it 
> 12?) then we punt on determining the dirty region and accumulate the 
> entire parent. This is a heuristic used to trade off figuring out how 
> big the dirty area is against just drawing it -- sometimes it is 
> cheaper to do the former, sometimes the latter.
>
> Also each individual dirty region probably comes with some overhead in 
> terms of setup for each render pass (each unique dirty area ends up 
> getting its own render pass), and this fixed cost has not been 
> analyzed and perhaps needs to be factored in to our determination of 
> the number of dirty regions we support, or the heuristic in any case.
>
> Are your slow examples reproducible? If so we need the test case. Is 
> there an issue filed? We can't fix things we can't reproduce. We spend 
> a
> *considerable* amount of time and energy on performance and for the 
> things we're measuring we're doing well. As the saying goes "what's 
> measured, improves". After the switch to gradle and the new project 
> layout, one thing I'm going to look at is using JMH[2] in OpenJFX so 
> we can write micro benchmarks and have them easy for everybody to run 
> and contribute to. Our current set of micro benchmarks are based on 
> the predecessor of JMH which was the JRockit benchmark suite and was 
> proprietary (hence we cannot just open source our existing benchmarks without doing some rewrite).
>
> [1] Attributed to Peter Drucker
> http://blog.johnrchildress.com/2012/06/05/key-business-metrics-and-mil
> estones/
> [2]
> http://hg.openjdk.java.net/code-tools/jmh/file/tip/jmh-samples/src/mai
> n/java/org/openjdk/jmh/samples/