Optimised, high-performance, multi-threaded rendering pipeline

[Felix Bembrick]

(Thanks to Kevin for lifting my "awaiting moderation" impasse).

So, with all the recent discussions regarding the great contribution by
Laurent Bourgès of MarlinFX, it was suggested that a separate thread be
started to discuss parallelisation of the JavaFX rendering pipeline in
general.

As has been correctly pointed-out, converting or modifying the existing
rendering pipeline into a fully multi-threaded and performant beast is
indeed quite a complex task.

But, that's exactly what myself and my colleagues have been working on for
about 2 years.

The result is what we call the Hyper Rendering Pipeline (HPR).

Work on HPR started when we developed FXMark and were (bitterly)
disappointed with the performance of the JavaFX scene graph.  Many JavaFX
developers have blogged about the need to dramatically minimise the number
of nodes (especially on embedded devices) in order to achieve even
"acceptable" performance.  Often it is the case that most (if not all
rendering) is eventually done in a single Canvas node.

Now, as well already know, the JavaFX Canvas does perform very well and the
recent awesome work (DemoFX) by Chris Newland, just for example, shows what
can be done with this one node.

But, the majority of the animation plumbing in JavaFX is related to the
scene graph itself and is designed to make use of multiple nodes and node
types.  At the moment, the performance of this scene graph is the Achilles
Heel of JavaFX (or at least one of them).

Enter HPR.

I personally have worked with a number of hardware-accelerated toolkits
over the years and am astounded by just how sluggish the rendering pipeline
for JavaFX is. When I am animating just a couple of hundred nodes using
JavaFX and transitions, I am lucky to get more than about 30 FPS, but on
the same (very powerful) machine, I can use other toolkits to render
thousands of "objects" and achieve frame rates well over 1000 FPS.

So, we refactored the entire scene graph rendering pipeline with the
following goals and principles:

1. It is written using JavaFX 9 and Java 9 (but could theoretically be
back-ported to JavaFX 8 though I see no reason to).

2. We analysed how other toolkits had optimised their own rendering
pipelines (especially Qt which has made some significant advances in this
area in recent years).  We also analysed recent examples of multi-threaded
rendering using the new Vulkan API.

3. We carefully analysed and determined which parts of the pipeline should
best utilise the CPU and which parts should best utilise the GPU.

4. For those parts most suited to the CPU, we use the advanced concurrency
features of Java 8/9 to maximise parallelisation and throughput by
utilising multiple cores & threads in as an efficient manner as possible.

5. We devoted a large amount of time to optimising the "communication"
between the CPU and GPU to be far less "chatty" and this alone led to some
huge performance gains.

6. We also looked at the structure of the scene graph itself and after
studying products such as OpenSceneGraph, we refactored the JavaFX scene
graph in such a way that it lends itself to optimised rendering much more
easily.

7. This is clearly not a "small" patch.  In fact to refer to it as a
"patch" is probably rather inappropriate.

The end result is that we now have a fully-functional prototype of HPR and,
already, we are seeing very significant performance improvements.

At the minimum, scene graph rendering performance has improved by 500% and,
with judicious and sometimes "tricky" use of caching, we have seen
improvements in performance of 10x or more.

And... we are only just *starting* with the performance optimisation phase.

The potential for HPR is massive as it opens-up the possibility for the
JavaFX scene graph and the animation/transition infrastructure to be used
for a whole new class of applications including games, advanced
visualisations etc., without having to rely on imperative programming of a
single Canvas node.

I believe that HPR, along with tremendous recent developments like JPro and
the outstanding work by Gluon on mobiles and embedded devices, could
position JavaFX to be the best graphics toolkit of any kind in any language
and, be the ONLY *truly* cross-platform graphics technology available.

WORA for graphics and UIs is finally within reach!

Blessings,

Felix