JavaFX performance for complex visualisations

[John C. Turnbull]

Thanks for responding to this Richard.

The smoothness issue is a bit baffling.  On my desktop PC (which is
virtually a super computer and includes the world's fastest GPU), all the
animations in Ensemble are very "unsmooth", that is to say that even the
simple Translate transition demo displays a jumpy red ball moving from side
to side.  It's so jumpy that it looks like I am actually running it on a
hand-held calculator or similar!  Conversely, the same demo on my 5-year-old
laptop is "super smooth" and indeed all the Ensemble demos run extremely
impressively on this machine.  Do you have any ideas on what is causing this
and/or suggestions on how to improve things?

Anyway, I am very excited by the question you posed at the end of your reply
in relation to building a graphics intensive demo application with JavaFX.
Another thread has started dealing with the type of game or application that
would be suitable but the thing for me which is most important is that this
demo *not* be a clone of some simple existing game but rather be truly
graphics intensive and demanding.  One suggestion was to re-implement
Asteroids because "If we can't handle an old 8-bit game from the early 80's
with nice fluid animations then we can give up and go home" but to me this
would be exactly what *not* to do.  The idea that JavaFX should be capable
of fluid animations for an old style game is fine but what have we proved if
the app does indeed run smoothly using JavaFX?  Nothing really.  We *must*
focus on something which would not have run on an old 8-bit computer.
Something that would test Flash today on reasonably powered modern machines.
Something that actually *requires* a UI toolkit as powerful and advanced as
JavaFX (or is supposed to be).  Otherwise we have not achieved anything.

Whatever is chosen, I will contribute in any way I can :-)

-jct

-----Original Message-----
From: Richard Bair [mailto:richard.bair at oracle.com] 
Sent: Wednesday, 5 December 2012 07:24
To: John C. Turnbull
Cc: openjfx-dev at openjdk.java.net
Subject: Re: JavaFX performance for complex visualisations

Hi John,

Just getting through my email and have had this one flagged since Devoxx
:-). I respond below to individual points but most of these are probably not
worth you responding back on (they're just informational on my part) --
rather, down at the end is the real question I want to have a discussion on
(and will be a lot of fun to solve while we're at it).

> 1. Do any such games, animations or visualisations exist yet?

Yes. Much like Swing, JavaFX is seeing a lot of adoption in the heavy
enterprise desktop sorts of use cases -- not the kind of thing you will find
when browsing the internet. Most of these folks don't advertise the
technology they're using, and most of them are not interested in letting
their competitors know what they're doing or what technology they're using.
Which basically means, if we haven't built a demo for it, there isn't
anything you will find in your travels.

> 2. If not, how does Oracle or anyone else actually know that JavaFX is 
> capable of supporting such applications?

That is a good question. Many of the folks I've talked to building such
applications have done incredible work + proof of concepts long before I
even knew they were using JavaFX. Now, each year we do pretty graphically
intense demos for JavaOne, so we have a pretty good feel for it. We have a
bunch of benchmarks where we crank the number of nodes up to pretty high
numbers, and we know from these benchmarks (at least) that we scale very
well compared to other technologies. But I know neither the samples nor the
results are publicly available. But at least I can answer the "how does
Oracle . know" part of the question.

We have had a performance team from the get-go who were solely dedicated to
writing benchmarks, running benchmarks, and analyzing results. This has been
incredibly helpful. We have results that come not just from the weekly
integrations but even from intra-week builds on specific scrum repositories.
We have benchmarks and analysis for both desktop and embedded and take
performance very seriously.

Now, there are a lot of ways to use the platform. Some things are better
tuned than others. Performance work is driven by tuning and benchmarks. So
when we run into issues we have to (a) determine how likely that is to be an
issue to the broader development community (b) figure out what the tradeoffs
are (usually performance & quality are directly in contradiction with each
other), and ( c) schedule & implement the fix. Of course, the same few
people (Jim, mostly) are involved in many of the performance fixes and
features (like Canvas), so scheduling is an issue and feedback from
customers as to what they are actually running into (vs. what a theoretical
issue is) has priority.

> 3. Do I have the wrong understanding that JavaFX is supposed to 
> support such applications?

We expect that JavaFX will be usable for casual games, but we are not
building a gaming platform to compete with Unity and the like. If we happen
to be just as good, then that's awesome, but it isn't our primary goal.
Advanced visualizations, on the other hand, are part of our target market
(medical, pharma, enterprise desktop, etc)

> 4. Is it possible that, for whatever reason, JavaFX is simply not 
> capable of supporting such applications?

There are natural trade-offs between an immediate mode (like Canvas) API and
a retained mode (like Scene Graph) API. Now that we have both (plus Java2D +
ImageWriter and such for the really crazy cases) we should be capable of a
wide range of use cases. However, some things are not yet optimized. But
there is no technical reason why we shouldn't be quite capable.

Another area we have not yet exposed API which is problematic for some
people is in the area of custom shaders. It may be that without some kind of
custom shader support (which would have to be made to work on D3D as well as
OpenGL) certain use cases cannot be easily fulfilled.

> My feeling that JavaFX can indeed support such applications is based 
> on the fact that it is hardware accelerated and therefore it should be 
> limited mostly by the capabilities of the graphics card and also 
> because it is often talked about in this way.  However, I have 
> observed varying levels of performance that don't quite follow these 
> principles such as JavaFX performing poorly with choppy/jittery 
> animations and transitions on my most powerful machine with an NVIDIA 
> GeForce GTX 690 (the current fastest graphics card in the world) but 
> performing quite well on machines with much lower specifications.

It is highly unlikely that this is due to performance issues. As mentioned
in the last few days on another thread, there are smoothness issues that are
completely separate from performance (although both are often talked about
together). We're working on the smoothness. Sometimes this is an application
problem as opposed to a platform problem (one such example is the
DisplayShelf sample, which needs to be rewritten).

> So I guess I am curious to know what kinds of testing and evaluations 
> Oracle has undertaken to determine the performance characteristics of 
> JavaFX and exactly what kinds of applications it is actually suitable 
> for.  For example, I am yet to see any JavaFX application with even 
> the sophistication of a Flash electronic greeting card or banner ad 
> and yet I assume JavaFX will be used for such purposes eventually.

I would imagine any of our demos from the last 3-4 years of JavaOne would
demonstrate much more complicated use cases than that! But, instead of me
just complaining -- lets build one! We'll add it both to Ensemble and our
test suite. We can start with something simple (like a greeting card) and
expand it out to something wildly complex. What would you like to see? One
thing I've wanted to build for years but haven't found the time for was a
tower defense game. But what would you think would be a good example of
graphics performance that would tell you "hey, this platform is great for
visualizations"?

Richard