Integrating native Open GL code in JavaFX and alternatives

[Robert Krüger]

Hi,

just to avoid, this gets buried in the other long thread where it
originated from (Migrating commercial applications from Swing to JFX),
I'm posting this again as a separate thread. I hope this is OK.

We are evaluating migrating our Swing Application which includes a
video player built using native code for decoding, rendering and
effects to JavaFX. Performance requirements are very ambitious (like
being close to the playback performance of the best native player, in
our case Quicktime for most formats we care about, for high definition
video).

Currently there is no way to do the same thing in JavaFX, which the
JOGL folks have already run into themselves (see
https://jogamp.org/bugzilla/show_bug.cgi?id=607).

We would either need support for getting a native Open GL context to
draw to a JFX component (panel, canvas whatever), so we can use native
code to do all sorts of stuff that happens in our video player or we
would need support for the following operations in JFX:

- shaders for converting pixel formats (e.g. YUV422->RGB, YUV420->RGB ....)
- shaders and frame buffer objects (render to texture) for applying
layered effects to video frames
- pixel buffer objects for transferring video images to textures (and back)
- interoperability with opencl and cuda (processing of video frames
with opencl/cuda and rendering with opengl)
- reuse of opengl code across platforms (including android and iOS)

We achieve that currently by having cross-platform C code that is
connected to each platform via a rather thin layer for communication
(e.g. to be able implement callbacks for OpenGL rendering parameters
controlled by a Java-UI). My guess is that it is more likely that you
will be able to expose the native context than providing all the
listed things. It is also a maybe a philosophical question whether you
want that but IMHO you close the door on an interesting range of
applications if you don't provide anything like that in some way
because you cannot implement a serious media application of a certain
kind without it and why reinvent the wheel? IMHO Java technologies in
the multi-media area (e.g. JavaSound, JMF) have in many cases suffered
from being good ideas but not making it to a production-quality state
because doing all that would have meant reinventing the wheel, which
in complex areas like the one described here is simply not feasible so
they IMHO remained being mostly of educational value for academic
environments. Acknowledging that in some areas it does make sense to
have some native code doesn't diminish the value of Java as a
platform. Having a lot of portable code in an application is still a
lot better than having one application per platform. Quite the
contrary, to me it would make Java/JavaFX more grown-up in the desktop
area if that was supported some more.

Sorry for being a bit philosophical in the end ;-).

Thanks a lot for your help. I'm really hoping we can come to the
conclusion that migrating to JavaFX is the right way to go for us.

Robert