Integrating native Open GL code in JavaFX and alternatives

[Scott Palmer]

+1

I see two basic needs:

1: A JAWT equivalent so we can at least get a handle to the Stage window.

2: Some means of embedding our own rendering surface.  Likely via a "texture" described in the native format for the graphics pipeline used (OpenGL or Direct3D)

In our Swing product (on Windows) we grab the native window handle of a heavyweight 'placholder' via JAWT and blit a bitmap directly to it - hardware assisted with a colorspace conversion, if I recall correctly.  JavaFX offers no equivalent.

I know that JavaFX will likely never allow the concept of embedding a heavyweight native window into a Scene.  I don't think that would be the right approach for it anyway.  But we can't even make a separate preview window natively and give it a proper parent window with the current APIs.

Scott

On 2013-01-25, at 4:14 AM, Robert Krüger <krueger at lesspain.de> wrote:

> Hi,
> 
> just to avoid, this gets buried in the other long thread where it
> originated from (Migrating commercial applications from Swing to JFX),
> I'm posting this again as a separate thread. I hope this is OK.
> 
> We are evaluating migrating our Swing Application which includes a
> video player built using native code for decoding, rendering and
> effects to JavaFX. Performance requirements are very ambitious (like
> being close to the playback performance of the best native player, in
> our case Quicktime for most formats we care about, for high definition
> video).
> 
> Currently there is no way to do the same thing in JavaFX, which the
> JOGL folks have already run into themselves (see
> https://jogamp.org/bugzilla/show_bug.cgi?id=607).
> 
> We would either need support for getting a native Open GL context to
> draw to a JFX component (panel, canvas whatever), so we can use native
> code to do all sorts of stuff that happens in our video player or we
> would need support for the following operations in JFX:
> 
> - shaders for converting pixel formats (e.g. YUV422->RGB, YUV420->RGB ....)
> - shaders and frame buffer objects (render to texture) for applying
> layered effects to video frames
> - pixel buffer objects for transferring video images to textures (and back)
> - interoperability with opencl and cuda (processing of video frames
> with opencl/cuda and rendering with opengl)
> - reuse of opengl code across platforms (including android and iOS)
> 
> We achieve that currently by having cross-platform C code that is
> connected to each platform via a rather thin layer for communication
> (e.g. to be able implement callbacks for OpenGL rendering parameters
> controlled by a Java-UI). My guess is that it is more likely that you
> will be able to expose the native context than providing all the
> listed things. It is also a maybe a philosophical question whether you
> want that but IMHO you close the door on an interesting range of
> applications if you don't provide anything like that in some way
> because you cannot implement a serious media application of a certain
> kind without it and why reinvent the wheel? IMHO Java technologies in
> the multi-media area (e.g. JavaSound, JMF) have in many cases suffered
> from being good ideas but not making it to a production-quality state
> because doing all that would have meant reinventing the wheel, which
> in complex areas like the one described here is simply not feasible so
> they IMHO remained being mostly of educational value for academic
> environments. Acknowledging that in some areas it does make sense to
> have some native code doesn't diminish the value of Java as a
> platform. Having a lot of portable code in an application is still a
> lot better than having one application per platform. Quite the
> contrary, to me it would make Java/JavaFX more grown-up in the desktop
> area if that was supported some more.
> 
> Sorry for being a bit philosophical in the end ;-).
> 
> Thanks a lot for your help. I'm really hoping we can come to the
> conclusion that migrating to JavaFX is the right way to go for us.
> 
> Robert