Integrating native Open GL code in JavaFX and alternatives
Artem Ananiev
artem.ananiev at oracle.com
Mon Jan 28 08:17:09 PST 2013
On 1/28/2013 7:48 PM, Sven Reimers wrote:
> The link
>
> https://jbs.oracle.com/bugs/browse/JDK-7154778
>
> does not work for "outsiders" - I think you are hinting at
>
> http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=7154778
Yes, my fault. Thanks for the right link.
Artem
> -Sven
>
>
> On Mon, Jan 28, 2013 at 4:16 PM, Artem Ananiev <artem.ananiev at oracle.com
> <mailto:artem.ananiev at oracle.com>> wrote:
>
>
> On 1/26/2013 12:04 AM, Scott Palmer wrote:
>
>
> On 2013-01-25, at 2:16 PM, Artem Ananiev
> <artem.ananiev at oracle.com <mailto:artem.ananiev at oracle.com>>
> wrote:
>
> A few comments from my side:
>
> 1. If a native Window/Scene handle is provided, it will be
> enough
> to embed whatever you need. It will even be possible to
> embed D3D
> content into JavaFX running with OpenGL pipeline :)
>
>
> There will be issues in JavaFX that we can't work around as
> easily as
> we code in Swing. For example clipping. How can my video preview
> overlay participate in a Scene such that I can put it inside a
> ScrollPane or SplitPane? With Swing I could manage this by
> placing a
> heavyweight component in the UI. Rectangular clipping was easy to
> deal with by wrapping in another heavyweight component - the OS
> windowing system took care of it. Later Swing got smarter about
> mixing heavy/light components. JavaFX doesn't have something
> comparable.
>
>
> To use heavyweight component in a scroll/split pane, we need
> heavyweight scroll/split pane :) In other words, such scenarios
> don't work well in Swing too.
>
> Do you mean limited heavyweight/lightweight component mixing,
> introduced to 6u10 and the forward-ported to 7? Note the word
> "limited" here: the mixing is really limited, it doesn't cover
> non-rectangular shapes, transparent components, and many more. It
> actually works fine for simplest scenarios only. The code we put to
> AWT to support the limited mixing is not something I would like to
> see in JavaFX.
>
>
> The question is what is the native handle? On Windows, HWND
> is the
> answer. On Mac: NSView or CALayer?
>
>
> CALayer seems to be the way things are going. NSView was the Java 6
> native layer, now in Java 7 it is CALayer. But this is native code…
> let there be a way to get either from a platform specific structure.
>
>
> Believe it or not, we will support both CALayer-based and
> NSView-based embedded windows in JDK8 (and probably next JDK7
> updates). Check the following JDK bug for details:
>
> https://jbs.oracle.com/bugs/__browse/JDK-7154778
> <https://jbs.oracle.com/bugs/browse/JDK-7154778>
>
> Classes are not public (java.*, javax.*) APIs, though, but are
> supposed to be backwards compatible. If we implement this
> functionality in JavaFX, we'll likely support both CALayers and NSViews.
>
>
> In the latter case we'll also need mechanism to flush embedded
> layer content into the parent (window) layer, and the thing will
> get complicated.
>
> 2. Another challenge with heavyweight embedding is
> integration into
> FX scene. Focus traversal, transforms, effects, modality, etc.
>
>
> Can be depending on the use case. For a video preview panel, I
> wouldn't' need most of that. #3 would be best.
>
> 3. Alternatively, we can try to implement semi-heavyweight
> embedding. What we need is to make Prism be able to render
> externally managed textures (GL, D3D). Input events will be
> handled
> as for any other FX nodes, transforms/effects will also be of no
> problem. What's questionable here is a) performance b)
> sharing GL
> context or D3D device between FX and application code. I'm not a
> graphics expert, but suspect there may be problems in this area.
>
>
> I agree. But I think it is worth getting a graphics expert (I'm not
> a OpenGL/D3D person either) to think about it over the weekend :-)
>
> Cheers,
>
> Scott
>
>
> Thanks,
>
> Artem
>
> Thanks,
>
> Artem
>
> On 1/25/2013 6:28 PM, Scott Palmer wrote:
>
> +1
>
> I see two basic needs:
>
> 1: A JAWT equivalent so we can at least get a handle to
> the Stage
> window.
>
> 2: Some means of embedding our own rendering surface.
> Likely via a
> "texture" described in the native format for the
> graphics pipeline
> used (OpenGL or Direct3D)
>
> In our Swing product (on Windows) we grab the native
> window handle of
> a heavyweight 'placholder' via JAWT and blit a bitmap
> directly to it
> - hardware assisted with a colorspace conversion, if I
> recall
> correctly. JavaFX offers no equivalent.
>
> I know that JavaFX will likely never allow the concept
> of embedding a
> heavyweight native window into a Scene. I don't think
> that would be
> the right approach for it anyway. But we can't even
> make a separate
> preview window natively and give it a proper parent
> window with the
> current APIs.
>
> Scott
>
> On 2013-01-25, at 4:14 AM, Robert Krüger
> <krueger at lesspain.de <mailto:krueger at lesspain.de>>
> wrote:
>
> Hi,
>
> just to avoid, this gets buried in the other long
> thread where it
> originated from (Migrating commercial applications
> from Swing to JFX),
> I'm posting this again as a separate thread. I hope
> this is OK.
>
> We are evaluating migrating our Swing Application
> which includes a
> video player built using native code for decoding,
> rendering and
> effects to JavaFX. Performance requirements are very
> ambitious (like
> being close to the playback performance of the best
> native player, in
> our case Quicktime for most formats we care about,
> for high definition
> video).
>
> Currently there is no way to do the same thing in
> JavaFX, which the
> JOGL folks have already run into themselves (see
> https://jogamp.org/bugzilla/__show_bug.cgi?id=607
> <https://jogamp.org/bugzilla/show_bug.cgi?id=607>).
>
> We would either need support for getting a native
> Open GL context to
> draw to a JFX component (panel, canvas whatever), so
> we can use native
> code to do all sorts of stuff that happens in our
> video player or we
> would need support for the following operations in JFX:
>
> - shaders for converting pixel formats (e.g.
> YUV422->RGB, YUV420->RGB ....)
> - shaders and frame buffer objects (render to
> texture) for applying
> layered effects to video frames
> - pixel buffer objects for transferring video images
> to textures (and back)
> - interoperability with opencl and cuda (processing
> of video frames
> with opencl/cuda and rendering with opengl)
> - reuse of opengl code across platforms (including
> android and iOS)
>
> We achieve that currently by having cross-platform C
> code that is
> connected to each platform via a rather thin layer
> for communication
> (e.g. to be able implement callbacks for OpenGL
> rendering parameters
> controlled by a Java-UI). My guess is that it is
> more likely that you
> will be able to expose the native context than
> providing all the
> listed things. It is also a maybe a philosophical
> question whether you
> want that but IMHO you close the door on an
> interesting range of
> applications if you don't provide anything like that
> in some way
> because you cannot implement a serious media
> application of a certain
> kind without it and why reinvent the wheel? IMHO
> Java technologies in
> the multi-media area (e.g. JavaSound, JMF) have in
> many cases suffered
> from being good ideas but not making it to a
> production-quality state
> because doing all that would have meant reinventing
> the wheel, which
> in complex areas like the one described here is
> simply not feasible so
> they IMHO remained being mostly of educational value
> for academic
> environments. Acknowledging that in some areas it
> does make sense to
> have some native code doesn't diminish the value of
> Java as a
> platform. Having a lot of portable code in an
> application is still a
> lot better than having one application per platform.
> Quite the
> contrary, to me it would make Java/JavaFX more
> grown-up in the desktop
> area if that was supported some more.
>
> Sorry for being a bit philosophical in the end ;-).
>
> Thanks a lot for your help. I'm really hoping we can
> come to the
> conclusion that migrating to JavaFX is the right way
> to go for us.
>
> Robert
>
>
>
>
>
>
> --
> Sven Reimers
>
> * Senior Expert Software Architect
> * NetBeans Dream Team Member: http://dreamteam.netbeans.org
> * NetBeans Governance Board Member:
> http://netbeans.org/about/os/governance.html
> * Community Leader NetBeans: http://community.java.net/netbeans
> Desktop Java:
> http://community.java.net/javadesktop
> * Duke's Choice Award Winner 2009
> * Blog: http://nbguru.blogspot.com
>
> * XING: https://www.xing.com/profile/Sven_Reimers8
> * LinkedIn: http://www.linkedin.com/in/svenreimers
>
> Join the NetBeans Groups:
> * XING: http://www.xing.com/group-20148.82db20
> * NUGM: http://haug-server.dyndns.org/display/NUGM/Home
> * LinkedIn: http://www.linkedin.com/groups?gid=1860468
> http://www.linkedin.com/groups?gid=107402
> http://www.linkedin.com/groups?gid=1684717
> * Oracle: https://mix.oracle.com/groups/18497
More information about the openjfx-dev
mailing list