Exposing native surface or opengl handle

Wed Jun 25 07:43:16 UTC 2014

Hey all,

it's great to have a new thread like this. Robert exactly pointed out what 
we actually need. 
I have seen an approach to integrate JFX in JOGL and vice versa. This approach
is always been a copying of the pixel buffers between those two frameworks.
From my perspective this is not a real good approach because of obvious 
performance issues.  Yesterday, I had a better idea.

my idea:
I know it is very hard to have JFX exposing a real GLCanvas/Context. I used 
JOGL for quite some time and I know the JFX rendering pipeline a bit.
Please correct me if I am wrong. The most applications need some point to draw. 
The best pointer would be an exposed FBO from Prism that can be used by
Jogl/LWJGL. Additionally we would need a possibility to share the JFX OGL-Context with another
one, so we could reuse this FBO in a second window. Okay, this wouldn't needed
if I could share textures over scenes.

I know there is one major limitation. In windows Prism is using DirectX by 
default, so there won't be a possible interaction with Jogl. I am sure some
DirectX guys really like to have there hands on the surface-layer as well ;)

Well, to used the Jogl way above we would also need a stable OGL implementation
of Prism for Windows. Last time I tried it it was not even comparable. I am not
sure why, but OGL on Mac and Linux works pretty good.

Matthias

--
Matthias Hänel
Geschäftsführer/ Managing Director

------------------------------------------------------------------------

UltraMixer Digital Audio Solutions
Am Waldschlößchen 2
01099 Dresden

-------------------------------------------------------------------------
info at ultramixer.com          http://www.ultramixer.com

Am 23.06.2014 um 17:43 schrieb Robert Krüger <krueger at lesspain.de>:

> Thanks a lot for the valuable and very encouraging info! I will track
> that issue and use that for further communication.
> 
> Robert
> 
> On Mon, Jun 23, 2014 at 5:15 PM, Stephen F Northover
> <steve.x.northover at oracle.com> wrote:
>> I'm sorry this thread scrolled away into the bitbucket in the sky.
>> 
>> Last JavaOne, we wrote a prototype that showed native integration on OS X.
>> We parented a native sheet dialog in an FX stage and providing an OpenGL
>> node.  The code was a prototype that worked only on OS X.  The Open GL node
>> allowed drawing JOGL and LWJGL to draw on a texture that was created by FX.
>> This mean that the OpenGL node could take part in FX animations and effects.
>> 
>> I will attach the prototype code to
>> https://javafx-jira.kenai.com/browse/RT-36215.  I need to find it and make
>> sure that it still compiles and works.  This week is M5 RDP2 and today is
>> likely to be a write off for a number of reasons.
>> 
>> https://wiki.openjdk.java.net/display/OpenJFX/8u20
>> 
>> Please ping me in the JIRA if the code doesn't show up sometime this week.
>> The prototype is the basis of one possible implementation and needs some
>> work.  There are other possible implementations and this is not the final
>> word on the issue.
>> 
>> Steve
>> 
>> 
>> On 2014-06-23, 10:03 AM, Robert Krüger wrote:
>>> 
>>> Sorry, my last reply did not go to the list. That was unintended.
>>> 
>>> Oracle-Team, please someone comment on this, at least on what should
>>> be done regarding a Jira Issue (or several ones?) to track any
>>> progress on this and collect ideas & requirements.
>>> 
>>> Best regards,
>>> 
>>> Robert
>>> 
>>> On Fri, Jun 13, 2014 at 10:41 PM, Robert Krüger <krueger at lesspain.de>
>>> wrote:
>>>> 
>>>> Thanks for the hint. I think this is similar to what a colleague of
>>>> mine did a while ago as a proof of concept using other com.sun.api
>>>> that then went away. As long as we're bundling the JRE with our
>>>> product and we're desperate enough to get it working, we might do
>>>> something like this but it's of course just a last resort. Lets hope
>>>> someone from Oracle says something.
>>>> 
>>>> 
>>>> 
>>>> On Fri, Jun 13, 2014 at 8:05 PM, Scott Palmer <swpalmer at gmail.com> wrote:
>>>>> 
>>>>> That’s basically it. It isn’t perfect, but its so simple I don’t see why
>>>>> it can’t be done quickly.  We are already talking about using native code to
>>>>> render.
>>>>> 
>>>>> That said, com.sun.glass.ui.Window contains the field we want:
>>>>> 
>>>>>     // Native object handle (HWND, or NSWindow*, etc.)
>>>>>     long ptr;
>>>>> 
>>>>> You could be evil and hack it now with reflection, but it relies on
>>>>> internal implementation details.
>>>>> 
>>>>> In my application I already create a native window for video preview…
>>>>> though not as a child of the FX window.  The problem is that there isn’t a
>>>>> straight-forward, reliable, supported way to get the window handle to use
>>>>> for the parent (JavaFX) window.  There are ways to hack it, but they aren’t
>>>>> pretty.
>>>>> 
>>>>> 
>>>>> Scott
>>>>> 
>>>>> On Jun 13, 2014, at 7:55 AM, Robert Krüger <krueger at lesspain.de> wrote:
>>>>> 
>>>>>> Just for my understanding: Your approach would be to get the native
>>>>>> window handle for the hosting JFX stage, then leave an open space in
>>>>>> the layout for e.g. the player canvas that will be implemented
>>>>>> natively and then create a native child window that just reacts to
>>>>>> move and resize events of its native parent?
>>>>>> 
>>>>>> On Fri, Jun 13, 2014 at 1:48 PM, Scott Palmer <swpalmer at gmail.com>
>>>>>> wrote:
>>>>>>> 
>>>>>>> This is critical, but I don't think we need to focus on a specific
>>>>>>> technology like Direct3D or OpenGL. As a first step all we need is a
>>>>>>> mechanism to get a native reference to the Window. Just like we can with
>>>>>>> JAWT.  I'm guessing that JavaFX doesn't use heavyweight child windows so we
>>>>>>> could add a new child window that we manage with our own code and it would
>>>>>>> appear on top of the JavaFX content.
>>>>>>> 
>>>>>>> Scott
>>>>>>> 
>>>>>>>> On Jun 13, 2014, at 3:08 AM, Felix Bembrick
>>>>>>>> <felix.bembrick at gmail.com> wrote:
>>>>>>>> 
>>>>>>>> I absolutely agree that such a feature is critical for the success
>>>>>>>> and
>>>>>>>> longevity of JavaFX.  I am *really* hoping for some heavily beefed-up
>>>>>>>> 3D
>>>>>>>> support in a JFX 8.* release or JFX 9.
>>>>>>>> 
>>>>>>>> I need my graphics toolkit (currently JavaFX) to be able to handle
>>>>>>>> everything from simple UIs with basic controls to complex 3D
>>>>>>>> visualisations, just like the underlying graphics API is capable of
>>>>>>>> (i.e.
>>>>>>>> OpenGL or Direct3D).  I strongly suspect though that focusing on
>>>>>>>> OpenGL
>>>>>>>> exclusively is the only viable way to go from a cost perspective
>>>>>>>> which
>>>>>>>> would mean JavaFX supporting OpenGL on Windows.
>>>>>>>> 
>>>>>>>> 
>>>>>>>>> On 13 June 2014 16:57, Robert Krüger <krueger at lesspain.de> wrote:
>>>>>>>>> 
>>>>>>>>> Hi,
>>>>>>>>> 
>>>>>>>>> it has been discussed a number of time in the passed but let me
>>>>>>>>> quickly summarize:
>>>>>>>>> 
>>>>>>>>> A number of people have requested a feature that provides the
>>>>>>>>> ability
>>>>>>>>> to have native code draw into a surface provided by a JavaFX
>>>>>>>>> application as fast as technically possible, i.e. with no
>>>>>>>>> indirection
>>>>>>>>> or copying because use cases for this were mostly cases where
>>>>>>>>> performance was critical, e.g. HD/UHD video players, real-time
>>>>>>>>> visualization etc. where losing even e few percent would make a
>>>>>>>>> software written in JavaFX unable to compete with native products
>>>>>>>>> (e.g. in the video area nobody will use a video player that is not
>>>>>>>>> able to play the content smoothly that VLC player or Quicktime can
>>>>>>>>> on
>>>>>>>>> the same machine). Some people already have libraries of native code
>>>>>>>>> that they have built over the years and would like to reuse. I would
>>>>>>>>> even go so far to say that our product will probably never be able
>>>>>>>>> to
>>>>>>>>> move to JFX (apart from mixing it a bit with Swing, which we
>>>>>>>>> currently
>>>>>>>>> see rather aa a migration strategy and not as the end result)
>>>>>>>>> without
>>>>>>>>> this problem solved.
>>>>>>>>> 
>>>>>>>>> In the past the reactions/signals from the Oracle team in this
>>>>>>>>> respect
>>>>>>>>> were mixed but some of it indicated that this topic was discussed in
>>>>>>>>> the past and might be revisited after the release of JFX 8. Now that
>>>>>>>>> the latter has happened I would like to ask what the plans for this
>>>>>>>>> are.
>>>>>>>>> 
>>>>>>>>> Is there a Jira Issue where we can track the progress of it?
>>>>>>>>> 
>>>>>>>>> If not, does it make sense if I open one, so people (probably the
>>>>>>>>> same
>>>>>>>>> ones that have participated in these discussions in the past like
>>>>>>>>> Scott and the Ultramixer guys etc.) can collect their
>>>>>>>>> requirements/thoughts?
>>>>>>>>> 
>>>>>>>>> Even if Oracle should decide not to do something about it, it would
>>>>>>>>> still be nice to get pointers in the code base to where this would
>>>>>>>>> be
>>>>>>>>> possible, even if it is non-portable. I know that for our product it
>>>>>>>>> would be totally OK to have different ways of doing this for each
>>>>>>>>> platform and to live with the limitation to not be able to
>>>>>>>>> manipulate
>>>>>>>>> the result in the FX scene graph apart from that the position of the
>>>>>>>>> surface moving with the hosting FX node and as far as I have
>>>>>>>>> understood others, it is the same for them.
>>>>>>>>> 
>>>>>>>>> Maybe a Jira issue could also be a good place to bundle information
>>>>>>>>> about approaches interested parties are willing to pursue.
>>>>>>>>> 
>>>>>>>>> Thoughts anyone?
>>>>>>>>> 
>>>>>>>>> Robert
>>>>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> --
>>>>>> Robert Krüger
>>>>>> Managing Partner
>>>>>> Lesspain GmbH & Co. KG
>>>>>> 
>>>>>> www.lesspain-software.com
>>>> 
>>>> 
>>>> 
>>>> --
>>>> Robert Krüger
>>>> Managing Partner
>>>> Lesspain GmbH & Co. KG
>>>> 
>>>> www.lesspain-software.com
>>> 
>>> 
>>> 
>> 
> 
> 
> 
> -- 
> Robert Krüger
> Managing Partner
> Lesspain GmbH & Co. KG
> 
> www.lesspain-software.com