Layering JavaFX onto an external rendering context

[Michael Paus]

Why don't you just give it a try?

I'd start with the native PixelBuffer option first because it is 
relatively simple and the performance is sufficient for most use cases. 
There is, of course, an overhead caused by the GPU -> CPU -> GPU memory 
transfer but it is not so bad if done right. I've been able to interface 
a JavaFX application with the external VLC video player that way. I 
could watch 4K videos without a problem and without stressing the CPU 
too much. I do not even have a dedicated GPU in my Mac Mini - just the 
internal Intel graphic. The nice thing about this approach is that the 
maximum overhead is constant. It depends only on your hardware and the 
size of your rendering area but not on the complexity of your rendering.

Another advantage is that this approach is fully integrated into JavaFX. 
This means you can do anything with the rendering you can do with a 
JavaFX ImageView. E.g., drawing something on top of the rendering via 
JavaFX costs you nothing. Even double-buffering is possible if you know 
how to do that.

And if this is not enough for you, there still is DriftFX which avoids 
this GPU -> CPU -> GPU memory transfer but is a bit more ambitious to 
handle. So I would keep that in mind as an option if necessary.

Michael

Am 07.03.21 um 13:22 schrieb Neil C Smith:
> Hi,
>
> A few comments from my perspective here, mainly coming from
> frustration at the number of times I've had to tell clients or users
> that JavaFX isn't a viable option for them ..
>
> On Sat, 6 Mar 2021 at 12:22, Mark Raynsford <org.openjdk at io7m.com> wrote:
>> there is a
>>       combinatorial explosion of possibilities with regards to which
>>       rendering API the _user_ might be using, and which rendering API
>>       JavaFX might be using (if any!).
> I agree that's a difficult one, but also surely not an insurmountable
> one?!  There are precedents around Java APIs that provide querying and
> access to optional support on a platform by platform basis.  A library
> integrating JavaFX with some other platform / renderer specific API is
> always going to have to handle the situation by either failing
> gracefully or falling back to a less performant alternative.
>
>> Traditional stateful APIs like
>>       OpenGL make it far too easy for external code to screw up the
>>       rendering context anyway, so exposing JavaFX's own context would
>>       be a bad idea.
> I remain of the view that this shouldn't really be that concerning?!
> It's a bug in the library if it does so.  The current PixelBuffer API
> already allows enough access for a library to *really* screw up the
> rendering context, by bringing the whole thing crashing down.  But,
> concurrency issues aside, adding that access has been a real boon.
> With access comes responsibility.
>
>> JavaFX _does_ have a software renderer. What if we could have JavaFX
>> work with entirely software rendering into an offscreen BufferedImage?
> Why BufferedImage?  What about a reuse / extension / parallel of
> PixelBuffer?  That could allow a range of implementation and the
> potential for using renderers that can output without a visible window
> to do this while at least keeping all pixel data off heap?
>
> Best wishes,
>
> Neil
>
>