[External] : Re: JavaFX Direct3D 12 rendering pipeline for Windows

[Nir Lisker]

Hi Lukasz,

Yes, FFM is quite a bit to take in. I haven't mastered the newest version
myself yet.

In many cases I mostly followed D3D9 backend's design, so the similarities
> can be there. =) But there definitely is room for improvement which I also
> see and would like to tackle before the backend reaches a "mature enough"
> state. When working on 3D part of JFX I also saw that there is some data duplication
> happening, especially with Light/Mesh/MeshView code. I'd like to trim
> this as well, but it was not a priority yet - I preferred to have a
> working code now and all sorts of cleanups or reorganizations like that can
> come later.
>
> Regarding FFM - that is an interesting idea that we could explore at some
> point, but I'm not sure if we could (or even should) move all the
> implementation details to Java and limit native calls to just D3D12.
> There are a few problems with that:
>

Because FFM saves a lot of handwriting of fragile JNI code, the sooner it's
adopted, the less headache you (or collaborators) will have. So while it's
true that there's a minimal working level the code should be brought to,
adopting FFM will mean rewriting a lot of JNI code. This is why I suggest
going with FFM as soon as that minimal level is reached. It has advantages
in type safety and compiler checks that JNI does not have, including the
ability to use objects-structs mapping instead of breaking them down to
JNI-supported primitives (long -> jlong etc.).


>   - There is quite a bit of logic which integrates better on C/C++ side
> with D3D12, mostly due to 12's low-level-ness. Rewriting those in Java
> feels like quite a bit of a task in itself, and might create more problems
> than it solves.
>

I'm not sure what you mean by this. If you mean the current sandbox code
that uses JNI will need to be rewritten, then yes, that will be a rewrite,
which is why I advocate to take advantage of FFM sooner rather than later.
If you're mostly copying from D3D9 then it does save some effort on the JNI
side. D3D12 (or any DLL) doesn't know what's calling it, whether it's C++
or Java, so there shouldn't be a difference in "integration". At least not
when I called native libraries, but perhaps I'm missing something. I can
also give a concrete example on some code snippet in our D3D9 or 12 if it
helps clarify my intention.


>   - Raising the abstraction from 12’s low-level approach instead of just
> making a “passthrough” is also something that’s quite frequently done,
> mostly to make the API a bit more useable and suitable for project’s
> specific needs (see ex. NativeBuffer abstractifying 12’s heap types into
> simply data being CPU visible, or code managing mid-frame data like
> CommandLists, shader constants, 2D vertices etc.). This could also help
> later down the line when adding potential new features, simplifying the
> calls made from Java side.
>

I'm not sure what this means either :p The abstraction that exists or will
exist in C/C++ that surrounds the D3D API can exist instead in Java. It's
the same abstraction level.


>   - There might be some implications related to using jextract on
> third-party headers that we are unsure of and would require additional
> approvals before proceeding with this specific approach.
>

OK, I don't know how this works with licensing if that's what you mean.
I've used jextract (and even wrote a JavaFX GUI wrapper for it) on 3rd
party headers and technically didn't see many issues.


> The most reasonable/probable FFM use would be to access a D3D12 backend
> DLL we create (so basically accessing current prism_d3d12.dll but with FFM
> and not JNI). But it is something that we can explore in the future, for
> now I think there are more higher-priority tasks related to this backend.


One of FFM's goals is to eliminate the need to create a bridge DLL like
prism_d3d12. I don't know if it's technically feasible with regard to how
native memory management works and because it works with C and not
completely with C++.

As for what to move to the Java side, there could be a performance tradeoff
between an increased number of native calls and the ability of the JIT to
optimize them. There are tricks there that we could use. There will be
quite a bit to explore when the minimal working state is reached. By the
way, Metal could use it too if it's not very far out, but I certainly would
not replace the current ES and D3D9 code with FFM.


On Thu, Oct 17, 2024 at 5:04 PM Lukasz Kostyra <lukasz.kostyra at oracle.com>
wrote:

> Hi Nir,
>
>
>
> Thanks for suggestions! Apologies for being a bit late, needed to wrap my
> head around FFM before responding.
>
>
>
> In many cases I mostly followed D3D9 backend's design, so the similarities
> can be there. =) But there definitely is room for improvement which I also
> see and would like to tackle before the backend reaches a "mature enough"
> state. When working on 3D part of JFX I also saw that there is some data
> duplication happening, especially with Light/Mesh/MeshView code. I'd like
> to trim this as well, but it was not a priority yet - I preferred to have
> a working code now and all sorts of cleanups or reorganizations like that
> can come later.
>
>
>
> Regarding FFM - that is an interesting idea that we could explore at some
> point, but I'm not sure if we could (or even should) move all the
> implementation details to Java and limit native calls to just D3D12.
> There are a few problems with that:
>
>   - There is quite a bit of logic which integrates better on C/C++ side
> with D3D12, mostly due to 12's low-level-ness. Rewriting those in Java
> feels like quite a bit of a task in itself, and might create more problems
> than it solves.
>
>   - Raising the abstraction from 12’s low-level approach instead of just
> making a “passthrough” is also something that’s quite frequently done,
> mostly to make the API a bit more useable and suitable for project’s
> specific needs (see ex. NativeBuffer abstractifying 12’s heap types into
> simply data being CPU visible, or code managing mid-frame data like
> CommandLists, shader constants, 2D vertices etc.). This could also help
> later down the line when adding potential new features, simplifying the
> calls made from Java side.
>
>   - There might be some implications related to using jextract on
> third-party headers that we are unsure of and would require additional
> approvals before proceeding with this specific approach.
>
>
>
> The most reasonable/probable FFM use would be to access a D3D12 backend
> DLL we create (so basically accessing current prism_d3d12.dll but with FFM
> and not JNI). But it is something that we can explore in the future, for
> now I think there are more higher-priority tasks related to this backend.
>
>
>
> Lukasz
>
>
>
> *From:* Nir Lisker <nlisker at gmail.com>
> *Sent:* Tuesday, 15 October 2024 14:28
> *To:* Lukasz Kostyra <lukasz.kostyra at oracle.com>
> *Cc:* openjfx-dev <openjfx-dev at openjdk.org>
> *Subject:* [External] : Re: JavaFX Direct3D 12 rendering pipeline for
> Windows
>
>
>
> This is exciting news!
>
>
>
> I looked at the code quickly. Here are my thoughts.
>
>
>
> My biggest remark is that JNI is still being used, which is being more and
> more restricted. Recently, Kevin announced that we will be FFM-ready,
> surely by the time this pipeline is released. I would like to see (and can
> help, having used FFM since incubation) D3D12 being based on FFM.
>
>
>
> This will also allow less cluttered code to be written with less effort.
> Consider this mess in D3D12NativeMeshView (exists in D3D9 also, not blaming
> you for this :) ):
>
>
>
>     private native void nSetLight(long ptr, int index, float x, float y,
> float z, float r, float g, float b,
>                                   float enabled, float ca, float la, float
> qa, float isAttenuated, float maxRange,
>                                   float dirX, float dirY, float dirZ,
> float innerAngle, float outerAngle, float falloff);
>
>
>
> That, in a pure Java world, would be `void nSetLight(Light)` with `Light`
> holding all its parameters. In fact, this could make whole files that just
> prepare the data for the native layer redundant. At least I know this is
> the case in the current D3D9 implementation.
>
>
>
> It might also be the case that the pointer juggling that we do currently
> (using `long ptr` everywhere) will not be needed because FFM does it for us
> (Arena takes care of the lifecycle context). However, when it comes to
> resource deallocation there are many caveats, so I'm speculating here.
>
>
>
> Another design element that I think we can get rid of is the duplication
> of data structures in the Java and native sides. if Java has a Light,
> Material, and Mesh, the native side doesn't need them in reality, but in
> D3D9 (and somewhat in D3D12) it does have them. Without checking, I suspect
> that this duplication leads to inefficient memory usage. I speculate that
> it was done to minimize the number of native calls, a consideration that
> could be overturned with FFM. In general, a lot of the code in C++ can be
> potentially moved to Java, with only the calls to DirectX being left of
> course.
>
>
>
> Thanks for this herculean effort!
>
>
>
> On Mon, Oct 14, 2024 at 7:10 PM Lukasz Kostyra <lukasz.kostyra at oracle.com>
> wrote:
>
> Hello openjfx-dev,
>
>
>
> we just pushed a prototype of a new JavaFX Direct3D 12 rendering pipeline
>
> for Windows to a new "direct3d12" branch on jfx-sandbox. It is more than an
>
> experiment branch - we intend to fully develop the D3D12 backend there.
>
>
>
> We're not necessarily looking for contributions at this point, but if
> anyone
>
> has early feedback about it or wants to try it by building it themselves,
>
> that would be fine. We also did not test it on a wider range of hardware,
> so
>
> your mileage may vary. While D3D12 pipeline will build by default, D3D9
>
> pipeline is still the default pick at runtime. To run anything on D3D12
>
> pipeline you need to force it with ex.:
>
>   java -Dprism.order=d3d12 ...
>
>
>
> Backend supports 2D rendering (albeit with some graphical issues here and
> there
>
> that need to be ironed out) and basic 3D rendering. Expect not everything
> fully
>
> working yet (ex. some gradients on 2D controls are incorrect, or 3D-in-2D
> will
>
> straight up not work) and the performance not matching D3D9 yet. Our goal
> is to
>
> first reach feature completion and then focus on performance.
>
>
>
> Lukasz
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/openjfx-dev/attachments/20241017/35d6afe1/attachment-0001.htm>