<div dir="ltr">Hi Lukasz,<div><br></div><div>Yes, FFM is quite a bit to take in. I haven't mastered the newest version myself yet.</div><div><br></div><div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">In many cases I mostly followed D3D9 backend's design, so the similarities can be there. =) But there definitely is room for improvement which I <span lang="EN-US">also see and </span>would like to tackle before the backend reaches a "mature enough" state. When working on 3D part of JFX I <span lang="EN-US">also </span>saw that there is some <span lang="EN-US">data</span> duplication happening<span lang="EN-US">,</span> especially with Light/Mesh/MeshView code. I'd like to trim this as well, but it was not a priority yet<span lang="EN-US"> - I preferred to have a working code now and all sorts of cleanups or reorganizations like that can come later</span>.<br> <br>Regarding FFM - that is an interesting idea that we could explore<span lang="EN-US"> at some point</span>, but I'm not sure if we could (or even should) move <span lang="EN-US">all the implementation details</span> to Java and limit native calls to just D3D12. There are a <span lang="EN-US">few</span> problems with that:<br></blockquote><div><br></div><div>Because FFM saves a lot of handwriting of fragile JNI code, the sooner it's adopted, the less headache you (or collaborators) will have. So while it's true that there's a minimal working level the code should be brought to, adopting FFM will mean rewriting a lot of JNI code. This is why I suggest going with FFM as soon as that minimal level is reached. It has advantages in type safety and compiler checks that JNI does not have, including the ability to use objects-structs mapping instead of breaking them down to JNI-supported primitives (long -> jlong etc.).<br></div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"> - There is quite a bit of logic which integrates better on C/C++ side with D3D12, mostly due to 12's low-level-ness<span lang="EN-US">. Rewriting those in Java feels like quite a bit of a task in itself, and might create more problems than it solves.<br></span></blockquote><div><br></div><div><div>I'm not sure what you mean by this. If you mean the current sandbox code that uses JNI will need to be rewritten, then yes, that will be a rewrite, which is why I advocate to take advantage of FFM sooner rather than later. If you're mostly copying from D3D9 then it does save some effort on the JNI side. D3D12 (or any DLL) doesn't know what's calling it, whether it's C++ or Java, so there shouldn't be a difference in "integration". At least not when I called native libraries, but perhaps I'm missing something. I can also give a concrete example on some code snippet in our D3D9 or 12 if it helps clarify my intention.</div></div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><span lang="EN-US"><u></u><u></u></span><span lang="EN-US"> - Raising the abstraction from 12’s low-level approach instead of just making a “passthrough” is also something that’s quite frequently done, mostly to make the API a bit more useable and suitable for project’s specific needs (see ex. NativeBuffer abstractifying 12’s heap types into simply data being CPU visible, or code managing mid-frame data like CommandLists, shader constants, 2D vertices etc.). This could also help later down the line when adding potential new features, simplifying the calls made from Java side.<br></span></blockquote><div><br></div><div>I'm not sure what this means either :p The abstraction that exists or will exist in C/C++ that surrounds the D3D API can exist instead in Java. It's the same abstraction level.</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><span lang="EN-US"></span> - There might be some<span lang="EN-US"> implications related to using jextract on third-party headers that we are unsure of and would require additional approvals before proceeding with this specific approach.<br></span></blockquote><div><br></div><div>OK, I don't know how this works with licensing if that's what you mean. I've used jextract (and even wrote a JavaFX GUI wrapper for it) on 3rd party headers and technically didn't see many issues.</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><span lang="EN-US">The</span><span lang="EN-US"> </span><span lang="EN-US">most reasonable/probable</span><span lang="EN-US"> </span><span lang="EN-US">FFM use </span>would <span lang="EN-US">be to access a D3D12 backend DLL we create (so basically accessing current prism_d3d12.dll but with FFM and not JNI). But it is something that we can explore in the future, for now I think there are more higher-priority tasks related to this backend.</span></blockquote><p class="MsoNormal"><u></u></p><p class="MsoNormal"><u></u></p></div><div><br></div><div>One of FFM's goals is to eliminate the need to create a bridge DLL like prism_d3d12. I don't know if it's technically feasible with regard to how native memory management works and because it works with C and not completely with C++.</div><div><br></div><div>As for what to move to the Java side, there could be a performance tradeoff between an increased number of native calls and the ability of the JIT to optimize them. There are tricks there that we could use. There will be quite a bit to explore when the minimal working state is reached. By the way, Metal could use it too if it's not very far out, but I certainly would not replace the current ES and D3D9 code with FFM.<br></div><div><br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Thu, Oct 17, 2024 at 5:04 PM Lukasz Kostyra <<a href="mailto:lukasz.kostyra@oracle.com">lukasz.kostyra@oracle.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div class="msg4664229473832342332">
<div lang="en-NL" style="overflow-wrap: break-word;">
<div class="m_4664229473832342332WordSection1">
<p class="MsoNormal"><span lang="EN-US">Hi Nir,<u></u><u></u></span></p>
<p class="MsoNormal"><span><u></u> <u></u></span></p>
<p class="MsoNormal"><span>Thanks for suggestions!</span><span>
<span lang="EN-US">Apologies for being a bit late, needed to wrap my head around FFM before responding.<u></u><u></u></span></span></p>
<p class="MsoNormal"><span lang="EN-US"><u></u> <u></u></span></p>
<p class="MsoNormal"><span>In many cases I mostly followed D3D9 backend's design, so the similarities can be there. =) But there definitely is room for improvement which I
</span><span lang="EN-US">also see and </span>
<span>would like to tackle before the backend reaches a "mature enough" state.</span><span>
</span><span>When working on 3D part of JFX I </span>
<span lang="EN-US">also </span><span>saw that there is some</span><span>
<span lang="EN-US">data</span></span><span> duplication happening</span><span lang="EN-US">,</span><span> especially with Light/Mesh/MeshView code. I'd
like to trim this as well, but it was not a priority yet</span><span lang="EN-US"> - I preferred to have a working code now and all sorts of cleanups or reorganizations like that can come later</span><span>.<u></u><u></u></span></p>
<p class="MsoNormal"><span><u></u> <u></u></span></p>
<p class="MsoNormal"><span>Regarding FFM - that is an interesting idea that we could explore</span><span lang="EN-US"> at some point</span><span>, but I'm
not sure if we could (or even should) move </span><span lang="EN-US">all the implementation details</span><span> to Java and limit native calls to just D3D12. There are a
</span><span lang="EN-US">few</span><span> problems with that:<u></u><u></u></span></p>
<p class="MsoNormal"><span> - There is quite a bit of logic which integrates better on C/C++ side with D3D12, mostly due to 12's low-level-ness</span><span lang="EN-US">. Rewriting those
in Java feels like quite a bit of a task in itself, and might create more problems than it solves.<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US"> - Raising the abstraction from 12’s low-level approach instead of just making a “passthrough” is also something that’s quite frequently done, mostly to make the API a bit more useable
and suitable for project’s specific needs (see ex. NativeBuffer abstractifying 12’s heap types into simply data being CPU visible, or code managing mid-frame data like CommandLists, shader constants, 2D vertices etc.). This could also help later down the line
when adding potential new features, simplifying the calls made from Java side.<u></u><u></u></span></p>
<p class="MsoNormal"><span> - There might be some</span><span lang="EN-US"> implications related to using jextract on third-party headers that we are unsure of and would require additional
approvals before proceeding with this specific approach.<u></u><u></u></span></p>
<p class="MsoNormal"><span><u></u> <u></u></span></p>
<p class="MsoNormal"><span lang="EN-US">The</span><span lang="EN-US">
</span><span lang="EN-US">most reasonable/probable</span><span lang="EN-US">
</span><span lang="EN-US">FFM use </span><span>would
</span><span lang="EN-US">be to access a D3D12 backend DLL we create (so basically accessing current prism_d3d12.dll but with FFM and not JNI). But it is something that we can explore in the future, for now I think there are
more higher-priority tasks related to this backend.<u></u><u></u></span></p>
<p class="MsoNormal"><span><u></u> <u></u></span></p>
<p class="MsoNormal"><span lang="EN-US">Lukasz<u></u><u></u></span></p>
<p class="MsoNormal"><span><u></u> <u></u></span></p>
<div style="border-right:none;border-bottom:none;border-left:none;border-top:1pt solid rgb(225,225,225);padding:3pt 0cm 0cm">
<p class="MsoNormal"><b><span lang="EN-US">From:</span></b><span lang="EN-US"> Nir Lisker <<a href="mailto:nlisker@gmail.com" target="_blank">nlisker@gmail.com</a>>
<br>
<b>Sent:</b> Tuesday, 15 October 2024 14:28<br>
<b>To:</b> Lukasz Kostyra <<a href="mailto:lukasz.kostyra@oracle.com" target="_blank">lukasz.kostyra@oracle.com</a>><br>
<b>Cc:</b> openjfx-dev <<a href="mailto:openjfx-dev@openjdk.org" target="_blank">openjfx-dev@openjdk.org</a>><br>
<b>Subject:</b> [External] : Re: JavaFX Direct3D 12 rendering pipeline for Windows<u></u><u></u></span></p>
</div>
<p class="MsoNormal"><u></u> <u></u></p>
<div>
<p class="MsoNormal">This is exciting news!<u></u><u></u></p>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">I looked at the code quickly. Here are my thoughts.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">My biggest remark is that JNI is still being used, which is being more and more restricted. Recently, Kevin announced that we will be FFM-ready, surely by the time this pipeline is released. I would like to see (and can help, having used
FFM since incubation) D3D12 being based on FFM.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">This will also allow less cluttered code to be written with less effort. Consider this mess in D3D12NativeMeshView (exists in D3D9 also, not blaming you for this :) ):<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal"> private native void nSetLight(long ptr, int index, float x, float y, float z, float r, float g, float b,<br>
float enabled, float ca, float la, float qa, float isAttenuated, float maxRange,<br>
float dirX, float dirY, float dirZ, float innerAngle, float outerAngle, float falloff);<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">That, in a pure Java world, would be `void nSetLight(Light)` with `Light` holding all its parameters. In fact, this could make whole files that just prepare the data for the native layer redundant. At least I know this is the case in the
current D3D9 implementation.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">It might also be the case that the pointer juggling that we do currently (using `long ptr` everywhere) will not be needed because FFM does it for us (Arena takes care of the lifecycle context). However, when it comes to resource deallocation
there are many caveats, so I'm speculating here.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">Another design element that I think we can get rid of is the duplication of data structures in the Java and native sides. if Java has a Light, Material, and Mesh, the native side doesn't need them in reality, but in D3D9 (and somewhat in
D3D12) it does have them. Without checking, I suspect that this duplication leads to inefficient memory usage. I speculate that it was done to minimize the number of native calls, a consideration that could be overturned with FFM. In general, a lot of the
code in C++ can be potentially moved to Java, with only the calls to DirectX being left of course.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">Thanks for this herculean effort!<u></u><u></u></p>
</div>
</div>
<p class="MsoNormal"><u></u> <u></u></p>
<div>
<div>
<p class="MsoNormal">On Mon, Oct 14, 2024 at 7:10 PM Lukasz Kostyra <<a href="mailto:lukasz.kostyra@oracle.com" target="_blank">lukasz.kostyra@oracle.com</a>> wrote:<u></u><u></u></p>
</div>
<blockquote style="border-top:none;border-right:none;border-bottom:none;border-left:1pt solid rgb(204,204,204);padding:0cm 0cm 0cm 6pt;margin:5pt 0cm 5pt 4.8pt">
<div>
<div>
<div>
<p class="MsoNormal">Hello openjfx-dev,<u></u><u></u></p>
<p class="MsoNormal"> <u></u><u></u></p>
<p class="MsoNormal">we just pushed a prototype of a new JavaFX Direct3D 12 rendering pipeline<u></u><u></u></p>
<p class="MsoNormal">for Windows to a new "direct3d12" branch on jfx-sandbox. It is more than an<u></u><u></u></p>
<p class="MsoNormal">experiment branch - we intend to fully develop the D3D12 backend there.<u></u><u></u></p>
<p class="MsoNormal"> <u></u><u></u></p>
<p class="MsoNormal">We're not necessarily looking for contributions at this point, but if anyone<u></u><u></u></p>
<p class="MsoNormal">has early feedback about it or wants to try it by building it themselves,<u></u><u></u></p>
<p class="MsoNormal">that would be fine. We also did not test it on a wider range of hardware, so<u></u><u></u></p>
<p class="MsoNormal">your mileage may vary. While D3D12 pipeline will build by default, D3D9<u></u><u></u></p>
<p class="MsoNormal">pipeline is still the default pick at runtime. To run anything on D3D12<u></u><u></u></p>
<p class="MsoNormal">pipeline you need to force it with ex.:<u></u><u></u></p>
<p class="MsoNormal"> java -Dprism.order=d3d12 ...<u></u><u></u></p>
<p class="MsoNormal"> <u></u><u></u></p>
<p class="MsoNormal">Backend supports 2D rendering (albeit with some graphical issues here and there<u></u><u></u></p>
<p class="MsoNormal">that need to be ironed out) and basic 3D rendering. Expect not everything fully<u></u><u></u></p>
<p class="MsoNormal">working yet (ex. some gradients on 2D controls are incorrect, or 3D-in-2D will<u></u><u></u></p>
<p class="MsoNormal">straight up not work) and the performance not matching D3D9 yet. Our goal is to<u></u><u></u></p>
<p class="MsoNormal">first reach feature completion and then focus on performance.<u></u><u></u></p>
<p class="MsoNormal"> <u></u><u></u></p>
<p class="MsoNormal">Lukasz<u></u><u></u></p>
</div>
</div>
</div>
</blockquote>
</div>
</div>
</div>
</div></blockquote></div>