[Feature Proposal] Vertex Colors on TriangleMesh

[Knee Snap]

Hi Michael,

Thanks for the response!

*> Have you thought about ways how we could allow deverlopers to define
custom formats? Maybe that would also require a new kind of Mesh that can
accept custom data.*
Yep, and while I'd very much like this eventually, this cannot occur before
user-supplied shaders are supported since otherwise there'd be nothing done
with any user-configured data. User-supplied shaders would be a very large
amount of work in many different aspects.
For example, there isn't even currently a way to tell if the 3D backend
JavaFX has running is OpenGL or Direct3D, let alone manage shaders.
I don't know that I'm qualified to even implement something so large for an
open-source immutable API yet. And, it would also ask more from
developers who utilize JavaFX, as they now need to write shaders for each
supported backend, and juggle a lot objects & concepts more when it comes
to rendering.
I still see value in keeping a platform-agnostic shader-free TriangleMesh
which has vertex coloring even after a hypothetical user-supplied shader
system, which could co-exist separate from the "TriangleMesh" as perhaps a
new "AdvancedTriangleMesh" class.
Now here's where I think you've brought up an interesting point and I'll
counter with a slight change to my design.

*> However, with the introduction of programmable shaders, the structure of
vertices can be freely defined by developers. It seems to me that your
proposed API would pick just one (maybe useful) scenario, and eternally
bake it into JavaFX APIs. What if we get custom shaders one day? Then we
would need a way to customize the vertex format again.*
Yes! It is just one scenario and shaders are the dream, but I still believe
it's worth baking into the JavaFX API due to how useful it is, I imagine
that this feature can co-exist separately from the hypothetical
user-supplied shader support.
I mentioned before in a previous email that I don't think having vertex
colors takes away from the ideal future where custom shaders are supported.
Since there seems to be a concern here, I'll expand upon what I meant by
that.

The current TriangleMesh.java is equivalent to a non-shader fixed rendering
pipeline, as it has an explicitly defined group of buffers. Some of which
(normals) are already optional.
The design of the future user-supplied shader system has not been made
worse by supporting vertex colors, as we already have the same exact
eternally baked API already inside TriangleMesh.java.

It's true though that TriangleMesh's existing buffers are mostly applicable
to pretty much every future shader, but it's not entirely the case.
Normals are also currently optional, and also might not be desired by
certain custom shaders when it's time. I'll concede that normals are
somewhat more common than vertex colors, but it's the same problem in the
same file.

Initially because of this observation about normals, my thought was that
TriangleMesh wouldn't be the parent class of any future mesh with shader
support. There would be two separate mesh base classes, TriangleMesh (fixed
capabilities), and something like AdvancedTriangleMesh (allowing
user-specified shaders).
Any solution which was good enough for normals is also good enough for
vertex colors right?

Well, after your critique, I've come up with another option, I'm curious if
you prefer this one.
Instead of modifying TriangleMesh.java and having an unrelated
AdvancedTriangleMesh class in the future, this one can consider that just
because normals are in TriangleMesh.java a bad thing doesn't mean vertex
colors need to clog it further.
So, I'd instead create "GouraudShadedTriangleMesh" which extends
TriangleMesh, and it's what would contain the vertex buffer, etc. So it can
have the coloring, and also allow a future AdvancedTriangleMesh to extend
the original TriangleMesh without having a hint of the vertex color buffer.

I believe this should adequately address why I think this is good to have
in the API forever.

*> Currently we have two predefined vertex formats:
VertexFormat.POINT_TEXCOORD and VertexFormat.POINT_NORMAL_TEXCOORD. It
doesn't seem obvious to me that the best way forward is to add more bespoke
formats here.*
I partially agree, and partially disagree. I agree that we should not be
adding VertexFormat options left and right. But the only other scenario I
can imagine as theoretically adding more VertexFormats is if it became time
to support quads. Either that or a new user-supplied shader system, but
that isn't really relevant.
Vertex colors feel to me like the final missing piece before the current
non-custom shader mesh support is fully featured, so I don't think it's
unreasonable to add new VertexFormat entries.
In theory it would be possible with the "GouraudShadedTriangleMesh" idea to
not actually create any new VertexFormat object/functionality, and keep
such functionality within the "GouraudShadedTriangleMesh" class itself.
Personally I think it depends entirely on the decision of "Would a
potential future user-supplied shader system extend TriangleMesh or should
it be its own thing?"

Thanks again for the feedback!


On Tue, Aug 20, 2024 at 10:51 PM Michael Strauß <michaelstrau2 at gmail.com>
wrote:

> Vertex colors usually refers to a technique where the vertex format is
> enriched with an additional color component, and this color component is
> then used in one of the shader stages to affect the computation in some way
> (usually by combining it with the texture color).
>
> As you correctly say, this technique is decades-old, it existed even back
> in the days of fixed function pipelines.
>
> However, with the introduction of programmable shaders, the structure of
> vertices can be freely defined by developers. It seems to me that your
> proposed API would pick just one (maybe useful) scenario, and eternally
> bake it into JavaFX APIs. What if we get custom shaders one day? Then we
> would need a way to customize the vertex format again.
>
> Currently we have two predefined vertex formats:
> VertexFormat.POINT_TEXCOORD and VertexFormat.POINT_NORMAL_TEXCOORD.
> It doesn't seem obvious to me that the best way forward is to add more
> bespoke formats here.
>
> Have you thought about ways how we could allow deverlopers to define
> custom formats? Maybe that would also require a new kind of Mesh that can
> accept custom data.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/openjfx-dev/attachments/20240820/7c428a6e/attachment-0001.htm>