[Feature Proposal] Vertex Colors on TriangleMesh

[Knee Snap]

Hi all, just sending an update on the proposal since I'm hoping to start
work on a draft PR for this, but I'd like feedback on the proposed API
(with & without the changes mentioned in the previous email).

Thanks!

On Tue, Aug 20, 2024 at 11:49 PM Knee Snap <kneester77 at gmail.com> wrote:

> Hi Michael,
>
> Thanks for the response!
>
> *> Have you thought about ways how we could allow deverlopers to define
> custom formats? Maybe that would also require a new kind of Mesh that can
> accept custom data.*
> Yep, and while I'd very much like this eventually, this cannot occur
> before user-supplied shaders are supported since otherwise there'd be
> nothing done with any user-configured data. User-supplied shaders would be
> a very large amount of work in many different aspects.
> For example, there isn't even currently a way to tell if the 3D backend
> JavaFX has running is OpenGL or Direct3D, let alone manage shaders.
> I don't know that I'm qualified to even implement something so large for
> an open-source immutable API yet. And, it would also ask more from
> developers who utilize JavaFX, as they now need to write shaders for each
> supported backend, and juggle a lot objects & concepts more when it comes
> to rendering.
> I still see value in keeping a platform-agnostic shader-free TriangleMesh
> which has vertex coloring even after a hypothetical user-supplied shader
> system, which could co-exist separate from the "TriangleMesh" as perhaps a
> new "AdvancedTriangleMesh" class.
> Now here's where I think you've brought up an interesting point and I'll
> counter with a slight change to my design.
>
> *> However, with the introduction of programmable shaders, the structure
> of vertices can be freely defined by developers. It seems to me that your
> proposed API would pick just one (maybe useful) scenario, and eternally
> bake it into JavaFX APIs. What if we get custom shaders one day? Then we
> would need a way to customize the vertex format again.*
> Yes! It is just one scenario and shaders are the dream, but I still
> believe it's worth baking into the JavaFX API due to how useful it is, I
> imagine that this feature can co-exist separately from the hypothetical
> user-supplied shader support.
> I mentioned before in a previous email that I don't think having vertex
> colors takes away from the ideal future where custom shaders are supported.
> Since there seems to be a concern here, I'll expand upon what I meant by
> that.
>
> The current TriangleMesh.java is equivalent to a non-shader fixed
> rendering pipeline, as it has an explicitly defined group of buffers. Some
> of which (normals) are already optional.
> The design of the future user-supplied shader system has not been made
> worse by supporting vertex colors, as we already have the same exact
> eternally baked API already inside TriangleMesh.java.
>
> It's true though that TriangleMesh's existing buffers are mostly
> applicable to pretty much every future shader, but it's not entirely the
> case.
> Normals are also currently optional, and also might not be desired by
> certain custom shaders when it's time. I'll concede that normals are
> somewhat more common than vertex colors, but it's the same problem in the
> same file.
>
> Initially because of this observation about normals, my thought was that
> TriangleMesh wouldn't be the parent class of any future mesh with shader
> support. There would be two separate mesh base classes, TriangleMesh (fixed
> capabilities), and something like AdvancedTriangleMesh (allowing
> user-specified shaders).
> Any solution which was good enough for normals is also good enough for
> vertex colors right?
>
> Well, after your critique, I've come up with another option, I'm curious
> if you prefer this one.
> Instead of modifying TriangleMesh.java and having an unrelated
> AdvancedTriangleMesh class in the future, this one can consider that just
> because normals are in TriangleMesh.java a bad thing doesn't mean vertex
> colors need to clog it further.
> So, I'd instead create "GouraudShadedTriangleMesh" which extends
> TriangleMesh, and it's what would contain the vertex buffer, etc. So it can
> have the coloring, and also allow a future AdvancedTriangleMesh to extend
> the original TriangleMesh without having a hint of the vertex color buffer.
>
> I believe this should adequately address why I think this is good to have
> in the API forever.
>
> *> Currently we have two predefined vertex formats:
> VertexFormat.POINT_TEXCOORD and VertexFormat.POINT_NORMAL_TEXCOORD. It
> doesn't seem obvious to me that the best way forward is to add more bespoke
> formats here.*
> I partially agree, and partially disagree. I agree that we should not be
> adding VertexFormat options left and right. But the only other scenario I
> can imagine as theoretically adding more VertexFormats is if it became time
> to support quads. Either that or a new user-supplied shader system, but
> that isn't really relevant.
> Vertex colors feel to me like the final missing piece before the current
> non-custom shader mesh support is fully featured, so I don't think it's
> unreasonable to add new VertexFormat entries.
> In theory it would be possible with the "GouraudShadedTriangleMesh" idea
> to not actually create any new VertexFormat object/functionality, and keep
> such functionality within the "GouraudShadedTriangleMesh" class itself.
> Personally I think it depends entirely on the decision of "Would a
> potential future user-supplied shader system extend TriangleMesh or should
> it be its own thing?"
>
> Thanks again for the feedback!
>
>
> On Tue, Aug 20, 2024 at 10:51 PM Michael Strauß <michaelstrau2 at gmail.com>
> wrote:
>
>> Vertex colors usually refers to a technique where the vertex format is
>> enriched with an additional color component, and this color component is
>> then used in one of the shader stages to affect the computation in some way
>> (usually by combining it with the texture color).
>>
>> As you correctly say, this technique is decades-old, it existed even back
>> in the days of fixed function pipelines.
>>
>> However, with the introduction of programmable shaders, the structure of
>> vertices can be freely defined by developers. It seems to me that your
>> proposed API would pick just one (maybe useful) scenario, and eternally
>> bake it into JavaFX APIs. What if we get custom shaders one day? Then we
>> would need a way to customize the vertex format again.
>>
>> Currently we have two predefined vertex formats:
>> VertexFormat.POINT_TEXCOORD and VertexFormat.POINT_NORMAL_TEXCOORD.
>> It doesn't seem obvious to me that the best way forward is to add more
>> bespoke formats here.
>>
>> Have you thought about ways how we could allow deverlopers to define
>> custom formats? Maybe that would also require a new kind of Mesh that can
>> accept custom data.
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/openjfx-dev/attachments/20240822/cc23b61f/attachment.htm>