From jackyguo579 at gmail.com Sat Mar 13 02:58:14 2021 From: jackyguo579 at gmail.com (superminerJG) Date: Fri, 12 Mar 2021 21:58:14 -0500 Subject: Implementing a shader API Message-ID: Hi everyone. This problem might have been discussed numerous times, and has been held off for quite a while. Although we are starting to see lighting improvements added to JavaFX, I feel that it's necessary for JavaFX to have some kind of custom shader pipeline. However, nobody has gotten around to doing it, so I thought that I could lend a hand. Before I get into things, I would like to know how and what I can contribute. The questions are at the bottom of the email. Thanks! - jgcodes Questions: - Can high school students sign the OCA? (I saw the job title field on the form and wasn't so sure I was legally allowed to sign it) - When I implement the shader API, should it use some kind of GLSL mapping, or should I try to extend JSL to work in 3D? - Should I have sent this email to the openjfx-dev mailing list and not this one? From johan.vos at gluonhq.com Tue Mar 16 08:40:02 2021 From: johan.vos at gluonhq.com (Johan Vos) Date: Tue, 16 Mar 2021 09:40:02 +0100 Subject: Implementing a shader API In-Reply-To: References: Message-ID: On Sat, Mar 13, 2021 at 3:58 AM superminerJG wrote: > Hi everyone. > [...] > Questions: > > - Can high school students sign the OCA? (I saw the job title field on > the form and wasn't so sure I was legally allowed to sign it) > If the answer to this was "NO", it would mean we're doing something terribly wrong that we need to fix. I have no immediate knowledge on how to do this, hence Nir's suggestion about asking on adoption-discuss would be good. > - When I implement the shader API, should it use some kind of GLSL > mapping, or should I try to extend JSL to work in 3D? > The latter sounds the least intrusive approach. Michael and Nir have interesting comments in their replies. It's not a simple thing. > - Should I have sent this email to the openjfx-dev mailing list and not > this one? > At this moment, I think openjfx-discuss is the best mailing list. There are a number of conceptual things that need to be sorted first, and it would be good to keep the discussion alive on this list. Once there is something tangible, e.g. a prototype, we can move the discussion to openjfx-dev. Thanks for the ideas! - Johan From jackyguo579 at gmail.com Wed Mar 17 01:15:07 2021 From: jackyguo579 at gmail.com (Jacky Guo) Date: Tue, 16 Mar 2021 21:15:07 -0400 Subject: Shader API proposal Message-ID: This is my proposal as to how the shader API should work: In javafx.scene.paint 2 new classes would be added: - Shader - ShadedMaterial ShadedMaterial would contain methods relating to the material itself, probably including texture maps and the like. Shader would contain more low-level functions, like passing values to the shader or similar. I'm not sure if all the functionality should belong to one class, but in my mind, separating it like this will have its benefits. In addition, I haven't dabbled much with native graphics, so I will be needing some guidance on that end. From kevin.rushforth at oracle.com Fri Mar 19 13:28:39 2021 From: kevin.rushforth at oracle.com (Kevin Rushforth) Date: Fri, 19 Mar 2021 06:28:39 -0700 Subject: Shader API proposal In-Reply-To: References: Message-ID: <4ab801b9-ccea-614c-d116-f1a5fcf687ea@oracle.com> This isn't the first time that the question of allowing some sort of application-provided shaders has come up. There is an Enhancement request in JBS to track this, JDK-8134662 [1], but it's just a placeholder. I don't want to discourage research in this area, but let me point out just a few of the challenges that providing such an API would pose: 1. JavaFX already uses implicitly-generated shaders -- Both Prism (for rendering) and Decora (for image-based filter effects) create build-time-generated vertex and fragment shaders. Prism uses these generated shaders to render shapes, text, and images. It passes a generated set of uniform and varying parameters, along with the necessary texture objects, to these shaders. It isn't clear how an application-provided shader would interact with this. There are several possible approaches one might take with various pros and cons. The Decora shaders are less likely to interfere, but that's another area where an application developer might want to use an application-provided shader. 2. Add missing operations to JSL language -- The JSL language is only complete enough to do what was needed to implement the Decora (image filter effects) and 2D Prism rendering shaders. As such it is a subset of what GLSL, HLSL, and MSL can provide. Along with this, the JSLC (JSL Compiler) would need to implement them, and we would need a much more robust set of tests for the compiler. 3. JSL Language specification -- right now it is an internal tool, so the level of documentation needed is very low. If we were to productize it, it would need a complete spec along the lines of the FXML or JavaFX CSS spec. I think research in this area would be interesting, as long as you look at it as a research project, and not as a feature that is likely to become part of JavaFX any time soon. -- Kevin [1] https://bugs.openjdk.java.net/browse/JDK-8134662 On 3/16/2021 6:15 PM, Jacky Guo wrote: > This is my proposal as to how the shader API should work: > > In javafx.scene.paint 2 new classes would be added: > > - Shader > - ShadedMaterial > > ShadedMaterial would contain methods relating to the material itself, > probably including texture maps and the like. > > Shader would contain more low-level functions, like passing values to the > shader or similar. > > I'm not sure if all the functionality should belong to one class, but in my > mind, separating it like this will have its benefits. > > In addition, I haven't dabbled much with native graphics, so I will be > needing some guidance on that end. From nlisker at gmail.com Fri Mar 19 16:01:37 2021 From: nlisker at gmail.com (Nir Lisker) Date: Fri, 19 Mar 2021 18:01:37 +0200 Subject: Shader API proposal In-Reply-To: References: Message-ID: I think that the best initial approach would be hands-on. Learn the current shader rendering chain: 1. The implementation of Shape3D, LightBase and PhongMaterial. 2. The native pipelines. The java side graphics contexts send the relevant data to the native context. 3. The shaders. The native contexts pass on the data to the shaders. Try to make modifications to the public Java API that requires modifications in the shaders and change them accordingly so you'll get a full start-to-finish test feature, This will give you an idea of what it entails. Even if you do this for only 1 of the pipelines it's good enough. >From there you can start abstracting the idea. That would be my suggestion. - Nir On Wed, Mar 17, 2021 at 3:15 AM Jacky Guo wrote: > This is my proposal as to how the shader API should work: > > In javafx.scene.paint 2 new classes would be added: > > - Shader > - ShadedMaterial > > ShadedMaterial would contain methods relating to the material itself, > probably including texture maps and the like. > > Shader would contain more low-level functions, like passing values to the > shader or similar. > > I'm not sure if all the functionality should belong to one class, but in > my mind, separating it like this will have its benefits. > > In addition, I haven't dabbled much with native graphics, so I will be > needing some guidance on that end. > From nlisker at gmail.com Fri Mar 26 15:34:58 2021 From: nlisker at gmail.com (Nir Lisker) Date: Fri, 26 Mar 2021 18:34:58 +0300 Subject: Implementing a shader API In-Reply-To: References:

<04bdb68c-3d7c-7880-842d-65223f5dc6f5@videotron.ca> Message-ID: Let's keep the discussion in the openjfx-discuss at openjdk.java.net list for now. I'm sending this mail on the dev list just for continuity. I'm also very clueless as to how JavaFX links D3D shaders at runtime. Can > someone give me a quick rundown on that? > The D3D shader pointers are in D3DPhongShader.h. There is 1 vertex shader and several pixel shaders that account for different illumination modes for performance reasons, and these are swapped at runtime in D3DMeshView.cc whenever a render is requested. In this class, the vertex shader is set with IDirect3DDevice9::SetVertexShader, taking in D3DPhongShader::getVertexShader. The pixel shader is set with D3DPhongShader::setPixelShader, taking in the illumination mode parameters and retrieving the appropriate shader. It is then set with IDirect3DDevice9::SetPixelShader. Regarding the shader API, if an error occurs during shader compilation, > should it simply default to the basic Phong shaders or throw an Exception > (likely in a new package with other shader stuff)? > If you are talking about a new API, then this is too detailed to figure out at this point. We need to step back and figure out how to have things working with each other before figuring out what to do on invalid input. On Fri, Mar 26, 2021 at 5:58 PM Jacky Guo wrote: > Regarding the shader API, if an error occurs during shader compilation, > should it simply default to the basic Phong shaders or throw an Exception > (likely in a new package with other shader stuff)? > > I'm also very clueless as to how JavaFX links D3D shaders at runtime. Can > someone give me a quick rundown on that? > > On Wed, Mar 17, 2021 at 9:06 PM Dan Howard wrote: > >> This is cool. You should like up with Almas Baimagambetov >> >> >> https://github.com/AlmasB >> >> >> On 3/15/2021 10:12 PM, Jacky Guo wrote: >> > Regarding uniform parameters (i.e. variables set from Java and called on >> > the shader): >> > Since OpenGL's data passing uses locations, you could simply store a >> map of >> > uniform parameters (or whatever their JSL equivalent is) to their >> > locations, and call the required methods and perform the necessary >> > type-safety checks. >> > For D3D, you would still use the parameter map, but only for type >> safety. >> > >> > This is assuming I understand how the APIs work and I am not overlooking >> > something crucial. >> > >> > On Mon, Mar 15, 2021 at 4:51 AM Nir Lisker wrote: >> > >> >> I've thought about this too for a bit. My question would be how to tell >> >> the CPU what data to pass to the shader. Take OpenGL for example: >> writing >> >> an API to pass a shader and compiling it in runtime is not hard, but >> you >> >> need the CPU side to feed the shader its parameters. You would need an >> API >> >> for arbitrary data that the shader has been preconfigured to handle. >> That's >> >> quite an undertaking. >> >> And what level of flexibility is this going to give? The shader might >> >> require data that the public Java API does not expose, so you are >> limited >> >> in that regard too. When I wrote the enhancements for the shaders, I >> also >> >> had to change the Java side, which you won't be able to do. >> >> On the D3D side, the shaders are precompiled, so it's a slightly >> different >> >> story. >> >> >> >> - Can high school students sign the OCA? (I saw the job title field >> >>> on the form and wasn't so sure I was legally allowed to sign it) >> >> >> >> Maybe ask on adoption-discuss at openjdk.java.net. >> >> >> >> On Mon, Mar 15, 2021 at 1:50 AM Michael Ennen >> >> wrote: >> >> >> >>> I messed around with adding features to JavaFX's "JSLC" (Java Shader >> >>> Language Compiler) which currently is only used >> >>> at compile-time for JavaFX to convert "JSL" shaders to various >> platform >> >>> dependent shader languages (GLSL, HLSL, etc.). >> >>> >> >>> In my opinion the ideal way to implement a public shader API would be >> to >> >>> extend JSLC to be able to be used either >> >>> at user application compile time, or more ideally, at application >> run-time >> >>> to dynamically generate platform dependent shaders >> >>> depending on what platform JavaFX is running on. Then a JavaFX >> application >> >>> developer can write platform independent >> >>> shaders in JSL. This would be quite a large undertaking and have a >> high >> >>> burden of maintenance to keep it current with >> >>> fast moving shader languages. >> >>> >> >>> I was able to convert JSLC grammar to antlr4 (from the previous antlr3 >> >>> version) in this merged commit (pre-dates the move >> >>> to Github): >> >>> >> >>> >> >>> >> https://github.com/openjdk/jfx/commit/52adea7c3635656421db766641416ff28213928f#diff-9c8e9478200eb2c014c196feec63537046751c0d6fe80f90b8506b11abfcc2f7 >> >>> >> >>> I then took advantage of a new feature in antlr4 which allows for >> >>> extracting embedded actions from the grammar definition file to a >> visitor >> >>> class: >> >>> >> >>> >> >>> >> https://github.com/openjdk/jfx/commit/8889330cc30a47361d47519be2ec662fb5ebe856#diff-9c8e9478200eb2c014c196feec63537046751c0d6fe80f90b8506b11abfcc2f7 >> >>> >> >>> I then, in a test repository did some work in fleshing out what it >> would >> >>> take to implement more features for JSLC which you can find (as messy >> >>> commits) here: >> >>> >> >>> https://github.com/brcolow/jslc >> >>> >> >>> That's just my two cents on how a public shader API should best be >> added >> >>> to >> >>> JavaFX, by building on previous work that exists in the source code >> >>> repository but is only >> >>> used in a very limited and small way - but there is much potential >> there >> >>> and I believe the original author of JSL's intent was to have it be a >> >>> public API feature to allow >> >>> JavaFX developers to write platform independent shaders in JSL. >> >>> >> >>> >> >>> On Sun, Mar 14, 2021 at 3:32 PM superminerJG >> >>> wrote: >> >>> >> >>>> Hi everyone. >> >>>> >> >>>> This problem might have been discussed numerous times, and has been >> held >> >>>> off for quite a while. >> >>>> >> >>>> Although we are starting to see lighting improvements added to >> JavaFX, I >> >>>> feel that it's necessary for JavaFX to have some kind of custom >> shader >> >>>> pipeline. However, nobody has gotten around to doing it, so I thought >> >>> that >> >>>> I could lend a hand. Before I get into things, I would like to know >> how >> >>> and >> >>>> what I can contribute. The questions are at the bottom of the email. >> >>>> >> >>>> Thanks! >> >>>> - jgcodes >> >>>> >> >>>> Questions: >> >>>> >> >>>> - Can high school students sign the OCA? (I saw the job title >> field >> >>> on >> >>>> the form and wasn't so sure I was legally allowed to sign it) >> >>>> - When I implement the shader API, should it use some kind of >> GLSL >> >>>> mapping, or should I try to extend JSL to work in 3D? >> >>>> - Should I have sent this email to another mailing list? >> >>>> >> >>> >> >>> -- >> >>> Michael Ennen >> >>> >> > From jackyguo579 at gmail.com Tue Mar 30 13:11:42 2021 From: jackyguo579 at gmail.com (Jacky Guo) Date: Tue, 30 Mar 2021 09:11:42 -0400 Subject: Implementing a shader API In-Reply-To: References: Message-ID: There are already two third-party libraries which do just that, namely LWJGL and JOGL. The reason I'm implementing custom shaders here is because even the old Java3D library had support for them. However, we're adapting JSL for it because different platforms require different shaders. (FX uses DirectX on Windows and OpenGL ES on most other platforms) On Sat, Mar 27, 2021 at 7:22 AM Eric Bresie wrote: > Don?t suppose integrating Java OpenGL if not already done would help? > > https://jogamp.org/jogl/www/ > > They are a little dated but there are the OpenGL related JSRs like > > JSR 239: JavaTM Binding for the OpenGL? ES API > https://www.jcp.org/en/jsr/detail?id=239 > I think this was targeted at Java ME mobile platforms > > JSR 231: JavaTM Binding for the OpenGL? API > https://www.jcp.org/en/jsr/detail?id=231 > > Or is some updated JSR really needed here? > > Eric Bresie > Ebresie at gmail.com > > On March 26, 2021 at 10:34:58 AM CDT, Nir Lisker > wrote: > Let's keep the discussion in the openjfx-discuss at openjdk.java.net list > for > now. I'm sending this mail on the dev list just for continuity. > > I'm also very clueless as to how JavaFX links D3D shaders at runtime. Can > > someone give me a quick rundown on that? > > > The D3D shader pointers are in D3DPhongShader.h. There is 1 vertex shader > and several pixel shaders that account for different illumination modes > for > performance reasons, and these are swapped at runtime in D3DMeshView.cc > whenever a render is requested. > In this class, the vertex shader is set > with IDirect3DDevice9::SetVertexShader, taking > in D3DPhongShader::getVertexShader. The pixel shader is set > with D3DPhongShader::setPixelShader, taking in the illumination mode > parameters and retrieving the appropriate shader. It is then set with > IDirect3DDevice9::SetPixelShader. > > Regarding the shader API, if an error occurs during shader compilation, > > should it simply default to the basic Phong shaders or throw an Exception > (likely in a new package with other shader stuff)? > > > If you are talking about a new API, then this is too detailed to figure > out > at this point. We need to step back and figure out how to have things > working with each other before figuring out what to do on invalid input. > > On Fri, Mar 26, 2021 at 5:58 PM Jacky Guo wrote: > > Regarding the shader API, if an error occurs during shader compilation, > should it simply default to the basic Phong shaders or throw an Exception > (likely in a new package with other shader stuff)? > > I'm also very clueless as to how JavaFX links D3D shaders at runtime. Can > someone give me a quick rundown on that? > > On Wed, Mar 17, 2021 at 9:06 PM Dan Howard wrote: > > This is cool. You should like up with Almas Baimagambetov > > > https://github.com/AlmasB > > > On 3/15/2021 10:12 PM, Jacky Guo wrote: > > Regarding uniform parameters (i.e. variables set from Java and called on > the shader): > Since OpenGL's data passing uses locations, you could simply store a > > map of > > uniform parameters (or whatever their JSL equivalent is) to their > locations, and call the required methods and perform the necessary > type-safety checks. > For D3D, you would still use the parameter map, but only for type > > safety. > > > This is assuming I understand how the APIs work and I am not overlooking > something crucial. > > On Mon, Mar 15, 2021 at 4:51 AM Nir Lisker wrote: > > I've thought about this too for a bit. My question would be how to tell > the CPU what data to pass to the shader. Take OpenGL for example: > > writing > > an API to pass a shader and compiling it in runtime is not hard, but > > you > > need the CPU side to feed the shader its parameters. You would need an > > API > > for arbitrary data that the shader has been preconfigured to handle. > > That's > > quite an undertaking. > And what level of flexibility is this going to give? The shader might > require data that the public Java API does not expose, so you are > > limited > > in that regard too. When I wrote the enhancements for the shaders, I > > also > > had to change the Java side, which you won't be able to do. > On the D3D side, the shaders are precompiled, so it's a slightly > > different > > story. > > - Can high school students sign the OCA? (I saw the job title field > > on the form and wasn't so sure I was legally allowed to sign it) > > > Maybe ask on adoption-discuss at openjdk.java.net. > > On Mon, Mar 15, 2021 at 1:50 AM Michael Ennen > wrote: > > I messed around with adding features to JavaFX's "JSLC" (Java Shader > Language Compiler) which currently is only used > at compile-time for JavaFX to convert "JSL" shaders to various > > platform > > dependent shader languages (GLSL, HLSL, etc.). > > In my opinion the ideal way to implement a public shader API would be > > to > > extend JSLC to be able to be used either > at user application compile time, or more ideally, at application > > run-time > > to dynamically generate platform dependent shaders > depending on what platform JavaFX is running on. Then a JavaFX > > application > > developer can write platform independent > shaders in JSL. This would be quite a large undertaking and have a > > high > > burden of maintenance to keep it current with > fast moving shader languages. > > I was able to convert JSLC grammar to antlr4 (from the previous antlr3 > version) in this merged commit (pre-dates the move > to Github): > > > > > https://github.com/openjdk/jfx/commit/52adea7c3635656421db766641416ff28213928f#diff-9c8e9478200eb2c014c196feec63537046751c0d6fe80f90b8506b11abfcc2f7 > > > I then took advantage of a new feature in antlr4 which allows for > extracting embedded actions from the grammar definition file to a > > visitor > > class: > > > > > https://github.com/openjdk/jfx/commit/8889330cc30a47361d47519be2ec662fb5ebe856#diff-9c8e9478200eb2c014c196feec63537046751c0d6fe80f90b8506b11abfcc2f7 > > > I then, in a test repository did some work in fleshing out what it > > would > > take to implement more features for JSLC which you can find (as messy > commits) here: > > https://github.com/brcolow/jslc > > That's just my two cents on how a public shader API should best be > > added > > to > JavaFX, by building on previous work that exists in the source code > repository but is only > used in a very limited and small way - but there is much potential > > there > > and I believe the original author of JSL's intent was to have it be a > public API feature to allow > JavaFX developers to write platform independent shaders in JSL. > > > On Sun, Mar 14, 2021 at 3:32 PM superminerJG > wrote: > > Hi everyone. > > This problem might have been discussed numerous times, and has been > > held > > off for quite a while. > > Although we are starting to see lighting improvements added to > > JavaFX, I > > feel that it's necessary for JavaFX to have some kind of custom > > shader > > pipeline. However, nobody has gotten around to doing it, so I thought > > that > > I could lend a hand. Before I get into things, I would like to know > > how > > and > > what I can contribute. The questions are at the bottom of the email. > > Thanks! > - jgcodes > > Questions: > > - Can high school students sign the OCA? (I saw the job title > > field > > on > > the form and wasn't so sure I was legally allowed to sign it) > - When I implement the shader API, should it use some kind of > > GLSL > > mapping, or should I try to extend JSL to work in 3D? > - Should I have sent this email to another mailing list? > > > -- > Michael Ennen > > > >