From asotona at openjdk.org Tue Apr 1 05:02:32 2025 From: asotona at openjdk.org (Adam Sotona) Date: Tue, 1 Apr 2025 05:02:32 GMT Subject: [code-reflection] RFR: Onnx direct reflection over records as tuples [v7] In-Reply-To: References: Message-ID: > This PR removes `ExplicitOnnxOps.LoopReturn` and reflects directly on Records. > > Unfortunately in some situations `new LoopReturn` cause javac crashing with IOOBE at `jdk.incubator.code/jdk.incubator.code.internal.ReflectMethods$BodyScanner.thisValue(ReflectMethods.java:689)`. > > Problematic use cases are at `WalkTheMazeTest::turnRight` and `WalkTheMazeTest::turnLeftWhileWall`. > While seamlessly working use cases are at `WalkTheMazeTest::walkAroundTheMaze` and `SimpleTest::forLoopAdd`. > > As a temporary workaround any static factory method returning a record is also treated as a tuple constructor. > > Please review. > > Thanks, > Adam Adam Sotona has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains 11 additional commits since the last revision: - recommended changes applied - debuf prints removal - LoopReturn rename to LoopResult - Cleanup after #376 fix. Thank you! - Merge remote-tracking branch 'babylon/code-reflection' into loops-cont5 - main loop in WalkTheMazeTest use tuple with three tensors - fixed model for tuples in loop - fixed test - support for record arguments - support for list and record return types - ... and 1 more: https://git.openjdk.org/babylon/compare/aae7db14...ff5fbbbc ------------- Changes: - all: https://git.openjdk.org/babylon/pull/375/files - new: https://git.openjdk.org/babylon/pull/375/files/a4e9c5dc..ff5fbbbc Webrevs: - full: https://webrevs.openjdk.org/?repo=babylon&pr=375&range=06 - incr: https://webrevs.openjdk.org/?repo=babylon&pr=375&range=05-06 Stats: 477 lines in 16 files changed: 333 ins; 37 del; 107 mod Patch: https://git.openjdk.org/babylon/pull/375.diff Fetch: git fetch https://git.openjdk.org/babylon.git pull/375/head:pull/375 PR: https://git.openjdk.org/babylon/pull/375 From asotona at openjdk.org Tue Apr 1 07:35:00 2025 From: asotona at openjdk.org (Adam Sotona) Date: Tue, 1 Apr 2025 07:35:00 GMT Subject: [code-reflection] RFR: Onnx direct reflection over records as tuples [v8] In-Reply-To: References: Message-ID: > This PR removes `ExplicitOnnxOps.LoopReturn` and reflects directly on Records. > > Unfortunately in some situations `new LoopReturn` cause javac crashing with IOOBE at `jdk.incubator.code/jdk.incubator.code.internal.ReflectMethods$BodyScanner.thisValue(ReflectMethods.java:689)`. > > Problematic use cases are at `WalkTheMazeTest::turnRight` and `WalkTheMazeTest::turnLeftWhileWall`. > While seamlessly working use cases are at `WalkTheMazeTest::walkAroundTheMaze` and `SimpleTest::forLoopAdd`. > > As a temporary workaround any static factory method returning a record is also treated as a tuple constructor. > > Please review. > > Thanks, > Adam Adam Sotona has updated the pull request incrementally with one additional commit since the last revision: WalkTheMazeTest cleanup ------------- Changes: - all: https://git.openjdk.org/babylon/pull/375/files - new: https://git.openjdk.org/babylon/pull/375/files/ff5fbbbc..b3024642 Webrevs: - full: https://webrevs.openjdk.org/?repo=babylon&pr=375&range=07 - incr: https://webrevs.openjdk.org/?repo=babylon&pr=375&range=06-07 Stats: 7 lines in 1 file changed: 0 ins; 6 del; 1 mod Patch: https://git.openjdk.org/babylon/pull/375.diff Fetch: git fetch https://git.openjdk.org/babylon.git pull/375/head:pull/375 PR: https://git.openjdk.org/babylon/pull/375 From asotona at openjdk.org Tue Apr 1 08:33:40 2025 From: asotona at openjdk.org (Adam Sotona) Date: Tue, 1 Apr 2025 08:33:40 GMT Subject: [code-reflection] RFR: Onnx direct reflection over records as tuples [v9] In-Reply-To: References: Message-ID: > This PR removes `ExplicitOnnxOps.LoopReturn` and reflects directly on Records. > > Unfortunately in some situations `new LoopReturn` cause javac crashing with IOOBE at `jdk.incubator.code/jdk.incubator.code.internal.ReflectMethods$BodyScanner.thisValue(ReflectMethods.java:689)`. > > Problematic use cases are at `WalkTheMazeTest::turnRight` and `WalkTheMazeTest::turnLeftWhileWall`. > While seamlessly working use cases are at `WalkTheMazeTest::walkAroundTheMaze` and `SimpleTest::forLoopAdd`. > > As a temporary workaround any static factory method returning a record is also treated as a tuple constructor. > > Please review. > > Thanks, > Adam Adam Sotona has updated the pull request incrementally with one additional commit since the last revision: removed debug output ------------- Changes: - all: https://git.openjdk.org/babylon/pull/375/files - new: https://git.openjdk.org/babylon/pull/375/files/b3024642..1e0bc54d Webrevs: - full: https://webrevs.openjdk.org/?repo=babylon&pr=375&range=08 - incr: https://webrevs.openjdk.org/?repo=babylon&pr=375&range=07-08 Stats: 1 line in 1 file changed: 0 ins; 1 del; 0 mod Patch: https://git.openjdk.org/babylon/pull/375.diff Fetch: git fetch https://git.openjdk.org/babylon.git pull/375/head:pull/375 PR: https://git.openjdk.org/babylon/pull/375 From asotona at openjdk.org Tue Apr 1 11:19:05 2025 From: asotona at openjdk.org (Adam Sotona) Date: Tue, 1 Apr 2025 11:19:05 GMT Subject: [code-reflection] RFR: Onnx direct reflection over records as tuples [v10] In-Reply-To: References: Message-ID: > This PR removes `ExplicitOnnxOps.LoopReturn` and reflects directly on Records. > > Unfortunately in some situations `new LoopReturn` cause javac crashing with IOOBE at `jdk.incubator.code/jdk.incubator.code.internal.ReflectMethods$BodyScanner.thisValue(ReflectMethods.java:689)`. > > Problematic use cases are at `WalkTheMazeTest::turnRight` and `WalkTheMazeTest::turnLeftWhileWall`. > While seamlessly working use cases are at `WalkTheMazeTest::walkAroundTheMaze` and `SimpleTest::forLoopAdd`. > > As a temporary workaround any static factory method returning a record is also treated as a tuple constructor. > > Please review. > > Thanks, > Adam Adam Sotona has updated the pull request incrementally with one additional commit since the last revision: nit change ------------- Changes: - all: https://git.openjdk.org/babylon/pull/375/files - new: https://git.openjdk.org/babylon/pull/375/files/1e0bc54d..767e7d1c Webrevs: - full: https://webrevs.openjdk.org/?repo=babylon&pr=375&range=09 - incr: https://webrevs.openjdk.org/?repo=babylon&pr=375&range=08-09 Stats: 4 lines in 1 file changed: 0 ins; 0 del; 4 mod Patch: https://git.openjdk.org/babylon/pull/375.diff Fetch: git fetch https://git.openjdk.org/babylon.git pull/375/head:pull/375 PR: https://git.openjdk.org/babylon/pull/375 From asotona at openjdk.org Wed Apr 2 07:42:20 2025 From: asotona at openjdk.org (Adam Sotona) Date: Wed, 2 Apr 2025 07:42:20 GMT Subject: [code-reflection] RFR: Onnx direct reflection over records as tuples [v11] In-Reply-To: References: Message-ID: > This PR removes `ExplicitOnnxOps.LoopReturn` and reflects directly on Records. > > Unfortunately in some situations `new LoopReturn` cause javac crashing with IOOBE at `jdk.incubator.code/jdk.incubator.code.internal.ReflectMethods$BodyScanner.thisValue(ReflectMethods.java:689)`. > > Problematic use cases are at `WalkTheMazeTest::turnRight` and `WalkTheMazeTest::turnLeftWhileWall`. > While seamlessly working use cases are at `WalkTheMazeTest::walkAroundTheMaze` and `SimpleTest::forLoopAdd`. > > As a temporary workaround any static factory method returning a record is also treated as a tuple constructor. > > Please review. > > Thanks, > Adam Adam Sotona has updated the pull request incrementally with one additional commit since the last revision: Applied OnnxTransformer patch from @PaulSandoz ------------- Changes: - all: https://git.openjdk.org/babylon/pull/375/files - new: https://git.openjdk.org/babylon/pull/375/files/767e7d1c..f4b441e7 Webrevs: - full: https://webrevs.openjdk.org/?repo=babylon&pr=375&range=10 - incr: https://webrevs.openjdk.org/?repo=babylon&pr=375&range=09-10 Stats: 106 lines in 2 files changed: 64 ins; 19 del; 23 mod Patch: https://git.openjdk.org/babylon/pull/375.diff Fetch: git fetch https://git.openjdk.org/babylon.git pull/375/head:pull/375 PR: https://git.openjdk.org/babylon/pull/375 From asotona at openjdk.org Wed Apr 2 08:20:13 2025 From: asotona at openjdk.org (Adam Sotona) Date: Wed, 2 Apr 2025 08:20:13 GMT Subject: git: openjdk/babylon: code-reflection: Onnx direct reflection over records as tuples Message-ID: Changeset: bd0569ee Branch: code-reflection Author: Adam Sotona Date: 2025-04-02 08:18:34 +0000 URL: https://git.openjdk.org/babylon/commit/bd0569ee1a32d3394a82bd36a29b67e7c2069b71 Onnx direct reflection over records as tuples ! cr-examples/onnx/src/main/java/oracle/code/onnx/ExplicitOnnxOperators.java ! cr-examples/onnx/src/main/java/oracle/code/onnx/OnnxProtoBuilder.java ! cr-examples/onnx/src/main/java/oracle/code/onnx/OnnxRuntime.java ! cr-examples/onnx/src/main/java/oracle/code/onnx/compiler/OnnxTransformer.java ! cr-examples/onnx/src/main/java/oracle/code/onnx/ir/ExplicitOnnxOps.java ! cr-examples/onnx/src/test/java/oracle/code/onnx/SimpleTest.java ! cr-examples/onnx/src/test/java/oracle/code/onnx/WalkTheMazeTest.java From asotona at openjdk.org Wed Apr 2 08:21:21 2025 From: asotona at openjdk.org (Adam Sotona) Date: Wed, 2 Apr 2025 08:21:21 GMT Subject: [code-reflection] RFR: Onnx direct reflection over records as tuples [v11] In-Reply-To: References: Message-ID: On Wed, 2 Apr 2025 07:42:20 GMT, Adam Sotona wrote: >> This PR removes `ExplicitOnnxOps.LoopReturn` and reflects directly on Records. >> >> Unfortunately in some situations `new LoopReturn` cause javac crashing with IOOBE at `jdk.incubator.code/jdk.incubator.code.internal.ReflectMethods$BodyScanner.thisValue(ReflectMethods.java:689)`. >> >> Problematic use cases are at `WalkTheMazeTest::turnRight` and `WalkTheMazeTest::turnLeftWhileWall`. >> While seamlessly working use cases are at `WalkTheMazeTest::walkAroundTheMaze` and `SimpleTest::forLoopAdd`. >> >> As a temporary workaround any static factory method returning a record is also treated as a tuple constructor. >> >> Please review. >> >> Thanks, >> Adam > > Adam Sotona has updated the pull request incrementally with one additional commit since the last revision: > > Applied OnnxTransformer patch from @PaulSandoz Thanks for the reviews! ------------- PR Comment: https://git.openjdk.org/babylon/pull/375#issuecomment-2771767386 From asotona at openjdk.org Wed Apr 2 08:21:22 2025 From: asotona at openjdk.org (Adam Sotona) Date: Wed, 2 Apr 2025 08:21:22 GMT Subject: [code-reflection] Integrated: Onnx direct reflection over records as tuples In-Reply-To: References: Message-ID: On Mon, 31 Mar 2025 11:05:56 GMT, Adam Sotona wrote: > This PR removes `ExplicitOnnxOps.LoopReturn` and reflects directly on Records. > > Unfortunately in some situations `new LoopReturn` cause javac crashing with IOOBE at `jdk.incubator.code/jdk.incubator.code.internal.ReflectMethods$BodyScanner.thisValue(ReflectMethods.java:689)`. > > Problematic use cases are at `WalkTheMazeTest::turnRight` and `WalkTheMazeTest::turnLeftWhileWall`. > While seamlessly working use cases are at `WalkTheMazeTest::walkAroundTheMaze` and `SimpleTest::forLoopAdd`. > > As a temporary workaround any static factory method returning a record is also treated as a tuple constructor. > > Please review. > > Thanks, > Adam This pull request has now been integrated. Changeset: bd0569ee Author: Adam Sotona URL: https://git.openjdk.org/babylon/commit/bd0569ee1a32d3394a82bd36a29b67e7c2069b71 Stats: 481 lines in 7 files changed: 223 ins; 168 del; 90 mod Onnx direct reflection over records as tuples ------------- PR: https://git.openjdk.org/babylon/pull/375 From gfrost at openjdk.org Wed Apr 2 14:19:26 2025 From: gfrost at openjdk.org (Gary Frost) Date: Wed, 2 Apr 2025 14:19:26 GMT Subject: git: openjdk/babylon: code-reflection: Hat merge cuda ptx ffi backends Message-ID: <3f8f2555-9249-4e0a-8e28-2490d35b4288@openjdk.org> Changeset: 225e2b75 Branch: code-reflection Author: Gary Frost Date: 2025-04-02 14:15:57 +0000 URL: https://git.openjdk.org/babylon/commit/225e2b7532722d51a0cb9ed01c7f2275d08a5260 Hat merge cuda ptx ffi backends ! hat/backends/ffi/cuda/CMakeLists.txt ! hat/backends/ffi/cuda/cpp/cuda_backend.cpp + hat/backends/ffi/cuda/cpp/cuda_backend_buffer.cpp + hat/backends/ffi/cuda/cpp/cuda_backend_kernel.cpp + hat/backends/ffi/cuda/cpp/cuda_backend_module.cpp + hat/backends/ffi/cuda/cpp/cuda_backend_queue.cpp + hat/backends/ffi/cuda/cpp/squares.cpp ! hat/backends/ffi/cuda/include/cuda_backend.h ! hat/backends/ffi/cuda/src/main/java/hat/backend/ffi/CudaBackend.java ! hat/backends/ffi/opencl/src/main/java/hat/backend/ffi/OpenCLBackend.java ! hat/backends/ffi/shared/include/shared.h + hat/backends/ffi/shared/src/main/java/hat/backend/ffi/Config.java ! hat/backends/ffi/shared/src/main/java/hat/backend/ffi/FFILib.java ! hat/examples/experiments/src/main/java/experiments/Mesh.java ! hat/examples/experiments/src/main/java/experiments/MinBufferTest.java ! hat/examples/nbody/src/main/java/nbody/opencl/OpenCLNBodyGLWindow.java ! hat/examples/squares/src/main/java/squares/Main.java ! hat/hat/run.java From gfrost at openjdk.org Wed Apr 2 14:20:27 2025 From: gfrost at openjdk.org (Gary Frost) Date: Wed, 2 Apr 2025 14:20:27 GMT Subject: [code-reflection] Integrated: Hat merge cuda ptx ffi backends Message-ID: The merge between ptx and cuda backends continues. We still have a bug using cuda streams (which we need for minimizing buffer copies). Added a pure C++ test (which launches squares directly from C++) it replicates the error we see with the squares example. Lots of small tidy ups in the code here, but no real fixes. Expect more cuda/ptx merge updates ------------- Commit messages: - whitespace - slowly tracking cuda issue. kernels can be launched provided we avoid using streams but we need streams to minimize buffer copies - synced with jetson/cuda - cuda backend compiles and recieves ptx. Failing to execute kernel Changes: https://git.openjdk.org/babylon/pull/377/files Webrev: https://webrevs.openjdk.org/?repo=babylon&pr=377&range=00 Stats: 1607 lines in 18 files changed: 1161 ins; 399 del; 47 mod Patch: https://git.openjdk.org/babylon/pull/377.diff Fetch: git fetch https://git.openjdk.org/babylon.git pull/377/head:pull/377 PR: https://git.openjdk.org/babylon/pull/377 From gfrost at openjdk.org Wed Apr 2 14:20:28 2025 From: gfrost at openjdk.org (Gary Frost) Date: Wed, 2 Apr 2025 14:20:28 GMT Subject: [code-reflection] Integrated: Hat merge cuda ptx ffi backends In-Reply-To: References: Message-ID: On Wed, 2 Apr 2025 14:07:34 GMT, Gary Frost wrote: > The merge between ptx and cuda backends continues. > > We still have a bug using cuda streams (which we need for minimizing buffer copies). > > Added a pure C++ test (which launches squares directly from C++) it replicates the error we see with the squares example. > > Lots of small tidy ups in the code here, but no real fixes. > > Expect more cuda/ptx merge updates This pull request has now been integrated. Changeset: 225e2b75 Author: Gary Frost URL: https://git.openjdk.org/babylon/commit/225e2b7532722d51a0cb9ed01c7f2275d08a5260 Stats: 1607 lines in 18 files changed: 1161 ins; 399 del; 47 mod Hat merge cuda ptx ffi backends ------------- PR: https://git.openjdk.org/babylon/pull/377 From duke at openjdk.org Thu Apr 3 06:54:13 2025 From: duke at openjdk.org (f) Date: Thu, 3 Apr 2025 06:54:13 GMT Subject: [code-reflection] RFR: Support storing the code that builds the code model [v21] In-Reply-To: References: Message-ID: On Thu, 27 Mar 2025 08:45:12 GMT, Mourad Abbay wrote: >> In this PR we allow the user to decide how to store the code model. The first option is `TEXT`, if this is selected, we store the textual representation of the code model. The second option is `CODE_BUILDR`, if this is selected, we store the code that builds the code model. All work done here is around the second option, because the first option is already supported. > > Mourad Abbay has updated the pull request incrementally with one additional commit since the last revision: > > Changes in code due to making opMethod signature uniform Has Babylon jdk supported llama, deepseak and yolo inference for at least one platform? How about to fully support one platform first? ie. one of ptx, spirv, cuda, hip, etc on linux. ------------- PR Comment: https://git.openjdk.org/babylon/pull/305#issuecomment-2774346940 From zjx001202 at gmail.com Thu Apr 3 15:55:31 2025 From: zjx001202 at gmail.com (Glavo) Date: Thu, 3 Apr 2025 23:55:31 +0800 Subject: Unable to build the code-reflection branch using JDK 24 as boot jdk Message-ID: Hi, I wanted to try using Babylon/HAT, so I built the JDK based on the latest commit (225e2b7) of the code-reflection branch. Unfortunately, if I use JDK 24 as the boot jdk, it quickly fails to build with this error: * For target buildtools_depend__the.COMPILE_DEPEND_batch: > error: cannot access module-info > bad class file: /modules/java.se/module-info.class > bad requires flag: ACC_TRANSITIVE (0x0020 > Please remove or make sure it appears in the correct subdirectory of > the classpath. > 1 error > * For target buildtools_jdk_tools_classes__the.BUILD_TOOLS_JDK_batch: > error: cannot access module-info > bad class file: /modules/java.se/module-info.class > bad requires flag: ACC_TRANSITIVE (0x0020 > Please remove or make sure it appears in the correct subdirectory of > the classpath. > 1 error But I switched to JDK 23.0.2 as the boot jdk and everything worked fine. Glavo -------------- next part -------------- An HTML attachment was scrubbed... URL: From mabbay at openjdk.org Thu Apr 3 18:21:32 2025 From: mabbay at openjdk.org (Mourad Abbay) Date: Thu, 3 Apr 2025 18:21:32 GMT Subject: git: openjdk/babylon: code-reflection: Represents literals of type byte/short as permitted in the Java language Message-ID: <25ecb57f-eb6a-4499-8bd4-6144b4bade42@openjdk.org> Changeset: fd979023 Branch: code-reflection Author: Mourad Abbay Date: 2025-04-03 18:19:45 +0000 URL: https://git.openjdk.org/babylon/commit/fd9790233138b1a9d622440287fccd56cd7e9f1b Represents literals of type byte/short as permitted in the Java language Reviewed-by: mcimadamore, psandoz ! src/jdk.incubator.code/share/classes/jdk/incubator/code/internal/ReflectMethods.java ! test/langtools/tools/javac/reflect/PatternsTest.java ! test/langtools/tools/javac/reflect/UnaryopTest.java From mabbay at openjdk.org Thu Apr 3 18:23:08 2025 From: mabbay at openjdk.org (Mourad Abbay) Date: Thu, 3 Apr 2025 18:23:08 GMT Subject: [code-reflection] Integrated: Represents literals of type byte/short as permitted in the Java language In-Reply-To: References: Message-ID: <2BhTsUGdcmCCCLwvMzKF7iec1tdfBmUgwgeYi6hUA_Q=.926ed3bd-be59-47cc-9290-d17ed467574b@github.com> On Wed, 12 Mar 2025 17:44:40 GMT, Mourad Abbay wrote: > Before this PR, we had literals of type byte/short in the code model, but that's not permitted by the Java language. In Java, a literals of type byte/short is a literal of type int + conversion to the type byte/short. This pull request has now been integrated. Changeset: fd979023 Author: Mourad Abbay URL: https://git.openjdk.org/babylon/commit/fd9790233138b1a9d622440287fccd56cd7e9f1b Stats: 127 lines in 3 files changed: 118 ins; 5 del; 4 mod Represents literals of type byte/short as permitted in the Java language Reviewed-by: mcimadamore, psandoz ------------- PR: https://git.openjdk.org/babylon/pull/351 From paul.sandoz at oracle.com Thu Apr 3 18:23:00 2025 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Thu, 3 Apr 2025 18:23:00 +0000 Subject: Unable to build the code-reflection branch using JDK 24 as boot jdk In-Reply-To: References: Message-ID: <5D76107D-786D-418F-A54B-790D05314D58@oracle.com> Hi Glavo, To use 24 as the boot JDK we need merge code-reflection with master, which we want to do anyway and will do so after https://github.com/openjdk/babylon/pull/305 is integrated (hopefully in the next few days). Paul. On Apr 3, 2025, at 8:55?AM, Glavo wrote: Hi, I wanted to try using Babylon/HAT, so I built the JDK based on the latest commit (225e2b7) of the code-reflection branch. Unfortunately, if I use JDK 24 as the boot jdk, it quickly fails to build with this error: * For target buildtools_depend__the.COMPILE_DEPEND_batch: error: cannot access module-info bad class file: /modules/java.se/module-info.class bad requires flag: ACC_TRANSITIVE (0x0020 Please remove or make sure it appears in the correct subdirectory of the classpath. 1 error * For target buildtools_jdk_tools_classes__the.BUILD_TOOLS_JDK_batch: error: cannot access module-info bad class file: /modules/java.se/module-info.class bad requires flag: ACC_TRANSITIVE (0x0020 Please remove or make sure it appears in the correct subdirectory of the classpath. 1 error But I switched to JDK 23.0.2 as the boot jdk and everything worked fine. Glavo -------------- next part -------------- An HTML attachment was scrubbed... URL: From mabbay at openjdk.org Thu Apr 3 18:27:49 2025 From: mabbay at openjdk.org (Mourad Abbay) Date: Thu, 3 Apr 2025 18:27:49 GMT Subject: [code-reflection] RFR: Support storing the code that builds the code model [v22] In-Reply-To: References: Message-ID: > In this PR we allow the user to decide how to store the code model. The first option is `TEXT`, if this is selected, we store the textual representation of the code model. The second option is `CODE_BUILDR`, if this is selected, we store the code that builds the code model. All work done here is around the second option, because the first option is already supported. Mourad Abbay has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains 25 additional commits since the last revision: - Merge branch 'code-reflection' into code-model-storage-option - Make return type of opMethod the same for all storage options - Changes in code due to making opMethod signature uniform - Make opMethod descriptor consistent for both storage options. - Don't rely on the assumption of 1d array - Merge branch 'code-reflection' into code-model-storage-option - Load OpFactory and TypeElementFactory before invocation of opMethod only if the opMethod has there params. - Fix the remaining compiler tests failures - Fix some of the test failures (3 remains) - Fix the remaining test failures of SwitchExpressionTest2 - ... and 15 more: https://git.openjdk.org/babylon/compare/3d8aac9e...3e3095cb ------------- Changes: - all: https://git.openjdk.org/babylon/pull/305/files - new: https://git.openjdk.org/babylon/pull/305/files/bf96a44a..3e3095cb Webrevs: - full: https://webrevs.openjdk.org/?repo=babylon&pr=305&range=21 - incr: https://webrevs.openjdk.org/?repo=babylon&pr=305&range=20-21 Stats: 12377 lines in 214 files changed: 7994 ins; 3410 del; 973 mod Patch: https://git.openjdk.org/babylon/pull/305.diff Fetch: git fetch https://git.openjdk.org/babylon.git pull/305/head:pull/305 PR: https://git.openjdk.org/babylon/pull/305 From gfrost at openjdk.org Fri Apr 4 13:17:29 2025 From: gfrost at openjdk.org (Gary Frost) Date: Fri, 4 Apr 2025 13:17:29 GMT Subject: [code-reflection] Integrated: Synchronizing all the ffi backend native code Message-ID: <6Wf_r8M4gy1Ip-9vZy6B0OGwyI3SnyOKLsxmSj5s0R0=.32b6c132-6cc7-48cd-93b2-99222b199368@github.com> The ongoing refactor of ffi backends continues. Moved common ffi code to ffi-shared (for all backends) Also changed the java backend so that they all used the same mh references. ------------- Commit messages: - Synchronizing all the ffi backend native code Changes: https://git.openjdk.org/babylon/pull/379/files Webrev: https://webrevs.openjdk.org/?repo=babylon&pr=379&range=00 Stats: 890 lines in 24 files changed: 371 ins; 409 del; 110 mod Patch: https://git.openjdk.org/babylon/pull/379.diff Fetch: git fetch https://git.openjdk.org/babylon.git pull/379/head:pull/379 PR: https://git.openjdk.org/babylon/pull/379 From gfrost at openjdk.org Fri Apr 4 13:17:30 2025 From: gfrost at openjdk.org (Gary Frost) Date: Fri, 4 Apr 2025 13:17:30 GMT Subject: [code-reflection] Integrated: Synchronizing all the ffi backend native code In-Reply-To: <6Wf_r8M4gy1Ip-9vZy6B0OGwyI3SnyOKLsxmSj5s0R0=.32b6c132-6cc7-48cd-93b2-99222b199368@github.com> References: <6Wf_r8M4gy1Ip-9vZy6B0OGwyI3SnyOKLsxmSj5s0R0=.32b6c132-6cc7-48cd-93b2-99222b199368@github.com> Message-ID: <67hUfs6_6Ckd9vufpPCtN4Slz6DvEvZMLQ_ZB2IQoFA=.0f953cde-e35f-43fe-9b2b-eb725abd34e9@github.com> On Fri, 4 Apr 2025 13:12:16 GMT, Gary Frost wrote: > The ongoing refactor of ffi backends continues. > > Moved common ffi code to ffi-shared (for all backends) > > Also changed the java backend so that they all used the same mh references. This pull request has now been integrated. Changeset: de4cceca Author: Gary Frost URL: https://git.openjdk.org/babylon/commit/de4cceca27c1ca176064917e2f8f4d9911e9cc90 Stats: 890 lines in 24 files changed: 371 ins; 409 del; 110 mod Synchronizing all the ffi backend native code ------------- PR: https://git.openjdk.org/babylon/pull/379 From gfrost at openjdk.org Fri Apr 4 13:18:38 2025 From: gfrost at openjdk.org (Gary Frost) Date: Fri, 4 Apr 2025 13:18:38 GMT Subject: git: openjdk/babylon: code-reflection: Synchronizing all the ffi backend native code Message-ID: Changeset: de4cceca Branch: code-reflection Author: Gary Frost Date: 2025-04-04 13:14:02 +0000 URL: https://git.openjdk.org/babylon/commit/de4cceca27c1ca176064917e2f8f4d9911e9cc90 Synchronizing all the ffi backend native code ! hat/backends/ffi/cuda/cpp/cuda_backend.cpp ! hat/backends/ffi/cuda/cpp/squares.cpp ! hat/backends/ffi/cuda/include/cuda_backend.h ! hat/backends/ffi/cuda/src/main/java/hat/backend/ffi/CudaBackend.java ! hat/backends/ffi/mock/cpp/mock_backend.cpp ! hat/backends/ffi/mock/src/main/java/hat/backend/ffi/MockBackend.java ! hat/backends/ffi/opencl/CMakeLists.txt ! hat/backends/ffi/opencl/cpp/opencl_backend.cpp ! hat/backends/ffi/opencl/cpp/opencl_backend_info.cpp ! hat/backends/ffi/opencl/cpp/opencl_backend_program.cpp + hat/backends/ffi/opencl/cpp/squares.cpp ! hat/backends/ffi/opencl/include/opencl_backend.h ! hat/backends/ffi/opencl/src/main/java/hat/backend/ffi/OpenCLBackend.java - hat/backends/ffi/opencl/src/main/java/hat/backend/ffi/OpenCLConfig.java ! hat/backends/ffi/ptx/src/main/java/hat/backend/ffi/PTXBackend.java ! hat/backends/ffi/shared/cpp/shared.cpp ! hat/backends/ffi/shared/include/shared.h ! hat/backends/ffi/shared/src/main/java/hat/backend/ffi/C99FFIBackend.java ! hat/backends/ffi/shared/src/main/java/hat/backend/ffi/Config.java ! hat/backends/ffi/shared/src/main/java/hat/backend/ffi/FFIBackend.java ! hat/backends/ffi/shared/src/main/java/hat/backend/ffi/FFIBackendDriver.java ! hat/backends/ffi/shared/src/main/java/hat/backend/ffi/FFILib.java ! hat/backends/ffi/spirv/cpp/spirv_backend.cpp ! hat/intellij/experiments.iml From mabbay at openjdk.org Fri Apr 4 15:05:14 2025 From: mabbay at openjdk.org (Mourad Abbay) Date: Fri, 4 Apr 2025 15:05:14 GMT Subject: [code-reflection] RFR: Support storing the code that builds the code model [v18] In-Reply-To: References: Message-ID: On Wed, 26 Mar 2025 09:47:25 GMT, Maurizio Cimadamore wrote: >> (of course we'd only do the above translation for ops that have a non-void result type) > > We can follow up on this improved translation at a later point. Now I see. The current approach is inserting vars only when required. You're saying we can insert Vars for all operation (which is a simpler approach) ? ------------- PR Review Comment: https://git.openjdk.org/babylon/pull/305#discussion_r2028991409 From gfrost at openjdk.org Fri Apr 4 15:50:19 2025 From: gfrost at openjdk.org (Gary Frost) Date: Fri, 4 Apr 2025 15:50:19 GMT Subject: [code-reflection] Integrated: Finally found and fixed cuda side CUstream bug Message-ID: Finally tracked down (and fixed) CUstream CUDA backend bug, which was halgting ability to minimize buffer copies. This checkin also includes a pure native raw opencl-squares and cuda-squares example code (no Java at all but using the ffi backend hooks to execute OpenCL/CUDA from native code) ------------- Commit messages: - Finally found and fixed cuda side CUstream bug Changes: https://git.openjdk.org/babylon/pull/380/files Webrev: https://webrevs.openjdk.org/?repo=babylon&pr=380&range=00 Stats: 79 lines in 8 files changed: 22 ins; 0 del; 57 mod Patch: https://git.openjdk.org/babylon/pull/380.diff Fetch: git fetch https://git.openjdk.org/babylon.git pull/380/head:pull/380 PR: https://git.openjdk.org/babylon/pull/380 From gfrost at openjdk.org Fri Apr 4 15:50:19 2025 From: gfrost at openjdk.org (Gary Frost) Date: Fri, 4 Apr 2025 15:50:19 GMT Subject: [code-reflection] Integrated: Finally found and fixed cuda side CUstream bug In-Reply-To: References: Message-ID: On Fri, 4 Apr 2025 15:45:30 GMT, Gary Frost wrote: > Finally tracked down (and fixed) CUstream CUDA backend bug, which was halgting ability to minimize buffer copies. > > This checkin also includes a pure native raw opencl-squares and cuda-squares example code (no Java at all but using the ffi backend hooks to execute OpenCL/CUDA from native code) This pull request has now been integrated. Changeset: e73510d8 Author: Gary Frost URL: https://git.openjdk.org/babylon/commit/e73510d8a2bf7ef23ef9dfc867088df0f6fad0cc Stats: 79 lines in 8 files changed: 22 ins; 0 del; 57 mod Finally found and fixed cuda side CUstream bug ------------- PR: https://git.openjdk.org/babylon/pull/380 From gfrost at openjdk.org Fri Apr 4 15:51:24 2025 From: gfrost at openjdk.org (Gary Frost) Date: Fri, 4 Apr 2025 15:51:24 GMT Subject: git: openjdk/babylon: code-reflection: Finally found and fixed cuda side CUstream bug Message-ID: <88dfbff0-27e6-4637-aec8-78a7beee8426@openjdk.org> Changeset: e73510d8 Branch: code-reflection Author: Gary Frost Date: 2025-04-04 15:47:29 +0000 URL: https://git.openjdk.org/babylon/commit/e73510d8a2bf7ef23ef9dfc867088df0f6fad0cc Finally found and fixed cuda side CUstream bug ! hat/backends/ffi/cuda/cpp/cuda_backend.cpp ! hat/backends/ffi/cuda/cpp/cuda_backend_buffer.cpp ! hat/backends/ffi/cuda/cpp/cuda_backend_kernel.cpp ! hat/backends/ffi/cuda/cpp/cuda_backend_module.cpp ! hat/backends/ffi/cuda/cpp/cuda_backend_queue.cpp ! hat/backends/ffi/cuda/cpp/squares.cpp ! hat/backends/ffi/cuda/include/cuda_backend.h ! hat/backends/ffi/shared/cpp/shared.cpp From mabbay at openjdk.org Fri Apr 4 17:04:31 2025 From: mabbay at openjdk.org (Mourad Abbay) Date: Fri, 4 Apr 2025 17:04:31 GMT Subject: git: openjdk/babylon: code-reflection: Support storing the code that builds the code model Message-ID: Changeset: fbc7e37b Branch: code-reflection Author: Mourad Abbay Date: 2025-04-04 17:03:56 +0000 URL: https://git.openjdk.org/babylon/commit/fbc7e37b3036e0e98edf3077227b3adea5ce466f Support storing the code that builds the code model Reviewed-by: mcimadamore ! src/java.base/share/classes/java/lang/invoke/InnerClassLambdaMetafactory.java ! src/jdk.incubator.code/share/classes/jdk/incubator/code/Op.java ! src/jdk.incubator.code/share/classes/jdk/incubator/code/bytecode/BytecodeGenerator.java + src/jdk.incubator.code/share/classes/jdk/incubator/code/internal/CodeModelToAST.java ! src/jdk.incubator.code/share/classes/jdk/incubator/code/internal/CodeReflectionSymbols.java ! src/jdk.incubator.code/share/classes/jdk/incubator/code/internal/ReflectMethods.java ! src/jdk.incubator.code/share/classes/jdk/incubator/code/writer/OpBuilder.java + test/jdk/java/lang/reflect/code/TestAddVarsWhenNecessary.java From mabbay at openjdk.org Fri Apr 4 17:07:06 2025 From: mabbay at openjdk.org (Mourad Abbay) Date: Fri, 4 Apr 2025 17:07:06 GMT Subject: [code-reflection] Integrated: Support storing the code that builds the code model In-Reply-To: References: Message-ID: <6UrP1qVUaQDhkFUQwQ-74d_NSMsCNSO-KcGAGUVLAV4=.b8a7deb4-32e5-47eb-9261-addc27c503e4@github.com> On Wed, 22 Jan 2025 12:01:19 GMT, Mourad Abbay wrote: > In this PR we allow the user to decide how to store the code model. The first option is `TEXT`, if this is selected, we store the textual representation of the code model. The second option is `CODE_BUILDR`, if this is selected, we store the code that builds the code model. All work done here is around the second option, because the first option is already supported. This pull request has now been integrated. Changeset: fbc7e37b Author: Mourad Abbay URL: https://git.openjdk.org/babylon/commit/fbc7e37b3036e0e98edf3077227b3adea5ce466f Stats: 507 lines in 8 files changed: 480 ins; 6 del; 21 mod Support storing the code that builds the code model Reviewed-by: mcimadamore ------------- PR: https://git.openjdk.org/babylon/pull/305 From gfrost at openjdk.org Sat Apr 5 11:06:10 2025 From: gfrost at openjdk.org (Gary Frost) Date: Sat, 5 Apr 2025 11:06:10 GMT Subject: git: openjdk/babylon: code-reflection: switch all uses of runtime API cudaXXX for device API cuXXX - to =?UTF-8?B?dHJhY+KApg==?= Message-ID: Changeset: c96ce3b3 Branch: code-reflection Author: Gary Frost Date: 2025-04-05 11:04:39 +0000 URL: https://git.openjdk.org/babylon/commit/c96ce3b38593d27d3fe3ffbd116692cb15207b38 switch all uses of runtime API cudaXXX for device API cuXXX - to trac? ! hat/backends/ffi/cuda/cpp/cuda_backend.cpp ! hat/backends/ffi/cuda/cpp/cuda_backend_buffer.cpp ! hat/backends/ffi/cuda/cpp/cuda_backend_kernel.cpp ! hat/backends/ffi/cuda/cpp/cuda_backend_module.cpp ! hat/backends/ffi/cuda/cpp/cuda_backend_queue.cpp ! hat/backends/ffi/cuda/include/cuda_backend.h From gfrost at openjdk.org Sat Apr 5 11:07:35 2025 From: gfrost at openjdk.org (Gary Frost) Date: Sat, 5 Apr 2025 11:07:35 GMT Subject: [code-reflection] Integrated: switch all uses of runtime API cudaXXX for device API cuXXX - to =?UTF-8?B?dHJhY+KApg==?= Message-ID: Switch all uses of CUDA runtime API (cudaXXX) for device API (cuXXX) - to track bug uncovered by heal example which seems to access thread local data (native code) and thus get the incrroect stream/context. Bug only shows up in heal example. This checkin cleans up API usage and uses a common mechanism to trace cuXX calls ------------- Commit messages: - switch all uses of runtime API cudaXXX for device API cuXXX - to track bug uncovered by heal example Changes: https://git.openjdk.org/babylon/pull/381/files Webrev: https://webrevs.openjdk.org/?repo=babylon&pr=381&range=00 Stats: 215 lines in 6 files changed: 32 ins; 108 del; 75 mod Patch: https://git.openjdk.org/babylon/pull/381.diff Fetch: git fetch https://git.openjdk.org/babylon.git pull/381/head:pull/381 PR: https://git.openjdk.org/babylon/pull/381 From gfrost at openjdk.org Sat Apr 5 11:07:35 2025 From: gfrost at openjdk.org (Gary Frost) Date: Sat, 5 Apr 2025 11:07:35 GMT Subject: [code-reflection] Integrated: switch all uses of runtime API cudaXXX for device API cuXXX - to =?UTF-8?B?dHJhY+KApg==?= In-Reply-To: References: Message-ID: On Sat, 5 Apr 2025 11:02:50 GMT, Gary Frost wrote: > Switch all uses of CUDA runtime API (cudaXXX) for device API (cuXXX) - to track bug uncovered by heal example which seems to access thread local data (native code) and thus get the incrroect stream/context. > > Bug only shows up in heal example. > > This checkin cleans up API usage and uses a common mechanism to trace cuXX calls This pull request has now been integrated. Changeset: c96ce3b3 Author: Gary Frost URL: https://git.openjdk.org/babylon/commit/c96ce3b38593d27d3fe3ffbd116692cb15207b38 Stats: 215 lines in 6 files changed: 32 ins; 108 del; 75 mod switch all uses of runtime API cudaXXX for device API cuXXX - to trac? ------------- PR: https://git.openjdk.org/babylon/pull/381 From gfrost at openjdk.org Sat Apr 5 16:28:30 2025 From: gfrost at openjdk.org (Gary Frost) Date: Sat, 5 Apr 2025 16:28:30 GMT Subject: git: openjdk/babylon: code-reflection: removed jextracted opencl dependency on life example Message-ID: <24171704-bc7a-48dc-8e81-9466de45b68a@openjdk.org> Changeset: 9dba3048 Branch: code-reflection Author: Gary Frost Date: 2025-04-05 16:27:01 +0000 URL: https://git.openjdk.org/babylon/commit/9dba3048c1ee6f1c107479a83fc8340153b97435 removed jextracted opencl dependency on life example ! hat/examples/life/pom.xml ! hat/examples/life/src/main/java/life/Main.java ! hat/examples/life/src/main/java/life/Viewer.java ! hat/examples/pom.xml ! hat/hat/bld.java ! hat/hat/run.java From gfrost at openjdk.org Sat Apr 5 16:30:34 2025 From: gfrost at openjdk.org (Gary Frost) Date: Sat, 5 Apr 2025 16:30:34 GMT Subject: [code-reflection] Integrated: removed jextracted opencl dependency on life example Message-ID: Removed the jextracted opencl dependencies on life example. We used the game of life example to prove the algorithm for minimizing buffer transfers using jextracted opencl Now we seem to have a stable version of the algorithm in HAT itself we can now remove the jexctracted opencl dependency. This also means we can build and run on cuda and test the algorithm there. ------------- Commit messages: - removed jextracted opencl dependency on life example Changes: https://git.openjdk.org/babylon/pull/382/files Webrev: https://webrevs.openjdk.org/?repo=babylon&pr=382&range=00 Stats: 96 lines in 6 files changed: 4 ins; 79 del; 13 mod Patch: https://git.openjdk.org/babylon/pull/382.diff Fetch: git fetch https://git.openjdk.org/babylon.git pull/382/head:pull/382 PR: https://git.openjdk.org/babylon/pull/382 From gfrost at openjdk.org Sat Apr 5 16:30:34 2025 From: gfrost at openjdk.org (Gary Frost) Date: Sat, 5 Apr 2025 16:30:34 GMT Subject: [code-reflection] Integrated: removed jextracted opencl dependency on life example In-Reply-To: References: Message-ID: On Sat, 5 Apr 2025 16:25:28 GMT, Gary Frost wrote: > Removed the jextracted opencl dependencies on life example. > > We used the game of life example to prove the algorithm for minimizing buffer transfers using jextracted opencl > > Now we seem to have a stable version of the algorithm in HAT itself we can now remove the jexctracted opencl dependency. > > This also means we can build and run on cuda and test the algorithm there. This pull request has now been integrated. Changeset: 9dba3048 Author: Gary Frost URL: https://git.openjdk.org/babylon/commit/9dba3048c1ee6f1c107479a83fc8340153b97435 Stats: 96 lines in 6 files changed: 4 ins; 79 del; 13 mod removed jextracted opencl dependency on life example ------------- PR: https://git.openjdk.org/babylon/pull/382 From gfrost at openjdk.org Sun Apr 6 11:37:01 2025 From: gfrost at openjdk.org (Gary Frost) Date: Sun, 6 Apr 2025 11:37:01 GMT Subject: git: openjdk/babylon: code-reflection: Moved PTX backend functionality to CUDA backend Message-ID: Changeset: d8a08236 Branch: code-reflection Author: Gary Frost Date: 2025-04-06 11:35:17 +0000 URL: https://git.openjdk.org/babylon/commit/d8a082365bb65eed025a3c9518918369af454592 Moved PTX backend functionality to CUDA backend ! hat/backends/ffi/cuda/cpp/cuda_backend.cpp ! hat/backends/ffi/cuda/include/cuda_backend.h ! hat/backends/ffi/cuda/src/main/java/hat/backend/ffi/CudaBackend.java + hat/backends/ffi/cuda/src/main/java/hat/backend/ffi/CudaC99HATKernelBuilder.java + hat/backends/ffi/cuda/src/main/java/hat/backend/ffi/PTXHATKernelBuilder.java = hat/backends/ffi/cuda/src/main/java/hat/backend/ffi/PTXPtrOp.java = hat/backends/ffi/cuda/src/main/java/hat/backend/ffi/PTXRegister.java ! hat/backends/ffi/pom.xml ! hat/backends/ffi/ptx/include/ptx_backend.h ! hat/backends/ffi/ptx/src/main/java/hat/backend/ffi/PTXBackend.java ! hat/backends/ffi/shared/cpp/shared.cpp ! hat/backends/ffi/shared/include/shared.h ! hat/backends/ffi/shared/src/main/java/hat/backend/ffi/Config.java ! hat/hat/bld.java From gfrost at openjdk.org Sun Apr 6 11:38:23 2025 From: gfrost at openjdk.org (Gary Frost) Date: Sun, 6 Apr 2025 11:38:23 GMT Subject: [code-reflection] Integrated: Moved PTX backend functionality to CUDA backend Message-ID: Moved the required PTX generation code into the more general CUDA backend. We can select PTX generation either by sending a config string including 'PTX' to the backend (default will be Cuda C99) We can also add PTX to HAT env var at launch HAT=INFO,PTX java @hat/run ffi-cuda squares vs HAT=INFO java @hat/run ffi-cude squares ------------- Commit messages: - Moved PTX backend functionality to CUDA backend Changes: https://git.openjdk.org/babylon/pull/384/files Webrev: https://webrevs.openjdk.org/?repo=babylon&pr=384&range=00 Stats: 1609 lines in 14 files changed: 1507 ins; 49 del; 53 mod Patch: https://git.openjdk.org/babylon/pull/384.diff Fetch: git fetch https://git.openjdk.org/babylon.git pull/384/head:pull/384 PR: https://git.openjdk.org/babylon/pull/384 From gfrost at openjdk.org Sun Apr 6 11:38:25 2025 From: gfrost at openjdk.org (Gary Frost) Date: Sun, 6 Apr 2025 11:38:25 GMT Subject: [code-reflection] Integrated: Moved PTX backend functionality to CUDA backend In-Reply-To: References: Message-ID: On Sun, 6 Apr 2025 11:33:49 GMT, Gary Frost wrote: > Moved the required PTX generation code into the more general CUDA backend. > > We can select PTX generation either by sending a config string including 'PTX' to the backend (default will be Cuda C99) > > We can also add PTX to HAT env var at launch > > > HAT=INFO,PTX java @hat/run ffi-cuda squares > > vs > > HAT=INFO java @hat/run ffi-cude squares This pull request has now been integrated. Changeset: d8a08236 Author: Gary Frost URL: https://git.openjdk.org/babylon/commit/d8a082365bb65eed025a3c9518918369af454592 Stats: 1609 lines in 14 files changed: 1507 ins; 49 del; 53 mod Moved PTX backend functionality to CUDA backend ------------- PR: https://git.openjdk.org/babylon/pull/384 From gfrost at openjdk.org Sun Apr 6 14:33:49 2025 From: gfrost at openjdk.org (Gary Frost) Date: Sun, 6 Apr 2025 14:33:49 GMT Subject: git: openjdk/babylon: code-reflection: Removed ptx backend Message-ID: <8cc98e86-5824-457a-9e9a-e34e9e624ba4@openjdk.org> Changeset: cb41d87e Branch: code-reflection Author: Gary Frost Date: 2025-04-06 14:32:28 +0000 URL: https://git.openjdk.org/babylon/commit/cb41d87e86e96336b1b50867d825deaf82e4d5dc Removed ptx backend - hat/backends/ffi/ptx/.gitignore - hat/backends/ffi/ptx/CMakeLists.txt - hat/backends/ffi/ptx/cpp/info.cpp - hat/backends/ffi/ptx/cpp/ptx_backend.cpp - hat/backends/ffi/ptx/include/ptx_backend.h - hat/backends/ffi/ptx/pom.xml - hat/backends/ffi/ptx/src/main/java/hat/backend/ffi/PTXBackend.java - hat/backends/ffi/ptx/src/main/java/hat/backend/ffi/PTXCodeBuilder.java - hat/backends/ffi/ptx/src/main/java/hat/backend/ffi/PTXDeviceInfo.java - hat/backends/ffi/ptx/src/main/java/hat/backend/ffi/PTXPtrOp.java - hat/backends/ffi/ptx/src/main/java/hat/backend/ffi/PTXRegister.java - hat/backends/ffi/ptx/src/main/java/hat/backend/ffi/TestIt.java - hat/backends/ffi/ptx/src/main/resources/META-INF/services/hat.backend.Backend ! hat/intellij/.idea/modules.xml - hat/intellij/backend_ffi_ptx.iml From gfrost at openjdk.org Sun Apr 6 14:35:12 2025 From: gfrost at openjdk.org (Gary Frost) Date: Sun, 6 Apr 2025 14:35:12 GMT Subject: [code-reflection] Integrated: Removed ptx backend Message-ID: The previous PR moved ptx backend functionality to Cuda backend. So just removing the redundant files and project artifacts ------------- Commit messages: - Removed ptx backend Changes: https://git.openjdk.org/babylon/pull/385/files Webrev: https://webrevs.openjdk.org/?repo=babylon&pr=385&range=00 Stats: 2467 lines in 15 files changed: 0 ins; 2467 del; 0 mod Patch: https://git.openjdk.org/babylon/pull/385.diff Fetch: git fetch https://git.openjdk.org/babylon.git pull/385/head:pull/385 PR: https://git.openjdk.org/babylon/pull/385 From gfrost at openjdk.org Sun Apr 6 14:35:12 2025 From: gfrost at openjdk.org (Gary Frost) Date: Sun, 6 Apr 2025 14:35:12 GMT Subject: [code-reflection] Integrated: Removed ptx backend In-Reply-To: References: Message-ID: On Sun, 6 Apr 2025 14:31:14 GMT, Gary Frost wrote: > The previous PR moved ptx backend functionality to Cuda backend. So just removing the redundant files and project artifacts This pull request has now been integrated. Changeset: cb41d87e Author: Gary Frost URL: https://git.openjdk.org/babylon/commit/cb41d87e86e96336b1b50867d825deaf82e4d5dc Stats: 2467 lines in 15 files changed: 0 ins; 2467 del; 0 mod Removed ptx backend ------------- PR: https://git.openjdk.org/babylon/pull/385 From duke at openjdk.org Mon Apr 7 15:20:52 2025 From: duke at openjdk.org (f) Date: Mon, 7 Apr 2025 15:20:52 GMT Subject: [code-reflection] RFR: 1D Matrix Multiplication example for HAT [v2] In-Reply-To: References: Message-ID: On Thu, 13 Feb 2025 06:33:11 GMT, Juan Fumero wrote: >> Juan Fumero has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 11 commits: >> >> - Merge branch 'code-reflection' into dev/examples >> - Minor fix seq-comparison code >> - Merge with latest develop >> - Merge branch 'code-reflection' into dev/examples >> - Merge branch 'code-reflection' into dev/examples >> - Merge branch 'code-reflection' into dev/examples >> - MatrixMult example moved to matmul directory >> - Merge branch 'code-reflection' into dev/examples >> - Precision control error down to 1% >> - Matrix-Multiplication checks >> - ... and 1 more: https://git.openjdk.org/babylon/compare/ee3da036...cd3c7ce9 > > Pending for review @jjfumero Hi, can use Babylon to implement operations to do llama inference now? Babylon has all the basic ops for at least one platform ie. cuda now? ------------- PR Comment: https://git.openjdk.org/babylon/pull/276#issuecomment-2783699286 From duke at openjdk.org Mon Apr 7 15:31:15 2025 From: duke at openjdk.org (f) Date: Mon, 7 Apr 2025 15:31:15 GMT Subject: [code-reflection] RFR: 1D Matrix Multiplication example for HAT [v2] In-Reply-To: References: Message-ID: On Thu, 13 Feb 2025 06:33:11 GMT, Juan Fumero wrote: >> Juan Fumero has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 11 commits: >> >> - Merge branch 'code-reflection' into dev/examples >> - Minor fix seq-comparison code >> - Merge with latest develop >> - Merge branch 'code-reflection' into dev/examples >> - Merge branch 'code-reflection' into dev/examples >> - Merge branch 'code-reflection' into dev/examples >> - MatrixMult example moved to matmul directory >> - Merge branch 'code-reflection' into dev/examples >> - Precision control error down to 1% >> - Matrix-Multiplication checks >> - ... and 1 more: https://git.openjdk.org/babylon/compare/ee3da036...cd3c7ce9 > > Pending for review @jjfumero Hi, can use Babylon to implement operations to do llama inference now? Babylon has all the basic ops for at least one platform ie. cuda now? ------------- PR Comment: https://git.openjdk.org/babylon/pull/276#issuecomment-2783733732 From duke at openjdk.org Mon Apr 7 16:23:04 2025 From: duke at openjdk.org (Juan Fumero) Date: Mon, 7 Apr 2025 16:23:04 GMT Subject: [code-reflection] RFR: 1D Matrix Multiplication example for HAT [v2] In-Reply-To: References: Message-ID: On Mon, 7 Apr 2025 15:17:37 GMT, f wrote: >> Pending for review > > @jjfumero Hi, can use Babylon to implement operations to do llama inference now? Babylon has all the basic ops for at least one platform ie. cuda now? Hi @SidneyLann , I am not the core maintainer of Babylon. Probably Gary Frost can help you with your questions. From my view, I think you need to access shared memory and some synchronisation primitives to be able to perform reductions. I am not sure if this is implemented in HAT yet. ------------- PR Comment: https://git.openjdk.org/babylon/pull/276#issuecomment-2783916596 From gfrost at openjdk.org Mon Apr 7 17:54:03 2025 From: gfrost at openjdk.org (Gary Frost) Date: Mon, 7 Apr 2025 17:54:03 GMT Subject: [code-reflection] RFR: 1D Matrix Multiplication example for HAT [v2] In-Reply-To: References: Message-ID: On Sat, 18 Jan 2025 06:27:18 GMT, Juan Fumero wrote: >> Add new example for 1D Matrix Multiplication in HAT. >> >> ### How to test? >> >> >> ## Compile >> java --add-modules jdk.incubator.code --enable-preview --source 24 bld >> >> ## Run with the OpenCL Backend >> java @bldr/hatrun ffi-opencl matmul >> >> ## Run with the CUDA Backend >> java @bldr/hatrun ffi-ptx matmul >> >> >> #### Note that the generated kernel for OpenCL contains a race condition: >> >> >> __kernel void matrixMultiplyKernel( >> __global KernelContext_t *kc, __global F32Array_t* matrixA, __global F32Array_t* matrixB, __global F32Array_t* matrixC, int size >> ){ >> kc->x=get_global_id(0); // << Shared struct across all threads to store the thread-id >> if(kc->xmaxX){ >> for(int j = 0; j> float acc = (float)0; >> for(int k = 0; k> acc=acc+matrixA->array[(long)(kc->x*size+k)]*matrixB->array[(long)(k*size+j)]; >> } >> matrixC->array[(long)(kc->x*size+j)]=acc; >> } >> } >> return; >> } >> >> >> After applying a patch provided by Gary Frost to solve the race condition, it works. >> >> Patch: >> >> >> diff --git a/hat/hat/src/main/java/hat/backend/c99codebuilders/C99HatKernelBuilder.java b/hat/hat/src/main/java/hat/backend/c99codebuilders/C99HatKernelBuilder.java >> index ade90914d7e..2719fed31ed 100644 >> --- a/hat/hat/src/main/java/hat/backend/c99codebuilders/C99HatKernelBuilder.java >> +++ b/hat/hat/src/main/java/hat/backend/c99codebuilders/C99HatKernelBuilder.java >> @@ -26,7 +26,6 @@ >> >> >> import hat.buffer.Buffer; >> -import hat.buffer.KernelContext; >> import hat.callgraph.KernelCallGraph; >> import hat.callgraph.KernelEntrypoint; >> import hat.optools.FuncOpWrapper; >> @@ -72,9 +71,13 @@ T typedefStructOrUnion(boolean isStruct, String name, Consumer consumer) { >> >> >> public final T scope() { >> - return >> - identifier("kc").rarrow().identifier("x").equals().globalId().semicolon().nl(); >> - //.identifier("kc").rarrow().identifier("maxX").equals().globalSize().semicolon().nl(); >> + >> + identifier("KernelContext_t").space().identifier("mine").semicolon().nl(); >> + identifier("KernelContext_t").asterisk().space().identifier("kc").equals().ampersand().identifier("mine").semicolon().nl(); >> + identifier("kc").rarrow().identifier("x").equals().globalId().semicolon().nl(); >> + identifier("kc").rarrow().identifier("maxX").equals().identifie... > > Juan Fumero has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 11 commits: > > - Merge branch 'code-reflection' into dev/examples > - Minor fix seq-comparison code > - Merge with latest develop > - Merge branch 'code-reflection' into dev/examples > - Merge branch 'code-reflection' into dev/examples > - Merge branch 'code-reflection' into dev/examples > - MatrixMult example moved to matmul directory > - Merge branch 'code-reflection' into dev/examples > - Precision control error down to 1% > - Matrix-Multiplication checks > - ... and 1 more: https://git.openjdk.org/babylon/compare/ee3da036...cd3c7ce9 Hi Sidney As Juan mentioned, I don't think we are there yet. But we have plans. We need to add low level primitives to HAT for matrix mul/scans etc to allow us to handoff to the vendor backends (GPU drivers) without unnecessary copies. Gary On Mon, Apr 7, 2025 at 5:20?PM Juan Fumero ***@***.***> wrote: > Hi @SidneyLann , I am not the core > maintainer of Babylon. Probably Gary Frost can help you with your > questions. From my view, I think you need to access shared memory and some > synchronisation primitives to be able to perform reductions. I am not sure > if this is implemented in HAT yet. > > ? > Reply to this email directly, view it on GitHub > , or > unsubscribe > > . > You are receiving this because you are subscribed to this thread.Message > ID: ***@***.***> > [image: jjfumero]*jjfumero* left a comment (openjdk/babylon#276) > > > Hi @SidneyLann , I am not the core > maintainer of Babylon. Probably Gary Frost can help you with your > questions. From my view, I think you need to access shared memory and some > synchronisation primitives to be able to perform reductions. I am not sure > if this is implemented in HAT yet. > > ? > Reply to this email directly, view it on GitHub > , or > unsubscribe > > . > You are receiving this because you are subscribed to this thread.Message > ID: ***@***.***> > ------------- PR Comment: https://git.openjdk.org/babylon/pull/276#issuecomment-2784126120 From mcimadamore at openjdk.org Mon Apr 7 21:23:34 2025 From: mcimadamore at openjdk.org (Maurizio Cimadamore) Date: Mon, 7 Apr 2025 21:23:34 GMT Subject: [code-reflection] RFR: Cleanup `CodeModelToAST` to use `Resolve` Message-ID: `CodeModelToAST` needs to convert field/method references back into javac var and method symbols. In javac backend classes this is typically done by using `Resolve::resolveInternalField/Method`. The current approach is instead to create new symbol instances from the contents of the field/method references, which is problematic because the code has to figure out properties that are not encoded in the references -- such as flags and other hidden information (such as constant values) that might be left there by javac. This PR rectifies that. I also cleaned up the code a bit by breaking very long lines and I have fixed an issue in the translation from TypeElement back to javac type as erased type (e.g. `Map`) were converted back to generic types (e.g. `Map`) thus leading to issues when calling the internal resolution methods. The only place left where we create a new symbol from scratch is with instance creation expressions -- to fix this we need to associate a `MethodRef` to `NewOp` nodes. ------------- Commit messages: - Cleanup code in CodeModelToAST Changes: https://git.openjdk.org/babylon/pull/386/files Webrev: https://webrevs.openjdk.org/?repo=babylon&pr=386&range=00 Stats: 84 lines in 2 files changed: 36 ins; 22 del; 26 mod Patch: https://git.openjdk.org/babylon/pull/386.diff Fetch: git fetch https://git.openjdk.org/babylon.git pull/386/head:pull/386 PR: https://git.openjdk.org/babylon/pull/386 From duke at openjdk.org Mon Apr 7 23:29:38 2025 From: duke at openjdk.org (f) Date: Mon, 7 Apr 2025 23:29:38 GMT Subject: [code-reflection] RFR: 1D Matrix Multiplication example for HAT [v2] In-Reply-To: References: Message-ID: On Mon, 7 Apr 2025 17:51:27 GMT, Gary Frost wrote: >> Juan Fumero has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 11 commits: >> >> - Merge branch 'code-reflection' into dev/examples >> - Minor fix seq-comparison code >> - Merge with latest develop >> - Merge branch 'code-reflection' into dev/examples >> - Merge branch 'code-reflection' into dev/examples >> - Merge branch 'code-reflection' into dev/examples >> - MatrixMult example moved to matmul directory >> - Merge branch 'code-reflection' into dev/examples >> - Precision control error down to 1% >> - Matrix-Multiplication checks >> - ... and 1 more: https://git.openjdk.org/babylon/compare/ee3da036...cd3c7ce9 > > Hi Sidney > > As Juan mentioned, I don't think we are there yet. But we have plans. > > We need to add low level primitives to HAT for matrix mul/scans etc to > allow us to handoff to the vendor backends (GPU drivers) without > unnecessary copies. > > Gary > > > On Mon, Apr 7, 2025 at 5:20?PM Juan Fumero ***@***.***> wrote: > >> Hi @SidneyLann , I am not the core >> maintainer of Babylon. Probably Gary Frost can help you with your >> questions. From my view, I think you need to access shared memory and some >> synchronisation primitives to be able to perform reductions. I am not sure >> if this is implemented in HAT yet. >> >> ? >> Reply to this email directly, view it on GitHub >> , or >> unsubscribe >> >> . >> You are receiving this because you are subscribed to this thread.Message >> ID: ***@***.***> >> [image: jjfumero]*jjfumero* left a comment (openjdk/babylon#276) >> >> >> Hi @SidneyLann , I am not the core >> maintainer of Babylon. Probably Gary Frost can help you with your >> questions. From my view, I think you need to access shared memory and some >> synchronisation primitives to be able to perform reductions. I am not sure >> if this is implemented in HAT yet. >> >> ? >> Reply to this email directly, view it on GitHub >> , or >> unsubscribe >> >> . >> You are receiving this because you are subscribed to this thread.Message >> ID: ***@***.***> >> @grfrost Hi Gray Are you develope many platforms(ptx,cuda,spirv,hip,etc) simultaneously?How about complete one platform(ie. cuda) first? ------------- PR Comment: https://git.openjdk.org/babylon/pull/276#issuecomment-2784852485 From mabbay at openjdk.org Tue Apr 8 16:03:01 2025 From: mabbay at openjdk.org (Mourad Abbay) Date: Tue, 8 Apr 2025 16:03:01 GMT Subject: [code-reflection] RFR: Remove the code model transformation: addVarsWhenNecessary. Message-ID: We remove the code model transformation: `addVarsWhenNecessary`. ------------- Commit messages: - Remove the code model transformation: addVarsWhenNecessary. Changes: https://git.openjdk.org/babylon/pull/387/files Webrev: https://webrevs.openjdk.org/?repo=babylon&pr=387&range=00 Stats: 102 lines in 1 file changed: 34 ins; 59 del; 9 mod Patch: https://git.openjdk.org/babylon/pull/387.diff Fetch: git fetch https://git.openjdk.org/babylon.git pull/387/head:pull/387 PR: https://git.openjdk.org/babylon/pull/387 From mcimadamore at openjdk.org Tue Apr 8 16:07:51 2025 From: mcimadamore at openjdk.org (Maurizio Cimadamore) Date: Tue, 8 Apr 2025 16:07:51 GMT Subject: [code-reflection] RFR: Remove the code model transformation: addVarsWhenNecessary. In-Reply-To: References: Message-ID: On Tue, 8 Apr 2025 15:58:24 GMT, Mourad Abbay wrote: > We remove the code model transformation: `addVarsWhenNecessary`. src/jdk.incubator.code/share/classes/jdk/incubator/code/internal/CodeModelToAST.java line 219: > 217: } > 218: > 219: private JCTree.JCExpression exprTree(Value v) { Why do we have nodes in the `valueToTree` map that are not variables? ------------- PR Review Comment: https://git.openjdk.org/babylon/pull/387#discussion_r2033540375 From mcimadamore at openjdk.org Tue Apr 8 16:14:40 2025 From: mcimadamore at openjdk.org (Maurizio Cimadamore) Date: Tue, 8 Apr 2025 16:14:40 GMT Subject: [code-reflection] RFR: Remove the code model transformation: addVarsWhenNecessary. In-Reply-To: <5NCcDRp2j9aYBTAB3EmOx1qnjpWABAn18G-cw--ASwE=.9711b01f-2e24-4155-9d6c-c131e7d64683@github.com> References: <5NCcDRp2j9aYBTAB3EmOx1qnjpWABAn18G-cw--ASwE=.9711b01f-2e24-4155-9d6c-c131e7d64683@github.com> Message-ID: <0twXkIXZQzuVtn2WM7p3wfMDRfg_0-5tWNyhYW5h0fc=.dc1ce9b7-da29-4598-a5da-e75e5ee20d0c@github.com> On Tue, 8 Apr 2025 16:08:55 GMT, Maurizio Cimadamore wrote: >> src/jdk.incubator.code/share/classes/jdk/incubator/code/internal/CodeModelToAST.java line 219: >> >>> 217: } >>> 218: >>> 219: private JCTree.JCExpression exprTree(Value v) { >> >> Why do we have nodes in the `valueToTree` map that are not variables? > > It seems to me that, with the exception of `return` we only deal with expressions? I see that we create some intermediate `Exec` nodes (e.g. for array access) but it seems to me that this indirection can be removed in this new code? It seems to me that, with the exception of `return` we only deal with expressions? I see that we create some intermediate `Exec` nodes (e.g. for array access) but it seems to me that this indirection can be removed in this new code? ------------- PR Review Comment: https://git.openjdk.org/babylon/pull/387#discussion_r2033549199 From mcimadamore at openjdk.org Tue Apr 8 16:14:40 2025 From: mcimadamore at openjdk.org (Maurizio Cimadamore) Date: Tue, 8 Apr 2025 16:14:40 GMT Subject: [code-reflection] RFR: Remove the code model transformation: addVarsWhenNecessary. In-Reply-To: References: Message-ID: <5NCcDRp2j9aYBTAB3EmOx1qnjpWABAn18G-cw--ASwE=.9711b01f-2e24-4155-9d6c-c131e7d64683@github.com> On Tue, 8 Apr 2025 16:05:01 GMT, Maurizio Cimadamore wrote: >> We remove the code model transformation: `addVarsWhenNecessary`. > > src/jdk.incubator.code/share/classes/jdk/incubator/code/internal/CodeModelToAST.java line 219: > >> 217: } >> 218: >> 219: private JCTree.JCExpression exprTree(Value v) { > > Why do we have nodes in the `valueToTree` map that are not variables? It seems to me that, with the exception of `return` we only deal with expressions? I see that we create some intermediate `Exec` nodes (e.g. for array access) but it seems to me that this indirection can be removed in this new code? ------------- PR Review Comment: https://git.openjdk.org/babylon/pull/387#discussion_r2033548716 From mcimadamore at openjdk.org Tue Apr 8 16:14:40 2025 From: mcimadamore at openjdk.org (Maurizio Cimadamore) Date: Tue, 8 Apr 2025 16:14:40 GMT Subject: [code-reflection] RFR: Remove the code model transformation: addVarsWhenNecessary. In-Reply-To: References: Message-ID: On Tue, 8 Apr 2025 15:58:24 GMT, Mourad Abbay wrote: > We remove the code model transformation: `addVarsWhenNecessary`. src/jdk.incubator.code/share/classes/jdk/incubator/code/internal/CodeModelToAST.java line 30: > 28: private final Map jtToType; > 29: private Symbol.MethodSymbol ms; > 30: private int c = 0; // used to name variables we introduce in the AST please rename to `localVarCount` or something like that src/jdk.incubator.code/share/classes/jdk/incubator/code/internal/CodeModelToAST.java line 198: > 196: default -> throw new IllegalStateException("Op -> JCTree not supported for :" + op.getClass().getName()); > 197: }; > 198: if (tree instanceof JCTree.JCExpression expr) { When we get here, in my mind there's only two cases: * `tree` is an expression -- in which case we treat the expression as the initializer of some synthetic variable that we create * `tree` is a `return` node -- in which case there's nothing to do -- e.g. no need to call `map` ! ------------- PR Review Comment: https://git.openjdk.org/babylon/pull/387#discussion_r2033554346 PR Review Comment: https://git.openjdk.org/babylon/pull/387#discussion_r2033552583 From mabbay at openjdk.org Tue Apr 8 16:40:39 2025 From: mabbay at openjdk.org (Mourad Abbay) Date: Tue, 8 Apr 2025 16:40:39 GMT Subject: [code-reflection] RFR: Remove the code model transformation: addVarsWhenNecessary. In-Reply-To: <0twXkIXZQzuVtn2WM7p3wfMDRfg_0-5tWNyhYW5h0fc=.dc1ce9b7-da29-4598-a5da-e75e5ee20d0c@github.com> References: <5NCcDRp2j9aYBTAB3EmOx1qnjpWABAn18G-cw--ASwE=.9711b01f-2e24-4155-9d6c-c131e7d64683@github.com> <0twXkIXZQzuVtn2WM7p3wfMDRfg_0-5tWNyhYW5h0fc=.dc1ce9b7-da29-4598-a5da-e75e5ee20d0c@github.com> Message-ID: On Tue, 8 Apr 2025 16:09:07 GMT, Maurizio Cimadamore wrote: >> It seems to me that, with the exception of `return` we only deal with expressions? I see that we create some intermediate `Exec` nodes (e.g. for array access) but it seems to me that this indirection can be removed in this new code? > > It seems to me that, with the exception of `return` we only deal with expressions? I see that we create some intermediate `Exec` nodes (e.g. for array access) but it seems to me that this indirection can be removed in this new code? Yes, we can map all operations to JCExpression, expect for return. ArrayStoreOp can also be mapped to an expression. I also noticed that we only deal with InvokeOp that has return type different than void, thus then can be mapped to expressions. ------------- PR Review Comment: https://git.openjdk.org/babylon/pull/387#discussion_r2033604628 From mabbay at openjdk.org Tue Apr 8 16:50:35 2025 From: mabbay at openjdk.org (Mourad Abbay) Date: Tue, 8 Apr 2025 16:50:35 GMT Subject: [code-reflection] RFR: Remove the code model transformation: addVarsWhenNecessary. [v2] In-Reply-To: References: Message-ID: > We remove the code model transformation: `addVarsWhenNecessary`. Mourad Abbay has updated the pull request incrementally with four additional commits since the last revision: - Refactor. - All InvokeOps (we can have) can be me mapped to JCExpression directly. - The mapping of returnValue to tree is not going to be requested, so no need to store it. - Map ArrayStoreOp to a JCExpression. ------------- Changes: - all: https://git.openjdk.org/babylon/pull/387/files - new: https://git.openjdk.org/babylon/pull/387/files/1645e037..cae6c352 Webrevs: - full: https://webrevs.openjdk.org/?repo=babylon&pr=387&range=01 - incr: https://webrevs.openjdk.org/?repo=babylon&pr=387&range=00-01 Stats: 8 lines in 1 file changed: 1 ins; 4 del; 3 mod Patch: https://git.openjdk.org/babylon/pull/387.diff Fetch: git fetch https://git.openjdk.org/babylon.git pull/387/head:pull/387 PR: https://git.openjdk.org/babylon/pull/387 From mcimadamore at openjdk.org Tue Apr 8 16:56:26 2025 From: mcimadamore at openjdk.org (Maurizio Cimadamore) Date: Tue, 8 Apr 2025 16:56:26 GMT Subject: git: openjdk/babylon: code-reflection: Cleanup `CodeModelToAST` to use `Resolve` Message-ID: Changeset: 9cddb871 Branch: code-reflection Author: Maurizio Cimadamore Date: 2025-04-08 16:54:07 +0000 URL: https://git.openjdk.org/babylon/commit/9cddb871638b7590033c0127f26e5e8ae41294a6 Cleanup `CodeModelToAST` to use `Resolve` ! src/jdk.incubator.code/share/classes/jdk/incubator/code/internal/CodeModelToAST.java ! src/jdk.incubator.code/share/classes/jdk/incubator/code/internal/ReflectMethods.java From mcimadamore at openjdk.org Tue Apr 8 16:56:44 2025 From: mcimadamore at openjdk.org (Maurizio Cimadamore) Date: Tue, 8 Apr 2025 16:56:44 GMT Subject: [code-reflection] Integrated: Cleanup `CodeModelToAST` to use `Resolve` In-Reply-To: References: Message-ID: On Mon, 7 Apr 2025 17:46:52 GMT, Maurizio Cimadamore wrote: > `CodeModelToAST` needs to convert field/method references back into javac var and method symbols. In javac backend classes this is typically done by using `Resolve::resolveInternalField/Method`. The current approach is instead to create new symbol instances from the contents of the field/method references, which is problematic because the code has to figure out properties that are not encoded in the references -- such as flags and other hidden information (such as constant values) that might be left there by javac. > > This PR rectifies that. I also cleaned up the code a bit by breaking very long lines and I have fixed an issue in the translation from TypeElement back to javac type as erased type (e.g. `Map`) were converted back to generic types (e.g. `Map`) thus leading to issues when calling the internal resolution methods. > > The only place left where we create a new symbol from scratch is with instance creation expressions -- to fix this we need to associate a `MethodRef` to `NewOp` nodes. This pull request has now been integrated. Changeset: 9cddb871 Author: Maurizio Cimadamore URL: https://git.openjdk.org/babylon/commit/9cddb871638b7590033c0127f26e5e8ae41294a6 Stats: 84 lines in 2 files changed: 36 ins; 22 del; 26 mod Cleanup `CodeModelToAST` to use `Resolve` ------------- PR: https://git.openjdk.org/babylon/pull/386 From mabbay at openjdk.org Tue Apr 8 17:12:13 2025 From: mabbay at openjdk.org (Mourad Abbay) Date: Tue, 8 Apr 2025 17:12:13 GMT Subject: [code-reflection] RFR: Remove the code model transformation: addVarsWhenNecessary. [v3] In-Reply-To: References: Message-ID: > We remove the code model transformation: `addVarsWhenNecessary`. Mourad Abbay has updated the pull request incrementally with one additional commit since the last revision: Simplify CodeModelToAST.exprTree ------------- Changes: - all: https://git.openjdk.org/babylon/pull/387/files - new: https://git.openjdk.org/babylon/pull/387/files/cae6c352..66ad9bf2 Webrevs: - full: https://webrevs.openjdk.org/?repo=babylon&pr=387&range=02 - incr: https://webrevs.openjdk.org/?repo=babylon&pr=387&range=01-02 Stats: 2 lines in 1 file changed: 0 ins; 2 del; 0 mod Patch: https://git.openjdk.org/babylon/pull/387.diff Fetch: git fetch https://git.openjdk.org/babylon.git pull/387/head:pull/387 PR: https://git.openjdk.org/babylon/pull/387 From mabbay at openjdk.org Tue Apr 8 17:25:55 2025 From: mabbay at openjdk.org (Mourad Abbay) Date: Tue, 8 Apr 2025 17:25:55 GMT Subject: [code-reflection] RFR: Remove the code model transformation: addVarsWhenNecessary. [v4] In-Reply-To: References: Message-ID: > We remove the code model transformation: `addVarsWhenNecessary`. Mourad Abbay has updated the pull request incrementally with one additional commit since the last revision: Map Value to VarSymbol ------------- Changes: - all: https://git.openjdk.org/babylon/pull/387/files - new: https://git.openjdk.org/babylon/pull/387/files/66ad9bf2..c50e3cb5 Webrevs: - full: https://webrevs.openjdk.org/?repo=babylon&pr=387&range=03 - incr: https://webrevs.openjdk.org/?repo=babylon&pr=387&range=02-03 Stats: 12 lines in 1 file changed: 0 ins; 6 del; 6 mod Patch: https://git.openjdk.org/babylon/pull/387.diff Fetch: git fetch https://git.openjdk.org/babylon.git pull/387/head:pull/387 PR: https://git.openjdk.org/babylon/pull/387 From mabbay at openjdk.org Tue Apr 8 17:45:13 2025 From: mabbay at openjdk.org (Mourad Abbay) Date: Tue, 8 Apr 2025 17:45:13 GMT Subject: [code-reflection] RFR: Remove the code model transformation: addVarsWhenNecessary. [v5] In-Reply-To: References: Message-ID: > We remove the code model transformation: `addVarsWhenNecessary`. Mourad Abbay has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains nine commits: - Remove unnecessary comments - Merge branch 'code-reflection' into remove-addVarsWhenNecessary # Conflicts: # src/jdk.incubator.code/share/classes/jdk/incubator/code/internal/CodeModelToAST.java - Map Value to VarSymbol - Simplify CodeModelToAST.exprTree - Refactor. - All InvokeOps (we can have) can be me mapped to JCExpression directly. - The mapping of returnValue to tree is not going to be requested, so no need to store it. - Map ArrayStoreOp to a JCExpression. - Remove the code model transformation: addVarsWhenNecessary. ------------- Changes: https://git.openjdk.org/babylon/pull/387/files Webrev: https://webrevs.openjdk.org/?repo=babylon&pr=387&range=04 Stats: 100 lines in 1 file changed: 25 ins; 63 del; 12 mod Patch: https://git.openjdk.org/babylon/pull/387.diff Fetch: git fetch https://git.openjdk.org/babylon.git pull/387/head:pull/387 PR: https://git.openjdk.org/babylon/pull/387 From mcimadamore at openjdk.org Tue Apr 8 17:51:42 2025 From: mcimadamore at openjdk.org (Maurizio Cimadamore) Date: Tue, 8 Apr 2025 17:51:42 GMT Subject: [code-reflection] RFR: Remove the code model transformation: addVarsWhenNecessary. [v5] In-Reply-To: References: Message-ID: On Tue, 8 Apr 2025 17:45:13 GMT, Mourad Abbay wrote: >> We remove the code model transformation: `addVarsWhenNecessary`. > > Mourad Abbay has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains nine commits: > > - Remove unnecessary comments > - Merge branch 'code-reflection' into remove-addVarsWhenNecessary > > # Conflicts: > # src/jdk.incubator.code/share/classes/jdk/incubator/code/internal/CodeModelToAST.java > - Map Value to VarSymbol > - Simplify CodeModelToAST.exprTree > - Refactor. > - All InvokeOps (we can have) can be me mapped to JCExpression directly. > - The mapping of returnValue to tree is not going to be requested, so no need to store it. > - Map ArrayStoreOp to a JCExpression. > - Remove the code model transformation: addVarsWhenNecessary. On another note: I realized that `CodeModelToAST` still dumps everything unconditionally to `System.out` -- we should drop that? ------------- PR Comment: https://git.openjdk.org/babylon/pull/387#issuecomment-2787225956 From mabbay at openjdk.org Tue Apr 8 18:25:21 2025 From: mabbay at openjdk.org (Mourad Abbay) Date: Tue, 8 Apr 2025 18:25:21 GMT Subject: [code-reflection] RFR: Remove the code model transformation: addVarsWhenNecessary. [v5] In-Reply-To: References: Message-ID: On Tue, 8 Apr 2025 17:49:09 GMT, Maurizio Cimadamore wrote: > On another note: I realized that `CodeModelToAST` still dumps everything unconditionally to `System.out` -- we should drop that? I removed that ------------- PR Comment: https://git.openjdk.org/babylon/pull/387#issuecomment-2787319009 From mcimadamore at openjdk.org Wed Apr 9 08:27:55 2025 From: mcimadamore at openjdk.org (Maurizio Cimadamore) Date: Wed, 9 Apr 2025 08:27:55 GMT Subject: [code-reflection] RFR: Remove the code model transformation: addVarsWhenNecessary. [v5] In-Reply-To: References: Message-ID: On Tue, 8 Apr 2025 17:45:13 GMT, Mourad Abbay wrote: >> We remove the code model transformation: `addVarsWhenNecessary`. > > Mourad Abbay has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains nine commits: > > - Remove unnecessary comments > - Merge branch 'code-reflection' into remove-addVarsWhenNecessary > > # Conflicts: > # src/jdk.incubator.code/share/classes/jdk/incubator/code/internal/CodeModelToAST.java > - Map Value to VarSymbol > - Simplify CodeModelToAST.exprTree > - Refactor. > - All InvokeOps (we can have) can be me mapped to JCExpression directly. > - The mapping of returnValue to tree is not going to be requested, so no need to store it. > - Map ArrayStoreOp to a JCExpression. > - Remove the code model transformation: addVarsWhenNecessary. Very nice cleanup! ------------- Marked as reviewed by mcimadamore (Reviewer). PR Review: https://git.openjdk.org/babylon/pull/387#pullrequestreview-2752550638 From gfrost at openjdk.org Wed Apr 9 12:05:40 2025 From: gfrost at openjdk.org (Gary Frost) Date: Wed, 9 Apr 2025 12:05:40 GMT Subject: git: openjdk/babylon: code-reflection: Hat merge cuda opencl backend common code 1 Message-ID: Changeset: 82ab10f4 Branch: code-reflection Author: Gary Frost Date: 2025-04-09 12:03:23 +0000 URL: https://git.openjdk.org/babylon/commit/82ab10f4030406aa44d02eac800c3bd27a96237a Hat merge cuda opencl backend common code 1 ! hat/backends/ffi/CMakeLists.txt ! hat/backends/ffi/mock/cpp/mock_backend.cpp ! hat/backends/ffi/opencl/CMakeLists.txt ! hat/backends/ffi/opencl/cpp/opencl_backend.cpp ! hat/backends/ffi/opencl/cpp/opencl_backend_buffer.cpp - hat/backends/ffi/opencl/cpp/opencl_backend_config.cpp ! hat/backends/ffi/opencl/cpp/opencl_backend_kernel.cpp ! hat/backends/ffi/opencl/cpp/opencl_backend_program.cpp ! hat/backends/ffi/opencl/cpp/opencl_backend_queue.cpp ! hat/backends/ffi/opencl/cpp/squares.cpp ! hat/backends/ffi/opencl/include/opencl_backend.h ! hat/backends/ffi/shared/cpp/shared.cpp ! hat/backends/ffi/shared/include/shared.h ! hat/backends/ffi/spirv/cpp/spirv_backend.cpp From gfrost at openjdk.org Wed Apr 9 12:08:23 2025 From: gfrost at openjdk.org (Gary Frost) Date: Wed, 9 Apr 2025 12:08:23 GMT Subject: [code-reflection] Integrated: Hat merge cuda opencl backend common code 1 In-Reply-To: References: Message-ID: <0I8TZzQiI2OcWMDrpgu6nlyeQhsZp4fyRk0kk6o4Xq0=.725dd122-d9f6-45bf-ab0c-be8424e19f93@github.com> On Wed, 9 Apr 2025 12:01:04 GMT, Gary Frost wrote: > More progress moving opencl/cuda ffi specific code to ffi-shared > > We still need to test and integrate the CUDA side. This pull request has now been integrated. Changeset: 82ab10f4 Author: Gary Frost URL: https://git.openjdk.org/babylon/commit/82ab10f4030406aa44d02eac800c3bd27a96237a Stats: 1239 lines in 14 files changed: 493 ins; 440 del; 306 mod Hat merge cuda opencl backend common code 1 ------------- PR: https://git.openjdk.org/babylon/pull/388 From gfrost at openjdk.org Wed Apr 9 12:08:23 2025 From: gfrost at openjdk.org (Gary Frost) Date: Wed, 9 Apr 2025 12:08:23 GMT Subject: [code-reflection] Integrated: Hat merge cuda opencl backend common code 1 Message-ID: More progress moving opencl/cuda ffi specific code to ffi-shared We still need to test and integrate the CUDA side. ------------- Commit messages: - Moved all kernel dispatch to ffi-shared, ready to refactor CUDA side now - Moving core/common backend functionality down to ffi-shared Changes: https://git.openjdk.org/babylon/pull/388/files Webrev: https://webrevs.openjdk.org/?repo=babylon&pr=388&range=00 Stats: 1239 lines in 14 files changed: 493 ins; 440 del; 306 mod Patch: https://git.openjdk.org/babylon/pull/388.diff Fetch: git fetch https://git.openjdk.org/babylon.git pull/388/head:pull/388 PR: https://git.openjdk.org/babylon/pull/388 From gfrost at openjdk.org Wed Apr 9 14:25:17 2025 From: gfrost at openjdk.org (Gary Frost) Date: Wed, 9 Apr 2025 14:25:17 GMT Subject: git: openjdk/babylon: code-reflection: replaced common ffi-cuda code with ffi-shared Message-ID: Changeset: 314ecacf Branch: code-reflection Author: Gary Frost Date: 2025-04-09 14:22:57 +0000 URL: https://git.openjdk.org/babylon/commit/314ecacf383c04809a87810929ac9bc038a9bca5 replaced common ffi-cuda code with ffi-shared ! hat/backends/ffi/cuda/cpp/cuda_backend.cpp ! hat/backends/ffi/cuda/cpp/cuda_backend_buffer.cpp ! hat/backends/ffi/cuda/cpp/cuda_backend_kernel.cpp ! hat/backends/ffi/cuda/cpp/cuda_backend_queue.cpp ! hat/backends/ffi/cuda/cpp/squares.cpp ! hat/backends/ffi/cuda/include/cuda_backend.h From gfrost at openjdk.org Wed Apr 9 14:26:14 2025 From: gfrost at openjdk.org (Gary Frost) Date: Wed, 9 Apr 2025 14:26:14 GMT Subject: [code-reflection] Integrated: replaced common ffi-cuda code with ffi-shared Message-ID: Replaced common code in ffi-cuda backend with common code in ffi-shared. This finally aligns the codebase for ptx, cuda and opencl. Also acts as stable API to refactor HIP and SPIRV backends. At this time CUDA backend can not minimize buffer copies (JVM crash). So that will be investigated in the next PR. ------------- Commit messages: - replaced common ffi-cuda code with ffi-shared Changes: https://git.openjdk.org/babylon/pull/389/files Webrev: https://webrevs.openjdk.org/?repo=babylon&pr=389&range=00 Stats: 336 lines in 6 files changed: 121 ins; 157 del; 58 mod Patch: https://git.openjdk.org/babylon/pull/389.diff Fetch: git fetch https://git.openjdk.org/babylon.git pull/389/head:pull/389 PR: https://git.openjdk.org/babylon/pull/389 From gfrost at openjdk.org Wed Apr 9 14:26:14 2025 From: gfrost at openjdk.org (Gary Frost) Date: Wed, 9 Apr 2025 14:26:14 GMT Subject: [code-reflection] Integrated: replaced common ffi-cuda code with ffi-shared In-Reply-To: References: Message-ID: On Wed, 9 Apr 2025 14:20:24 GMT, Gary Frost wrote: > Replaced common code in ffi-cuda backend with common code in ffi-shared. > > This finally aligns the codebase for ptx, cuda and opencl. > > Also acts as stable API to refactor HIP and SPIRV backends. > > At this time CUDA backend can not minimize buffer copies (JVM crash). So that will be investigated in the next PR. This pull request has now been integrated. Changeset: 314ecacf Author: Gary Frost URL: https://git.openjdk.org/babylon/commit/314ecacf383c04809a87810929ac9bc038a9bca5 Stats: 336 lines in 6 files changed: 121 ins; 157 del; 58 mod replaced common ffi-cuda code with ffi-shared ------------- PR: https://git.openjdk.org/babylon/pull/389 From gfrost at openjdk.org Wed Apr 9 14:39:46 2025 From: gfrost at openjdk.org (Gary Frost) Date: Wed, 9 Apr 2025 14:39:46 GMT Subject: [code-reflection] Integrated: Add return values to setArg kernel methods Message-ID: <_pGP88llP5ow87IT_yuO03gBl-hNrgWqg3bcoE4F0kY=.db0d7b20-e220-4b4c-a433-185f672ca169@github.com> Sadly introduced a bug in last PR. Added previously ommited return true for CudaKernel::setArgXX methods. They both now return true. ------------- Commit messages: - Add return values to setArg kernel methods Changes: https://git.openjdk.org/babylon/pull/390/files Webrev: https://webrevs.openjdk.org/?repo=babylon&pr=390&range=00 Stats: 3 lines in 1 file changed: 2 ins; 0 del; 1 mod Patch: https://git.openjdk.org/babylon/pull/390.diff Fetch: git fetch https://git.openjdk.org/babylon.git pull/390/head:pull/390 PR: https://git.openjdk.org/babylon/pull/390 From gfrost at openjdk.org Wed Apr 9 14:39:47 2025 From: gfrost at openjdk.org (Gary Frost) Date: Wed, 9 Apr 2025 14:39:47 GMT Subject: [code-reflection] Integrated: Add return values to setArg kernel methods In-Reply-To: <_pGP88llP5ow87IT_yuO03gBl-hNrgWqg3bcoE4F0kY=.db0d7b20-e220-4b4c-a433-185f672ca169@github.com> References: <_pGP88llP5ow87IT_yuO03gBl-hNrgWqg3bcoE4F0kY=.db0d7b20-e220-4b4c-a433-185f672ca169@github.com> Message-ID: On Wed, 9 Apr 2025 14:32:15 GMT, Gary Frost wrote: > Sadly introduced a bug in last PR. > > Added previously ommited return true for CudaKernel::setArgXX methods. They both now return true. This pull request has now been integrated. Changeset: 2c6582ef Author: Gary Frost URL: https://git.openjdk.org/babylon/commit/2c6582ef9686e015b15d06db8e98b368b16cdae0 Stats: 3 lines in 1 file changed: 2 ins; 0 del; 1 mod Add return values to setArg kernel methods ------------- PR: https://git.openjdk.org/babylon/pull/390 From gfrost at openjdk.org Wed Apr 9 14:39:32 2025 From: gfrost at openjdk.org (Gary Frost) Date: Wed, 9 Apr 2025 14:39:32 GMT Subject: git: openjdk/babylon: code-reflection: Add return values to setArg kernel methods Message-ID: <39bf3916-0212-4240-a82c-4b21e5201cdd@openjdk.org> Changeset: 2c6582ef Branch: code-reflection Author: Gary Frost Date: 2025-04-09 14:36:02 +0000 URL: https://git.openjdk.org/babylon/commit/2c6582ef9686e015b15d06db8e98b368b16cdae0 Add return values to setArg kernel methods ! hat/backends/ffi/cuda/cpp/cuda_backend_kernel.cpp From mabbay at openjdk.org Wed Apr 9 23:20:38 2025 From: mabbay at openjdk.org (Mourad Abbay) Date: Wed, 9 Apr 2025 23:20:38 GMT Subject: git: openjdk/babylon: code-reflection: Remove the code model transformation: addVarsWhenNecessary. Message-ID: Changeset: e093b1f6 Branch: code-reflection Author: Mourad Abbay Date: 2025-04-09 23:18:52 +0000 URL: https://git.openjdk.org/babylon/commit/e093b1f6427aa9a6d23c28aaed7077ee142aa660 Remove the code model transformation: addVarsWhenNecessary. Reviewed-by: mcimadamore ! src/jdk.incubator.code/share/classes/jdk/incubator/code/internal/CodeModelToAST.java From mabbay at openjdk.org Wed Apr 9 23:21:48 2025 From: mabbay at openjdk.org (Mourad Abbay) Date: Wed, 9 Apr 2025 23:21:48 GMT Subject: [code-reflection] Integrated: Remove the code model transformation: addVarsWhenNecessary. In-Reply-To: References: Message-ID: On Tue, 8 Apr 2025 15:58:24 GMT, Mourad Abbay wrote: > We remove the code model transformation: `addVarsWhenNecessary`. This pull request has now been integrated. Changeset: e093b1f6 Author: Mourad Abbay URL: https://git.openjdk.org/babylon/commit/e093b1f6427aa9a6d23c28aaed7077ee142aa660 Stats: 100 lines in 1 file changed: 25 ins; 63 del; 12 mod Remove the code model transformation: addVarsWhenNecessary. Reviewed-by: mcimadamore ------------- PR: https://git.openjdk.org/babylon/pull/387 From mcimadamore at openjdk.org Thu Apr 10 11:43:29 2025 From: mcimadamore at openjdk.org (Maurizio Cimadamore) Date: Thu, 10 Apr 2025 11:43:29 GMT Subject: [code-reflection] RFR: Improve resolution of types in `CodeModelToAST` In-Reply-To: References: Message-ID: On Thu, 10 Apr 2025 11:37:16 GMT, Maurizio Cimadamore wrote: > The `CodeModelToAST` class features an ad-hoc table of types, which is hard-wired to include all the types that can come up in the code model generated by `OpBuilder`. > This approach is fragile (if new types are relied upon by `OpBuilder`, they need to be added there), and duplicates type resolution tables that javac has to maintain anyway. > > This PR replaces the ad-hoc type table with more idiomatic javac type lookup logic. src/jdk.incubator.code/share/classes/jdk/incubator/code/internal/CodeModelToAST.java line 62: > 60: yield new Type.ClassType(enclosing, typeArgs, typeElementToType(ct.rawType()).tsym); > 61: } > 62: case ClassType ct -> types.erasure(syms.enterClass(attrEnv.toplevel.modle, ct.toClassName())); note: the module passed here `attrEnv.toplevel.modle` is *not* the module the type we are looking for belongs to. It is the module of the current compilation unit -- in other words, this module is used to determine which packages are visible from the compilation unit. ------------- PR Review Comment: https://git.openjdk.org/babylon/pull/391#discussion_r2037149944 From mcimadamore at openjdk.org Thu Apr 10 11:43:28 2025 From: mcimadamore at openjdk.org (Maurizio Cimadamore) Date: Thu, 10 Apr 2025 11:43:28 GMT Subject: [code-reflection] RFR: Improve resolution of types in `CodeModelToAST` Message-ID: The `CodeModelToAST` class features an ad-hoc table of types, which is hard-wired to include all the types that can come up in the code model generated by `OpBuilder`. This approach is fragile (if new types are relied upon by `OpBuilder`, they need to be added there), and duplicates type resolution tables that javac has to maintain anyway. This PR replaces the ad-hoc type table with more idiomatic javac type lookup logic. ------------- Commit messages: - Initial push Changes: https://git.openjdk.org/babylon/pull/391/files Webrev: https://webrevs.openjdk.org/?repo=babylon&pr=391&range=00 Stats: 64 lines in 1 file changed: 8 ins; 52 del; 4 mod Patch: https://git.openjdk.org/babylon/pull/391.diff Fetch: git fetch https://git.openjdk.org/babylon.git pull/391/head:pull/391 PR: https://git.openjdk.org/babylon/pull/391 From asotona at openjdk.org Thu Apr 10 14:40:09 2025 From: asotona at openjdk.org (Adam Sotona) Date: Thu, 10 Apr 2025 14:40:09 GMT Subject: [code-reflection] RFR: Support for Onnx functions Message-ID: Support for Onnx functions: - Onnx model is represented by `ModuleOp`. - Initializers are self-described by `OnnxType.Initializer` type (no more by-passing initializers values from `OnnxTransformer`). - `OnnxTransformer` is decomposed into small independent transformations. - Single-use functions are inlined, multiple-use functions are declared in the model. - Protobuf model is improved to better reflect Onnx model. Sample Onnx model: module ()void -> { func @"step" (%0 : tensor, %1 : tensor, %2 : init, oracle.code.onnx.WalkTheMazeTest.directionEast>, %3 : init, oracle.code.onnx.WalkTheMazeTest.stepEast>, %4 : init, oracle.code.onnx.WalkTheMazeTest.directionNorth>, %5 : init, oracle.code.onnx.WalkTheMazeTest.stepNorth>, %6 : init, oracle.code.onnx.WalkTheMazeTest.directionWest>, %7 : init, oracle.code.onnx.WalkTheMazeTest.stepWest>, %8 : init, oracle.code.onnx.WalkTheMazeTest.stepSouth>)tensor -> { %9 : tensor = Equal %1 %2; %10 : tensor = If %9 ()tensor -> { %11 : tensor = Add %0 %3; return %11 @loc="102:17"; } ()tensor -> { %12 : tensor = Equal %1 %4; %13 : tensor = If %12 ()tensor -> { %14 : tensor = Add %0 %5; return %14 @loc="104:25"; } ()tensor -> { %15 : tensor = Equal %1 %6; %16 : tensor = If %15 ()tensor -> { %17 : tensor = Add %0 %7; return %17 @loc="106:29"; } ()tensor -> { %18 : tensor = Add %0 %8; return %18 @loc="107:29"; }; return %16 @loc="105:25"; }; return %13 @loc="103:17"; }; return %10 @loc="101:9"; }; func @"turnLeft" (%19 : tensor, %20 : init, oracle.code.onnx.WalkTheMazeTest.directionEast>, %21 : init, oracle.code.onnx.WalkTheMazeTest.directionNorth>, %22 : init, oracle.code.onnx.WalkTheMazeTest.directionWest>, %23 : init, oracle.code.onnx.WalkTheMazeTest.directionSouth>)tensor -> { %24 : tensor = Equal %19 %20; %25 : tensor = If %24 ()tensor -> { %26 : tensor = Identity %21; return %26 @loc="80:17"; } ()tensor -> { %27 : tensor = Equal %19 %21; %28 : tensor = If %27 ()tensor -> { %29 : tensor = Identity %22; return %29 @loc="82:25"; } ()tensor -> { %30 : tensor = Equal %19 %22; %31 : tensor = If %30 ()tensor -> { %32 : tensor = Identity %23; return %32 @loc="84:29"; } ()tensor -> { %33 : tensor = Identity %20; return %33 @loc="85:29"; }; return %31 @loc="83:25"; }; return %28 @loc="81:17"; }; return %25 @loc="79:9"; }; func @"isWallAt" (%34 : tensor, %35 : init, oracle.code.onnx.WalkTheMazeTest.maze>, %36 : init, oracle.code.onnx.WalkTheMazeTest.oneOne>, %37 : init, oracle.code.onnx.WalkTheMazeTest.wall>)tensor -> { %38 : tensor = Add %34 %36; %39 : tensor = Slice %35 %34 %38; %40 : tensor = CastLike %39 %37; %41 : tensor = Equal %40 %37; return %41 @loc="96:9"; }; func @"walkAroundTheMaze" (%42 : init, oracle.code.onnx.WalkTheMazeTest.directionEast>, %43 : init, oracle.code.onnx.WalkTheMazeTest.stepEast>, %44 : init, oracle.code.onnx.WalkTheMazeTest.directionNorth>, %45 : init, oracle.code.onnx.WalkTheMazeTest.stepNorth>, %46 : init, oracle.code.onnx.WalkTheMazeTest.directionWest>, %47 : init, oracle.code.onnx.WalkTheMazeTest.stepWest>, %48 : init, oracle.code.onnx.WalkTheMazeTest.stepSouth>, %49 : init, oracle.code.onnx.WalkTheMazeTest.directionSouth>, %50 : init, oracle.code.onnx.WalkTheMazeTest.maze>, %51 : init, oracle.code.onnx.WalkTheMazeTest.oneOne>, %52 : init, oracle.code.onnx.WalkTheMazeTest.wall>, %53 : init, oracle.code.onnx.WalkTheMazeTest.homePos>, %54 : init, oracle.code.onnx.WalkTheMazeTest.limit>, %55 : init, oracle.code.onnx.WalkT heMazeTest._true>, %56 : init, oracle.code.onnx.WalkTheMazeTest.three>, %57 : init, oracle.code.onnx.WalkTheMazeTest.scalarShape>)tensor -> { %58 : tensor = Cast %42 @to="2"; %59 : Tuple, oracle.code.onnx.WalkTheMazeTest.homePos>, init, oracle.code.onnx.WalkTheMazeTest.directionEast>, tensor> = tuple %53 %42 %58; %60 : Tuple, tensor, tensor> = Loop %54 %55 %59 (%61 : tensor, %62 : tensor, %63 : Tuple, tensor, tensor>)Tuple, Tuple, tensor, tensor>> -> { %64 : tensor = tuple.load %63 @"0"; %65 : tensor = tuple.load %63 @"1"; %66 : tensor = func.call %64 %65 %42 %43 %44 %45 %46 %47 %48 @"step"; %67 : tensor = tuple.load %63 @"1"; %68 : tensor = Loop %56 %55 %67 (%69 : tensor, %70 : tensor, %71 : tensor)Tuple, tensor> -> { %72 : tensor = func.call %71 %42 %44 %46 %49 @"turnLeft"; %73 : Tuple, tensor> = tuple %70 %72; return %73 @loc="90:46"; }; %74 : tensor = func.call %66 %68 %42 %43 %44 %45 %46 %47 %48 @"step"; %75 : tensor = func.call %74 %50 %51 %52 @"isWallAt"; %76 : tensor = Reshape %75 %57; %77 : tensor = Loop %54 %76 %68 (%78 : tensor, %79 : tensor, %80 : tensor)Tuple, tensor> -> { %81 : tensor = func.call %80 %42 %44 %46 %49 @"turnLeft"; %82 : tensor = func.call %66 %81 %42 %43 %44 %45 %46 %47 %48 @"step"; %83 : tensor = func.call %82 %50 %51 %52 @"isWallAt"; %84 : Tuple, tensor> = tuple %83 %81; return %84 @loc="120:17"; }; %85 : tensor = Equal %66 %53; %86 : tensor = ReduceMin %85; %87 : tensor = Not %86; %88 : tensor = tuple.load %63 @"2"; %89 : tensor = Cast %77 @to="2"; %90 : tensor = Concat %88 %89 @axis="0"; %91 : Tuple, tensor, tensor> = tuple %66 %77 %90; %92 : Tuple, Tuple, tensor, tensor>> = tuple %87 %91; return %92 @loc="140:13"; }; %93 : tensor = tuple.load %60 @"2"; return %93 @loc="148:69"; }; unreachable; }; Protobuf model fragment of the sample Onnx model: ![WalkTheMaze](https://github.com/user-attachments/assets/ca7a8288-bd4e-42d4-967a-fc560a425344) ------------- Commit messages: - OnnxProtoBuilder.Indexer improved to match computed global names by OpWriter - OnnxProtoBuilder support for custom domain name and named graphs + cleanup - model main func name calculation - iniling single-called functions - OnnxTransformer cleanup - Initializers encoded into OnnxType.Initializer - OnnxProtoBuilder support for ModuleOp - work in progress - OnnxProtoBuilder support for ModuleOp - work in progress - fixed SimpleTest::testConcat to avoid non-tensor in args - Onnx model functions into module - work in progress - ... and 1 more: https://git.openjdk.org/babylon/compare/bd0569ee...3c68af28 Changes: https://git.openjdk.org/babylon/pull/378/files Webrev: https://webrevs.openjdk.org/?repo=babylon&pr=378&range=00 Stats: 487 lines in 8 files changed: 315 ins; 78 del; 94 mod Patch: https://git.openjdk.org/babylon/pull/378.diff Fetch: git fetch https://git.openjdk.org/babylon.git pull/378/head:pull/378 PR: https://git.openjdk.org/babylon/pull/378 From gfrost at openjdk.org Thu Apr 10 15:13:26 2025 From: gfrost at openjdk.org (Gary Frost) Date: Thu, 10 Apr 2025 15:13:26 GMT Subject: git: openjdk/babylon: code-reflection: Enabled MINIMIZE_COPIES for ffi-cuda backend Message-ID: <7da9a384-f595-4892-840e-931ef78124cf@openjdk.org> Changeset: 6e2d0993 Branch: code-reflection Author: Gary Frost Date: 2025-04-10 15:12:12 +0000 URL: https://git.openjdk.org/babylon/commit/6e2d099326d9cd524e94f9b18c1d801ff679462b Enabled MINIMIZE_COPIES for ffi-cuda backend ! hat/backends/ffi/cuda/cpp/cuda_backend_buffer.cpp ! hat/backends/ffi/cuda/cpp/cuda_backend_queue.cpp ! hat/backends/ffi/cuda/include/cuda_backend.h ! hat/backends/ffi/opencl/src/main/java/hat/backend/ffi/OpenCLBackend.java ! hat/backends/ffi/shared/src/main/java/hat/backend/ffi/C99FFIBackend.java From gfrost at openjdk.org Thu Apr 10 15:15:10 2025 From: gfrost at openjdk.org (Gary Frost) Date: Thu, 10 Apr 2025 15:15:10 GMT Subject: [code-reflection] Integrated: Enabled MINIMIZE_COPIES for ffi-cuda backend Message-ID: I enabled minimize copies mode for ffi-cuda. Some apps seem to benefit (mandel and violajones). Healing brush still an issue, it seems the driver backends do not like enqueuing copies (to or from device) on a thread different from the one where thee 'cuStream' was created (heal example is the only one that does this - as it creates backend in main thread but dispatches kernels in the swing thread). More investigation is needed. ------------- Commit messages: - Enabled MINIMIZE_COPIES for ffi-cuda backend Changes: https://git.openjdk.org/babylon/pull/392/files Webrev: https://webrevs.openjdk.org/?repo=babylon&pr=392&range=00 Stats: 215 lines in 5 files changed: 129 ins; 77 del; 9 mod Patch: https://git.openjdk.org/babylon/pull/392.diff Fetch: git fetch https://git.openjdk.org/babylon.git pull/392/head:pull/392 PR: https://git.openjdk.org/babylon/pull/392 From gfrost at openjdk.org Thu Apr 10 15:15:11 2025 From: gfrost at openjdk.org (Gary Frost) Date: Thu, 10 Apr 2025 15:15:11 GMT Subject: [code-reflection] Integrated: Enabled MINIMIZE_COPIES for ffi-cuda backend In-Reply-To: References: Message-ID: On Thu, 10 Apr 2025 15:10:32 GMT, Gary Frost wrote: > I enabled minimize copies mode for ffi-cuda. Some apps seem to benefit (mandel and violajones). > > > Healing brush still an issue, it seems the driver backends do not like enqueuing copies (to or from device) on a thread different from the one where thee 'cuStream' was created (heal example is the only one that does this - as it creates backend in main thread but dispatches kernels in the swing thread). > > More investigation is needed. This pull request has now been integrated. Changeset: 6e2d0993 Author: Gary Frost URL: https://git.openjdk.org/babylon/commit/6e2d099326d9cd524e94f9b18c1d801ff679462b Stats: 215 lines in 5 files changed: 129 ins; 77 del; 9 mod Enabled MINIMIZE_COPIES for ffi-cuda backend ------------- PR: https://git.openjdk.org/babylon/pull/392 From mabbay at openjdk.org Thu Apr 10 16:32:53 2025 From: mabbay at openjdk.org (Mourad Abbay) Date: Thu, 10 Apr 2025 16:32:53 GMT Subject: [code-reflection] RFR: Improve resolution of types in `CodeModelToAST` In-Reply-To: References: Message-ID: <076Ug70ccmEPZyx5RSoZXGZ2mnrSS55enD8-iKoXDzs=.52f06e23-264d-4f50-bf84-d44fa9a567b1@github.com> On Thu, 10 Apr 2025 11:37:16 GMT, Maurizio Cimadamore wrote: > The `CodeModelToAST` class features an ad-hoc table of types, which is hard-wired to include all the types that can come up in the code model generated by `OpBuilder`. > This approach is fragile (if new types are relied upon by `OpBuilder`, they need to be added there), and duplicates type resolution tables that javac has to maintain anyway. > > This PR replaces the ad-hoc type table with more idiomatic javac type lookup logic. src/jdk.incubator.code/share/classes/jdk/incubator/code/internal/CodeModelToAST.java line 48: > 46: > 47: private Type typeElementToType(TypeElement te) { > 48: JavaType jt = (JavaType) te; I suggest we remove the cast and switch over the param `te` ------------- PR Review Comment: https://git.openjdk.org/babylon/pull/391#discussion_r2037816364 From mcimadamore at openjdk.org Fri Apr 11 09:55:43 2025 From: mcimadamore at openjdk.org (Maurizio Cimadamore) Date: Fri, 11 Apr 2025 09:55:43 GMT Subject: git: openjdk/babylon: code-reflection: Improve resolution of types in `CodeModelToAST` Message-ID: Changeset: 1a28d369 Branch: code-reflection Author: Maurizio Cimadamore Date: 2025-04-11 09:55:09 +0000 URL: https://git.openjdk.org/babylon/commit/1a28d3693feff089bb111c89a23d5b073dc95c1e Improve resolution of types in `CodeModelToAST` ! src/jdk.incubator.code/share/classes/jdk/incubator/code/internal/CodeModelToAST.java From mcimadamore at openjdk.org Fri Apr 11 09:59:32 2025 From: mcimadamore at openjdk.org (Maurizio Cimadamore) Date: Fri, 11 Apr 2025 09:59:32 GMT Subject: [code-reflection] RFR: Improve resolution of types in `CodeModelToAST` [v2] In-Reply-To: References: Message-ID: > The `CodeModelToAST` class features an ad-hoc table of types, which is hard-wired to include all the types that can come up in the code model generated by `OpBuilder`. > This approach is fragile (if new types are relied upon by `OpBuilder`, they need to be added there), and duplicates type resolution tables that javac has to maintain anyway. > > This PR replaces the ad-hoc type table with more idiomatic javac type lookup logic. Maurizio Cimadamore has updated the pull request incrementally with one additional commit since the last revision: Address review comments ------------- Changes: - all: https://git.openjdk.org/babylon/pull/391/files - new: https://git.openjdk.org/babylon/pull/391/files/01eacfce..3e5f5b6e Webrevs: - full: https://webrevs.openjdk.org/?repo=babylon&pr=391&range=01 - incr: https://webrevs.openjdk.org/?repo=babylon&pr=391&range=00-01 Stats: 3 lines in 1 file changed: 0 ins; 1 del; 2 mod Patch: https://git.openjdk.org/babylon/pull/391.diff Fetch: git fetch https://git.openjdk.org/babylon.git pull/391/head:pull/391 PR: https://git.openjdk.org/babylon/pull/391 From mcimadamore at openjdk.org Fri Apr 11 09:59:33 2025 From: mcimadamore at openjdk.org (Maurizio Cimadamore) Date: Fri, 11 Apr 2025 09:59:33 GMT Subject: [code-reflection] Integrated: Improve resolution of types in `CodeModelToAST` In-Reply-To: References: Message-ID: On Thu, 10 Apr 2025 11:37:16 GMT, Maurizio Cimadamore wrote: > The `CodeModelToAST` class features an ad-hoc table of types, which is hard-wired to include all the types that can come up in the code model generated by `OpBuilder`. > This approach is fragile (if new types are relied upon by `OpBuilder`, they need to be added there), and duplicates type resolution tables that javac has to maintain anyway. > > This PR replaces the ad-hoc type table with more idiomatic javac type lookup logic. This pull request has now been integrated. Changeset: 1a28d369 Author: Maurizio Cimadamore URL: https://git.openjdk.org/babylon/commit/1a28d3693feff089bb111c89a23d5b073dc95c1e Stats: 66 lines in 1 file changed: 8 ins; 53 del; 5 mod Improve resolution of types in `CodeModelToAST` ------------- PR: https://git.openjdk.org/babylon/pull/391 From mcimadamore at openjdk.org Fri Apr 11 12:17:59 2025 From: mcimadamore at openjdk.org (Maurizio Cimadamore) Date: Fri, 11 Apr 2025 12:17:59 GMT Subject: [code-reflection] RFR: Further cleanup of `CodeModelToAST` and regularize support for varargs instance creation expressions Message-ID: This PR makes the code in `CodeModelToAST` that deal with instance creation expression (`NewOp`) a bit closer to the code that deals with method calls (`InvokeOp`). I've added a method, called `constructorTypeToSymbol` which takes the constructor type of a `NewOp` and looks up the constructor symbol with that type. Then I moved the logic that sets `varargElements` on the created `JCMethodInvocation` nodes into a separate, reusable routine. This is now called also for varargs instance creation expression. Of course, to do that I had to add support for a new varargs attribute in the `NewOp` class -- which closely follows the one used by `InvokeOp`. This means that in `ReflectMethod` we can now reify in the model as to whether a constructor call was varargs or not. I've also added some arg count validation in `NewOp`, very similar to the one that appears in `InvokeOp`. ------------- Commit messages: - Add validation - Initial push Changes: https://git.openjdk.org/babylon/pull/393/files Webrev: https://webrevs.openjdk.org/?repo=babylon&pr=393&range=00 Stats: 194 lines in 7 files changed: 151 ins; 14 del; 29 mod Patch: https://git.openjdk.org/babylon/pull/393.diff Fetch: git fetch https://git.openjdk.org/babylon.git pull/393/head:pull/393 PR: https://git.openjdk.org/babylon/pull/393 From mabbay at openjdk.org Fri Apr 11 16:35:47 2025 From: mabbay at openjdk.org (Mourad Abbay) Date: Fri, 11 Apr 2025 16:35:47 GMT Subject: [code-reflection] RFR: Further cleanup of `CodeModelToAST` and regularize support for varargs instance creation expressions In-Reply-To: References: Message-ID: <1KJ964q93UyVlvcOq97UwfQ8ZwNHrCQEhWOJqYyWK9A=.8a94c4cc-4c7b-42e1-8d4e-5a074b0f4602@github.com> On Fri, 11 Apr 2025 12:03:04 GMT, Maurizio Cimadamore wrote: > This PR makes the code in `CodeModelToAST` that deal with instance creation expression (`NewOp`) a bit closer to the code that deals with method calls (`InvokeOp`). I've added a method, called `constructorTypeToSymbol` which takes the constructor type of a `NewOp` and looks up the constructor symbol with that type. > > Then I moved the logic that sets `varargElements` on the created `JCMethodInvocation` nodes into a separate, reusable routine. This is now called also for varargs instance creation expression. > > Of course, to do that I had to add support for a new varargs attribute in the `NewOp` class -- which closely follows the one used by `InvokeOp`. This means that in `ReflectMethod` we can now reify in the model as to whether a constructor call was varargs or not. > > I've also added some arg count validation in `NewOp`, very similar to the one that appears in `InvokeOp`. src/jdk.incubator.code/share/classes/jdk/incubator/code/internal/CodeModelToAST.java line 87: > 85: setVarargs(methodInvocation, invokeOp.invokeDescriptor().type()); > 86: } > 87: if (invokeOp.result().uses().isEmpty()) { This part was removed by a previous PR. Now it's going to be reintroduced. ------------- PR Review Comment: https://git.openjdk.org/babylon/pull/393#discussion_r2039899322 From mabbay at openjdk.org Fri Apr 11 17:03:45 2025 From: mabbay at openjdk.org (Mourad Abbay) Date: Fri, 11 Apr 2025 17:03:45 GMT Subject: [code-reflection] RFR: Further cleanup of `CodeModelToAST` and regularize support for varargs instance creation expressions In-Reply-To: References: Message-ID: On Fri, 11 Apr 2025 12:03:04 GMT, Maurizio Cimadamore wrote: > This PR makes the code in `CodeModelToAST` that deal with instance creation expression (`NewOp`) a bit closer to the code that deals with method calls (`InvokeOp`). I've added a method, called `constructorTypeToSymbol` which takes the constructor type of a `NewOp` and looks up the constructor symbol with that type. > > Then I moved the logic that sets `varargElements` on the created `JCMethodInvocation` nodes into a separate, reusable routine. This is now called also for varargs instance creation expression. > > Of course, to do that I had to add support for a new varargs attribute in the `NewOp` class -- which closely follows the one used by `InvokeOp`. This means that in `ReflectMethod` we can now reify in the model as to whether a constructor call was varargs or not. > > I've also added some arg count validation in `NewOp`, very similar to the one that appears in `InvokeOp`. src/jdk.incubator.code/share/classes/jdk/incubator/code/internal/CodeModelToAST.java line 108: > 106: case CoreOp.ConstantOp constantOp -> treeMaker.Literal(constantOp.value()); > 107: case CoreOp.InvokeOp invokeOp -> invokeOpToJCMethodInvocation(invokeOp); > 108: case CoreOp.NewOp newOp when newOp.resultType() instanceof ArrayType at -> { Shouldn't we have a method like `boolean NewOp.isArrayCreation()` ? ------------- PR Review Comment: https://git.openjdk.org/babylon/pull/393#discussion_r2039939212 From mcimadamore at openjdk.org Fri Apr 11 17:24:41 2025 From: mcimadamore at openjdk.org (Maurizio Cimadamore) Date: Fri, 11 Apr 2025 17:24:41 GMT Subject: [code-reflection] RFR: Further cleanup of `CodeModelToAST` and regularize support for varargs instance creation expressions In-Reply-To: <1KJ964q93UyVlvcOq97UwfQ8ZwNHrCQEhWOJqYyWK9A=.8a94c4cc-4c7b-42e1-8d4e-5a074b0f4602@github.com> References: <1KJ964q93UyVlvcOq97UwfQ8ZwNHrCQEhWOJqYyWK9A=.8a94c4cc-4c7b-42e1-8d4e-5a074b0f4602@github.com> Message-ID: On Fri, 11 Apr 2025 16:32:38 GMT, Mourad Abbay wrote: >> This PR makes the code in `CodeModelToAST` that deal with instance creation expression (`NewOp`) a bit closer to the code that deals with method calls (`InvokeOp`). I've added a method, called `constructorTypeToSymbol` which takes the constructor type of a `NewOp` and looks up the constructor symbol with that type. >> >> Then I moved the logic that sets `varargElements` on the created `JCMethodInvocation` nodes into a separate, reusable routine. This is now called also for varargs instance creation expression. >> >> Of course, to do that I had to add support for a new varargs attribute in the `NewOp` class -- which closely follows the one used by `InvokeOp`. This means that in `ReflectMethod` we can now reify in the model as to whether a constructor call was varargs or not. >> >> I've also added some arg count validation in `NewOp`, very similar to the one that appears in `InvokeOp`. > > src/jdk.incubator.code/share/classes/jdk/incubator/code/internal/CodeModelToAST.java line 87: > >> 85: setVarargs(methodInvocation, invokeOp.invokeDescriptor().type()); >> 86: } >> 87: if (invokeOp.result().uses().isEmpty()) { > > This part was removed by a previous PR. Now it's going to be reintroduced. good catch - seems like a merge issue > src/jdk.incubator.code/share/classes/jdk/incubator/code/internal/CodeModelToAST.java line 108: > >> 106: case CoreOp.ConstantOp constantOp -> treeMaker.Literal(constantOp.value()); >> 107: case CoreOp.InvokeOp invokeOp -> invokeOpToJCMethodInvocation(invokeOp); >> 108: case CoreOp.NewOp newOp when newOp.resultType() instanceof ArrayType at -> { > > Shouldn't we have a method like `boolean NewOp.isArrayCreation()` ? Yeah - this will probably be helpful -- but I'd prefer that to be addressed separately ------------- PR Review Comment: https://git.openjdk.org/babylon/pull/393#discussion_r2039970993 PR Review Comment: https://git.openjdk.org/babylon/pull/393#discussion_r2039971894 From mcimadamore at openjdk.org Fri Apr 11 17:36:31 2025 From: mcimadamore at openjdk.org (Maurizio Cimadamore) Date: Fri, 11 Apr 2025 17:36:31 GMT Subject: [code-reflection] RFR: Further cleanup of `CodeModelToAST` and regularize support for varargs instance creation expressions [v2] In-Reply-To: References: Message-ID: > This PR makes the code in `CodeModelToAST` that deal with instance creation expression (`NewOp`) a bit closer to the code that deals with method calls (`InvokeOp`). I've added a method, called `constructorTypeToSymbol` which takes the constructor type of a `NewOp` and looks up the constructor symbol with that type. > > Then I moved the logic that sets `varargElements` on the created `JCMethodInvocation` nodes into a separate, reusable routine. This is now called also for varargs instance creation expression. > > Of course, to do that I had to add support for a new varargs attribute in the `NewOp` class -- which closely follows the one used by `InvokeOp`. This means that in `ReflectMethod` we can now reify in the model as to whether a constructor call was varargs or not. > > I've also added some arg count validation in `NewOp`, very similar to the one that appears in `InvokeOp`. Maurizio Cimadamore has updated the pull request incrementally with one additional commit since the last revision: Address review comments ------------- Changes: - all: https://git.openjdk.org/babylon/pull/393/files - new: https://git.openjdk.org/babylon/pull/393/files/1d1056c9..9fe9b812 Webrevs: - full: https://webrevs.openjdk.org/?repo=babylon&pr=393&range=01 - incr: https://webrevs.openjdk.org/?repo=babylon&pr=393&range=00-01 Stats: 3 lines in 1 file changed: 0 ins; 3 del; 0 mod Patch: https://git.openjdk.org/babylon/pull/393.diff Fetch: git fetch https://git.openjdk.org/babylon.git pull/393/head:pull/393 PR: https://git.openjdk.org/babylon/pull/393 From mabbay at openjdk.org Fri Apr 11 18:51:38 2025 From: mabbay at openjdk.org (Mourad Abbay) Date: Fri, 11 Apr 2025 18:51:38 GMT Subject: [code-reflection] RFR: Further cleanup of `CodeModelToAST` and regularize support for varargs instance creation expressions [v2] In-Reply-To: References: Message-ID: <7MuWlDzkRN3FYbtZdoCWmqxG28z8gHrYXx1qm0F0YNs=.4c784119-360f-481e-a5e3-46a417298ad0@github.com> On Fri, 11 Apr 2025 17:36:31 GMT, Maurizio Cimadamore wrote: >> This PR makes the code in `CodeModelToAST` that deal with instance creation expression (`NewOp`) a bit closer to the code that deals with method calls (`InvokeOp`). I've added a method, called `constructorTypeToSymbol` which takes the constructor type of a `NewOp` and looks up the constructor symbol with that type. >> >> Then I moved the logic that sets `varargElements` on the created `JCMethodInvocation` nodes into a separate, reusable routine. This is now called also for varargs instance creation expression. >> >> Of course, to do that I had to add support for a new varargs attribute in the `NewOp` class -- which closely follows the one used by `InvokeOp`. This means that in `ReflectMethod` we can now reify in the model as to whether a constructor call was varargs or not. >> >> I've also added some arg count validation in `NewOp`, very similar to the one that appears in `InvokeOp`. > > Maurizio Cimadamore has updated the pull request incrementally with one additional commit since the last revision: > > Address review comments Look good to me ------------- PR Comment: https://git.openjdk.org/babylon/pull/393#issuecomment-2797774224 From asotona at openjdk.org Mon Apr 14 13:12:47 2025 From: asotona at openjdk.org (Adam Sotona) Date: Mon, 14 Apr 2025 13:12:47 GMT Subject: [code-reflection] RFR: Support for Onnx functions [v2] In-Reply-To: References: Message-ID: > Support for Onnx functions: > - Onnx model is represented by `ModuleOp`. > - Initializers are self-described by `OnnxType.Initializer` type (no more by-passing initializers values from `OnnxTransformer`). > - `OnnxTransformer` is decomposed into small independent transformations. > - Single-use functions are inlined, multiple-use functions are declared in the model. > - Protobuf model is improved to better reflect Onnx model. > > Sample Onnx model: > > module ()void -> { > func @"step" (%0 : tensor, %1 : tensor, %2 : init, oracle.code.onnx.WalkTheMazeTest.directionEast>, %3 : init, oracle.code.onnx.WalkTheMazeTest.stepEast>, %4 : init, oracle.code.onnx.WalkTheMazeTest.directionNorth>, %5 : init, oracle.code.onnx.WalkTheMazeTest.stepNorth>, %6 : init, oracle.code.onnx.WalkTheMazeTest.directionWest>, %7 : init, oracle.code.onnx.WalkTheMazeTest.stepWest>, %8 : init, oracle.code.onnx.WalkTheMazeTest.stepSouth>)tensor -> { > %9 : tensor = Equal %1 %2; > %10 : tensor = If %9 > ()tensor -> { > %11 : tensor = Add %0 %3; > return %11 @loc="102:17"; > } > ()tensor -> { > %12 : tensor = Equal %1 %4; > %13 : tensor = If %12 > ()tensor -> { > %14 : tensor = Add %0 %5; > return %14 @loc="104:25"; > } > ()tensor -> { > %15 : tensor = Equal %1 %6; > %16 : tensor = If %15 > ()tensor -> { > %17 : tensor = Add %0 %7; > return %17 @loc="106:29"; > } > ()tensor -> { > %18 : tensor = Add %0 %8; > return %18 @loc="107:29"; > }; > return %16 @loc="105:25"; > }; > return %13 @loc="103:17"; > }; > return %10 @loc="101:9"; > }; > func @"turnLeft" (%19 : tensor, %20 : init, oracle.code.onnx.WalkTheMazeTest.directionEast>, %21 : init, oracle.code.onnx.WalkTheMazeTest.directionNorth>, %22 : init, ... Adam Sotona has updated the pull request incrementally with one additional commit since the last revision: Protobuf model input and output names prefixed with % to match code model names. ------------- Changes: - all: https://git.openjdk.org/babylon/pull/378/files - new: https://git.openjdk.org/babylon/pull/378/files/3c68af28..1b97c6d6 Webrevs: - full: https://webrevs.openjdk.org/?repo=babylon&pr=378&range=01 - incr: https://webrevs.openjdk.org/?repo=babylon&pr=378&range=00-01 Stats: 2 lines in 2 files changed: 0 ins; 1 del; 1 mod Patch: https://git.openjdk.org/babylon/pull/378.diff Fetch: git fetch https://git.openjdk.org/babylon.git pull/378/head:pull/378 PR: https://git.openjdk.org/babylon/pull/378 From mabbay at openjdk.org Mon Apr 14 23:40:29 2025 From: mabbay at openjdk.org (Mourad Abbay) Date: Mon, 14 Apr 2025 23:40:29 GMT Subject: [code-reflection] RFR: Add ConstructorRef Message-ID: This is Maurizio's work. This PR tries to make `NewOp` similar to how `InvokeOp` is designed. It does this by adding a descriptor for constructors: `ConstructorRef`. `ConstructorRef` is wrapper around `FunctionType`, providing functionalities for resolution to a reflective instance. Some changes are common between this PR and #393. ------------- Commit messages: - Remove trailing whitespaces - No need to pass refType during creation of ConstructorRef - Merge branch 'code-reflection' into new_op_constr_ref - Enhance ConstructorRef::resolveToHandle to cover array creation - Merge branch 'code-reflection' into new_op_constr_ref - Merge branch 'code-reflection' into new_op_constr_ref - Clean up CodeModelToAST - Fix tests - Initial push - Cleanup code in CodeModelToAST Changes: https://git.openjdk.org/babylon/pull/395/files Webrev: https://webrevs.openjdk.org/?repo=babylon&pr=395&range=00 Stats: 569 lines in 28 files changed: 386 ins; 65 del; 118 mod Patch: https://git.openjdk.org/babylon/pull/395.diff Fetch: git fetch https://git.openjdk.org/babylon.git pull/395/head:pull/395 PR: https://git.openjdk.org/babylon/pull/395 From mabbay at openjdk.org Mon Apr 14 23:44:51 2025 From: mabbay at openjdk.org (Mourad Abbay) Date: Mon, 14 Apr 2025 23:44:51 GMT Subject: [code-reflection] RFR: Add ConstructorRef [v2] In-Reply-To: References: Message-ID: > This is Maurizio's work. > This PR tries to make `NewOp` similar to how `InvokeOp` is designed. It does this by adding a descriptor for constructors: `ConstructorRef`. > `ConstructorRef` is wrapper around `FunctionType`, providing functionalities for resolution to a reflective instance. > Some changes are common between this PR and #393. Mourad Abbay has updated the pull request incrementally with one additional commit since the last revision: Remove code that was deleted by a previous PR ------------- Changes: - all: https://git.openjdk.org/babylon/pull/395/files - new: https://git.openjdk.org/babylon/pull/395/files/2acdd441..9e815f03 Webrevs: - full: https://webrevs.openjdk.org/?repo=babylon&pr=395&range=01 - incr: https://webrevs.openjdk.org/?repo=babylon&pr=395&range=00-01 Stats: 3 lines in 1 file changed: 0 ins; 3 del; 0 mod Patch: https://git.openjdk.org/babylon/pull/395.diff Fetch: git fetch https://git.openjdk.org/babylon.git pull/395/head:pull/395 PR: https://git.openjdk.org/babylon/pull/395 From asotona at openjdk.org Tue Apr 15 04:49:01 2025 From: asotona at openjdk.org (Adam Sotona) Date: Tue, 15 Apr 2025 04:49:01 GMT Subject: [code-reflection] Integrated: Support for Onnx functions In-Reply-To: References: Message-ID: On Fri, 4 Apr 2025 09:18:54 GMT, Adam Sotona wrote: > Support for Onnx functions: > - Onnx model is represented by `ModuleOp`. > - Initializers are self-described by `OnnxType.Initializer` type (no more by-passing initializers values from `OnnxTransformer`). > - `OnnxTransformer` is decomposed into small independent transformations. > - Single-use functions are inlined, multiple-use functions are declared in the model. > - Protobuf model is improved to better reflect Onnx model. > > Sample Onnx model: > > module ()void -> { > func @"step" (%0 : tensor, %1 : tensor, %2 : init, oracle.code.onnx.WalkTheMazeTest.directionEast>, %3 : init, oracle.code.onnx.WalkTheMazeTest.stepEast>, %4 : init, oracle.code.onnx.WalkTheMazeTest.directionNorth>, %5 : init, oracle.code.onnx.WalkTheMazeTest.stepNorth>, %6 : init, oracle.code.onnx.WalkTheMazeTest.directionWest>, %7 : init, oracle.code.onnx.WalkTheMazeTest.stepWest>, %8 : init, oracle.code.onnx.WalkTheMazeTest.stepSouth>)tensor -> { > %9 : tensor = Equal %1 %2; > %10 : tensor = If %9 > ()tensor -> { > %11 : tensor = Add %0 %3; > return %11 @loc="102:17"; > } > ()tensor -> { > %12 : tensor = Equal %1 %4; > %13 : tensor = If %12 > ()tensor -> { > %14 : tensor = Add %0 %5; > return %14 @loc="104:25"; > } > ()tensor -> { > %15 : tensor = Equal %1 %6; > %16 : tensor = If %15 > ()tensor -> { > %17 : tensor = Add %0 %7; > return %17 @loc="106:29"; > } > ()tensor -> { > %18 : tensor = Add %0 %8; > return %18 @loc="107:29"; > }; > return %16 @loc="105:25"; > }; > return %13 @loc="103:17"; > }; > return %10 @loc="101:9"; > }; > func @"turnLeft" (%19 : tensor, %20 : init, oracle.code.onnx.WalkTheMazeTest.directionEast>, %21 : init, oracle.code.onnx.WalkTheMazeTest.directionNorth>, %22 : init, ... This pull request has now been integrated. Changeset: ddad895d Author: Adam Sotona URL: https://git.openjdk.org/babylon/commit/ddad895d66529fdd297b7a02219f19c7e79238cf Stats: 486 lines in 8 files changed: 314 ins; 78 del; 94 mod Support for Onnx functions ------------- PR: https://git.openjdk.org/babylon/pull/378 From asotona at openjdk.org Tue Apr 15 04:49:03 2025 From: asotona at openjdk.org (Adam Sotona) Date: Tue, 15 Apr 2025 04:49:03 GMT Subject: git: openjdk/babylon: code-reflection: Support for Onnx functions Message-ID: <452b03b8-c5d1-4ce2-ba71-a1999682f033@openjdk.org> Changeset: ddad895d Branch: code-reflection Author: Adam Sotona Date: 2025-04-15 04:46:22 +0000 URL: https://git.openjdk.org/babylon/commit/ddad895d66529fdd297b7a02219f19c7e79238cf Support for Onnx functions ! cr-examples/onnx/src/main/java/oracle/code/onnx/OnnxProtoBuilder.java ! cr-examples/onnx/src/main/java/oracle/code/onnx/OnnxRuntime.java ! cr-examples/onnx/src/main/java/oracle/code/onnx/compiler/OnnxPartialEvaluator.java ! cr-examples/onnx/src/main/java/oracle/code/onnx/compiler/OnnxTransformer.java ! cr-examples/onnx/src/main/java/oracle/code/onnx/ir/OnnxType.java ! cr-examples/onnx/src/test/java/oracle/code/onnx/CNNTest.java ! cr-examples/onnx/src/test/java/oracle/code/onnx/RuntimeTest.java ! cr-examples/onnx/src/test/java/oracle/code/onnx/SimpleTest.java From psandoz at openjdk.org Tue Apr 15 16:47:56 2025 From: psandoz at openjdk.org (Paul Sandoz) Date: Tue, 15 Apr 2025 16:47:56 GMT Subject: [code-reflection] RFR: Add ConstructorRef [v2] In-Reply-To: References: Message-ID: On Mon, 14 Apr 2025 23:44:51 GMT, Mourad Abbay wrote: >> This is Maurizio's work. >> This PR tries to make `NewOp` similar to how `InvokeOp` is designed. It does this by adding a descriptor for constructors: `ConstructorRef`. >> `ConstructorRef` is wrapper around `FunctionType`, providing functionalities for resolution to a reflective instance. >> Some changes are common between this PR and #393. > > Mourad Abbay has updated the pull request incrementally with one additional commit since the last revision: > > Remove code that was deleted by a previous PR Looking good. Can you also please test the examples and HAT? src/jdk.incubator.code/share/classes/jdk/incubator/code/op/CoreOp.java line 1690: > 1688: this(isVarArgs, resultType, > 1689: ConstructorRef.constructor(constructorType), args); > 1690: } Let's remove these methods, so the constructors only accept `ConstructorRef` src/jdk.incubator.code/share/classes/jdk/incubator/code/op/CoreOp.java line 1724: > 1722: public FunctionType constructorType() { > 1723: return constructorDescriptor.type(); > 1724: } We can remove this method. src/jdk.incubator.code/share/classes/jdk/incubator/code/type/ConstructorRef.java line 44: > 42: public sealed interface ConstructorRef extends TypeVarRef.Owner permits ConstructorRefImpl { > 43: > 44: TypeElement refType(); I am unsure about this method, but make it a default method that returns the result of `type().returnType()`, then the implementation does not need to do anything special. We might need to revisit the terminology, but we can do that later. src/jdk.incubator.code/share/classes/jdk/incubator/code/type/ConstructorRef.java line 65: > 63: static ConstructorRef constructor(Class refType, MethodType mt) { > 64: return constructor(refType, mt.parameterList()); > 65: } Drop the `refType` parameter and use `mt.returnType()` src/jdk.incubator.code/share/classes/jdk/incubator/code/type/ConstructorRef.java line 77: > 75: static ConstructorRef constructor(TypeElement refType, List params) { > 76: return constructor(functionType(refType, params)); > 77: } Add an overloaded method of signature `(TypeElement refType, TypeElement... params)` src/jdk.incubator.code/share/classes/jdk/incubator/code/type/CoreTypeFactory.java line 131: > 129: constructTypeArgument(tree, 0, NO_WILDCARDS)); > 130: } else if (parts.length == 3) { > 131: // method or constructor type-var This strongly indicates we need to make refs be type elements, perhaps extending from say `JavaRef`, representing the common super class for symbolic references to Java things distinct from a `JavaType`. They formally become part of the code model and then we can manage this more uniformly. We can revisit later (including the distinction between structured string encodings and human readable encodings for Java types and refs). ------------- PR Review: https://git.openjdk.org/babylon/pull/395#pullrequestreview-2769064565 PR Review Comment: https://git.openjdk.org/babylon/pull/395#discussion_r2045053285 PR Review Comment: https://git.openjdk.org/babylon/pull/395#discussion_r2045055643 PR Review Comment: https://git.openjdk.org/babylon/pull/395#discussion_r2045024340 PR Review Comment: https://git.openjdk.org/babylon/pull/395#discussion_r2045029812 PR Review Comment: https://git.openjdk.org/babylon/pull/395#discussion_r2045032566 PR Review Comment: https://git.openjdk.org/babylon/pull/395#discussion_r2045048385 From psandoz at openjdk.org Tue Apr 15 16:51:00 2025 From: psandoz at openjdk.org (Paul Sandoz) Date: Tue, 15 Apr 2025 16:51:00 GMT Subject: [code-reflection] RFR: Add ConstructorRef [v2] In-Reply-To: References: Message-ID: On Mon, 14 Apr 2025 23:44:51 GMT, Mourad Abbay wrote: >> This is Maurizio's work. >> This PR tries to make `NewOp` similar to how `InvokeOp` is designed. It does this by adding a descriptor for constructors: `ConstructorRef`. >> `ConstructorRef` is wrapper around `FunctionType`, providing functionalities for resolution to a reflective instance. >> Some changes are common between this PR and #393. > > Mourad Abbay has updated the pull request incrementally with one additional commit since the last revision: > > Remove code that was deleted by a previous PR src/jdk.incubator.code/share/classes/jdk/incubator/code/op/CoreOp.java line 3932: > 3930: * @return the instance creation operation > 3931: */ > 3932: public static NewOp _new(boolean isVarargs, TypeElement returnType, ConstructorRef constructorDescriptor, Update all the factory methods to accept `ConstructorRef` instead of `FunctionType`. ------------- PR Review Comment: https://git.openjdk.org/babylon/pull/395#discussion_r2045060683 From gfrost at openjdk.org Wed Apr 16 11:06:41 2025 From: gfrost at openjdk.org (Gary Frost) Date: Wed, 16 Apr 2025 11:06:41 GMT Subject: [code-reflection] Integrated: Hat separate jextract builds In-Reply-To: References: Message-ID: On Wed, 16 Apr 2025 10:56:31 GMT, Gary Frost wrote: > This is an infrastructure/build change. > > Instead of relying on script 'capabilities' we now use cmake to jextract. This makes more sense because cmake 'knows' where the include/lib files are. > > Previous capabilities used cmake under the hood, but we did not have a (clean) jextract solution for maven. > > Now we can delegate to cmake 'to extract' artifacts (in the same way we delegate to build ffi backends) > > I will make the pom changes as a separate PR. This pull request has now been integrated. Changeset: d6a24cf1 Author: Gary Frost URL: https://git.openjdk.org/babylon/commit/d6a24cf10c1ca82722cd6ade2a2db12ccf4c4419 Stats: 1290 lines in 32 files changed: 678 ins; 479 del; 133 mod Hat separate jextract builds ------------- PR: https://git.openjdk.org/babylon/pull/397 From gfrost at openjdk.org Wed Apr 16 11:06:40 2025 From: gfrost at openjdk.org (Gary Frost) Date: Wed, 16 Apr 2025 11:06:40 GMT Subject: [code-reflection] Integrated: Hat separate jextract builds Message-ID: This is an infrastructure/build change. Instead of relying on script 'capabilities' we now use cmake to jextract. This makes more sense because cmake 'knows' where the include/lib files are. Previous capabilities used cmake under the hood, but we did not have a (clean) jextract solution for maven. Now we can delegate to cmake 'to extract' artifacts (in the same way we delegate to build ffi backends) I will make the pom changes as a separate PR. ------------- Commit messages: - whitespace - using cmake to extract cuda, opencl and opengl - cleaned up bld.java - moved jextract out of main build Changes: https://git.openjdk.org/babylon/pull/397/files Webrev: https://webrevs.openjdk.org/?repo=babylon&pr=397&range=00 Stats: 1290 lines in 32 files changed: 678 ins; 479 del; 133 mod Patch: https://git.openjdk.org/babylon/pull/397.diff Fetch: git fetch https://git.openjdk.org/babylon.git pull/397/head:pull/397 PR: https://git.openjdk.org/babylon/pull/397 From gfrost at openjdk.org Wed Apr 16 11:06:51 2025 From: gfrost at openjdk.org (Gary Frost) Date: Wed, 16 Apr 2025 11:06:51 GMT Subject: git: openjdk/babylon: code-reflection: Hat separate jextract builds Message-ID: Changeset: d6a24cf1 Branch: code-reflection Author: Gary Frost Date: 2025-04-16 11:03:07 +0000 URL: https://git.openjdk.org/babylon/commit/d6a24cf10c1ca82722cd6ade2a2db12ccf4c4419 Hat separate jextract builds ! cr-examples/onnx/README.md ! cr-examples/spirv/README.md ! cr-examples/triton/README.md ! hat/backends/ffi/mock/CMakeLists.txt ! hat/backends/ffi/opencl/CMakeLists.txt - hat/backends/ffi/shared/src/main/resources/META-INF/services/hat.backend.Backend = hat/backends/java/mt/src/main/java/hat/backend/java/JavaMultiThreadedBackend.java = hat/backends/java/seq/src/main/java/hat/backend/java/JavaSequentialBackend.java ! hat/backends/java/seq/src/main/resources/META-INF/services/hat.backend.Backend ! hat/docs/hat-04-02-cascade-interface-mapping.md ! hat/docs/hat-06-kernel-analysis.md ! hat/examples/nbody/src/main/java/nbody/Main.java ! hat/examples/nbody/src/main/java/nbody/opencl/OpenCLNBodyGLWindow.java ! hat/examples/violajones/src/main/java/violajones/attic/ViolaJones.java ! hat/extractions/.gitignore + hat/extractions/CMakeLists.txt + hat/extractions/README.txt + hat/extractions/cuda/CMakeLists.txt + hat/extractions/opencl/CMakeLists.txt - hat/extractions/opencl/pom.xml + hat/extractions/opengl/CMakeLists.txt - hat/extractions/opengl/pom.xml - hat/extractions/pom.xml ! hat/hat-core/src/main/java/hat/backend/Backend.java ! hat/hat-core/src/main/java/hat/ifacemapper/BufferState.java ! hat/hat-core/src/main/test/hat/SquaresTest.java ! hat/hat/Script.java ! hat/hat/bld.java + hat/hat/extract + hat/hat/extract.java ! hat/hat/run.java ! hat/hat/sanity.java From gfrost at openjdk.org Wed Apr 16 15:07:25 2025 From: gfrost at openjdk.org (Gary Frost) Date: Wed, 16 Apr 2025 15:07:25 GMT Subject: git: openjdk/babylon: code-reflection: updated cuda build to use cmake extractions Message-ID: <05ce4d1c-3a7c-4357-9996-1fb2385669d3@openjdk.org> Changeset: 6e5284aa Branch: code-reflection Author: Gary Frost Date: 2025-04-16 15:03:52 +0000 URL: https://git.openjdk.org/babylon/commit/6e5284aa0ad5d192ce030c255a01f2a79ca83a45 updated cuda build to use cmake extractions ! hat/backends/ffi/cuda/CMakeLists.txt ! hat/extractions/CMakeLists.txt ! hat/extractions/cuda/CMakeLists.txt ! hat/extractions/opengl/CMakeLists.txt ! hat/hat/bld.java From gfrost at openjdk.org Wed Apr 16 15:07:39 2025 From: gfrost at openjdk.org (Gary Frost) Date: Wed, 16 Apr 2025 15:07:39 GMT Subject: [code-reflection] Integrated: updated cuda build to use cmake extractions Message-ID: <4o_A84ayxSSykPpyfhlg6fMo3Srf_kVYfqbTVtsiJ0g=.da7fec63-cd24-47ed-89c6-475e961827d7@github.com> This is just update from CUDA bld to allow it to use cmake to jextract cuda and opengl. Finally builds from `java @hat/bld` are consitant between mac and linux ------------- Commit messages: - updated cuda build to use cmake extractions Changes: https://git.openjdk.org/babylon/pull/398/files Webrev: https://webrevs.openjdk.org/?repo=babylon&pr=398&range=00 Stats: 80 lines in 5 files changed: 47 ins; 8 del; 25 mod Patch: https://git.openjdk.org/babylon/pull/398.diff Fetch: git fetch https://git.openjdk.org/babylon.git pull/398/head:pull/398 PR: https://git.openjdk.org/babylon/pull/398 From gfrost at openjdk.org Wed Apr 16 15:07:40 2025 From: gfrost at openjdk.org (Gary Frost) Date: Wed, 16 Apr 2025 15:07:40 GMT Subject: [code-reflection] Integrated: updated cuda build to use cmake extractions In-Reply-To: <4o_A84ayxSSykPpyfhlg6fMo3Srf_kVYfqbTVtsiJ0g=.da7fec63-cd24-47ed-89c6-475e961827d7@github.com> References: <4o_A84ayxSSykPpyfhlg6fMo3Srf_kVYfqbTVtsiJ0g=.da7fec63-cd24-47ed-89c6-475e961827d7@github.com> Message-ID: On Wed, 16 Apr 2025 15:01:35 GMT, Gary Frost wrote: > This is just update from CUDA bld to allow it to use cmake to jextract cuda and opengl. > > Finally builds from `java @hat/bld` are consitant between mac and linux This pull request has now been integrated. Changeset: 6e5284aa Author: Gary Frost URL: https://git.openjdk.org/babylon/commit/6e5284aa0ad5d192ce030c255a01f2a79ca83a45 Stats: 80 lines in 5 files changed: 47 ins; 8 del; 25 mod updated cuda build to use cmake extractions ------------- PR: https://git.openjdk.org/babylon/pull/398 From psandoz at openjdk.org Wed Apr 16 15:42:08 2025 From: psandoz at openjdk.org (Paul Sandoz) Date: Wed, 16 Apr 2025 15:42:08 GMT Subject: [code-reflection] RFR: Removal of OnnxType.Initializer type In-Reply-To: References: Message-ID: On Wed, 16 Apr 2025 09:43:37 GMT, Adam Sotona wrote: > This PR removes `OnnxType.Initializer` type and it passes sequence of `FieldRef` of the initializers directly from `OnnxTransformer` to `OnnxRuntime` instead. Marked as reviewed by psandoz (Lead). ------------- PR Review: https://git.openjdk.org/babylon/pull/396#pullrequestreview-2772994522 From asotona at openjdk.org Wed Apr 16 16:17:17 2025 From: asotona at openjdk.org (Adam Sotona) Date: Wed, 16 Apr 2025 16:17:17 GMT Subject: git: openjdk/babylon: code-reflection: Removal of OnnxType.Initializer type Message-ID: <8bbc8e36-bedb-4ec2-93a5-086a103c62cc@openjdk.org> Changeset: ca5db647 Branch: code-reflection Author: Adam Sotona Date: 2025-04-16 16:16:26 +0000 URL: https://git.openjdk.org/babylon/commit/ca5db64727bd2f35c269912e59dcef5a69209b57 Removal of OnnxType.Initializer type Reviewed-by: psandoz ! cr-examples/onnx/src/main/java/oracle/code/onnx/OnnxRuntime.java ! cr-examples/onnx/src/main/java/oracle/code/onnx/compiler/OnnxTransformer.java ! cr-examples/onnx/src/main/java/oracle/code/onnx/ir/OnnxType.java ! cr-examples/onnx/src/test/java/oracle/code/onnx/CNNTest.java From asotona at openjdk.org Wed Apr 16 16:20:01 2025 From: asotona at openjdk.org (Adam Sotona) Date: Wed, 16 Apr 2025 16:20:01 GMT Subject: [code-reflection] RFR: Removal of OnnxType.Initializer type In-Reply-To: References: Message-ID: <38IXjaHTp-eLxJ2D-z9oOeW0qiMWLqQutdmNydtuCwo=.ce18876d-547c-424d-99fe-232179efaa1e@github.com> On Wed, 16 Apr 2025 09:43:37 GMT, Adam Sotona wrote: > This PR removes `OnnxType.Initializer` type and it passes sequence of `FieldRef` of the initializers directly from `OnnxTransformer` to `OnnxRuntime` instead. Thank you for the review. ------------- PR Comment: https://git.openjdk.org/babylon/pull/396#issuecomment-2810075316 From asotona at openjdk.org Wed Apr 16 16:20:01 2025 From: asotona at openjdk.org (Adam Sotona) Date: Wed, 16 Apr 2025 16:20:01 GMT Subject: [code-reflection] Integrated: Removal of OnnxType.Initializer type In-Reply-To: References: Message-ID: On Wed, 16 Apr 2025 09:43:37 GMT, Adam Sotona wrote: > This PR removes `OnnxType.Initializer` type and it passes sequence of `FieldRef` of the initializers directly from `OnnxTransformer` to `OnnxRuntime` instead. This pull request has now been integrated. Changeset: ca5db647 Author: Adam Sotona URL: https://git.openjdk.org/babylon/commit/ca5db64727bd2f35c269912e59dcef5a69209b57 Stats: 67 lines in 4 files changed: 6 ins; 44 del; 17 mod Removal of OnnxType.Initializer type Reviewed-by: psandoz ------------- PR: https://git.openjdk.org/babylon/pull/396 From gfrost at openjdk.org Thu Apr 17 10:48:55 2025 From: gfrost at openjdk.org (Gary Frost) Date: Thu, 17 Apr 2025 10:48:55 GMT Subject: git: openjdk/babylon: code-reflection: Git use cmake jextract in maven Message-ID: Changeset: 0f39dd1c Branch: code-reflection Author: Gary Frost Date: 2025-04-17 10:47:59 +0000 URL: https://git.openjdk.org/babylon/commit/0f39dd1cd1bd4fedeabd4d919beffde6e36f7d00 Git use cmake jextract in maven + hat/examples/nbody/pom.xml ! hat/examples/pom.xml ! hat/extractions/CMakeLists.txt = hat/extractions/opencl/pom.xml = hat/extractions/opengl/pom.xml + hat/extractions/pom.xml ! hat/hat/sanity.java ! hat/maven-build.bash ! hat/pom.xml ! hat/wrap/glwrap/pom.xml From gfrost at openjdk.org Thu Apr 17 10:50:41 2025 From: gfrost at openjdk.org (Gary Frost) Date: Thu, 17 Apr 2025 10:50:41 GMT Subject: [code-reflection] Integrated: Git use cmake jextract in maven Message-ID: Previously we added cmake artifacts (CMakeFiles.txt) to opencl, opengl and cuda subdirs in extractions. Here we added the resulting java source files to the maven pom hierarchy. Upshot of which, we (on Mac with OpenCL) use maven to build everything.. well nearly, we still need to manually run cmake So this should work bash maven-build.bash java @hat/run ffi-opencl nbody Note that the maven build bash script actually has a new step to run cmake on extractions dir echo We need to run cmake in extractions dir first pushd extractions rm -rf cmake-build-debug cmake -B cmake-build-debug cmake --build cmake-build-debug --target extract popd echo Now we can use maven mvn -e clean compile package install ``` I will remove this and put it into the maven build, when I grok how to add a source generator to maven. ------------- Commit messages: - Arrange for maven-build.bash to call cmake in extractions dir - Allow maven to use cmake extracted files Changes: https://git.openjdk.org/babylon/pull/400/files Webrev: https://webrevs.openjdk.org/?repo=babylon&pr=400&range=00 Stats: 171 lines in 10 files changed: 150 ins; 2 del; 19 mod Patch: https://git.openjdk.org/babylon/pull/400.diff Fetch: git fetch https://git.openjdk.org/babylon.git pull/400/head:pull/400 PR: https://git.openjdk.org/babylon/pull/400 From gfrost at openjdk.org Thu Apr 17 10:50:43 2025 From: gfrost at openjdk.org (Gary Frost) Date: Thu, 17 Apr 2025 10:50:43 GMT Subject: [code-reflection] Integrated: Git use cmake jextract in maven In-Reply-To: References: Message-ID: On Thu, 17 Apr 2025 10:45:56 GMT, Gary Frost wrote: > Previously we added cmake artifacts (CMakeFiles.txt) to opencl, opengl and cuda subdirs in extractions. > > Here we added the resulting java source files to the maven pom hierarchy. > > Upshot of which, we (on Mac with OpenCL) use maven to build everything.. well nearly, we still need to manually run cmake > > So this should work > > bash maven-build.bash > java @hat/run ffi-opencl nbody > > Note that the maven build bash script actually has a new step to run cmake on extractions dir > > > echo We need to run cmake in extractions dir first > pushd extractions > rm -rf cmake-build-debug > cmake -B cmake-build-debug > cmake --build cmake-build-debug --target extract > popd > echo Now we can use maven > mvn -e clean compile package install > ``` > > I will remove this and put it into the maven build, when I grok how to add a source generator to maven. This pull request has now been integrated. Changeset: 0f39dd1c Author: Gary Frost URL: https://git.openjdk.org/babylon/commit/0f39dd1cd1bd4fedeabd4d919beffde6e36f7d00 Stats: 171 lines in 10 files changed: 150 ins; 2 del; 19 mod Git use cmake jextract in maven ------------- PR: https://git.openjdk.org/babylon/pull/400 From asotona at openjdk.org Thu Apr 17 12:55:39 2025 From: asotona at openjdk.org (Adam Sotona) Date: Thu, 17 Apr 2025 12:55:39 GMT Subject: [code-reflection] RFR: OnnxProtoModel implementation Message-ID: Changes in this PR: - `OnnxProtoModel` implementation replacing and enhancing basic `OnnxProtoPrinter` functionality - added missing copyright headers ------------- Commit messages: - copyright headers - fixed OnnxProtoModel - Merge remote-tracking branch 'babylon/code-reflection' into onnx-parser - OnnxProtoModel implementation replaces OnnxProtoPrinter - draft of ModelProto - draft of OnnxProtoParser - draft of OnnxProtoParser - draft of OnnxProtoParser - OnnxProtoPrinter improvements Changes: https://git.openjdk.org/babylon/pull/399/files Webrev: https://webrevs.openjdk.org/?repo=babylon&pr=399&range=00 Stats: 787 lines in 7 files changed: 531 ins; 256 del; 0 mod Patch: https://git.openjdk.org/babylon/pull/399.diff Fetch: git fetch https://git.openjdk.org/babylon.git pull/399/head:pull/399 PR: https://git.openjdk.org/babylon/pull/399 From asotona at openjdk.org Thu Apr 17 13:23:43 2025 From: asotona at openjdk.org (Adam Sotona) Date: Thu, 17 Apr 2025 13:23:43 GMT Subject: [code-reflection] RFR: OnnxProtoModel implementation [v2] In-Reply-To: References: Message-ID: > Changes in this PR: > - `OnnxProtoModel` implementation replacing and enhancing basic `OnnxProtoPrinter` functionality > - added missing copyright headers Adam Sotona has updated the pull request incrementally with one additional commit since the last revision: OnnxProtoModel::toText improved to allow dump all data ------------- Changes: - all: https://git.openjdk.org/babylon/pull/399/files - new: https://git.openjdk.org/babylon/pull/399/files/9748e144..a1930237 Webrevs: - full: https://webrevs.openjdk.org/?repo=babylon&pr=399&range=01 - incr: https://webrevs.openjdk.org/?repo=babylon&pr=399&range=00-01 Stats: 16 lines in 1 file changed: 6 ins; 0 del; 10 mod Patch: https://git.openjdk.org/babylon/pull/399.diff Fetch: git fetch https://git.openjdk.org/babylon.git pull/399/head:pull/399 PR: https://git.openjdk.org/babylon/pull/399 From psandoz at openjdk.org Thu Apr 17 23:07:33 2025 From: psandoz at openjdk.org (Paul Sandoz) Date: Thu, 17 Apr 2025 23:07:33 GMT Subject: [code-reflection] RFR: Merge master Message-ID: Merging with master, updating to Java 25, requiring 24 as a boot JDK. All examples and HAT are also updated. All tests pass. I found an issue with compilation of the test [TextMatrix.java](https://github.com/openjdk/babylon/blob/code-reflection/cr-examples/triton/src/test/java/oracle/code/triton/TestMatrix.java). It looks like the encoding of the code model as a method that builds the code model is reaching class file method size limits. So for now i have switched the default encoding back to `TEXT` until this is resolved. ------------- Commit messages: - Merge remote-tracking branch 'upstream/master' into merge-master - 8353655: Clean up and open source KeyEvent related tests (Part 1) - 8354789: Unnecessary null check in sun.awt.windows.WToolkit.getFontPeer - 8354464: Additional cleanup setting up native.encoding - 8354541: Remove Shenandoah post barrier expand loop opts - 8354552: Open source a few Swing tests - 8354451: Open source some more Swing popup menu tests - 8354106: Clean up and open source KeyEvent related tests (Part 2) - 8354522: Clones of DecimalFormat cause interferences when used concurrently - 8353070: Clean up and open source couple AWT Graphics related tests (Part 1) - ... and 2793 more: https://git.openjdk.org/babylon/compare/0f39dd1c...1154738c The webrevs contain the adjustments done while merging with regards to each parent branch: - code-reflection: https://webrevs.openjdk.org/?repo=babylon&pr=401&range=00.0 - master: https://webrevs.openjdk.org/?repo=babylon&pr=401&range=00.1 Changes: https://git.openjdk.org/babylon/pull/401/files Stats: 1241316 lines in 16416 files changed: 577041 ins; 565734 del; 98541 mod Patch: https://git.openjdk.org/babylon/pull/401.diff Fetch: git fetch https://git.openjdk.org/babylon.git pull/401/head:pull/401 PR: https://git.openjdk.org/babylon/pull/401 From gfrost at openjdk.org Fri Apr 18 09:21:09 2025 From: gfrost at openjdk.org (Gary Frost) Date: Fri, 18 Apr 2025 09:21:09 GMT Subject: git: openjdk/babylon: code-reflection: Hat more maven pom cleanup Message-ID: <1e14bd84-fbd3-4ab9-80e3-1909c9bbce6f@openjdk.org> Changeset: 7807eff2 Branch: code-reflection Author: Gary Frost Date: 2025-04-18 09:19:02 +0000 URL: https://git.openjdk.org/babylon/commit/7807eff2c27b9fc7cba534a291439370566018cf Hat more maven pom cleanup ! hat/backends/ffi/pom.xml ! hat/examples/pom.xml ! hat/extractions/CMakeLists.txt = hat/extractions/cuda/pom.xml ! hat/extractions/opencl/CMakeLists.txt ! hat/extractions/opencl/pom.xml ! hat/extractions/opengl/CMakeLists.txt ! hat/extractions/opengl/pom.xml ! hat/extractions/pom.xml - hat/hat/extract - hat/hat/extract.java ! hat/maven-build.bash ! hat/wrap/pom.xml From gfrost at openjdk.org Fri Apr 18 09:23:08 2025 From: gfrost at openjdk.org (Gary Frost) Date: Fri, 18 Apr 2025 09:23:08 GMT Subject: [code-reflection] Integrated: Hat more maven pom cleanup Message-ID: <3MkNkvI-Yv01haux80SSKUjOZmS3XeAQlJYPsP5D_EA=.f9e954e9-480d-4872-a945-78c2c9608dd6@github.com> Discovered how to run cmake from maven in generate-sources phase. So we no longer have to call cmake manually before building with maven. ------------- Commit messages: - added -Popengl,opencl activations - cmake jextract added to maven pom.xml generate-sources - Commenting out dumping of vars - ffi backends use ffi-cmake-build-debug dir - create activations for opencl,opengl and cuda Changes: https://git.openjdk.org/babylon/pull/402/files Webrev: https://webrevs.openjdk.org/?repo=babylon&pr=402&range=00 Stats: 436 lines in 13 files changed: 181 ins; 199 del; 56 mod Patch: https://git.openjdk.org/babylon/pull/402.diff Fetch: git fetch https://git.openjdk.org/babylon.git pull/402/head:pull/402 PR: https://git.openjdk.org/babylon/pull/402 From gfrost at openjdk.org Fri Apr 18 09:23:08 2025 From: gfrost at openjdk.org (Gary Frost) Date: Fri, 18 Apr 2025 09:23:08 GMT Subject: [code-reflection] Integrated: Hat more maven pom cleanup In-Reply-To: <3MkNkvI-Yv01haux80SSKUjOZmS3XeAQlJYPsP5D_EA=.f9e954e9-480d-4872-a945-78c2c9608dd6@github.com> References: <3MkNkvI-Yv01haux80SSKUjOZmS3XeAQlJYPsP5D_EA=.f9e954e9-480d-4872-a945-78c2c9608dd6@github.com> Message-ID: <9-Hg7IbCPh0q_q0AQcoJz_rkflA675_Wob1FRXRX-L8=.d6f313b5-44af-45b2-99d4-c3189aceefd4@github.com> On Fri, 18 Apr 2025 09:16:54 GMT, Gary Frost wrote: > Discovered how to run cmake from maven in generate-sources phase. So we no longer have to call cmake manually before building with maven. This pull request has now been integrated. Changeset: 7807eff2 Author: Gary Frost URL: https://git.openjdk.org/babylon/commit/7807eff2c27b9fc7cba534a291439370566018cf Stats: 436 lines in 13 files changed: 181 ins; 199 del; 56 mod Hat more maven pom cleanup ------------- PR: https://git.openjdk.org/babylon/pull/402 From mabbay at openjdk.org Fri Apr 18 15:58:42 2025 From: mabbay at openjdk.org (Mourad Abbay) Date: Fri, 18 Apr 2025 15:58:42 GMT Subject: [code-reflection] RFR: Add ConstructorRef [v3] In-Reply-To: References: Message-ID: <2Akw4wEO8NyLOdJUDJppc6cMSttGy8ri6Ib38Gvflp4=.fcd60ac1-1093-4cab-9e37-c845fc244d9c@github.com> > This is Maurizio's work. > This PR tries to make `NewOp` similar to how `InvokeOp` is designed. It does this by adding a descriptor for constructors: `ConstructorRef`. > `ConstructorRef` is wrapper around `FunctionType`, providing functionalities for resolution to a reflective instance. > Some changes are common between this PR and #393. Mourad Abbay has updated the pull request incrementally with one additional commit since the last revision: Apply Paul's comments ------------- Changes: - all: https://git.openjdk.org/babylon/pull/395/files - new: https://git.openjdk.org/babylon/pull/395/files/9e815f03..a49f2790 Webrevs: - full: https://webrevs.openjdk.org/?repo=babylon&pr=395&range=02 - incr: https://webrevs.openjdk.org/?repo=babylon&pr=395&range=01-02 Stats: 182 lines in 11 files changed: 9 ins; 139 del; 34 mod Patch: https://git.openjdk.org/babylon/pull/395.diff Fetch: git fetch https://git.openjdk.org/babylon.git pull/395/head:pull/395 PR: https://git.openjdk.org/babylon/pull/395 From psandoz at openjdk.org Fri Apr 18 16:05:39 2025 From: psandoz at openjdk.org (Paul Sandoz) Date: Fri, 18 Apr 2025 16:05:39 GMT Subject: [code-reflection] RFR: Merge master [v2] In-Reply-To: References: Message-ID: <12yuLsX-iXVK_F6Suf9RHGMP88g7-hWTQVUYLssxJug=.0af24173-ee7b-48fc-b015-7346504b1708@github.com> > Merging with master, updating to Java 25, requiring 24 as a boot JDK. > > All examples and HAT are also updated. All tests pass. > > I found an issue with compilation of the test [TextMatrix.java](https://github.com/openjdk/babylon/blob/code-reflection/cr-examples/triton/src/test/java/oracle/code/triton/TestMatrix.java). It looks like the encoding of the code model as a method that builds the code model is reaching class file method size limits. So for now i have switched the default encoding back to `TEXT` until this is resolved. Paul Sandoz has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 2804 commits: - Merge remote-tracking branch 'upstream/code-reflection' into merge-master - Merge remote-tracking branch 'upstream/master' into merge-master - 8353655: Clean up and open source KeyEvent related tests (Part 1) Reviewed-by: abhiscxk - 8354789: Unnecessary null check in sun.awt.windows.WToolkit.getFontPeer Reviewed-by: aivanov, tr - 8354464: Additional cleanup setting up native.encoding Reviewed-by: naoto, alanb - 8354541: Remove Shenandoah post barrier expand loop opts Reviewed-by: shade - 8354552: Open source a few Swing tests Reviewed-by: dnguyen, psadhukhan - 8354451: Open source some more Swing popup menu tests Reviewed-by: jdv, kizune - 8354106: Clean up and open source KeyEvent related tests (Part 2) Reviewed-by: abhiscxk, prr - 8354522: Clones of DecimalFormat cause interferences when used concurrently Reviewed-by: jlu, naoto - ... and 2794 more: https://git.openjdk.org/babylon/compare/7807eff2...667ffbff ------------- Changes: https://git.openjdk.org/babylon/pull/401/files Webrev: Webrev is not available because diff is too large Stats: 1241313 lines in 16414 files changed: 577041 ins; 565734 del; 98538 mod Patch: https://git.openjdk.org/babylon/pull/401.diff Fetch: git fetch https://git.openjdk.org/babylon.git pull/401/head:pull/401 PR: https://git.openjdk.org/babylon/pull/401 From psandoz at openjdk.org Fri Apr 18 19:14:18 2025 From: psandoz at openjdk.org (Paul Sandoz) Date: Fri, 18 Apr 2025 19:14:18 GMT Subject: [code-reflection] Integrated: Merge master In-Reply-To: References: Message-ID: <6Ud81lvoUqbyugNtG_6HEuYJkf3WXON6zcupaMj57Oo=.f7c46e79-eabd-42e2-9a50-9768eab77246@github.com> On Thu, 17 Apr 2025 23:02:36 GMT, Paul Sandoz wrote: > Merging with master, updating to Java 25, requiring 24 as a boot JDK. > > All examples and HAT are also updated. All tests pass. > > I found an issue with compilation of the test [TextMatrix.java](https://github.com/openjdk/babylon/blob/code-reflection/cr-examples/triton/src/test/java/oracle/code/triton/TestMatrix.java). It looks like the encoding of the code model as a method that builds the code model is reaching class file method size limits. So for now i have switched the default encoding back to `TEXT` until this is resolved. This pull request has now been integrated. Changeset: 9ca6a424 Author: Paul Sandoz URL: https://git.openjdk.org/babylon/commit/9ca6a4243794b974562c43ed6072dc0575c418ba Stats: 1241313 lines in 16414 files changed: 577041 ins; 565734 del; 98538 mod Merge ------------- PR: https://git.openjdk.org/babylon/pull/401 From psandoz at openjdk.org Fri Apr 18 19:36:56 2025 From: psandoz at openjdk.org (Paul Sandoz) Date: Fri, 18 Apr 2025 19:36:56 GMT Subject: [code-reflection] RFR: Add ConstructorRef [v3] In-Reply-To: <2Akw4wEO8NyLOdJUDJppc6cMSttGy8ri6Ib38Gvflp4=.fcd60ac1-1093-4cab-9e37-c845fc244d9c@github.com> References: <2Akw4wEO8NyLOdJUDJppc6cMSttGy8ri6Ib38Gvflp4=.fcd60ac1-1093-4cab-9e37-c845fc244d9c@github.com> Message-ID: On Fri, 18 Apr 2025 15:58:42 GMT, Mourad Abbay wrote: >> This is Maurizio's work. >> This PR tries to make `NewOp` similar to how `InvokeOp` is designed. It does this by adding a descriptor for constructors: `ConstructorRef`. >> `ConstructorRef` is wrapper around `FunctionType`, providing functionalities for resolution to a reflective instance. >> Some changes are common between this PR and #393. > > Mourad Abbay has updated the pull request incrementally with one additional commit since the last revision: > > Apply Paul's comments Marked as reviewed by psandoz (Lead). ------------- PR Review: https://git.openjdk.org/babylon/pull/395#pullrequestreview-2779291657 From mabbay at openjdk.org Fri Apr 18 20:36:14 2025 From: mabbay at openjdk.org (Mourad Abbay) Date: Fri, 18 Apr 2025 20:36:14 GMT Subject: git: openjdk/babylon: code-reflection: Add ConstructorRef Message-ID: <1dd53115-c476-4ea8-b375-6be1a3c1da25@openjdk.org> Changeset: 5238a40d Branch: code-reflection Author: Mourad Abbay Date: 2025-04-18 20:33:33 +0000 URL: https://git.openjdk.org/babylon/commit/5238a40d11397b9896756a674158378223405388 Add ConstructorRef Reviewed-by: psandoz ! src/jdk.incubator.code/share/classes/jdk/incubator/code/analysis/StringConcatTransformer.java ! src/jdk.incubator.code/share/classes/jdk/incubator/code/bytecode/BytecodeGenerator.java ! src/jdk.incubator.code/share/classes/jdk/incubator/code/bytecode/BytecodeLift.java ! src/jdk.incubator.code/share/classes/jdk/incubator/code/bytecode/UnresolvedTypesTransformer.java ! src/jdk.incubator.code/share/classes/jdk/incubator/code/internal/CodeModelToAST.java ! src/jdk.incubator.code/share/classes/jdk/incubator/code/internal/ReflectMethods.java ! src/jdk.incubator.code/share/classes/jdk/incubator/code/interpreter/Interpreter.java ! src/jdk.incubator.code/share/classes/jdk/incubator/code/op/CoreOp.java ! src/jdk.incubator.code/share/classes/jdk/incubator/code/op/ExtendedOp.java ! src/jdk.incubator.code/share/classes/jdk/incubator/code/parser/impl/DescParser.java + src/jdk.incubator.code/share/classes/jdk/incubator/code/type/ConstructorRef.java ! src/jdk.incubator.code/share/classes/jdk/incubator/code/type/CoreTypeFactory.java ! src/jdk.incubator.code/share/classes/jdk/incubator/code/type/JavaType.java ! src/jdk.incubator.code/share/classes/jdk/incubator/code/type/MethodRef.java ! src/jdk.incubator.code/share/classes/jdk/incubator/code/type/TypeVarRef.java + src/jdk.incubator.code/share/classes/jdk/incubator/code/type/impl/ConstructorRefImpl.java ! src/jdk.incubator.code/share/classes/jdk/incubator/code/writer/OpBuilder.java - test/jdk/java/lang/reflect/code/TestAddVarsWhenNecessary.java ! test/jdk/java/lang/reflect/code/type/TestReferences.java ! test/langtools/tools/javac/reflect/BlockTest.java ! test/langtools/tools/javac/reflect/BoxingConversionTest.java ! test/langtools/tools/javac/reflect/FieldAccessTest.java ! test/langtools/tools/javac/reflect/ImplicitConversionTest.java ! test/langtools/tools/javac/reflect/LocalClassTest.java ! test/langtools/tools/javac/reflect/MethodReferenceTest.java ! test/langtools/tools/javac/reflect/NewArrayTest.java ! test/langtools/tools/javac/reflect/NewTest.java ! test/langtools/tools/javac/reflect/NullTest.java ! test/langtools/tools/javac/reflect/QuotableIntersectionTest.java ! test/langtools/tools/javac/reflect/QuotableSubtypeTest.java ! test/langtools/tools/javac/reflect/QuotedTest.java ! test/langtools/tools/javac/reflect/SwitchExpressionTest2.java ! test/langtools/tools/javac/reflect/SwitchStatementTest.java ! test/langtools/tools/javac/reflect/ThrowTest.java From mabbay at openjdk.org Fri Apr 18 20:36:14 2025 From: mabbay at openjdk.org (Mourad Abbay) Date: Fri, 18 Apr 2025 20:36:14 GMT Subject: [code-reflection] Integrated: Add ConstructorRef In-Reply-To: References: Message-ID: On Mon, 14 Apr 2025 23:33:33 GMT, Mourad Abbay wrote: > This is Maurizio's work. > This PR tries to make `NewOp` similar to how `InvokeOp` is designed. It does this by adding a descriptor for constructors: `ConstructorRef`. > `ConstructorRef` is wrapper around `FunctionType`, providing functionalities for resolution to a reflective instance. > Some changes are common between this PR and #393. This pull request has now been integrated. Changeset: 5238a40d Author: Mourad Abbay URL: https://git.openjdk.org/babylon/commit/5238a40d11397b9896756a674158378223405388 Stats: 727 lines in 34 files changed: 386 ins; 198 del; 143 mod Add ConstructorRef Reviewed-by: psandoz ------------- PR: https://git.openjdk.org/babylon/pull/395 From gfrost at openjdk.org Sat Apr 19 15:38:22 2025 From: gfrost at openjdk.org (Gary Frost) Date: Sat, 19 Apr 2025 15:38:22 GMT Subject: git: openjdk/babylon: code-reflection: Cleanup bld.java and Script.java Reconstructed - from failed merge Message-ID: <03329d6a-6d5f-4471-97ca-05f7600f1a4e@openjdk.org> Changeset: 8e81823c Branch: code-reflection Author: Gary Frost Date: 2025-04-19 15:36:45 +0000 URL: https://git.openjdk.org/babylon/commit/8e81823c80510c3e2bb9cb2589be0fe8ed810a18 Cleanup bld.java and Script.java Reconstructed - from failed merge ! hat/extractions/.gitignore ! hat/hat/Script.java ! hat/hat/bld ! hat/hat/bld.java ! hat/hat/clean.java - hat/hat/mkpoms - hat/hat/mkpoms.java From gfrost at openjdk.org Sat Apr 19 15:39:32 2025 From: gfrost at openjdk.org (Gary Frost) Date: Sat, 19 Apr 2025 15:39:32 GMT Subject: [code-reflection] Integrated: Cleanup bld.java and Script.java Reconstructed - from failed merge Message-ID: <6UMaOKo7z4CSLlCZIehp0GvWU14_mvTLR9hnKjMxUhY=.b963357a-eedf-49f4-9f04-d415c1d691d1@github.com> Finally got out of my merge mess (I think). Here we have a pruned version of hat/hat/Script.java and a cleaned up hat/hat/bld.java After moving jextract out to cmake, and making cmake callable from maven. I decided to cleanup hat/hat/bld.java which had a complicated 'capability' mechanism for emulating what cmake can do. I removed all of that. Now hat/bld.java just delegates to the same cmake files as maven. ------------- Commit messages: - Cleanup bld.java and Script.java Reconstructed - from failed merge Changes: https://git.openjdk.org/babylon/pull/405/files Webrev: https://webrevs.openjdk.org/?repo=babylon&pr=405&range=00 Stats: 3436 lines in 7 files changed: 118 ins; 3209 del; 109 mod Patch: https://git.openjdk.org/babylon/pull/405.diff Fetch: git fetch https://git.openjdk.org/babylon.git pull/405/head:pull/405 PR: https://git.openjdk.org/babylon/pull/405 From gfrost at openjdk.org Sat Apr 19 15:39:32 2025 From: gfrost at openjdk.org (Gary Frost) Date: Sat, 19 Apr 2025 15:39:32 GMT Subject: [code-reflection] Integrated: Cleanup bld.java and Script.java Reconstructed - from failed merge In-Reply-To: <6UMaOKo7z4CSLlCZIehp0GvWU14_mvTLR9hnKjMxUhY=.b963357a-eedf-49f4-9f04-d415c1d691d1@github.com> References: <6UMaOKo7z4CSLlCZIehp0GvWU14_mvTLR9hnKjMxUhY=.b963357a-eedf-49f4-9f04-d415c1d691d1@github.com> Message-ID: On Sat, 19 Apr 2025 15:35:05 GMT, Gary Frost wrote: > Finally got out of my merge mess (I think). > > Here we have a pruned version of hat/hat/Script.java and a cleaned up hat/hat/bld.java > > After moving jextract out to cmake, and making cmake callable from maven. I decided to cleanup hat/hat/bld.java which had a complicated 'capability' mechanism for emulating what cmake can do. I removed all of that. Now hat/bld.java just delegates to the same cmake files as maven. This pull request has now been integrated. Changeset: 8e81823c Author: Gary Frost URL: https://git.openjdk.org/babylon/commit/8e81823c80510c3e2bb9cb2589be0fe8ed810a18 Stats: 3436 lines in 7 files changed: 118 ins; 3209 del; 109 mod Cleanup bld.java and Script.java Reconstructed - from failed merge ------------- PR: https://git.openjdk.org/babylon/pull/405 From gfrost at openjdk.org Sun Apr 20 12:29:19 2025 From: gfrost at openjdk.org (Gary Frost) Date: Sun, 20 Apr 2025 12:29:19 GMT Subject: git: openjdk/babylon: code-reflection: Automatically detect which event helper to exclude in glwrap Message-ID: Changeset: fa9fc2ff Branch: code-reflection Author: Gary Frost Date: 2025-04-20 12:26:59 +0000 URL: https://git.openjdk.org/babylon/commit/fa9fc2ff23ba65085f5ee745e173e50e7e16d483 Automatically detect which event helper to exclude in glwrap ! hat/hat/Script.java ! hat/hat/bld.java ! hat/wrap/pom.xml From gfrost at openjdk.org Sun Apr 20 12:30:21 2025 From: gfrost at openjdk.org (Gary Frost) Date: Sun, 20 Apr 2025 12:30:21 GMT Subject: [code-reflection] Integrated: Automatically detect which event helper to exclude in glwrap Message-ID: Merged required cuda bld.java changes Sadly extracted opengl is subtley different on mac and linux. We need to exclude specific wrapper classes, depending on files in the jextracteOpenCL.jar. This PR also has logic to detect this so we don't have to remember to comment in/out 'exclusion' patterns. ------------- Commit messages: - Automatically detect which event helper to exclude in glwrap Changes: https://git.openjdk.org/babylon/pull/406/files Webrev: https://webrevs.openjdk.org/?repo=babylon&pr=406&range=00 Stats: 94 lines in 3 files changed: 74 ins; 9 del; 11 mod Patch: https://git.openjdk.org/babylon/pull/406.diff Fetch: git fetch https://git.openjdk.org/babylon.git pull/406/head:pull/406 PR: https://git.openjdk.org/babylon/pull/406 From gfrost at openjdk.org Sun Apr 20 12:30:21 2025 From: gfrost at openjdk.org (Gary Frost) Date: Sun, 20 Apr 2025 12:30:21 GMT Subject: [code-reflection] Integrated: Automatically detect which event helper to exclude in glwrap In-Reply-To: References: Message-ID: On Sun, 20 Apr 2025 12:24:43 GMT, Gary Frost wrote: > Merged required cuda bld.java changes > > Sadly extracted opengl is subtley different on mac and linux. We need to exclude specific wrapper classes, depending on files in the jextracteOpenCL.jar. > > This PR also has logic to detect this so we don't have to remember to comment in/out 'exclusion' patterns. This pull request has now been integrated. Changeset: fa9fc2ff Author: Gary Frost URL: https://git.openjdk.org/babylon/commit/fa9fc2ff23ba65085f5ee745e173e50e7e16d483 Stats: 94 lines in 3 files changed: 74 ins; 9 del; 11 mod Automatically detect which event helper to exclude in glwrap ------------- PR: https://git.openjdk.org/babylon/pull/406 From psandoz at openjdk.org Sun Apr 20 20:47:38 2025 From: psandoz at openjdk.org (Paul Sandoz) Date: Sun, 20 Apr 2025 20:47:38 GMT Subject: [code-reflection] RFR: Immutble ExternalizedOp.attributes Message-ID: The attributes map of `ExternalizedOp` no longer needs to be immutable. `OpBuilder` is modified accordingly to use `Map.of`. ------------- Commit messages: - Merge branch 'code-reflection' into extern-op-attr-map - Mutability not requred. Changes: https://git.openjdk.org/babylon/pull/407/files Webrev: https://webrevs.openjdk.org/?repo=babylon&pr=407&range=00 Stats: 30 lines in 2 files changed: 8 ins; 8 del; 14 mod Patch: https://git.openjdk.org/babylon/pull/407.diff Fetch: git fetch https://git.openjdk.org/babylon.git pull/407/head:pull/407 PR: https://git.openjdk.org/babylon/pull/407 From psandoz at openjdk.org Sun Apr 20 22:22:33 2025 From: psandoz at openjdk.org (Paul Sandoz) Date: Sun, 20 Apr 2025 22:22:33 GMT Subject: [code-reflection] RFR: Immutble ExternalizedOp.attributes [v2] In-Reply-To: References: Message-ID: > The attributes map of `ExternalizedOp` no longer needs to be immutable. `OpBuilder` is modified accordingly to use `Map.of`. > > In addition `OpBuilder` is also modified to reuse previously constructed type elements, and construct externalized ops using a constructor fewer parameters, if some are defaults. Paul Sandoz has updated the pull request incrementally with one additional commit since the last revision: Share types. ------------- Changes: - all: https://git.openjdk.org/babylon/pull/407/files - new: https://git.openjdk.org/babylon/pull/407/files/c9e7addd..40650ec0 Webrevs: - full: https://webrevs.openjdk.org/?repo=babylon&pr=407&range=01 - incr: https://webrevs.openjdk.org/?repo=babylon&pr=407&range=00-01 Stats: 40 lines in 2 files changed: 28 ins; 0 del; 12 mod Patch: https://git.openjdk.org/babylon/pull/407.diff Fetch: git fetch https://git.openjdk.org/babylon.git pull/407/head:pull/407 PR: https://git.openjdk.org/babylon/pull/407 From gfrost at openjdk.org Mon Apr 21 10:05:48 2025 From: gfrost at openjdk.org (Gary Frost) Date: Mon, 21 Apr 2025 10:05:48 GMT Subject: git: openjdk/babylon: code-reflection: For bld.java added javac diagnostics to JavaResult Message-ID: <5a855899-4b03-486a-9d12-a45330421381@openjdk.org> Changeset: 65b8762b Branch: code-reflection Author: Gary Frost Date: 2025-04-21 10:04:32 +0000 URL: https://git.openjdk.org/babylon/commit/65b8762b068469637e3c07f91533dabcdbef28c4 For bld.java added javac diagnostics to JavaResult ! hat/hat/Script.java ! hat/hat/bld.java From gfrost at openjdk.org Mon Apr 21 10:08:51 2025 From: gfrost at openjdk.org (Gary Frost) Date: Mon, 21 Apr 2025 10:08:51 GMT Subject: [code-reflection] Integrated: For bld.java added javac diagnostics to JavaResult In-Reply-To: References: Message-ID: On Mon, 21 Apr 2025 10:02:11 GMT, Gary Frost wrote: > Mostly cosmetic changes to Script.java and bld.java. > > Specifically now we show colorized 'diagnostics' from javac. So errors RED in output. This pull request has now been integrated. Changeset: 65b8762b Author: Gary Frost URL: https://git.openjdk.org/babylon/commit/65b8762b068469637e3c07f91533dabcdbef28c4 Stats: 222 lines in 2 files changed: 123 ins; 33 del; 66 mod For bld.java added javac diagnostics to JavaResult ------------- PR: https://git.openjdk.org/babylon/pull/408 From gfrost at openjdk.org Mon Apr 21 10:08:51 2025 From: gfrost at openjdk.org (Gary Frost) Date: Mon, 21 Apr 2025 10:08:51 GMT Subject: [code-reflection] Integrated: For bld.java added javac diagnostics to JavaResult Message-ID: Mostly cosmetic changes to Script.java and bld.java. Specifically now we show colorized 'diagnostics' from javac. So errors RED in output. ------------- Commit messages: - For bld.java added javac diagnostics to JavaResult Changes: https://git.openjdk.org/babylon/pull/408/files Webrev: https://webrevs.openjdk.org/?repo=babylon&pr=408&range=00 Stats: 222 lines in 2 files changed: 123 ins; 33 del; 66 mod Patch: https://git.openjdk.org/babylon/pull/408.diff Fetch: git fetch https://git.openjdk.org/babylon.git pull/408/head:pull/408 PR: https://git.openjdk.org/babylon/pull/408 From psandoz at openjdk.org Mon Apr 21 20:43:36 2025 From: psandoz at openjdk.org (Paul Sandoz) Date: Mon, 21 Apr 2025 20:43:36 GMT Subject: [code-reflection] RFR: Immutble ExternalizedOp.attributes [v3] In-Reply-To: References: Message-ID: > The attributes map of `ExternalizedOp` no longer needs to be immutable. `OpBuilder` is modified accordingly to use `Map.of`. > > In addition `OpBuilder` is also modified to reuse previously constructed type elements. Paul Sandoz has updated the pull request incrementally with one additional commit since the last revision: Remove special constructor (revisit later). ------------- Changes: - all: https://git.openjdk.org/babylon/pull/407/files - new: https://git.openjdk.org/babylon/pull/407/files/40650ec0..de14df0e Webrevs: - full: https://webrevs.openjdk.org/?repo=babylon&pr=407&range=02 - incr: https://webrevs.openjdk.org/?repo=babylon&pr=407&range=01-02 Stats: 31 lines in 2 files changed: 0 ins; 23 del; 8 mod Patch: https://git.openjdk.org/babylon/pull/407.diff Fetch: git fetch https://git.openjdk.org/babylon.git pull/407/head:pull/407 PR: https://git.openjdk.org/babylon/pull/407 From psandoz at openjdk.org Mon Apr 21 20:52:07 2025 From: psandoz at openjdk.org (Paul Sandoz) Date: Mon, 21 Apr 2025 20:52:07 GMT Subject: git: openjdk/babylon: code-reflection: Immutble ExternalizedOp.attributes Message-ID: Changeset: d41b2ba4 Branch: code-reflection Author: Paul Sandoz Date: 2025-04-21 20:50:50 +0000 URL: https://git.openjdk.org/babylon/commit/d41b2ba42565b2adb7a02015a396e6f9417a9f57 Immutble ExternalizedOp.attributes ! src/jdk.incubator.code/share/classes/jdk/incubator/code/op/ExternalizableOp.java ! src/jdk.incubator.code/share/classes/jdk/incubator/code/writer/OpBuilder.java From psandoz at openjdk.org Mon Apr 21 20:53:55 2025 From: psandoz at openjdk.org (Paul Sandoz) Date: Mon, 21 Apr 2025 20:53:55 GMT Subject: [code-reflection] Integrated: Immutble ExternalizedOp.attributes In-Reply-To: References: Message-ID: On Sun, 20 Apr 2025 20:43:12 GMT, Paul Sandoz wrote: > The attributes map of `ExternalizedOp` no longer needs to be immutable. `OpBuilder` is modified accordingly to use `Map.of`. > > In addition `OpBuilder` is also modified to reuse previously constructed type elements. > > This fixes issues compiling Triton matrix test cases when the CODE_BUILDER option is used to store models in class files. However, before we can switch back the default storage option to CODE_BUILDER we need to address the translation to AST. Specifically, we need to significantly reduce the number of locals that are declared and used. Once that is addressed i believe storage in the class file for the CODE_BUILDER option will be within ~2x in size of the TEXT option for small models and maybe smaller for larger models as more information is likely shared. This pull request has now been integrated. Changeset: d41b2ba4 Author: Paul Sandoz URL: https://git.openjdk.org/babylon/commit/d41b2ba42565b2adb7a02015a396e6f9417a9f57 Stats: 39 lines in 2 files changed: 13 ins; 8 del; 18 mod Immutble ExternalizedOp.attributes ------------- PR: https://git.openjdk.org/babylon/pull/407 From asotona at openjdk.org Tue Apr 22 08:56:29 2025 From: asotona at openjdk.org (Adam Sotona) Date: Tue, 22 Apr 2025 08:56:29 GMT Subject: [code-reflection] RFR: OnnxProtoModel implementation [v3] In-Reply-To: References: Message-ID: > Changes in this PR: > - `OnnxProtoModel` implementation replacing and enhancing basic `OnnxProtoPrinter` functionality > - added missing copyright headers Adam Sotona has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 12 commits: - OnnxPartialEvaluator sync with the latest changes - Merge remote-tracking branch 'babylon/code-reflection' into onnx-parser - OnnxProtoModel::toText improved to allow dump all data - copyright headers - fixed OnnxProtoModel - Merge remote-tracking branch 'babylon/code-reflection' into onnx-parser - OnnxProtoModel implementation replaces OnnxProtoPrinter - draft of ModelProto - draft of OnnxProtoParser - draft of OnnxProtoParser - ... and 2 more: https://git.openjdk.org/babylon/compare/d41b2ba4...c93d78ef ------------- Changes: https://git.openjdk.org/babylon/pull/399/files Webrev: https://webrevs.openjdk.org/?repo=babylon&pr=399&range=02 Stats: 795 lines in 8 files changed: 537 ins; 256 del; 2 mod Patch: https://git.openjdk.org/babylon/pull/399.diff Fetch: git fetch https://git.openjdk.org/babylon.git pull/399/head:pull/399 PR: https://git.openjdk.org/babylon/pull/399 From asotona at openjdk.org Tue Apr 22 12:11:21 2025 From: asotona at openjdk.org (Adam Sotona) Date: Tue, 22 Apr 2025 12:11:21 GMT Subject: [code-reflection] RFR: OnnxProtoModel implementation [v4] In-Reply-To: References: Message-ID: <1k_E8J78ir0O5PUeJLG3Xe-LnZY0XqneKqxDQU7irXc=.2a8acf2b-a229-4095-bfde-defeeae65b8c@github.com> > Changes in this PR: > - `OnnxProtoModel` implementation replacing and enhancing basic `OnnxProtoPrinter` functionality > - added missing copyright headers Adam Sotona has updated the pull request incrementally with one additional commit since the last revision: OnnxProtoModel minor fixes ------------- Changes: - all: https://git.openjdk.org/babylon/pull/399/files - new: https://git.openjdk.org/babylon/pull/399/files/c93d78ef..8271e2c3 Webrevs: - full: https://webrevs.openjdk.org/?repo=babylon&pr=399&range=03 - incr: https://webrevs.openjdk.org/?repo=babylon&pr=399&range=02-03 Stats: 72 lines in 1 file changed: 0 ins; 0 del; 72 mod Patch: https://git.openjdk.org/babylon/pull/399.diff Fetch: git fetch https://git.openjdk.org/babylon.git pull/399/head:pull/399 PR: https://git.openjdk.org/babylon/pull/399 From asotona at openjdk.org Tue Apr 22 12:36:47 2025 From: asotona at openjdk.org (Adam Sotona) Date: Tue, 22 Apr 2025 12:36:47 GMT Subject: git: openjdk/babylon: code-reflection: OnnxProtoModel implementation Message-ID: Changeset: 93b4b32c Branch: code-reflection Author: Adam Sotona Date: 2025-04-22 12:34:18 +0000 URL: https://git.openjdk.org/babylon/commit/93b4b32cb0c7c5288b6b76e0a2c71261a41a58ad OnnxProtoModel implementation ! cr-examples/onnx/src/main/java/oracle/code/onnx/OnnxProtoBuilder.java - cr-examples/onnx/src/main/java/oracle/code/onnx/OnnxProtoPrinter.java ! cr-examples/onnx/src/main/java/oracle/code/onnx/OnnxRuntime.java ! cr-examples/onnx/src/main/java/oracle/code/onnx/compiler/OnnxPartialEvaluator.java + cr-examples/onnx/src/main/java/oracle/code/onnx/proto/OnnxProtoModel.java ! cr-examples/onnx/src/test/java/oracle/code/onnx/RuntimeTest.java ! cr-examples/onnx/src/test/java/oracle/code/onnx/SimpleTest.java ! cr-examples/onnx/src/test/java/oracle/code/onnx/WalkTheMazeTest.java From asotona at openjdk.org Tue Apr 22 12:37:09 2025 From: asotona at openjdk.org (Adam Sotona) Date: Tue, 22 Apr 2025 12:37:09 GMT Subject: [code-reflection] Integrated: OnnxProtoModel implementation In-Reply-To: References: Message-ID: On Wed, 16 Apr 2025 18:21:12 GMT, Adam Sotona wrote: > Changes in this PR: > - `OnnxProtoModel` implementation replacing and enhancing basic `OnnxProtoPrinter` functionality > - added missing copyright headers This pull request has now been integrated. Changeset: 93b4b32c Author: Adam Sotona URL: https://git.openjdk.org/babylon/commit/93b4b32cb0c7c5288b6b76e0a2c71261a41a58ad Stats: 795 lines in 8 files changed: 537 ins; 256 del; 2 mod OnnxProtoModel implementation ------------- PR: https://git.openjdk.org/babylon/pull/399 From mabbay at openjdk.org Wed Apr 23 23:25:52 2025 From: mabbay at openjdk.org (Mourad Abbay) Date: Wed, 23 Apr 2025 23:25:52 GMT Subject: [code-reflection] RFR: Reduce number of local variables when transforming code model builder to AST Message-ID: <29bswmLdSbwIRO0ccjwuoErnoPDG59QNfP7ptIn--GI=.ab85bf3f-a930-4b43-bcde-c142b247edb6@github.com> For CODE_BUILDER option, we transform code model builder to equivalent AST. We were introducing local variables for very expressions, as a result we were closer to the limit on local variables. In this PR, we add local variables when it's needed. ------------- Commit messages: - Reduce number of local variables when transforming code model builder to AST Changes: https://git.openjdk.org/babylon/pull/410/files Webrev: https://webrevs.openjdk.org/?repo=babylon&pr=410&range=00 Stats: 149 lines in 2 files changed: 93 ins; 24 del; 32 mod Patch: https://git.openjdk.org/babylon/pull/410.diff Fetch: git fetch https://git.openjdk.org/babylon.git pull/410/head:pull/410 PR: https://git.openjdk.org/babylon/pull/410 From psandoz at openjdk.org Thu Apr 24 00:41:54 2025 From: psandoz at openjdk.org (Paul Sandoz) Date: Thu, 24 Apr 2025 00:41:54 GMT Subject: [code-reflection] RFR: Reduce number of local variables when transforming code model builder to AST In-Reply-To: <29bswmLdSbwIRO0ccjwuoErnoPDG59QNfP7ptIn--GI=.ab85bf3f-a930-4b43-bcde-c142b247edb6@github.com> References: <29bswmLdSbwIRO0ccjwuoErnoPDG59QNfP7ptIn--GI=.ab85bf3f-a930-4b43-bcde-c142b247edb6@github.com> Message-ID: On Wed, 23 Apr 2025 23:20:11 GMT, Mourad Abbay wrote: > For CODE_BUILDER option, we transform code model builder to equivalent AST. We were introducing local variables for very expressions, as a result we were closer to the limit on local variables. In this PR, we add local variables where it's needed. This fits nicely into the current code. Looking more closely. You don't explicitly traverse the root expressions graphs, you just use the root values, traverse (equivalently) on what they depend on per the op's operands via `opToTree`, and prune using the mappings in `valueToTree`, which all makes sense. We can avoid creation of the expression graphs. Instead, you can traverse the model identifying values that are the value roots i.e., those for the invocations as we do now *and* those with 0 or > 1 uses, and accumulate them in a list, called say `rootValues`. I think the explicit expression graph generation was useful (for me at least) to help better understand the problem and visualize but we don't need it anymore. However, the concept/pattern remains. src/jdk.incubator.code/share/classes/jdk/incubator/code/internal/CodeModelToAST.java line 78: > 76: } > 77: } > 78: Use a switch expression. src/jdk.incubator.code/share/classes/jdk/incubator/code/internal/CodeModelToAST.java line 124: > 122: Op.Result.class, Op.class); > 123: final MethodRef BLOCK_BUILDER_PARAM = MethodRef.method(Block.Builder.class, "parameter", > 124: Block.Parameter.class, TypeElement.class); Move to static final fields. src/jdk.incubator.code/share/classes/jdk/incubator/code/internal/CodeModelToAST.java line 138: > 136: // Variable declarations modeling local variables > 137: case CoreOp.VarOp vop -> vop.operands().get(0) instanceof Op.Result; > 138: // An operation result with no uses or more than one Update comment src/jdk.incubator.code/share/classes/jdk/incubator/code/internal/CodeModelToAST.java line 144: > 142: > 143: var stats = new ListBuffer(); > 144: // instead, we will traverse the root expr graphs instead of what? Perhaps say "Traverse the root expr graphs instead of traversing the model" src/jdk.incubator.code/share/classes/jdk/incubator/code/internal/CodeModelToAST.java line 254: > 252: java.util.List> edges = new ArrayList<>(); > 253: for (Value operand : op.result().dependsOn()) { > 254: // Ignore edge for operand if also used by other operations Update comment ------------- PR Review: https://git.openjdk.org/babylon/pull/410#pullrequestreview-2788971845 PR Review Comment: https://git.openjdk.org/babylon/pull/410#discussion_r2057075028 PR Review Comment: https://git.openjdk.org/babylon/pull/410#discussion_r2057075924 PR Review Comment: https://git.openjdk.org/babylon/pull/410#discussion_r2057076112 PR Review Comment: https://git.openjdk.org/babylon/pull/410#discussion_r2057085253 PR Review Comment: https://git.openjdk.org/babylon/pull/410#discussion_r2057082183 From mabbay at openjdk.org Thu Apr 24 04:05:45 2025 From: mabbay at openjdk.org (Mourad Abbay) Date: Thu, 24 Apr 2025 04:05:45 GMT Subject: [code-reflection] RFR: Reduce number of local variables when transforming code model builder to AST [v2] In-Reply-To: <29bswmLdSbwIRO0ccjwuoErnoPDG59QNfP7ptIn--GI=.ab85bf3f-a930-4b43-bcde-c142b247edb6@github.com> References: <29bswmLdSbwIRO0ccjwuoErnoPDG59QNfP7ptIn--GI=.ab85bf3f-a930-4b43-bcde-c142b247edb6@github.com> Message-ID: > For CODE_BUILDER option, we transform code model builder to equivalent AST. We were introducing local variables for very expressions, as a result we were closer to the limit on local variables. In this PR, we add local variables where it's needed. Mourad Abbay has updated the pull request incrementally with one additional commit since the last revision: Apply Paul's comments ------------- Changes: - all: https://git.openjdk.org/babylon/pull/410/files - new: https://git.openjdk.org/babylon/pull/410/files/cf8f38d9..f64ebf4f Webrevs: - full: https://webrevs.openjdk.org/?repo=babylon&pr=410&range=01 - incr: https://webrevs.openjdk.org/?repo=babylon&pr=410&range=00-01 Stats: 85 lines in 1 file changed: 4 ins; 67 del; 14 mod Patch: https://git.openjdk.org/babylon/pull/410.diff Fetch: git fetch https://git.openjdk.org/babylon.git pull/410/head:pull/410 PR: https://git.openjdk.org/babylon/pull/410 From psandoz at openjdk.org Thu Apr 24 16:36:06 2025 From: psandoz at openjdk.org (Paul Sandoz) Date: Thu, 24 Apr 2025 16:36:06 GMT Subject: [code-reflection] RFR: Reduce number of local variables when transforming code model builder to AST [v2] In-Reply-To: References: <29bswmLdSbwIRO0ccjwuoErnoPDG59QNfP7ptIn--GI=.ab85bf3f-a930-4b43-bcde-c142b247edb6@github.com> Message-ID: <5ewhwdj27enEfiAXObiRtCw3Y6iJNKVKZpgC1VkdzdU=.676d08e8-6ab3-4eac-8c0e-ac4b4043cfcd@github.com> On Thu, 24 Apr 2025 04:05:45 GMT, Mourad Abbay wrote: >> For CODE_BUILDER option, we transform code model builder to equivalent AST. We were introducing local variables for very expressions, as a result we were closer to the limit on local variables. In this PR, we add local variables where it's needed. > > Mourad Abbay has updated the pull request incrementally with one additional commit since the last revision: > > Apply Paul's comments Very good. ------------- Marked as reviewed by psandoz (Lead). PR Review: https://git.openjdk.org/babylon/pull/410#pullrequestreview-2791844380 From psandoz at openjdk.org Thu Apr 24 19:18:40 2025 From: psandoz at openjdk.org (Paul Sandoz) Date: Thu, 24 Apr 2025 19:18:40 GMT Subject: [code-reflection] RFR: Define JavaRef top-type for all Java refs Message-ID: This represents a modest step towards bringing java refs into the code model as type elements, and perhaps more formally enabling traversal and transformation of a type element's tree structure more directly than doing so via externalization. ------------- Commit messages: - Define JavaRef top-type for all Java refs. Changes: https://git.openjdk.org/babylon/pull/411/files Webrev: https://webrevs.openjdk.org/?repo=babylon&pr=411&range=00 Stats: 62 lines in 7 files changed: 35 ins; 9 del; 18 mod Patch: https://git.openjdk.org/babylon/pull/411.diff Fetch: git fetch https://git.openjdk.org/babylon.git pull/411/head:pull/411 PR: https://git.openjdk.org/babylon/pull/411 From psandoz at openjdk.org Thu Apr 24 19:35:04 2025 From: psandoz at openjdk.org (Paul Sandoz) Date: Thu, 24 Apr 2025 19:35:04 GMT Subject: [code-reflection] RFR: Define JavaRef top-type for all Java refs [v2] In-Reply-To: References: Message-ID: <1tI8KmqbupSzxtfsXQFT8_LJNUCBav-W9BynqYjbc5k=.7fa03669-2413-4cd1-b254-65dafe002380@github.com> > This represents a modest step towards bringing java refs into the code model as type elements, and perhaps more formally enabling traversal and transformation of a type element's tree structure more directly than doing so via externalization. Paul Sandoz has updated the pull request incrementally with one additional commit since the last revision: Revert unintended changes. ------------- Changes: - all: https://git.openjdk.org/babylon/pull/411/files - new: https://git.openjdk.org/babylon/pull/411/files/e3a74b3f..408353c6 Webrevs: - full: https://webrevs.openjdk.org/?repo=babylon&pr=411&range=01 - incr: https://webrevs.openjdk.org/?repo=babylon&pr=411&range=00-01 Stats: 26 lines in 2 files changed: 9 ins; 3 del; 14 mod Patch: https://git.openjdk.org/babylon/pull/411.diff Fetch: git fetch https://git.openjdk.org/babylon.git pull/411/head:pull/411 PR: https://git.openjdk.org/babylon/pull/411 From psandoz at openjdk.org Thu Apr 24 22:31:00 2025 From: psandoz at openjdk.org (Paul Sandoz) Date: Thu, 24 Apr 2025 22:31:00 GMT Subject: [code-reflection] RFR: Reduce number of local variables when transforming code model builder to AST [v2] In-Reply-To: References: <29bswmLdSbwIRO0ccjwuoErnoPDG59QNfP7ptIn--GI=.ab85bf3f-a930-4b43-bcde-c142b247edb6@github.com> Message-ID: On Thu, 24 Apr 2025 04:05:45 GMT, Mourad Abbay wrote: >> For CODE_BUILDER option, we transform code model builder to equivalent AST. We were introducing local variables for very expressions, as a result we were closer to the limit on local variables. In this PR, we add local variables where it's needed. > > Mourad Abbay has updated the pull request incrementally with one additional commit since the last revision: > > Apply Paul's comments I notice we are generating the `LocalVariableTable` which also means the local variable names get placed in the constant pool. Perhaps there is a way to avoid generating that for this method as i think it provides little value for debugging? ------------- PR Comment: https://git.openjdk.org/babylon/pull/410#issuecomment-2828997354 From mabbay at openjdk.org Fri Apr 25 00:16:09 2025 From: mabbay at openjdk.org (Mourad Abbay) Date: Fri, 25 Apr 2025 00:16:09 GMT Subject: git: openjdk/babylon: code-reflection: Reduce number of local variables when transforming code model builder to AST Message-ID: Changeset: f41fa393 Branch: code-reflection Author: Mourad Abbay Date: 2025-04-25 00:14:42 +0000 URL: https://git.openjdk.org/babylon/commit/f41fa3934e2aa966cbd585777fc44d9bd63f026a Reduce number of local variables when transforming code model builder to AST Reviewed-by: psandoz ! src/jdk.incubator.code/share/classes/jdk/incubator/code/internal/CodeModelToAST.java ! src/jdk.incubator.code/share/classes/jdk/incubator/code/internal/ReflectMethods.java From mabbay at openjdk.org Fri Apr 25 00:19:48 2025 From: mabbay at openjdk.org (Mourad Abbay) Date: Fri, 25 Apr 2025 00:19:48 GMT Subject: [code-reflection] RFR: Reduce number of local variables when transforming code model builder to AST [v3] In-Reply-To: <29bswmLdSbwIRO0ccjwuoErnoPDG59QNfP7ptIn--GI=.ab85bf3f-a930-4b43-bcde-c142b247edb6@github.com> References: <29bswmLdSbwIRO0ccjwuoErnoPDG59QNfP7ptIn--GI=.ab85bf3f-a930-4b43-bcde-c142b247edb6@github.com> Message-ID: > For CODE_BUILDER option, we transform code model builder to equivalent AST. We were introducing local variables for very expressions, as a result we were closer to the limit on local variables. In this PR, we add local variables where it's needed. Mourad Abbay has updated the pull request incrementally with one additional commit since the last revision: Add SYNTHETIC flag to the var symbol we add to the AST ------------- Changes: - all: https://git.openjdk.org/babylon/pull/410/files - new: https://git.openjdk.org/babylon/pull/410/files/f64ebf4f..18202044 Webrevs: - full: https://webrevs.openjdk.org/?repo=babylon&pr=410&range=02 - incr: https://webrevs.openjdk.org/?repo=babylon&pr=410&range=01-02 Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod Patch: https://git.openjdk.org/babylon/pull/410.diff Fetch: git fetch https://git.openjdk.org/babylon.git pull/410/head:pull/410 PR: https://git.openjdk.org/babylon/pull/410 From mabbay at openjdk.org Fri Apr 25 00:19:49 2025 From: mabbay at openjdk.org (Mourad Abbay) Date: Fri, 25 Apr 2025 00:19:49 GMT Subject: [code-reflection] RFR: Reduce number of local variables when transforming code model builder to AST [v2] In-Reply-To: References: <29bswmLdSbwIRO0ccjwuoErnoPDG59QNfP7ptIn--GI=.ab85bf3f-a930-4b43-bcde-c142b247edb6@github.com> Message-ID: On Thu, 24 Apr 2025 22:27:59 GMT, Paul Sandoz wrote: > I notice we are generating the `LocalVariableTable` which also means the local variable names get placed in the constant pool. Perhaps there is a way to avoid generating that for this method as i think it provides little value for debugging? > > Compiling with and without `-g` can show the approximate difference. I believe for larger models it can be significant. There is a way. I added the flag SYNTHETIC to the var symbol. ------------- PR Comment: https://git.openjdk.org/babylon/pull/410#issuecomment-2829104649 From mabbay at openjdk.org Fri Apr 25 00:19:49 2025 From: mabbay at openjdk.org (Mourad Abbay) Date: Fri, 25 Apr 2025 00:19:49 GMT Subject: [code-reflection] Integrated: Reduce number of local variables when transforming code model builder to AST In-Reply-To: <29bswmLdSbwIRO0ccjwuoErnoPDG59QNfP7ptIn--GI=.ab85bf3f-a930-4b43-bcde-c142b247edb6@github.com> References: <29bswmLdSbwIRO0ccjwuoErnoPDG59QNfP7ptIn--GI=.ab85bf3f-a930-4b43-bcde-c142b247edb6@github.com> Message-ID: On Wed, 23 Apr 2025 23:20:11 GMT, Mourad Abbay wrote: > For CODE_BUILDER option, we transform code model builder to equivalent AST. We were introducing local variables for very expressions, as a result we were closer to the limit on local variables. In this PR, we add local variables where it's needed. This pull request has now been integrated. Changeset: f41fa393 Author: Mourad Abbay URL: https://git.openjdk.org/babylon/commit/f41fa3934e2aa966cbd585777fc44d9bd63f026a Stats: 117 lines in 2 files changed: 56 ins; 50 del; 11 mod Reduce number of local variables when transforming code model builder to AST Reviewed-by: psandoz ------------- PR: https://git.openjdk.org/babylon/pull/410 From psandoz at openjdk.org Fri Apr 25 16:26:07 2025 From: psandoz at openjdk.org (Paul Sandoz) Date: Fri, 25 Apr 2025 16:26:07 GMT Subject: git: openjdk/babylon: code-reflection: Define JavaRef top-type for all Java refs Message-ID: Changeset: d484e537 Branch: code-reflection Author: Paul Sandoz Date: 2025-04-25 16:24:03 +0000 URL: https://git.openjdk.org/babylon/commit/d484e537cb41b737d54837fb23b8157abf2823e7 Define JavaRef top-type for all Java refs ! src/jdk.incubator.code/share/classes/jdk/incubator/code/type/ConstructorRef.java ! src/jdk.incubator.code/share/classes/jdk/incubator/code/type/FieldRef.java + src/jdk.incubator.code/share/classes/jdk/incubator/code/type/JavaRef.java ! src/jdk.incubator.code/share/classes/jdk/incubator/code/type/MethodRef.java ! src/jdk.incubator.code/share/classes/jdk/incubator/code/type/RecordTypeRef.java From psandoz at openjdk.org Fri Apr 25 16:27:12 2025 From: psandoz at openjdk.org (Paul Sandoz) Date: Fri, 25 Apr 2025 16:27:12 GMT Subject: [code-reflection] Integrated: Define JavaRef top-type for all Java refs In-Reply-To: References: Message-ID: On Thu, 24 Apr 2025 19:10:50 GMT, Paul Sandoz wrote: > This represents a modest step towards bringing java refs into the code model as type elements, and perhaps more formally enabling traversal and transformation of a type element's tree structure more directly than doing so via externalization. This pull request has now been integrated. Changeset: d484e537 Author: Paul Sandoz URL: https://git.openjdk.org/babylon/commit/d484e537cb41b737d54837fb23b8157abf2823e7 Stats: 36 lines in 5 files changed: 32 ins; 0 del; 4 mod Define JavaRef top-type for all Java refs ------------- PR: https://git.openjdk.org/babylon/pull/411 From gfrost at openjdk.org Sat Apr 26 09:45:27 2025 From: gfrost at openjdk.org (Gary Frost) Date: Sat, 26 Apr 2025 09:45:27 GMT Subject: git: openjdk/babylon: code-reflection: merged changes needed to get opencl + opengl on linux to work Message-ID: Changeset: 9a2c92fa Branch: code-reflection Author: Gary Frost Date: 2025-04-26 09:44:44 +0000 URL: https://git.openjdk.org/babylon/commit/9a2c92fa35f29b6cc44cb002bdc0fefbf184a2de merged changes needed to get opencl + opengl on linux to work ! hat/backends/ffi/shared/cpp/json.cpp ! hat/extractions/cuda/CMakeLists.txt ! hat/extractions/opencl/CMakeLists.txt ! hat/extractions/opengl/CMakeLists.txt ! hat/hat/bld.java ! hat/hat/run.java From gfrost at openjdk.org Sat Apr 26 09:47:36 2025 From: gfrost at openjdk.org (Gary Frost) Date: Sat, 26 Apr 2025 09:47:36 GMT Subject: [code-reflection] Integrated: merged changes needed to get opencl + opengl on linux to work Message-ID: These changes were needed to get HAT and examples/samples to build and run on linux x86 with opencl and opengl Mostly cmake changes for opengl = cl ------------- Commit messages: - merged changes needed to get opencl + opengl on linux to work Changes: https://git.openjdk.org/babylon/pull/412/files Webrev: https://webrevs.openjdk.org/?repo=babylon&pr=412&range=00 Stats: 54 lines in 6 files changed: 25 ins; 0 del; 29 mod Patch: https://git.openjdk.org/babylon/pull/412.diff Fetch: git fetch https://git.openjdk.org/babylon.git pull/412/head:pull/412 PR: https://git.openjdk.org/babylon/pull/412 From gfrost at openjdk.org Sat Apr 26 09:47:36 2025 From: gfrost at openjdk.org (Gary Frost) Date: Sat, 26 Apr 2025 09:47:36 GMT Subject: [code-reflection] Integrated: merged changes needed to get opencl + opengl on linux to work In-Reply-To: References: Message-ID: On Sat, 26 Apr 2025 09:43:08 GMT, Gary Frost wrote: > These changes were needed to get HAT and examples/samples to build and run on linux x86 with opencl and opengl > > Mostly cmake changes for opengl = cl This pull request has now been integrated. Changeset: 9a2c92fa Author: Gary Frost URL: https://git.openjdk.org/babylon/commit/9a2c92fa35f29b6cc44cb002bdc0fefbf184a2de Stats: 54 lines in 6 files changed: 25 ins; 0 del; 29 mod merged changes needed to get opencl + opengl on linux to work ------------- PR: https://git.openjdk.org/babylon/pull/412 From mcimadamore at openjdk.org Mon Apr 28 11:59:15 2025 From: mcimadamore at openjdk.org (Maurizio Cimadamore) Date: Mon, 28 Apr 2025 11:59:15 GMT Subject: [code-reflection] RFR: Add ConstructorRef [v3] In-Reply-To: <2Akw4wEO8NyLOdJUDJppc6cMSttGy8ri6Ib38Gvflp4=.fcd60ac1-1093-4cab-9e37-c845fc244d9c@github.com> References: <2Akw4wEO8NyLOdJUDJppc6cMSttGy8ri6Ib38Gvflp4=.fcd60ac1-1093-4cab-9e37-c845fc244d9c@github.com> Message-ID: On Fri, 18 Apr 2025 15:58:42 GMT, Mourad Abbay wrote: >> This is Maurizio's work. >> This PR tries to make `NewOp` similar to how `InvokeOp` is designed. It does this by adding a descriptor for constructors: `ConstructorRef`. >> `ConstructorRef` is wrapper around `FunctionType`, providing functionalities for resolution to a reflective instance. >> Some changes are common between this PR and #393. > > Mourad Abbay has updated the pull request incrementally with one additional commit since the last revision: > > Apply Paul's comments Thanks @mabbay for shepherding this PR! ------------- PR Comment: https://git.openjdk.org/babylon/pull/395#issuecomment-2835005097 From mcimadamore at openjdk.org Mon Apr 28 12:03:25 2025 From: mcimadamore at openjdk.org (Maurizio Cimadamore) Date: Mon, 28 Apr 2025 12:03:25 GMT Subject: [code-reflection] RFR: Define JavaRef top-type for all Java refs [v2] In-Reply-To: <1tI8KmqbupSzxtfsXQFT8_LJNUCBav-W9BynqYjbc5k=.7fa03669-2413-4cd1-b254-65dafe002380@github.com> References: <1tI8KmqbupSzxtfsXQFT8_LJNUCBav-W9BynqYjbc5k=.7fa03669-2413-4cd1-b254-65dafe002380@github.com> Message-ID: On Thu, 24 Apr 2025 19:35:04 GMT, Paul Sandoz wrote: >> This represents a modest step towards bringing java refs into the code model as type elements, and perhaps more formally enabling traversal and transformation of a type element's tree structure more directly than doing so via externalization. > > Paul Sandoz has updated the pull request incrementally with one additional commit since the last revision: > > Revert unintended changes. src/jdk.incubator.code/share/classes/jdk/incubator/code/type/JavaRef.java line 11: > 9: * can be resolved to an instance of {@link java.lang.reflect.Type Type}. > 10: */ > 11: public sealed interface JavaRef Shouldn't `TypeVarRef` also extend from this? ------------- PR Review Comment: https://git.openjdk.org/babylon/pull/411#discussion_r2063513372 From mcimadamore at openjdk.org Mon Apr 28 13:34:16 2025 From: mcimadamore at openjdk.org (Maurizio Cimadamore) Date: Mon, 28 Apr 2025 13:34:16 GMT Subject: [code-reflection] RFR: Reduce number of local variables when transforming code model builder to AST [v3] In-Reply-To: References: <29bswmLdSbwIRO0ccjwuoErnoPDG59QNfP7ptIn--GI=.ab85bf3f-a930-4b43-bcde-c142b247edb6@github.com> Message-ID: On Fri, 25 Apr 2025 00:19:48 GMT, Mourad Abbay wrote: >> For CODE_BUILDER option, we transform code model builder to equivalent AST. We were introducing local variables for very expressions, as a result we were closer to the limit on local variables. In this PR, we add local variables where it's needed. > > Mourad Abbay has updated the pull request incrementally with one additional commit since the last revision: > > Add SYNTHETIC flag to the var symbol we add to the AST Very nice change. This nicely compensates for the fact that javac doesn't perform any optimization on the generated code. Another possible optimization I noticed: if the expression being transformed is constant (e.g. it has a constant type -- `tree.type.constValue() != null`) -- then we don't need a local variable -- because javac will inline the constant value anyway (e.g. emit an `ldc` or something like that). Since there are quite a few string constants in the generated code this might further improve things. (That said, there doesn't seem to be any value that's used more than once that is also a constant) ------------- PR Review: https://git.openjdk.org/babylon/pull/410#pullrequestreview-2799315825 From psandoz at openjdk.org Mon Apr 28 16:03:04 2025 From: psandoz at openjdk.org (Paul Sandoz) Date: Mon, 28 Apr 2025 16:03:04 GMT Subject: [code-reflection] RFR: Define JavaRef top-type for all Java refs [v2] In-Reply-To: References: <1tI8KmqbupSzxtfsXQFT8_LJNUCBav-W9BynqYjbc5k=.7fa03669-2413-4cd1-b254-65dafe002380@github.com> Message-ID: On Mon, 28 Apr 2025 12:00:29 GMT, Maurizio Cimadamore wrote: >> Paul Sandoz has updated the pull request incrementally with one additional commit since the last revision: >> >> Revert unintended changes. > > src/jdk.incubator.code/share/classes/jdk/incubator/code/type/JavaRef.java line 11: > >> 9: * can be resolved to an instance of {@link java.lang.reflect.Type Type}. >> 10: */ >> 11: public sealed interface JavaRef > > Shouldn't `TypeVarRef` also extend from this? I always thought of it as just a java type element, correlated with `java.lang.reflect.TypeVariable` since that is what it resolves to. Why thinking about this PR and more broadly I was pondering whether to rename `TypeVarRef` to `TypeVarRefType` or more verbosely as `TypeVariableType`, since i must admit to to finding the `Ref` in `TypeVarRef` a little confusing as presumably it is referring to the type variable, but then `ClassType` refers to the class, etc. ------------- PR Review Comment: https://git.openjdk.org/babylon/pull/411#discussion_r2063996973 From mcimadamore at openjdk.org Mon Apr 28 16:11:06 2025 From: mcimadamore at openjdk.org (Maurizio Cimadamore) Date: Mon, 28 Apr 2025 16:11:06 GMT Subject: [code-reflection] RFR: Define JavaRef top-type for all Java refs [v2] In-Reply-To: References: <1tI8KmqbupSzxtfsXQFT8_LJNUCBav-W9BynqYjbc5k=.7fa03669-2413-4cd1-b254-65dafe002380@github.com> Message-ID: On Mon, 28 Apr 2025 15:59:59 GMT, Paul Sandoz wrote: >> src/jdk.incubator.code/share/classes/jdk/incubator/code/type/JavaRef.java line 11: >> >>> 9: * can be resolved to an instance of {@link java.lang.reflect.Type Type}. >>> 10: */ >>> 11: public sealed interface JavaRef >> >> Shouldn't `TypeVarRef` also extend from this? > > I always thought of it as just a java type element, correlated with `java.lang.reflect.TypeVariable` since that is what it resolves to. > > When thinking about this PR and more broadly I was pondering whether to rename `TypeVarRef` to `TypeVarRefType` or more verbosely as `TypeVariableType`, since i must admit to finding the `Ref` in `TypeVarRef` a little confusing as presumably it is referring to the type variable, but then `ClassType` refers to the class, etc. > I always thought of it as just a java type element, correlated with `java.lang.reflect.TypeVariable` since that is what it resolves to. > > Why thinking about this PR and more broadly I was pondering whether to rename `TypeVarRef` to `TypeVarRefType` or more verbosely as `TypeVariableType`, since i must admit to to finding the `Ref` in `TypeVarRef` a little confusing as presumably it is referring to the type variable, but then `ClassType` refers to the class, etc. Yeah - we went down the "ref" path because, well, it points to a declaration (of a type-variable). But I agree that `TypeVariable` or `TypeVariableType` is the better name. This is squarely a "type" - something that can be used where a type is expected, and something that can be turned into a `j.l.r.Type` ------------- PR Review Comment: https://git.openjdk.org/babylon/pull/411#discussion_r2064010599 From mcimadamore at openjdk.org Mon Apr 28 18:37:03 2025 From: mcimadamore at openjdk.org (Maurizio Cimadamore) Date: Mon, 28 Apr 2025 18:37:03 GMT Subject: [code-reflection] RFR: Drop unused local variables from generated op method Message-ID: The generated op method contains some local variables that are never used. This PR fixes that, further reducing the number of generated locals per synthetic op method. While this is not a groundbreaking improvement, in some complex tests I've seen a reduction of locals up to 20%. ------------- Commit messages: - Initial push Changes: https://git.openjdk.org/babylon/pull/413/files Webrev: https://webrevs.openjdk.org/?repo=babylon&pr=413&range=00 Stats: 16 lines in 1 file changed: 8 ins; 0 del; 8 mod Patch: https://git.openjdk.org/babylon/pull/413.diff Fetch: git fetch https://git.openjdk.org/babylon.git pull/413/head:pull/413 PR: https://git.openjdk.org/babylon/pull/413 From psandoz at openjdk.org Mon Apr 28 18:37:03 2025 From: psandoz at openjdk.org (Paul Sandoz) Date: Mon, 28 Apr 2025 18:37:03 GMT Subject: [code-reflection] RFR: Drop unused local variables from generated op method In-Reply-To: References: Message-ID: On Mon, 28 Apr 2025 18:28:37 GMT, Maurizio Cimadamore wrote: > The generated op method contains some local variables that are never used. > This PR fixes that, further reducing the number of generated locals per synthetic op method. > While this is not a groundbreaking improvement, in some complex tests I've seen a reduction of locals up to 20%. Nice incremental change. I was going to suggest more generally matching against `JavaStatement`, but then `ArrayStoreOp` implements `JavaExpression` and `JavaStatement`, and in the latter case the `uses` would be important if not explicitly matching. ------------- Marked as reviewed by psandoz (Lead). PR Review: https://git.openjdk.org/babylon/pull/413#pullrequestreview-2800300656 From mcimadamore at openjdk.org Mon Apr 28 18:37:04 2025 From: mcimadamore at openjdk.org (Maurizio Cimadamore) Date: Mon, 28 Apr 2025 18:37:04 GMT Subject: [code-reflection] RFR: Drop unused local variables from generated op method In-Reply-To: References: Message-ID: On Mon, 28 Apr 2025 18:28:37 GMT, Maurizio Cimadamore wrote: > The generated op method contains some local variables that are never used. > This PR fixes that, further reducing the number of generated locals per synthetic op method. > While this is not a groundbreaking improvement, in some complex tests I've seen a reduction of locals up to 20%. src/jdk.incubator.code/share/classes/jdk/incubator/code/internal/CodeModelToAST.java line 124: > 122: > 123: java.util.List rootValues = funcOp.traverse(new ArrayList<>(), (l, ce) -> { > 124: boolean isRoot = switch (ce) { I've rewrote the code here, as the condition `uses().size() != 1` was very subtle, and picking up things with no usages which, in principle, should not be "roots". Now the list of root values we expect is captured explicitly in the code. src/jdk.incubator.code/share/classes/jdk/incubator/code/internal/CodeModelToAST.java line 146: > 144: valueToTree.put(root, tree); > 145: } else { > 146: tree = treeMaker.Exec(e); This is the heart of the fix - if there's no usage, just emit as expression statement. I have tried with an alternate fix which, in case of exactly one use, would "inline" the use (avoiding the local). While this delivered massive reduction in the number of locals, the generated model was no longer compatible as some values were being reordered. ------------- PR Review Comment: https://git.openjdk.org/babylon/pull/413#discussion_r2064261368 PR Review Comment: https://git.openjdk.org/babylon/pull/413#discussion_r2064268094 From mcimadamore at openjdk.org Mon Apr 28 18:37:04 2025 From: mcimadamore at openjdk.org (Maurizio Cimadamore) Date: Mon, 28 Apr 2025 18:37:04 GMT Subject: [code-reflection] RFR: Drop unused local variables from generated op method In-Reply-To: References: Message-ID: <5zzzDlmcJNs2_POmr2YlIlr2HUDmpjyn8En46iEzpB8=.72825fc7-7bc2-474d-bc29-72a02ddff0fe@github.com> On Mon, 28 Apr 2025 18:30:11 GMT, Maurizio Cimadamore wrote: >> The generated op method contains some local variables that are never used. >> This PR fixes that, further reducing the number of generated locals per synthetic op method. >> While this is not a groundbreaking improvement, in some complex tests I've seen a reduction of locals up to 20%. > > src/jdk.incubator.code/share/classes/jdk/incubator/code/internal/CodeModelToAST.java line 124: > >> 122: >> 123: java.util.List rootValues = funcOp.traverse(new ArrayList<>(), (l, ce) -> { >> 124: boolean isRoot = switch (ce) { > > I've rewrote the code here, as the condition `uses().size() != 1` was very subtle, and picking up things with no usages which, in principle, should not be "roots". Now the list of root values we expect is captured explicitly in the code. Array stores ops and return ops were examples of ops implicitly picked up by the old code. ------------- PR Review Comment: https://git.openjdk.org/babylon/pull/413#discussion_r2064262206 From asotona at openjdk.org Mon Apr 28 20:12:06 2025 From: asotona at openjdk.org (Adam Sotona) Date: Mon, 28 Apr 2025 20:12:06 GMT Subject: git: openjdk/babylon: code-reflection: Onnx proto model test Message-ID: Changeset: f4c8a1cb Branch: code-reflection Author: Adam Sotona Date: 2025-04-28 20:10:43 +0000 URL: https://git.openjdk.org/babylon/commit/f4c8a1cbc04dd3f01696239ffde37e466725c538 Onnx proto model test ! cr-examples/onnx/opgen/pom.xml ! cr-examples/onnx/opgen/src/main/java/oracle/code/onnx/opgen/OpGen.java ! cr-examples/onnx/src/main/java/oracle/code/onnx/ir/OnnxOp.java ! cr-examples/onnx/src/main/java/oracle/code/onnx/ir/OnnxOps.java ! cr-examples/onnx/src/main/java/oracle/code/onnx/ir/OnnxType.java ! cr-examples/onnx/src/main/java/oracle/code/onnx/proto/OnnxProtoModel.java + cr-examples/onnx/src/test/java/oracle/code/onnx/proto/OnnxProtoModelTest.java From asotona at openjdk.org Mon Apr 28 20:14:48 2025 From: asotona at openjdk.org (Adam Sotona) Date: Mon, 28 Apr 2025 20:14:48 GMT Subject: [code-reflection] Integrated: Onnx proto model test Message-ID: Onnx proto model test ------------- Commit messages: - draft of OnnxProtoModelTest::cnnLiftTest - OnnxProtoModelTest - work in progress - fixed OpGen to avoid CCE - reverted OnnxOps change - reverted OnnxOps change - OnnxProtoModelTest - work in progress - OnnxProtoModelTest - work in progress - OnnxProtoToModelTest - work in progress - Initial work on OnnxProtoToModelTest Changes: https://git.openjdk.org/babylon/pull/409/files Webrev: https://webrevs.openjdk.org/?repo=babylon&pr=409&range=00 Stats: 361 lines in 7 files changed: 332 ins; 9 del; 20 mod Patch: https://git.openjdk.org/babylon/pull/409.diff Fetch: git fetch https://git.openjdk.org/babylon.git pull/409/head:pull/409 PR: https://git.openjdk.org/babylon/pull/409 From asotona at openjdk.org Mon Apr 28 20:14:49 2025 From: asotona at openjdk.org (Adam Sotona) Date: Mon, 28 Apr 2025 20:14:49 GMT Subject: [code-reflection] Integrated: Onnx proto model test In-Reply-To: References: Message-ID: On Wed, 23 Apr 2025 15:10:21 GMT, Adam Sotona wrote: > Onnx proto model test This pull request has now been integrated. Changeset: f4c8a1cb Author: Adam Sotona URL: https://git.openjdk.org/babylon/commit/f4c8a1cbc04dd3f01696239ffde37e466725c538 Stats: 361 lines in 7 files changed: 332 ins; 9 del; 20 mod Onnx proto model test ------------- PR: https://git.openjdk.org/babylon/pull/409 From mabbay at openjdk.org Tue Apr 29 08:41:01 2025 From: mabbay at openjdk.org (Mourad Abbay) Date: Tue, 29 Apr 2025 08:41:01 GMT Subject: [code-reflection] RFR: Drop unused local variables from generated op method In-Reply-To: References: Message-ID: <-qWphYkDvqZcV5052EALEFd6Kde9Bk6NMpnAIUAXMDE=.112a006d-e9fb-4639-872c-e88b66c1c146@github.com> On Mon, 28 Apr 2025 18:28:37 GMT, Maurizio Cimadamore wrote: > The generated op method contains some local variables that are never used. > This PR fixes that, further reducing the number of generated locals per synthetic op method. > While this is not a groundbreaking improvement, in some complex tests I've seen a reduction of locals up to 20%. Look good to me ------------- PR Comment: https://git.openjdk.org/babylon/pull/413#issuecomment-2837953535 From mcimadamore at openjdk.org Tue Apr 29 08:50:44 2025 From: mcimadamore at openjdk.org (Maurizio Cimadamore) Date: Tue, 29 Apr 2025 08:50:44 GMT Subject: git: openjdk/babylon: code-reflection: Drop unused local variables from generated op method Message-ID: Changeset: 4f3248ab Branch: code-reflection Author: Maurizio Cimadamore Date: 2025-04-29 08:48:59 +0000 URL: https://git.openjdk.org/babylon/commit/4f3248ab8d843615190397efd178b4e709877cca Drop unused local variables from generated op method Reviewed-by: psandoz ! src/jdk.incubator.code/share/classes/jdk/incubator/code/internal/CodeModelToAST.java From mcimadamore at openjdk.org Tue Apr 29 08:51:59 2025 From: mcimadamore at openjdk.org (Maurizio Cimadamore) Date: Tue, 29 Apr 2025 08:51:59 GMT Subject: [code-reflection] Integrated: Drop unused local variables from generated op method In-Reply-To: References: Message-ID: <-sH-acLuugQlnjnQ904cyB6Yj-r10p7RzuSjjpw1IZ0=.3907072f-73d0-436a-a25b-2379005f3213@github.com> On Mon, 28 Apr 2025 18:28:37 GMT, Maurizio Cimadamore wrote: > The generated op method contains some local variables that are never used. > This PR fixes that, further reducing the number of generated locals per synthetic op method. > While this is not a groundbreaking improvement, in some complex tests I've seen a reduction of locals up to 20%. This pull request has now been integrated. Changeset: 4f3248ab Author: Maurizio Cimadamore URL: https://git.openjdk.org/babylon/commit/4f3248ab8d843615190397efd178b4e709877cca Stats: 16 lines in 1 file changed: 8 ins; 0 del; 8 mod Drop unused local variables from generated op method Reviewed-by: psandoz ------------- PR: https://git.openjdk.org/babylon/pull/413 From mabbay at openjdk.org Tue Apr 29 10:57:38 2025 From: mabbay at openjdk.org (Mourad Abbay) Date: Tue, 29 Apr 2025 10:57:38 GMT Subject: [code-reflection] RFR: Method code model uniquness Message-ID: Multiple `Method` objects that represent the same java method, have different code models. This is because every time we call `Class.getMethod` or `Class.getDeclaredMethod` we get a newly created instance. One possible solution is to attach the model to the root `Method` object. ------------- Commit messages: - Write test that shows the issue Changes: https://git.openjdk.org/babylon/pull/415/files Webrev: https://webrevs.openjdk.org/?repo=babylon&pr=415&range=00 Stats: 32 lines in 1 file changed: 32 ins; 0 del; 0 mod Patch: https://git.openjdk.org/babylon/pull/415.diff Fetch: git fetch https://git.openjdk.org/babylon.git pull/415/head:pull/415 PR: https://git.openjdk.org/babylon/pull/415 From mabbay at openjdk.org Tue Apr 29 12:10:34 2025 From: mabbay at openjdk.org (Mourad Abbay) Date: Tue, 29 Apr 2025 12:10:34 GMT Subject: [code-reflection] RFR: Method code model uniquness [v2] In-Reply-To: References: Message-ID: <1zU_o89vvJYSFw0rHd0GYm6xoCm4I7DyoscVlXpF1-U=.8fad2667-0a86-4918-9ee9-c106cdd41ee4@github.com> > Multiple `Method` objects that represent the same java method, have different code models. This is because every time we call `Class.getMethod` or `Class.getDeclaredMethod` we get a newly created instance. One possible solution is to attach the model to the root `Method` object. Mourad Abbay has updated the pull request incrementally with one additional commit since the last revision: Ensure instances that reflect the same method have the same model ------------- Changes: - all: https://git.openjdk.org/babylon/pull/415/files - new: https://git.openjdk.org/babylon/pull/415/files/114c6a59..f88f4e2b Webrevs: - full: https://webrevs.openjdk.org/?repo=babylon&pr=415&range=01 - incr: https://webrevs.openjdk.org/?repo=babylon&pr=415&range=00-01 Stats: 20 lines in 2 files changed: 11 ins; 0 del; 9 mod Patch: https://git.openjdk.org/babylon/pull/415.diff Fetch: git fetch https://git.openjdk.org/babylon.git pull/415/head:pull/415 PR: https://git.openjdk.org/babylon/pull/415 From mcimadamore at openjdk.org Tue Apr 29 14:13:07 2025 From: mcimadamore at openjdk.org (Maurizio Cimadamore) Date: Tue, 29 Apr 2025 14:13:07 GMT Subject: [code-reflection] RFR: Method code model uniquness [v2] In-Reply-To: <1zU_o89vvJYSFw0rHd0GYm6xoCm4I7DyoscVlXpF1-U=.8fad2667-0a86-4918-9ee9-c106cdd41ee4@github.com> References: <1zU_o89vvJYSFw0rHd0GYm6xoCm4I7DyoscVlXpF1-U=.8fad2667-0a86-4918-9ee9-c106cdd41ee4@github.com> Message-ID: On Tue, 29 Apr 2025 12:10:34 GMT, Mourad Abbay wrote: >> Multiple `Method` objects that represent the same java method, have different code models. This is because every time we call `Class.getMethod` or `Class.getDeclaredMethod` we get a newly created instance. One possible solution is to attach the model to the root `Method` object. > > Mourad Abbay has updated the pull request incrementally with one additional commit since the last revision: > > Ensure instances that reflect the same method have the same model src/java.base/share/classes/java/lang/reflect/Method.java line 233: > 231: /* package */ > 232: Optional setCodeModelIfNeeded(Function> modelFactory) { > 233: Optional localRef = root.codeModel; I was looking at `Method` and found this method that does more or less a similar operation: // Accessor for generic info repository @Override MethodRepository getGenericInfo() { var genericInfo = this.genericInfo; if (genericInfo == null) { var root = this.root; if (root != null) { genericInfo = root.getGenericInfo(); } else { genericInfo = MethodRepository.make(getGenericSignature(), getFactory()); } this.genericInfo = genericInfo; } return genericInfo; } Should we copy/reuse same logic? ------------- PR Review Comment: https://git.openjdk.org/babylon/pull/415#discussion_r2066617876 From mcimadamore at openjdk.org Tue Apr 29 14:13:07 2025 From: mcimadamore at openjdk.org (Maurizio Cimadamore) Date: Tue, 29 Apr 2025 14:13:07 GMT Subject: [code-reflection] RFR: Method code model uniquness [v2] In-Reply-To: References: <1zU_o89vvJYSFw0rHd0GYm6xoCm4I7DyoscVlXpF1-U=.8fad2667-0a86-4918-9ee9-c106cdd41ee4@github.com> Message-ID: On Tue, 29 Apr 2025 14:09:04 GMT, Maurizio Cimadamore wrote: >> Mourad Abbay has updated the pull request incrementally with one additional commit since the last revision: >> >> Ensure instances that reflect the same method have the same model > > src/java.base/share/classes/java/lang/reflect/Method.java line 233: > >> 231: /* package */ >> 232: Optional setCodeModelIfNeeded(Function> modelFactory) { >> 233: Optional localRef = root.codeModel; > > I was looking at `Method` and found this method that does more or less a similar operation: > > > // Accessor for generic info repository > @Override > MethodRepository getGenericInfo() { > var genericInfo = this.genericInfo; > if (genericInfo == null) { > var root = this.root; > if (root != null) { > genericInfo = root.getGenericInfo(); > } else { > genericInfo = MethodRepository.make(getGenericSignature(), getFactory()); > } > this.genericInfo = genericInfo; > } > return genericInfo; > } > > > Should we copy/reuse same logic? E.g. note that `root` can be null! ------------- PR Review Comment: https://git.openjdk.org/babylon/pull/415#discussion_r2066621384 From psandoz at openjdk.org Tue Apr 29 16:34:00 2025 From: psandoz at openjdk.org (Paul Sandoz) Date: Tue, 29 Apr 2025 16:34:00 GMT Subject: [code-reflection] RFR: Method code model uniquness [v2] In-Reply-To: <1zU_o89vvJYSFw0rHd0GYm6xoCm4I7DyoscVlXpF1-U=.8fad2667-0a86-4918-9ee9-c106cdd41ee4@github.com> References: <1zU_o89vvJYSFw0rHd0GYm6xoCm4I7DyoscVlXpF1-U=.8fad2667-0a86-4918-9ee9-c106cdd41ee4@github.com> Message-ID: <4cOq4JZvyTa-Cc6PlMYBlBTnES74Y7zTz9Q8ro5am9w=.7cdf58ce-9f26-4215-82fc-563b58329478@github.com> On Tue, 29 Apr 2025 12:10:34 GMT, Mourad Abbay wrote: >> Multiple `Method` objects that represent the same java method, have different code models. This is because every time we call `Class.getMethod` or `Class.getDeclaredMethod` we get a newly created instance. One possible solution is to attach the model to the root `Method` object. > > Mourad Abbay has updated the pull request incrementally with one additional commit since the last revision: > > Ensure instances that reflect the same method have the same model Given the underlying root method is held in an instance of `ReflectionData` which in turn is held in a soft reference stored on the class we may not need to do anything specific to ensure code models don't hang around too long after they are no longer used. If we find we need to, e.g., because the non-root method instances are cached or something connected is, we can update to store the model in a weak reference. However, we probably need to do something like that for the code models of lambda expressions, when you get to that, and if so i think we should do so for both cases. ------------- PR Review: https://git.openjdk.org/babylon/pull/415#pullrequestreview-2804224298 From psandoz at openjdk.org Tue Apr 29 16:34:00 2025 From: psandoz at openjdk.org (Paul Sandoz) Date: Tue, 29 Apr 2025 16:34:00 GMT Subject: [code-reflection] RFR: Method code model uniquness [v2] In-Reply-To: References: <1zU_o89vvJYSFw0rHd0GYm6xoCm4I7DyoscVlXpF1-U=.8fad2667-0a86-4918-9ee9-c106cdd41ee4@github.com> Message-ID: On Tue, 29 Apr 2025 14:09:55 GMT, Maurizio Cimadamore wrote: >> src/java.base/share/classes/java/lang/reflect/Method.java line 233: >> >>> 231: /* package */ >>> 232: Optional setCodeModelIfNeeded(Function> modelFactory) { >>> 233: Optional localRef = root.codeModel; >> >> I was looking at `Method` and found this method that does more or less a similar operation: >> >> >> // Accessor for generic info repository >> @Override >> MethodRepository getGenericInfo() { >> var genericInfo = this.genericInfo; >> if (genericInfo == null) { >> var root = this.root; >> if (root != null) { >> genericInfo = root.getGenericInfo(); >> } else { >> genericInfo = MethodRepository.make(getGenericSignature(), getFactory()); >> } >> this.genericInfo = genericInfo; >> } >> return genericInfo; >> } >> >> >> Should we copy/reuse same logic? > > E.g. note that `root` can be null! Yes, pattern is i think better. It recursively enters the root instance, whose root has to be `null`, hence the check to determine whether to recurse or do the work. ------------- PR Review Comment: https://git.openjdk.org/babylon/pull/415#discussion_r2066885346 From psandoz at openjdk.org Tue Apr 29 22:30:35 2025 From: psandoz at openjdk.org (Paul Sandoz) Date: Tue, 29 Apr 2025 22:30:35 GMT Subject: [code-reflection] RFR: JavaRef extends TypeElement Message-ID: <67BtibiQvMvf_mN3zQEpynENB26pqjxkUyVprL1xeVM=.bfab593c-f8fd-41f1-b527-36b384b52177@github.com> Each implementation of `JavaRef` supports an external form, using the sigil prefix `&` to indicate a reference of some kind. The externalized form of `TypeVarRef` is changed to be uniformly structured. This required update to the code model text of a few tests. Arguably we are taking a step back in terms of readability, but we can address that later - it was easier to update a few tests rather than preserve the existing encoding in a few places. We still rely on the bespoke string form for java refs, which is used as the value of an externalized attribute. Ideally such attribute values are either instances of `JavaRef` or `ExternalizedTypeElement` to be transformed into the appropriate `JavaRef`. We can address that later to further separate out parsing. I believe we have what we need to further enhance the code model builder to not rely on bespoke parsing logic of Java types. We can either construct `ExternalizedTypeElement` tree instances explicitly or parse from a very simple grammar. If we are careful i believe we can share the results of nested type elements if reused e.g. as in `List` and `Set`. More generally it now means we can generate a simple s-expression-like tree for the whole code model, e.g., a string where the `(` and `)` characters represent tree structure and say `L` represents a leaf node, and a list of leaf node values in topological order. -- Below is the type grammar, which i believe is consistent with what we have implemented. # Type element grammar ## General structure identifier string name identifier sigil # | . | & | + | - | [ type sigil identifier sigil identifier identifier < types > sigil identifier < types > types type type , types ## Core types varType "var" < type > tupleType "tuple" "tuple" < types > funcType "func" < types > # Java types javaType primitiveType | classType | wildCardType | arrayType | typeVariableType javaType-no-wildCardType primitiveType | classType | arrayType | typeVariableType primitiveType boolean | byte | ... | void classType name name < paramTypes > . < enclosingType , innerType > paramTypes javaType javaType , javaTypes enclosingType classType innerType classType wildcardType + < wildcardTypeBound > - < wildcardTypeBound > wildcardTypeBound javaType-no-wildCardType arrayType dims < javaType > dims [ [ dims typeVariableType "#" name < typeVariableTypeOwner , typeVariableTypeBound > typeVariableTypeOwner classType | methodRef | constructorRef typeVariableTypeBound javaType-no-wildCardType ## Java refs methodRef & "m" < type , name , funcType > constructorRef & "c" < funcType > fieldRef & "f" < type , name , type > recordRef & "r" < type > & "r" < type , recordRefComponentTypes > recordRefComponentTypes type , name type , name , recordRefComponentTypes ------------- Commit messages: - Remove comment. - Remove redundant code. - JavaRef extends TypeElement Changes: https://git.openjdk.org/babylon/pull/416/files Webrev: https://webrevs.openjdk.org/?repo=babylon&pr=416&range=00 Stats: 291 lines in 14 files changed: 214 ins; 22 del; 55 mod Patch: https://git.openjdk.org/babylon/pull/416.diff Fetch: git fetch https://git.openjdk.org/babylon.git pull/416/head:pull/416 PR: https://git.openjdk.org/babylon/pull/416 From mabbay at openjdk.org Tue Apr 29 23:43:14 2025 From: mabbay at openjdk.org (Mourad Abbay) Date: Tue, 29 Apr 2025 23:43:14 GMT Subject: [code-reflection] RFR: Method code model uniquness [v3] In-Reply-To: References: Message-ID: <281FsFAzaOYRCle_hk1XGa8DkK_fsodCAQ02TpC44gA=.f4f48305-fc7e-4220-87be-2cb84514a78a@github.com> > Multiple `Method` objects that represent the same java method, have different code models. This is because every time we call `Class.getMethod` or `Class.getDeclaredMethod` we get a newly created instance. One possible solution is to attach the model to the root `Method` object. Mourad Abbay has updated the pull request incrementally with one additional commit since the last revision: Apply Maurizio's suggestion ------------- Changes: - all: https://git.openjdk.org/babylon/pull/415/files - new: https://git.openjdk.org/babylon/pull/415/files/f88f4e2b..4c2b662d Webrevs: - full: https://webrevs.openjdk.org/?repo=babylon&pr=415&range=02 - incr: https://webrevs.openjdk.org/?repo=babylon&pr=415&range=01-02 Stats: 8 lines in 1 file changed: 4 ins; 0 del; 4 mod Patch: https://git.openjdk.org/babylon/pull/415.diff Fetch: git fetch https://git.openjdk.org/babylon.git pull/415/head:pull/415 PR: https://git.openjdk.org/babylon/pull/415 From mabbay at openjdk.org Wed Apr 30 01:19:55 2025 From: mabbay at openjdk.org (Mourad Abbay) Date: Wed, 30 Apr 2025 01:19:55 GMT Subject: [code-reflection] RFR: Method code model uniquness [v4] In-Reply-To: References: Message-ID: > Multiple `Method` objects that represent the same java method, have different code models. This is because every time we call `Class.getMethod` or `Class.getDeclaredMethod` we get a newly created instance. One possible solution is to attach the model to the root `Method` object. Mourad Abbay has updated the pull request incrementally with one additional commit since the last revision: Avoid caching Method.codeModel on non-root instances ------------- Changes: - all: https://git.openjdk.org/babylon/pull/415/files - new: https://git.openjdk.org/babylon/pull/415/files/4c2b662d..71f52f60 Webrevs: - full: https://webrevs.openjdk.org/?repo=babylon&pr=415&range=03 - incr: https://webrevs.openjdk.org/?repo=babylon&pr=415&range=02-03 Stats: 8 lines in 1 file changed: 3 ins; 4 del; 1 mod Patch: https://git.openjdk.org/babylon/pull/415.diff Fetch: git fetch https://git.openjdk.org/babylon.git pull/415/head:pull/415 PR: https://git.openjdk.org/babylon/pull/415 From mcimadamore at openjdk.org Wed Apr 30 08:19:09 2025 From: mcimadamore at openjdk.org (Maurizio Cimadamore) Date: Wed, 30 Apr 2025 08:19:09 GMT Subject: [code-reflection] RFR: Method code model uniquness [v4] In-Reply-To: References: Message-ID: On Wed, 30 Apr 2025 01:19:55 GMT, Mourad Abbay wrote: >> Multiple `Method` objects that represent the same java method, have different code models. This is because every time we call `Class.getMethod` or `Class.getDeclaredMethod` we get a newly created instance. One possible solution is to attach the model to the root `Method` object. > > Mourad Abbay has updated the pull request incrementally with one additional commit since the last revision: > > Avoid caching Method.codeModel on non-root instances src/java.base/share/classes/java/lang/reflect/Method.java line 233: > 231: /* package */ > 232: Optional setCodeModelIfNeeded(Function> modelFactory) { > 233: if (root != null) { This logic is subtly different to the one I pointed out -- this always recurse if the root is non null. The one I pointed out calls the method on the root, but than saves the result on the copy locally. I'm not sure which is better -- but it seems odd to use slightly different idioms in the same class to do effectively the same thing? ------------- PR Review Comment: https://git.openjdk.org/babylon/pull/415#discussion_r2068136860 From mcimadamore at openjdk.org Wed Apr 30 08:33:04 2025 From: mcimadamore at openjdk.org (Maurizio Cimadamore) Date: Wed, 30 Apr 2025 08:33:04 GMT Subject: [code-reflection] RFR: JavaRef extends TypeElement In-Reply-To: <67BtibiQvMvf_mN3zQEpynENB26pqjxkUyVprL1xeVM=.bfab593c-f8fd-41f1-b527-36b384b52177@github.com> References: <67BtibiQvMvf_mN3zQEpynENB26pqjxkUyVprL1xeVM=.bfab593c-f8fd-41f1-b527-36b384b52177@github.com> Message-ID: On Tue, 29 Apr 2025 22:12:34 GMT, Paul Sandoz wrote: > Each implementation of `JavaRef` supports an external form, using the sigil prefix `&` to indicate a reference of some kind. > > The externalized form of `TypeVarRef` is changed to be uniformly structured. This required update to the code model text of a few tests. Arguably we are taking a step back in terms of readability, but we can address that later - it was easier to update a few tests rather than preserve the existing encoding in a few places. > > We still rely on the bespoke string form for java refs, which is used as the value of an externalized attribute. Ideally such attribute values are either instances of `JavaRef` or `ExternalizedTypeElement` to be transformed into the appropriate `JavaRef`. We can address that later to further separate out parsing. > > I believe we have what we need to further enhance the code model builder to not rely on bespoke parsing logic of types and refs. We can either construct `ExternalizedTypeElement` tree instances explicitly or parse from a very simple grammar. If we are careful i believe we can share the results of nested type elements if reused e.g. as in `List` and `Set`. > > More generally it now means we can generate a simple s-expression-like tree for the whole code model, e.g., a string where the `(` and `)` characters represent tree structure and say `L` represents a leaf node, and a list of leaf node values in topological order. > > -- > > Below is the type grammar, which i believe is consistent with what we have implemented. > > # Type element grammar > > ## General structure > > > identifier > string > > name > identifier > > sigil > # | . | & | + | - | [ > > type > sigil > identifier > sigil identifier > identifier < types > > sigil identifier < types > > > types > type > type , types > > > ## Core types > > > varType > "var" < type > > > tupleType > "tuple" > "tuple" < types > > > funcType > "func" < types > > > > # Java types > > > javaType > primitiveType | classType | wildCardType | arrayType | typeVariableType > > javaType-no-wildCardType > primitiveType | classType | arrayType | typeVariableType > > primitiveType > boolean | byte | ... | void > > classType > name > name < paramTypes > > . < enclosingType , innerType > > paramTypes > javaType > javaType , javaTypes > enclosingType > classType > innerType > classType > > wildcardType > + < wildcardTypeBound > > - < wildcardTypeBound > > wildcardTypeBound > javaType-no-wildCardType > > arrayType > dims < javaType > > dims > [ > [ dims > > typeVariableType > "#" name < typeVariableTypeOwner , typeVariableTypeBound > > typeVa... In the grammar for Java types, it would perhaps be better to start with: javaType primitiveType | classType | arrayType | typeVariableType javaTypeArgument wildcardType | javaType This allows you to use `javaType` w/o fears -- and only use wildcards where needed -- which is for class type parameters (in fact, we could also inline `wildcardType` into `paramTypes`). Otherwise, a lot of care is needed to use the correct production -- for instance, I see that `arrayType` seems to allow wildcard types (as written) -- which is against the JLS. (Of course the JavaType API has similar problems) ------------- PR Comment: https://git.openjdk.org/babylon/pull/416#issuecomment-2841217811 From mcimadamore at openjdk.org Wed Apr 30 08:36:00 2025 From: mcimadamore at openjdk.org (Maurizio Cimadamore) Date: Wed, 30 Apr 2025 08:36:00 GMT Subject: [code-reflection] RFR: JavaRef extends TypeElement In-Reply-To: <67BtibiQvMvf_mN3zQEpynENB26pqjxkUyVprL1xeVM=.bfab593c-f8fd-41f1-b527-36b384b52177@github.com> References: <67BtibiQvMvf_mN3zQEpynENB26pqjxkUyVprL1xeVM=.bfab593c-f8fd-41f1-b527-36b384b52177@github.com> Message-ID: On Tue, 29 Apr 2025 22:12:34 GMT, Paul Sandoz wrote: > Each implementation of `JavaRef` supports an external form, using the sigil prefix `&` to indicate a reference of some kind. > > The externalized form of `TypeVarRef` is changed to be uniformly structured. This required update to the code model text of a few tests. Arguably we are taking a step back in terms of readability, but we can address that later - it was easier to update a few tests rather than preserve the existing encoding in a few places. > > We still rely on the bespoke string form for java refs, which is used as the value of an externalized attribute. Ideally such attribute values are either instances of `JavaRef` or `ExternalizedTypeElement` to be transformed into the appropriate `JavaRef`. We can address that later to further separate out parsing. > > I believe we have what we need to further enhance the code model builder to not rely on bespoke parsing logic of types and refs. We can either construct `ExternalizedTypeElement` tree instances explicitly or parse from a very simple grammar. If we are careful i believe we can share the results of nested type elements if reused e.g. as in `List` and `Set`. > > More generally it now means we can generate a simple s-expression-like tree for the whole code model, e.g., a string where the `(` and `)` characters represent tree structure and say `L` represents a leaf node, and a list of leaf node values in topological order. > > -- > > Below is the type grammar, which i believe is consistent with what we have implemented. > > # Type element grammar > > ## General structure > > > identifier > string > > name > identifier > > sigil > # | . | & | + | - | [ > > type > sigil > identifier > sigil identifier > identifier < types > > sigil identifier < types > > > types > type > type , types > > > ## Core types > > > varType > "var" < type > > > tupleType > "tuple" > "tuple" < types > > > funcType > "func" < types > > > > # Java types > > > javaType > primitiveType | classType | wildCardType | arrayType | typeVariableType > > javaType-no-wildCardType > primitiveType | classType | arrayType | typeVariableType > > primitiveType > boolean | byte | ... | void > > classType > name > name < paramTypes > > . < enclosingType , innerType > > paramTypes > javaType > javaType , javaTypes > enclosingType > classType > innerType > classType > > wildcardType > + < wildcardTypeBound > > - < wildcardTypeBound > > wildcardTypeBound > javaType-no-wildCardType > > arrayType > dims < javaType > > dims > [ > [ dims > > typeVariableType > "#" name < typeVariableTypeOwner , typeVariableTypeBound > > typeVa... src/jdk.incubator.code/share/classes/jdk/incubator/code/type/TypeVarRef.java line 105: > 103: @Override > 104: public ExternalizedTypeElement externalize() { > 105: ExternalizedTypeElement eOwner = switch (owner) { Nice! ------------- PR Review Comment: https://git.openjdk.org/babylon/pull/416#discussion_r2068164672 From mcimadamore at openjdk.org Wed Apr 30 08:41:56 2025 From: mcimadamore at openjdk.org (Maurizio Cimadamore) Date: Wed, 30 Apr 2025 08:41:56 GMT Subject: [code-reflection] RFR: JavaRef extends TypeElement In-Reply-To: <67BtibiQvMvf_mN3zQEpynENB26pqjxkUyVprL1xeVM=.bfab593c-f8fd-41f1-b527-36b384b52177@github.com> References: <67BtibiQvMvf_mN3zQEpynENB26pqjxkUyVprL1xeVM=.bfab593c-f8fd-41f1-b527-36b384b52177@github.com> Message-ID: On Tue, 29 Apr 2025 22:12:34 GMT, Paul Sandoz wrote: > Each implementation of `JavaRef` supports an external form, using the sigil prefix `&` to indicate a reference of some kind. > > The externalized form of `TypeVarRef` is changed to be uniformly structured. This required update to the code model text of a few tests. Arguably we are taking a step back in terms of readability, but we can address that later - it was easier to update a few tests rather than preserve the existing encoding in a few places. > > We still rely on the bespoke string form for java refs, which is used as the value of an externalized attribute. Ideally such attribute values are either instances of `JavaRef` or `ExternalizedTypeElement` to be transformed into the appropriate `JavaRef`. We can address that later to further separate out parsing. > > I believe we have what we need to further enhance the code model builder to not rely on bespoke parsing logic of types and refs. We can either construct `ExternalizedTypeElement` tree instances explicitly or parse from a very simple grammar. If we are careful i believe we can share the results of nested type elements if reused e.g. as in `List` and `Set`. > > More generally it now means we can generate a simple s-expression-like tree for the whole code model, e.g., a string where the `(` and `)` characters represent tree structure and say `L` represents a leaf node, and a list of leaf node values in topological order. > > -- > > Below is the type grammar, which i believe is consistent with what we have implemented. > > # Type element grammar > > ## General structure > > > identifier > string > > name > identifier > > sigil > # | . | & | + | - | [ > > type > sigil > identifier > sigil identifier > identifier < types > > sigil identifier < types > > > types > type > type , types > > > ## Core types > > > varType > "var" < type > > > tupleType > "tuple" > "tuple" < types > > > funcType > "func" < types > > > > # Java types > > > javaType > primitiveType | classType | wildCardType | arrayType | typeVariableType > > javaType-no-wildCardType > primitiveType | classType | arrayType | typeVariableType > > primitiveType > boolean | byte | ... | void > > classType > name > name < paramTypes > > . < enclosingType , innerType > > paramTypes > javaType > javaType , javaTypes > enclosingType > classType > innerType > classType > > wildcardType > + < wildcardTypeBound > > - < wildcardTypeBound > > wildcardTypeBound > javaType-no-wildCardType > > arrayType > dims < javaType > > dims > [ > [ dims > > typeVariableType > "#" name < typeVariableTypeOwner , typeVariableTypeBound > > typeVa... Re s-expressions, it struck me that an alternate, more verbose representation, would be one where we have no sigils, but we use function application, and the function name determines the type being constructed. E.g. instead of: `Map` we could do: `class` Instead of: `#T` we could do: `tvar` There's not much between the two. But if we're after uniformity, the latter feels more uniform, as it treats the "name" as a true type factory, and gives all types a more uniform shape: type identifier identifier < types > I think we started off where we are now because we were attracted by the similarity with generic types. But it's becoming increasingly clear that we will need to display these types in a more human-readable fashion anyway -- at which point, do we still care if the grammar of a (class) type element is different from that of a generic Java type? (In fact, I'd consider replacing `<>` with `()`). ------------- PR Comment: https://git.openjdk.org/babylon/pull/416#issuecomment-2841240483 From mcimadamore at openjdk.org Wed Apr 30 08:47:01 2025 From: mcimadamore at openjdk.org (Maurizio Cimadamore) Date: Wed, 30 Apr 2025 08:47:01 GMT Subject: [code-reflection] RFR: JavaRef extends TypeElement In-Reply-To: <67BtibiQvMvf_mN3zQEpynENB26pqjxkUyVprL1xeVM=.bfab593c-f8fd-41f1-b527-36b384b52177@github.com> References: <67BtibiQvMvf_mN3zQEpynENB26pqjxkUyVprL1xeVM=.bfab593c-f8fd-41f1-b527-36b384b52177@github.com> Message-ID: On Tue, 29 Apr 2025 22:12:34 GMT, Paul Sandoz wrote: > Each implementation of `JavaRef` supports an external form, using the sigil prefix `&` to indicate a reference of some kind. > > The externalized form of `TypeVarRef` is changed to be uniformly structured. This required update to the code model text of a few tests. Arguably we are taking a step back in terms of readability, but we can address that later - it was easier to update a few tests rather than preserve the existing encoding in a few places. > > We still rely on the bespoke string form for java refs, which is used as the value of an externalized attribute. Ideally such attribute values are either instances of `JavaRef` or `ExternalizedTypeElement` to be transformed into the appropriate `JavaRef`. We can address that later to further separate out parsing. > > I believe we have what we need to further enhance the code model builder to not rely on bespoke parsing logic of types and refs. We can either construct `ExternalizedTypeElement` tree instances explicitly or parse from a very simple grammar. If we are careful i believe we can share the results of nested type elements if reused e.g. as in `List` and `Set`. > > More generally it now means we can generate a simple s-expression-like tree for the whole code model, e.g., a string where the `(` and `)` characters represent tree structure and say `L` represents a leaf node, and a list of leaf node values in topological order. > > -- > > Below is the type grammar, which i believe is consistent with what we have implemented. > > # Type element grammar > > ## General structure > > > identifier > string > > name > identifier > > sigil > # | . | & | + | - | [ > > type > sigil > identifier > sigil identifier > identifier < types > > sigil identifier < types > > > types > type > type , types > > > ## Core types > > > varType > "var" < type > > > tupleType > "tuple" > "tuple" < types > > > funcType > "func" < types > > > > # Java types > > > javaType > primitiveType | classType | wildCardType | arrayType | typeVariableType > > javaType-no-wildCardType > primitiveType | classType | arrayType | typeVariableType > > primitiveType > boolean | byte | ... | void > > classType > name > name < paramTypes > > . < enclosingType , innerType > > paramTypes > javaType > javaType , javaTypes > enclosingType > classType > innerType > classType > > wildcardType > + < wildcardTypeBound > > - < wildcardTypeBound > > wildcardTypeBound > javaType-no-wildCardType > > arrayType > dims < javaType > > dims > [ > [ dims > > typeVariableType > "#" name < typeVariableTypeOwner , typeVariableTypeBound > > typeVa... src/jdk.incubator.code/share/classes/jdk/incubator/code/type/ConstructorRef.java line 47: > 45: FunctionType type(); > 46: > 47: default TypeElement refType() { I don't get this default method -- the concrete impl overrides it. ------------- PR Review Comment: https://git.openjdk.org/babylon/pull/416#discussion_r2068183388 From mcimadamore at openjdk.org Wed Apr 30 08:51:55 2025 From: mcimadamore at openjdk.org (Maurizio Cimadamore) Date: Wed, 30 Apr 2025 08:51:55 GMT Subject: [code-reflection] RFR: JavaRef extends TypeElement In-Reply-To: <67BtibiQvMvf_mN3zQEpynENB26pqjxkUyVprL1xeVM=.bfab593c-f8fd-41f1-b527-36b384b52177@github.com> References: <67BtibiQvMvf_mN3zQEpynENB26pqjxkUyVprL1xeVM=.bfab593c-f8fd-41f1-b527-36b384b52177@github.com> Message-ID: On Tue, 29 Apr 2025 22:12:34 GMT, Paul Sandoz wrote: > Each implementation of `JavaRef` supports an external form, using the sigil prefix `&` to indicate a reference of some kind. > > The externalized form of `TypeVarRef` is changed to be uniformly structured. This required update to the code model text of a few tests. Arguably we are taking a step back in terms of readability, but we can address that later - it was easier to update a few tests rather than preserve the existing encoding in a few places. > > We still rely on the bespoke string form for java refs, which is used as the value of an externalized attribute. Ideally such attribute values are either instances of `JavaRef` or `ExternalizedTypeElement` to be transformed into the appropriate `JavaRef`. We can address that later to further separate out parsing. > > I believe we have what we need to further enhance the code model builder to not rely on bespoke parsing logic of types and refs. We can either construct `ExternalizedTypeElement` tree instances explicitly or parse from a very simple grammar. If we are careful i believe we can share the results of nested type elements if reused e.g. as in `List` and `Set`. > > More generally it now means we can generate a simple s-expression-like tree for the whole code model, e.g., a string where the `(` and `)` characters represent tree structure and say `L` represents a leaf node, and a list of leaf node values in topological order. > > -- > > Below is the type grammar, which i believe is consistent with what we have implemented. > > # Type element grammar > > ## General structure > > > identifier > string > > name > identifier > > sigil > # | . | & | + | - | [ > > type > sigil > identifier > sigil identifier > identifier < types > > sigil identifier < types > > > types > type > type , types > > > ## Core types > > > varType > "var" < type > > > tupleType > "tuple" > "tuple" < types > > > funcType > "func" < types > > > > # Java types > > > javaType > primitiveType | classType | wildCardType | arrayType | typeVariableType > > javaType-no-wildCardType > primitiveType | classType | arrayType | typeVariableType > > primitiveType > boolean | byte | ... | void > > classType > name > name < paramTypes > > . < enclosingType , innerType > > paramTypes > javaType > javaType , javaTypes > enclosingType > classType > innerType > classType > > wildcardType > + < wildcardTypeBound > > - < wildcardTypeBound > > wildcardTypeBound > javaType-no-wildCardType > > arrayType > dims < javaType > > dims > [ > [ dims > > typeVariableType > "#" name < typeVariableTypeOwner , typeVariableTypeBound > > typeVa... src/jdk.incubator.code/share/classes/jdk/incubator/code/type/CoreTypeFactory.java line 135: > 133: } > 134: > 135: return JavaType.typeVarRef(name, This is much nicer! No longer need to monkey around with string representation -- and can rely on nested type elements instead! ------------- PR Review Comment: https://git.openjdk.org/babylon/pull/416#discussion_r2068190952 From mcimadamore at openjdk.org Wed Apr 30 08:59:56 2025 From: mcimadamore at openjdk.org (Maurizio Cimadamore) Date: Wed, 30 Apr 2025 08:59:56 GMT Subject: [code-reflection] RFR: JavaRef extends TypeElement In-Reply-To: <67BtibiQvMvf_mN3zQEpynENB26pqjxkUyVprL1xeVM=.bfab593c-f8fd-41f1-b527-36b384b52177@github.com> References: <67BtibiQvMvf_mN3zQEpynENB26pqjxkUyVprL1xeVM=.bfab593c-f8fd-41f1-b527-36b384b52177@github.com> Message-ID: On Tue, 29 Apr 2025 22:12:34 GMT, Paul Sandoz wrote: > Each implementation of `JavaRef` supports an external form, using the sigil prefix `&` to indicate a reference of some kind. > > The externalized form of `TypeVarRef` is changed to be uniformly structured. This required update to the code model text of a few tests. Arguably we are taking a step back in terms of readability, but we can address that later - it was easier to update a few tests rather than preserve the existing encoding in a few places. > > We still rely on the bespoke string form for java refs, which is used as the value of an externalized attribute. Ideally such attribute values are either instances of `JavaRef` or `ExternalizedTypeElement` to be transformed into the appropriate `JavaRef`. We can address that later to further separate out parsing. > > I believe we have what we need to further enhance the code model builder to not rely on bespoke parsing logic of types and refs. We can either construct `ExternalizedTypeElement` tree instances explicitly or parse from a very simple grammar. If we are careful i believe we can share the results of nested type elements if reused e.g. as in `List` and `Set`. > > More generally it now means we can generate a simple s-expression-like tree for the whole code model, e.g., a string where the `(` and `)` characters represent tree structure and say `L` represents a leaf node, and a list of leaf node values in topological order. > > -- > > Below is the type grammar, which i believe is consistent with what we have implemented. > > # Type element grammar > > ## General structure > > > identifier > string > > name > identifier > > sigil > # | . | & | + | - | [ > > type > sigil > identifier > sigil identifier > identifier < types > > sigil identifier < types > > > types > type > type , types > > > ## Core types > > > varType > "var" < type > > > tupleType > "tuple" > "tuple" < types > > > funcType > "func" < types > > > > # Java types > > > javaType > primitiveType | classType | wildCardType | arrayType | typeVariableType > > javaType-no-wildCardType > primitiveType | classType | arrayType | typeVariableType > > primitiveType > boolean | byte | ... | void > > classType > name > name < paramTypes > > . < enclosingType , innerType > > paramTypes > javaType > javaType , javaTypes > enclosingType > classType > innerType > classType > > wildcardType > + < wildcardTypeBound > > - < wildcardTypeBound > > wildcardTypeBound > javaType-no-wildCardType > > arrayType > dims < javaType > > dims > [ > [ dims > > typeVariableType > "#" name < typeVariableTypeOwner , typeVariableTypeBound > > typeVa... test/jdk/java/lang/reflect/code/type/TestReferences.java line 52: > 50: {"java.io.PrintStream::println(java.lang.String)void", "java.io.PrintStream", "println"}, > 51: {"MethodReferenceTest$A::m(java.lang.Object)java.lang.Object", "MethodReferenceTest$A", "m"}, > 52: {"R<#T>::n()#T", "R<#T>", "n"} Parsing of MethodRef should eventually disappear, right? E.g. one thing is parsing the externalized method ref -- (which can be done in a principled way) but parsing a MethodRef using a slightly different externalized string (like in this test) seems confusing. (Same for all JavaRefs, really) ------------- PR Review Comment: https://git.openjdk.org/babylon/pull/416#discussion_r2068204591 From mcimadamore at openjdk.org Wed Apr 30 09:06:54 2025 From: mcimadamore at openjdk.org (Maurizio Cimadamore) Date: Wed, 30 Apr 2025 09:06:54 GMT Subject: [code-reflection] RFR: JavaRef extends TypeElement In-Reply-To: <67BtibiQvMvf_mN3zQEpynENB26pqjxkUyVprL1xeVM=.bfab593c-f8fd-41f1-b527-36b384b52177@github.com> References: <67BtibiQvMvf_mN3zQEpynENB26pqjxkUyVprL1xeVM=.bfab593c-f8fd-41f1-b527-36b384b52177@github.com> Message-ID: <_B-hgiRGV8keEwNF0JQ6Ek9hlk226Y_D-fyxssIBghk=.b7858021-8e64-4caa-ab3e-663fd37031f5@github.com> On Tue, 29 Apr 2025 22:12:34 GMT, Paul Sandoz wrote: > Each implementation of `JavaRef` supports an external form, using the sigil prefix `&` to indicate a reference of some kind. > > The externalized form of `TypeVarRef` is changed to be uniformly structured. This required update to the code model text of a few tests. Arguably we are taking a step back in terms of readability, but we can address that later - it was easier to update a few tests rather than preserve the existing encoding in a few places. > > We still rely on the bespoke string form for java refs, which is used as the value of an externalized attribute. Ideally such attribute values are either instances of `JavaRef` or `ExternalizedTypeElement` to be transformed into the appropriate `JavaRef`. We can address that later to further separate out parsing. > > I believe we have what we need to further enhance the code model builder to not rely on bespoke parsing logic of types and refs. We can either construct `ExternalizedTypeElement` tree instances explicitly or parse from a very simple grammar. If we are careful i believe we can share the results of nested type elements if reused e.g. as in `List` and `Set`. > > More generally it now means we can generate a simple s-expression-like tree for the whole code model, e.g., a string where the `(` and `)` characters represent tree structure and say `L` represents a leaf node, and a list of leaf node values in topological order. > > -- > > Below is the type grammar, which i believe is consistent with what we have implemented. > > # Type element grammar > > ## General structure > > > identifier > string > > name > identifier > > sigil > # | . | & | + | - | [ > > type > sigil > identifier > sigil identifier > identifier < types > > sigil identifier < types > > > types > type > type , types > > > ## Core types > > > varType > "var" < type > > > tupleType > "tuple" > "tuple" < types > > > funcType > "func" < types > > > > # Java types > > > javaType > primitiveType | classType | wildCardType | arrayType | typeVariableType > > javaType-no-wildCardType > primitiveType | classType | arrayType | typeVariableType > > primitiveType > boolean | byte | ... | void > > classType > name > name < paramTypes > > . < enclosingType , innerType > > paramTypes > javaType > javaType , javaTypes > enclosingType > classType > innerType > classType > > wildcardType > + < wildcardTypeBound > > - < wildcardTypeBound > > wildcardTypeBound > javaType-no-wildCardType > > arrayType > dims < javaType > > dims > [ > [ dims > > typeVariableType > "#" name < typeVariableTypeOwner , typeVariableTypeBound > > typeVa... src/jdk.incubator.code/share/classes/jdk/incubator/code/type/JavaRef.java line 21: > 19: // @@@ Make RecordTypeRef.ComponentRef implement JavaRef? > 20: // - resolve to accessor method > 21: // - (RecordTypeRef resolves to Type.) Shouldn't it resolve to `RecordComponent` ? ------------- PR Review Comment: https://git.openjdk.org/babylon/pull/416#discussion_r2068217813 From mcimadamore at openjdk.org Wed Apr 30 09:06:55 2025 From: mcimadamore at openjdk.org (Maurizio Cimadamore) Date: Wed, 30 Apr 2025 09:06:55 GMT Subject: [code-reflection] RFR: JavaRef extends TypeElement In-Reply-To: References: <67BtibiQvMvf_mN3zQEpynENB26pqjxkUyVprL1xeVM=.bfab593c-f8fd-41f1-b527-36b384b52177@github.com> Message-ID: On Wed, 30 Apr 2025 08:56:40 GMT, Maurizio Cimadamore wrote: >> Each implementation of `JavaRef` supports an external form, using the sigil prefix `&` to indicate a reference of some kind. >> >> The externalized form of `TypeVarRef` is changed to be uniformly structured. This required update to the code model text of a few tests. Arguably we are taking a step back in terms of readability, but we can address that later - it was easier to update a few tests rather than preserve the existing encoding in a few places. >> >> We still rely on the bespoke string form for java refs, which is used as the value of an externalized attribute. Ideally such attribute values are either instances of `JavaRef` or `ExternalizedTypeElement` to be transformed into the appropriate `JavaRef`. We can address that later to further separate out parsing. >> >> I believe we have what we need to further enhance the code model builder to not rely on bespoke parsing logic of types and refs. We can either construct `ExternalizedTypeElement` tree instances explicitly or parse from a very simple grammar. If we are careful i believe we can share the results of nested type elements if reused e.g. as in `List` and `Set`. >> >> More generally it now means we can generate a simple s-expression-like tree for the whole code model, e.g., a string where the `(` and `)` characters represent tree structure and say `L` represents a leaf node, and a list of leaf node values in topological order. >> >> -- >> >> Below is the type grammar, which i believe is consistent with what we have implemented. >> >> # Type element grammar >> >> ## General structure >> >> >> identifier >> string >> >> name >> identifier >> >> sigil >> # | . | & | + | - | [ >> >> type >> sigil >> identifier >> sigil identifier >> identifier < types > >> sigil identifier < types > >> >> types >> type >> type , types >> >> >> ## Core types >> >> >> varType >> "var" < type > >> >> tupleType >> "tuple" >> "tuple" < types > >> >> funcType >> "func" < types > >> >> >> # Java types >> >> >> javaType >> primitiveType | classType | wildCardType | arrayType | typeVariableType >> >> javaType-no-wildCardType >> primitiveType | classType | arrayType | typeVariableType >> >> primitiveType >> boolean | byte | ... | void >> >> classType >> name >> name < paramTypes > >> . < enclosingType , innerType > >> paramTypes >> javaType >> javaType , javaTypes >> enclosingType >> classType >> innerType >> classType >> >> wildcardType >> + < wildcardTypeBound > >> - < wildcardTypeBound > >> wildcardTypeB... > > test/jdk/java/lang/reflect/code/type/TestReferences.java line 52: > >> 50: {"java.io.PrintStream::println(java.lang.String)void", "java.io.PrintStream", "println"}, >> 51: {"MethodReferenceTest$A::m(java.lang.Object)java.lang.Object", "MethodReferenceTest$A", "m"}, >> 52: {"R<#T>::n()#T", "R<#T>", "n"} > > Parsing of MethodRef should eventually disappear, right? E.g. one thing is parsing the externalized method ref -- (which can be done in a principled way) but parsing a MethodRef using a slightly different externalized string (like in this test) seems confusing. (Same for all JavaRefs, really) Ok, this is covered by your `We still rely on the bespoke string form for java refs, which is used as the value of an externalized attribute` comment. And also explains why there are only so few changes in tests (I was expecting anything using invoke, or field load to need a refresh :-) ). ------------- PR Review Comment: https://git.openjdk.org/babylon/pull/416#discussion_r2068215031 From mcimadamore at openjdk.org Wed Apr 30 09:14:58 2025 From: mcimadamore at openjdk.org (Maurizio Cimadamore) Date: Wed, 30 Apr 2025 09:14:58 GMT Subject: [code-reflection] RFR: JavaRef extends TypeElement In-Reply-To: <67BtibiQvMvf_mN3zQEpynENB26pqjxkUyVprL1xeVM=.bfab593c-f8fd-41f1-b527-36b384b52177@github.com> References: <67BtibiQvMvf_mN3zQEpynENB26pqjxkUyVprL1xeVM=.bfab593c-f8fd-41f1-b527-36b384b52177@github.com> Message-ID: <9XgdeyH_PJAL8pymlcEcCpVh_n8xbJobcL5vS-GtcBM=.7362e9dc-077c-4f74-8e30-ebc7157a2b76@github.com> On Tue, 29 Apr 2025 22:12:34 GMT, Paul Sandoz wrote: > Each implementation of `JavaRef` supports an external form, using the sigil prefix `&` to indicate a reference of some kind. > > The externalized form of `TypeVarRef` is changed to be uniformly structured. This required update to the code model text of a few tests. Arguably we are taking a step back in terms of readability, but we can address that later - it was easier to update a few tests rather than preserve the existing encoding in a few places. > > We still rely on the bespoke string form for java refs, which is used as the value of an externalized attribute. Ideally such attribute values are either instances of `JavaRef` or `ExternalizedTypeElement` to be transformed into the appropriate `JavaRef`. We can address that later to further separate out parsing. > > I believe we have what we need to further enhance the code model builder to not rely on bespoke parsing logic of types and refs. We can either construct `ExternalizedTypeElement` tree instances explicitly or parse from a very simple grammar. If we are careful i believe we can share the results of nested type elements if reused e.g. as in `List` and `Set`. > > More generally it now means we can generate a simple s-expression-like tree for the whole code model, e.g., a string where the `(` and `)` characters represent tree structure and say `L` represents a leaf node, and a list of leaf node values in topological order. > > -- > > Below is the type grammar, which i believe is consistent with what we have implemented. > > # Type element grammar > > ## General structure > > > identifier > string > > name > identifier > > sigil > # | . | & | + | - | [ > > type > sigil > identifier > sigil identifier > identifier < types > > sigil identifier < types > > > types > type > type , types > > > ## Core types > > > varType > "var" < type > > > tupleType > "tuple" > "tuple" < types > > > funcType > "func" < types > > > > # Java types > > > javaType > primitiveType | classType | wildCardType | arrayType | typeVariableType > > javaType-no-wildCardType > primitiveType | classType | arrayType | typeVariableType > > primitiveType > boolean | byte | ... | void > > classType > name > name < paramTypes > > . < enclosingType , innerType > > paramTypes > javaType > javaType , javaTypes > enclosingType > classType > innerType > classType > > wildcardType > + < wildcardTypeBound > > - < wildcardTypeBound > > wildcardTypeBound > javaType-no-wildCardType > > arrayType > dims < javaType > > dims > [ > [ dims > > typeVariableType > "#" name < typeVariableTypeOwner , typeVariableTypeBound > > typeVa... Very good cleanup! src/jdk.incubator.code/share/classes/jdk/incubator/code/type/TypeVarRef.java line 117: > 115: public String toString() { > 116: // @@@ required to pass TestJavaType.java > 117: return name; What should the `toString()` method in JavaType do? Should they return a human readable representation that will then be embedded in the code model (e.g. when calling `toText`) ? Or should we also have `toText` on all `TypeElement`s ? ------------- Marked as reviewed by mcimadamore (Reviewer). PR Review: https://git.openjdk.org/babylon/pull/416#pullrequestreview-2806344809 PR Review Comment: https://git.openjdk.org/babylon/pull/416#discussion_r2068228209 From psandoz at openjdk.org Wed Apr 30 18:04:05 2025 From: psandoz at openjdk.org (Paul Sandoz) Date: Wed, 30 Apr 2025 18:04:05 GMT Subject: [code-reflection] RFR: JavaRef extends TypeElement In-Reply-To: References: <67BtibiQvMvf_mN3zQEpynENB26pqjxkUyVprL1xeVM=.bfab593c-f8fd-41f1-b527-36b384b52177@github.com> Message-ID: On Wed, 30 Apr 2025 08:44:12 GMT, Maurizio Cimadamore wrote: >> Each implementation of `JavaRef` supports an external form, using the sigil prefix `&` to indicate a reference of some kind. >> >> The externalized form of `TypeVarRef` is changed to be uniformly structured. This required update to the code model text of a few tests. Arguably we are taking a step back in terms of readability, but we can address that later - it was easier to update a few tests rather than preserve the existing encoding in a few places. >> >> We still rely on the bespoke string form for java refs, which is used as the value of an externalized attribute. Ideally such attribute values are either instances of `JavaRef` or `ExternalizedTypeElement` to be transformed into the appropriate `JavaRef`. We can address that later to further separate out parsing. >> >> I believe we have what we need to further enhance the code model builder to not rely on bespoke parsing logic of types and refs. We can either construct `ExternalizedTypeElement` tree instances explicitly or parse from a very simple grammar. If we are careful i believe we can share the results of nested type elements if reused e.g. as in `List` and `Set`. >> >> More generally it now means we can generate a simple s-expression-like tree for the whole code model, e.g., a string where the `(` and `)` characters represent tree structure and say `L` represents a leaf node, and a list of leaf node values in topological order. >> >> -- >> >> Below is the type grammar, which i believe is consistent with what we have implemented. >> >> # Type element grammar >> >> ## General structure >> >> >> identifier >> string >> >> name >> identifier >> >> sigil >> # | . | & | + | - | [ >> >> type >> sigil >> identifier >> sigil identifier >> identifier < types > >> sigil identifier < types > >> >> types >> type >> type , types >> >> >> ## Core types >> >> >> varType >> "var" < type > >> >> tupleType >> "tuple" >> "tuple" < types > >> >> funcType >> "func" < types > >> >> >> # Java types >> >> >> javaType >> primitiveType | classType | wildCardType | arrayType | typeVariableType >> >> javaType-no-wildCardType >> primitiveType | classType | arrayType | typeVariableType >> >> primitiveType >> boolean | byte | ... | void >> >> classType >> name >> name < paramTypes > >> . < enclosingType , innerType > >> paramTypes >> javaType >> javaType , javaTypes >> enclosingType >> classType >> innerType >> classType >> >> wildcardType >> + < wildcardTypeBound > >> - < wildcardTypeBound > >> wildcardTypeB... > > src/jdk.incubator.code/share/classes/jdk/incubator/code/type/ConstructorRef.java line 47: > >> 45: FunctionType type(); >> 46: >> 47: default TypeElement refType() { > > I don't get this default method -- the concrete impl overrides it. I'll clean that up, there is no need for the extra field in `ConstructorRefImpl`. > src/jdk.incubator.code/share/classes/jdk/incubator/code/type/JavaRef.java line 21: > >> 19: // @@@ Make RecordTypeRef.ComponentRef implement JavaRef? >> 20: // - resolve to accessor method >> 21: // - (RecordTypeRef resolves to Type.) > > Shouldn't it resolve to `RecordComponent` ? Yes! ------------- PR Review Comment: https://git.openjdk.org/babylon/pull/416#discussion_r2069199522 PR Review Comment: https://git.openjdk.org/babylon/pull/416#discussion_r2069200684 From psandoz at openjdk.org Wed Apr 30 18:11:06 2025 From: psandoz at openjdk.org (Paul Sandoz) Date: Wed, 30 Apr 2025 18:11:06 GMT Subject: [code-reflection] RFR: JavaRef extends TypeElement In-Reply-To: <9XgdeyH_PJAL8pymlcEcCpVh_n8xbJobcL5vS-GtcBM=.7362e9dc-077c-4f74-8e30-ebc7157a2b76@github.com> References: <67BtibiQvMvf_mN3zQEpynENB26pqjxkUyVprL1xeVM=.bfab593c-f8fd-41f1-b527-36b384b52177@github.com> <9XgdeyH_PJAL8pymlcEcCpVh_n8xbJobcL5vS-GtcBM=.7362e9dc-077c-4f74-8e30-ebc7157a2b76@github.com> Message-ID: On Wed, 30 Apr 2025 09:11:04 GMT, Maurizio Cimadamore wrote: >> Each implementation of `JavaRef` supports an external form, using the sigil prefix `&` to indicate a reference of some kind. >> >> The externalized form of `TypeVarRef` is changed to be uniformly structured. This required update to the code model text of a few tests. Arguably we are taking a step back in terms of readability, but we can address that later - it was easier to update a few tests rather than preserve the existing encoding in a few places. >> >> We still rely on the bespoke string form for java refs, which is used as the value of an externalized attribute. Ideally such attribute values are either instances of `JavaRef` or `ExternalizedTypeElement` to be transformed into the appropriate `JavaRef`. We can address that later to further separate out parsing. >> >> I believe we have what we need to further enhance the code model builder to not rely on bespoke parsing logic of types and refs. We can either construct `ExternalizedTypeElement` tree instances explicitly or parse from a very simple grammar. If we are careful i believe we can share the results of nested type elements if reused e.g. as in `List` and `Set`. >> >> More generally it now means we can generate a simple s-expression-like tree for the whole code model, e.g., a string where the `(` and `)` characters represent tree structure and say `L` represents a leaf node, and a list of leaf node values in topological order. >> >> -- >> >> Below is the type grammar, which i believe is consistent with what we have implemented. >> >> # Type element grammar >> >> ## General structure >> >> >> identifier >> string >> >> name >> identifier >> >> sigil >> # | . | & | + | - | [ >> >> type >> sigil >> identifier >> sigil identifier >> identifier < types > >> sigil identifier < types > >> >> types >> type >> type , types >> >> >> ## Core types >> >> >> varType >> "var" < type > >> >> tupleType >> "tuple" >> "tuple" < types > >> >> funcType >> "func" < types > >> >> >> # Java types >> >> >> javaType >> primitiveType | classType | wildCardType | arrayType | typeVariableType >> >> javaType-no-wildCardType >> primitiveType | classType | arrayType | typeVariableType >> >> primitiveType >> boolean | byte | ... | void >> >> classType >> name >> name < paramTypes > >> . < enclosingType , innerType > >> paramTypes >> javaType >> javaType , javaTypes >> enclosingType >> classType >> innerType >> classType >> >> wildcardType >> + < wildcardTypeBound > >> - < wildcardTypeBound > >> wildcardTypeB... > > src/jdk.incubator.code/share/classes/jdk/incubator/code/type/TypeVarRef.java line 117: > >> 115: public String toString() { >> 116: // @@@ required to pass TestJavaType.java >> 117: return name; > > What should the `toString()` method in JavaType do? Should they return a human readable representation that will then be embedded in the code model (e.g. when calling `toText`) ? Or should we also have `toText` on all `TypeElement`s ? Hmm... in this case `toString()` seems appropriate to return a human readable representation (that `OpWriter` writes and `OpParser` parses, but we don't have to specify the grammar), given that i would not expect these strings to be very large, unlike the text of code models. (The comment indicates some testing modifications would be required for type variable types.) The default implementation of `TypeElement.toString` could be `return externalize().toString()`. ------------- PR Review Comment: https://git.openjdk.org/babylon/pull/416#discussion_r2069209651 From psandoz at openjdk.org Wed Apr 30 18:13:57 2025 From: psandoz at openjdk.org (Paul Sandoz) Date: Wed, 30 Apr 2025 18:13:57 GMT Subject: [code-reflection] RFR: JavaRef extends TypeElement In-Reply-To: References: <67BtibiQvMvf_mN3zQEpynENB26pqjxkUyVprL1xeVM=.bfab593c-f8fd-41f1-b527-36b384b52177@github.com> Message-ID: <6HmFvtmVnSn_UEDmvOXOlv5XFADrX-TI3R9dfl3gC5c=.f35c26d0-9f44-424e-9b49-1c671e73ead6@github.com> On Wed, 30 Apr 2025 09:02:38 GMT, Maurizio Cimadamore wrote: >> test/jdk/java/lang/reflect/code/type/TestReferences.java line 52: >> >>> 50: {"java.io.PrintStream::println(java.lang.String)void", "java.io.PrintStream", "println"}, >>> 51: {"MethodReferenceTest$A::m(java.lang.Object)java.lang.Object", "MethodReferenceTest$A", "m"}, >>> 52: {"R<#T>::n()#T", "R<#T>", "n"} >> >> Parsing of MethodRef should eventually disappear, right? E.g. one thing is parsing the externalized method ref -- (which can be done in a principled way) but parsing a MethodRef using a slightly different externalized string (like in this test) seems confusing. (Same for all JavaRefs, really) > > Ok, this is covered by your `We still rely on the bespoke string form for java refs, which is used as the value of an externalized attribute` comment. And also explains why there are only so few changes in tests (I was expecting anything using invoke, or field load to need a refresh :-) ). Indeed. The parsing of java refs should eventually become a private thing of the `OpParser`. So `OpWriter` can use `JavaRef.toString` i.e., an identical approach proposed for `TypeElement`. ------------- PR Review Comment: https://git.openjdk.org/babylon/pull/416#discussion_r2069213667 From psandoz at openjdk.org Wed Apr 30 18:26:02 2025 From: psandoz at openjdk.org (Paul Sandoz) Date: Wed, 30 Apr 2025 18:26:02 GMT Subject: [code-reflection] RFR: JavaRef extends TypeElement In-Reply-To: References: <67BtibiQvMvf_mN3zQEpynENB26pqjxkUyVprL1xeVM=.bfab593c-f8fd-41f1-b527-36b384b52177@github.com> Message-ID: On Wed, 30 Apr 2025 08:29:59 GMT, Maurizio Cimadamore wrote: > In the grammar for Java types, it would perhaps be better to start with: > > ``` > javaType > primitiveType | classType | arrayType | typeVariableType > > javaTypeArgument > wildcardType | javaType > ``` > Good point, I was basically trying to reverse engineer from our current implementation to more easily spot patterns/structure. It indicates we have some bugs in our implementation of the java type factories! Let's address this separately (and we can place the grammar in the repo somewhere). ------------- PR Comment: https://git.openjdk.org/babylon/pull/416#issuecomment-2842909769 From psandoz at openjdk.org Wed Apr 30 18:37:59 2025 From: psandoz at openjdk.org (Paul Sandoz) Date: Wed, 30 Apr 2025 18:37:59 GMT Subject: [code-reflection] RFR: JavaRef extends TypeElement In-Reply-To: References: <67BtibiQvMvf_mN3zQEpynENB26pqjxkUyVprL1xeVM=.bfab593c-f8fd-41f1-b527-36b384b52177@github.com> Message-ID: On Wed, 30 Apr 2025 08:39:08 GMT, Maurizio Cimadamore wrote: > Re s-expressions, it struck me that an alternate, more verbose representation, would be one where we have no sigils, but we use function application, and the function name determines the type being constructed. E.g. instead of: That seems more principled. We could consider that after we have tackled the human readability of Java types. What might also help is what dialect the externalized type is associated with (and more generally for ops), which allows for a better association of factories. For non-Java types, the approach works quite nicely so far in many cases e.g., in the Triton or ONNX examples, where we don't really require sigils. Dialect identification would be useful because we can mix types from different dialects e.g., Triton reuses Java primitive types such as for `tensor`, so we would use the externalized form of the Java type here rather than the human readable form given it is nested. ------------- PR Comment: https://git.openjdk.org/babylon/pull/416#issuecomment-2842936925 From psandoz at openjdk.org Wed Apr 30 18:45:13 2025 From: psandoz at openjdk.org (Paul Sandoz) Date: Wed, 30 Apr 2025 18:45:13 GMT Subject: [code-reflection] RFR: JavaRef extends TypeElement [v2] In-Reply-To: <67BtibiQvMvf_mN3zQEpynENB26pqjxkUyVprL1xeVM=.bfab593c-f8fd-41f1-b527-36b384b52177@github.com> References: <67BtibiQvMvf_mN3zQEpynENB26pqjxkUyVprL1xeVM=.bfab593c-f8fd-41f1-b527-36b384b52177@github.com> Message-ID: <_F67lOshTs1HY5ESN41xnN30psFJjIpMKrD9ypAOSNI=.47aba314-a778-4dd0-b0cc-9d83812964e6@github.com> > Each implementation of `JavaRef` supports an external form, using the sigil prefix `&` to indicate a reference of some kind. > > The externalized form of `TypeVarRef` is changed to be uniformly structured. This required update to the code model text of a few tests. Arguably we are taking a step back in terms of readability, but we can address that later - it was easier to update a few tests rather than preserve the existing encoding in a few places. > > We still rely on the bespoke string form for java refs, which is used as the value of an externalized attribute. Ideally such attribute values are either instances of `JavaRef` or `ExternalizedTypeElement` to be transformed into the appropriate `JavaRef`. We can address that later to further separate out parsing. > > I believe we have what we need to further enhance the code model builder to not rely on bespoke parsing logic of types and refs. We can either construct `ExternalizedTypeElement` tree instances explicitly or parse from a very simple grammar. If we are careful i believe we can share the results of nested type elements if reused e.g. as in `List` and `Set`. > > More generally it now means we can generate a simple s-expression-like tree for the whole code model, e.g., a string where the `(` and `)` characters represent tree structure and say `L` represents a leaf node, and a list of leaf node values in topological order. > > -- > > Below is the type grammar, which i believe is consistent with what we have implemented. > > # Type element grammar > > ## General structure > > > identifier > string > > name > identifier > > sigil > # | . | & | + | - | [ > > type > sigil > identifier > sigil identifier > identifier < types > > sigil identifier < types > > > types > type > type , types > > > ## Core types > > > varType > "var" < type > > > tupleType > "tuple" > "tuple" < types > > > funcType > "func" < types > > > > # Java types > > > javaType > primitiveType | classType | wildCardType | arrayType | typeVariableType > > javaType-no-wildCardType > primitiveType | classType | arrayType | typeVariableType > > primitiveType > boolean | byte | ... | void > > classType > name > name < paramTypes > > . < enclosingType , innerType > > paramTypes > javaType > javaType , javaTypes > enclosingType > classType > innerType > classType > > wildcardType > + < wildcardTypeBound > > - < wildcardTypeBound > > wildcardTypeBound > javaType-no-wildCardType > > arrayType > dims < javaType > > dims > [ > [ dims > > typeVariableType > "#" name < typeVariableTypeOwner , typeVariableTypeBound > > typeVa... Paul Sandoz has updated the pull request incrementally with one additional commit since the last revision: Clean up ConstructorRef ------------- Changes: - all: https://git.openjdk.org/babylon/pull/416/files - new: https://git.openjdk.org/babylon/pull/416/files/27da7be0..52a7ec90 Webrevs: - full: https://webrevs.openjdk.org/?repo=babylon&pr=416&range=01 - incr: https://webrevs.openjdk.org/?repo=babylon&pr=416&range=00-01 Stats: 22 lines in 2 files changed: 1 ins; 14 del; 7 mod Patch: https://git.openjdk.org/babylon/pull/416.diff Fetch: git fetch https://git.openjdk.org/babylon.git pull/416/head:pull/416 PR: https://git.openjdk.org/babylon/pull/416 From psandoz at openjdk.org Wed Apr 30 19:33:00 2025 From: psandoz at openjdk.org (Paul Sandoz) Date: Wed, 30 Apr 2025 19:33:00 GMT Subject: git: openjdk/babylon: code-reflection: JavaRef extends TypeElement Message-ID: Changeset: de806577 Branch: code-reflection Author: Paul Sandoz Date: 2025-04-30 19:31:22 +0000 URL: https://git.openjdk.org/babylon/commit/de806577d30f4b792e231a94546d8fac2d4695ef JavaRef extends TypeElement Reviewed-by: mcimadamore ! src/jdk.incubator.code/share/classes/jdk/incubator/code/TypeElement.java ! src/jdk.incubator.code/share/classes/jdk/incubator/code/parser/impl/DescParser.java ! src/jdk.incubator.code/share/classes/jdk/incubator/code/type/ConstructorRef.java ! src/jdk.incubator.code/share/classes/jdk/incubator/code/type/CoreTypeFactory.java ! src/jdk.incubator.code/share/classes/jdk/incubator/code/type/JavaRef.java ! src/jdk.incubator.code/share/classes/jdk/incubator/code/type/TypeVarRef.java ! src/jdk.incubator.code/share/classes/jdk/incubator/code/type/impl/ConstructorRefImpl.java ! src/jdk.incubator.code/share/classes/jdk/incubator/code/type/impl/FieldRefImpl.java ! src/jdk.incubator.code/share/classes/jdk/incubator/code/type/impl/MethodRefImpl.java ! src/jdk.incubator.code/share/classes/jdk/incubator/code/type/impl/RecordTypeRefImpl.java ! test/jdk/java/lang/reflect/code/type/TestReferences.java ! test/langtools/tools/javac/reflect/DenotableTypesTest.java ! test/langtools/tools/javac/reflect/IntersectionTypeTest.java ! test/langtools/tools/javac/reflect/PatternTest2.java From psandoz at openjdk.org Wed Apr 30 19:34:00 2025 From: psandoz at openjdk.org (Paul Sandoz) Date: Wed, 30 Apr 2025 19:34:00 GMT Subject: [code-reflection] Integrated: JavaRef extends TypeElement In-Reply-To: <67BtibiQvMvf_mN3zQEpynENB26pqjxkUyVprL1xeVM=.bfab593c-f8fd-41f1-b527-36b384b52177@github.com> References: <67BtibiQvMvf_mN3zQEpynENB26pqjxkUyVprL1xeVM=.bfab593c-f8fd-41f1-b527-36b384b52177@github.com> Message-ID: On Tue, 29 Apr 2025 22:12:34 GMT, Paul Sandoz wrote: > Each implementation of `JavaRef` supports an external form, using the sigil prefix `&` to indicate a reference of some kind. > > The externalized form of `TypeVarRef` is changed to be uniformly structured. This required update to the code model text of a few tests. Arguably we are taking a step back in terms of readability, but we can address that later - it was easier to update a few tests rather than preserve the existing encoding in a few places. > > We still rely on the bespoke string form for java refs, which is used as the value of an externalized attribute. Ideally such attribute values are either instances of `JavaRef` or `ExternalizedTypeElement` to be transformed into the appropriate `JavaRef`. We can address that later to further separate out parsing. > > I believe we have what we need to further enhance the code model builder to not rely on bespoke parsing logic of types and refs. We can either construct `ExternalizedTypeElement` tree instances explicitly or parse from a very simple grammar. If we are careful i believe we can share the results of nested type elements if reused e.g. as in `List` and `Set`. > > More generally it now means we can generate a simple s-expression-like tree for the whole code model, e.g., a string where the `(` and `)` characters represent tree structure and say `L` represents a leaf node, and a list of leaf node values in topological order. > > -- > > Below is the type grammar, which i believe is consistent with what we have implemented. > > # Type element grammar > > ## General structure > > > identifier > string > > name > identifier > > sigil > # | . | & | + | - | [ > > type > sigil > identifier > sigil identifier > identifier < types > > sigil identifier < types > > > types > type > type , types > > > ## Core types > > > varType > "var" < type > > > tupleType > "tuple" > "tuple" < types > > > funcType > "func" < types > > > > # Java types > > > javaType > primitiveType | classType | wildCardType | arrayType | typeVariableType > > javaType-no-wildCardType > primitiveType | classType | arrayType | typeVariableType > > primitiveType > boolean | byte | ... | void > > classType > name > name < paramTypes > > . < enclosingType , innerType > > paramTypes > javaType > javaType , javaTypes > enclosingType > classType > innerType > classType > > wildcardType > + < wildcardTypeBound > > - < wildcardTypeBound > > wildcardTypeBound > javaType-no-wildCardType > > arrayType > dims < javaType > > dims > [ > [ dims > > typeVariableType > "#" name < typeVariableTypeOwner , typeVariableTypeBound > > typeVa... This pull request has now been integrated. Changeset: de806577 Author: Paul Sandoz URL: https://git.openjdk.org/babylon/commit/de806577d30f4b792e231a94546d8fac2d4695ef Stats: 309 lines in 14 files changed: 215 ins; 36 del; 58 mod JavaRef extends TypeElement Reviewed-by: mcimadamore ------------- PR: https://git.openjdk.org/babylon/pull/416 From psandoz at openjdk.org Wed Apr 30 19:46:42 2025 From: psandoz at openjdk.org (Paul Sandoz) Date: Wed, 30 Apr 2025 19:46:42 GMT Subject: [code-reflection] RFR: Rename TypeVarRef to TypeVariableType Message-ID: Rename `TypeVarRef` to `TypeVariableType` ------------- Commit messages: - Rename TypeVarRef to TypeVariableType Changes: https://git.openjdk.org/babylon/pull/417/files Webrev: https://webrevs.openjdk.org/?repo=babylon&pr=417&range=00 Stats: 19 lines in 7 files changed: 0 ins; 4 del; 15 mod Patch: https://git.openjdk.org/babylon/pull/417.diff Fetch: git fetch https://git.openjdk.org/babylon.git pull/417/head:pull/417 PR: https://git.openjdk.org/babylon/pull/417 From mcimadamore at openjdk.org Wed Apr 30 21:11:05 2025 From: mcimadamore at openjdk.org (Maurizio Cimadamore) Date: Wed, 30 Apr 2025 21:11:05 GMT Subject: [code-reflection] RFR: Rename TypeVarRef to TypeVariableType In-Reply-To: References: Message-ID: On Wed, 30 Apr 2025 19:41:32 GMT, Paul Sandoz wrote: > Rename `TypeVarRef` to `TypeVariableType` New name looks much better! ------------- Marked as reviewed by mcimadamore (Reviewer). PR Review: https://git.openjdk.org/babylon/pull/417#pullrequestreview-2808451543 From mcimadamore at openjdk.org Wed Apr 30 21:19:04 2025 From: mcimadamore at openjdk.org (Maurizio Cimadamore) Date: Wed, 30 Apr 2025 21:19:04 GMT Subject: [code-reflection] RFR: JavaRef extends TypeElement [v2] In-Reply-To: References: <67BtibiQvMvf_mN3zQEpynENB26pqjxkUyVprL1xeVM=.bfab593c-f8fd-41f1-b527-36b384b52177@github.com> Message-ID: <4CNchtey30bFot_54QaAw8WJrhQOgkNZGr7uZFukWNI=.dad6402a-82ac-46c2-9fcb-6a90ddf1300b@github.com> On Wed, 30 Apr 2025 18:01:27 GMT, Paul Sandoz wrote: >> src/jdk.incubator.code/share/classes/jdk/incubator/code/type/JavaRef.java line 21: >> >>> 19: // @@@ Make RecordTypeRef.ComponentRef implement JavaRef? >>> 20: // - resolve to accessor method >>> 21: // - (RecordTypeRef resolves to Type.) >> >> Shouldn't it resolve to `RecordComponent` ? > > Yes! Question - if Java code calls a method on a record... is the "ref" attached to the invoke operation a record type ref? I don't think that is the case, as I don't remember any special logic in javac to emit a record type ref when generating field/method references? ------------- PR Review Comment: https://git.openjdk.org/babylon/pull/416#discussion_r2069453336 From mcimadamore at openjdk.org Wed Apr 30 21:23:03 2025 From: mcimadamore at openjdk.org (Maurizio Cimadamore) Date: Wed, 30 Apr 2025 21:23:03 GMT Subject: [code-reflection] RFR: JavaRef extends TypeElement [v2] In-Reply-To: References: <67BtibiQvMvf_mN3zQEpynENB26pqjxkUyVprL1xeVM=.bfab593c-f8fd-41f1-b527-36b384b52177@github.com> <9XgdeyH_PJAL8pymlcEcCpVh_n8xbJobcL5vS-GtcBM=.7362e9dc-077c-4f74-8e30-ebc7157a2b76@github.com> Message-ID: On Wed, 30 Apr 2025 18:08:25 GMT, Paul Sandoz wrote: >> src/jdk.incubator.code/share/classes/jdk/incubator/code/type/TypeVarRef.java line 117: >> >>> 115: public String toString() { >>> 116: // @@@ required to pass TestJavaType.java >>> 117: return name; >> >> What should the `toString()` method in JavaType do? Should they return a human readable representation that will then be embedded in the code model (e.g. when calling `toText`) ? Or should we also have `toText` on all `TypeElement`s ? > > Hmm... in this case `toString()` seems appropriate to return a human readable representation (that `OpWriter` writes and `OpParser` parses, but we don't have to specify the grammar), given that i would not expect these strings to be very large, unlike the text of code models. > > (The comment indicates some testing modifications would be required for type variable types.) > The default implementation of `TypeElement.toString` could be `return externalize().toString()`. The issue is that there is a clear tension between things being parseable and things being readable. If you want things to be parseable, you need to include all the information that is in the ref/type. For instance, a type-variable should have the owner printed out -- as well as the bound. But for a type-variable, a human readable representation would just expect to see the tvar name (and maybe the owner, at least in some ambiguous cases). But such a representation would be lossy, and not parseable. ------------- PR Review Comment: https://git.openjdk.org/babylon/pull/416#discussion_r2069458102 From mcimadamore at openjdk.org Wed Apr 30 21:27:58 2025 From: mcimadamore at openjdk.org (Maurizio Cimadamore) Date: Wed, 30 Apr 2025 21:27:58 GMT Subject: [code-reflection] RFR: JavaRef extends TypeElement [v2] In-Reply-To: <_F67lOshTs1HY5ESN41xnN30psFJjIpMKrD9ypAOSNI=.47aba314-a778-4dd0-b0cc-9d83812964e6@github.com> References: <67BtibiQvMvf_mN3zQEpynENB26pqjxkUyVprL1xeVM=.bfab593c-f8fd-41f1-b527-36b384b52177@github.com> <_F67lOshTs1HY5ESN41xnN30psFJjIpMKrD9ypAOSNI=.47aba314-a778-4dd0-b0cc-9d83812964e6@github.com> Message-ID: <5nFyHreXt2N7xLEs8iktY1eDUFiwAq3QVYGRbSAv7LM=.718f173d-447e-4a26-8ad7-a57310c1e37b@github.com> On Wed, 30 Apr 2025 18:45:13 GMT, Paul Sandoz wrote: >> Each implementation of `JavaRef` supports an external form, using the sigil prefix `&` to indicate a reference of some kind. >> >> The externalized form of `TypeVarRef` is changed to be uniformly structured. This required update to the code model text of a few tests. Arguably we are taking a step back in terms of readability, but we can address that later - it was easier to update a few tests rather than preserve the existing encoding in a few places. >> >> We still rely on the bespoke string form for java refs, which is used as the value of an externalized attribute. Ideally such attribute values are either instances of `JavaRef` or `ExternalizedTypeElement` to be transformed into the appropriate `JavaRef`. We can address that later to further separate out parsing. >> >> I believe we have what we need to further enhance the code model builder to not rely on bespoke parsing logic of types and refs. We can either construct `ExternalizedTypeElement` tree instances explicitly or parse from a very simple grammar. If we are careful i believe we can share the results of nested type elements if reused e.g. as in `List` and `Set`. >> >> More generally it now means we can generate a simple s-expression-like tree for the whole code model, e.g., a string where the `(` and `)` characters represent tree structure and say `L` represents a leaf node, and a list of leaf node values in topological order. >> >> -- >> >> Below is the type grammar, which i believe is consistent with what we have implemented. >> >> # Type element grammar >> >> ## General structure >> >> >> identifier >> string >> >> name >> identifier >> >> sigil >> # | . | & | + | - | [ >> >> type >> sigil >> identifier >> sigil identifier >> identifier < types > >> sigil identifier < types > >> >> types >> type >> type , types >> >> >> ## Core types >> >> >> varType >> "var" < type > >> >> tupleType >> "tuple" >> "tuple" < types > >> >> funcType >> "func" < types > >> >> >> # Java types >> >> >> javaType >> primitiveType | classType | wildCardType | arrayType | typeVariableType >> >> javaType-no-wildCardType >> primitiveType | classType | arrayType | typeVariableType >> >> primitiveType >> boolean | byte | ... | void >> >> classType >> name >> name < paramTypes > >> . < enclosingType , innerType > >> paramTypes >> javaType >> javaType , javaTypes >> enclosingType >> classType >> innerType >> classType >> >> wildcardType >> + < wildcardTypeBound > >> - < wildcardTypeBound > >> wildcardTypeB... > > Paul Sandoz has updated the pull request incrementally with one additional commit since the last revision: > > Clean up ConstructorRef I notice that `TypeVarRef.Owner` is very close to being a `JavaRef`. Except that a class type is not a ref (which is what we discussed offline yesterday). ------------- PR Comment: https://git.openjdk.org/babylon/pull/416#issuecomment-2843320043 From mcimadamore at openjdk.org Wed Apr 30 21:33:56 2025 From: mcimadamore at openjdk.org (Maurizio Cimadamore) Date: Wed, 30 Apr 2025 21:33:56 GMT Subject: [code-reflection] RFR: JavaRef extends TypeElement [v2] In-Reply-To: <_F67lOshTs1HY5ESN41xnN30psFJjIpMKrD9ypAOSNI=.47aba314-a778-4dd0-b0cc-9d83812964e6@github.com> References: <67BtibiQvMvf_mN3zQEpynENB26pqjxkUyVprL1xeVM=.bfab593c-f8fd-41f1-b527-36b384b52177@github.com> <_F67lOshTs1HY5ESN41xnN30psFJjIpMKrD9ypAOSNI=.47aba314-a778-4dd0-b0cc-9d83812964e6@github.com> Message-ID: On Wed, 30 Apr 2025 18:45:13 GMT, Paul Sandoz wrote: >> Each implementation of `JavaRef` supports an external form, using the sigil prefix `&` to indicate a reference of some kind. >> >> The externalized form of `TypeVarRef` is changed to be uniformly structured. This required update to the code model text of a few tests. Arguably we are taking a step back in terms of readability, but we can address that later - it was easier to update a few tests rather than preserve the existing encoding in a few places. >> >> We still rely on the bespoke string form for java refs, which is used as the value of an externalized attribute. Ideally such attribute values are either instances of `JavaRef` or `ExternalizedTypeElement` to be transformed into the appropriate `JavaRef`. We can address that later to further separate out parsing. >> >> I believe we have what we need to further enhance the code model builder to not rely on bespoke parsing logic of types and refs. We can either construct `ExternalizedTypeElement` tree instances explicitly or parse from a very simple grammar. If we are careful i believe we can share the results of nested type elements if reused e.g. as in `List` and `Set`. >> >> More generally it now means we can generate a simple s-expression-like tree for the whole code model, e.g., a string where the `(` and `)` characters represent tree structure and say `L` represents a leaf node, and a list of leaf node values in topological order. >> >> -- >> >> Below is the type grammar, which i believe is consistent with what we have implemented. >> >> # Type element grammar >> >> ## General structure >> >> >> identifier >> string >> >> name >> identifier >> >> sigil >> # | . | & | + | - | [ >> >> type >> sigil >> identifier >> sigil identifier >> identifier < types > >> sigil identifier < types > >> >> types >> type >> type , types >> >> >> ## Core types >> >> >> varType >> "var" < type > >> >> tupleType >> "tuple" >> "tuple" < types > >> >> funcType >> "func" < types > >> >> >> # Java types >> >> >> javaType >> primitiveType | classType | wildCardType | arrayType | typeVariableType >> >> javaType-no-wildCardType >> primitiveType | classType | arrayType | typeVariableType >> >> primitiveType >> boolean | byte | ... | void >> >> classType >> name >> name < paramTypes > >> . < enclosingType , innerType > >> paramTypes >> javaType >> javaType , javaTypes >> enclosingType >> classType >> innerType >> classType >> >> wildcardType >> + < wildcardTypeBound > >> - < wildcardTypeBound > >> wildcardTypeB... > > Paul Sandoz has updated the pull request incrementally with one additional commit since the last revision: > > Clean up ConstructorRef Honestly, I wonder if the concept of `JavaRef` still carry its own weight. What it it was just `TypeElement`/`JavaType` ? What if the hierarchy of Java "types" was just flat? My feeling is that if we introduce distinctions, then it's easy to see cracks -- such as the fact that records are refs, but classes are not -- or the fact that a type-variable is both a refs (it points to a type var decl), but also not a ref (it represents a type-variable _use_). ------------- PR Comment: https://git.openjdk.org/babylon/pull/416#issuecomment-2843331390 From psandoz at openjdk.org Wed Apr 30 22:07:48 2025 From: psandoz at openjdk.org (Paul Sandoz) Date: Wed, 30 Apr 2025 22:07:48 GMT Subject: git: openjdk/babylon: code-reflection: Rename TypeVarRef to TypeVariableType Message-ID: <796421ba-9348-40b8-85c9-d00d65e51f6e@openjdk.org> Changeset: 2b0d5a2d Branch: code-reflection Author: Paul Sandoz Date: 2025-04-30 22:07:13 +0000 URL: https://git.openjdk.org/babylon/commit/2b0d5a2d4f6185ea194a28211ae1520e614423de Rename TypeVarRef to TypeVariableType Reviewed-by: mcimadamore ! src/jdk.incubator.code/share/classes/jdk/incubator/code/type/ClassType.java ! src/jdk.incubator.code/share/classes/jdk/incubator/code/type/ConstructorRef.java ! src/jdk.incubator.code/share/classes/jdk/incubator/code/type/CoreTypeFactory.java ! src/jdk.incubator.code/share/classes/jdk/incubator/code/type/JavaType.java ! src/jdk.incubator.code/share/classes/jdk/incubator/code/type/MethodRef.java = src/jdk.incubator.code/share/classes/jdk/incubator/code/type/TypeVariableType.java ! test/jdk/java/lang/reflect/code/type/TestErasure.java From psandoz at openjdk.org Wed Apr 30 22:10:00 2025 From: psandoz at openjdk.org (Paul Sandoz) Date: Wed, 30 Apr 2025 22:10:00 GMT Subject: [code-reflection] Integrated: Rename TypeVarRef to TypeVariableType In-Reply-To: References: Message-ID: On Wed, 30 Apr 2025 19:41:32 GMT, Paul Sandoz wrote: > Rename `TypeVarRef` to `TypeVariableType` This pull request has now been integrated. Changeset: 2b0d5a2d Author: Paul Sandoz URL: https://git.openjdk.org/babylon/commit/2b0d5a2d4f6185ea194a28211ae1520e614423de Stats: 19 lines in 7 files changed: 0 ins; 4 del; 15 mod Rename TypeVarRef to TypeVariableType Reviewed-by: mcimadamore ------------- PR: https://git.openjdk.org/babylon/pull/417 From psandoz at openjdk.org Wed Apr 30 23:07:03 2025 From: psandoz at openjdk.org (Paul Sandoz) Date: Wed, 30 Apr 2025 23:07:03 GMT Subject: [code-reflection] RFR: JavaRef extends TypeElement [v2] In-Reply-To: References: <67BtibiQvMvf_mN3zQEpynENB26pqjxkUyVprL1xeVM=.bfab593c-f8fd-41f1-b527-36b384b52177@github.com> <_F67lOshTs1HY5ESN41xnN30psFJjIpMKrD9ypAOSNI=.47aba314-a778-4dd0-b0cc-9d83812964e6@github.com> Message-ID: On Wed, 30 Apr 2025 21:31:20 GMT, Maurizio Cimadamore wrote: > Honestly, I wonder if the concept of `JavaRef` still carry its own weight. What it it was just `TypeElement`/`JavaType` ? What if the hierarchy of Java "types" was just flat? My feeling is that if we introduce distinctions, then it's easy to see cracks -- such as the fact that records are refs, but classes are not -- or the fact that a type-variable is both a refs (it points to a type var decl), but also not a ref (it represents a type-variable _use_). It can go away if there are no suitable methods that are applicable to all refs. I was hoping there might be something around resolution or erasure. We could also relocate all the factory methods. I wish we could find a better name than `Ref`. Record ref is needed for the modeling of record patterns, describing a record's components (types and names) so we can lower without appealing to reflection. I suspect we may need something similar for constructor patterns and member patterns. So when we model those maybe we might find a more general way of expressing the sequence of named parameters and the owner of those? I suppose we could introduce `RecordType` as a sub class of `ClassType`, but there is no need to directly express the records components in any hierarchical relationship of that because they are not pertinent to the type itself e.g., the externalized form would not include them. `TypeVariableType` is a curious beast, since it refers to the declaring owner that gives the type variable name its identity. It is clearly interchangeable with `java.lang.reflect.type.TypeVariable`. ------------- PR Comment: https://git.openjdk.org/babylon/pull/416#issuecomment-2843634356