From mabbay at openjdk.org Thu May 1 01:16:56 2025 From: mabbay at openjdk.org (Mourad Abbay) Date: Thu, 1 May 2025 01:16:56 GMT Subject: [code-reflection] RFR: Method code model uniquness [v4] In-Reply-To: References: Message-ID: On Wed, 30 Apr 2025 08:16:36 GMT, Maurizio Cimadamore wrote: >> Mourad Abbay has updated the pull request incrementally with one additional commit since the last revision: >> >> Avoid caching Method.codeModel on non-root instances > > src/java.base/share/classes/java/lang/reflect/Method.java line 233: > >> 231: /* package */ >> 232: Optional setCodeModelIfNeeded(Function> modelFactory) { >> 233: if (root != null) { > > This logic is subtly different to the one I pointed out -- this always recurse if the root is non null. The one I pointed out calls the method on the root, but than saves the result on the copy locally. I'm not sure which is better -- but it seems odd to use slightly different idioms in the same class to do effectively the same thing? I applied the pattern you pointed out in the third commit. In this version I intentionally wanted to have the model stored in the root method, this way we only have one reference to the model. ------------- PR Review Comment: https://git.openjdk.org/babylon/pull/415#discussion_r2069736239 From mabbay at openjdk.org Thu May 1 01:20:02 2025 From: mabbay at openjdk.org (Mourad Abbay) Date: Thu, 1 May 2025 01:20:02 GMT Subject: [code-reflection] RFR: Further cleanup of `CodeModelToAST` and regularize support for varargs instance creation expressions [v2] In-Reply-To: References: Message-ID: On Fri, 11 Apr 2025 17:36:31 GMT, Maurizio Cimadamore wrote: >> This PR makes the code in `CodeModelToAST` that deal with instance creation expression (`NewOp`) a bit closer to the code that deals with method calls (`InvokeOp`). I've added a method, called `constructorTypeToSymbol` which takes the constructor type of a `NewOp` and looks up the constructor symbol with that type. >> >> Then I moved the logic that sets `varargElements` on the created `JCMethodInvocation` nodes into a separate, reusable routine. This is now called also for varargs instance creation expression. >> >> Of course, to do that I had to add support for a new varargs attribute in the `NewOp` class -- which closely follows the one used by `InvokeOp`. This means that in `ReflectMethod` we can now reify in the model as to whether a constructor call was varargs or not. >> >> I've also added some arg count validation in `NewOp`, very similar to the one that appears in `InvokeOp`. > > Maurizio Cimadamore has updated the pull request incrementally with one additional commit since the last revision: > > Address review comments Can we resolve conflicts and merge ? ------------- PR Comment: https://git.openjdk.org/babylon/pull/393#issuecomment-2843863729 From mcimadamore at openjdk.org Thu May 1 08:47:10 2025 From: mcimadamore at openjdk.org (Maurizio Cimadamore) Date: Thu, 1 May 2025 08:47:10 GMT Subject: [code-reflection] RFR: Further cleanup of `CodeModelToAST` and regularize support for varargs instance creation expressions [v2] In-Reply-To: References: Message-ID: On Thu, 1 May 2025 01:17:30 GMT, Mourad Abbay wrote: >> Maurizio Cimadamore has updated the pull request incrementally with one additional commit since the last revision: >> >> Address review comments > > Can we resolve conflicts and merge ? @mabbay I think this has already been merged - probably as part of https://git.openjdk.org/babylon/pull/395 ------------- PR Comment: https://git.openjdk.org/babylon/pull/393#issuecomment-2844393733 From mcimadamore at openjdk.org Thu May 1 08:47:10 2025 From: mcimadamore at openjdk.org (Maurizio Cimadamore) Date: Thu, 1 May 2025 08:47:10 GMT Subject: [code-reflection] Withdrawn: Further cleanup of `CodeModelToAST` and regularize support for varargs instance creation expressions In-Reply-To: References: Message-ID: On Fri, 11 Apr 2025 12:03:04 GMT, Maurizio Cimadamore wrote: > This PR makes the code in `CodeModelToAST` that deal with instance creation expression (`NewOp`) a bit closer to the code that deals with method calls (`InvokeOp`). I've added a method, called `constructorTypeToSymbol` which takes the constructor type of a `NewOp` and looks up the constructor symbol with that type. > > Then I moved the logic that sets `varargElements` on the created `JCMethodInvocation` nodes into a separate, reusable routine. This is now called also for varargs instance creation expression. > > Of course, to do that I had to add support for a new varargs attribute in the `NewOp` class -- which closely follows the one used by `InvokeOp`. This means that in `ReflectMethod` we can now reify in the model as to whether a constructor call was varargs or not. > > I've also added some arg count validation in `NewOp`, very similar to the one that appears in `InvokeOp`. This pull request has been closed without being integrated. ------------- PR: https://git.openjdk.org/babylon/pull/393 From mcimadamore at openjdk.org Thu May 1 08:36:07 2025 From: mcimadamore at openjdk.org (Maurizio Cimadamore) Date: Thu, 1 May 2025 08:36:07 GMT Subject: [code-reflection] RFR: JavaRef extends TypeElement [v2] In-Reply-To: References: <67BtibiQvMvf_mN3zQEpynENB26pqjxkUyVprL1xeVM=.bfab593c-f8fd-41f1-b527-36b384b52177@github.com> <_F67lOshTs1HY5ESN41xnN30psFJjIpMKrD9ypAOSNI=.47aba314-a778-4dd0-b0cc-9d83812964e6@github.com> Message-ID: On Wed, 30 Apr 2025 23:02:57 GMT, Paul Sandoz wrote: > It can go away if there are no suitable methods that are applicable to all refs. I was hoping there might be something around resolution or erasure. We could also relocate all the factory methods. I wish we could find a better name than `Ref`. I agree --- I was wondering if we could have a resolve method in all Java "types" (we should find a better word for it) -- some resolve to a `j.l.r.Type`, other will resolve to `Method`/`Constructor`/`Field`/`Class`/`RecordComponent`. But they _all_ can be resolved, in a way. Not sure if this is a stretch too far -- but I wonder if the separation between "symbolic reference" and "type" -- which is useful when writing a compiler -- is something worth calling attention for here, or if we're better off just pretending they are all Java "symbols" of some kind (some representing types, some representing something else). ------------- PR Comment: https://git.openjdk.org/babylon/pull/416#issuecomment-2844380563 From mcimadamore at openjdk.org Thu May 1 08:36:58 2025 From: mcimadamore at openjdk.org (Maurizio Cimadamore) Date: Thu, 1 May 2025 08:36:58 GMT Subject: [code-reflection] RFR: Method code model uniquness [v4] In-Reply-To: References: Message-ID: On Wed, 30 Apr 2025 01:19:55 GMT, Mourad Abbay wrote: >> Multiple `Method` objects that represent the same java method, have different code models. This is because every time we call `Class.getMethod` or `Class.getDeclaredMethod` we get a newly created instance. One possible solution is to attach the model to the root `Method` object. > > Mourad Abbay has updated the pull request incrementally with one additional commit since the last revision: > > Avoid caching Method.codeModel on non-root instances Marked as reviewed by mcimadamore (Reviewer). ------------- PR Review: https://git.openjdk.org/babylon/pull/415#pullrequestreview-2809361456 From psandoz at openjdk.org Thu May 1 15:40:04 2025 From: psandoz at openjdk.org (Paul Sandoz) Date: Thu, 1 May 2025 15:40:04 GMT Subject: [code-reflection] RFR: JavaRef extends TypeElement [v2] In-Reply-To: <4CNchtey30bFot_54QaAw8WJrhQOgkNZGr7uZFukWNI=.dad6402a-82ac-46c2-9fcb-6a90ddf1300b@github.com> References: <67BtibiQvMvf_mN3zQEpynENB26pqjxkUyVprL1xeVM=.bfab593c-f8fd-41f1-b527-36b384b52177@github.com> <4CNchtey30bFot_54QaAw8WJrhQOgkNZGr7uZFukWNI=.dad6402a-82ac-46c2-9fcb-6a90ddf1300b@github.com> Message-ID: On Wed, 30 Apr 2025 21:15:49 GMT, Maurizio Cimadamore wrote: >> Yes! > > Question - if Java code calls a method on a record... is the "ref" attached to the invoke operation a record type ref? I don't think that is the case, as I don't remember any special logic in javac to emit a record type ref when generating field/method references? No, it's an invocation operation to the method whose name is the same as the component name. When we lower a record pattern we emit the invocation using the detail in the record ref, see [here](https://github.com/openjdk/babylon/blob/5238a40d11397b9896756a674158378223405388/src/jdk.incubator.code/share/classes/jdk/incubator/code/op/ExtendedOp.java#L3141): Value nestedTarget = currentBlock.op(CoreOp.invoke(rpOp.recordDescriptor().methodForComponent(i), target)); where @Override public MethodRef methodForComponent(int i) { if (i < 0 || i >= components.size()) { throw new IndexOutOfBoundsException(); } ComponentRef c = components.get(i); return MethodRef.method(recordType, c.name(), c.type()); } ------------- PR Review Comment: https://git.openjdk.org/babylon/pull/416#discussion_r2070420230 From psandoz at openjdk.org Fri May 2 17:21:10 2025 From: psandoz at openjdk.org (Paul Sandoz) Date: Fri, 2 May 2025 17:21:10 GMT Subject: [code-reflection] RFR: Method code model uniquness [v4] In-Reply-To: References: Message-ID: On Wed, 30 Apr 2025 01:19:55 GMT, Mourad Abbay wrote: >> Multiple `Method` objects that represent the same java method, have different code models. This is because every time we call `Class.getMethod` or `Class.getDeclaredMethod` we get a newly created instance. One possible solution is to attach the model to the root `Method` object. > > Mourad Abbay has updated the pull request incrementally with one additional commit since the last revision: > > Avoid caching Method.codeModel on non-root instances Marked as reviewed by psandoz (Lead). ------------- PR Review: https://git.openjdk.org/babylon/pull/415#pullrequestreview-2812530331 From psandoz at openjdk.org Fri May 2 17:33:18 2025 From: psandoz at openjdk.org (Paul Sandoz) Date: Fri, 2 May 2025 17:33:18 GMT Subject: [code-reflection] RFR: Build ref attributes from externalized type elems Message-ID: This removes the dependence on specific parsing of refs. Now they are type elements we can use its string form. ------------- Commit messages: - Build ref attributes from externalized type elems Changes: https://git.openjdk.org/babylon/pull/418/files Webrev: https://webrevs.openjdk.org/?repo=babylon&pr=418&range=00 Stats: 28 lines in 1 file changed: 0 ins; 28 del; 0 mod Patch: https://git.openjdk.org/babylon/pull/418.diff Fetch: git fetch https://git.openjdk.org/babylon.git pull/418/head:pull/418 PR: https://git.openjdk.org/babylon/pull/418 From psandoz at openjdk.org Fri May 2 17:40:30 2025 From: psandoz at openjdk.org (Paul Sandoz) Date: Fri, 2 May 2025 17:40:30 GMT Subject: git: openjdk/babylon: code-reflection: Build ref attributes from externalized type elems Message-ID: Changeset: f1a6fa99 Branch: code-reflection Author: Paul Sandoz Date: 2025-05-02 17:39:16 +0000 URL: https://git.openjdk.org/babylon/commit/f1a6fa99fbd41c501e57e14746a47e00215cefb8 Build ref attributes from externalized type elems Reviewed-by: mcimadamore ! src/jdk.incubator.code/share/classes/jdk/incubator/code/writer/OpBuilder.java From mcimadamore at openjdk.org Fri May 2 17:42:11 2025 From: mcimadamore at openjdk.org (Maurizio Cimadamore) Date: Fri, 2 May 2025 17:42:11 GMT Subject: [code-reflection] RFR: Build ref attributes from externalized type elems In-Reply-To: References: Message-ID: On Fri, 2 May 2025 17:27:46 GMT, Paul Sandoz wrote: > This removes the dependence on specific parsing of refs. Now they are type elements we can use its string form. Looks good! ------------- Marked as reviewed by mcimadamore (Reviewer). PR Review: https://git.openjdk.org/babylon/pull/418#pullrequestreview-2812566257 From psandoz at openjdk.org Fri May 2 17:42:11 2025 From: psandoz at openjdk.org (Paul Sandoz) Date: Fri, 2 May 2025 17:42:11 GMT Subject: [code-reflection] Integrated: Build ref attributes from externalized type elems In-Reply-To: References: Message-ID: On Fri, 2 May 2025 17:27:46 GMT, Paul Sandoz wrote: > This removes the dependence on specific parsing of refs. Now they are type elements we can use its string form. This pull request has now been integrated. Changeset: f1a6fa99 Author: Paul Sandoz URL: https://git.openjdk.org/babylon/commit/f1a6fa99fbd41c501e57e14746a47e00215cefb8 Stats: 28 lines in 1 file changed: 0 ins; 28 del; 0 mod Build ref attributes from externalized type elems Reviewed-by: mcimadamore ------------- PR: https://git.openjdk.org/babylon/pull/418 From psandoz at openjdk.org Fri May 2 21:02:28 2025 From: psandoz at openjdk.org (Paul Sandoz) Date: Fri, 2 May 2025 21:02:28 GMT Subject: [code-reflection] RFR: Build extern type element tree Message-ID: The model that builds the model now explicitly builds the externalized type element tree rather than parsing it from the string form. Externalized type element instances are reused. ------------- Commit messages: - Remove unused field. - Build extern type element tree Changes: https://git.openjdk.org/babylon/pull/419/files Webrev: https://webrevs.openjdk.org/?repo=babylon&pr=419&range=00 Stats: 86 lines in 3 files changed: 77 ins; 4 del; 5 mod Patch: https://git.openjdk.org/babylon/pull/419.diff Fetch: git fetch https://git.openjdk.org/babylon.git pull/419/head:pull/419 PR: https://git.openjdk.org/babylon/pull/419 From psandoz at openjdk.org Fri May 2 21:05:04 2025 From: psandoz at openjdk.org (Paul Sandoz) Date: Fri, 2 May 2025 21:05:04 GMT Subject: [code-reflection] RFR: Build extern type element tree In-Reply-To: References: Message-ID: On Fri, 2 May 2025 20:58:09 GMT, Paul Sandoz wrote: > The model that builds the model now explicitly builds the externalized type element tree rather than parsing it from the string form. Externalized type element instances are reused. This PR also fixed an issue where the array operation operands were incorrect (the index and element values were the swapped - it worked because the incorrect order was consistently used). ------------- PR Comment: https://git.openjdk.org/babylon/pull/419#issuecomment-2848114765 From psandoz at openjdk.org Fri May 2 21:12:45 2025 From: psandoz at openjdk.org (Paul Sandoz) Date: Fri, 2 May 2025 21:12:45 GMT Subject: [code-reflection] RFR: Build extern type element tree [v2] In-Reply-To: References: Message-ID: > The model that builds the model now explicitly builds the externalized type element tree rather than parsing it from the string form. Externalized type element instances are reused. Paul Sandoz has updated the pull request incrementally with one additional commit since the last revision: Remove unused import ------------- Changes: - all: https://git.openjdk.org/babylon/pull/419/files - new: https://git.openjdk.org/babylon/pull/419/files/7648d19e..f9f9cb20 Webrevs: - full: https://webrevs.openjdk.org/?repo=babylon&pr=419&range=01 - incr: https://webrevs.openjdk.org/?repo=babylon&pr=419&range=00-01 Stats: 1 line in 1 file changed: 0 ins; 1 del; 0 mod Patch: https://git.openjdk.org/babylon/pull/419.diff Fetch: git fetch https://git.openjdk.org/babylon.git pull/419/head:pull/419 PR: https://git.openjdk.org/babylon/pull/419 From mcimadamore at openjdk.org Fri May 2 21:16:55 2025 From: mcimadamore at openjdk.org (Maurizio Cimadamore) Date: Fri, 2 May 2025 21:16:55 GMT Subject: [code-reflection] RFR: Build extern type element tree [v2] In-Reply-To: References: Message-ID: On Fri, 2 May 2025 21:12:45 GMT, Paul Sandoz wrote: >> The model that builds the model now explicitly builds the externalized type element tree rather than parsing it from the string form. Externalized type element instances are reused. > > Paul Sandoz has updated the pull request incrementally with one additional commit since the last revision: > > Remove unused import nice! ------------- Marked as reviewed by mcimadamore (Reviewer). PR Review: https://git.openjdk.org/babylon/pull/419#pullrequestreview-2812917880 From psandoz at openjdk.org Fri May 2 21:16:56 2025 From: psandoz at openjdk.org (Paul Sandoz) Date: Fri, 2 May 2025 21:16:56 GMT Subject: [code-reflection] RFR: Build extern type element tree [v2] In-Reply-To: References: Message-ID: <8t_zV2jWehOyi45UGgXE9UESdZ7k03mspF2klgcAfMU=.6eb388a7-9011-4b16-900b-eb1e116cc128@github.com> On Fri, 2 May 2025 21:12:45 GMT, Paul Sandoz wrote: >> The model that builds the model now explicitly builds the externalized type element tree rather than parsing it from the string form. Externalized type element instances are reused. > > Paul Sandoz has updated the pull request incrementally with one additional commit since the last revision: > > Remove unused import src/jdk.incubator.code/share/classes/jdk/incubator/code/TypeElement.java line 113: > 111: ExternalizedTypeElement a3, ExternalizedTypeElement a4) { > 112: return new ExternalizedTypeElement(s, List.of(a1, a2, a3, a4)); > 113: } It's possible we don't need these methods. They were included so that it was more efficient to encode ex type elem construction in the class file. Since instances used more than once are shared the small incremental cost of using the list accepting method is likely a good tradeoff. (General idea - we should annotate methods that are referenced by the compiler and runtime and used for generating and producing code models, such that we know it will cause issues if we remove or refactor them.) ------------- PR Review Comment: https://git.openjdk.org/babylon/pull/419#discussion_r2072155374 From psandoz at openjdk.org Fri May 2 22:39:00 2025 From: psandoz at openjdk.org (Paul Sandoz) Date: Fri, 2 May 2025 22:39:00 GMT Subject: git: openjdk/babylon: code-reflection: Build extern type element tree Message-ID: Changeset: 28ff84b6 Branch: code-reflection Author: Paul Sandoz Date: 2025-05-02 22:37:00 +0000 URL: https://git.openjdk.org/babylon/commit/28ff84b69187d3554f7a09c3b3e53eb2603bf90f Build extern type element tree Reviewed-by: mcimadamore ! src/jdk.incubator.code/share/classes/jdk/incubator/code/TypeElement.java ! src/jdk.incubator.code/share/classes/jdk/incubator/code/internal/CodeModelToAST.java ! src/jdk.incubator.code/share/classes/jdk/incubator/code/writer/OpBuilder.java From psandoz at openjdk.org Fri May 2 22:39:58 2025 From: psandoz at openjdk.org (Paul Sandoz) Date: Fri, 2 May 2025 22:39:58 GMT Subject: [code-reflection] Integrated: Build extern type element tree In-Reply-To: References: Message-ID: <5dK7gntv3UX2fYS3PRDd6n3ya4OHJ_hoS-QAseGSs8k=.84a7062e-8b17-4958-8c41-caaa2f54ab1d@github.com> On Fri, 2 May 2025 20:58:09 GMT, Paul Sandoz wrote: > The model that builds the model now explicitly builds the externalized type element tree rather than parsing it from the string form. Externalized type element instances are reused. This pull request has now been integrated. Changeset: 28ff84b6 Author: Paul Sandoz URL: https://git.openjdk.org/babylon/commit/28ff84b69187d3554f7a09c3b3e53eb2603bf90f Stats: 85 lines in 3 files changed: 76 ins; 4 del; 5 mod Build extern type element tree Reviewed-by: mcimadamore ------------- PR: https://git.openjdk.org/babylon/pull/419 From mabbay at openjdk.org Fri May 2 23:53:00 2025 From: mabbay at openjdk.org (Mourad Abbay) Date: Fri, 2 May 2025 23:53:00 GMT Subject: [code-reflection] Integrated: Method code model uniquness In-Reply-To: References: Message-ID: On Tue, 29 Apr 2025 10:52:58 GMT, Mourad Abbay wrote: > Multiple `Method` objects that represent the same java method, have different code models. This is because every time we call `Class.getMethod` or `Class.getDeclaredMethod` we get a newly created instance. One possible solution is to attach the model to the root `Method` object. This pull request has now been integrated. Changeset: b6232637 Author: Mourad Abbay URL: https://git.openjdk.org/babylon/commit/b6232637eca8a3a5fca317cf0f9c8402e8745146 Stats: 48 lines in 2 files changed: 46 ins; 0 del; 2 mod Method code model uniquness Reviewed-by: psandoz, mcimadamore ------------- PR: https://git.openjdk.org/babylon/pull/415 From mabbay at openjdk.org Fri May 2 23:52:50 2025 From: mabbay at openjdk.org (Mourad Abbay) Date: Fri, 2 May 2025 23:52:50 GMT Subject: git: openjdk/babylon: code-reflection: Method code model uniquness Message-ID: Changeset: b6232637 Branch: code-reflection Author: Mourad Abbay Date: 2025-05-02 23:50:27 +0000 URL: https://git.openjdk.org/babylon/commit/b6232637eca8a3a5fca317cf0f9c8402e8745146 Method code model uniquness Reviewed-by: psandoz, mcimadamore ! src/java.base/share/classes/java/lang/reflect/Method.java + test/langtools/tools/javac/reflect/MethodModelTest.java From mabbay at openjdk.org Sat May 3 08:49:14 2025 From: mabbay at openjdk.org (Mourad Abbay) Date: Sat, 3 May 2025 08:49:14 GMT Subject: [code-reflection] RFR: Lambda model uniqueness Message-ID: <5pt5GFZHVDGTp2_9GLFo9kjrE4XatA23wwjDg9OOf9M=.700bfa02-c542-4187-aeec-00a2969f960a@github.com> Lambdas that share the same proxy class, have the same implementation and therefore have the same code model generated by the compiler. In this PR, we share the code model across instances of the same proxy class. This also means we reduce execution time as we only need to invoke `opMethod` once for given proxy class. With these changes we can compare lambda code model based on identity. ------------- Commit messages: - Rename a test - Share the compiler generated code model of a lambda across instances of its proxy class Changes: https://git.openjdk.org/babylon/pull/420/files Webrev: https://webrevs.openjdk.org/?repo=babylon&pr=420&range=00 Stats: 112 lines in 3 files changed: 93 ins; 12 del; 7 mod Patch: https://git.openjdk.org/babylon/pull/420.diff Fetch: git fetch https://git.openjdk.org/babylon.git pull/420/head:pull/420 PR: https://git.openjdk.org/babylon/pull/420 From mabbay at openjdk.org Sat May 3 08:59:16 2025 From: mabbay at openjdk.org (Mourad Abbay) Date: Sat, 3 May 2025 08:59:16 GMT Subject: [code-reflection] RFR: Lambda model uniqueness [v2] In-Reply-To: <5pt5GFZHVDGTp2_9GLFo9kjrE4XatA23wwjDg9OOf9M=.700bfa02-c542-4187-aeec-00a2969f960a@github.com> References: <5pt5GFZHVDGTp2_9GLFo9kjrE4XatA23wwjDg9OOf9M=.700bfa02-c542-4187-aeec-00a2969f960a@github.com> Message-ID: > Lambdas that share the same proxy class, have the same implementation and therefore have the same code model generated by the compiler. In this PR, we share the code model across instances of the same proxy class. > This also means we reduce execution time as we only need to invoke `opMethod` once for given proxy class. > With these changes we can compare lambda code model based on identity. Mourad Abbay has updated the pull request incrementally with one additional commit since the last revision: Define static fields and update comments ------------- Changes: - all: https://git.openjdk.org/babylon/pull/420/files - new: https://git.openjdk.org/babylon/pull/420/files/4f9fd9a8..c690b993 Webrevs: - full: https://webrevs.openjdk.org/?repo=babylon&pr=420&range=01 - incr: https://webrevs.openjdk.org/?repo=babylon&pr=420&range=00-01 Stats: 8 lines in 1 file changed: 2 ins; 2 del; 4 mod Patch: https://git.openjdk.org/babylon/pull/420.diff Fetch: git fetch https://git.openjdk.org/babylon.git pull/420/head:pull/420 PR: https://git.openjdk.org/babylon/pull/420 From mabbay at openjdk.org Sun May 4 11:36:56 2025 From: mabbay at openjdk.org (Mourad Abbay) Date: Sun, 4 May 2025 11:36:56 GMT Subject: [code-reflection] RFR: Lambda model uniqueness [v3] In-Reply-To: <5pt5GFZHVDGTp2_9GLFo9kjrE4XatA23wwjDg9OOf9M=.700bfa02-c542-4187-aeec-00a2969f960a@github.com> References: <5pt5GFZHVDGTp2_9GLFo9kjrE4XatA23wwjDg9OOf9M=.700bfa02-c542-4187-aeec-00a2969f960a@github.com> Message-ID: > Lambdas that share the same proxy class, have the same implementation and therefore have the same code model generated by the compiler. In this PR, we share the code model across instances of the same proxy class. > This also means we reduce execution time as we only need to invoke `opMethod` once for given proxy class. > With these changes we can compare lambda code model based on identity. Mourad Abbay has updated the pull request incrementally with one additional commit since the last revision: Remove static field as it causes compilation error ------------- Changes: - all: https://git.openjdk.org/babylon/pull/420/files - new: https://git.openjdk.org/babylon/pull/420/files/c690b993..2b524aa6 Webrevs: - full: https://webrevs.openjdk.org/?repo=babylon&pr=420&range=02 - incr: https://webrevs.openjdk.org/?repo=babylon&pr=420&range=01-02 Stats: 7 lines in 1 file changed: 2 ins; 1 del; 4 mod Patch: https://git.openjdk.org/babylon/pull/420.diff Fetch: git fetch https://git.openjdk.org/babylon.git pull/420/head:pull/420 PR: https://git.openjdk.org/babylon/pull/420 From mabbay at openjdk.org Sun May 4 13:49:46 2025 From: mabbay at openjdk.org (Mourad Abbay) Date: Sun, 4 May 2025 13:49:46 GMT Subject: [code-reflection] RFR: Lambda model uniqueness [v4] In-Reply-To: <5pt5GFZHVDGTp2_9GLFo9kjrE4XatA23wwjDg9OOf9M=.700bfa02-c542-4187-aeec-00a2969f960a@github.com> References: <5pt5GFZHVDGTp2_9GLFo9kjrE4XatA23wwjDg9OOf9M=.700bfa02-c542-4187-aeec-00a2969f960a@github.com> Message-ID: > Lambdas that share the same proxy class, have the same implementation and therefore have the same code model generated by the compiler. In this PR, we share the code model across instances of the same proxy class. > This also means we reduce execution time as we only need to invoke `opMethod` once for given proxy class. > With these changes we can compare lambda code model based on identity. Mourad Abbay has updated the pull request incrementally with one additional commit since the last revision: Create Quoted object lazily ------------- Changes: - all: https://git.openjdk.org/babylon/pull/420/files - new: https://git.openjdk.org/babylon/pull/420/files/2b524aa6..fc94c111 Webrevs: - full: https://webrevs.openjdk.org/?repo=babylon&pr=420&range=03 - incr: https://webrevs.openjdk.org/?repo=babylon&pr=420&range=02-03 Stats: 86 lines in 1 file changed: 80 ins; 3 del; 3 mod Patch: https://git.openjdk.org/babylon/pull/420.diff Fetch: git fetch https://git.openjdk.org/babylon.git pull/420/head:pull/420 PR: https://git.openjdk.org/babylon/pull/420 From asotona at openjdk.org Mon May 5 16:56:28 2025 From: asotona at openjdk.org (Adam Sotona) Date: Mon, 5 May 2025 16:56:28 GMT Subject: [code-reflection] RFR: Onnx model and protobuf model improvements Message-ID: Onnx model and protobuf model improvements. Onnx protobuf constants, builder and model generated. ------------- Commit messages: - ProtoGen work in progress - ProtoGen work in progress - ProtoGen work in progress - ProtoGen work in progress - ProtoGen work in progress - ProtoGen work in progress - implementation of ProtoGen - OnnxProtoBuilder separation of the generated code - Towards a fully generated OnnxProtoBuilder - AttributeType.TENSOR holds Tensor instead of byte array, AttributeType.TENSOR holds array of Tensors Changes: https://git.openjdk.org/babylon/pull/414/files Webrev: https://webrevs.openjdk.org/?repo=babylon&pr=414&range=00 Stats: 4342 lines in 13 files changed: 3192 ins; 1042 del; 108 mod Patch: https://git.openjdk.org/babylon/pull/414.diff Fetch: git fetch https://git.openjdk.org/babylon.git pull/414/head:pull/414 PR: https://git.openjdk.org/babylon/pull/414 From asotona at openjdk.org Tue May 6 06:30:53 2025 From: asotona at openjdk.org (Adam Sotona) Date: Tue, 6 May 2025 06:30:53 GMT Subject: [code-reflection] RFR: Onnx model and protobuf model improvements [v2] In-Reply-To: References: Message-ID: <3s-tm094Vgg6jcpXtbiS9rszFZY-VhMzgy-6iVrca_M=.99155d12-4641-40b5-9919-ce2f76b2e8cc@github.com> > Onnx model and protobuf model improvements. > Onnx protobuf constants, builder and model generated. Adam Sotona has updated the pull request incrementally with one additional commit since the last revision: NIT ProtoGen fix ------------- Changes: - all: https://git.openjdk.org/babylon/pull/414/files - new: https://git.openjdk.org/babylon/pull/414/files/3be8f709..6d2adfa1 Webrevs: - full: https://webrevs.openjdk.org/?repo=babylon&pr=414&range=01 - incr: https://webrevs.openjdk.org/?repo=babylon&pr=414&range=00-01 Stats: 3 lines in 2 files changed: 1 ins; 2 del; 0 mod Patch: https://git.openjdk.org/babylon/pull/414.diff Fetch: git fetch https://git.openjdk.org/babylon.git pull/414/head:pull/414 PR: https://git.openjdk.org/babylon/pull/414 From asotona at openjdk.org Tue May 6 06:41:16 2025 From: asotona at openjdk.org (Adam Sotona) Date: Tue, 6 May 2025 06:41:16 GMT Subject: [code-reflection] RFR: Onnx model and protobuf model improvements [v3] In-Reply-To: References: Message-ID: > Onnx model and protobuf model improvements. > Onnx protobuf constants, builder and model generated. Adam Sotona has updated the pull request incrementally with one additional commit since the last revision: NIT ProtoGen fix ------------- Changes: - all: https://git.openjdk.org/babylon/pull/414/files - new: https://git.openjdk.org/babylon/pull/414/files/6d2adfa1..3fc910ee Webrevs: - full: https://webrevs.openjdk.org/?repo=babylon&pr=414&range=02 - incr: https://webrevs.openjdk.org/?repo=babylon&pr=414&range=01-02 Stats: 2 lines in 2 files changed: 0 ins; 0 del; 2 mod Patch: https://git.openjdk.org/babylon/pull/414.diff Fetch: git fetch https://git.openjdk.org/babylon.git pull/414/head:pull/414 PR: https://git.openjdk.org/babylon/pull/414 From asotona at openjdk.org Tue May 6 07:38:55 2025 From: asotona at openjdk.org (Adam Sotona) Date: Tue, 6 May 2025 07:38:55 GMT Subject: git: openjdk/babylon: code-reflection: Onnx model and protobuf model improvements Message-ID: <3697becd-78d6-40af-8816-169b2633299a@openjdk.org> Changeset: 6c7ea689 Branch: code-reflection Author: Adam Sotona Date: 2025-05-06 07:36:32 +0000 URL: https://git.openjdk.org/babylon/commit/6c7ea6892cf8349de40ced435e3d7cf55850d0b0 Onnx model and protobuf model improvements ! cr-examples/onnx/opgen/src/main/java/oracle/code/onnx/OpSchema.java ! cr-examples/onnx/opgen/src/main/java/oracle/code/onnx/opgen/OpGen.java + cr-examples/onnx/opgen/src/main/java/oracle/code/onnx/protogen/ProtoGen.java ! cr-examples/onnx/src/main/java/oracle/code/onnx/OnnxProtoBuilder.java ! cr-examples/onnx/src/main/java/oracle/code/onnx/OnnxRuntime.java ! cr-examples/onnx/src/main/java/oracle/code/onnx/ir/OnnxOps.java + cr-examples/onnx/src/main/java/oracle/code/onnx/proto/OnnxBuilder.java + cr-examples/onnx/src/main/java/oracle/code/onnx/proto/OnnxConstants.java + cr-examples/onnx/src/main/java/oracle/code/onnx/proto/OnnxModel.java - cr-examples/onnx/src/main/java/oracle/code/onnx/proto/OnnxProtoModel.java ! cr-examples/onnx/src/test/java/oracle/code/onnx/RuntimeTest.java + cr-examples/onnx/src/test/java/oracle/code/onnx/proto/OnnxModelTest.java - cr-examples/onnx/src/test/java/oracle/code/onnx/proto/OnnxProtoModelTest.java From asotona at openjdk.org Tue May 6 07:41:59 2025 From: asotona at openjdk.org (Adam Sotona) Date: Tue, 6 May 2025 07:41:59 GMT Subject: [code-reflection] RFR: Onnx model and protobuf model improvements [v4] In-Reply-To: References: Message-ID: > Onnx model and protobuf model improvements. > Onnx protobuf constants, builder and model generated. Adam Sotona has updated the pull request incrementally with one additional commit since the last revision: Update ProtoGen.java ------------- Changes: - all: https://git.openjdk.org/babylon/pull/414/files - new: https://git.openjdk.org/babylon/pull/414/files/3fc910ee..ad654250 Webrevs: - full: https://webrevs.openjdk.org/?repo=babylon&pr=414&range=03 - incr: https://webrevs.openjdk.org/?repo=babylon&pr=414&range=02-03 Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod Patch: https://git.openjdk.org/babylon/pull/414.diff Fetch: git fetch https://git.openjdk.org/babylon.git pull/414/head:pull/414 PR: https://git.openjdk.org/babylon/pull/414 From asotona at openjdk.org Tue May 6 07:42:00 2025 From: asotona at openjdk.org (Adam Sotona) Date: Tue, 6 May 2025 07:42:00 GMT Subject: [code-reflection] Integrated: Onnx model and protobuf model improvements In-Reply-To: References: Message-ID: On Tue, 29 Apr 2025 08:47:27 GMT, Adam Sotona wrote: > Onnx model and protobuf model improvements. > Onnx protobuf constants, builder and model generated. This pull request has now been integrated. Changeset: 6c7ea689 Author: Adam Sotona URL: https://git.openjdk.org/babylon/commit/6c7ea6892cf8349de40ced435e3d7cf55850d0b0 Stats: 4341 lines in 13 files changed: 3191 ins; 1042 del; 108 mod Onnx model and protobuf model improvements ------------- PR: https://git.openjdk.org/babylon/pull/414 From mcimadamore at openjdk.org Tue May 6 10:52:35 2025 From: mcimadamore at openjdk.org (Maurizio Cimadamore) Date: Tue, 6 May 2025 10:52:35 GMT Subject: [code-reflection] RFR: Lambda model uniqueness [v4] In-Reply-To: References: <5pt5GFZHVDGTp2_9GLFo9kjrE4XatA23wwjDg9OOf9M=.700bfa02-c542-4187-aeec-00a2969f960a@github.com> Message-ID: On Sun, 4 May 2025 13:49:46 GMT, Mourad Abbay wrote: >> Lambdas that share the same proxy class, have the same implementation and therefore have the same code model generated by the compiler. In this PR, we share the code model across instances of the same proxy class. >> This also means we reduce execution time as we only need to invoke `opMethod` once for given proxy class. >> With these changes we can compare lambda code model based on identity. > > Mourad Abbay has updated the pull request incrementally with one additional commit since the last revision: > > Create Quoted object lazily src/java.base/share/classes/java/lang/invoke/InnerClassLambdaMetafactory.java line 402: > 400: public void accept(CodeBuilder cob) { > 401: if (factoryType.parameterCount() == 0 && disableEagerInitialization) { > 402: ClassDesc lambdaTypeDescriptor = classDesc(factoryType.returnType()); This bit of code seems to be the same as the one in `generateClassInitializer` -- which is now no longer used? ------------- PR Review Comment: https://git.openjdk.org/babylon/pull/420#discussion_r2075209858 From mcimadamore at openjdk.org Tue May 6 11:04:38 2025 From: mcimadamore at openjdk.org (Maurizio Cimadamore) Date: Tue, 6 May 2025 11:04:38 GMT Subject: [code-reflection] RFR: Lambda model uniqueness [v4] In-Reply-To: References: <5pt5GFZHVDGTp2_9GLFo9kjrE4XatA23wwjDg9OOf9M=.700bfa02-c542-4187-aeec-00a2969f960a@github.com> Message-ID: On Sun, 4 May 2025 13:49:46 GMT, Mourad Abbay wrote: >> Lambdas that share the same proxy class, have the same implementation and therefore have the same code model generated by the compiler. In this PR, we share the code model across instances of the same proxy class. >> This also means we reduce execution time as we only need to invoke `opMethod` once for given proxy class. >> With these changes we can compare lambda code model based on identity. > > Mourad Abbay has updated the pull request incrementally with one additional commit since the last revision: > > Create Quoted object lazily src/java.base/share/classes/java/lang/invoke/InnerClassLambdaMetafactory.java line 496: > 494: // push the receiver on the stack for operand of put field instruction > 495: cob.aload(0) > 496: // load class data: CodeReflectionSupport.HANDLE_MAKE_QUOTED Who uses `generateQuotedFieldInitializer` ? src/java.base/share/classes/java/lang/invoke/InnerClassLambdaMetafactory.java line 691: > 689: } > 690: > 691: private void createQuotedObj(CodeBuilder cob) { Seems like the code here duplicates `generateQuotedFieldInitializer` ------------- PR Review Comment: https://git.openjdk.org/babylon/pull/420#discussion_r2075225001 PR Review Comment: https://git.openjdk.org/babylon/pull/420#discussion_r2075227202 From mcimadamore at openjdk.org Tue May 6 11:09:30 2025 From: mcimadamore at openjdk.org (Maurizio Cimadamore) Date: Tue, 6 May 2025 11:09:30 GMT Subject: [code-reflection] RFR: Lambda model uniqueness [v4] In-Reply-To: References: <5pt5GFZHVDGTp2_9GLFo9kjrE4XatA23wwjDg9OOf9M=.700bfa02-c542-4187-aeec-00a2969f960a@github.com> Message-ID: On Sun, 4 May 2025 13:49:46 GMT, Mourad Abbay wrote: >> Lambdas that share the same proxy class, have the same implementation and therefore have the same code model generated by the compiler. In this PR, we share the code model across instances of the same proxy class. >> This also means we reduce execution time as we only need to invoke `opMethod` once for given proxy class. >> With these changes we can compare lambda code model based on identity. > > Mourad Abbay has updated the pull request incrementally with one additional commit since the last revision: > > Create Quoted object lazily It seems to me this patch is doing several things at once. While sharing the model instance in a new static field, I'm less sure about the fact that the lazy initialization of the `Quoted` object is worth the extra code complexity (e.g. double checked locking) -- after all, computing the `Quoted` instance in the constructor of the generated class should not mess up with uniqueness (as there will still be a new instance of `Quoted` per new instance of the generated class?) ------------- PR Review: https://git.openjdk.org/babylon/pull/420#pullrequestreview-2817857995 From mabbay at openjdk.org Tue May 6 11:14:19 2025 From: mabbay at openjdk.org (Mourad Abbay) Date: Tue, 6 May 2025 11:14:19 GMT Subject: [code-reflection] RFR: Lambda model uniqueness [v5] In-Reply-To: <5pt5GFZHVDGTp2_9GLFo9kjrE4XatA23wwjDg9OOf9M=.700bfa02-c542-4187-aeec-00a2969f960a@github.com> References: <5pt5GFZHVDGTp2_9GLFo9kjrE4XatA23wwjDg9OOf9M=.700bfa02-c542-4187-aeec-00a2969f960a@github.com> Message-ID: <1Qeup2IWyFbh4jMPpXP2qZ0Sun7IJBBzetH4iWA6tXY=.e2fe5bf2-513f-4c04-9c52-cc7c6f460414@github.com> > Lambdas that share the same proxy class, have the same implementation and therefore have the same code model generated by the compiler. In this PR, we share the code model across instances of the same proxy class. > This also means we reduce execution time as we only need to invoke `opMethod` once for given proxy class. > With these changes we can compare lambda code model based on identity. Mourad Abbay has updated the pull request incrementally with one additional commit since the last revision: Remove unused code ------------- Changes: - all: https://git.openjdk.org/babylon/pull/420/files - new: https://git.openjdk.org/babylon/pull/420/files/fc94c111..849aa6e6 Webrevs: - full: https://webrevs.openjdk.org/?repo=babylon&pr=420&range=04 - incr: https://webrevs.openjdk.org/?repo=babylon&pr=420&range=03-04 Stats: 55 lines in 1 file changed: 0 ins; 55 del; 0 mod Patch: https://git.openjdk.org/babylon/pull/420.diff Fetch: git fetch https://git.openjdk.org/babylon.git pull/420/head:pull/420 PR: https://git.openjdk.org/babylon/pull/420 From mabbay at openjdk.org Tue May 6 11:28:27 2025 From: mabbay at openjdk.org (Mourad Abbay) Date: Tue, 6 May 2025 11:28:27 GMT Subject: [code-reflection] RFR: Lambda model uniqueness [v4] In-Reply-To: References: <5pt5GFZHVDGTp2_9GLFo9kjrE4XatA23wwjDg9OOf9M=.700bfa02-c542-4187-aeec-00a2969f960a@github.com> Message-ID: On Tue, 6 May 2025 11:06:32 GMT, Maurizio Cimadamore wrote: > It seems to me this patch is doing several things at once. While sharing the model instance in a new static field, I'm less sure about the fact that the lazy initialization of the `Quoted` object is worth the extra code complexity (e.g. double checked locking) -- after all, computing the `Quoted` instance in the constructor of the generated class should not mess up with uniqueness (as there will still be a new instance of `Quoted` per new instance of the generated class?) The end goal is to lazily invoke `opMethod`. This saves space as the model is in memory when needed. It also saves time. Quoted object depends on the result of opMethod, so Quoted object must be initialized lazily. ------------- PR Comment: https://git.openjdk.org/babylon/pull/420#issuecomment-2854216310 From mcimadamore at openjdk.org Tue May 6 17:12:28 2025 From: mcimadamore at openjdk.org (Maurizio Cimadamore) Date: Tue, 6 May 2025 17:12:28 GMT Subject: [code-reflection] RFR: Lambda model uniqueness [v4] In-Reply-To: References: <5pt5GFZHVDGTp2_9GLFo9kjrE4XatA23wwjDg9OOf9M=.700bfa02-c542-4187-aeec-00a2969f960a@github.com> Message-ID: On Tue, 6 May 2025 11:24:50 GMT, Mourad Abbay wrote: > > It seems to me this patch is doing several things at once. While sharing the model instance in a new static field, I'm less sure about the fact that the lazy initialization of the `Quoted` object is worth the extra code complexity (e.g. double checked locking) -- after all, computing the `Quoted` instance in the constructor of the generated class should not mess up with uniqueness (as there will still be a new instance of `Quoted` per new instance of the generated class?) > > The end goal is to lazily invoke `opMethod`. This saves space as the model is in memory when needed. It also saves time. Quoted object depends on the result of opMethod, so Quoted object must be initialized lazily. Maybe I'm reading the code wrong -- but it seems to me that we cache the FuncOp in a static field. This static field is initialized with the lambda class (in the static initializer), so there's no laziness there -- but the same `FuncOp` can be shared across all instances of the lambda class. Then there is the method to create the `Quoted` object; this method needs the `FuncOp` as well as the captured value -- and pass everything to the interpreter -- this is done in `QuotedHelper`. It seems to me that the new code is lazily computing the quoted instance -- e.g. if nobody wants the `Quoted` object, then we won't try to compute it eagerly when the lambda instance is constructed. So, it seems to me that opMethod is always called eagerly in the class initializer. The only thing that is lazy is the interpreter code to generated the `Quoted`. Am I reading incorrectly? ------------- PR Comment: https://git.openjdk.org/babylon/pull/420#issuecomment-2855331142 From psandoz at openjdk.org Tue May 6 17:12:28 2025 From: psandoz at openjdk.org (Paul Sandoz) Date: Tue, 6 May 2025 17:12:28 GMT Subject: [code-reflection] RFR: Lambda model uniqueness [v5] In-Reply-To: <1Qeup2IWyFbh4jMPpXP2qZ0Sun7IJBBzetH4iWA6tXY=.e2fe5bf2-513f-4c04-9c52-cc7c6f460414@github.com> References: <5pt5GFZHVDGTp2_9GLFo9kjrE4XatA23wwjDg9OOf9M=.700bfa02-c542-4187-aeec-00a2969f960a@github.com> <1Qeup2IWyFbh4jMPpXP2qZ0Sun7IJBBzetH4iWA6tXY=.e2fe5bf2-513f-4c04-9c52-cc7c6f460414@github.com> Message-ID: On Tue, 6 May 2025 11:14:19 GMT, Mourad Abbay wrote: >> Lambdas that share the same proxy class, have the same implementation and therefore have the same code model generated by the compiler. In this PR, we share the code model across instances of the same proxy class. >> This also means we reduce execution time as we only need to invoke `opMethod` once for given proxy class. >> With these changes we can compare lambda code model based on identity. > > Mourad Abbay has updated the pull request incrementally with one additional commit since the last revision: > > Remove unused code src/java.base/share/classes/java/lang/invoke/InnerClassLambdaMetafactory.java line 397: > 395: } > 396: > 397: private void generateStaticInitializer(ClassBuilder clb) { [Class initialization method](https://docs.oracle.com/javase/specs/jvms/se24/html/jvms-2.html#jvms-2.9.2) is the correct term, recommend renaming the method to `generateClassInitializationMethod`. src/java.base/share/classes/java/lang/invoke/InnerClassLambdaMetafactory.java line 398: > 396: > 397: private void generateStaticInitializer(ClassBuilder clb) { > 398: clb.withMethodBody(CLASS_INIT_NAME, MTD_void, ACC_STATIC, new Consumer() { This might always create a class initializer method, which might be empty. I think we should condition the creation. src/java.base/share/classes/java/lang/invoke/InnerClassLambdaMetafactory.java line 582: > 580: @Override > 581: public void accept(CodeBuilder cob) { > 582: // bytecode for double-checked locking (copied from the compiler) I wonder if you can rewrite this using `BlockCodeBuilder` and `CatchBuilder` (see usages in `CodeBuilder` methods)? If so i think it would be easier to comprehend as the structure of the code becomes evident. src/java.base/share/classes/java/lang/invoke/InnerClassLambdaMetafactory.java line 605: > 603: .aload(1) > 604: .ifnonnull(l2); > 605: createQuotedObj(cob); I find this method confusing because we are performing DCL over the quoted instance and not the code model. I can see why you did that. Can we refine this in a subsequent PR to do so on the model and then we can create `Quoted` instance each time? Later we can turn `Quoted` into a value class, and for now we can mark it as value-based. Then i think that allows us to adjust later if we so wish to store the code model in a weak reference ------------- PR Review Comment: https://git.openjdk.org/babylon/pull/420#discussion_r2075885528 PR Review Comment: https://git.openjdk.org/babylon/pull/420#discussion_r2075882688 PR Review Comment: https://git.openjdk.org/babylon/pull/420#discussion_r2075880526 PR Review Comment: https://git.openjdk.org/babylon/pull/420#discussion_r2075922108 From mabbay at openjdk.org Tue May 6 18:15:36 2025 From: mabbay at openjdk.org (Mourad Abbay) Date: Tue, 6 May 2025 18:15:36 GMT Subject: [code-reflection] RFR: Lambda model uniqueness [v4] In-Reply-To: References: <5pt5GFZHVDGTp2_9GLFo9kjrE4XatA23wwjDg9OOf9M=.700bfa02-c542-4187-aeec-00a2969f960a@github.com> Message-ID: On Tue, 6 May 2025 17:09:26 GMT, Maurizio Cimadamore wrote: > > > It seems to me this patch is doing several things at once. While sharing the model instance in a new static field, I'm less sure about the fact that the lazy initialization of the `Quoted` object is worth the extra code complexity (e.g. double checked locking) -- after all, computing the `Quoted` instance in the constructor of the generated class should not mess up with uniqueness (as there will still be a new instance of `Quoted` per new instance of the generated class?) > > > > > > The end goal is to lazily invoke `opMethod`. This saves space as the model is in memory when needed. It also saves time. Quoted object depends on the result of opMethod, so Quoted object must be initialized lazily. > > Maybe I'm reading the code wrong -- but it seems to me that we cache the FuncOp in a static field. This static field is initialized with the lambda class (in the static initializer), so there's no laziness there -- but the same `FuncOp` can be shared across all instances of the lambda class. Then there is the method to create the `Quoted` object; this method needs the `FuncOp` as well as the captured value -- and pass everything to the interpreter -- this is done in `QuotedHelper`. It seems to me that the new code is lazily computing the quoted instance -- e.g. if nobody wants the `Quoted` object, then we won't try to compute it eagerly when the lambda instance is constructed. > > So, it seems to me that opMethod is always called eagerly in the class initializer. The only thing that is lazy is the interpreter code to generated the `Quoted`. Am I reading incorrectly? You are right. In a previous comment, I said that the end goal is to invoke opMethod lazily. I am still working to achieve that goal. ------------- PR Comment: https://git.openjdk.org/babylon/pull/420#issuecomment-2855498382 From mcimadamore at openjdk.org Wed May 7 09:24:36 2025 From: mcimadamore at openjdk.org (Maurizio Cimadamore) Date: Wed, 7 May 2025 09:24:36 GMT Subject: [code-reflection] RFR: Lambda model uniqueness [v4] In-Reply-To: References: <5pt5GFZHVDGTp2_9GLFo9kjrE4XatA23wwjDg9OOf9M=.700bfa02-c542-4187-aeec-00a2969f960a@github.com> Message-ID: On Tue, 6 May 2025 18:13:07 GMT, Mourad Abbay wrote: > > > > It seems to me this patch is doing several things at once. While sharing the model instance in a new static field, I'm less sure about the fact that the lazy initialization of the `Quoted` object is worth the extra code complexity (e.g. double checked locking) -- after all, computing the `Quoted` instance in the constructor of the generated class should not mess up with uniqueness (as there will still be a new instance of `Quoted` per new instance of the generated class?) > > > > > > > > > The end goal is to lazily invoke `opMethod`. This saves space as the model is in memory when needed. It also saves time. Quoted object depends on the result of opMethod, so Quoted object must be initialized lazily. > > > > > > Maybe I'm reading the code wrong -- but it seems to me that we cache the FuncOp in a static field. This static field is initialized with the lambda class (in the static initializer), so there's no laziness there -- but the same `FuncOp` can be shared across all instances of the lambda class. Then there is the method to create the `Quoted` object; this method needs the `FuncOp` as well as the captured value -- and pass everything to the interpreter -- this is done in `QuotedHelper`. It seems to me that the new code is lazily computing the quoted instance -- e.g. if nobody wants the `Quoted` object, then we won't try to compute it eagerly when the lambda instance is constructed. > > So, it seems to me that opMethod is always called eagerly in the class initializer. The only thing that is lazy is the interpreter code to generated the `Quoted`. Am I reading incorrectly? > > You are right. In a previous comment, I said that the end goal is to invoke opMethod lazily. I am still working to achieve that goal. I guess the reason I'm confused is that this PR is called "lambda model uniqueness". IMHO, caching the call to opMethod in a static field (like you do in this PR) is enough to achieve "model uniqueness". If you also want to tackle laziness, that's ok, but I feel (and perhaps Paul's comment goes in this direction), that it would be better to do that in a separate PR? ------------- PR Comment: https://git.openjdk.org/babylon/pull/420#issuecomment-2857824709 From gfrost at openjdk.org Wed May 7 11:47:20 2025 From: gfrost at openjdk.org (Gary Frost) Date: Wed, 7 May 2025 11:47:20 GMT Subject: git: openjdk/babylon: code-reflection: Fixes required for maven/cmake on ubuntu - note cuda ptx working, =?UTF-8?B?Y3Vk4oCm?= Message-ID: <1e35e689-a764-48b0-88a7-bb1dd539554b@openjdk.org> Changeset: 04277c28 Branch: code-reflection Author: Gary Frost Date: 2025-05-07 11:46:12 +0000 URL: https://git.openjdk.org/babylon/commit/04277c28eb71a92de0cc68d422ed5a3070ccef8a Fixes required for maven/cmake on ubuntu - note cuda ptx working, cud? ! hat/backends/ffi/cuda/CMakeLists.txt ! hat/backends/ffi/cuda/cpp/cuda_backend.cpp ! hat/extractions/cuda/CMakeLists.txt ! hat/wrap/glwrap/pom.xml From gfrost at openjdk.org Wed May 7 11:49:51 2025 From: gfrost at openjdk.org (Gary Frost) Date: Wed, 7 May 2025 11:49:51 GMT Subject: [code-reflection] Integrated: Fixes required for maven/cmake on ubuntu - note cuda ptx working, =?UTF-8?B?Y3Vk4oCm?= Message-ID: Changes neeed to build opencl and cuda on ubuntu Mostly cmake realted. CMake creates inconsistent vars for CUDA include deps, this breaks ffi-backend build and jextract Similarly pom.xml needs a conditional file exclusion for wrapping opengl. ------------- Commit messages: - Fixes required for maven/cmake on ubuntu - note cuda ptx working, cuda code gen failing Changes: https://git.openjdk.org/babylon/pull/421/files Webrev: https://webrevs.openjdk.org/?repo=babylon&pr=421&range=00 Stats: 21 lines in 4 files changed: 13 ins; 3 del; 5 mod Patch: https://git.openjdk.org/babylon/pull/421.diff Fetch: git fetch https://git.openjdk.org/babylon.git pull/421/head:pull/421 PR: https://git.openjdk.org/babylon/pull/421 From gfrost at openjdk.org Wed May 7 11:49:51 2025 From: gfrost at openjdk.org (Gary Frost) Date: Wed, 7 May 2025 11:49:51 GMT Subject: [code-reflection] Integrated: Fixes required for maven/cmake on ubuntu - note cuda ptx working, =?UTF-8?B?Y3Vk4oCm?= In-Reply-To: References: Message-ID: On Wed, 7 May 2025 11:44:06 GMT, Gary Frost wrote: > Changes neeed to build opencl and cuda on ubuntu > > Mostly cmake realted. CMake creates inconsistent vars for CUDA include deps, this breaks ffi-backend build and jextract > > Similarly pom.xml needs a conditional file exclusion for wrapping opengl. This pull request has now been integrated. Changeset: 04277c28 Author: Gary Frost URL: https://git.openjdk.org/babylon/commit/04277c28eb71a92de0cc68d422ed5a3070ccef8a Stats: 21 lines in 4 files changed: 13 ins; 3 del; 5 mod Fixes required for maven/cmake on ubuntu - note cuda ptx working, cud? ------------- PR: https://git.openjdk.org/babylon/pull/421 From mabbay at openjdk.org Wed May 7 14:59:37 2025 From: mabbay at openjdk.org (Mourad Abbay) Date: Wed, 7 May 2025 14:59:37 GMT Subject: [code-reflection] RFR: Lambda model uniqueness [v5] In-Reply-To: <1Qeup2IWyFbh4jMPpXP2qZ0Sun7IJBBzetH4iWA6tXY=.e2fe5bf2-513f-4c04-9c52-cc7c6f460414@github.com> References: <5pt5GFZHVDGTp2_9GLFo9kjrE4XatA23wwjDg9OOf9M=.700bfa02-c542-4187-aeec-00a2969f960a@github.com> <1Qeup2IWyFbh4jMPpXP2qZ0Sun7IJBBzetH4iWA6tXY=.e2fe5bf2-513f-4c04-9c52-cc7c6f460414@github.com> Message-ID: On Tue, 6 May 2025 11:14:19 GMT, Mourad Abbay wrote: >> Lambdas that share the same proxy class, have the same implementation and therefore have the same code model generated by the compiler. In this PR, we share the code model across instances of the same proxy class. >> This also means we reduce execution time as we only need to invoke `opMethod` once for given proxy class. >> With these changes we can compare lambda code model based on identity. > > Mourad Abbay has refreshed the contents of this pull request, and previous commits have been removed. Incremental views are not available. I will work on lazy computation of lambda model and Quoted object in a separate PR. ------------- PR Comment: https://git.openjdk.org/babylon/pull/420#issuecomment-2858926587 From mabbay at openjdk.org Wed May 7 17:15:21 2025 From: mabbay at openjdk.org (Mourad Abbay) Date: Wed, 7 May 2025 17:15:21 GMT Subject: [code-reflection] RFR: Lambda model uniqueness [v6] In-Reply-To: <5pt5GFZHVDGTp2_9GLFo9kjrE4XatA23wwjDg9OOf9M=.700bfa02-c542-4187-aeec-00a2969f960a@github.com> References: <5pt5GFZHVDGTp2_9GLFo9kjrE4XatA23wwjDg9OOf9M=.700bfa02-c542-4187-aeec-00a2969f960a@github.com> Message-ID: > Lambdas that share the same proxy class, have the same implementation and therefore have the same code model generated by the compiler. In this PR, we share the code model across instances of the same proxy class. > This also means we reduce execution time as we only need to invoke `opMethod` once for given proxy class. > With these changes we can compare lambda code model based on identity. Mourad Abbay has refreshed the contents of this pull request, and previous commits have been removed. The incremental views will show differences compared to the previous content of the PR. The pull request contains one new commit since the last revision: Apply Paul's comments ------------- Changes: - all: https://git.openjdk.org/babylon/pull/420/files - new: https://git.openjdk.org/babylon/pull/420/files/849aa6e6..ade697ba Webrevs: - full: https://webrevs.openjdk.org/?repo=babylon&pr=420&range=05 - incr: https://webrevs.openjdk.org/?repo=babylon&pr=420&range=04-05 Stats: 146 lines in 1 file changed: 61 ins; 80 del; 5 mod Patch: https://git.openjdk.org/babylon/pull/420.diff Fetch: git fetch https://git.openjdk.org/babylon.git pull/420/head:pull/420 PR: https://git.openjdk.org/babylon/pull/420 From mabbay at openjdk.org Wed May 7 17:47:24 2025 From: mabbay at openjdk.org (Mourad Abbay) Date: Wed, 7 May 2025 17:47:24 GMT Subject: [code-reflection] RFR: Lambda model uniqueness [v7] In-Reply-To: <5pt5GFZHVDGTp2_9GLFo9kjrE4XatA23wwjDg9OOf9M=.700bfa02-c542-4187-aeec-00a2969f960a@github.com> References: <5pt5GFZHVDGTp2_9GLFo9kjrE4XatA23wwjDg9OOf9M=.700bfa02-c542-4187-aeec-00a2969f960a@github.com> Message-ID: > Lambdas that share the same proxy class, have the same implementation and therefore have the same code model generated by the compiler. In this PR, we share the code model across instances of the same proxy class. > This also means we reduce execution time as we only need to invoke `opMethod` once for given proxy class. > With these changes we can compare lambda code model based on identity. Mourad Abbay has updated the pull request incrementally with one additional commit since the last revision: Remove unused method ------------- Changes: - all: https://git.openjdk.org/babylon/pull/420/files - new: https://git.openjdk.org/babylon/pull/420/files/ade697ba..a9935e70 Webrevs: - full: https://webrevs.openjdk.org/?repo=babylon&pr=420&range=06 - incr: https://webrevs.openjdk.org/?repo=babylon&pr=420&range=05-06 Stats: 22 lines in 1 file changed: 0 ins; 22 del; 0 mod Patch: https://git.openjdk.org/babylon/pull/420.diff Fetch: git fetch https://git.openjdk.org/babylon.git pull/420/head:pull/420 PR: https://git.openjdk.org/babylon/pull/420 From mabbay at openjdk.org Wed May 7 17:52:44 2025 From: mabbay at openjdk.org (Mourad Abbay) Date: Wed, 7 May 2025 17:52:44 GMT Subject: [code-reflection] RFR: Create Quoted instance when it's requested Message-ID: <84_eHlapXNQnZlQWoS_4LjHxDF6A1qQBLP72kpHpnSc=.c47a9616-16c6-468a-bcae-6cfb0a3293b6@github.com> This PR moves construction of Quoted instance from constructor to the method that returns it. This simple change will allow us to lazily invoke `opMethod`, something I will work on in a follow up PR. This changes are based on #420. ------------- Commit messages: - Merge branch 'lambda-model-uniqueness' into Quoted-obj-on-demand - Remove unused method - Create Quoted instance when it's requested - Apply Paul's comments - Remove static field as it causes compilation error - Define static fields and update comments - Rename a test - Share the compiler generated code model of a lambda across instances of its proxy class Changes: https://git.openjdk.org/babylon/pull/422/files Webrev: https://webrevs.openjdk.org/?repo=babylon&pr=422&range=00 Stats: 241 lines in 4 files changed: 106 ins; 117 del; 18 mod Patch: https://git.openjdk.org/babylon/pull/422.diff Fetch: git fetch https://git.openjdk.org/babylon.git pull/422/head:pull/422 PR: https://git.openjdk.org/babylon/pull/422 From mabbay at openjdk.org Wed May 7 18:31:35 2025 From: mabbay at openjdk.org (Mourad Abbay) Date: Wed, 7 May 2025 18:31:35 GMT Subject: [code-reflection] RFR: Lambda model uniqueness [v8] In-Reply-To: <5pt5GFZHVDGTp2_9GLFo9kjrE4XatA23wwjDg9OOf9M=.700bfa02-c542-4187-aeec-00a2969f960a@github.com> References: <5pt5GFZHVDGTp2_9GLFo9kjrE4XatA23wwjDg9OOf9M=.700bfa02-c542-4187-aeec-00a2969f960a@github.com> Message-ID: > Lambdas that share the same proxy class, have the same implementation and therefore have the same code model generated by the compiler. In this PR, we share the code model across instances of the same proxy class. > This also means we reduce execution time as we only need to invoke `opMethod` once for given proxy class. > With these changes we can compare lambda code model based on identity. Mourad Abbay has updated the pull request incrementally with one additional commit since the last revision: Test lambda model uniqueness in parallel execution ------------- Changes: - all: https://git.openjdk.org/babylon/pull/420/files - new: https://git.openjdk.org/babylon/pull/420/files/a9935e70..31ac31ad Webrevs: - full: https://webrevs.openjdk.org/?repo=babylon&pr=420&range=07 - incr: https://webrevs.openjdk.org/?repo=babylon&pr=420&range=06-07 Stats: 19 lines in 1 file changed: 17 ins; 0 del; 2 mod Patch: https://git.openjdk.org/babylon/pull/420.diff Fetch: git fetch https://git.openjdk.org/babylon.git pull/420/head:pull/420 PR: https://git.openjdk.org/babylon/pull/420 From psandoz at openjdk.org Wed May 7 19:01:09 2025 From: psandoz at openjdk.org (Paul Sandoz) Date: Wed, 7 May 2025 19:01:09 GMT Subject: [code-reflection] RFR: Lambda model uniqueness [v8] In-Reply-To: References: <5pt5GFZHVDGTp2_9GLFo9kjrE4XatA23wwjDg9OOf9M=.700bfa02-c542-4187-aeec-00a2969f960a@github.com> Message-ID: On Wed, 7 May 2025 18:31:35 GMT, Mourad Abbay wrote: >> Lambdas that share the same proxy class, have the same implementation and therefore have the same code model generated by the compiler. In this PR, we share the code model across instances of the same proxy class. >> This also means we reduce execution time as we only need to invoke `opMethod` once for given proxy class. >> With these changes we can compare lambda code model based on identity. > > Mourad Abbay has updated the pull request incrementally with one additional commit since the last revision: > > Test lambda model uniqueness in parallel execution Marked as reviewed by psandoz (Lead). ------------- PR Review: https://git.openjdk.org/babylon/pull/420#pullrequestreview-2822899580 From mabbay at openjdk.org Wed May 7 19:22:02 2025 From: mabbay at openjdk.org (Mourad Abbay) Date: Wed, 7 May 2025 19:22:02 GMT Subject: git: openjdk/babylon: code-reflection: Lambda model uniqueness Message-ID: Changeset: 6e062d51 Branch: code-reflection Author: Mourad Abbay Date: 2025-05-07 19:20:19 +0000 URL: https://git.openjdk.org/babylon/commit/6e062d51a7c4a05a4263def900d7fc7d9c9b9e0b Lambda model uniqueness Reviewed-by: psandoz ! src/java.base/share/classes/java/lang/invoke/InnerClassLambdaMetafactory.java + test/langtools/tools/javac/reflect/LambdaModelUniquenessTest.java = test/langtools/tools/javac/reflect/MethodModelUniquenessTest.java From mabbay at openjdk.org Wed May 7 19:23:20 2025 From: mabbay at openjdk.org (Mourad Abbay) Date: Wed, 7 May 2025 19:23:20 GMT Subject: [code-reflection] Integrated: Lambda model uniqueness In-Reply-To: <5pt5GFZHVDGTp2_9GLFo9kjrE4XatA23wwjDg9OOf9M=.700bfa02-c542-4187-aeec-00a2969f960a@github.com> References: <5pt5GFZHVDGTp2_9GLFo9kjrE4XatA23wwjDg9OOf9M=.700bfa02-c542-4187-aeec-00a2969f960a@github.com> Message-ID: On Sat, 3 May 2025 08:44:57 GMT, Mourad Abbay wrote: > Lambdas that share the same proxy class, have the same implementation and therefore have the same code model generated by the compiler. In this PR, we share the code model across instances of the same proxy class. > This also means we reduce execution time as we only need to invoke `opMethod` once for given proxy class. > With these changes we can compare lambda code model based on identity. This pull request has now been integrated. Changeset: 6e062d51 Author: Mourad Abbay URL: https://git.openjdk.org/babylon/commit/6e062d51a7c4a05a4263def900d7fc7d9c9b9e0b Stats: 134 lines in 3 files changed: 97 ins; 17 del; 20 mod Lambda model uniqueness Reviewed-by: psandoz ------------- PR: https://git.openjdk.org/babylon/pull/420 From psandoz at openjdk.org Wed May 7 19:34:13 2025 From: psandoz at openjdk.org (Paul Sandoz) Date: Wed, 7 May 2025 19:34:13 GMT Subject: [code-reflection] RFR: Create Quoted instance when it's requested In-Reply-To: <84_eHlapXNQnZlQWoS_4LjHxDF6A1qQBLP72kpHpnSc=.c47a9616-16c6-468a-bcae-6cfb0a3293b6@github.com> References: <84_eHlapXNQnZlQWoS_4LjHxDF6A1qQBLP72kpHpnSc=.c47a9616-16c6-468a-bcae-6cfb0a3293b6@github.com> Message-ID: <2dIof0tq-LOkfTrik8LQk9DZ6kgiUXGuk52lIlPjk5I=.e5022a9e-3ac4-4255-b387-84a65a8e6964@github.com> On Wed, 7 May 2025 17:48:14 GMT, Mourad Abbay wrote: > This PR moves construction of Quoted instance from constructor to the method that returns it. This simple change will allow us to lazily invoke `opMethod`, something I will work on in a follow up PR. > > This changes are based on #420. I totally forgot we currently run the lambda code model through the interpreter to obtain the quoted instance! I don't think we should modify the meta factory as i suggested, apologies for misleading you. At some point I think we need to remove the dependency on the interpreter - so we explicitly validate the lambda code model and the operate on it directly to create the `Quoted` instance. (For specification I think it is important we retain the current lambda code model program behavior.) In fact i am not sure it is worth doing anything lazy right now in subsequent PRs and a better focus of effort would be to remove the dependency on the interpreter. After that we can address lazy construction of models. ------------- PR Review: https://git.openjdk.org/babylon/pull/422#pullrequestreview-2822973831 From mabbay at openjdk.org Wed May 7 19:43:12 2025 From: mabbay at openjdk.org (Mourad Abbay) Date: Wed, 7 May 2025 19:43:12 GMT Subject: [code-reflection] Withdrawn: Create Quoted instance when it's requested In-Reply-To: <84_eHlapXNQnZlQWoS_4LjHxDF6A1qQBLP72kpHpnSc=.c47a9616-16c6-468a-bcae-6cfb0a3293b6@github.com> References: <84_eHlapXNQnZlQWoS_4LjHxDF6A1qQBLP72kpHpnSc=.c47a9616-16c6-468a-bcae-6cfb0a3293b6@github.com> Message-ID: On Wed, 7 May 2025 17:48:14 GMT, Mourad Abbay wrote: > This PR moves construction of Quoted instance from constructor to the method that returns it. This simple change will allow us to lazily invoke `opMethod`, something I will work on in a follow up PR. > > This changes are based on #420. This pull request has been closed without being integrated. ------------- PR: https://git.openjdk.org/babylon/pull/422 From mabbay at openjdk.org Mon May 12 15:15:57 2025 From: mabbay at openjdk.org (Mourad Abbay) Date: Mon, 12 May 2025 15:15:57 GMT Subject: [code-reflection] RFR: Replace the use of Interpreter to create Quoted instance with specialized code Message-ID: Creating Quoted instance was done by invoking the Interpreter. The Interpreter is too general and if someone could somehow inject any code model we will interpret that as well. This PR replace the use of the interpreter with specialized code. ------------- Commit messages: - Replace the use of Interpreter to create Quoted instance with specialized code Changes: https://git.openjdk.org/babylon/pull/424/files Webrev: https://webrevs.openjdk.org/?repo=babylon&pr=424&range=00 Stats: 32 lines in 3 files changed: 22 ins; 2 del; 8 mod Patch: https://git.openjdk.org/babylon/pull/424.diff Fetch: git fetch https://git.openjdk.org/babylon.git pull/424/head:pull/424 PR: https://git.openjdk.org/babylon/pull/424 From mabbay at openjdk.org Mon May 12 21:46:27 2025 From: mabbay at openjdk.org (Mourad Abbay) Date: Mon, 12 May 2025 21:46:27 GMT Subject: [code-reflection] RFR: Disallow insertion of a root op in a block Message-ID: <8sB7qlbnONF4wEI39cUCoik0sXf2Oj8NV7TsJ-ZbDWA=.a4cd10a1-ef17-4170-8697-4fd15557b568@github.com> A root operation shouldn't be inserted in a block. This changes enforce this rule. ------------- Commit messages: - Disallow insertion of a root op in a block Changes: https://git.openjdk.org/babylon/pull/425/files Webrev: https://webrevs.openjdk.org/?repo=babylon&pr=425&range=00 Stats: 42 lines in 4 files changed: 41 ins; 0 del; 1 mod Patch: https://git.openjdk.org/babylon/pull/425.diff Fetch: git fetch https://git.openjdk.org/babylon.git pull/425/head:pull/425 PR: https://git.openjdk.org/babylon/pull/425 From gfrost at openjdk.org Tue May 13 12:25:11 2025 From: gfrost at openjdk.org (Gary Frost) Date: Tue, 13 May 2025 12:25:11 GMT Subject: git: openjdk/babylon: code-reflection: Removed hardcoded nvcc path Message-ID: <43b383d4-a0ba-4690-b283-5a7e43fc5955@openjdk.org> Changeset: b958a301 Branch: code-reflection Author: Gary Frost Date: 2025-05-13 12:23:41 +0000 URL: https://git.openjdk.org/babylon/commit/b958a30173082927d2ed5c7468955340a6fe35a0 Removed hardcoded nvcc path ! hat/backends/ffi/cuda/cpp/cuda_backend.cpp From gfrost at openjdk.org Tue May 13 12:26:17 2025 From: gfrost at openjdk.org (Gary Frost) Date: Tue, 13 May 2025 12:26:17 GMT Subject: [code-reflection] Integrated: Removed hardcoded nvcc path Message-ID: Rookie mistake. I had accidently hardcoded path to nvcc in nvidia backend. ------------- Commit messages: - Removed hardcoded nvcc path Changes: https://git.openjdk.org/babylon/pull/426/files Webrev: https://webrevs.openjdk.org/?repo=babylon&pr=426&range=00 Stats: 7 lines in 1 file changed: 0 ins; 1 del; 6 mod Patch: https://git.openjdk.org/babylon/pull/426.diff Fetch: git fetch https://git.openjdk.org/babylon.git pull/426/head:pull/426 PR: https://git.openjdk.org/babylon/pull/426 From gfrost at openjdk.org Tue May 13 12:26:18 2025 From: gfrost at openjdk.org (Gary Frost) Date: Tue, 13 May 2025 12:26:18 GMT Subject: [code-reflection] Integrated: Removed hardcoded nvcc path In-Reply-To: References: Message-ID: On Tue, 13 May 2025 12:22:02 GMT, Gary Frost wrote: > Rookie mistake. I had accidently hardcoded path to nvcc in nvidia backend. This pull request has now been integrated. Changeset: b958a301 Author: Gary Frost URL: https://git.openjdk.org/babylon/commit/b958a30173082927d2ed5c7468955340a6fe35a0 Stats: 7 lines in 1 file changed: 0 ins; 1 del; 6 mod Removed hardcoded nvcc path ------------- PR: https://git.openjdk.org/babylon/pull/426 From psandoz at openjdk.org Tue May 13 19:26:17 2025 From: psandoz at openjdk.org (Paul Sandoz) Date: Tue, 13 May 2025 19:26:17 GMT Subject: [code-reflection] RFR: Disallow insertion of a root op in a block In-Reply-To: <8sB7qlbnONF4wEI39cUCoik0sXf2Oj8NV7TsJ-ZbDWA=.a4cd10a1-ef17-4170-8697-4fd15557b568@github.com> References: <8sB7qlbnONF4wEI39cUCoik0sXf2Oj8NV7TsJ-ZbDWA=.a4cd10a1-ef17-4170-8697-4fd15557b568@github.com> Message-ID: On Mon, 12 May 2025 21:42:35 GMT, Mourad Abbay wrote: > A root operation shouldn't be inserted in a block. This changes enforce this rule. That was a reasonable attempt, but it will not work because a func op can be a descendant of a module op (there might be one or two tests that fail). Nor will it generalize for operations of other dialects. We need a general solution that can be applied to any operation. By default we would apply the solution to func ops extracted from classfiles. I think we need to assign a special value to `Op.result` that indicates it is bound as a root but `Op.parentBlock()` still returns `null`. ------------- PR Review: https://git.openjdk.org/babylon/pull/425#pullrequestreview-2837884766 From psandoz at openjdk.org Tue May 13 20:22:10 2025 From: psandoz at openjdk.org (Paul Sandoz) Date: Tue, 13 May 2025 20:22:10 GMT Subject: [code-reflection] RFR: Replace the use of Interpreter to create Quoted instance with specialized code In-Reply-To: References: Message-ID: On Mon, 12 May 2025 15:10:21 GMT, Mourad Abbay wrote: > Creating Quoted instance was done by invoking the Interpreter. The Interpreter is too general and if someone could somehow inject any code model we will interpret that as well. This PR replace the use of the interpreter with specialized code. We need to formalize this more. We could have two static factory methods: 1. `FuncOp CoreOp.quoteOp(Op l)` that transforms an op into a quoted func op. (There should be code around that already does this, but i have not checked where it is.) 2. `Quoted CoreOp.quotedOp(FuncOp f, Object... args)` that computes a `Quoted` instance from a quoted func op. Such that: LambdaOp l1 = ... FuncOp qlf1 = quoteOp(l1); Quoted q1 = quotedOp(qlf1, args); LambdaOp l2 = (LambdaOp) q1.op(); FuncOp qlf2 = quoteOp(l2); // l1 and l2 are structurally the same // qlf1 and qlf2 are structurally the same We need to clearly specify the model produced by `quoteOp` and therefore the validation requirements of `quoted` become clear. We might be able to present as a transformation to aid the specification. At least the following needs to be validated. - the func op's body contains one block consisting of, in order, zero or more var ops, a quoted op, and a return op - there are N var ops for N block parameters - each var op is uniquely initialized with a block parameter (possibly in order) - the return op returns the result of the quoted op - the quoted op's body contains one block - the block consist in order of the op that is quoted and a yield op yielding the result of that op to be quoted - the op to be quoted captures the results of all the var ops declared in the func op, and all such usages are for var load ops That is applicable to quoting of lambda ops and closure ops. ------------- PR Review: https://git.openjdk.org/babylon/pull/424#pullrequestreview-2838021289 From psandoz at openjdk.org Tue May 13 20:26:34 2025 From: psandoz at openjdk.org (Paul Sandoz) Date: Tue, 13 May 2025 20:26:34 GMT Subject: [code-reflection] RFR: Replace the use of Interpreter to create Quoted instance with specialized code In-Reply-To: References: Message-ID: On Mon, 12 May 2025 15:10:21 GMT, Mourad Abbay wrote: > Creating Quoted instance was done by invoking the Interpreter. The Interpreter is too general and if someone could somehow inject any code model we will interpret that as well. This PR replace the use of the interpreter with specialized code. src/jdk.incubator.code/share/classes/jdk/incubator/code/interpreter/Interpreter.java line 230: > 228: } > 229: > 230: public static final class VarBox You don't need to make this public. Look what it implements and provide your own implementation. Separately, we likely need to address how we expose a runtime value correspond to a symbolic `Var` value. We will encounter similar issues with tuples and funcs, so we need some general mapping i expect. This may become easier to identify as we refactor the code base more cleanly into dialects. ------------- PR Review Comment: https://git.openjdk.org/babylon/pull/424#discussion_r2087574913 From psandoz at openjdk.org Wed May 14 22:32:50 2025 From: psandoz at openjdk.org (Paul Sandoz) Date: Wed, 14 May 2025 22:32:50 GMT Subject: git: openjdk/babylon: code-reflection: Update JSON API Message-ID: <92b8dc27-965a-4e68-925e-929f0483bcba@openjdk.org> Changeset: 2870cdd5 Branch: code-reflection Author: Paul Sandoz Date: 2025-05-14 22:31:53 +0000 URL: https://git.openjdk.org/babylon/commit/2870cdd52f1aab1b0565d4caf48dfe82856a0c38 Update JSON API ! cr-examples/onnx/opgen/src/main/java/oracle/code/json/Json.java ! cr-examples/onnx/opgen/src/main/java/oracle/code/json/JsonArray.java - cr-examples/onnx/opgen/src/main/java/oracle/code/json/JsonArrayImpl.java ! cr-examples/onnx/opgen/src/main/java/oracle/code/json/JsonBoolean.java - cr-examples/onnx/opgen/src/main/java/oracle/code/json/JsonBooleanImpl.java - cr-examples/onnx/opgen/src/main/java/oracle/code/json/JsonDocumentInfo.java - cr-examples/onnx/opgen/src/main/java/oracle/code/json/JsonGenerator.java ! cr-examples/onnx/opgen/src/main/java/oracle/code/json/JsonNull.java - cr-examples/onnx/opgen/src/main/java/oracle/code/json/JsonNullImpl.java ! cr-examples/onnx/opgen/src/main/java/oracle/code/json/JsonNumber.java - cr-examples/onnx/opgen/src/main/java/oracle/code/json/JsonNumberImpl.java ! cr-examples/onnx/opgen/src/main/java/oracle/code/json/JsonObject.java - cr-examples/onnx/opgen/src/main/java/oracle/code/json/JsonObjectImpl.java ! cr-examples/onnx/opgen/src/main/java/oracle/code/json/JsonParseException.java - cr-examples/onnx/opgen/src/main/java/oracle/code/json/JsonParser.java ! cr-examples/onnx/opgen/src/main/java/oracle/code/json/JsonString.java - cr-examples/onnx/opgen/src/main/java/oracle/code/json/JsonStringImpl.java ! cr-examples/onnx/opgen/src/main/java/oracle/code/json/JsonValue.java - cr-examples/onnx/opgen/src/main/java/oracle/code/json/JsonValueImpl.java + cr-examples/onnx/opgen/src/main/java/oracle/code/json/impl/JsonArrayImpl.java + cr-examples/onnx/opgen/src/main/java/oracle/code/json/impl/JsonBooleanImpl.java + cr-examples/onnx/opgen/src/main/java/oracle/code/json/impl/JsonNullImpl.java + cr-examples/onnx/opgen/src/main/java/oracle/code/json/impl/JsonNumberImpl.java + cr-examples/onnx/opgen/src/main/java/oracle/code/json/impl/JsonObjectImpl.java + cr-examples/onnx/opgen/src/main/java/oracle/code/json/impl/JsonParser.java + cr-examples/onnx/opgen/src/main/java/oracle/code/json/impl/JsonStringImpl.java + cr-examples/onnx/opgen/src/main/java/oracle/code/json/impl/Utils.java ! cr-examples/onnx/opgen/src/main/java/oracle/code/onnx/opgen/OpSchemaParser.java ! cr-examples/onnx/src/main/java/oracle/code/onnx/OnnxOperators.java From psandoz at openjdk.org Wed May 14 22:35:42 2025 From: psandoz at openjdk.org (Paul Sandoz) Date: Wed, 14 May 2025 22:35:42 GMT Subject: [code-reflection] Integrated: Update JSON API Message-ID: Update JSON API ------------- Commit messages: - Update JSON API. Changes: https://git.openjdk.org/babylon/pull/427/files Webrev: https://webrevs.openjdk.org/?repo=babylon&pr=427&range=00 Stats: 3299 lines in 29 files changed: 1613 ins; 1570 del; 116 mod Patch: https://git.openjdk.org/babylon/pull/427.diff Fetch: git fetch https://git.openjdk.org/babylon.git pull/427/head:pull/427 PR: https://git.openjdk.org/babylon/pull/427 From psandoz at openjdk.org Wed May 14 22:35:45 2025 From: psandoz at openjdk.org (Paul Sandoz) Date: Wed, 14 May 2025 22:35:45 GMT Subject: [code-reflection] Integrated: Update JSON API In-Reply-To: References: Message-ID: On Wed, 14 May 2025 22:28:55 GMT, Paul Sandoz wrote: > Update JSON API This pull request has now been integrated. Changeset: 2870cdd5 Author: Paul Sandoz URL: https://git.openjdk.org/babylon/commit/2870cdd52f1aab1b0565d4caf48dfe82856a0c38 Stats: 3299 lines in 29 files changed: 1613 ins; 1570 del; 116 mod Update JSON API ------------- PR: https://git.openjdk.org/babylon/pull/427 From gfrost at openjdk.org Fri May 16 11:50:19 2025 From: gfrost at openjdk.org (Gary Frost) Date: Fri, 16 May 2025 11:50:19 GMT Subject: git: openjdk/babylon: code-reflection: Add rocm/hip cmake rules - hip backend work in progress Message-ID: <9bdc5364-e57d-4e49-aac4-0e0278c3ae6d@openjdk.org> Changeset: 75fdb9c3 Branch: code-reflection Author: Gary Frost Date: 2025-05-16 11:48:47 +0000 URL: https://git.openjdk.org/babylon/commit/75fdb9c3cc92df47ccf5acd36e9d5ecaa718b170 Add rocm/hip cmake rules - hip backend work in progress ! hat/backends/ffi/CMakeLists.txt ! hat/backends/ffi/cuda/include/cuda_backend.h ! hat/backends/ffi/hip/CMakeLists.txt ! hat/backends/ffi/hip/include/hip_backend.h From gfrost at openjdk.org Fri May 16 11:51:43 2025 From: gfrost at openjdk.org (Gary Frost) Date: Fri, 16 May 2025 11:51:43 GMT Subject: [code-reflection] Integrated: Add rocm/hip cmake rules - hip backend work in progress Message-ID: <0Gfw8HqIEQl5w8gF4Dbo2Yv7KYYTcrOJuyssVvWjkhQ=.dc6d3505-2e22-4bb3-8250-18caad1d354b@github.com> Trying to get the HIPBackend to build. Discovered that rocm/hip includes a 'FindHIP' so added that here.. Still working on getting the backend to build. ------------- Commit messages: - Add rocm/hip cmake rules - hip backend work in progress Changes: https://git.openjdk.org/babylon/pull/430/files Webrev: https://webrevs.openjdk.org/?repo=babylon&pr=430&range=00 Stats: 156 lines in 4 files changed: 106 ins; 17 del; 33 mod Patch: https://git.openjdk.org/babylon/pull/430.diff Fetch: git fetch https://git.openjdk.org/babylon.git pull/430/head:pull/430 PR: https://git.openjdk.org/babylon/pull/430 From gfrost at openjdk.org Fri May 16 11:51:43 2025 From: gfrost at openjdk.org (Gary Frost) Date: Fri, 16 May 2025 11:51:43 GMT Subject: [code-reflection] Integrated: Add rocm/hip cmake rules - hip backend work in progress In-Reply-To: <0Gfw8HqIEQl5w8gF4Dbo2Yv7KYYTcrOJuyssVvWjkhQ=.dc6d3505-2e22-4bb3-8250-18caad1d354b@github.com> References: <0Gfw8HqIEQl5w8gF4Dbo2Yv7KYYTcrOJuyssVvWjkhQ=.dc6d3505-2e22-4bb3-8250-18caad1d354b@github.com> Message-ID: <6uY23e9dw0UeSjYrdvOAe3HBKTEnkr3_Rl9lb_N_MsI=.9d83ad2e-3712-4176-a5a1-fd819fa1d321@github.com> On Fri, 16 May 2025 11:46:32 GMT, Gary Frost wrote: > Trying to get the HIPBackend to build. > > Discovered that rocm/hip includes a 'FindHIP' so added that here.. Still working on getting the backend to build. This pull request has now been integrated. Changeset: 75fdb9c3 Author: Gary Frost URL: https://git.openjdk.org/babylon/commit/75fdb9c3cc92df47ccf5acd36e9d5ecaa718b170 Stats: 156 lines in 4 files changed: 106 ins; 17 del; 33 mod Add rocm/hip cmake rules - hip backend work in progress ------------- PR: https://git.openjdk.org/babylon/pull/430 From mabbay at openjdk.org Tue May 20 09:24:51 2025 From: mabbay at openjdk.org (Mourad Abbay) Date: Tue, 20 May 2025 09:24:51 GMT Subject: [code-reflection] RFR: Replace the use of Interpreter to create Quoted instance with specialized code [v2] In-Reply-To: References: Message-ID: > Creating Quoted instance was done by invoking the Interpreter. The Interpreter is too general and if someone could somehow inject any code model we will interpret that as well. This PR replace the use of the interpreter with specialized code. Mourad Abbay has updated the pull request incrementally with one additional commit since the last revision: Implement quoteOp method ------------- Changes: - all: https://git.openjdk.org/babylon/pull/424/files - new: https://git.openjdk.org/babylon/pull/424/files/96197901..634799b8 Webrevs: - full: https://webrevs.openjdk.org/?repo=babylon&pr=424&range=01 - incr: https://webrevs.openjdk.org/?repo=babylon&pr=424&range=00-01 Stats: 68 lines in 2 files changed: 68 ins; 0 del; 0 mod Patch: https://git.openjdk.org/babylon/pull/424.diff Fetch: git fetch https://git.openjdk.org/babylon.git pull/424/head:pull/424 PR: https://git.openjdk.org/babylon/pull/424 From mabbay at openjdk.org Tue May 20 10:06:00 2025 From: mabbay at openjdk.org (Mourad Abbay) Date: Tue, 20 May 2025 10:06:00 GMT Subject: [code-reflection] RFR: Replace the use of Interpreter to create Quoted instance with specialized code [v3] In-Reply-To: References: Message-ID: <5X5BVXL3hx0fQiB50lZOsgP2Fn3APgXZosO9RbmznDs=.f7b6973d-29fa-4a14-aba8-ec97aaa0d83b@github.com> > Creating Quoted instance was done by invoking the Interpreter. The Interpreter is too general and if someone could somehow inject any code model we will interpret that as well. This PR replace the use of the interpreter with specialized code. Mourad Abbay has updated the pull request incrementally with one additional commit since the last revision: Uniformly deal with captured values and operands ------------- Changes: - all: https://git.openjdk.org/babylon/pull/424/files - new: https://git.openjdk.org/babylon/pull/424/files/634799b8..fbef70f1 Webrevs: - full: https://webrevs.openjdk.org/?repo=babylon&pr=424&range=02 - incr: https://webrevs.openjdk.org/?repo=babylon&pr=424&range=01-02 Stats: 26 lines in 1 file changed: 21 ins; 1 del; 4 mod Patch: https://git.openjdk.org/babylon/pull/424.diff Fetch: git fetch https://git.openjdk.org/babylon.git pull/424/head:pull/424 PR: https://git.openjdk.org/babylon/pull/424 From mcimadamore at openjdk.org Wed May 21 11:37:54 2025 From: mcimadamore at openjdk.org (Maurizio Cimadamore) Date: Wed, 21 May 2025 11:37:54 GMT Subject: [code-reflection] RFR: Regularize support for Java types/references Message-ID: This PR regularizes the support for Java types in the code model. There are two competing constraints to balance: on the one hand, the Java types should be readable from humans when inspecting the textual form of code model; on the other hand, we don't want to bake in Java-specific type parsing in the core code model API. Unfortunately, the current implementation scores poorly on both aspects, as (a) textual Java types can become very hard to read (e.g. `.<., NewTest$B$C>`), and (b) the parsing logic (defined in `DescParser`) contains several ad-hoc extensions that are only there to support Java types. To address this, this PR introduces two different kinds of externalized type forms: *inflated* type forms and *flattened* type forms. An inflated type form closely follow the structure of the `JavaType` or `JavaRef` it models. For instance, the Java type `Foo` is modelled as follows: java.type.class(Foo, java.type.primitive(void), java.type.wildcard(EXTENDS, java.type.class(Bar, java.type.primitive(void)))) Inflated type forms can be *flattened* (using `JavaTypeUtils::flatten`), to derive a form that is more suitable for humans. For instance, the above inflated form is flattened as follows: java.type:"Foo" Conversely, flattened type forms can be *inflated* back (using `JavaTypeUtils::inflate`) to their original inflated form. This distinction between inflated and flattened forms allow us to massively simplify `DescParser` -- which no longer has to worries about the syntactic vagaries of Java types. All that complexity is now pushed onto the flattened type forms support. The goal is to progressively make flattened Java type forms an "implementation detail" (more on that below). To accommodate flattened and inflated type forms, two changes are needed: * in `OpWriter` we need to flatten an inflated type form (if possible) before writing it out as a string (this allows for the textual form of a code model to be more readable) * in `OpParser` we need to inflate a flattened type form (if possible) before handing the type form back to the rest of the parsing machinery (which will e.g. invoke the type factory on such inflated type form) All Java types and references now follow this pattern: 1. the `externalize` method returns an inflated type form 2. the `toString` method returns a readable string (the one associated with the corresponding flat type form) 3. the `ofString` factory parses the flat string into an inflated form, then turn that inflated form into either a `JavaType` or `JavaRef`, as needed Note that (3) is the only reason as to why the flattened type forms surface into the API (since the factories are public, we would need to specify what strings they accept). Crucially, these factories are only used when parsing op attributes -- where Java types and refs are just modeled as plain flat strings -- e.g.: %6 : java.type:"int" = invoke %5 @"java.lang.Object::hashCode():int"; Note how the Java method reference after `@` is encoded as a plain string, which is then parser by the `InvokeOp` factory using `MethodRef::ofString`. #### Implementation This PR moves all the support for inflated and flattened Java type forms into a single class, namely `JavaTypeUtils`. This class contains factories to create the various inflated forms, as well as method to parse them, to turn them into flat strings, or to turn them into `JavaType` or `JavaRef` objects. The code in this class is a bit tedious, as I wanted to express all the functionality we needed in terms of externalized type forms. This allows us to reduce the impact of Java types on the rest of the code model (e.g. the changes to `OpParser` and `OpWriter` are literally one liners -- at the cost of making the implementation of some of these methods slightly more convoluted. For instance, it would have been easier to define a mapping from `JavaType` into a flat string (as we could leverage the `toString` method on the various `JavaType` subclasses). But doing so would create some issues, as now `OpWriter` would need to sometimes call `externalize` on the type, sometimes call `toString` on it -- similar idiosyncrasies would show up on the `OpParser` side. #### Future work The next step in this journey (not addressed by this PR) would be to parse Java type/ref attributes in a more structural fashion, so that we can let go of the `ofString` factories. And, after that, another possible improvement would be to augment and generalize the grammar supported by `DescParser` to cover more interesting structural forms -- something like: node: ident '<' node* '>' literal This would allow us to model all externalized type forms as more general *nodes* -- but will also allow us to model other kinds of interesting structure more directly, such as location information, Java literals and, possibly, even Java annotations. ------------- Commit messages: - Fix JavaType grammar - Drop redundant field - Add Javadoc to `JavaTypeUtils` - Cleanup exType parser - Cleanup and strengthen code that tests for the specific kind of externalized type - Remove parens from method type var external type strings - All tests pass - Drop Java-type specific code in ExternalizedTypeElement - Fix ClassType::rawType() to take enclosing type into account - Re-enable all the test cases in `TestJavaType` - ... and 6 more: https://git.openjdk.org/babylon/compare/2b0d5a2d...b76890dc Changes: https://git.openjdk.org/babylon/pull/432/files Webrev: https://webrevs.openjdk.org/?repo=babylon&pr=432&range=00 Stats: 7361 lines in 72 files changed: 840 ins; 450 del; 6071 mod Patch: https://git.openjdk.org/babylon/pull/432.diff Fetch: git fetch https://git.openjdk.org/babylon.git pull/432/head:pull/432 PR: https://git.openjdk.org/babylon/pull/432 From mcimadamore at openjdk.org Wed May 21 11:37:59 2025 From: mcimadamore at openjdk.org (Maurizio Cimadamore) Date: Wed, 21 May 2025 11:37:59 GMT Subject: [code-reflection] RFR: Regularize support for Java types/references In-Reply-To: References: Message-ID: On Wed, 21 May 2025 10:00:12 GMT, Maurizio Cimadamore wrote: > This PR regularizes the support for Java types in the code model. There are two competing constraints to balance: on the one hand, the Java types should be readable from humans when inspecting the textual form of code model; on the other hand, we don't want to bake in Java-specific type parsing in the core code model API. > > Unfortunately, the current implementation scores poorly on both aspects, as (a) textual Java types can become very hard to read (e.g. `.<., NewTest$B$C>`), and (b) the parsing logic (defined in `DescParser`) contains several ad-hoc extensions that are only there to support Java types. > > To address this, this PR introduces two different kinds of externalized type forms: *inflated* type forms and *flattened* type forms. An inflated type form closely follow the structure of the `JavaType` or `JavaRef` it models. For instance, the Java type `Foo` is modelled as follows: > > > java.type.class(Foo, java.type.primitive(void), > java.type.wildcard(EXTENDS, > java.type.class(Bar, java.type.primitive(void)))) > > > Inflated type forms can be *flattened* (using `JavaTypeUtils::flatten`), to derive a form that is more suitable for humans. For instance, the above inflated form is flattened as follows: > > > java.type:"Foo" > > Conversely, flattened type forms can be *inflated* back (using `JavaTypeUtils::inflate`) to their original inflated form. > > This distinction between inflated and flattened forms allow us to massively simplify `DescParser` -- which no longer has to worries about the syntactic vagaries of Java types. All that complexity is now pushed onto the flattened type forms support. The goal is to progressively make flattened Java type forms an "implementation detail" (more on that below). > > To accommodate flattened and inflated type forms, two changes are needed: > * in `OpWriter` we need to flatten an inflated type form (if possible) before writing it out as a string (this allows for the textual form of a code model to be more readable) > * in `OpParser` we need to inflate a flattened type form (if possible) before handing the type form back to the rest of the parsing machinery (which will e.g. invoke the type factory on such inflated type form) > > All Java types and references now follow this pattern: > 1. the `externalize` method returns an inflated type form > 2. the `toString` method returns a readable string (the one associated with the corresponding flat type ... src/jdk.incubator.code/share/classes/jdk/incubator/code/TypeElement.java line 48: > 46: } > 47: > 48: // Unpack array-like identifier [+ This code was removed, as it was specific to Java types src/jdk.incubator.code/share/classes/jdk/incubator/code/parser/OpParser.java line 127: > 125: public final class OpParser { > 126: > 127: static final TypeElement.ExternalizedTypeElement VOID = JavaType.VOID.externalize(); This is a workaround: the parser uses a private void type -- but such a type is not handled by either the core type factory or the Java type factory. So injecting such type into the parsed model results into issues. src/jdk.incubator.code/share/classes/jdk/incubator/code/parser/OpParser.java line 606: > 604: > 605: TypeElement.ExternalizedTypeElement parseExTypeElem() { > 606: return JavaTypeUtils.inflate(DescParser.parseExTypeElem(lexer)); When parsing an externalized type form, inflate it fist (from its readable form, if it's a Java type). src/jdk.incubator.code/share/classes/jdk/incubator/code/parser/impl/DescParser.java line 48: > 46: } > 47: > 48: // ExType: Note that the grammar here is very simple, as we no longer have to have support for Java types in here. An externalized type form is a type identifier, followed by zero or more externalized type arguments. An identifier is a qualified identifier -- each part in the identifier can be either string literals or identifiers. Various parts can be separated either with `.` or `:`. src/jdk.incubator.code/share/classes/jdk/incubator/code/type/ArrayType.java line 82: > 80: @Override > 81: public ExternalizedTypeElement externalize() { > 82: return JavaTypeUtils.arrayType(componentType.externalize()); This pattern is common to all types and refs: * the `externalize` method returns an inflated externalized form built with the corresponding factory in `JavaTypeUtils` * the `toString` method returns a flat string, obtained with the corresponding method in `JavaTypeUtils` * the `ofString` factory parses a flat string and turns it into an inflated form, which is then parsed into a `JavaType` or `JavaRef`, using corresponding methods in `JavaTypeUtils` src/jdk.incubator.code/share/classes/jdk/incubator/code/type/CoreTypeFactory.java line 100: > 98: @Override > 99: public TypeElement constructType(TypeElement.ExternalizedTypeElement tree) { > 100: return JavaTypeUtils.toJavaType(tree); Note how parsing an inflated type form is now done entirely inside `JavaTypeUtils`. I'm not entirely sure whether the Java type factory should support both Java type and reference inflated forms. For now I went with just types -- but if we want to push this further (e.g. to cover also op attributes) we would need the factory to support both types and refs (which is a trivial change to do). src/jdk.incubator.code/share/classes/jdk/incubator/code/type/impl/JavaTypeUtils.java line 655: > 653: } > 654: > 655: // JavaRef: Note: the grammar of ref and types is partially overlapping -- for this reason it's slightly easier to keep the two parsing methods separate. If we wanted to unify more, we could require all refs to be prefixed with a `&` which would simplify things a bit (and also be compatible with what we do to disambiguate between class and method type-variable owner). src/jdk.incubator.code/share/classes/jdk/incubator/code/writer/OpWriter.java line 561: > 559: > 560: void writeType(TypeElement te) { > 561: write(JavaTypeUtils.flatten(te.externalize()).toString()); When writing a type element into textual form, flatten it first (to make it more readable if it's a Java type). test/langtools/tools/javac/reflect/CodeReflectionTester.java line 72: > 70: } > 71: > 72: static void checkModel(Member member, String found, IR ir) { This new method allows the harness to print the model we found on the compiled class even in the face of an error when parsing the `@IR` annotation. This is very handy when bulk-updating the tests, as it means that parser errors will cause the harness to just dump the "expected" model for all the methods in the class (instead of throwing on the first parser error). test/langtools/tools/javac/reflect/ForLoopTest.java line 42: > 40: %2 : Var>"> = var %1 @"ll"; > 41: java.enhancedFor > 42: ()java.type:"java.util.List>" -> { Here and elsewhere -- it seems like block names like `^expr` or `^def` are no longer generated. But this PR doesn't change that (in fact, I tried creating the model for a simple for loop without these changes, and I still don't see the body names). I believe this has to do with the fact that e.g. for ops are composed of _bodies_, and each body in the op has a single block (thus, it is the entry block for that body). And, entry block names are skipped. The test was probably still working fine (despite the mismatch) because we parse the IR string, then generate the textual form again before comparing it -- this will drop the entry block names -- meaning the "expected" IR will match the one generated by javac. ------------- PR Review Comment: https://git.openjdk.org/babylon/pull/432#discussion_r2099884559 PR Review Comment: https://git.openjdk.org/babylon/pull/432#discussion_r2099888570 PR Review Comment: https://git.openjdk.org/babylon/pull/432#discussion_r2099894899 PR Review Comment: https://git.openjdk.org/babylon/pull/432#discussion_r2099898042 PR Review Comment: https://git.openjdk.org/babylon/pull/432#discussion_r2099902238 PR Review Comment: https://git.openjdk.org/babylon/pull/432#discussion_r2100046974 PR Review Comment: https://git.openjdk.org/babylon/pull/432#discussion_r2100043265 PR Review Comment: https://git.openjdk.org/babylon/pull/432#discussion_r2099893171 PR Review Comment: https://git.openjdk.org/babylon/pull/432#discussion_r2099890681 PR Review Comment: https://git.openjdk.org/babylon/pull/432#discussion_r2099974036 From mcimadamore at openjdk.org Wed May 21 14:35:08 2025 From: mcimadamore at openjdk.org (Maurizio Cimadamore) Date: Wed, 21 May 2025 14:35:08 GMT Subject: [code-reflection] RFR: Regularize support for Java types/references In-Reply-To: References: Message-ID: On Wed, 21 May 2025 11:30:40 GMT, Maurizio Cimadamore wrote: >> This PR regularizes the support for Java types in the code model. There are two competing constraints to balance: on the one hand, the Java types should be readable from humans when inspecting the textual form of code model; on the other hand, we don't want to bake in Java-specific type parsing in the core code model API. >> >> Unfortunately, the current implementation scores poorly on both aspects, as (a) textual Java types can become very hard to read (e.g. `.<., NewTest$B$C>`), and (b) the parsing logic (defined in `DescParser`) contains several ad-hoc extensions that are only there to support Java types. >> >> To address this, this PR introduces two different kinds of externalized type forms: *inflated* type forms and *flattened* type forms. An inflated type form closely follow the structure of the `JavaType` or `JavaRef` it models. For instance, the Java type `Foo` is modelled as follows: >> >> >> java.type.class(Foo, java.type.primitive(void), >> java.type.wildcard(EXTENDS, >> java.type.class(Bar, java.type.primitive(void)))) >> >> >> Inflated type forms can be *flattened* (using `JavaTypeUtils::flatten`), to derive a form that is more suitable for humans. For instance, the above inflated form is flattened as follows: >> >> >> java.type:"Foo" >> >> Conversely, flattened type forms can be *inflated* back (using `JavaTypeUtils::inflate`) to their original inflated form. >> >> This distinction between inflated and flattened forms allow us to massively simplify `DescParser` -- which no longer has to worries about the syntactic vagaries of Java types. All that complexity is now pushed onto the flattened type forms support. The goal is to progressively make flattened Java type forms an "implementation detail" (more on that below). >> >> To accommodate flattened and inflated type forms, two changes are needed: >> * in `OpWriter` we need to flatten an inflated type form (if possible) before writing it out as a string (this allows for the textual form of a code model to be more readable) >> * in `OpParser` we need to inflate a flattened type form (if possible) before handing the type form back to the rest of the parsing machinery (which will e.g. invoke the type factory on such inflated type form) >> >> All Java types and references now follow this pattern: >> 1. the `externalize` method returns an inflated type form >> 2. the `toString` method returns a readable stri... > > src/jdk.incubator.code/share/classes/jdk/incubator/code/type/impl/JavaTypeUtils.java line 655: > >> 653: } >> 654: >> 655: // JavaRef: > > Note: the grammar of ref and types is partially overlapping -- for this reason it's slightly easier to keep the two parsing methods separate. If we wanted to unify more, we could require all refs to be prefixed with a `&` which would simplify things a bit (and also be compatible with what we do to disambiguate between class and method type-variable owner). Also, note: the grammar for method ref changed, as this now requires the type separator `:` to be used between the params and the return type (in a way that is more similar to what's done for field refs). If we don't want to have this incompatibility we can roll back. ------------- PR Review Comment: https://git.openjdk.org/babylon/pull/432#discussion_r2100456761 From mcimadamore at openjdk.org Wed May 21 15:23:11 2025 From: mcimadamore at openjdk.org (Maurizio Cimadamore) Date: Wed, 21 May 2025 15:23:11 GMT Subject: [code-reflection] RFR: Regularize support for Java types/references In-Reply-To: References: Message-ID: On Wed, 21 May 2025 10:00:12 GMT, Maurizio Cimadamore wrote: > The next step in this journey (not addressed by this PR) would be to parse Java type/ref attributes in a more structural fashion, so that we can let go of the `ofString` factories. The patch below contains a sketch of how we might go in that direction: https://github.com/mcimadamore/babylon/compare/externalized_Java_type_cleanup...mcimadamore:babylon:remove_of_string_type_factories?expand=1 The fact the I was able to cook it in an hour or so makes me confident about the general direction set by this PR. ------------- PR Comment: https://git.openjdk.org/babylon/pull/432#issuecomment-2898354029 From mcimadamore at openjdk.org Wed May 21 15:42:30 2025 From: mcimadamore at openjdk.org (Maurizio Cimadamore) Date: Wed, 21 May 2025 15:42:30 GMT Subject: [code-reflection] RFR: Regularize support for Java types/references [v2] In-Reply-To: References: Message-ID: <2ofYQ2zcEP3Kc3IPNfp0Rmvoz11uWV_Jb3wNHGfbcls=.fa20cc1a-f974-4dfd-ba84-fdce7735a6ce@github.com> > This PR regularizes the support for Java types in the code model. There are two competing constraints to balance: on the one hand, the Java types should be readable from humans when inspecting the textual form of code model; on the other hand, we don't want to bake in Java-specific type parsing in the core code model API. > > Unfortunately, the current implementation scores poorly on both aspects, as (a) textual Java types can become very hard to read (e.g. `.<., NewTest$B$C>`), and (b) the parsing logic (defined in `DescParser`) contains several ad-hoc extensions that are only there to support Java types. > > To address this, this PR introduces two different kinds of externalized type forms: *inflated* type forms and *flattened* type forms. An inflated type form closely follow the structure of the `JavaType` or `JavaRef` it models. For instance, the Java type `Foo` is modelled as follows: > > > java.type.class(Foo, java.type.primitive(void), > java.type.wildcard(EXTENDS, > java.type.class(Bar, java.type.primitive(void)))) > > > Inflated type forms can be *flattened* (using `JavaTypeUtils::flatten`), to derive a form that is more suitable for humans. For instance, the above inflated form is flattened as follows: > > > java.type:"Foo" > > Conversely, flattened type forms can be *inflated* back (using `JavaTypeUtils::inflate`) to their original inflated form. > > This distinction between inflated and flattened forms allow us to massively simplify `DescParser` -- which no longer has to worries about the syntactic vagaries of Java types. All that complexity is now pushed onto the flattened type forms support. The goal is to progressively make flattened Java type forms an "implementation detail" (more on that below). > > To accommodate flattened and inflated type forms, two changes are needed: > * in `OpWriter` we need to flatten an inflated type form (if possible) before writing it out as a string (this allows for the textual form of a code model to be more readable) > * in `OpParser` we need to inflate a flattened type form (if possible) before handing the type form back to the rest of the parsing machinery (which will e.g. invoke the type factory on such inflated type form) > > All Java types and references now follow this pattern: > 1. the `externalize` method returns an inflated type form > 2. the `toString` method returns a readable string (the one associated with the corresponding flat type ... Maurizio Cimadamore has updated the pull request incrementally with one additional commit since the last revision: Simplify parser code to use `Lexer::is` ------------- Changes: - all: https://git.openjdk.org/babylon/pull/432/files - new: https://git.openjdk.org/babylon/pull/432/files/b76890dc..cf48f74c Webrevs: - full: https://webrevs.openjdk.org/?repo=babylon&pr=432&range=01 - incr: https://webrevs.openjdk.org/?repo=babylon&pr=432&range=00-01 Stats: 13 lines in 2 files changed: 0 ins; 0 del; 13 mod Patch: https://git.openjdk.org/babylon/pull/432.diff Fetch: git fetch https://git.openjdk.org/babylon.git pull/432/head:pull/432 PR: https://git.openjdk.org/babylon/pull/432 From mabbay at openjdk.org Thu May 22 02:50:07 2025 From: mabbay at openjdk.org (Mourad Abbay) Date: Thu, 22 May 2025 02:50:07 GMT Subject: [code-reflection] RFR: Replace the use of Interpreter to create Quoted instance with specialized code [v4] In-Reply-To: References: Message-ID: > Creating Quoted instance was done by invoking the Interpreter. The Interpreter is too general and if someone could somehow inject any code model we will interpret that as well. This PR replace the use of the interpreter with specialized code. Mourad Abbay has updated the pull request incrementally with one additional commit since the last revision: Implement quotedOp method ------------- Changes: - all: https://git.openjdk.org/babylon/pull/424/files - new: https://git.openjdk.org/babylon/pull/424/files/fbef70f1..ca090586 Webrevs: - full: https://webrevs.openjdk.org/?repo=babylon&pr=424&range=03 - incr: https://webrevs.openjdk.org/?repo=babylon&pr=424&range=02-03 Stats: 36 lines in 2 files changed: 36 ins; 0 del; 0 mod Patch: https://git.openjdk.org/babylon/pull/424.diff Fetch: git fetch https://git.openjdk.org/babylon.git pull/424/head:pull/424 PR: https://git.openjdk.org/babylon/pull/424 From mcimadamore at openjdk.org Thu May 22 10:49:10 2025 From: mcimadamore at openjdk.org (Maurizio Cimadamore) Date: Thu, 22 May 2025 10:49:10 GMT Subject: [code-reflection] RFR: Replace the use of Interpreter to create Quoted instance with specialized code [v4] In-Reply-To: References: Message-ID: On Thu, 22 May 2025 02:50:07 GMT, Mourad Abbay wrote: >> Creating Quoted instance was done by invoking the Interpreter. The Interpreter is too general and if someone could somehow inject any code model we will interpret that as well. This PR replace the use of the interpreter with specialized code. > > Mourad Abbay has updated the pull request incrementally with one additional commit since the last revision: > > Implement quotedOp method src/jdk.incubator.code/share/classes/jdk/incubator/code/internal/QuotedHelper.java line 39: > 37: public class QuotedHelper { > 38: > 39: public static Quoted makeQuoted(FuncOp op, Object[] args) { Shouldn't this use the new helper method you've defined? And shouldn't javac use the other helper method to create the FuncOp in the first place? ------------- PR Review Comment: https://git.openjdk.org/babylon/pull/424#discussion_r2102252452 From mcimadamore at openjdk.org Thu May 22 10:52:17 2025 From: mcimadamore at openjdk.org (Maurizio Cimadamore) Date: Thu, 22 May 2025 10:52:17 GMT Subject: [code-reflection] RFR: Replace the use of Interpreter to create Quoted instance with specialized code [v4] In-Reply-To: References: Message-ID: On Thu, 22 May 2025 02:50:07 GMT, Mourad Abbay wrote: >> Creating Quoted instance was done by invoking the Interpreter. The Interpreter is too general and if someone could somehow inject any code model we will interpret that as well. This PR replace the use of the interpreter with specialized code. > > Mourad Abbay has updated the pull request incrementally with one additional commit since the last revision: > > Implement quotedOp method src/jdk.incubator.code/share/classes/jdk/incubator/code/op/CoreOp.java line 4495: > 4493: public static Quoted quotedOp(FuncOp funcOp, Object[] args) { > 4494: > 4495: assert funcOp.body().blocks().size() == 1; I believe the checks you enforce are correct -- but leaving these checks as plain `assert` feel odd -- e.g. this is the contract of an API method, so the contract of such method should be fully specified, and real exceptions must be thrown. With `assert`, the issues will only be reported if the user enables assertion when running their code (actually worse, since these assertions are inside JDK, they would need to enable "system assertions" -- which is to say that they will never see this code fail in the way you intended). ------------- PR Review Comment: https://git.openjdk.org/babylon/pull/424#discussion_r2102257549 From mcimadamore at openjdk.org Thu May 22 11:17:21 2025 From: mcimadamore at openjdk.org (Maurizio Cimadamore) Date: Thu, 22 May 2025 11:17:21 GMT Subject: [code-reflection] RFR: Replace the use of Interpreter to create Quoted instance with specialized code [v4] In-Reply-To: References: Message-ID: On Thu, 22 May 2025 02:50:07 GMT, Mourad Abbay wrote: >> Creating Quoted instance was done by invoking the Interpreter. The Interpreter is too general and if someone could somehow inject any code model we will interpret that as well. This PR replace the use of the interpreter with specialized code. > > Mourad Abbay has updated the pull request incrementally with one additional commit since the last revision: > > Implement quotedOp method src/jdk.incubator.code/share/classes/jdk/incubator/code/op/CoreOp.java line 4478: > 4476: b.op(o); > 4477: > 4478: for (Value k : m.keySet()) { Why is this loop needed? Is this to restore the original mapping in the copy context? If so, perhaps the code could be strengthened by surrounding `b.op` with a `try/finally` and then restore the mapping in the finally block? (But I wonder if the API could offer more help here -- e.g. some kind of temp value mapping) ------------- PR Review Comment: https://git.openjdk.org/babylon/pull/424#discussion_r2102298167 From mcimadamore at openjdk.org Thu May 22 11:17:21 2025 From: mcimadamore at openjdk.org (Maurizio Cimadamore) Date: Thu, 22 May 2025 11:17:21 GMT Subject: [code-reflection] RFR: Replace the use of Interpreter to create Quoted instance with specialized code [v4] In-Reply-To: References: Message-ID: On Thu, 22 May 2025 11:12:56 GMT, Maurizio Cimadamore wrote: >> Mourad Abbay has updated the pull request incrementally with one additional commit since the last revision: >> >> Implement quotedOp method > > src/jdk.incubator.code/share/classes/jdk/incubator/code/op/CoreOp.java line 4478: > >> 4476: b.op(o); >> 4477: >> 4478: for (Value k : m.keySet()) { > > Why is this loop needed? Is this to restore the original mapping in the copy context? If so, perhaps the code could be strengthened by surrounding `b.op` with a `try/finally` and then restore the mapping in the finally block? > > (But I wonder if the API could offer more help here -- e.g. some kind of temp value mapping) Iterating on `m.entrySet` would allow to skip the extra call to `m.get` ? (an overload in `CopyContext::mapValues` that also accepts a `Map` also seems fair game here) ------------- PR Review Comment: https://git.openjdk.org/babylon/pull/424#discussion_r2102299413 From mcimadamore at openjdk.org Thu May 22 17:33:36 2025 From: mcimadamore at openjdk.org (Maurizio Cimadamore) Date: Thu, 22 May 2025 17:33:36 GMT Subject: [code-reflection] RFR: Regularize support for Java types/references [v3] In-Reply-To: References: Message-ID: > This PR regularizes the support for Java types in the code model. There are two competing constraints to balance: on the one hand, the Java types should be readable from humans when inspecting the textual form of code model; on the other hand, we don't want to bake in Java-specific type parsing in the core code model API. > > Unfortunately, the current implementation scores poorly on both aspects, as (a) textual Java types can become very hard to read (e.g. `.<., NewTest$B$C>`), and (b) the parsing logic (defined in `DescParser`) contains several ad-hoc extensions that are only there to support Java types. > > To address this, this PR introduces two different kinds of externalized type forms: *inflated* type forms and *flattened* type forms. An inflated type form closely follow the structure of the `JavaType` or `JavaRef` it models. For instance, the Java type `Foo` is modelled as follows: > > > java.type.class(Foo, java.type.primitive(void), > java.type.wildcard(EXTENDS, > java.type.class(Bar, java.type.primitive(void)))) > > > Inflated type forms can be *flattened* (using `JavaTypeUtils::flatten`), to derive a form that is more suitable for humans. For instance, the above inflated form is flattened as follows: > > > java.type:"Foo" > > Conversely, flattened type forms can be *inflated* back (using `JavaTypeUtils::inflate`) to their original inflated form. > > This distinction between inflated and flattened forms allow us to massively simplify `DescParser` -- which no longer has to worries about the syntactic vagaries of Java types. All that complexity is now pushed onto the flattened type forms support. The goal is to progressively make flattened Java type forms an "implementation detail" (more on that below). > > To accommodate flattened and inflated type forms, two changes are needed: > * in `OpWriter` we need to flatten an inflated type form (if possible) before writing it out as a string (this allows for the textual form of a code model to be more readable) > * in `OpParser` we need to inflate a flattened type form (if possible) before handing the type form back to the rest of the parsing machinery (which will e.g. invoke the type factory on such inflated type form) > > All Java types and references now follow this pattern: > 1. the `externalize` method returns an inflated type form > 2. the `toString` method returns a readable string (the one associated with the corresponding flat type ... Maurizio Cimadamore has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains 19 additional commits since the last revision: - Enhance type factory to create refs - Merge branch 'code-reflection' into externalized_Java_type_cleanup - Simplify parser code to use `Lexer::is` - Fix JavaType grammar - Drop redundant field - Add Javadoc to `JavaTypeUtils` - Cleanup exType parser - Cleanup and strengthen code that tests for the specific kind of externalized type - Remove parens from method type var external type strings - All tests pass - ... and 9 more: https://git.openjdk.org/babylon/compare/d1e22c58...067d4371 ------------- Changes: - all: https://git.openjdk.org/babylon/pull/432/files - new: https://git.openjdk.org/babylon/pull/432/files/cf48f74c..067d4371 Webrevs: - full: https://webrevs.openjdk.org/?repo=babylon&pr=432&range=02 - incr: https://webrevs.openjdk.org/?repo=babylon&pr=432&range=01-02 Stats: 8122 lines in 58 files changed: 5141 ins; 2683 del; 298 mod Patch: https://git.openjdk.org/babylon/pull/432.diff Fetch: git fetch https://git.openjdk.org/babylon.git pull/432/head:pull/432 PR: https://git.openjdk.org/babylon/pull/432 From mcimadamore at openjdk.org Thu May 22 18:10:23 2025 From: mcimadamore at openjdk.org (Maurizio Cimadamore) Date: Thu, 22 May 2025 18:10:23 GMT Subject: [code-reflection] RFR: Regularize support for Java types/references [v4] In-Reply-To: References: Message-ID: > This PR regularizes the support for Java types in the code model. There are two competing constraints to balance: on the one hand, the Java types should be readable from humans when inspecting the textual form of code model; on the other hand, we don't want to bake in Java-specific type parsing in the core code model API. > > Unfortunately, the current implementation scores poorly on both aspects, as (a) textual Java types can become very hard to read (e.g. `.<., NewTest$B$C>`), and (b) the parsing logic (defined in `DescParser`) contains several ad-hoc extensions that are only there to support Java types. > > To address this, this PR introduces two different kinds of externalized type forms: *inflated* type forms and *flattened* type forms. An inflated type form closely follow the structure of the `JavaType` or `JavaRef` it models. For instance, the Java type `Foo` is modelled as follows: > > > java.type.class(Foo, java.type.primitive(void), > java.type.wildcard(EXTENDS, > java.type.class(Bar, java.type.primitive(void)))) > > > Inflated type forms can be *flattened* (using `JavaTypeUtils::flatten`), to derive a form that is more suitable for humans. For instance, the above inflated form is flattened as follows: > > > java.type:"Foo" > > Conversely, flattened type forms can be *inflated* back (using `JavaTypeUtils::inflate`) to their original inflated form. > > This distinction between inflated and flattened forms allow us to massively simplify `DescParser` -- which no longer has to worries about the syntactic vagaries of Java types. All that complexity is now pushed onto the flattened type forms support. The goal is to progressively make flattened Java type forms an "implementation detail" (more on that below). > > To accommodate flattened and inflated type forms, two changes are needed: > * in `OpWriter` we need to flatten an inflated type form (if possible) before writing it out as a string (this allows for the textual form of a code model to be more readable) > * in `OpParser` we need to inflate a flattened type form (if possible) before handing the type form back to the rest of the parsing machinery (which will e.g. invoke the type factory on such inflated type form) > > All Java types and references now follow this pattern: > 1. the `externalize` method returns an inflated type form > 2. the `toString` method returns a readable string (the one associated with the corresponding flat type ... Maurizio Cimadamore has updated the pull request incrementally with one additional commit since the last revision: Simplify JavaTypeUtils to use ExternalizedTypeElement factories ------------- Changes: - all: https://git.openjdk.org/babylon/pull/432/files - new: https://git.openjdk.org/babylon/pull/432/files/067d4371..8cc17354 Webrevs: - full: https://webrevs.openjdk.org/?repo=babylon&pr=432&range=03 - incr: https://webrevs.openjdk.org/?repo=babylon&pr=432&range=02-03 Stats: 19 lines in 1 file changed: 0 ins; 0 del; 19 mod Patch: https://git.openjdk.org/babylon/pull/432.diff Fetch: git fetch https://git.openjdk.org/babylon.git pull/432/head:pull/432 PR: https://git.openjdk.org/babylon/pull/432 From mabbay at openjdk.org Thu May 22 23:40:22 2025 From: mabbay at openjdk.org (Mourad Abbay) Date: Thu, 22 May 2025 23:40:22 GMT Subject: [code-reflection] RFR: Replace the use of Interpreter to create Quoted instance with specialized code [v5] In-Reply-To: References: Message-ID: <7IEAWEx_3-JfyFolkhZSYccpMK-j6c0E6HVmrL14bmU=.f9943b4c-f91f-466b-9e71-d9c8640c93f0@github.com> > Creating Quoted instance was done by invoking the Interpreter. The Interpreter is too general and if someone could somehow inject any code model we will interpret that as well. This PR replace the use of the interpreter with specialized code. Mourad Abbay has refreshed the contents of this pull request, and previous commits have been removed. The incremental views will show differences compared to the previous content of the PR. The pull request contains two new commits since the last revision: - Focus on captured values for the moment - Implement quotedOp method ------------- Changes: - all: https://git.openjdk.org/babylon/pull/424/files - new: https://git.openjdk.org/babylon/pull/424/files/ca090586..1bca3137 Webrevs: - full: https://webrevs.openjdk.org/?repo=babylon&pr=424&range=04 - incr: https://webrevs.openjdk.org/?repo=babylon&pr=424&range=03-04 Stats: 11 lines in 2 files changed: 5 ins; 1 del; 5 mod Patch: https://git.openjdk.org/babylon/pull/424.diff Fetch: git fetch https://git.openjdk.org/babylon.git pull/424/head:pull/424 PR: https://git.openjdk.org/babylon/pull/424 From psandoz at openjdk.org Thu May 22 23:55:05 2025 From: psandoz at openjdk.org (Paul Sandoz) Date: Thu, 22 May 2025 23:55:05 GMT Subject: [code-reflection] RFR: Replace the use of Interpreter to create Quoted instance with specialized code [v5] In-Reply-To: <7IEAWEx_3-JfyFolkhZSYccpMK-j6c0E6HVmrL14bmU=.f9943b4c-f91f-466b-9e71-d9c8640c93f0@github.com> References: <7IEAWEx_3-JfyFolkhZSYccpMK-j6c0E6HVmrL14bmU=.f9943b4c-f91f-466b-9e71-d9c8640c93f0@github.com> Message-ID: On Thu, 22 May 2025 23:40:22 GMT, Mourad Abbay wrote: >> Creating Quoted instance was done by invoking the Interpreter. The Interpreter is too general and if someone could somehow inject any code model we will interpret that as well. This PR replace the use of the interpreter with specialized code. > > Mourad Abbay has refreshed the contents of this pull request, and previous commits have been removed. The incremental views will show differences compared to the previous content of the PR. The pull request contains two new commits since the last revision: > > - Focus on captured values for the moment > - Implement quotedOp method src/jdk.incubator.code/share/classes/jdk/incubator/code/op/CoreOp.java line 4443: > 4441: return func("q", FunctionType.VOID).body(block -> { > 4442: > 4443: List capturedValues = op.capturedValues(); Don't forget operands, they can occur in order before any captured variables. ------------- PR Review Comment: https://git.openjdk.org/babylon/pull/424#discussion_r2103576263 From psandoz at openjdk.org Fri May 23 00:01:59 2025 From: psandoz at openjdk.org (Paul Sandoz) Date: Fri, 23 May 2025 00:01:59 GMT Subject: [code-reflection] RFR: Replace the use of Interpreter to create Quoted instance with specialized code [v5] In-Reply-To: <7IEAWEx_3-JfyFolkhZSYccpMK-j6c0E6HVmrL14bmU=.f9943b4c-f91f-466b-9e71-d9c8640c93f0@github.com> References: <7IEAWEx_3-JfyFolkhZSYccpMK-j6c0E6HVmrL14bmU=.f9943b4c-f91f-466b-9e71-d9c8640c93f0@github.com> Message-ID: On Thu, 22 May 2025 23:40:22 GMT, Mourad Abbay wrote: >> Creating Quoted instance was done by invoking the Interpreter. The Interpreter is too general and if someone could somehow inject any code model we will interpret that as well. This PR replace the use of the interpreter with specialized code. > > Mourad Abbay has refreshed the contents of this pull request, and previous commits have been removed. The incremental views will show differences compared to the previous content of the PR. The pull request contains two new commits since the last revision: > > - Focus on captured values for the moment > - Implement quotedOp method src/jdk.incubator.code/share/classes/jdk/incubator/code/op/CoreOp.java line 4439: > 4437: } > 4438: > 4439: public static FuncOp quoteOp(Op op) { See also the method `BytecodeGenerator::quote` it does the same thing specific to `LambdaOp`, and we can generalize to `Op` and then the bytecode generator can reuse the general one. ------------- PR Review Comment: https://git.openjdk.org/babylon/pull/424#discussion_r2103580823 From mabbay at openjdk.org Fri May 23 00:16:21 2025 From: mabbay at openjdk.org (Mourad Abbay) Date: Fri, 23 May 2025 00:16:21 GMT Subject: [code-reflection] RFR: Replace the use of Interpreter to create Quoted instance with specialized code [v6] In-Reply-To: References: Message-ID: > Creating Quoted instance was done by invoking the Interpreter. The Interpreter is too general and if someone could somehow inject any code model we will interpret that as well. This PR replace the use of the interpreter with specialized code. Mourad Abbay has updated the pull request incrementally with three additional commits since the last revision: - Add comments - Add a check - Change the return type of quoted func op ------------- Changes: - all: https://git.openjdk.org/babylon/pull/424/files - new: https://git.openjdk.org/babylon/pull/424/files/1bca3137..8d3a227b Webrevs: - full: https://webrevs.openjdk.org/?repo=babylon&pr=424&range=05 - incr: https://webrevs.openjdk.org/?repo=babylon&pr=424&range=04-05 Stats: 14 lines in 2 files changed: 9 ins; 2 del; 3 mod Patch: https://git.openjdk.org/babylon/pull/424.diff Fetch: git fetch https://git.openjdk.org/babylon.git pull/424/head:pull/424 PR: https://git.openjdk.org/babylon/pull/424 From mcimadamore at openjdk.org Fri May 23 10:24:10 2025 From: mcimadamore at openjdk.org (Maurizio Cimadamore) Date: Fri, 23 May 2025 10:24:10 GMT Subject: [code-reflection] RFR: Regularize support for Java types/references [v5] In-Reply-To: References: Message-ID: > This PR regularizes the support for Java types in the code model. There are two competing constraints to balance: on the one hand, the Java types should be readable from humans when inspecting the textual form of code model; on the other hand, we don't want to bake in Java-specific type parsing in the core code model API. > > Unfortunately, the current implementation scores poorly on both aspects, as (a) textual Java types can become very hard to read (e.g. `.<., NewTest$B$C>`), and (b) the parsing logic (defined in `DescParser`) contains several ad-hoc extensions that are only there to support Java types. > > To address this, this PR introduces two different kinds of externalized type forms: *inflated* type forms and *flattened* type forms. An inflated type form closely follow the structure of the `JavaType` or `JavaRef` it models. For instance, the Java type `Foo` is modelled as follows: > > > java.type.class(Foo, java.type.primitive(void), > java.type.wildcard(EXTENDS, > java.type.class(Bar, java.type.primitive(void)))) > > > Inflated type forms can be *flattened* (using `JavaTypeUtils::flatten`), to derive a form that is more suitable for humans. For instance, the above inflated form is flattened as follows: > > > java.type:"Foo" > > Conversely, flattened type forms can be *inflated* back (using `JavaTypeUtils::inflate`) to their original inflated form. > > This distinction between inflated and flattened forms allow us to massively simplify `DescParser` -- which no longer has to worries about the syntactic vagaries of Java types. All that complexity is now pushed onto the flattened type forms support. The goal is to progressively make flattened Java type forms an "implementation detail" (more on that below). > > To accommodate flattened and inflated type forms, two changes are needed: > * in `OpWriter` we need to flatten an inflated type form (if possible) before writing it out as a string (this allows for the textual form of a code model to be more readable) > * in `OpParser` we need to inflate a flattened type form (if possible) before handing the type form back to the rest of the parsing machinery (which will e.g. invoke the type factory on such inflated type form) > > All Java types and references now follow this pattern: > 1. the `externalize` method returns an inflated type form > 2. the `toString` method returns a readable string (the one associated with the corresponding flat type ... Maurizio Cimadamore has updated the pull request incrementally with one additional commit since the last revision: Fix examples ------------- Changes: - all: https://git.openjdk.org/babylon/pull/432/files - new: https://git.openjdk.org/babylon/pull/432/files/8cc17354..c89986c0 Webrevs: - full: https://webrevs.openjdk.org/?repo=babylon&pr=432&range=04 - incr: https://webrevs.openjdk.org/?repo=babylon&pr=432&range=03-04 Stats: 585 lines in 11 files changed: 6 ins; 0 del; 579 mod Patch: https://git.openjdk.org/babylon/pull/432.diff Fetch: git fetch https://git.openjdk.org/babylon.git pull/432/head:pull/432 PR: https://git.openjdk.org/babylon/pull/432 From mcimadamore at openjdk.org Fri May 23 10:24:10 2025 From: mcimadamore at openjdk.org (Maurizio Cimadamore) Date: Fri, 23 May 2025 10:24:10 GMT Subject: [code-reflection] RFR: Regularize support for Java types/references [v5] In-Reply-To: References: Message-ID: On Fri, 23 May 2025 10:20:58 GMT, Maurizio Cimadamore wrote: >> This PR regularizes the support for Java types in the code model. There are two competing constraints to balance: on the one hand, the Java types should be readable from humans when inspecting the textual form of code model; on the other hand, we don't want to bake in Java-specific type parsing in the core code model API. >> >> Unfortunately, the current implementation scores poorly on both aspects, as (a) textual Java types can become very hard to read (e.g. `.<., NewTest$B$C>`), and (b) the parsing logic (defined in `DescParser`) contains several ad-hoc extensions that are only there to support Java types. >> >> To address this, this PR introduces two different kinds of externalized type forms: *inflated* type forms and *flattened* type forms. An inflated type form closely follow the structure of the `JavaType` or `JavaRef` it models. For instance, the Java type `Foo` is modelled as follows: >> >> >> java.type.class(Foo, java.type.primitive(void), >> java.type.wildcard(EXTENDS, >> java.type.class(Bar, java.type.primitive(void)))) >> >> >> Inflated type forms can be *flattened* (using `JavaTypeUtils::flatten`), to derive a form that is more suitable for humans. For instance, the above inflated form is flattened as follows: >> >> >> java.type:"Foo" >> >> Conversely, flattened type forms can be *inflated* back (using `JavaTypeUtils::inflate`) to their original inflated form. >> >> This distinction between inflated and flattened forms allow us to massively simplify `DescParser` -- which no longer has to worries about the syntactic vagaries of Java types. All that complexity is now pushed onto the flattened type forms support. The goal is to progressively make flattened Java type forms an "implementation detail" (more on that below). >> >> To accommodate flattened and inflated type forms, two changes are needed: >> * in `OpWriter` we need to flatten an inflated type form (if possible) before writing it out as a string (this allows for the textual form of a code model to be more readable) >> * in `OpParser` we need to inflate a flattened type form (if possible) before handing the type form back to the rest of the parsing machinery (which will e.g. invoke the type factory on such inflated type form) >> >> All Java types and references now follow this pattern: >> 1. the `externalize` method returns an inflated type form >> 2. the `toString` method returns a readable stri... > > Maurizio Cimadamore has updated the pull request incrementally with one additional commit since the last revision: > > Fix examples cr-examples/triton/src/main/java/oracle/code/triton/MLIRGenerator.java line 167: > 165: > 166: for (int i = 0; i < type.length(); ++i) { > 167: if (i + 19 < type.length() && type.substring(i, i + 19).equals("java.type.primitive")) { I had to "tolerate" the new inflated external forms here (by skipping them). The logic here seems to assume that both class and primitive types appear "naked" in the type strings, which is no longer the case. ------------- PR Review Comment: https://git.openjdk.org/babylon/pull/432#discussion_r2104291191 From mcimadamore at openjdk.org Fri May 23 10:28:01 2025 From: mcimadamore at openjdk.org (Maurizio Cimadamore) Date: Fri, 23 May 2025 10:28:01 GMT Subject: [code-reflection] RFR: Regularize support for Java types/references [v5] In-Reply-To: References: Message-ID: On Fri, 23 May 2025 10:24:10 GMT, Maurizio Cimadamore wrote: >> This PR regularizes the support for Java types in the code model. There are two competing constraints to balance: on the one hand, the Java types should be readable from humans when inspecting the textual form of code model; on the other hand, we don't want to bake in Java-specific type parsing in the core code model API. >> >> Unfortunately, the current implementation scores poorly on both aspects, as (a) textual Java types can become very hard to read (e.g. `.<., NewTest$B$C>`), and (b) the parsing logic (defined in `DescParser`) contains several ad-hoc extensions that are only there to support Java types. >> >> To address this, this PR introduces two different kinds of externalized type forms: *inflated* type forms and *flattened* type forms. An inflated type form closely follow the structure of the `JavaType` or `JavaRef` it models. For instance, the Java type `Foo` is modelled as follows: >> >> >> java.type.class(Foo, java.type.primitive(void), >> java.type.wildcard(EXTENDS, >> java.type.class(Bar, java.type.primitive(void)))) >> >> >> Inflated type forms can be *flattened* (using `JavaTypeUtils::flatten`), to derive a form that is more suitable for humans. For instance, the above inflated form is flattened as follows: >> >> >> java.type:"Foo" >> >> Conversely, flattened type forms can be *inflated* back (using `JavaTypeUtils::inflate`) to their original inflated form. >> >> This distinction between inflated and flattened forms allow us to massively simplify `DescParser` -- which no longer has to worries about the syntactic vagaries of Java types. All that complexity is now pushed onto the flattened type forms support. The goal is to progressively make flattened Java type forms an "implementation detail" (more on that below). >> >> To accommodate flattened and inflated type forms, two changes are needed: >> * in `OpWriter` we need to flatten an inflated type form (if possible) before writing it out as a string (this allows for the textual form of a code model to be more readable) >> * in `OpParser` we need to inflate a flattened type form (if possible) before handing the type form back to the rest of the parsing machinery (which will e.g. invoke the type factory on such inflated type form) >> >> All Java types and references now follow this pattern: >> 1. the `externalize` method returns an inflated type form >> 2. the `toString` method returns a readable stri... > > Maurizio Cimadamore has updated the pull request incrementally with one additional commit since the last revision: > > Fix examples cr-examples/triton/src/test/java/oracle/code/triton/TestAddKernel.java line 109: > 107: @TritonCodeModel(""" > 108: module ()java.type:"void" -> { > 109: tt.func @"add_kernel2_ptr>_ptr>_ptr>_int_64_void" (%0 : ptr, %1 : ptr, %2 : ptr, %3 : java.type:"int")java.type:"void" -> { Note (in this and other tests) -- the signature of the triton function contains the literal externalized type string of the various parameter/return types. These are NOT flattened strings because e.g. `Ptr::toString` does not flatten (nor it knows how to, if we were to use the public API). This leads to some asymmetry, where in some cases the inflated form appears in full (as here), while in other cases the short type string appears (e.g. because the type is a Java type, and its `toString` is more friendly). I had half a mind to make `toString` of all triton types more friendly (e.g. not rely on externalize) -- but ultimately I decided to just stick with this very basic conversion -- perhaps we can come back to this later? src/jdk.incubator.code/share/classes/jdk/incubator/code/type/CoreTypeFactory.java line 99: > 97: case INFLATED_TYPE -> JavaTypeUtils.toJavaType(tree); > 98: case INFLATED_REF -> JavaTypeUtils.toJavaRef(tree); > 99: default -> throw new UnsupportedOperationException("Unsupported: " + tree); I'm starting to think this should return `null`, as we used that as "composition" signal? ------------- PR Review Comment: https://git.openjdk.org/babylon/pull/432#discussion_r2104295819 PR Review Comment: https://git.openjdk.org/babylon/pull/432#discussion_r2104297544 From psandoz at openjdk.org Fri May 23 15:25:01 2025 From: psandoz at openjdk.org (Paul Sandoz) Date: Fri, 23 May 2025 15:25:01 GMT Subject: [code-reflection] RFR: Regularize support for Java types/references [v5] In-Reply-To: References: Message-ID: On Fri, 23 May 2025 10:24:08 GMT, Maurizio Cimadamore wrote: >> Maurizio Cimadamore has updated the pull request incrementally with one additional commit since the last revision: >> >> Fix examples > > cr-examples/triton/src/test/java/oracle/code/triton/TestAddKernel.java line 109: > >> 107: @TritonCodeModel(""" >> 108: module ()java.type:"void" -> { >> 109: tt.func @"add_kernel2_ptr>_ptr>_ptr>_int_64_void" (%0 : ptr, %1 : ptr, %2 : ptr, %3 : java.type:"int")java.type:"void" -> { > > Note (in this and other tests) -- the signature of the triton function contains the literal externalized type string of the various parameter/return types. These are NOT flattened strings because e.g. `Ptr::toString` does not flatten (nor it knows how to, if we were to use the public API). This leads to some asymmetry, where in some cases the inflated form appears in full (as here), while in other cases the short type string appears (e.g. because the type is a Java type, and its `toString` is more friendly). > > I had half a mind to make `toString` of all triton types more friendly (e.g. not rely on externalize) -- but ultimately I decided to just stick with this very basic conversion -- perhaps we can come back to this later? This is fine IMO. If need be we can revisit later on. ------------- PR Review Comment: https://git.openjdk.org/babylon/pull/432#discussion_r2104823252 From psandoz at openjdk.org Fri May 23 15:28:14 2025 From: psandoz at openjdk.org (Paul Sandoz) Date: Fri, 23 May 2025 15:28:14 GMT Subject: [code-reflection] RFR: Regularize support for Java types/references [v5] In-Reply-To: References: Message-ID: On Fri, 23 May 2025 10:20:57 GMT, Maurizio Cimadamore wrote: >> Maurizio Cimadamore has updated the pull request incrementally with one additional commit since the last revision: >> >> Fix examples > > cr-examples/triton/src/main/java/oracle/code/triton/MLIRGenerator.java line 167: > >> 165: >> 166: for (int i = 0; i < type.length(); ++i) { >> 167: if (i + 19 < type.length() && type.substring(i, i + 19).equals("java.type.primitive")) { > > I had to "tolerate" the new inflated external forms here (by skipping them). The logic here seems to assume that both class and primitive types appear "naked" in the type strings, which is no longer the case. It's likely this area (Triton code model -> Triton MLIR -> SPIRV) will bit rot as we don't have the expertise and hardware to maintain and fully test. We might need to move it to a separate area if it becomes more burdensome. So what you did seems fine. ------------- PR Review Comment: https://git.openjdk.org/babylon/pull/432#discussion_r2104828454 From psandoz at openjdk.org Fri May 23 15:31:11 2025 From: psandoz at openjdk.org (Paul Sandoz) Date: Fri, 23 May 2025 15:31:11 GMT Subject: [code-reflection] RFR: Regularize support for Java types/references [v5] In-Reply-To: References: Message-ID: On Fri, 23 May 2025 10:24:10 GMT, Maurizio Cimadamore wrote: >> This PR regularizes the support for Java types in the code model. There are two competing constraints to balance: on the one hand, the Java types should be readable from humans when inspecting the textual form of code model; on the other hand, we don't want to bake in Java-specific type parsing in the core code model API. >> >> Unfortunately, the current implementation scores poorly on both aspects, as (a) textual Java types can become very hard to read (e.g. `.<., NewTest$B$C>`), and (b) the parsing logic (defined in `DescParser`) contains several ad-hoc extensions that are only there to support Java types. >> >> To address this, this PR introduces two different kinds of externalized type forms: *inflated* type forms and *flattened* type forms. An inflated type form closely follow the structure of the `JavaType` or `JavaRef` it models. For instance, the Java type `Foo` is modelled as follows: >> >> >> java.type.class(Foo, java.type.primitive(void), >> java.type.wildcard(EXTENDS, >> java.type.class(Bar, java.type.primitive(void)))) >> >> >> Inflated type forms can be *flattened* (using `JavaTypeUtils::flatten`), to derive a form that is more suitable for humans. For instance, the above inflated form is flattened as follows: >> >> >> java.type:"Foo" >> >> Conversely, flattened type forms can be *inflated* back (using `JavaTypeUtils::inflate`) to their original inflated form. >> >> This distinction between inflated and flattened forms allow us to massively simplify `DescParser` -- which no longer has to worries about the syntactic vagaries of Java types. All that complexity is now pushed onto the flattened type forms support. The goal is to progressively make flattened Java type forms an "implementation detail" (more on that below). >> >> To accommodate flattened and inflated type forms, two changes are needed: >> * in `OpWriter` we need to flatten an inflated type form (if possible) before writing it out as a string (this allows for the textual form of a code model to be more readable) >> * in `OpParser` we need to inflate a flattened type form (if possible) before handing the type form back to the rest of the parsing machinery (which will e.g. invoke the type factory on such inflated type form) >> >> All Java types and references now follow this pattern: >> 1. the `externalize` method returns an inflated type form >> 2. the `toString` method returns a readable stri... > > Maurizio Cimadamore has updated the pull request incrementally with one additional commit since the last revision: > > Fix examples Very nice work. This regularizes the code and clearly separates the boundaries. It provide a stronger foundation for further improvements. ------------- Marked as reviewed by psandoz (Lead). PR Review: https://git.openjdk.org/babylon/pull/432#pullrequestreview-2864827491 From mabbay at openjdk.org Sat May 24 09:04:39 2025 From: mabbay at openjdk.org (Mourad Abbay) Date: Sat, 24 May 2025 09:04:39 GMT Subject: [code-reflection] RFR: Replace the use of Interpreter to create Quoted instance with specialized code [v7] In-Reply-To: References: Message-ID: > Creating Quoted instance was done by invoking the Interpreter. The Interpreter is too general and if someone could somehow inject any code model we will interpret that as well. This PR replace the use of the interpreter with specialized code. Mourad Abbay has updated the pull request incrementally with one additional commit since the last revision: Support quoting of an op that has operands ------------- Changes: - all: https://git.openjdk.org/babylon/pull/424/files - new: https://git.openjdk.org/babylon/pull/424/files/8d3a227b..20a7fb46 Webrevs: - full: https://webrevs.openjdk.org/?repo=babylon&pr=424&range=06 - incr: https://webrevs.openjdk.org/?repo=babylon&pr=424&range=05-06 Stats: 80 lines in 3 files changed: 69 ins; 4 del; 7 mod Patch: https://git.openjdk.org/babylon/pull/424.diff Fetch: git fetch https://git.openjdk.org/babylon.git pull/424/head:pull/424 PR: https://git.openjdk.org/babylon/pull/424 From mcimadamore at openjdk.org Mon May 26 09:35:54 2025 From: mcimadamore at openjdk.org (Maurizio Cimadamore) Date: Mon, 26 May 2025 09:35:54 GMT Subject: [code-reflection] Integrated: Regularize support for Java types/references In-Reply-To: References: Message-ID: On Wed, 21 May 2025 10:00:12 GMT, Maurizio Cimadamore wrote: > This PR regularizes the support for Java types in the code model. There are two competing constraints to balance: on the one hand, the Java types should be readable from humans when inspecting the textual form of code model; on the other hand, we don't want to bake in Java-specific type parsing in the core code model API. > > Unfortunately, the current implementation scores poorly on both aspects, as (a) textual Java types can become very hard to read (e.g. `.<., NewTest$B$C>`), and (b) the parsing logic (defined in `DescParser`) contains several ad-hoc extensions that are only there to support Java types. > > To address this, this PR introduces two different kinds of externalized type forms: *inflated* type forms and *flattened* type forms. An inflated type form closely follow the structure of the `JavaType` or `JavaRef` it models. For instance, the Java type `Foo` is modelled as follows: > > > java.type.class(Foo, java.type.primitive(void), > java.type.wildcard(EXTENDS, > java.type.class(Bar, java.type.primitive(void)))) > > > Inflated type forms can be *flattened* (using `JavaTypeUtils::flatten`), to derive a form that is more suitable for humans. For instance, the above inflated form is flattened as follows: > > > java.type:"Foo" > > Conversely, flattened type forms can be *inflated* back (using `JavaTypeUtils::inflate`) to their original inflated form. > > This distinction between inflated and flattened forms allow us to massively simplify `DescParser` -- which no longer has to worries about the syntactic vagaries of Java types. All that complexity is now pushed onto the flattened type forms support. The goal is to progressively make flattened Java type forms an "implementation detail" (more on that below). > > To accommodate flattened and inflated type forms, two changes are needed: > * in `OpWriter` we need to flatten an inflated type form (if possible) before writing it out as a string (this allows for the textual form of a code model to be more readable) > * in `OpParser` we need to inflate a flattened type form (if possible) before handing the type form back to the rest of the parsing machinery (which will e.g. invoke the type factory on such inflated type form) > > All Java types and references now follow this pattern: > 1. the `externalize` method returns an inflated type form > 2. the `toString` method returns a readable string (the one associated with the corresponding flat type ... This pull request has now been integrated. Changeset: 4278b5b4 Author: Maurizio Cimadamore URL: https://git.openjdk.org/babylon/commit/4278b5b437334a93cb266bee43c9c922e8fb2611 Stats: 7952 lines in 83 files changed: 846 ins; 452 del; 6654 mod Regularize support for Java types/references Reviewed-by: psandoz ------------- PR: https://git.openjdk.org/babylon/pull/432 From mcimadamore at openjdk.org Mon May 26 09:42:24 2025 From: mcimadamore at openjdk.org (Maurizio Cimadamore) Date: Mon, 26 May 2025 09:42:24 GMT Subject: git: openjdk/babylon: code-reflection: Regularize support for Java types/references Message-ID: <7f1d8d39-a551-4536-9fef-5a81b76fdcd5@openjdk.org> Changeset: 4278b5b4 Branch: code-reflection Author: Maurizio Cimadamore Date: 2025-05-26 09:32:55 +0000 URL: https://git.openjdk.org/babylon/commit/4278b5b437334a93cb266bee43c9c922e8fb2611 Regularize support for Java types/references Reviewed-by: psandoz ! cr-examples/spirv/src/main/java/intel/code/spirv/SpirvModuleGenerator.java ! cr-examples/triton/src/main/java/oracle/code/triton/MLIRGenerator.java ! cr-examples/triton/src/test/java/oracle/code/triton/TestAddKernel.java ! cr-examples/triton/src/test/java/oracle/code/triton/TestBroadcast.java ! cr-examples/triton/src/test/java/oracle/code/triton/TestCdiv.java ! cr-examples/triton/src/test/java/oracle/code/triton/TestCountedLoop.java ! cr-examples/triton/src/test/java/oracle/code/triton/TestMatrix.java ! cr-examples/triton/src/test/java/oracle/code/triton/TestMatrixFp16.java ! cr-examples/triton/src/test/java/oracle/code/triton/TestSoftMax.java ! cr-examples/triton/src/test/java/oracle/code/triton/TestZeros.java ! hat/backends/ffi/spirv/src/main/java/intel/code/spirv/SpirvModuleGenerator.java ! src/jdk.incubator.code/share/classes/jdk/incubator/code/TypeElement.java ! src/jdk.incubator.code/share/classes/jdk/incubator/code/parser/OpParser.java ! src/jdk.incubator.code/share/classes/jdk/incubator/code/parser/impl/DescParser.java ! src/jdk.incubator.code/share/classes/jdk/incubator/code/type/ArrayType.java ! src/jdk.incubator.code/share/classes/jdk/incubator/code/type/ClassType.java ! src/jdk.incubator.code/share/classes/jdk/incubator/code/type/ConstructorRef.java ! src/jdk.incubator.code/share/classes/jdk/incubator/code/type/CoreTypeFactory.java ! src/jdk.incubator.code/share/classes/jdk/incubator/code/type/FieldRef.java ! src/jdk.incubator.code/share/classes/jdk/incubator/code/type/JavaRef.java ! src/jdk.incubator.code/share/classes/jdk/incubator/code/type/JavaType.java ! src/jdk.incubator.code/share/classes/jdk/incubator/code/type/MethodRef.java ! src/jdk.incubator.code/share/classes/jdk/incubator/code/type/PrimitiveType.java ! src/jdk.incubator.code/share/classes/jdk/incubator/code/type/RecordTypeRef.java ! src/jdk.incubator.code/share/classes/jdk/incubator/code/type/TypeVariableType.java ! src/jdk.incubator.code/share/classes/jdk/incubator/code/type/WildcardType.java ! src/jdk.incubator.code/share/classes/jdk/incubator/code/type/impl/ConstructorRefImpl.java ! src/jdk.incubator.code/share/classes/jdk/incubator/code/type/impl/FieldRefImpl.java + src/jdk.incubator.code/share/classes/jdk/incubator/code/type/impl/JavaTypeUtils.java ! src/jdk.incubator.code/share/classes/jdk/incubator/code/type/impl/MethodRefImpl.java ! src/jdk.incubator.code/share/classes/jdk/incubator/code/type/impl/RecordTypeRefImpl.java ! src/jdk.incubator.code/share/classes/jdk/incubator/code/writer/OpWriter.java ! test/jdk/java/lang/reflect/code/TestLiveness.java ! test/jdk/java/lang/reflect/code/TestNormalizeBlocksTransformer.java ! test/jdk/java/lang/reflect/code/TestUsesDependsOn.java ! test/jdk/java/lang/reflect/code/lower/TestLoop.java ! test/jdk/java/lang/reflect/code/lower/TestSynchronized.java ! test/jdk/java/lang/reflect/code/parser/TestParse.java ! test/jdk/java/lang/reflect/code/type/TestJavaType.java ! test/jdk/java/lang/reflect/code/type/TestReferences.java ! test/langtools/tools/javac/reflect/ArrayAccessTest.java ! test/langtools/tools/javac/reflect/AssertTest.java ! test/langtools/tools/javac/reflect/BinopTest.java ! test/langtools/tools/javac/reflect/BlockTest.java ! test/langtools/tools/javac/reflect/BoxingConversionTest.java ! test/langtools/tools/javac/reflect/BreakContinueTest.java ! test/langtools/tools/javac/reflect/CastInstanceOfTest.java ! test/langtools/tools/javac/reflect/CodeReflectionTester.java ! test/langtools/tools/javac/reflect/ConditionalAndOrTest.java ! test/langtools/tools/javac/reflect/ConditionalExpressionTest.java ! test/langtools/tools/javac/reflect/ConstantsTest.java ! test/langtools/tools/javac/reflect/DenotableTypesTest.java ! test/langtools/tools/javac/reflect/EnumAccessTest.java ! test/langtools/tools/javac/reflect/FieldAccessTest.java ! test/langtools/tools/javac/reflect/ForLoopTest.java ! test/langtools/tools/javac/reflect/IfTest.java ! test/langtools/tools/javac/reflect/ImplicitConversionTest.java ! test/langtools/tools/javac/reflect/IntersectionTypeTest.java ! test/langtools/tools/javac/reflect/LambdaTest.java ! test/langtools/tools/javac/reflect/LocalClassTest.java ! test/langtools/tools/javac/reflect/LocalVarTest.java ! test/langtools/tools/javac/reflect/MethodCallTest.java ! test/langtools/tools/javac/reflect/MethodReferenceTest.java ! test/langtools/tools/javac/reflect/NewArrayTest.java ! test/langtools/tools/javac/reflect/NewTest.java ! test/langtools/tools/javac/reflect/NullTest.java ! test/langtools/tools/javac/reflect/PatternTest2.java ! test/langtools/tools/javac/reflect/PatternsTest.java ! test/langtools/tools/javac/reflect/PrimitiveCastTest.java ! test/langtools/tools/javac/reflect/QuotableIntersectionTest.java ! test/langtools/tools/javac/reflect/QuotableSubtypeTest.java ! test/langtools/tools/javac/reflect/QuotedTest.java ! test/langtools/tools/javac/reflect/StringConcatTest.java ! test/langtools/tools/javac/reflect/SuperTest.java ! test/langtools/tools/javac/reflect/SwitchExpressionTest.java ! test/langtools/tools/javac/reflect/SwitchExpressionTest2.java ! test/langtools/tools/javac/reflect/SwitchStatementTest.java ! test/langtools/tools/javac/reflect/SynchronizedTest.java ! test/langtools/tools/javac/reflect/ThrowTest.java ! test/langtools/tools/javac/reflect/TryTest.java ! test/langtools/tools/javac/reflect/UnaryopTest.java ! test/langtools/tools/javac/reflect/UnreachableTest.java ! test/langtools/tools/javac/reflect/WhileLoopTest.java From mcimadamore at openjdk.org Mon May 26 11:46:48 2025 From: mcimadamore at openjdk.org (Maurizio Cimadamore) Date: Mon, 26 May 2025 11:46:48 GMT Subject: [code-reflection] RFR: Regularize parsing of refs/types in op attributes Message-ID: This PR builds on https://git.openjdk.org/babylon/pull/432 and simplifies the way in which the various op factories handle attributes that are either `JavaRef` or `JavaType`. Previously, such attribute values were written as strings -- and then the op factory would need to parse them back using one of the `JavaType::ofString`, or `XYZRef::ofString` parsing factories. In this PR, type elements are serialized into strings using `TypeElement::externalize` (as usual, we also apply flattening, to turn the external forms into something more readable). When parsing back, we then check whether we're staring at an externalized form or not. If so, we parse it back, we inflate it and turn it back into a `TypeElement` using the selected type factory. As a result op factories can just work on type elements directly, all the parsing will already have occurred at that point. This simplfies the op factory code, and, crucially, allows us to drop the various `ofString` factories. This means that the code model API no longer relies on the textual form of Java types/refs. #### Next steps * enhance attribute parsing to special case all Java literals (so that we can drop even more parsing in the common cases) * unify grammar between attributes and types ------------- Commit messages: - Drop unused imports - Fix examples - Drop ofString factories and fix up tests - Merge branch 'code-reflection' into remove_of_string_type_factories - All tests pass - Fix examples - Simplify JavaTypeUtils to use ExternalizedTypeElement factories - Enhance type factory to create refs - Merge branch 'code-reflection' into externalized_Java_type_cleanup - Simplify parser code to use `Lexer::is` - ... and 16 more: https://git.openjdk.org/babylon/compare/4278b5b4...6cd1e041 Changes: https://git.openjdk.org/babylon/pull/433/files Webrev: https://webrevs.openjdk.org/?repo=babylon&pr=433&range=00 Stats: 729 lines in 49 files changed: 48 ins; 37 del; 644 mod Patch: https://git.openjdk.org/babylon/pull/433.diff Fetch: git fetch https://git.openjdk.org/babylon.git pull/433/head:pull/433 PR: https://git.openjdk.org/babylon/pull/433 From gfrost at openjdk.org Mon May 26 14:53:15 2025 From: gfrost at openjdk.org (Gary Frost) Date: Mon, 26 May 2025 14:53:15 GMT Subject: [code-reflection] Integrated: add extern C and minize square output Message-ID: Mock backend was broken, compiled, but fgailed to link at runtime. I added `extern"C"` to getBackend method. Also cleaned up/minimized output from squares and violajones examples. Ensured that `java @hat/clean` removed cached src dirs (this needs refactoring). ------------- Commit messages: - ws - add extern C and minize square output Changes: https://git.openjdk.org/babylon/pull/434/files Webrev: https://webrevs.openjdk.org/?repo=babylon&pr=434&range=00 Stats: 17 lines in 4 files changed: 9 ins; 0 del; 8 mod Patch: https://git.openjdk.org/babylon/pull/434.diff Fetch: git fetch https://git.openjdk.org/babylon.git pull/434/head:pull/434 PR: https://git.openjdk.org/babylon/pull/434 From gfrost at openjdk.org Mon May 26 14:53:29 2025 From: gfrost at openjdk.org (Gary Frost) Date: Mon, 26 May 2025 14:53:29 GMT Subject: git: openjdk/babylon: code-reflection: add extern C and minize square output Message-ID: Changeset: 288b6628 Branch: code-reflection Author: Gary Frost Date: 2025-05-26 14:50:29 +0000 URL: https://git.openjdk.org/babylon/commit/288b662861f8141d7e85104f409504ce158a0e7c add extern C and minize square output ! hat/backends/ffi/mock/cpp/mock_backend.cpp ! hat/examples/squares/src/main/java/squares/Main.java ! hat/examples/violajones/src/main/java/violajones/Main.java ! hat/hat/clean.java From gfrost at openjdk.org Mon May 26 14:53:15 2025 From: gfrost at openjdk.org (Gary Frost) Date: Mon, 26 May 2025 14:53:15 GMT Subject: [code-reflection] Integrated: add extern C and minize square output In-Reply-To: References: Message-ID: On Mon, 26 May 2025 14:45:19 GMT, Gary Frost wrote: > Mock backend was broken, compiled, but fgailed to link at runtime. > > I added `extern"C"` to getBackend method. > > Also cleaned up/minimized output from squares and violajones examples. > > Ensured that `java @hat/clean` removed cached src dirs (this needs refactoring). This pull request has now been integrated. Changeset: 288b6628 Author: Gary Frost URL: https://git.openjdk.org/babylon/commit/288b662861f8141d7e85104f409504ce158a0e7c Stats: 17 lines in 4 files changed: 9 ins; 0 del; 8 mod add extern C and minize square output ------------- PR: https://git.openjdk.org/babylon/pull/434 From mabbay at openjdk.org Tue May 27 00:51:46 2025 From: mabbay at openjdk.org (Mourad Abbay) Date: Tue, 27 May 2025 00:51:46 GMT Subject: [code-reflection] RFR: Replace the use of Interpreter to create Quoted instance with specialized code [v8] In-Reply-To: References: Message-ID: > Creating Quoted instance was done by invoking the Interpreter. The Interpreter is too general and if someone could somehow inject any code model we will interpret that as well. This PR replace the use of the interpreter with specialized code. Mourad Abbay has updated the pull request incrementally with one additional commit since the last revision: Update BytecodeGenerator to use CoreOp.quoteOp ------------- Changes: - all: https://git.openjdk.org/babylon/pull/424/files - new: https://git.openjdk.org/babylon/pull/424/files/20a7fb46..9f3db051 Webrevs: - full: https://webrevs.openjdk.org/?repo=babylon&pr=424&range=07 - incr: https://webrevs.openjdk.org/?repo=babylon&pr=424&range=06-07 Stats: 41 lines in 1 file changed: 0 ins; 40 del; 1 mod Patch: https://git.openjdk.org/babylon/pull/424.diff Fetch: git fetch https://git.openjdk.org/babylon.git pull/424/head:pull/424 PR: https://git.openjdk.org/babylon/pull/424 From psandoz at openjdk.org Tue May 27 17:45:04 2025 From: psandoz at openjdk.org (Paul Sandoz) Date: Tue, 27 May 2025 17:45:04 GMT Subject: [code-reflection] RFR: Regularize parsing of refs/types in op attributes In-Reply-To: References: Message-ID: On Mon, 26 May 2025 11:41:15 GMT, Maurizio Cimadamore wrote: > This PR builds on https://git.openjdk.org/babylon/pull/432 and simplifies the way in which the various op factories handle attributes that are either `JavaRef` or `JavaType`. > Previously, such attribute values were written as strings -- and then the op factory would need to parse them back using one of the `JavaType::ofString`, or `XYZRef::ofString` parsing factories. > > In this PR, type elements are serialized into strings using `TypeElement::externalize` (as usual, we also apply flattening, to turn the external forms into something more readable). > When parsing back, we then check whether we're staring at an externalized form or not. If so, we parse it back, we inflate it and turn it back into a `TypeElement` using the selected type factory. > > As a result op factories can just work on type elements directly, all the parsing will already have occurred at that point. This simplfies the op factory code, and, crucially, allows us to drop the various `ofString` factories. This means that the code model API no longer relies on the textual form of Java types/refs. > > #### Next steps > * enhance attribute parsing to special case all Java literals (so that we can drop even more parsing in the common cases) > * unify grammar between attributes and types Looks good. ------------- PR Review: https://git.openjdk.org/babylon/pull/433#pullrequestreview-2871992553 From mabbay at openjdk.org Tue May 27 23:04:09 2025 From: mabbay at openjdk.org (Mourad Abbay) Date: Tue, 27 May 2025 23:04:09 GMT Subject: [code-reflection] RFR: Replace the use of Interpreter to create Quoted instance with specialized code [v4] In-Reply-To: References: Message-ID: <7T5xK7Q6lVfKRywCC6wP-6lRMwQ3B1k-REHGoHNd_i0=.4f0e7537-e809-4e7b-857a-e244adfc0c9a@github.com> On Thu, 22 May 2025 10:46:27 GMT, Maurizio Cimadamore wrote: >> Mourad Abbay has refreshed the contents of this pull request, and previous commits have been removed. Incremental views are not available. > > src/jdk.incubator.code/share/classes/jdk/incubator/code/internal/QuotedHelper.java line 39: > >> 37: public class QuotedHelper { >> 38: >> 39: public static Quoted makeQuoted(FuncOp op, Object[] args) { > > Shouldn't this use the new helper method you've defined? > And shouldn't javac use the other helper method to create the FuncOp in the first place? I'm looking to see how CoreOp.quoteOp can be used in javac. ------------- PR Review Comment: https://git.openjdk.org/babylon/pull/424#discussion_r2110463934 From mcimadamore at openjdk.org Wed May 28 09:28:34 2025 From: mcimadamore at openjdk.org (Maurizio Cimadamore) Date: Wed, 28 May 2025 09:28:34 GMT Subject: git: openjdk/babylon: code-reflection: Regularize parsing of refs/types in op attributes Message-ID: <39517a3e-26a7-4002-8a51-73cd45f228d2@openjdk.org> Changeset: 44735d0d Branch: code-reflection Author: Maurizio Cimadamore Date: 2025-05-28 09:26:31 +0000 URL: https://git.openjdk.org/babylon/commit/44735d0d39e2c1141332bfa75d6b4ec822d58013 Regularize parsing of refs/types in op attributes ! cr-examples/spirv/src/main/java/intel/code/spirv/SpirvModuleGenerator.java ! cr-examples/triton/src/main/java/oracle/code/triton/TritonTransformer.java ! hat/backends/ffi/spirv/src/main/java/intel/code/spirv/SpirvModuleGenerator.java ! src/jdk.incubator.code/share/classes/jdk/incubator/code/op/CoreOp.java ! src/jdk.incubator.code/share/classes/jdk/incubator/code/op/ExtendedOp.java ! src/jdk.incubator.code/share/classes/jdk/incubator/code/parser/OpParser.java ! src/jdk.incubator.code/share/classes/jdk/incubator/code/type/ConstructorRef.java ! src/jdk.incubator.code/share/classes/jdk/incubator/code/type/FieldRef.java ! src/jdk.incubator.code/share/classes/jdk/incubator/code/type/JavaType.java ! src/jdk.incubator.code/share/classes/jdk/incubator/code/type/MethodRef.java ! src/jdk.incubator.code/share/classes/jdk/incubator/code/type/RecordTypeRef.java ! src/jdk.incubator.code/share/classes/jdk/incubator/code/writer/OpWriter.java ! test/jdk/java/lang/reflect/code/TestNormalizeBlocksTransformer.java ! test/jdk/java/lang/reflect/code/type/TestJavaType.java ! test/jdk/java/lang/reflect/code/type/TestReferences.java ! test/langtools/tools/javac/reflect/ArrayAccessTest.java ! test/langtools/tools/javac/reflect/BlockTest.java ! test/langtools/tools/javac/reflect/BoxingConversionTest.java ! test/langtools/tools/javac/reflect/CastInstanceOfTest.java ! test/langtools/tools/javac/reflect/ConditionalExpressionTest.java ! test/langtools/tools/javac/reflect/ConstantsTest.java ! test/langtools/tools/javac/reflect/DenotableTypesTest.java ! test/langtools/tools/javac/reflect/EnumAccessTest.java ! test/langtools/tools/javac/reflect/FieldAccessTest.java ! test/langtools/tools/javac/reflect/ForLoopTest.java ! test/langtools/tools/javac/reflect/IfTest.java ! test/langtools/tools/javac/reflect/ImplicitConversionTest.java ! test/langtools/tools/javac/reflect/IntersectionTypeTest.java ! test/langtools/tools/javac/reflect/LambdaTest.java ! test/langtools/tools/javac/reflect/LocalClassTest.java ! test/langtools/tools/javac/reflect/MethodCallTest.java ! test/langtools/tools/javac/reflect/MethodReferenceTest.java ! test/langtools/tools/javac/reflect/NewArrayTest.java ! test/langtools/tools/javac/reflect/NewTest.java ! test/langtools/tools/javac/reflect/NullTest.java ! test/langtools/tools/javac/reflect/PatternTest2.java ! test/langtools/tools/javac/reflect/PatternsTest.java ! test/langtools/tools/javac/reflect/QuotableIntersectionTest.java ! test/langtools/tools/javac/reflect/QuotableSubtypeTest.java ! test/langtools/tools/javac/reflect/QuotedTest.java ! test/langtools/tools/javac/reflect/SuperTest.java ! test/langtools/tools/javac/reflect/SwitchExpressionTest.java ! test/langtools/tools/javac/reflect/SwitchExpressionTest2.java ! test/langtools/tools/javac/reflect/SwitchStatementTest.java ! test/langtools/tools/javac/reflect/SynchronizedTest.java ! test/langtools/tools/javac/reflect/ThrowTest.java ! test/langtools/tools/javac/reflect/TryTest.java ! test/langtools/tools/javac/reflect/UnaryopTest.java ! test/langtools/tools/javac/reflect/WhileLoopTest.java From mcimadamore at openjdk.org Wed May 28 09:29:07 2025 From: mcimadamore at openjdk.org (Maurizio Cimadamore) Date: Wed, 28 May 2025 09:29:07 GMT Subject: [code-reflection] Integrated: Regularize parsing of refs/types in op attributes In-Reply-To: References: Message-ID: <4Aj9w5jMCyc8L9nu1lz4q99llBOY39geTPlPcq_DCF4=.20b95f55-a2a4-41e0-b568-5cf4b6800e16@github.com> On Mon, 26 May 2025 11:41:15 GMT, Maurizio Cimadamore wrote: > This PR builds on https://git.openjdk.org/babylon/pull/432 and simplifies the way in which the various op factories handle attributes that are either `JavaRef` or `JavaType`. > Previously, such attribute values were written as strings -- and then the op factory would need to parse them back using one of the `JavaType::ofString`, or `XYZRef::ofString` parsing factories. > > In this PR, type elements are serialized into strings using `TypeElement::externalize` (as usual, we also apply flattening, to turn the external forms into something more readable). > When parsing back, we then check whether we're staring at an externalized form or not. If so, we parse it back, we inflate it and turn it back into a `TypeElement` using the selected type factory. > > As a result op factories can just work on type elements directly, all the parsing will already have occurred at that point. This simplfies the op factory code, and, crucially, allows us to drop the various `ofString` factories. This means that the code model API no longer relies on the textual form of Java types/refs. > > #### Next steps > * enhance attribute parsing to special case all Java literals (so that we can drop even more parsing in the common cases) > * unify grammar between attributes and types This pull request has now been integrated. Changeset: 44735d0d Author: Maurizio Cimadamore URL: https://git.openjdk.org/babylon/commit/44735d0d39e2c1141332bfa75d6b4ec822d58013 Stats: 729 lines in 49 files changed: 48 ins; 37 del; 644 mod Regularize parsing of refs/types in op attributes ------------- PR: https://git.openjdk.org/babylon/pull/433 From mabbay at openjdk.org Wed May 28 10:20:03 2025 From: mabbay at openjdk.org (Mourad Abbay) Date: Wed, 28 May 2025 10:20:03 GMT Subject: [code-reflection] RFR: Replace the use of Interpreter to create Quoted instance with specialized code [v4] In-Reply-To: <7T5xK7Q6lVfKRywCC6wP-6lRMwQ3B1k-REHGoHNd_i0=.4f0e7537-e809-4e7b-857a-e244adfc0c9a@github.com> References: <7T5xK7Q6lVfKRywCC6wP-6lRMwQ3B1k-REHGoHNd_i0=.4f0e7537-e809-4e7b-857a-e244adfc0c9a@github.com> Message-ID: On Tue, 27 May 2025 23:01:18 GMT, Mourad Abbay wrote: >> src/jdk.incubator.code/share/classes/jdk/incubator/code/internal/QuotedHelper.java line 39: >> >>> 37: public class QuotedHelper { >>> 38: >>> 39: public static Quoted makeQuoted(FuncOp op, Object[] args) { >> >> Shouldn't this use the new helper method you've defined? >> And shouldn't javac use the other helper method to create the FuncOp in the first place? > > I'm looking to see how CoreOp.quoteOp can be used in javac. CoreOp.quoteOp result is different from what javac produces. For instance, CoreOp.quoteOp doesn't keep captured constants in the model. We can update it to do so, but then the goal of such as method was to take an operation from its original context by representing its captured values and operands as parameters. ------------- PR Review Comment: https://git.openjdk.org/babylon/pull/424#discussion_r2111488510 From mabbay at openjdk.org Wed May 28 10:23:01 2025 From: mabbay at openjdk.org (Mourad Abbay) Date: Wed, 28 May 2025 10:23:01 GMT Subject: [code-reflection] RFR: Replace the use of Interpreter to create Quoted instance with specialized code [v4] In-Reply-To: References: Message-ID: <4-DPT3NQoGYmlFJz5De5cF5RUP9F1C2O7MEw8w8eJs8=.aa8707ae-597b-4511-84a8-89eeb1bea62a@github.com> On Thu, 22 May 2025 10:49:33 GMT, Maurizio Cimadamore wrote: >> Mourad Abbay has updated the pull request incrementally with one additional commit since the last revision: >> >> Implement quotedOp method > > src/jdk.incubator.code/share/classes/jdk/incubator/code/op/CoreOp.java line 4494: > >> 4492: >> 4493: public static Quoted quotedOp(FuncOp funcOp, Object[] args) { >> 4494: > > I believe the checks you enforce are correct -- but leaving these checks as plain `assert` feel odd -- e.g. this is the contract of an API method, so the contract of such method should be fully specified, and real exceptions must be thrown. With `assert`, the issues will only be reported if the user enables assertion when running their code (actually worse, since these assertions are inside JDK, they would need to enable "system assertions" -- which is to say that they will never see this code fail in the way you intended). I will update to throw in case a check fails. ------------- PR Review Comment: https://git.openjdk.org/babylon/pull/424#discussion_r2111493009 From mabbay at openjdk.org Wed May 28 10:38:29 2025 From: mabbay at openjdk.org (Mourad Abbay) Date: Wed, 28 May 2025 10:38:29 GMT Subject: [code-reflection] RFR: Replace the use of Interpreter to create Quoted instance with specialized code [v4] In-Reply-To: References: Message-ID: On Thu, 22 May 2025 11:13:40 GMT, Maurizio Cimadamore wrote: >> src/jdk.incubator.code/share/classes/jdk/incubator/code/op/CoreOp.java line 4477: >> >>> 4475: >>> 4476: b.op(o); >>> 4477: >> >> Why is this loop needed? Is this to restore the original mapping in the copy context? If so, perhaps the code could be strengthened by surrounding `b.op` with a `try/finally` and then restore the mapping in the finally block? >> >> (But I wonder if the API could offer more help here -- e.g. some kind of temp value mapping) > > Iterating on `m.entrySet` would allow to skip the extra call to `m.get` ? > > (an overload in `CopyContext::mapValues` that also accepts a `Map` also seems fair game here) > Why is this loop needed? Is this to restore the original mapping in the copy context? If so, perhaps the code could be strengthened by surrounding `b.op` with a `try/finally` and then restore the mapping in the finally block? > > (But I wonder if the API could offer more help here -- e.g. some kind of temp value mapping) Maybe when appending an op, we can pass a map of values that overrides the mapping in context temporally ? ------------- PR Review Comment: https://git.openjdk.org/babylon/pull/424#discussion_r2111522551 From mcimadamore at openjdk.org Wed May 28 10:51:05 2025 From: mcimadamore at openjdk.org (Maurizio Cimadamore) Date: Wed, 28 May 2025 10:51:05 GMT Subject: [code-reflection] RFR: Replace the use of Interpreter to create Quoted instance with specialized code [v4] In-Reply-To: References: <7T5xK7Q6lVfKRywCC6wP-6lRMwQ3B1k-REHGoHNd_i0=.4f0e7537-e809-4e7b-857a-e244adfc0c9a@github.com> Message-ID: On Wed, 28 May 2025 10:17:53 GMT, Mourad Abbay wrote: >> I'm looking to see how CoreOp.quoteOp can be used in javac. > > CoreOp.quoteOp result is different from what javac produces. For instance, CoreOp.quoteOp doesn't keep captured constants in the model. We can update it to do so, but then the goal of such as method was to take an operation from its original context by representing its captured values and operands as parameters. I'm a bit confused -- if the new `CoreOp` methods cannot be used in javac/`QuotedHelper` -- then why are they in this PR? @PaulSandoz did you intend for the new methods to be the official way to deal with quoted models both in javac and elsewhere -- or was that just an API suggestion unrelated from the goal of this PR? (I'm a bit confused) ------------- PR Review Comment: https://git.openjdk.org/babylon/pull/424#discussion_r2111544145 From mcimadamore at openjdk.org Wed May 28 11:46:05 2025 From: mcimadamore at openjdk.org (Maurizio Cimadamore) Date: Wed, 28 May 2025 11:46:05 GMT Subject: [code-reflection] RFR: Replace the use of Interpreter to create Quoted instance with specialized code [v4] In-Reply-To: References: <7T5xK7Q6lVfKRywCC6wP-6lRMwQ3B1k-REHGoHNd_i0=.4f0e7537-e809-4e7b-857a-e244adfc0c9a@github.com> Message-ID: On Wed, 28 May 2025 10:48:04 GMT, Maurizio Cimadamore wrote: >> CoreOp.quoteOp result is different from what javac produces. For instance, CoreOp.quoteOp doesn't keep captured constants in the model. We can update it to do so, but then the goal of such as method was to take an operation from its original context by representing its captured values and operands as parameters. > > I'm a bit confused -- if the new `CoreOp` methods cannot be used in javac/`QuotedHelper` -- then why are they in this PR? @PaulSandoz did you intend for the new methods to be the official way to deal with quoted models both in javac and elsewhere -- or was that just an API suggestion unrelated from the goal of this PR? (I'm a bit confused) I checked back javac code, and javac generates a `funcOp` which more or less adheres to the characteristics described here -- more specifically, the entry block of the func generated by javac has N vars and N block parameters. The only discrepancy is that javac needs to deal with cases where the referred captured symbol is a constant (e.g. `final String s = "hello"`). In such cases, the generated code for the lambda still relies on a variable -- but that variable is _not_ initialized with a block parameter of the generated func -- it is instead initialized to a constant value. IMHO generalizing the new function to deal with such "local" vars could be useful (and will make javac generated code compliant with the rules we set). When implementing support for captured constants, we debated this a bit -- the alternative would have been to just "inline" the reference to the constant into the lambda body -- but that seemed a less transparent approach, so we decided to keep the reference to an enclosing var there -- so that e.g. the name of the capture symbol is still visible. If we still feel this is important, then I'd say the new functions should support this idiom. ------------- PR Review Comment: https://git.openjdk.org/babylon/pull/424#discussion_r2111640598 From mcimadamore at openjdk.org Wed May 28 12:02:03 2025 From: mcimadamore at openjdk.org (Maurizio Cimadamore) Date: Wed, 28 May 2025 12:02:03 GMT Subject: [code-reflection] RFR: Replace the use of Interpreter to create Quoted instance with specialized code [v4] In-Reply-To: References: <7T5xK7Q6lVfKRywCC6wP-6lRMwQ3B1k-REHGoHNd_i0=.4f0e7537-e809-4e7b-857a-e244adfc0c9a@github.com> Message-ID: On Wed, 28 May 2025 11:43:43 GMT, Maurizio Cimadamore wrote: >> I'm a bit confused -- if the new `CoreOp` methods cannot be used in javac/`QuotedHelper` -- then why are they in this PR? @PaulSandoz did you intend for the new methods to be the official way to deal with quoted models both in javac and elsewhere -- or was that just an API suggestion unrelated from the goal of this PR? (I'm a bit confused) > > I checked back javac code, and javac generates a `funcOp` which more or less adheres to the characteristics described here -- more specifically, the entry block of the func generated by javac has N vars and N block parameters. The only discrepancy is that javac needs to deal with cases where the referred captured symbol is a constant (e.g. `final String s = "hello"`). In such cases, the generated code for the lambda still relies on a variable -- but that variable is _not_ initialized with a block parameter of the generated func -- it is instead initialized to a constant value. IMHO generalizing the new function to deal with such "local" vars could be useful (and will make javac generated code compliant with the rules we set). > > When implementing support for captured constants, we debated this a bit -- the alternative would have been to just "inline" the reference to the constant into the lambda body -- but that seemed a less transparent approach, so we decided to keep the reference to an enclosing var there -- so that e.g. the name of the capture symbol is still visible. If we still feel this is important, then I'd say the new functions should support this idiom. To be clear: I think using `CoreOp::quoteOp` inside javac is a no-go. Javac has a lot of special code to allow to generate the model for a lambda "in isolation" -- this involves computing all the captures upfront, and then setting up a block with enough `Var` to satisfy the generation of the lambda body. References to captured symbols inside the lambda body are replaced with reference to such generated `Var`s -- this is roughly what `CoreOp::quoteOp` does -- but that method assumes we have an `Op` to begin with -- whereas javac is trying to generate that op as well, so I think what javac does looks ok. What seems more doable (and desirable IMHO) is to replace `QuotedHelper` with the other function -- `CoreOp::quotedOp` -- but to do this, `CoreOp::quotedOp` needs to relax some of its checks to tolerate (local) `Var` initialized with an (immediately preceding) constant op. ------------- PR Review Comment: https://git.openjdk.org/babylon/pull/424#discussion_r2111668620 From mcimadamore at openjdk.org Wed May 28 12:02:03 2025 From: mcimadamore at openjdk.org (Maurizio Cimadamore) Date: Wed, 28 May 2025 12:02:03 GMT Subject: [code-reflection] RFR: Replace the use of Interpreter to create Quoted instance with specialized code [v4] In-Reply-To: References: <7T5xK7Q6lVfKRywCC6wP-6lRMwQ3B1k-REHGoHNd_i0=.4f0e7537-e809-4e7b-857a-e244adfc0c9a@github.com> Message-ID: <2r2EKKymyDf21OdOFR2wtnGG4PF8JR1f9HKOxflTOL0=.c1e6cb43-b629-401a-97e4-554a4fca9b50@github.com> On Wed, 28 May 2025 11:57:12 GMT, Maurizio Cimadamore wrote: >> I checked back javac code, and javac generates a `funcOp` which more or less adheres to the characteristics described here -- more specifically, the entry block of the func generated by javac has N vars and N block parameters. The only discrepancy is that javac needs to deal with cases where the referred captured symbol is a constant (e.g. `final String s = "hello"`). In such cases, the generated code for the lambda still relies on a variable -- but that variable is _not_ initialized with a block parameter of the generated func -- it is instead initialized to a constant value. IMHO generalizing the new function to deal with such "local" vars could be useful (and will make javac generated code compliant with the rules we set). >> >> When implementing support for captured constants, we debated this a bit -- the alternative would have been to just "inline" the reference to the constant into the lambda body -- but that seemed a less transparent approach, so we decided to keep the reference to an enclosing var there -- so that e.g. the name of the capture symbol is still visible. If we still feel this is important, then I'd say the new functions should support this idiom. > > To be clear: I think using `CoreOp::quoteOp` inside javac is a no-go. Javac has a lot of special code to allow to generate the model for a lambda "in isolation" -- this involves computing all the captures upfront, and then setting up a block with enough `Var` to satisfy the generation of the lambda body. References to captured symbols inside the lambda body are replaced with reference to such generated `Var`s -- this is roughly what `CoreOp::quoteOp` does -- but that method assumes we have an `Op` to begin with -- whereas javac is trying to generate that op as well, so I think what javac does looks ok. > > What seems more doable (and desirable IMHO) is to replace `QuotedHelper` with the other function -- `CoreOp::quotedOp` -- but to do this, `CoreOp::quotedOp` needs to relax some of its checks to tolerate (local) `Var` initialized with an (immediately preceding) constant op. (I have no objection to this PR going ahead as is -- what I find mildly strange is that we tied this PR with the addition of these two new API functions in `CoreOp`, even though we ultimately don't have a definite answer on how to reuse said functions from the original code in this PR). ------------- PR Review Comment: https://git.openjdk.org/babylon/pull/424#discussion_r2111672142 From asotona at openjdk.org Wed May 28 14:46:23 2025 From: asotona at openjdk.org (Adam Sotona) Date: Wed, 28 May 2025 14:46:23 GMT Subject: [code-reflection] RFR: Onnx fixes to support LLM + Onnx GenAI FFM API Message-ID: - Manually added some contrib ops - Implemented support for records/tuples in arguments and as initializers - Fixed declaration of op set imports in functions - Generated FFM interface to Onnx GenAI API - Onnx op domain made a part of the op name - Added `forceInline` switch to OnnxTransformer ------------- Commit messages: - added forceInline switch to OnnxTransformer - temporary disable functions due to Onnx problem with unused outputs in functions - OnnxProtoBuilder fixed tuples expansion in function arguments - fixed missing operator set imports for functions - support for tuples of initializers - OnnxTransformer fixed remapInitializers to support records/tuples - fixed OnnxTransformer to convert filed load types - removal of Optional from SkipSimplifiedLayerNormalization result record - OnnxTransformer support for optional record components - fixed ExplicitOnnxOps attribute value types - ... and 12 more: https://git.openjdk.org/babylon/compare/6c7ea689...5b12215e Changes: https://git.openjdk.org/babylon/pull/429/files Webrev: https://webrevs.openjdk.org/?repo=babylon&pr=429&range=00 Stats: 8510 lines in 9 files changed: 8412 ins; 27 del; 71 mod Patch: https://git.openjdk.org/babylon/pull/429.diff Fetch: git fetch https://git.openjdk.org/babylon.git pull/429/head:pull/429 PR: https://git.openjdk.org/babylon/pull/429 From psandoz at openjdk.org Wed May 28 15:42:06 2025 From: psandoz at openjdk.org (Paul Sandoz) Date: Wed, 28 May 2025 15:42:06 GMT Subject: [code-reflection] RFR: Replace the use of Interpreter to create Quoted instance with specialized code [v4] In-Reply-To: <2r2EKKymyDf21OdOFR2wtnGG4PF8JR1f9HKOxflTOL0=.c1e6cb43-b629-401a-97e4-554a4fca9b50@github.com> References: <7T5xK7Q6lVfKRywCC6wP-6lRMwQ3B1k-REHGoHNd_i0=.4f0e7537-e809-4e7b-857a-e244adfc0c9a@github.com> <2r2EKKymyDf21OdOFR2wtnGG4PF8JR1f9HKOxflTOL0=.c1e6cb43-b629-401a-97e4-554a4fca9b50@github.com> Message-ID: <6liehN7xS40W_IYn-WmgXg2u0Gb-Baya4m71KFMeo30=.9f8f21ec-009a-49d0-b37f-a0e8a2a6ed04@github.com> On Wed, 28 May 2025 11:59:16 GMT, Maurizio Cimadamore wrote: >> To be clear: I think using `CoreOp::quoteOp` inside javac is a no-go. Javac has a lot of special code to allow to generate the model for a lambda "in isolation" -- this involves computing all the captures upfront, and then setting up a block with enough `Var` to satisfy the generation of the lambda body. References to captured symbols inside the lambda body are replaced with reference to such generated `Var`s -- this is roughly what `CoreOp::quoteOp` does -- but that method assumes we have an `Op` to begin with -- whereas javac is trying to generate that op as well, so I think what javac does looks ok. >> >> What seems more doable (and desirable IMHO) is to replace `QuotedHelper` with the other function -- `CoreOp::quotedOp` -- but to do this, `CoreOp::quotedOp` needs to relax some of its checks to tolerate (local) `Var` initialized with an (immediately preceding) constant op. > > (I have no objection to this PR going ahead as is -- what I find mildly strange is that we tied this PR with the addition of these two new API functions in `CoreOp`, even though we ultimately don't have a definite answer on how to reuse said functions from the original code in this PR). > @PaulSandoz did you intend for the new methods to be the official way to deal with quoted models both in javac and elsewhere -- or was that just an API suggestion unrelated from the goal of this PR? (I'm a bit confused) The main goal is to replace the use of the interpreter, but we really need to specify what is going on. To achieve that we can define two interrelated methods. The first, `Op -> FuncOp`, that extracts an operation out if its context and quotes it, and the second, FuncOp -> (Op, SequencesSet), that validates and returns the operation that is quoted and the values it captures [*] from which it is possible to create a `Quoted` instance that maps symbolic values to runtime values. It was not entirely clear to me how generally reusable the first might by javac but the specification can still be referred to. The bytecode generator can probably reuse it. [*] normally the function parameters wrapped in vars, but as you noted there might be embedded constants. ------------- PR Review Comment: https://git.openjdk.org/babylon/pull/424#discussion_r2112216751 From mcimadamore at openjdk.org Wed May 28 15:47:12 2025 From: mcimadamore at openjdk.org (Maurizio Cimadamore) Date: Wed, 28 May 2025 15:47:12 GMT Subject: [code-reflection] RFR: Enhance grammar of op attributes Message-ID: This PR enhances the grammar of op attributes, so as to avoid use of "stringy" attributes where possible. Now all attributes that are Java primitives are externalized and re-parsed back automatically. This means that op factories no longer need to test for attribute values of type `String` and then manually parse that string back into the correct primitive, which makes the code more robust. This change caused some reshuffling in the tests -- most notably, when fixing the triton test, the new logic highlighted what was probably an "issue" in the triton transformer (see related comment). ------------- Commit messages: - Update attribute grammar doc in OpParser - Simplify code in ArithMathOps.ConstantOp - Initial push Changes: https://git.openjdk.org/babylon/pull/435/files Webrev: https://webrevs.openjdk.org/?repo=babylon&pr=435&range=00 Stats: 960 lines in 52 files changed: 41 ins; 79 del; 840 mod Patch: https://git.openjdk.org/babylon/pull/435.diff Fetch: git fetch https://git.openjdk.org/babylon.git pull/435/head:pull/435 PR: https://git.openjdk.org/babylon/pull/435 From mcimadamore at openjdk.org Wed May 28 15:47:13 2025 From: mcimadamore at openjdk.org (Maurizio Cimadamore) Date: Wed, 28 May 2025 15:47:13 GMT Subject: [code-reflection] RFR: Enhance grammar of op attributes In-Reply-To: References: Message-ID: <5fgKTTXU_BoOSqtRSmKhn6Pryt5l6qu5JmGsW6KhNdo=.d870047f-092b-4d70-add0-d3480c248639@github.com> On Wed, 28 May 2025 15:28:30 GMT, Maurizio Cimadamore wrote: > This PR enhances the grammar of op attributes, so as to avoid use of "stringy" attributes where possible. Now all attributes that are Java primitives are externalized and re-parsed back automatically. This means that op factories no longer need to test for attribute values of type `String` and then manually parse that string back into the correct primitive, which makes the code more robust. > > This change caused some reshuffling in the tests -- most notably, when fixing the triton test, the new logic highlighted what was probably an "issue" in the triton transformer (see related comment). cr-examples/triton/src/main/java/oracle/code/triton/TritonTransformer.java line 1056: > 1054: b = block.context().getValue(b); > 1055: // Computed result is tensor of floats, regardless of inputs > 1056: Object zero = 0.0f; The previous constant was `0.0` which parses as a `double` constant, not a `float` one. Of course the string representation of both would be the same -- but now that `AttributeWriter` appends the correct suffix, this issue led to weird mismatches in tests. ------------- PR Review Comment: https://git.openjdk.org/babylon/pull/435#discussion_r2112222554 From mcimadamore at openjdk.org Wed May 28 15:50:09 2025 From: mcimadamore at openjdk.org (Maurizio Cimadamore) Date: Wed, 28 May 2025 15:50:09 GMT Subject: [code-reflection] RFR: Replace the use of Interpreter to create Quoted instance with specialized code [v4] In-Reply-To: <6liehN7xS40W_IYn-WmgXg2u0Gb-Baya4m71KFMeo30=.9f8f21ec-009a-49d0-b37f-a0e8a2a6ed04@github.com> References: <7T5xK7Q6lVfKRywCC6wP-6lRMwQ3B1k-REHGoHNd_i0=.4f0e7537-e809-4e7b-857a-e244adfc0c9a@github.com> <2r2EKKymyDf21OdOFR2wtnGG4PF8JR1f9HKOxflTOL0=.c1e6cb43-b629-401a-97e4-554a4fca9b50@github.com> <6liehN7xS40W_IYn-WmgXg2u0Gb-Baya4m71KFMeo30=.9f8f21ec-009a-49d0-b37f-a0e8a2a6ed04@github.com> Message-ID: On Wed, 28 May 2025 15:39:23 GMT, Paul Sandoz wrote: > The main goal is to replace the use of the interpreter, but we really need to specify what is going on Right -- that was my feeling. Then I suppose @mabbay found that the model produced by javac for quoted lambdas does not respect the specification we set forth. Doesn't that defeat the point of having the spec in the first place? ------------- PR Review Comment: https://git.openjdk.org/babylon/pull/424#discussion_r2112227966 From psandoz at openjdk.org Wed May 28 15:54:41 2025 From: psandoz at openjdk.org (Paul Sandoz) Date: Wed, 28 May 2025 15:54:41 GMT Subject: [code-reflection] RFR: Debug appropriate code element toString Message-ID: The `toString` of code elements (op, body, block) can be a little less verbose and better for debugging, especially so for blocks where we can include the index. ------------- Commit messages: - Debug appropriate code element toString Changes: https://git.openjdk.org/babylon/pull/436/files Webrev: https://webrevs.openjdk.org/?repo=babylon&pr=436&range=00 Stats: 20 lines in 3 files changed: 20 ins; 0 del; 0 mod Patch: https://git.openjdk.org/babylon/pull/436.diff Fetch: git fetch https://git.openjdk.org/babylon.git pull/436/head:pull/436 PR: https://git.openjdk.org/babylon/pull/436 From psandoz at openjdk.org Wed May 28 16:00:06 2025 From: psandoz at openjdk.org (Paul Sandoz) Date: Wed, 28 May 2025 16:00:06 GMT Subject: [code-reflection] RFR: Replace the use of Interpreter to create Quoted instance with specialized code [v4] In-Reply-To: References: <7T5xK7Q6lVfKRywCC6wP-6lRMwQ3B1k-REHGoHNd_i0=.4f0e7537-e809-4e7b-857a-e244adfc0c9a@github.com> <2r2EKKymyDf21OdOFR2wtnGG4PF8JR1f9HKOxflTOL0=.c1e6cb43-b629-401a-97e4-554a4fca9b50@github.com> <6liehN7xS40W_IYn-WmgXg2u0Gb-Baya4m71KFMeo30=.9f8f21ec-009a-49d0-b37f-a0e8a2a6ed04@github.com> Message-ID: <_uJuoRieuGSGmoH8koBl3MpgE12e51-1_J88DmYv9j0=.21e6a350-9362-4c0c-9a7b-184d255e2b13@github.com> On Wed, 28 May 2025 15:45:06 GMT, Maurizio Cimadamore wrote: > > The main goal is to replace the use of the interpreter, but we really need to specify what is going on > > Right -- that was my feeling. Then I suppose @mabbay found that the model produced by javac for quoted lambdas does not respect the specification we set forth. Doesn't that defeat the point of having the spec in the first place? I presume you are referring to how javac handles constants? I was expecting we would update the specification and implementation accordingly to allow for those i.e. additional captures can occur that are not function parameters and are the result of a constant operations. So the extract+quote produces a valid model but represents a subset of what is possible. ------------- PR Review Comment: https://git.openjdk.org/babylon/pull/424#discussion_r2112250713 From mcimadamore at openjdk.org Wed May 28 16:05:07 2025 From: mcimadamore at openjdk.org (Maurizio Cimadamore) Date: Wed, 28 May 2025 16:05:07 GMT Subject: [code-reflection] RFR: Debug appropriate code element toString In-Reply-To: References: Message-ID: On Wed, 28 May 2025 15:50:19 GMT, Paul Sandoz wrote: > The `toString` of code elements (op, body, block) can be a little less verbose and better for debugging, especially so for blocks where we can include the index. Marked as reviewed by mcimadamore (Reviewer). ------------- PR Review: https://git.openjdk.org/babylon/pull/436#pullrequestreview-2875657398 From psandoz at openjdk.org Wed May 28 16:13:00 2025 From: psandoz at openjdk.org (Paul Sandoz) Date: Wed, 28 May 2025 16:13:00 GMT Subject: [code-reflection] RFR: Replace the use of Interpreter to create Quoted instance with specialized code [v4] In-Reply-To: <_uJuoRieuGSGmoH8koBl3MpgE12e51-1_J88DmYv9j0=.21e6a350-9362-4c0c-9a7b-184d255e2b13@github.com> References: <7T5xK7Q6lVfKRywCC6wP-6lRMwQ3B1k-REHGoHNd_i0=.4f0e7537-e809-4e7b-857a-e244adfc0c9a@github.com> <2r2EKKymyDf21OdOFR2wtnGG4PF8JR1f9HKOxflTOL0=.c1e6cb43-b629-401a-97e4-554a4fca9b50@github.com> <6liehN7xS40W_IYn-WmgXg2u0Gb-Baya4m71KFMeo30=.9f8f21ec-009a-49d0-b37f-a0e8a2a6ed04@github.com> <_uJuoRieuGSGmoH8koBl3MpgE12e51-1_J88DmYv9j0=.21e6a350-9362-4c0c-9a7b-184d255e2b13@github.com> Message-ID: On Wed, 28 May 2025 15:57:06 GMT, Paul Sandoz wrote: > So the extract+quote produces a valid model but represents a subset of what is possible. Note we could refine this to detect if the captured value is the result of a constant and extract the constant operation. Something to consider later maybe after the first round. ------------- PR Review Comment: https://git.openjdk.org/babylon/pull/424#discussion_r2112275021 From psandoz at openjdk.org Wed May 28 16:17:00 2025 From: psandoz at openjdk.org (Paul Sandoz) Date: Wed, 28 May 2025 16:17:00 GMT Subject: git: openjdk/babylon: code-reflection: Debug appropriate code element toString Message-ID: Changeset: d376169f Branch: code-reflection Author: Paul Sandoz Date: 2025-05-28 16:14:18 +0000 URL: https://git.openjdk.org/babylon/commit/d376169ffc65d17c83cedae8415a0d8be8f261e0 Debug appropriate code element toString Reviewed-by: mcimadamore ! src/jdk.incubator.code/share/classes/jdk/incubator/code/Block.java ! src/jdk.incubator.code/share/classes/jdk/incubator/code/Body.java ! src/jdk.incubator.code/share/classes/jdk/incubator/code/Op.java From psandoz at openjdk.org Wed May 28 16:17:02 2025 From: psandoz at openjdk.org (Paul Sandoz) Date: Wed, 28 May 2025 16:17:02 GMT Subject: [code-reflection] Integrated: Debug appropriate code element toString In-Reply-To: References: Message-ID: On Wed, 28 May 2025 15:50:19 GMT, Paul Sandoz wrote: > The `toString` of code elements (op, body, block) can be a little less verbose and better for debugging, especially so for blocks where we can include the index. This pull request has now been integrated. Changeset: d376169f Author: Paul Sandoz URL: https://git.openjdk.org/babylon/commit/d376169ffc65d17c83cedae8415a0d8be8f261e0 Stats: 20 lines in 3 files changed: 20 ins; 0 del; 0 mod Debug appropriate code element toString Reviewed-by: mcimadamore ------------- PR: https://git.openjdk.org/babylon/pull/436 From psandoz at openjdk.org Wed May 28 16:20:08 2025 From: psandoz at openjdk.org (Paul Sandoz) Date: Wed, 28 May 2025 16:20:08 GMT Subject: [code-reflection] RFR: Enhance grammar of op attributes In-Reply-To: References: Message-ID: On Wed, 28 May 2025 15:28:30 GMT, Maurizio Cimadamore wrote: > This PR enhances the grammar of op attributes, so as to avoid use of "stringy" attributes where possible. Now all attributes that are Java primitives are externalized and re-parsed back automatically. This means that op factories no longer need to test for attribute values of type `String` and then manually parse that string back into the correct primitive, which makes the code more robust. > > This change caused some reshuffling in the tests -- most notably, when fixing the triton test, the new logic highlighted what was probably an "issue" in the triton transformer (see related comment). src/jdk.incubator.code/share/classes/jdk/incubator/code/op/CoreOp.java line 1281: > 1279: > 1280: static Object processConstantValue(TypeElement t, Object value) { > 1281: if (t.equals(JavaType.BOOLEAN) && value instanceof Boolean) { Nice. Can the IDE refactor the body of this method to a switch expression? ------------- PR Review Comment: https://git.openjdk.org/babylon/pull/435#discussion_r2112288529 From psandoz at openjdk.org Wed May 28 16:25:04 2025 From: psandoz at openjdk.org (Paul Sandoz) Date: Wed, 28 May 2025 16:25:04 GMT Subject: [code-reflection] RFR: Enhance grammar of op attributes In-Reply-To: References: Message-ID: <2GfYrOS_WYjA8_bX98Yl5AVccAsqQT2Ovd7IOGAGzBI=.bd560ec6-e9a2-44ba-93e2-41ae47623326@github.com> On Wed, 28 May 2025 15:28:30 GMT, Maurizio Cimadamore wrote: > This PR enhances the grammar of op attributes, so as to avoid use of "stringy" attributes where possible. Now all attributes that are Java primitives are externalized and re-parsed back automatically. This means that op factories no longer need to test for attribute values of type `String` and then manually parse that string back into the correct primitive, which makes the code more robust. > > This change caused some reshuffling in the tests -- most notably, when fixing the triton test, the new logic highlighted what was probably an "issue" in the triton transformer (see related comment). Very good. We are back to a similar position we were at ages ago (expressing various literals), but the architecture is way better. ------------- Marked as reviewed by psandoz (Lead). PR Review: https://git.openjdk.org/babylon/pull/435#pullrequestreview-2875713972 From mcimadamore at openjdk.org Wed May 28 17:11:02 2025 From: mcimadamore at openjdk.org (Maurizio Cimadamore) Date: Wed, 28 May 2025 17:11:02 GMT Subject: [code-reflection] RFR: Enhance grammar of op attributes In-Reply-To: References: Message-ID: On Wed, 28 May 2025 16:17:52 GMT, Paul Sandoz wrote: >> This PR enhances the grammar of op attributes, so as to avoid use of "stringy" attributes where possible. Now all attributes that are Java primitives are externalized and re-parsed back automatically. This means that op factories no longer need to test for attribute values of type `String` and then manually parse that string back into the correct primitive, which makes the code more robust. >> >> This change caused some reshuffling in the tests -- most notably, when fixing the triton test, the new logic highlighted what was probably an "issue" in the triton transformer (see related comment). > > src/jdk.incubator.code/share/classes/jdk/incubator/code/op/CoreOp.java line 1281: > >> 1279: >> 1280: static Object processConstantValue(TypeElement t, Object value) { >> 1281: if (t.equals(JavaType.BOOLEAN) && value instanceof Boolean) { > > Nice. Can the IDE refactor the body of this method to a switch expression? We can't really -- because the switch is on `JavaType` constant (and we don't support constant patterns -- at least yet :-) ). So, at least "as written", we can't easily turn this into a switch. ------------- PR Review Comment: https://git.openjdk.org/babylon/pull/435#discussion_r2112379040 From mcimadamore at openjdk.org Wed May 28 17:15:03 2025 From: mcimadamore at openjdk.org (Maurizio Cimadamore) Date: Wed, 28 May 2025 17:15:03 GMT Subject: [code-reflection] RFR: Replace the use of Interpreter to create Quoted instance with specialized code [v4] In-Reply-To: References: <7T5xK7Q6lVfKRywCC6wP-6lRMwQ3B1k-REHGoHNd_i0=.4f0e7537-e809-4e7b-857a-e244adfc0c9a@github.com> <2r2EKKymyDf21OdOFR2wtnGG4PF8JR1f9HKOxflTOL0=.c1e6cb43-b629-401a-97e4-554a4fca9b50@github.com> <6liehN7xS40W_IYn-WmgXg2u0Gb-Baya4m71KFMeo30=.9f8f21ec-009a-49d0-b37f-a0e8a2a6ed04@github.com> <_uJuoRieuGSGmoH8koBl3MpgE12e51-1_J88DmYv9j0=.21e6a350-9362-4c0c-9a7b-184d255e2b13@github.com> Message-ID: On Wed, 28 May 2025 16:10:22 GMT, Paul Sandoz wrote: >>> > The main goal is to replace the use of the interpreter, but we really need to specify what is going on >>> >>> Right -- that was my feeling. Then I suppose @mabbay found that the model produced by javac for quoted lambdas does not respect the specification we set forth. Doesn't that defeat the point of having the spec in the first place? >> >> I presume you are referring to how javac handles constants? I was expecting we would update the specification and implementation accordingly to allow for those i.e. additional captures can occur that are not function parameters and are the result of a constant operations. So the extract+quote produces a valid model but represents a subset of what is possible. > >> So the extract+quote produces a valid model but represents a subset of what is possible. > > Note we could refine this to detect if the captured value is the result of a constant and extract the constant operation. Something to consider later maybe after the first round. > I was expecting we would update the specification and implementation accordingly to allow for those i.e. additional captures can occur that are not function parameters and are the result of a constant operations. That was my expectation as well -- e.g. concretely, I think our goal for this PR is to have `QuotedHelper` use the new methods on the model javac generates, correct? ------------- PR Review Comment: https://git.openjdk.org/babylon/pull/424#discussion_r2112384021 From psandoz at openjdk.org Wed May 28 17:31:59 2025 From: psandoz at openjdk.org (Paul Sandoz) Date: Wed, 28 May 2025 17:31:59 GMT Subject: [code-reflection] RFR: Replace the use of Interpreter to create Quoted instance with specialized code [v4] In-Reply-To: References: <7T5xK7Q6lVfKRywCC6wP-6lRMwQ3B1k-REHGoHNd_i0=.4f0e7537-e809-4e7b-857a-e244adfc0c9a@github.com> <2r2EKKymyDf21OdOFR2wtnGG4PF8JR1f9HKOxflTOL0=.c1e6cb43-b629-401a-97e4-554a4fca9b50@github.com> <6liehN7xS40W_IYn-WmgXg2u0Gb-Baya4m71KFMeo30=.9f8f21ec-009a-49d0-b37f-a0e8a2a6ed04@github.com> <_uJuoRieuGSGmoH8koBl3MpgE12e51-1_J88DmYv9j0=.21e6a350-9362-4c0c-9a7b-184d255e2b13@github.com> Message-ID: On Wed, 28 May 2025 17:12:05 GMT, Maurizio Cimadamore wrote: > e.g. concretely, I think our goal for this PR is to have QuotedHelper use the new methods on the model javac generates, correct Yes. ------------- PR Review Comment: https://git.openjdk.org/babylon/pull/424#discussion_r2112415667 From mcimadamore at openjdk.org Thu May 29 16:42:46 2025 From: mcimadamore at openjdk.org (Maurizio Cimadamore) Date: Thu, 29 May 2025 16:42:46 GMT Subject: [code-reflection] RFR: Enhance grammar of op attributes [v2] In-Reply-To: References: Message-ID: > This PR enhances the grammar of op attributes, so as to avoid use of "stringy" attributes where possible. Now all attributes that are Java primitives are externalized and re-parsed back automatically. This means that op factories no longer need to test for attribute values of type `String` and then manually parse that string back into the correct primitive, which makes the code more robust. > > This change caused some reshuffling in the tests -- most notably, when fixing the triton test, the new logic highlighted what was probably an "issue" in the triton transformer (see related comment). Maurizio Cimadamore has updated the pull request incrementally with one additional commit since the last revision: Simplify code and add comment ------------- Changes: - all: https://git.openjdk.org/babylon/pull/435/files - new: https://git.openjdk.org/babylon/pull/435/files/3db61a94..b3139a49 Webrevs: - full: https://webrevs.openjdk.org/?repo=babylon&pr=435&range=01 - incr: https://webrevs.openjdk.org/?repo=babylon&pr=435&range=00-01 Stats: 5 lines in 1 file changed: 3 ins; 2 del; 0 mod Patch: https://git.openjdk.org/babylon/pull/435.diff Fetch: git fetch https://git.openjdk.org/babylon.git pull/435/head:pull/435 PR: https://git.openjdk.org/babylon/pull/435 From asotona at openjdk.org Thu May 29 17:06:46 2025 From: asotona at openjdk.org (Adam Sotona) Date: Thu, 29 May 2025 17:06:46 GMT Subject: [code-reflection] RFR: Onnx fixes to support LLM + Onnx GenAI FFM API [v2] In-Reply-To: References: Message-ID: > - Manually added some contrib ops > - Implemented support for records/tuples in arguments and as initializers > - Fixed declaration of op set imports in functions > - Generated FFM interface to Onnx GenAI API > - Onnx op domain made a part of the op name > - Added `forceInline` switch to OnnxTransformer Adam Sotona has updated the pull request incrementally with one additional commit since the last revision: OnnxProtoBuilder accepts explicit value names to support Onnx GenAI API naming constraints OnnxTransformer produces parameters (inputs) and return tuple elements (outputs) value name mapping ------------- Changes: - all: https://git.openjdk.org/babylon/pull/429/files - new: https://git.openjdk.org/babylon/pull/429/files/5b12215e..36d1eff6 Webrevs: - full: https://webrevs.openjdk.org/?repo=babylon&pr=429&range=01 - incr: https://webrevs.openjdk.org/?repo=babylon&pr=429&range=00-01 Stats: 79 lines in 2 files changed: 60 ins; 0 del; 19 mod Patch: https://git.openjdk.org/babylon/pull/429.diff Fetch: git fetch https://git.openjdk.org/babylon.git pull/429/head:pull/429 PR: https://git.openjdk.org/babylon/pull/429 From mcimadamore at openjdk.org Thu May 29 17:12:21 2025 From: mcimadamore at openjdk.org (Maurizio Cimadamore) Date: Thu, 29 May 2025 17:12:21 GMT Subject: git: openjdk/babylon: code-reflection: Enhance grammar of op attributes Message-ID: <2fbc4b46-af64-4af3-ae8c-d2e5e972bb0e@openjdk.org> Changeset: 66fd88ad Branch: code-reflection Author: Maurizio Cimadamore Date: 2025-05-29 17:10:29 +0000 URL: https://git.openjdk.org/babylon/commit/66fd88ad919193d7cb5d706c1d306d9cae54b781 Enhance grammar of op attributes Reviewed-by: psandoz ! cr-examples/triton/src/main/java/oracle/code/triton/ArithMathOps.java ! cr-examples/triton/src/main/java/oracle/code/triton/TritonOps.java ! cr-examples/triton/src/main/java/oracle/code/triton/TritonTransformer.java ! cr-examples/triton/src/test/java/oracle/code/triton/TestAddKernel.java ! cr-examples/triton/src/test/java/oracle/code/triton/TestBroadcast.java ! cr-examples/triton/src/test/java/oracle/code/triton/TestCdiv.java ! cr-examples/triton/src/test/java/oracle/code/triton/TestCountedLoop.java ! cr-examples/triton/src/test/java/oracle/code/triton/TestMatrix.java ! cr-examples/triton/src/test/java/oracle/code/triton/TestMatrixFp16.java ! cr-examples/triton/src/test/java/oracle/code/triton/TestSoftMax.java ! cr-examples/triton/src/test/java/oracle/code/triton/TestZeros.java ! src/jdk.incubator.code/share/classes/jdk/incubator/code/op/CoreOp.java ! src/jdk.incubator.code/share/classes/jdk/incubator/code/parser/OpParser.java ! src/jdk.incubator.code/share/classes/jdk/incubator/code/writer/OpWriter.java ! test/jdk/java/lang/reflect/code/TestLiveness.java ! test/jdk/java/lang/reflect/code/TestNormalizeBlocksTransformer.java ! test/jdk/java/lang/reflect/code/lower/TestLoop.java ! test/jdk/java/lang/reflect/code/lower/TestSynchronized.java ! test/jdk/java/lang/reflect/code/parser/TestParse.java ! test/langtools/tools/javac/reflect/ArrayAccessTest.java ! test/langtools/tools/javac/reflect/AssertTest.java ! test/langtools/tools/javac/reflect/BinopTest.java ! test/langtools/tools/javac/reflect/BlockTest.java ! test/langtools/tools/javac/reflect/BoxingConversionTest.java ! test/langtools/tools/javac/reflect/BreakContinueTest.java ! test/langtools/tools/javac/reflect/CastInstanceOfTest.java ! test/langtools/tools/javac/reflect/ConditionalAndOrTest.java ! test/langtools/tools/javac/reflect/ConstantsTest.java ! test/langtools/tools/javac/reflect/DenotableTypesTest.java ! test/langtools/tools/javac/reflect/FieldAccessTest.java ! test/langtools/tools/javac/reflect/ForLoopTest.java ! test/langtools/tools/javac/reflect/IfTest.java ! test/langtools/tools/javac/reflect/ImplicitConversionTest.java ! test/langtools/tools/javac/reflect/LambdaTest.java ! test/langtools/tools/javac/reflect/LocalVarTest.java ! test/langtools/tools/javac/reflect/MethodCallTest.java ! test/langtools/tools/javac/reflect/NewArrayTest.java ! test/langtools/tools/javac/reflect/NewTest.java ! test/langtools/tools/javac/reflect/NullTest.java ! test/langtools/tools/javac/reflect/PatternsTest.java ! test/langtools/tools/javac/reflect/QuotableIntersectionTest.java ! test/langtools/tools/javac/reflect/QuotableSubtypeTest.java ! test/langtools/tools/javac/reflect/QuotedTest.java ! test/langtools/tools/javac/reflect/SuperTest.java ! test/langtools/tools/javac/reflect/SwitchExpressionTest.java ! test/langtools/tools/javac/reflect/SwitchExpressionTest2.java ! test/langtools/tools/javac/reflect/SwitchStatementTest.java ! test/langtools/tools/javac/reflect/SynchronizedTest.java ! test/langtools/tools/javac/reflect/TryTest.java ! test/langtools/tools/javac/reflect/UnaryopTest.java ! test/langtools/tools/javac/reflect/UnreachableTest.java ! test/langtools/tools/javac/reflect/WhileLoopTest.java From mcimadamore at openjdk.org Thu May 29 17:13:06 2025 From: mcimadamore at openjdk.org (Maurizio Cimadamore) Date: Thu, 29 May 2025 17:13:06 GMT Subject: [code-reflection] Integrated: Enhance grammar of op attributes In-Reply-To: References: Message-ID: <0eU77jCyhnBQz4KYNrrebcbrvx8OasqrtS3dB7DRxCk=.0a21034b-87c2-487e-b8ac-4eda54e98577@github.com> On Wed, 28 May 2025 15:28:30 GMT, Maurizio Cimadamore wrote: > This PR enhances the grammar of op attributes, so as to avoid use of "stringy" attributes where possible. Now all attributes that are Java primitives are externalized and re-parsed back automatically. This means that op factories no longer need to test for attribute values of type `String` and then manually parse that string back into the correct primitive, which makes the code more robust. > > This change caused some reshuffling in the tests -- most notably, when fixing the triton test, the new logic highlighted what was probably an "issue" in the triton transformer (see related comment). This pull request has now been integrated. Changeset: 66fd88ad Author: Maurizio Cimadamore URL: https://git.openjdk.org/babylon/commit/66fd88ad919193d7cb5d706c1d306d9cae54b781 Stats: 965 lines in 52 files changed: 44 ins; 81 del; 840 mod Enhance grammar of op attributes Reviewed-by: psandoz ------------- PR: https://git.openjdk.org/babylon/pull/435 From mabbay at openjdk.org Thu May 29 23:03:24 2025 From: mabbay at openjdk.org (Mourad Abbay) Date: Thu, 29 May 2025 23:03:24 GMT Subject: [code-reflection] RFR: Replace the use of Interpreter to create Quoted instance with specialized code [v9] In-Reply-To: References: Message-ID: <07q8C_1RQ9GxivbiwmoHirs_N8pjsJzh7Ov4Cy1EqyM=.ab8ed3c8-aedf-4f67-8cb8-a0a54d086493@github.com> > Creating Quoted instance was done by invoking the Interpreter. The Interpreter is too general and if someone could somehow inject any code model we will interpret that as well. This PR replace the use of the interpreter with specialized code. Mourad Abbay has updated the pull request incrementally with one additional commit since the last revision: Update CoreOp.quotedOp to reuse implementation of ByteCodeGenerator.quote. ------------- Changes: - all: https://git.openjdk.org/babylon/pull/424/files - new: https://git.openjdk.org/babylon/pull/424/files/9f3db051..02f272c1 Webrevs: - full: https://webrevs.openjdk.org/?repo=babylon&pr=424&range=08 - incr: https://webrevs.openjdk.org/?repo=babylon&pr=424&range=07-08 Stats: 66 lines in 1 file changed: 1 ins; 36 del; 29 mod Patch: https://git.openjdk.org/babylon/pull/424.diff Fetch: git fetch https://git.openjdk.org/babylon.git pull/424/head:pull/424 PR: https://git.openjdk.org/babylon/pull/424 From mabbay at openjdk.org Thu May 29 23:44:37 2025 From: mabbay at openjdk.org (Mourad Abbay) Date: Thu, 29 May 2025 23:44:37 GMT Subject: [code-reflection] RFR: Replace the use of Interpreter to create Quoted instance with specialized code [v10] In-Reply-To: References: Message-ID: > Creating Quoted instance was done by invoking the Interpreter. The Interpreter is too general and if someone could somehow inject any code model we will interpret that as well. This PR replace the use of the interpreter with specialized code. Mourad Abbay has updated the pull request incrementally with one additional commit since the last revision: loosen the checks ------------- Changes: - all: https://git.openjdk.org/babylon/pull/424/files - new: https://git.openjdk.org/babylon/pull/424/files/02f272c1..2c3d805a Webrevs: - full: https://webrevs.openjdk.org/?repo=babylon&pr=424&range=09 - incr: https://webrevs.openjdk.org/?repo=babylon&pr=424&range=08-09 Stats: 60 lines in 2 files changed: 13 ins; 29 del; 18 mod Patch: https://git.openjdk.org/babylon/pull/424.diff Fetch: git fetch https://git.openjdk.org/babylon.git pull/424/head:pull/424 PR: https://git.openjdk.org/babylon/pull/424 From mabbay at openjdk.org Fri May 30 14:55:30 2025 From: mabbay at openjdk.org (Mourad Abbay) Date: Fri, 30 May 2025 14:55:30 GMT Subject: [code-reflection] RFR: Replace the use of Interpreter to create Quoted instance with specialized code [v11] In-Reply-To: References: Message-ID: <9vAhpiJHLkNxQGDmI7dpIfg_j7DzCUv-dYICQ9LX8OM=.b9a67755-eba1-4259-a872-6e025dbca1b6@github.com> > Creating Quoted instance was done by invoking the Interpreter. The Interpreter is too general and if someone could somehow inject any code model we will interpret that as well. This PR replace the use of the interpreter with specialized code. Mourad Abbay has updated the pull request incrementally with two additional commits since the last revision: - Get rid of assert statements - Add checks ------------- Changes: - all: https://git.openjdk.org/babylon/pull/424/files - new: https://git.openjdk.org/babylon/pull/424/files/2c3d805a..01961184 Webrevs: - full: https://webrevs.openjdk.org/?repo=babylon&pr=424&range=10 - incr: https://webrevs.openjdk.org/?repo=babylon&pr=424&range=09-10 Stats: 66 lines in 2 files changed: 61 ins; 0 del; 5 mod Patch: https://git.openjdk.org/babylon/pull/424.diff Fetch: git fetch https://git.openjdk.org/babylon.git pull/424/head:pull/424 PR: https://git.openjdk.org/babylon/pull/424 From mabbay at openjdk.org Fri May 30 23:29:25 2025 From: mabbay at openjdk.org (Mourad Abbay) Date: Fri, 30 May 2025 23:29:25 GMT Subject: [code-reflection] RFR: Replace the use of Interpreter to create Quoted instance with specialized code [v12] In-Reply-To: References: Message-ID: > Creating Quoted instance was done by invoking the Interpreter. The Interpreter is too general and if someone could somehow inject any code model we will interpret that as well. This PR replace the use of the interpreter with specialized code. Mourad Abbay has updated the pull request incrementally with two additional commits since the last revision: - Remove unused methods - Update QuotedHelper.makeQuoted to use CoreOp.quotedOp (that contains checks) ------------- Changes: - all: https://git.openjdk.org/babylon/pull/424/files - new: https://git.openjdk.org/babylon/pull/424/files/01961184..04ab281d Webrevs: - full: https://webrevs.openjdk.org/?repo=babylon&pr=424&range=11 - incr: https://webrevs.openjdk.org/?repo=babylon&pr=424&range=10-11 Stats: 40 lines in 2 files changed: 8 ins; 16 del; 16 mod Patch: https://git.openjdk.org/babylon/pull/424.diff Fetch: git fetch https://git.openjdk.org/babylon.git pull/424/head:pull/424 PR: https://git.openjdk.org/babylon/pull/424 From mabbay at openjdk.org Sat May 31 03:13:12 2025 From: mabbay at openjdk.org (Mourad Abbay) Date: Sat, 31 May 2025 03:13:12 GMT Subject: [code-reflection] RFR: Replace the use of Interpreter to create Quoted instance with specialized code [v13] In-Reply-To: References: Message-ID: > Creating Quoted instance was done by invoking the Interpreter. The Interpreter is too general and if someone could somehow inject any code model we will interpret that as well. This PR replace the use of the interpreter with specialized code. Mourad Abbay has updated the pull request incrementally with one additional commit since the last revision: Add error messages ------------- Changes: - all: https://git.openjdk.org/babylon/pull/424/files - new: https://git.openjdk.org/babylon/pull/424/files/04ab281d..93c1f3a8 Webrevs: - full: https://webrevs.openjdk.org/?repo=babylon&pr=424&range=12 - incr: https://webrevs.openjdk.org/?repo=babylon&pr=424&range=11-12 Stats: 15 lines in 1 file changed: 4 ins; 0 del; 11 mod Patch: https://git.openjdk.org/babylon/pull/424.diff Fetch: git fetch https://git.openjdk.org/babylon.git pull/424/head:pull/424 PR: https://git.openjdk.org/babylon/pull/424