From mabbay at openjdk.org  Sat Mar  1 00:16:25 2025
From: mabbay at openjdk.org (Mourad Abbay)
Date: Sat, 1 Mar 2025 00:16:25 GMT
Subject: [code-reflection] RFR: Support storing the code that builds the
 code model [v13]
In-Reply-To: <dexv8sQV_NcJsJIgMopd_M4wSWoZya9dU0ACwDtEL0E=.4ac0b80f-24fb-440f-a18d-bbfd07a681bc@github.com>
References: <dexv8sQV_NcJsJIgMopd_M4wSWoZya9dU0ACwDtEL0E=.4ac0b80f-24fb-440f-a18d-bbfd07a681bc@github.com>
Message-ID: <QYNsQWAlhofLGp3ASP9RZdRKaLQaLo37tnfXyT0gs9E=.56cf8369-768c-4de5-a434-058f513b0f34@github.com>

> In this PR we allow the user to decide how to store the code model. The first option is `TEXT`, if this is selected, we store the textual representation of the code model. The second option is `CODE_BUILDR`, if this is selected, we store the code that builds the code model. All work done here is around the second option, because the first option is already supported.

Mourad Abbay has updated the pull request incrementally with one additional commit since the last revision:

  Fix almost all test failures of SwitchExpressionTest2 (one remaining)

-------------

Changes:
  - all: https://git.openjdk.org/babylon/pull/305/files
  - new: https://git.openjdk.org/babylon/pull/305/files/a1d134c3..5f02f73e

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=babylon&pr=305&range=12
 - incr: https://webrevs.openjdk.org/?repo=babylon&pr=305&range=11-12

  Stats: 21 lines in 2 files changed: 20 ins; 0 del; 1 mod
  Patch: https://git.openjdk.org/babylon/pull/305.diff
  Fetch: git fetch https://git.openjdk.org/babylon.git pull/305/head:pull/305

PR: https://git.openjdk.org/babylon/pull/305

From asotona at openjdk.org  Sun Mar  2 14:58:05 2025
From: asotona at openjdk.org (Adam Sotona)
Date: Sun, 2 Mar 2025 14:58:05 GMT
Subject: [code-reflection] RFR: Model lifetimes of onnx session-related
 objects more explicitly
In-Reply-To: <aZZu9PNMaJT-9_gw8beTFcUO1r5IkAVKV0w4SDudTk8=.4dec9faf-3708-47cb-8cdf-5ab45e00f35c@github.com>
References: <s1RyI3MmwaIiMa3H__eLqjIAHbGb058ONRI3UZVtUkY=.4fd6ec15-bf2b-4f10-b168-c7d9d0dc23ed@github.com>
 <36W4fRHm2uq3icp4f1mc3Yi73d_a4_zrLI7ZXBOipsU=.5051d503-b012-4c11-9230-3cce24bc94a2@github.com>
 <aZZu9PNMaJT-9_gw8beTFcUO1r5IkAVKV0w4SDudTk8=.4dec9faf-3708-47cb-8cdf-5ab45e00f35c@github.com>
Message-ID: <1LmqgYmR4xi_zElwPTJfXqR9XM5mnWcpR1lvetay6TU=.9cbb6b35-18df-42be-ab21-897ce1158f06@github.com>

On Fri, 28 Feb 2025 22:21:30 GMT, Maurizio Cimadamore <mcimadamore at openjdk.org> wrote:

> I think you are on to something here. I am wondering if we can refame it like this:
> 
> ```
> try (OnnxRuntime or = OnnxRuntime.of(MethodHandles.lookup(), Area.ofConfined()) {
>   or.execute(() -> cnn(imageTensor))
> }
> ```
> 
> ?
> 
> I realize there can be only one runtime instance of a native ONNX runtime but i don't think we should be beholden to that constraint regarding the scoping of executing one or more models and with tensor memory under same lookup and arena (and even execution mode). Maybe there is a better name than `OnnxRuntime` to avoid any confusion.

We can do that, however it requires to involve the same (and for sure the only one) runtime instance to create all tensors (including the imageTensor in the fragment above). The runtime must be the only one, however tensors can be created anywhere and many models/sessions executed also.

-------------

PR Comment: https://git.openjdk.org/babylon/pull/332#issuecomment-2692768714

From asotona at openjdk.org  Sun Mar  2 15:04:02 2025
From: asotona at openjdk.org (Adam Sotona)
Date: Sun, 2 Mar 2025 15:04:02 GMT
Subject: [code-reflection] RFR: Model lifetimes of onnx session-related
 objects more explicitly
In-Reply-To: <s1RyI3MmwaIiMa3H__eLqjIAHbGb058ONRI3UZVtUkY=.4fd6ec15-bf2b-4f10-b168-c7d9d0dc23ed@github.com>
References: <s1RyI3MmwaIiMa3H__eLqjIAHbGb058ONRI3UZVtUkY=.4fd6ec15-bf2b-4f10-b168-c7d9d0dc23ed@github.com>
Message-ID: <BNAYB2Z26uKX7i1qUgxMhbIUaebHgTXTWmfCe_x6AaQ=.8e3599e9-8361-4363-bd65-4f76f807aedb@github.com>

On Fri, 28 Feb 2025 12:42:24 GMT, Maurizio Cimadamore <mcimadamore at openjdk.org> wrote:

> The class representing an onnx session is auto closeable. But, in the current code, a session is closed immediately after its `run` method is called. This is problematic because a session returns some ORTValues (tensors) which also need to be freed, but that cannot be freed immediately after calling `run` (as they need to be used by clients).
> 
> To address this problem, I tweaked the session code to accept an external arena. All the allocation of session-related data structures now happens using that external arena. This means that the client can now be in charge of managing the lifetime of a session (see changes to MNIST demo).
> 
> To test, I tweaked the MNIST code to do 10K iterations on each button pressed. Predictably, a single button pressed resulted in over 3g of memory being leaked. With these changes the memory arrives at ~400K (there is still some minor leak, but not sure worth pushing more).
> 
> If the changes to the demo are not deemed good, I can withdraw this PR -- I mostly wanted to capture the result of my exploration somewhere.

> > I think you are on to something here. I am wondering if we can refame it like this:
> > ```
> > try (OnnxRuntime or = OnnxRuntime.of(MethodHandles.lookup(), Area.ofConfined()) {
> >   or.execute(() -> cnn(imageTensor))
> > }
> > ```
> > 
> > 
> >     
> >       
> >     
> > 
> >       
> >     
> > 
> >     
> >   
> > ?
> > I realize there can be only one runtime instance of a native ONNX runtime but i don't think we should be beholden to that constraint regarding the scoping of executing one or more models and with tensor memory under same lookup and arena (and even execution mode). Maybe there is a better name than `OnnxRuntime` to avoid any confusion.
> 
> I think you are correct in that the high-level API we want doesn't really need to expose all this ONNX "cruft" (e.g. it seems neither high-level nor low-level enough). From the perspective of the Java client, there should be some kind of ONNX "session" (which might, or might not be related to the ONNX session struct in the C API) -- which has a user-defined lifetime. The only interesting method in this session object is "run" and when you "close", the lifetime ends, and all the resources are garbage collected.
> 
> Maybe we can even do something like:
> 
> ```java
> try (OnnxSession onnx = OnnxSession.of()) {
>   or.execute(lookup, () -> cnn(imageTensor))
> }
> ```
> 
> And hide the confined arena creation inside the API alltogether. (I'm not sure where `lookup` goes).

`() -> cnn(imageTensor)` produces a model, which is a session.

I would not recommend to call it a session nor a runtime if we want to create a layer between the mandatory singleton runtime and per-model sessions. 
If we want to match the Onnx API, it should be called OnnxEnvironment, it is pre-configured to run multiple sessions and it is not necessary to create a tensor.

-------------

PR Comment: https://git.openjdk.org/babylon/pull/332#issuecomment-2692770903

From asotona at openjdk.org  Sun Mar  2 15:25:09 2025
From: asotona at openjdk.org (Adam Sotona)
Date: Sun, 2 Mar 2025 15:25:09 GMT
Subject: [code-reflection] RFR: Model lifetimes of onnx session-related
 objects more explicitly
In-Reply-To: <s1RyI3MmwaIiMa3H__eLqjIAHbGb058ONRI3UZVtUkY=.4fd6ec15-bf2b-4f10-b168-c7d9d0dc23ed@github.com>
References: <s1RyI3MmwaIiMa3H__eLqjIAHbGb058ONRI3UZVtUkY=.4fd6ec15-bf2b-4f10-b168-c7d9d0dc23ed@github.com>
Message-ID: <QLbBRgHS8tV8Wv3uTZ-KzMTA5038pV_RCrXUyLIFGfk=.1f184469-9227-45b7-8477-d0016a4c101d@github.com>

On Fri, 28 Feb 2025 12:42:24 GMT, Maurizio Cimadamore <mcimadamore at openjdk.org> wrote:

> The class representing an onnx session is auto closeable. But, in the current code, a session is closed immediately after its `run` method is called. This is problematic because a session returns some ORTValues (tensors) which also need to be freed, but that cannot be freed immediately after calling `run` (as they need to be used by clients).
> 
> To address this problem, I tweaked the session code to accept an external arena. All the allocation of session-related data structures now happens using that external arena. This means that the client can now be in charge of managing the lifetime of a session (see changes to MNIST demo).
> 
> To test, I tweaked the MNIST code to do 10K iterations on each button pressed. Predictably, a single button pressed resulted in over 3g of memory being leaked. With these changes the memory arrives at ~400K (there is still some minor leak, but not sure worth pushing more).
> 
> If the changes to the demo are not deemed good, I can withdraw this PR -- I mostly wanted to capture the result of my exploration somewhere.

cr-examples/onnx/src/test/java/oracle/code/onnx/MNISTDemo.java line 151:

> 149:                     var imageTensor = Tensor.ofShape(new long[]{1, 1, IMAGE_SIZE, IMAGE_SIZE}, imageData);
> 150: 
> 151:                     try (Arena onnxSession = Arena.ofConfined()) {

Please don't call it a session. Onnx session is created below during the execution.

cr-examples/onnx/src/test/java/oracle/code/onnx/RuntimeTest.java line 18:

> 16:     public void test() throws Exception {
> 17:         var ort = OnnxRuntime.getInstance();
> 18:         try (Arena sessionArena = Arena.ofConfined()) {

The same naming confusion is here. Below are created two sessions.

-------------

PR Review Comment: https://git.openjdk.org/babylon/pull/332#discussion_r1976652470
PR Review Comment: https://git.openjdk.org/babylon/pull/332#discussion_r1976652723

From mabbay at openjdk.org  Mon Mar  3 00:19:41 2025
From: mabbay at openjdk.org (Mourad Abbay)
Date: Mon, 3 Mar 2025 00:19:41 GMT
Subject: [code-reflection] RFR: Support storing the code that builds the
 code model [v14]
In-Reply-To: <dexv8sQV_NcJsJIgMopd_M4wSWoZya9dU0ACwDtEL0E=.4ac0b80f-24fb-440f-a18d-bbfd07a681bc@github.com>
References: <dexv8sQV_NcJsJIgMopd_M4wSWoZya9dU0ACwDtEL0E=.4ac0b80f-24fb-440f-a18d-bbfd07a681bc@github.com>
Message-ID: <QOL8mMvsrJ1lMuUnmBsZrx5tabYiQcPOe_Rq9QlbuZU=.ac03a120-b663-4097-8dfa-80cdab9f94c1@github.com>

> In this PR we allow the user to decide how to store the code model. The first option is `TEXT`, if this is selected, we store the textual representation of the code model. The second option is `CODE_BUILDR`, if this is selected, we store the code that builds the code model. All work done here is around the second option, because the first option is already supported.

Mourad Abbay has updated the pull request incrementally with one additional commit since the last revision:

  Fix the remaining test failures of SwitchExpressionTest2

-------------

Changes:
  - all: https://git.openjdk.org/babylon/pull/305/files
  - new: https://git.openjdk.org/babylon/pull/305/files/5f02f73e..c17be6b6

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=babylon&pr=305&range=13
 - incr: https://webrevs.openjdk.org/?repo=babylon&pr=305&range=12-13

  Stats: 2 lines in 2 files changed: 0 ins; 0 del; 2 mod
  Patch: https://git.openjdk.org/babylon/pull/305.diff
  Fetch: git fetch https://git.openjdk.org/babylon.git pull/305/head:pull/305

PR: https://git.openjdk.org/babylon/pull/305

From mcimadamore at openjdk.org  Mon Mar  3 09:47:07 2025
From: mcimadamore at openjdk.org (Maurizio Cimadamore)
Date: Mon, 3 Mar 2025 09:47:07 GMT
Subject: [code-reflection] RFR: Model lifetimes of onnx session-related
 objects more explicitly
In-Reply-To: <QLbBRgHS8tV8Wv3uTZ-KzMTA5038pV_RCrXUyLIFGfk=.1f184469-9227-45b7-8477-d0016a4c101d@github.com>
References: <s1RyI3MmwaIiMa3H__eLqjIAHbGb058ONRI3UZVtUkY=.4fd6ec15-bf2b-4f10-b168-c7d9d0dc23ed@github.com>
 <QLbBRgHS8tV8Wv3uTZ-KzMTA5038pV_RCrXUyLIFGfk=.1f184469-9227-45b7-8477-d0016a4c101d@github.com>
Message-ID: <VkSguuPJo-2RqULvFq67UwbuIhYXLmvHkwfJCvEmnK4=.cbb3441a-de4f-46b6-8eed-352a5e285ee2@github.com>

On Sun, 2 Mar 2025 15:20:40 GMT, Adam Sotona <asotona at openjdk.org> wrote:

>> The class representing an onnx session is auto closeable. But, in the current code, a session is closed immediately after its `run` method is called. This is problematic because a session returns some ORTValues (tensors) which also need to be freed, but that cannot be freed immediately after calling `run` (as they need to be used by clients).
>> 
>> To address this problem, I tweaked the session code to accept an external arena. All the allocation of session-related data structures now happens using that external arena. This means that the client can now be in charge of managing the lifetime of a session (see changes to MNIST demo).
>> 
>> To test, I tweaked the MNIST code to do 10K iterations on each button pressed. Predictably, a single button pressed resulted in over 3g of memory being leaked. With these changes the memory arrives at ~400K (there is still some minor leak, but not sure worth pushing more).
>> 
>> If the changes to the demo are not deemed good, I can withdraw this PR -- I mostly wanted to capture the result of my exploration somewhere.
>
> cr-examples/onnx/src/test/java/oracle/code/onnx/MNISTDemo.java line 151:
> 
>> 149:                     var imageTensor = Tensor.ofShape(new long[]{1, 1, IMAGE_SIZE, IMAGE_SIZE}, imageData);
>> 150: 
>> 151:                     try (Arena onnxSession = Arena.ofConfined()) {
> 
> Please don't call it a session. Onnx session is created below during the execution.

I can surely change the name -- but note that the MNISTDemo code does NOT create an onnx session explicitly -- not does it care whether a session or an env is constructed under the hood.

I believe the question here is what kind of high-level API would it make sense for babylon users to interact with -- which is a separate question from "how should Java bindings for ONNX look like". It seems we're going in a direction (not just in this PR, but in the last 2-3) where we keep hiding more and more of the ONNX stuff from the user facing code, and we just want to provide a simple abstraction to execute a model (expressed as a reflectable method).

> The same naming confusion is here. Below are created two sessions.

Well, this test creates an onnx session. The session constructor now takes an arena (the arena for the session) -- so the test now also creates an arena for that session and calls it "sessionArena".  I doesn't seem too confusing to me -- IMHO what you are really objecting to is the need of creating two things -- rather than the name?

-------------

PR Review Comment: https://git.openjdk.org/babylon/pull/332#discussion_r1977177840
PR Review Comment: https://git.openjdk.org/babylon/pull/332#discussion_r1977182923

From mcimadamore at openjdk.org  Mon Mar  3 09:47:07 2025
From: mcimadamore at openjdk.org (Maurizio Cimadamore)
Date: Mon, 3 Mar 2025 09:47:07 GMT
Subject: [code-reflection] RFR: Model lifetimes of onnx session-related
 objects more explicitly
In-Reply-To: <VkSguuPJo-2RqULvFq67UwbuIhYXLmvHkwfJCvEmnK4=.cbb3441a-de4f-46b6-8eed-352a5e285ee2@github.com>
References: <s1RyI3MmwaIiMa3H__eLqjIAHbGb058ONRI3UZVtUkY=.4fd6ec15-bf2b-4f10-b168-c7d9d0dc23ed@github.com>
 <QLbBRgHS8tV8Wv3uTZ-KzMTA5038pV_RCrXUyLIFGfk=.1f184469-9227-45b7-8477-d0016a4c101d@github.com>
 <VkSguuPJo-2RqULvFq67UwbuIhYXLmvHkwfJCvEmnK4=.cbb3441a-de4f-46b6-8eed-352a5e285ee2@github.com>
Message-ID: <3nBw2IHNdH8bvmYvap-7zl3nwMO5TIBQ2CpAFtCJh9U=.c0cb071b-cb94-44dc-8770-6dab53d90efc@github.com>

On Mon, 3 Mar 2025 09:39:17 GMT, Maurizio Cimadamore <mcimadamore at openjdk.org> wrote:

> I believe the question here is what kind of high-level API would it make sense for babylon users to interact with -- which is a separate question from "how should Java bindings for ONNX look like"

(at least that's how I interpreted @PaulSandoz comment: https://github.com/openjdk/babylon/pull/332#issuecomment-2691545090)

-------------

PR Review Comment: https://git.openjdk.org/babylon/pull/332#discussion_r1977187114

From mcimadamore at openjdk.org  Mon Mar  3 09:53:03 2025
From: mcimadamore at openjdk.org (Maurizio Cimadamore)
Date: Mon, 3 Mar 2025 09:53:03 GMT
Subject: [code-reflection] RFR: Model lifetimes of onnx session-related
 objects more explicitly
In-Reply-To: <s1RyI3MmwaIiMa3H__eLqjIAHbGb058ONRI3UZVtUkY=.4fd6ec15-bf2b-4f10-b168-c7d9d0dc23ed@github.com>
References: <s1RyI3MmwaIiMa3H__eLqjIAHbGb058ONRI3UZVtUkY=.4fd6ec15-bf2b-4f10-b168-c7d9d0dc23ed@github.com>
Message-ID: <ycOZZ03lvi4_b2d5aiKS_q3oo5I_CQcak5GaOaNdmDg=.ae32c17a-e806-49b1-986a-7d129d151b81@github.com>

On Fri, 28 Feb 2025 12:42:24 GMT, Maurizio Cimadamore <mcimadamore at openjdk.org> wrote:

> The class representing an onnx session is auto closeable. But, in the current code, a session is closed immediately after its `run` method is called. This is problematic because a session returns some ORTValues (tensors) which also need to be freed, but that cannot be freed immediately after calling `run` (as they need to be used by clients).
> 
> To address this problem, I tweaked the session code to accept an external arena. All the allocation of session-related data structures now happens using that external arena. This means that the client can now be in charge of managing the lifetime of a session (see changes to MNIST demo).
> 
> To test, I tweaked the MNIST code to do 10K iterations on each button pressed. Predictably, a single button pressed resulted in over 3g of memory being leaked. With these changes the memory arrives at ~400K (there is still some minor leak, but not sure worth pushing more).
> 
> If the changes to the demo are not deemed good, I can withdraw this PR -- I mostly wanted to capture the result of my exploration somewhere.

A possible way forward:
* create a class called OnnxArena which implements FFM's Arena
* move all the model-related execute methods from OnnxRuntime to OnnxArena
* OnnxArena will create a confined arena internally -- to deal with the lifetime of an underlying onnx session

Then the MNIST demo code will look something like:


try (OnnxArena onnxArena = OnnxArena.of()) {
      var prediction = onnxArena.execute(MethodHandles.lookup(),
                           () -> cnn(imageTensor));
      var result = prediction.data().toArray(ValueLayout.JAVA_FLOAT);
      ...
} // close the onnx session, and release all memory allocated within the arena

-------------

PR Comment: https://git.openjdk.org/babylon/pull/332#issuecomment-2693812580

From asotona at openjdk.org  Mon Mar  3 10:07:06 2025
From: asotona at openjdk.org (Adam Sotona)
Date: Mon, 3 Mar 2025 10:07:06 GMT
Subject: [code-reflection] RFR: Model lifetimes of onnx session-related
 objects more explicitly
In-Reply-To: <s1RyI3MmwaIiMa3H__eLqjIAHbGb058ONRI3UZVtUkY=.4fd6ec15-bf2b-4f10-b168-c7d9d0dc23ed@github.com>
References: <s1RyI3MmwaIiMa3H__eLqjIAHbGb058ONRI3UZVtUkY=.4fd6ec15-bf2b-4f10-b168-c7d9d0dc23ed@github.com>
Message-ID: <fccNBWrXkNrHThTW-aQuej30idPn97nEZFt2mKP8LO0=.b2030b0a-c2ec-489c-9efc-bc7b43e5bbe4@github.com>

On Fri, 28 Feb 2025 12:42:24 GMT, Maurizio Cimadamore <mcimadamore at openjdk.org> wrote:

> The class representing an onnx session is auto closeable. But, in the current code, a session is closed immediately after its `run` method is called. This is problematic because a session returns some ORTValues (tensors) which also need to be freed, but that cannot be freed immediately after calling `run` (as they need to be used by clients).
> 
> To address this problem, I tweaked the session code to accept an external arena. All the allocation of session-related data structures now happens using that external arena. This means that the client can now be in charge of managing the lifetime of a session (see changes to MNIST demo).
> 
> To test, I tweaked the MNIST code to do 10K iterations on each button pressed. Predictably, a single button pressed resulted in over 3g of memory being leaked. With these changes the memory arrives at ~400K (there is still some minor leak, but not sure worth pushing more).
> 
> If the changes to the demo are not deemed good, I can withdraw this PR -- I mostly wanted to capture the result of my exploration somewhere.

I'm a bit confused. Are we discussing to hide Onnx API from the Onnx API and expose the whole stuff as an extension of FFM API / Arena ?

I was thinking a bit opposite - to wrap the low-level FFM components like memory segments and arenas handling inside the Onnx API. Practically it may look almost the same, just the naming should match what Onnx users are familiar with:


try (OnnxEnvironment onnxEnv= OnnxEnvironment.of(logLevel)) {
      var prediction = onnxEnv.execute(MethodHandles.lookup(),
                           () -> cnn(imageTensor));
      var result = prediction.data().toArray(ValueLayout.JAVA_FLOAT);
      ...
}


However that does not solve constrution of tensors. We have two options:
- Tesor.ofXyz(arena, ...
- onnxEnv.createTensorXyz(...

-------------

PR Comment: https://git.openjdk.org/babylon/pull/332#issuecomment-2693852933

From mcimadamore at openjdk.org  Mon Mar  3 11:27:07 2025
From: mcimadamore at openjdk.org (Maurizio Cimadamore)
Date: Mon, 3 Mar 2025 11:27:07 GMT
Subject: [code-reflection] RFR: Model lifetimes of onnx session-related
 objects more explicitly
In-Reply-To: <fccNBWrXkNrHThTW-aQuej30idPn97nEZFt2mKP8LO0=.b2030b0a-c2ec-489c-9efc-bc7b43e5bbe4@github.com>
References: <s1RyI3MmwaIiMa3H__eLqjIAHbGb058ONRI3UZVtUkY=.4fd6ec15-bf2b-4f10-b168-c7d9d0dc23ed@github.com>
 <fccNBWrXkNrHThTW-aQuej30idPn97nEZFt2mKP8LO0=.b2030b0a-c2ec-489c-9efc-bc7b43e5bbe4@github.com>
Message-ID: <uu5i_yRLCEQuVsWmtMtUZUw45ALWgXVj0ttsihyXfao=.47ec6635-967c-4604-9a6d-7426b627c0e5@github.com>

On Mon, 3 Mar 2025 10:04:53 GMT, Adam Sotona <asotona at openjdk.org> wrote:

> I'm a bit confused. Are we discussing to hide Onnx API from the Onnx API and expose the whole stuff as an extension of FFM API / Arena ?

I'm not sure either. But I think following the abstractions of the C API too literally might be a siren song. If we go down that path there's multiple layers of stuff to create, each with different lifetimes:

* the onnx runtime, ortAPI -- which has a global lifetime
* an onnx env which can be created/released
* an onnx session, which can be created (inside an env) and released
* tensors, which are OrtValues and can be created with any lifetime (but must be released with `ReleaseValue`)

If we expose all this to developers mistakes will be likely (e.g. forgetting to close/release memory). The Python API went a completely different route -- and only seems to expose a session API:

https://onnxruntime.ai/docs/api/python/api_summary.html#

My general feeling is that 90% of what the user wants is some autocloseable concept in which temporary resources (e.g. tensors) can be allocated -- and in which loading/executing a model is possible (and when you close, then everything is deallocated). I believe for tesnors you probably want two options:

* a way to create "global" tensors -- associated with their own automatic arena (so that they are freed based on GC)
* a way to create "local" tensors -- tied to a particular session -- this is useful e.g. to model results of model execution

I think our suggestions are in reality quite similar -- replace OnnxEnvironment with OnnxArena and you get the same thing. I'm not too biased on how we want to call these things -- as long as we don't introduce too much confusion with names in the ONNX API. IMHO what you call "OnnxEnvironment" is not really an OnnxEnv (e.g. an env cannot really execute anything) -- it's more a Java view of how interacting with the ONNX API should be -- so I'd prefer to call it with a separate name -- and then define how it maps on the Onnx API concepts.

If we go for a separate `OnnxEnvironment`-like class (e.g. something that is not an arena) then that class should also expose an arena, in case developers want to use the lifetime of the environment to perform memory segment allocation, memory map files, etc. (but maybe this is something we can add later, depending on use cases -- if you are confident that developers might never need to spell "MemorySegment" -- then it might not be necessary)

-------------

PR Comment: https://git.openjdk.org/babylon/pull/332#issuecomment-2694049448

From mcimadamore at openjdk.org  Mon Mar  3 11:27:07 2025
From: mcimadamore at openjdk.org (Maurizio Cimadamore)
Date: Mon, 3 Mar 2025 11:27:07 GMT
Subject: [code-reflection] RFR: Model lifetimes of onnx session-related
 objects more explicitly
In-Reply-To: <s1RyI3MmwaIiMa3H__eLqjIAHbGb058ONRI3UZVtUkY=.4fd6ec15-bf2b-4f10-b168-c7d9d0dc23ed@github.com>
References: <s1RyI3MmwaIiMa3H__eLqjIAHbGb058ONRI3UZVtUkY=.4fd6ec15-bf2b-4f10-b168-c7d9d0dc23ed@github.com>
Message-ID: <HjHws21VIksQRibzUcF9yN2Tz8faefalOEw06ZXMfY8=.38836328-5df3-4b4c-bce3-12e30581433e@github.com>

On Fri, 28 Feb 2025 12:42:24 GMT, Maurizio Cimadamore <mcimadamore at openjdk.org> wrote:

> The class representing an onnx session is auto closeable. But, in the current code, a session is closed immediately after its `run` method is called. This is problematic because a session returns some ORTValues (tensors) which also need to be freed, but that cannot be freed immediately after calling `run` (as they need to be used by clients).
> 
> To address this problem, I tweaked the session code to accept an external arena. All the allocation of session-related data structures now happens using that external arena. This means that the client can now be in charge of managing the lifetime of a session (see changes to MNIST demo).
> 
> To test, I tweaked the MNIST code to do 10K iterations on each button pressed. Predictably, a single button pressed resulted in over 3g of memory being leaked. With these changes the memory arrives at ~400K (there is still some minor leak, but not sure worth pushing more).
> 
> If the changes to the demo are not deemed good, I can withdraw this PR -- I mostly wanted to capture the result of my exploration somewhere.

Meta-comment -- it would perhaps be better to split this PR from the discussion of what a better high-level API should be? This PR really adds ways to correctly manage memory that is local to a session, so that there's no memory leaks. If we don't like having developers manually create an arena we can fix that later in a more API-oriented PR (as this PR should introduce enough machinery to implement whatever API is desirable on top).

-------------

PR Comment: https://git.openjdk.org/babylon/pull/332#issuecomment-2694054943

From asotona at openjdk.org  Mon Mar  3 12:11:35 2025
From: asotona at openjdk.org (Adam Sotona)
Date: Mon, 3 Mar 2025 12:11:35 GMT
Subject: [code-reflection] RFR: Model lifetimes of onnx session-related
 objects more explicitly
In-Reply-To: <s1RyI3MmwaIiMa3H__eLqjIAHbGb058ONRI3UZVtUkY=.4fd6ec15-bf2b-4f10-b168-c7d9d0dc23ed@github.com>
References: <s1RyI3MmwaIiMa3H__eLqjIAHbGb058ONRI3UZVtUkY=.4fd6ec15-bf2b-4f10-b168-c7d9d0dc23ed@github.com>
Message-ID: <hfaSeayACexE6T34CQ_n2ge3w9BSjnfS1HzRowf8IGQ=.5c12ec50-127b-4a94-8b5c-f6f5a3116903@github.com>

On Fri, 28 Feb 2025 12:42:24 GMT, Maurizio Cimadamore <mcimadamore at openjdk.org> wrote:

> The class representing an onnx session is auto closeable. But, in the current code, a session is closed immediately after its `run` method is called. This is problematic because a session returns some ORTValues (tensors) which also need to be freed, but that cannot be freed immediately after calling `run` (as they need to be used by clients).
> 
> To address this problem, I tweaked the session code to accept an external arena. All the allocation of session-related data structures now happens using that external arena. This means that the client can now be in charge of managing the lifetime of a session (see changes to MNIST demo).
> 
> To test, I tweaked the MNIST code to do 10K iterations on each button pressed. Predictably, a single button pressed resulted in over 3g of memory being leaked. With these changes the memory arrives at ~400K (there is still some minor leak, but not sure worth pushing more).
> 
> If the changes to the demo are not deemed good, I can withdraw this PR -- I mostly wanted to capture the result of my exploration somewhere.

Considering different use cases is this PR step forward, however further changes need deeper discussion. Letting the arena lifecycle on user is probably the best we can do now. I can imagine use cases where the model/session is used once and the resulting tensor suppose to live long outside of the try block. And I can also imagine a 100k of repeated execution of the same model/session, where session-long living results would cause significant memory leaks.

My the only remaining concern is related to the naming of Arena instance "onnxSession".

-------------

Marked as reviewed by asotona (Reviewer).

PR Review: https://git.openjdk.org/babylon/pull/332#pullrequestreview-2653948088

From mcimadamore at openjdk.org  Mon Mar  3 12:21:38 2025
From: mcimadamore at openjdk.org (Maurizio Cimadamore)
Date: Mon, 3 Mar 2025 12:21:38 GMT
Subject: [code-reflection] RFR: Model lifetimes of onnx session-related
 objects more explicitly [v2]
In-Reply-To: <s1RyI3MmwaIiMa3H__eLqjIAHbGb058ONRI3UZVtUkY=.4fd6ec15-bf2b-4f10-b168-c7d9d0dc23ed@github.com>
References: <s1RyI3MmwaIiMa3H__eLqjIAHbGb058ONRI3UZVtUkY=.4fd6ec15-bf2b-4f10-b168-c7d9d0dc23ed@github.com>
Message-ID: <jAwD9NICmwuLtwNwFt0Bng4_LDcVbn36tIyz2MH7m04=.5596a6fb-daf9-412b-8a77-3f1dc993fade@github.com>

> The class representing an onnx session is auto closeable. But, in the current code, a session is closed immediately after its `run` method is called. This is problematic because a session returns some ORTValues (tensors) which also need to be freed, but that cannot be freed immediately after calling `run` (as they need to be used by clients).
> 
> To address this problem, I tweaked the session code to accept an external arena. All the allocation of session-related data structures now happens using that external arena. This means that the client can now be in charge of managing the lifetime of a session (see changes to MNIST demo).
> 
> To test, I tweaked the MNIST code to do 10K iterations on each button pressed. Predictably, a single button pressed resulted in over 3g of memory being leaked. With these changes the memory arrives at ~400K (there is still some minor leak, but not sure worth pushing more).
> 
> If the changes to the demo are not deemed good, I can withdraw this PR -- I mostly wanted to capture the result of my exploration somewhere.

Maurizio Cimadamore has updated the pull request incrementally with one additional commit since the last revision:

  Rename local arena variables

-------------

Changes:
  - all: https://git.openjdk.org/babylon/pull/332/files
  - new: https://git.openjdk.org/babylon/pull/332/files/09c802a1..615c5503

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=babylon&pr=332&range=01
 - incr: https://webrevs.openjdk.org/?repo=babylon&pr=332&range=00-01

  Stats: 9 lines in 2 files changed: 0 ins; 0 del; 9 mod
  Patch: https://git.openjdk.org/babylon/pull/332.diff
  Fetch: git fetch https://git.openjdk.org/babylon.git pull/332/head:pull/332

PR: https://git.openjdk.org/babylon/pull/332

From mcimadamore at openjdk.org  Mon Mar  3 12:21:39 2025
From: mcimadamore at openjdk.org (Maurizio Cimadamore)
Date: Mon, 3 Mar 2025 12:21:39 GMT
Subject: [code-reflection] RFR: Model lifetimes of onnx session-related
 objects more explicitly
In-Reply-To: <s1RyI3MmwaIiMa3H__eLqjIAHbGb058ONRI3UZVtUkY=.4fd6ec15-bf2b-4f10-b168-c7d9d0dc23ed@github.com>
References: <s1RyI3MmwaIiMa3H__eLqjIAHbGb058ONRI3UZVtUkY=.4fd6ec15-bf2b-4f10-b168-c7d9d0dc23ed@github.com>
Message-ID: <DC_i7BRKfKblxX0tpg5qsSiLgWPgO4DRl2c8aevBZgg=.e26eb2c6-6417-4db2-974e-83c9718dd24c@github.com>

On Fri, 28 Feb 2025 12:42:24 GMT, Maurizio Cimadamore <mcimadamore at openjdk.org> wrote:

> The class representing an onnx session is auto closeable. But, in the current code, a session is closed immediately after its `run` method is called. This is problematic because a session returns some ORTValues (tensors) which also need to be freed, but that cannot be freed immediately after calling `run` (as they need to be used by clients).
> 
> To address this problem, I tweaked the session code to accept an external arena. All the allocation of session-related data structures now happens using that external arena. This means that the client can now be in charge of managing the lifetime of a session (see changes to MNIST demo).
> 
> To test, I tweaked the MNIST code to do 10K iterations on each button pressed. Predictably, a single button pressed resulted in over 3g of memory being leaked. With these changes the memory arrives at ~400K (there is still some minor leak, but not sure worth pushing more).
> 
> If the changes to the demo are not deemed good, I can withdraw this PR -- I mostly wanted to capture the result of my exploration somewhere.

This is an example of a possible higher-level API (built on top of the changes in this PR)

https://github.com/mcimadamore/babylon/compare/onnx_session_lifetime...mcimadamore:babylon:high-level-API?expand=1

For now -- it is added as an inner class of OnnxRuntime, since there's so much reuse (e.g. of `runtimeAddress` and other related helper functions). But the idea is that users will do:


try (OnnxRuntime.Environment env = OnnxRuntime.newEnv()) {
    env.execute(...)
    ...
} // close resources

-------------

PR Comment: https://git.openjdk.org/babylon/pull/332#issuecomment-2694179389

From asotona at openjdk.org  Mon Mar  3 12:28:06 2025
From: asotona at openjdk.org (Adam Sotona)
Date: Mon, 3 Mar 2025 12:28:06 GMT
Subject: [code-reflection] RFR: Model lifetimes of onnx session-related
 objects more explicitly [v2]
In-Reply-To: <jAwD9NICmwuLtwNwFt0Bng4_LDcVbn36tIyz2MH7m04=.5596a6fb-daf9-412b-8a77-3f1dc993fade@github.com>
References: <s1RyI3MmwaIiMa3H__eLqjIAHbGb058ONRI3UZVtUkY=.4fd6ec15-bf2b-4f10-b168-c7d9d0dc23ed@github.com>
 <jAwD9NICmwuLtwNwFt0Bng4_LDcVbn36tIyz2MH7m04=.5596a6fb-daf9-412b-8a77-3f1dc993fade@github.com>
Message-ID: <citoyU-6nLIW9pAypvS8pBzfRFQzAl8qgBhp0IDLpwk=.0b1562a7-953d-4306-8345-9a132076e77d@github.com>

On Mon, 3 Mar 2025 12:21:38 GMT, Maurizio Cimadamore <mcimadamore at openjdk.org> wrote:

>> The class representing an onnx session is auto closeable. But, in the current code, a session is closed immediately after its `run` method is called. This is problematic because a session returns some ORTValues (tensors) which also need to be freed, but that cannot be freed immediately after calling `run` (as they need to be used by clients).
>> 
>> To address this problem, I tweaked the session code to accept an external arena. All the allocation of session-related data structures now happens using that external arena. This means that the client can now be in charge of managing the lifetime of a session (see changes to MNIST demo).
>> 
>> To test, I tweaked the MNIST code to do 10K iterations on each button pressed. Predictably, a single button pressed resulted in over 3g of memory being leaked. With these changes the memory arrives at ~400K (there is still some minor leak, but not sure worth pushing more).
>> 
>> If the changes to the demo are not deemed good, I can withdraw this PR -- I mostly wanted to capture the result of my exploration somewhere.
>
> Maurizio Cimadamore has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Rename local arena variables

Marked as reviewed by asotona (Reviewer).

-------------

PR Review: https://git.openjdk.org/babylon/pull/332#pullrequestreview-2653983689

From duke at openjdk.org  Mon Mar  3 13:13:08 2025
From: duke at openjdk.org (Adam Pocock)
Date: Mon, 3 Mar 2025 13:13:08 GMT
Subject: [code-reflection] RFR: Model lifetimes of onnx session-related
 objects more explicitly [v2]
In-Reply-To: <jAwD9NICmwuLtwNwFt0Bng4_LDcVbn36tIyz2MH7m04=.5596a6fb-daf9-412b-8a77-3f1dc993fade@github.com>
References: <s1RyI3MmwaIiMa3H__eLqjIAHbGb058ONRI3UZVtUkY=.4fd6ec15-bf2b-4f10-b168-c7d9d0dc23ed@github.com>
 <jAwD9NICmwuLtwNwFt0Bng4_LDcVbn36tIyz2MH7m04=.5596a6fb-daf9-412b-8a77-3f1dc993fade@github.com>
Message-ID: <QFqHCcmnpXhzp2tZ8tEtfe6DjXWbshos4qPEVPFJ6GQ=.2ddbd8ad-84f1-4f3a-8555-dc6e282f7c3b@github.com>

On Mon, 3 Mar 2025 12:21:38 GMT, Maurizio Cimadamore <mcimadamore at openjdk.org> wrote:

>> The class representing an onnx session is auto closeable. But, in the current code, a session is closed immediately after its `run` method is called. This is problematic because a session returns some ORTValues (tensors) which also need to be freed, but that cannot be freed immediately after calling `run` (as they need to be used by clients).
>> 
>> To address this problem, I tweaked the session code to accept an external arena. All the allocation of session-related data structures now happens using that external arena. This means that the client can now be in charge of managing the lifetime of a session (see changes to MNIST demo).
>> 
>> To test, I tweaked the MNIST code to do 10K iterations on each button pressed. Predictably, a single button pressed resulted in over 3g of memory being leaked. With these changes the memory arrives at ~400K (there is still some minor leak, but not sure worth pushing more).
>> 
>> If the changes to the demo are not deemed good, I can withdraw this PR -- I mostly wanted to capture the result of my exploration somewhere.
>
> Maurizio Cimadamore has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Rename local arena variables

Releasing the ONNX environment will cause it to crash the next time one is created when certain kinds of sessions are created. It needs to be created at most once per JVM instantiation.

-------------

PR Comment: https://git.openjdk.org/babylon/pull/332#issuecomment-2694347199

From duke at openjdk.org  Mon Mar  3 14:05:08 2025
From: duke at openjdk.org (Adam Pocock)
Date: Mon, 3 Mar 2025 14:05:08 GMT
Subject: [code-reflection] RFR: Model lifetimes of onnx session-related
 objects more explicitly
In-Reply-To: <uu5i_yRLCEQuVsWmtMtUZUw45ALWgXVj0ttsihyXfao=.47ec6635-967c-4604-9a6d-7426b627c0e5@github.com>
References: <s1RyI3MmwaIiMa3H__eLqjIAHbGb058ONRI3UZVtUkY=.4fd6ec15-bf2b-4f10-b168-c7d9d0dc23ed@github.com>
 <fccNBWrXkNrHThTW-aQuej30idPn97nEZFt2mKP8LO0=.b2030b0a-c2ec-489c-9efc-bc7b43e5bbe4@github.com>
 <uu5i_yRLCEQuVsWmtMtUZUw45ALWgXVj0ttsihyXfao=.47ec6635-967c-4604-9a6d-7426b627c0e5@github.com>
Message-ID: <meJpQ9Tc0W5jtqd31R1Dacl6eqXaGJtzoutcyWejriM=.e927b29f-1ee7-45c4-98f6-3113e6d65b77@github.com>

On Mon, 3 Mar 2025 11:21:38 GMT, Maurizio Cimadamore <mcimadamore at openjdk.org> wrote:

> > I'm a bit confused. Are we discussing to hide Onnx API from the Onnx API and expose the whole stuff as an extension of FFM API / Arena ?
> 
> I'm not sure either. But I think following the abstractions of the C API too literally might be a siren song. If we go down that path there's multiple layers of stuff to create, each with different lifetimes:
> 
>     * the onnx runtime, ortAPI -- which has a global lifetime
> 
>     * an onnx env which can be created/released
> 
>     * an onnx session, which can be created (inside an env) and released
> 
>     * tensors, which are OrtValues and can be created with any lifetime (but must be released with `ReleaseValue`)
> 
> 
> If we expose all this to developers mistakes will be likely (e.g. forgetting to close/release memory). The Python API went a completely different route -- and only seems to expose a session API:
> 
> https://onnxruntime.ai/docs/api/python/api_summary.html#
> 
> My general feeling is that 90% of what the user wants is some autocloseable concept in which temporary resources (e.g. tensors) can be allocated -- and in which loading/executing a model is possible (and when you close, then everything is deallocated). I believe for tesnors you probably want two options:
> 
>     * a way to create "global" tensors -- associated with their own automatic arena (so that they are freed based on GC)
> 
>     * a way to create "local" tensors -- tied to a particular session -- this is useful e.g. to model results of model execution
> 
> 
> I think our suggestions are in reality quite similar -- replace OnnxEnvironment with OnnxArena and you get the same thing. I'm not too biased on how we want to call these things -- as long as we don't introduce too much confusion with names in the ONNX API. IMHO what you call "OnnxEnvironment" is not really an OnnxEnv (e.g. an env cannot really execute anything) -- it's more a Java view of how interacting with the ONNX API should be -- so I'd prefer to call it with a separate name -- and then define how it maps on the Onnx API concepts.
> 
> If we go for a separate `OnnxEnvironment`-like class (e.g. something that is not an arena) then that class should also expose an arena, in case developers want to use the lifetime of the environment to perform memory segment allocation, memory map files, etc. (but maybe this is something we can add later, depending on use cases -- if you are confident that developers might never need to spell "MemorySegment" -- then it might not be necessary)

The ONNX Runtime Java API does explicitly model the lifetimes of those things aside from the OrtApi struct which is static and the environment which has a shutdown hook & has a no-op close method. We needed to do this as users want to be able to create environments using global thread pools which means they need to control environment creation, but the environment must be created at most once per process. The Python API is currently trying to figure out how to expose that and it'll be a bit annoying for them as their environment is implicit. There is an allocator concept too which I've ignored in the Java API but have had several requests to expose (it'll allow things like allocating directly onto the GPU, which is more useful when you can express GPU memory segments, not sure how to do that in the existing JNI based API), that maps more directly onto the arenas and I'd base the API around that if I was writing it today.

-------------

PR Comment: https://git.openjdk.org/babylon/pull/332#issuecomment-2694475543

From mcimadamore at openjdk.org  Mon Mar  3 14:31:17 2025
From: mcimadamore at openjdk.org (Maurizio Cimadamore)
Date: Mon, 3 Mar 2025 14:31:17 GMT
Subject: [code-reflection] RFR: Model lifetimes of onnx session-related
 objects more explicitly [v2]
In-Reply-To: <QFqHCcmnpXhzp2tZ8tEtfe6DjXWbshos4qPEVPFJ6GQ=.2ddbd8ad-84f1-4f3a-8555-dc6e282f7c3b@github.com>
References: <s1RyI3MmwaIiMa3H__eLqjIAHbGb058ONRI3UZVtUkY=.4fd6ec15-bf2b-4f10-b168-c7d9d0dc23ed@github.com>
 <jAwD9NICmwuLtwNwFt0Bng4_LDcVbn36tIyz2MH7m04=.5596a6fb-daf9-412b-8a77-3f1dc993fade@github.com>
 <QFqHCcmnpXhzp2tZ8tEtfe6DjXWbshos4qPEVPFJ6GQ=.2ddbd8ad-84f1-4f3a-8555-dc6e282f7c3b@github.com>
Message-ID: <X60Vyu558n8eK8_wjNJLt6lrqJ_nP2PMJLCEPS9v5Cg=.a60e2157-fa78-4b23-81cd-7117fc84a572@github.com>

On Mon, 3 Mar 2025 13:10:50 GMT, Adam Pocock <duke at openjdk.org> wrote:

> Releasing the ONNX environment will cause it to crash the next time one is created when certain kinds of sessions are created. It needs to be created at most once per JVM instantiation.

OK - this is not documented anywhere though. Is it a bug in the C impl? I've tested my toy API with an autocloseable environment and I have no problem creating a new environment and disposing it after each new model execution in the MNIST demo (I repeated 10K times). Is there some high-level documentation which explains how these various abstractions are meant to be used?

If environment really has to be global -- that leaves us again with the problem of needed to have a lifetime per session (because a session can create a lot of resources that you don't want to just keep accumulating -- which is fine, but then the API would look more similar to the Python one (e.g. if there's only one environment, then having it implicit seems like a good 80/20 compromise to me).

-------------

PR Comment: https://git.openjdk.org/babylon/pull/332#issuecomment-2694582050

From mcimadamore at openjdk.org  Mon Mar  3 14:37:04 2025
From: mcimadamore at openjdk.org (Maurizio Cimadamore)
Date: Mon, 3 Mar 2025 14:37:04 GMT
Subject: [code-reflection] RFR: Model lifetimes of onnx session-related
 objects more explicitly
In-Reply-To: <meJpQ9Tc0W5jtqd31R1Dacl6eqXaGJtzoutcyWejriM=.e927b29f-1ee7-45c4-98f6-3113e6d65b77@github.com>
References: <s1RyI3MmwaIiMa3H__eLqjIAHbGb058ONRI3UZVtUkY=.4fd6ec15-bf2b-4f10-b168-c7d9d0dc23ed@github.com>
 <fccNBWrXkNrHThTW-aQuej30idPn97nEZFt2mKP8LO0=.b2030b0a-c2ec-489c-9efc-bc7b43e5bbe4@github.com>
 <uu5i_yRLCEQuVsWmtMtUZUw45ALWgXVj0ttsihyXfao=.47ec6635-967c-4604-9a6d-7426b627c0e5@github.com>
 <meJpQ9Tc0W5jtqd31R1Dacl6eqXaGJtzoutcyWejriM=.e927b29f-1ee7-45c4-98f6-3113e6d65b77@github.com>
Message-ID: <6Uy5H8qZycvg0Vx9CG0iSXF6quE2xgOS0gJdj6lOeg0=.d7952fc3-6a60-4aa1-8fd2-3f0ede6bd4b9@github.com>

On Mon, 3 Mar 2025 13:50:39 GMT, Adam Pocock <duke at openjdk.org> wrote:

> There is an allocator concept too which I've ignored in the Java API but have had several requests to expose (it'll allow things like allocating directly onto the GPU, which is more useful when you can express GPU memory segments, not sure how to do that in the existing JNI based API), that maps more directly onto the arenas and I'd base the API around that if I was writing it today.

I see - I have seen allocators on the C API. I'm not sure those needs to be exposed -- I'd assume these are mostly there to allow e.g. allocating a tensor on the GPU (probably in a transparent fashion). For the Java API, it seems to me that the capabilities exposed by an ORTAllocator can be emulated by using memory segment + arena + reinterpret, and then use the tensor factory that accepts existing data:

https://onnxruntime.ai/docs/api/c/struct_ort_api.html#a2aad3ccd68c66d0b38bdb966467d9324

-------------

PR Comment: https://git.openjdk.org/babylon/pull/332#issuecomment-2694607649

From duke at openjdk.org  Mon Mar  3 14:47:06 2025
From: duke at openjdk.org (Adam Pocock)
Date: Mon, 3 Mar 2025 14:47:06 GMT
Subject: [code-reflection] RFR: Model lifetimes of onnx session-related
 objects more explicitly [v2]
In-Reply-To: <X60Vyu558n8eK8_wjNJLt6lrqJ_nP2PMJLCEPS9v5Cg=.a60e2157-fa78-4b23-81cd-7117fc84a572@github.com>
References: <s1RyI3MmwaIiMa3H__eLqjIAHbGb058ONRI3UZVtUkY=.4fd6ec15-bf2b-4f10-b168-c7d9d0dc23ed@github.com>
 <jAwD9NICmwuLtwNwFt0Bng4_LDcVbn36tIyz2MH7m04=.5596a6fb-daf9-412b-8a77-3f1dc993fade@github.com>
 <QFqHCcmnpXhzp2tZ8tEtfe6DjXWbshos4qPEVPFJ6GQ=.2ddbd8ad-84f1-4f3a-8555-dc6e282f7c3b@github.com>
 <X60Vyu558n8eK8_wjNJLt6lrqJ_nP2PMJLCEPS9v5Cg=.a60e2157-fa78-4b23-81cd-7117fc84a572@github.com>
Message-ID: <SCzGQtBqn0K5v43FpZpR4sqCTQoXtKFmS1pCu8-wWig=.a77503c8-b6b5-478b-84e8-e073249f6a7b@github.com>

On Mon, 3 Mar 2025 14:28:55 GMT, Maurizio Cimadamore <mcimadamore at openjdk.org> wrote:

> > Releasing the ONNX environment will cause it to crash the next time one is created when certain kinds of sessions are created. It needs to be created at most once per JVM instantiation.
> 
> OK - this is not documented anywhere though. Is it a bug in the C impl? I've tested my toy API with an autocloseable environment and I have no problem creating a new environment and disposing it after each new model execution in the MNIST demo (I repeated 10K times). Is there some high-level documentation which explains how these various abstractions are meant to be used?
> 
> If environment really has to be global -- that leaves us again with the problem of needed to have a lifetime per session (because a session can create a lot of resources that you don't want to just keep accumulating -- which is fine, but then the API would look more similar to the Python one (e.g. if there's only one environment, then having it implicit seems like a good 80/20 compromise to me).

I documented it in the [Java API](https://github.com/microsoft/onnxruntime/blob/main/java/src/main/java/ai/onnxruntime/OrtEnvironment.java#L20). It's a consequence of how they load in CUDA & other execution providers, I don't think it breaks if it's CPU only (but it is allowed to), and it's not considered a bug in the C API (though the lack of documentation is a problem). The description of how I worked around it is [here](https://github.com/microsoft/onnxruntime/pull/10670). They added support for custom environment creation to [C# in 2023](https://github.com/microsoft/onnxruntime/pull/14723), and there are requests for it in Python. I'm not sure why it's not documented on the C API itself, but the C API has a bunch of undocumented interactions that you only find when they bite you (e.g., the last one I hit was that the CUDA session options update method overwrites all the options rather than adding options, so I added a note for that to the C API docs).

If you want to replicate the functionality of the existing Java API then environment creation control needs to be done by the user so they can specify a global thread pool (which is then shared across all sessions created from that environment) as there are people who use that functionality. But it depends what the scope is here, it can be implicit if you're not trying to provide a production ready API where people need that level of resource control.

-------------

PR Comment: https://git.openjdk.org/babylon/pull/332#issuecomment-2694638609

From duke at openjdk.org  Mon Mar  3 14:53:21 2025
From: duke at openjdk.org (Adam Pocock)
Date: Mon, 3 Mar 2025 14:53:21 GMT
Subject: [code-reflection] RFR: Model lifetimes of onnx session-related
 objects more explicitly
In-Reply-To: <6Uy5H8qZycvg0Vx9CG0iSXF6quE2xgOS0gJdj6lOeg0=.d7952fc3-6a60-4aa1-8fd2-3f0ede6bd4b9@github.com>
References: <s1RyI3MmwaIiMa3H__eLqjIAHbGb058ONRI3UZVtUkY=.4fd6ec15-bf2b-4f10-b168-c7d9d0dc23ed@github.com>
 <fccNBWrXkNrHThTW-aQuej30idPn97nEZFt2mKP8LO0=.b2030b0a-c2ec-489c-9efc-bc7b43e5bbe4@github.com>
 <uu5i_yRLCEQuVsWmtMtUZUw45ALWgXVj0ttsihyXfao=.47ec6635-967c-4604-9a6d-7426b627c0e5@github.com>
 <meJpQ9Tc0W5jtqd31R1Dacl6eqXaGJtzoutcyWejriM=.e927b29f-1ee7-45c4-98f6-3113e6d65b77@github.com>
 <6Uy5H8qZycvg0Vx9CG0iSXF6quE2xgOS0gJdj6lOeg0=.d7952fc3-6a60-4aa1-8fd2-3f0ede6bd4b9@github.com>
Message-ID: <fA_32yXXImUInhPdMUff-4_BlWqMuxxzJQJyeVqWwD8=.a1f661c1-ed3a-465a-b24f-5b7696722c1d@github.com>

On Mon, 3 Mar 2025 14:34:57 GMT, Maurizio Cimadamore <mcimadamore at openjdk.org> wrote:

> > There is an allocator concept too which I've ignored in the Java API but have had several requests to expose (it'll allow things like allocating directly onto the GPU, which is more useful when you can express GPU memory segments, not sure how to do that in the existing JNI based API), that maps more directly onto the arenas and I'd base the API around that if I was writing it today.
> 
> I see - I have seen allocators on the C API. I'm not sure those needs to be exposed -- I'd assume these are mostly there to allow e.g. allocating a tensor on the GPU (probably in a transparent fashion). For the Java API, it seems to me that the capabilities exposed by an ORTAllocator can be emulated by using memory segment + arena + reinterpret, and then use the tensor factory that accepts existing data:
> 
> https://onnxruntime.ai/docs/api/c/struct_ort_api.html#a2aad3ccd68c66d0b38bdb966467d9324

Yeah, I think it can be hidden for most use cases (the concept exists in the Java API but it's an internal implementation detail). The allocator is used in a bunch of other utility methods where it allocates strings & other objects in addition to tensor creation, but those can typically be put on the CPU default one.

-------------

PR Comment: https://git.openjdk.org/babylon/pull/332#issuecomment-2694655713

From asotona at openjdk.org  Mon Mar  3 15:24:14 2025
From: asotona at openjdk.org (Adam Sotona)
Date: Mon, 3 Mar 2025 15:24:14 GMT
Subject: [code-reflection] RFR: Model lifetimes of onnx session-related
 objects more explicitly [v2]
In-Reply-To: <jAwD9NICmwuLtwNwFt0Bng4_LDcVbn36tIyz2MH7m04=.5596a6fb-daf9-412b-8a77-3f1dc993fade@github.com>
References: <s1RyI3MmwaIiMa3H__eLqjIAHbGb058ONRI3UZVtUkY=.4fd6ec15-bf2b-4f10-b168-c7d9d0dc23ed@github.com>
 <jAwD9NICmwuLtwNwFt0Bng4_LDcVbn36tIyz2MH7m04=.5596a6fb-daf9-412b-8a77-3f1dc993fade@github.com>
Message-ID: <myICRMLMqR-jGl4bkU9Q1vzJQEw54qLTGzoei9_iEyk=.72035675-32f2-4446-943f-24831214fa7e@github.com>

On Mon, 3 Mar 2025 12:21:38 GMT, Maurizio Cimadamore <mcimadamore at openjdk.org> wrote:

>> The class representing an onnx session is auto closeable. But, in the current code, a session is closed immediately after its `run` method is called. This is problematic because a session returns some ORTValues (tensors) which also need to be freed, but that cannot be freed immediately after calling `run` (as they need to be used by clients).
>> 
>> To address this problem, I tweaked the session code to accept an external arena. All the allocation of session-related data structures now happens using that external arena. This means that the client can now be in charge of managing the lifetime of a session (see changes to MNIST demo).
>> 
>> To test, I tweaked the MNIST code to do 10K iterations on each button pressed. Predictably, a single button pressed resulted in over 3g of memory being leaked. With these changes the memory arrives at ~400K (there is still some minor leak, but not sure worth pushing more).
>> 
>> If the changes to the demo are not deemed good, I can withdraw this PR -- I mostly wanted to capture the result of my exploration somewhere.
>
> Maurizio Cimadamore has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Rename local arena variables

I would suggest to merge this PR first as a fix.

Current OnnxRuntime is a simple prototype focused on a single use case and needs to be refactored (maybe multiple times) to physically support all potential usecases (which we didn't even collected yet). A nice API box might be designed around it later.

>From all the discussion here I see the instantiation of the environment should be explicit (even doing it twice may cause a crash) and it might theoretically serve as a wrapper for default arena/allocator. However an option to shorten lifetime of the execution results should remain.  

I propose an alternative solution - create a new Arena.ofAuto() for each model/session execution and associate the results with it, so they are GCed and released when no more needed.

-------------

PR Comment: https://git.openjdk.org/babylon/pull/332#issuecomment-2694749938

From mcimadamore at openjdk.org  Mon Mar  3 15:42:23 2025
From: mcimadamore at openjdk.org (Maurizio Cimadamore)
Date: Mon, 3 Mar 2025 15:42:23 GMT
Subject: [code-reflection] RFR: Model lifetimes of onnx session-related
 objects more explicitly [v2]
In-Reply-To: <myICRMLMqR-jGl4bkU9Q1vzJQEw54qLTGzoei9_iEyk=.72035675-32f2-4446-943f-24831214fa7e@github.com>
References: <s1RyI3MmwaIiMa3H__eLqjIAHbGb058ONRI3UZVtUkY=.4fd6ec15-bf2b-4f10-b168-c7d9d0dc23ed@github.com>
 <jAwD9NICmwuLtwNwFt0Bng4_LDcVbn36tIyz2MH7m04=.5596a6fb-daf9-412b-8a77-3f1dc993fade@github.com>
 <myICRMLMqR-jGl4bkU9Q1vzJQEw54qLTGzoei9_iEyk=.72035675-32f2-4446-943f-24831214fa7e@github.com>
Message-ID: <R3aQP0btoh_pYY8cMXZLSGqvTmbI4q80iYWx4QkmypM=.db816fed-90a4-4067-b8c8-a04e15efa496@github.com>

On Mon, 3 Mar 2025 15:21:34 GMT, Adam Sotona <asotona at openjdk.org> wrote:

> I would suggest to merge this PR first as a fix.

If there's consensus, I'm happy to do that - let's hear from @PaulSandoz 

> 
> Current OnnxRuntime is a simple prototype focused on a single use case and needs to be refactored (maybe multiple times) to physically support all potential usecases (which we didn't even collected yet). A nice API box might be designed around it later.

If environment can only be created once, this seems to strongly argue for hiding the environment creation -- and instead expose the desired environment options in some other way.

> 
> From all the discussion here I see the instantiation of the environment should be explicit (even doing it twice may cause a crash) and it might theoretically serve as a wrapper for default arena/allocator. However an option to shorten lifetime of the execution results should remain.
> 
> I propose an alternative solution - create a new Arena.ofAuto() for each model/session execution and associate the results with it, so they are GCed and released when no more needed.

Using automatic arena is a good way to keep the API simple, but it is IMHO insufficient to deal with timely deallocation -- see the experiment mentioned here:

https://github.com/openjdk/babylon/pull/332#issuecomment-2690607634

-------------

PR Comment: https://git.openjdk.org/babylon/pull/332#issuecomment-2694799786

From asotona at openjdk.org  Mon Mar  3 19:12:36 2025
From: asotona at openjdk.org (Adam Sotona)
Date: Mon, 3 Mar 2025 19:12:36 GMT
Subject: [code-reflection] RFR: MNISTDemo split
Message-ID: <syO0vyd9dZAHxc0kBvYeMwA8igZgmJl6P3lZfMnTUVY=.dec8e631-1063-4de7-ace0-db1876da7db8@github.com>

MNISTDemo separation of UI

-------------

Commit messages:
 - MNISTDemo split

Changes: https://git.openjdk.org/babylon/pull/333/files
  Webrev: https://webrevs.openjdk.org/?repo=babylon&pr=333&range=00
  Stats: 168 lines in 2 files changed: 100 ins; 61 del; 7 mod
  Patch: https://git.openjdk.org/babylon/pull/333.diff
  Fetch: git fetch https://git.openjdk.org/babylon.git pull/333/head:pull/333

PR: https://git.openjdk.org/babylon/pull/333

From asotona at openjdk.org  Mon Mar  3 19:23:20 2025
From: asotona at openjdk.org (Adam Sotona)
Date: Mon, 3 Mar 2025 19:23:20 GMT
Subject: [code-reflection] RFR: MNISTDemo split [v2]
In-Reply-To: <syO0vyd9dZAHxc0kBvYeMwA8igZgmJl6P3lZfMnTUVY=.dec8e631-1063-4de7-ace0-db1876da7db8@github.com>
References: <syO0vyd9dZAHxc0kBvYeMwA8igZgmJl6P3lZfMnTUVY=.dec8e631-1063-4de7-ace0-db1876da7db8@github.com>
Message-ID: <rCfOLb7rP0g8Grz9l_axxj62hfxEHQLgk68wk0Wi4yw=.829e1370-920d-4a1e-8404-664d681b689b@github.com>

> MNISTDemo separation of UI

Adam Sotona has updated the pull request incrementally with one additional commit since the last revision:

  repackaging

-------------

Changes:
  - all: https://git.openjdk.org/babylon/pull/333/files
  - new: https://git.openjdk.org/babylon/pull/333/files/e862726b..c9adc405

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=babylon&pr=333&range=01
 - incr: https://webrevs.openjdk.org/?repo=babylon&pr=333&range=00-01

  Stats: 26 lines in 14 files changed: 2 ins; 1 del; 23 mod
  Patch: https://git.openjdk.org/babylon/pull/333.diff
  Fetch: git fetch https://git.openjdk.org/babylon.git pull/333/head:pull/333

PR: https://git.openjdk.org/babylon/pull/333

From psandoz at openjdk.org  Mon Mar  3 19:29:11 2025
From: psandoz at openjdk.org (Paul Sandoz)
Date: Mon, 3 Mar 2025 19:29:11 GMT
Subject: [code-reflection] RFR: MNISTDemo split [v2]
In-Reply-To: <rCfOLb7rP0g8Grz9l_axxj62hfxEHQLgk68wk0Wi4yw=.829e1370-920d-4a1e-8404-664d681b689b@github.com>
References: <syO0vyd9dZAHxc0kBvYeMwA8igZgmJl6P3lZfMnTUVY=.dec8e631-1063-4de7-ace0-db1876da7db8@github.com>
 <rCfOLb7rP0g8Grz9l_axxj62hfxEHQLgk68wk0Wi4yw=.829e1370-920d-4a1e-8404-664d681b689b@github.com>
Message-ID: <VKbdHRl1Ie7wgGFg14kQmuAZznY-gOgdIsT2wBqJ4nU=.a4db8dab-88f7-4f35-ba32-5387474b0854@github.com>

On Mon, 3 Mar 2025 19:23:20 GMT, Adam Sotona <asotona at openjdk.org> wrote:

>> MNISTDemo separation of UI
>
> Adam Sotona has updated the pull request incrementally with one additional commit since the last revision:
> 
>   repackaging

Marked as reviewed by psandoz (Lead).

-------------

PR Review: https://git.openjdk.org/babylon/pull/333#pullrequestreview-2655075198

From duke at openjdk.org  Mon Mar  3 19:36:07 2025
From: duke at openjdk.org (1235nice)
Date: Mon, 3 Mar 2025 19:36:07 GMT
Subject: [code-reflection] RFR: MNISTDemo split [v2]
In-Reply-To: <rCfOLb7rP0g8Grz9l_axxj62hfxEHQLgk68wk0Wi4yw=.829e1370-920d-4a1e-8404-664d681b689b@github.com>
References: <syO0vyd9dZAHxc0kBvYeMwA8igZgmJl6P3lZfMnTUVY=.dec8e631-1063-4de7-ace0-db1876da7db8@github.com>
 <rCfOLb7rP0g8Grz9l_axxj62hfxEHQLgk68wk0Wi4yw=.829e1370-920d-4a1e-8404-664d681b689b@github.com>
Message-ID: <Yc-x-5tTYM9BVwqi7a0mY3_dHdWpCFnb-d3VIALFI3g=.927c811a-eb43-4ff4-97c3-fe0f0dc8aa2d@github.com>

On Mon, 3 Mar 2025 19:23:20 GMT, Adam Sotona <asotona at openjdk.org> wrote:

>> MNISTDemo separation of UI
>
> Adam Sotona has updated the pull request incrementally with one additional commit since the last revision:
> 
>   repackaging

> ### Webrevs
> * 01: [Full](https://webrevs.openjdk.org/?repo=babylon&pr=333&range=01) - [Incremental](https://webrevs.openjdk.org/?repo=babylon&pr=333&range=00-01) ([c9adc405](https://git.openjdk.org/babylon/pull/333/files/c9adc4051181473016286a484436febaecaacfdc))
> * 00: [Full](https://webrevs.openjdk.org/?repo=babylon&pr=333&range=00) ([e862726b](https://git.openjdk.org/babylon/pull/333/files/e862726b39558ba52f4de30d40d4ca81bcc7aba0))

Hmm good

-------------

PR Comment: https://git.openjdk.org/babylon/pull/333#issuecomment-2695338104
PR Comment: https://git.openjdk.org/babylon/pull/333#issuecomment-2695338792

From asotona at openjdk.org  Mon Mar  3 19:42:09 2025
From: asotona at openjdk.org (Adam Sotona)
Date: Mon, 3 Mar 2025 19:42:09 GMT
Subject: git: openjdk/babylon: code-reflection: MNISTDemo split
Message-ID: <3725caa7-76d0-49b8-ab1e-c4f100ab8df9@openjdk.org>

Changeset: 630b8c95
Branch: code-reflection
Author:    Adam Sotona <asotona at openjdk.org>
Date:      2025-03-03 19:41:25 +0000
URL:       https://git.openjdk.org/babylon/commit/630b8c9585fe8765c188fe163fdf13c3abfe0ac6

MNISTDemo split

Reviewed-by: psandoz

! cr-examples/onnx/src/main/java/oracle/code/onnx/OnnxProtoBuilder.java
! cr-examples/onnx/src/test/java/oracle/code/onnx/CNNTest.java
- cr-examples/onnx/src/test/java/oracle/code/onnx/MNISTDemo.java
+ cr-examples/onnx/src/test/java/oracle/code/onnx/mnist/MNISTDemo.java
+ cr-examples/onnx/src/test/java/oracle/code/onnx/mnist/MNISTDemoUI.java
= cr-examples/onnx/src/test/resources/oracle/code/onnx/mnist/conv1-bias-float-le
= cr-examples/onnx/src/test/resources/oracle/code/onnx/mnist/conv1-weight-float-le
= cr-examples/onnx/src/test/resources/oracle/code/onnx/mnist/conv2-bias-float-le
= cr-examples/onnx/src/test/resources/oracle/code/onnx/mnist/conv2-weight-float-le
= cr-examples/onnx/src/test/resources/oracle/code/onnx/mnist/fc1-bias-float-le
= cr-examples/onnx/src/test/resources/oracle/code/onnx/mnist/fc1-weight-float-le
= cr-examples/onnx/src/test/resources/oracle/code/onnx/mnist/fc2-bias-float-le
= cr-examples/onnx/src/test/resources/oracle/code/onnx/mnist/fc2-weight-float-le
= cr-examples/onnx/src/test/resources/oracle/code/onnx/mnist/fc3-bias-float-le
= cr-examples/onnx/src/test/resources/oracle/code/onnx/mnist/fc3-weight-float-le


From asotona at openjdk.org  Mon Mar  3 19:44:04 2025
From: asotona at openjdk.org (Adam Sotona)
Date: Mon, 3 Mar 2025 19:44:04 GMT
Subject: [code-reflection] RFR: MNISTDemo split [v2]
In-Reply-To: <rCfOLb7rP0g8Grz9l_axxj62hfxEHQLgk68wk0Wi4yw=.829e1370-920d-4a1e-8404-664d681b689b@github.com>
References: <syO0vyd9dZAHxc0kBvYeMwA8igZgmJl6P3lZfMnTUVY=.dec8e631-1063-4de7-ace0-db1876da7db8@github.com>
 <rCfOLb7rP0g8Grz9l_axxj62hfxEHQLgk68wk0Wi4yw=.829e1370-920d-4a1e-8404-664d681b689b@github.com>
Message-ID: <5AwvjcqovpCl-4wvkDoEOUNzlHwOw5eMAcl-CE6s_X0=.c1d37eff-5564-40f7-8caa-27554fa6bf45@github.com>

On Mon, 3 Mar 2025 19:23:20 GMT, Adam Sotona <asotona at openjdk.org> wrote:

>> MNISTDemo separation of UI
>
> Adam Sotona has updated the pull request incrementally with one additional commit since the last revision:
> 
>   repackaging

Thank you.

-------------

PR Comment: https://git.openjdk.org/babylon/pull/333#issuecomment-2695371916

From asotona at openjdk.org  Mon Mar  3 19:44:05 2025
From: asotona at openjdk.org (Adam Sotona)
Date: Mon, 3 Mar 2025 19:44:05 GMT
Subject: [code-reflection] Integrated: MNISTDemo split
In-Reply-To: <syO0vyd9dZAHxc0kBvYeMwA8igZgmJl6P3lZfMnTUVY=.dec8e631-1063-4de7-ace0-db1876da7db8@github.com>
References: <syO0vyd9dZAHxc0kBvYeMwA8igZgmJl6P3lZfMnTUVY=.dec8e631-1063-4de7-ace0-db1876da7db8@github.com>
Message-ID: <HFx-PwwrvejtcIdvOyLSionlHgN-41Ept-PcKf63EMc=.8a298a4c-d0fb-4c86-936e-bd35adeefa4d@github.com>

On Mon, 3 Mar 2025 19:07:53 GMT, Adam Sotona <asotona at openjdk.org> wrote:

> MNISTDemo separation of UI

This pull request has now been integrated.

Changeset: 630b8c95
Author:    Adam Sotona <asotona at openjdk.org>
URL:       https://git.openjdk.org/babylon/commit/630b8c9585fe8765c188fe163fdf13c3abfe0ac6
Stats:     400 lines in 15 files changed: 210 ins; 170 del; 20 mod

MNISTDemo split

Reviewed-by: psandoz

-------------

PR: https://git.openjdk.org/babylon/pull/333

From psandoz at openjdk.org  Mon Mar  3 20:52:02 2025
From: psandoz at openjdk.org (Paul Sandoz)
Date: Mon, 3 Mar 2025 20:52:02 GMT
Subject: [code-reflection] RFR: Model lifetimes of onnx session-related
 objects more explicitly [v2]
In-Reply-To: <jAwD9NICmwuLtwNwFt0Bng4_LDcVbn36tIyz2MH7m04=.5596a6fb-daf9-412b-8a77-3f1dc993fade@github.com>
References: <s1RyI3MmwaIiMa3H__eLqjIAHbGb058ONRI3UZVtUkY=.4fd6ec15-bf2b-4f10-b168-c7d9d0dc23ed@github.com>
 <jAwD9NICmwuLtwNwFt0Bng4_LDcVbn36tIyz2MH7m04=.5596a6fb-daf9-412b-8a77-3f1dc993fade@github.com>
Message-ID: <4gHdm8gLth9-ZJX0kOwT0sGZ04ta28VomzD7T-s15-0=.6dfcb22d-32fe-4880-b66e-b080ee3fae25@github.com>

On Mon, 3 Mar 2025 12:21:38 GMT, Maurizio Cimadamore <mcimadamore at openjdk.org> wrote:

>> The class representing an onnx session is auto closeable. But, in the current code, a session is closed immediately after its `run` method is called. This is problematic because a session returns some ORTValues (tensors) which also need to be freed, but that cannot be freed immediately after calling `run` (as they need to be used by clients).
>> 
>> To address this problem, I tweaked the session code to accept an external arena. All the allocation of session-related data structures now happens using that external arena. This means that the client can now be in charge of managing the lifetime of a session (see changes to MNIST demo).
>> 
>> To test, I tweaked the MNIST code to do 10K iterations on each button pressed. Predictably, a single button pressed resulted in over 3g of memory being leaked. With these changes the memory arrives at ~400K (there is still some minor leak, but not sure worth pushing more).
>> 
>> If the changes to the demo are not deemed good, I can withdraw this PR -- I mostly wanted to capture the result of my exploration somewhere.
>
> Maurizio Cimadamore has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Rename local arena variables

I think it will take us more time to work out the API layering and names that don't conflict with ONNX concepts. Focusing on the memory management for now, with an Arena, is i think like the right thing to do so it behaves correctly and does not leak and can deallocate, if chosen to, in a timely manner.

Later we can then improve on the API layering. There is 1) the binding to the C API, creating an Java API that is idiomatically like C, 2) something that wraps the binding in idiomatic Java, and 3) something higher that encapsulates the scripting whose implementation is composed from 2. Meaning we should eventually be able to support the deployment of existing ONNX models and/or those generated from Java code.

-------------

PR Comment: https://git.openjdk.org/babylon/pull/332#issuecomment-2695503565

From mcimadamore at openjdk.org  Mon Mar  3 21:50:55 2025
From: mcimadamore at openjdk.org (Maurizio Cimadamore)
Date: Mon, 3 Mar 2025 21:50:55 GMT
Subject: [code-reflection] RFR: Model lifetimes of onnx session-related
 objects more explicitly [v3]
In-Reply-To: <s1RyI3MmwaIiMa3H__eLqjIAHbGb058ONRI3UZVtUkY=.4fd6ec15-bf2b-4f10-b168-c7d9d0dc23ed@github.com>
References: <s1RyI3MmwaIiMa3H__eLqjIAHbGb058ONRI3UZVtUkY=.4fd6ec15-bf2b-4f10-b168-c7d9d0dc23ed@github.com>
Message-ID: <flOOMoS3SJ-qGRJwj9wYaL0S5yxHfgE3Uw-fjE52V98=.f75af1e2-1b39-4dd5-8c09-82d88c026ea1@github.com>

> The class representing an onnx session is auto closeable. But, in the current code, a session is closed immediately after its `run` method is called. This is problematic because a session returns some ORTValues (tensors) which also need to be freed, but that cannot be freed immediately after calling `run` (as they need to be used by clients).
> 
> To address this problem, I tweaked the session code to accept an external arena. All the allocation of session-related data structures now happens using that external arena. This means that the client can now be in charge of managing the lifetime of a session (see changes to MNIST demo).
> 
> To test, I tweaked the MNIST code to do 10K iterations on each button pressed. Predictably, a single button pressed resulted in over 3g of memory being leaked. With these changes the memory arrives at ~400K (there is still some minor leak, but not sure worth pushing more).
> 
> If the changes to the demo are not deemed good, I can withdraw this PR -- I mostly wanted to capture the result of my exploration somewhere.

Maurizio Cimadamore has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains four commits:

 - Merge branch 'code-reflection' into onnx_session_lifetime
 - Rename local arena variables
 - Fix bug: replace confined arena with automatic arena in OnnxInterpreter
 - Model lifetimes of session-related objects more explicitly

-------------

Changes: https://git.openjdk.org/babylon/pull/332/files
  Webrev: Webrev is not available because diff is too large
  Stats: 342089 lines in 3277 files changed: 103895 ins; 210551 del; 27643 mod
  Patch: https://git.openjdk.org/babylon/pull/332.diff
  Fetch: git fetch https://git.openjdk.org/babylon.git pull/332/head:pull/332

PR: https://git.openjdk.org/babylon/pull/332

From mcimadamore at openjdk.org  Mon Mar  3 22:01:52 2025
From: mcimadamore at openjdk.org (Maurizio Cimadamore)
Date: Mon, 3 Mar 2025 22:01:52 GMT
Subject: [code-reflection] RFR: Model lifetimes of onnx session-related
 objects more explicitly [v4]
In-Reply-To: <s1RyI3MmwaIiMa3H__eLqjIAHbGb058ONRI3UZVtUkY=.4fd6ec15-bf2b-4f10-b168-c7d9d0dc23ed@github.com>
References: <s1RyI3MmwaIiMa3H__eLqjIAHbGb058ONRI3UZVtUkY=.4fd6ec15-bf2b-4f10-b168-c7d9d0dc23ed@github.com>
Message-ID: <D5p7Pe8weF6OFxm-KDGn8LPeXYGJ_36V-i6mVPy8YA4=.abcd4ddd-6230-4f1e-a58a-b56caa273cd3@github.com>

> The class representing an onnx session is auto closeable. But, in the current code, a session is closed immediately after its `run` method is called. This is problematic because a session returns some ORTValues (tensors) which also need to be freed, but that cannot be freed immediately after calling `run` (as they need to be used by clients).
> 
> To address this problem, I tweaked the session code to accept an external arena. All the allocation of session-related data structures now happens using that external arena. This means that the client can now be in charge of managing the lifetime of a session (see changes to MNIST demo).
> 
> To test, I tweaked the MNIST code to do 10K iterations on each button pressed. Predictably, a single button pressed resulted in over 3g of memory being leaked. With these changes the memory arrives at ~400K (there is still some minor leak, but not sure worth pushing more).
> 
> If the changes to the demo are not deemed good, I can withdraw this PR -- I mostly wanted to capture the result of my exploration somewhere.

Maurizio Cimadamore has refreshed the contents of this pull request, and previous commits have been removed. The incremental views will show differences compared to the previous content of the PR. The pull request contains one new commit since the last revision:

  Merge branch 'code-reflection' into onnx_session_lifetime

-------------

Changes:
  - all: https://git.openjdk.org/babylon/pull/332/files
  - new: https://git.openjdk.org/babylon/pull/332/files/0605aa4b..b7fa2555

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=babylon&pr=332&range=03
 - incr: Webrev is not available because diff is too large

  Stats: 341809 lines in 3274 files changed: 210309 ins; 103791 del; 27709 mod
  Patch: https://git.openjdk.org/babylon/pull/332.diff
  Fetch: git fetch https://git.openjdk.org/babylon.git pull/332/head:pull/332

PR: https://git.openjdk.org/babylon/pull/332

From mcimadamore at openjdk.org  Mon Mar  3 22:01:53 2025
From: mcimadamore at openjdk.org (Maurizio Cimadamore)
Date: Mon, 3 Mar 2025 22:01:53 GMT
Subject: [code-reflection] RFR: Model lifetimes of onnx session-related
 objects more explicitly [v4]
In-Reply-To: <D5p7Pe8weF6OFxm-KDGn8LPeXYGJ_36V-i6mVPy8YA4=.abcd4ddd-6230-4f1e-a58a-b56caa273cd3@github.com>
References: <s1RyI3MmwaIiMa3H__eLqjIAHbGb058ONRI3UZVtUkY=.4fd6ec15-bf2b-4f10-b168-c7d9d0dc23ed@github.com>
 <D5p7Pe8weF6OFxm-KDGn8LPeXYGJ_36V-i6mVPy8YA4=.abcd4ddd-6230-4f1e-a58a-b56caa273cd3@github.com>
Message-ID: <v9b4SO1GmSBH1GtvWttVdb-CugsRywzKXBV1htztqi8=.a1c6178f-054c-41d7-9868-d68feaebb24f@github.com>

On Mon, 3 Mar 2025 21:58:57 GMT, Maurizio Cimadamore <mcimadamore at openjdk.org> wrote:

>> The class representing an onnx session is auto closeable. But, in the current code, a session is closed immediately after its `run` method is called. This is problematic because a session returns some ORTValues (tensors) which also need to be freed, but that cannot be freed immediately after calling `run` (as they need to be used by clients).
>> 
>> To address this problem, I tweaked the session code to accept an external arena. All the allocation of session-related data structures now happens using that external arena. This means that the client can now be in charge of managing the lifetime of a session (see changes to MNIST demo).
>> 
>> To test, I tweaked the MNIST code to do 10K iterations on each button pressed. Predictably, a single button pressed resulted in over 3g of memory being leaked. With these changes the memory arrives at ~400K (there is still some minor leak, but not sure worth pushing more).
>> 
>> If the changes to the demo are not deemed good, I can withdraw this PR -- I mostly wanted to capture the result of my exploration somewhere.
>
> Maurizio Cimadamore has refreshed the contents of this pull request, and previous commits have been removed. The incremental views will show differences compared to the previous content of the PR. The pull request contains one new commit since the last revision:
> 
>   Merge branch 'code-reflection' into onnx_session_lifetime

cr-examples/onnx/src/test/java/oracle/code/onnx/mnist/MNISTDemo.java line 111:

> 109:     public static float[] classify(float[] imageData) {
> 110:         try (Arena arena = Arena.ofConfined()) {
> 111:             var imageTensor = Tensor.ofShape(new long[]{1, 1, IMAGE_SIZE, IMAGE_SIZE}, imageData);

I believe it would be useful to have overloads of the tensor factories taking an arena...

-------------

PR Review Comment: https://git.openjdk.org/babylon/pull/332#discussion_r1978303702

From mcimadamore at openjdk.org  Mon Mar  3 22:07:02 2025
From: mcimadamore at openjdk.org (Maurizio Cimadamore)
Date: Mon, 3 Mar 2025 22:07:02 GMT
Subject: git: openjdk/babylon: code-reflection: Model lifetimes of onnx
 session-related objects more explicitly
Message-ID: <6978fa7e-b355-4201-94a4-85b66248849c@openjdk.org>

Changeset: 0380011c
Branch: code-reflection
Author:    Maurizio Cimadamore <mcimadamore at openjdk.org>
Date:      2025-03-03 22:04:23 +0000
URL:       https://git.openjdk.org/babylon/commit/0380011c784d9b40fe9c5de08109cc753f0cad90

Model lifetimes of onnx session-related objects more explicitly

Reviewed-by: asotona

! cr-examples/onnx/src/main/java/oracle/code/onnx/OnnxInterpreter.java
! cr-examples/onnx/src/main/java/oracle/code/onnx/OnnxRuntime.java
! cr-examples/onnx/src/test/java/oracle/code/onnx/RuntimeTest.java
! cr-examples/onnx/src/test/java/oracle/code/onnx/mnist/MNISTDemo.java


From mcimadamore at openjdk.org  Mon Mar  3 22:07:46 2025
From: mcimadamore at openjdk.org (Maurizio Cimadamore)
Date: Mon, 3 Mar 2025 22:07:46 GMT
Subject: [code-reflection] RFR: Model lifetimes of onnx session-related
 objects more explicitly [v5]
In-Reply-To: <s1RyI3MmwaIiMa3H__eLqjIAHbGb058ONRI3UZVtUkY=.4fd6ec15-bf2b-4f10-b168-c7d9d0dc23ed@github.com>
References: <s1RyI3MmwaIiMa3H__eLqjIAHbGb058ONRI3UZVtUkY=.4fd6ec15-bf2b-4f10-b168-c7d9d0dc23ed@github.com>
Message-ID: <Ny6Sl8zEvk6muIFdS496A1C2GiRWjoz6HSleK72-R-s=.72468bfa-ab01-4d60-b95a-38f80c4d9f5d@github.com>

> The class representing an onnx session is auto closeable. But, in the current code, a session is closed immediately after its `run` method is called. This is problematic because a session returns some ORTValues (tensors) which also need to be freed, but that cannot be freed immediately after calling `run` (as they need to be used by clients).
> 
> To address this problem, I tweaked the session code to accept an external arena. All the allocation of session-related data structures now happens using that external arena. This means that the client can now be in charge of managing the lifetime of a session (see changes to MNIST demo).
> 
> To test, I tweaked the MNIST code to do 10K iterations on each button pressed. Predictably, a single button pressed resulted in over 3g of memory being leaked. With these changes the memory arrives at ~400K (there is still some minor leak, but not sure worth pushing more).
> 
> If the changes to the demo are not deemed good, I can withdraw this PR -- I mostly wanted to capture the result of my exploration somewhere.

Maurizio Cimadamore has updated the pull request incrementally with one additional commit since the last revision:

  Fix missing arena parameter

-------------

Changes:
  - all: https://git.openjdk.org/babylon/pull/332/files
  - new: https://git.openjdk.org/babylon/pull/332/files/b7fa2555..9ecbe73f

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=babylon&pr=332&range=04
 - incr: https://webrevs.openjdk.org/?repo=babylon&pr=332&range=03-04

  Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod
  Patch: https://git.openjdk.org/babylon/pull/332.diff
  Fetch: git fetch https://git.openjdk.org/babylon.git pull/332/head:pull/332

PR: https://git.openjdk.org/babylon/pull/332

From mcimadamore at openjdk.org  Mon Mar  3 22:07:47 2025
From: mcimadamore at openjdk.org (Maurizio Cimadamore)
Date: Mon, 3 Mar 2025 22:07:47 GMT
Subject: [code-reflection] Integrated: Model lifetimes of onnx session-related
 objects more explicitly
In-Reply-To: <s1RyI3MmwaIiMa3H__eLqjIAHbGb058ONRI3UZVtUkY=.4fd6ec15-bf2b-4f10-b168-c7d9d0dc23ed@github.com>
References: <s1RyI3MmwaIiMa3H__eLqjIAHbGb058ONRI3UZVtUkY=.4fd6ec15-bf2b-4f10-b168-c7d9d0dc23ed@github.com>
Message-ID: <DsupjSLEPD29QLTqvsGhZCikqQC5LGi_irn0BfnWl6A=.dbdbf8f5-dc98-4175-8035-5998e44c31d7@github.com>

On Fri, 28 Feb 2025 12:42:24 GMT, Maurizio Cimadamore <mcimadamore at openjdk.org> wrote:

> The class representing an onnx session is auto closeable. But, in the current code, a session is closed immediately after its `run` method is called. This is problematic because a session returns some ORTValues (tensors) which also need to be freed, but that cannot be freed immediately after calling `run` (as they need to be used by clients).
> 
> To address this problem, I tweaked the session code to accept an external arena. All the allocation of session-related data structures now happens using that external arena. This means that the client can now be in charge of managing the lifetime of a session (see changes to MNIST demo).
> 
> To test, I tweaked the MNIST code to do 10K iterations on each button pressed. Predictably, a single button pressed resulted in over 3g of memory being leaked. With these changes the memory arrives at ~400K (there is still some minor leak, but not sure worth pushing more).
> 
> If the changes to the demo are not deemed good, I can withdraw this PR -- I mostly wanted to capture the result of my exploration somewhere.

This pull request has now been integrated.

Changeset: 0380011c
Author:    Maurizio Cimadamore <mcimadamore at openjdk.org>
URL:       https://git.openjdk.org/babylon/commit/0380011c784d9b40fe9c5de08109cc753f0cad90
Stats:     84 lines in 4 files changed: 25 ins; 7 del; 52 mod

Model lifetimes of onnx session-related objects more explicitly

Reviewed-by: asotona

-------------

PR: https://git.openjdk.org/babylon/pull/332

From psandoz at openjdk.org  Mon Mar  3 22:11:17 2025
From: psandoz at openjdk.org (Paul Sandoz)
Date: Mon, 3 Mar 2025 22:11:17 GMT
Subject: [code-reflection] RFR: Model lifetimes of onnx session-related
 objects more explicitly [v5]
In-Reply-To: <Ny6Sl8zEvk6muIFdS496A1C2GiRWjoz6HSleK72-R-s=.72468bfa-ab01-4d60-b95a-38f80c4d9f5d@github.com>
References: <s1RyI3MmwaIiMa3H__eLqjIAHbGb058ONRI3UZVtUkY=.4fd6ec15-bf2b-4f10-b168-c7d9d0dc23ed@github.com>
 <Ny6Sl8zEvk6muIFdS496A1C2GiRWjoz6HSleK72-R-s=.72468bfa-ab01-4d60-b95a-38f80c4d9f5d@github.com>
Message-ID: <LZJy7e6WX05haGDskbuDjBhX-AFh8iDbY8s0ew7mc5o=.c46bfba6-e8bc-487c-800a-a51b8a4859b9@github.com>

On Mon, 3 Mar 2025 22:07:46 GMT, Maurizio Cimadamore <mcimadamore at openjdk.org> wrote:

>> The class representing an onnx session is auto closeable. But, in the current code, a session is closed immediately after its `run` method is called. This is problematic because a session returns some ORTValues (tensors) which also need to be freed, but that cannot be freed immediately after calling `run` (as they need to be used by clients).
>> 
>> To address this problem, I tweaked the session code to accept an external arena. All the allocation of session-related data structures now happens using that external arena. This means that the client can now be in charge of managing the lifetime of a session (see changes to MNIST demo).
>> 
>> To test, I tweaked the MNIST code to do 10K iterations on each button pressed. Predictably, a single button pressed resulted in over 3g of memory being leaked. With these changes the memory arrives at ~400K (there is still some minor leak, but not sure worth pushing more).
>> 
>> If the changes to the demo are not deemed good, I can withdraw this PR -- I mostly wanted to capture the result of my exploration somewhere.
>
> Maurizio Cimadamore has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Fix missing arena parameter

Marked as reviewed by psandoz (Lead).

cr-examples/onnx/src/main/java/oracle/code/onnx/OnnxRuntime.java line 138:

> 136:     }
> 137: 
> 138:     public static <T> Tensor<T> execute(MethodHandles.Lookup l, OnnxFunction<Tensor<T>> codeLambda, Arena sessionArena) {

Can we make the `codeLambda` be the last parameter? Then the arguments are more easily identifiable, with the "configuration" stuff prefixed.

-------------

PR Review: https://git.openjdk.org/babylon/pull/332#pullrequestreview-2655464038
PR Review Comment: https://git.openjdk.org/babylon/pull/332#discussion_r1978311695

From mabbay at openjdk.org  Mon Mar  3 22:52:06 2025
From: mabbay at openjdk.org (Mourad Abbay)
Date: Mon, 3 Mar 2025 22:52:06 GMT
Subject: [code-reflection] RFR: Support storing the code that builds the
 code model [v15]
In-Reply-To: <dexv8sQV_NcJsJIgMopd_M4wSWoZya9dU0ACwDtEL0E=.4ac0b80f-24fb-440f-a18d-bbfd07a681bc@github.com>
References: <dexv8sQV_NcJsJIgMopd_M4wSWoZya9dU0ACwDtEL0E=.4ac0b80f-24fb-440f-a18d-bbfd07a681bc@github.com>
Message-ID: <mds6gzsejWT8DWFwdI0UEnfJ01KfpFbhyKasgoaOzDI=.6a16ec70-3a2a-4153-b7ec-e7acd35de3fb@github.com>

> In this PR we allow the user to decide how to store the code model. The first option is `TEXT`, if this is selected, we store the textual representation of the code model. The second option is `CODE_BUILDR`, if this is selected, we store the code that builds the code model. All work done here is around the second option, because the first option is already supported.

Mourad Abbay has updated the pull request incrementally with one additional commit since the last revision:

  Fix some of the test failures (3 remains)

-------------

Changes:
  - all: https://git.openjdk.org/babylon/pull/305/files
  - new: https://git.openjdk.org/babylon/pull/305/files/c17be6b6..cd143c37

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=babylon&pr=305&range=14
 - incr: https://webrevs.openjdk.org/?repo=babylon&pr=305&range=13-14

  Stats: 19 lines in 1 file changed: 17 ins; 0 del; 2 mod
  Patch: https://git.openjdk.org/babylon/pull/305.diff
  Fetch: git fetch https://git.openjdk.org/babylon.git pull/305/head:pull/305

PR: https://git.openjdk.org/babylon/pull/305

From mcimadamore at openjdk.org  Tue Mar  4 08:02:29 2025
From: mcimadamore at openjdk.org (Maurizio Cimadamore)
Date: Tue, 4 Mar 2025 08:02:29 GMT
Subject: [code-reflection] RFR: Model lifetimes of onnx session-related
 objects more explicitly [v5]
In-Reply-To: <LZJy7e6WX05haGDskbuDjBhX-AFh8iDbY8s0ew7mc5o=.c46bfba6-e8bc-487c-800a-a51b8a4859b9@github.com>
References: <s1RyI3MmwaIiMa3H__eLqjIAHbGb058ONRI3UZVtUkY=.4fd6ec15-bf2b-4f10-b168-c7d9d0dc23ed@github.com>
 <Ny6Sl8zEvk6muIFdS496A1C2GiRWjoz6HSleK72-R-s=.72468bfa-ab01-4d60-b95a-38f80c4d9f5d@github.com>
 <LZJy7e6WX05haGDskbuDjBhX-AFh8iDbY8s0ew7mc5o=.c46bfba6-e8bc-487c-800a-a51b8a4859b9@github.com>
Message-ID: <f-yH1OOfeEMzipy4CwFJrh3rMtpvt7yhXvP4Gb2DpDQ=.58c4f0f7-923d-4ca0-82a7-6a2518a0e931@github.com>

On Mon, 3 Mar 2025 22:06:44 GMT, Paul Sandoz <psandoz at openjdk.org> wrote:

>> Maurizio Cimadamore has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   Fix missing arena parameter
>
> cr-examples/onnx/src/main/java/oracle/code/onnx/OnnxRuntime.java line 138:
> 
>> 136:     }
>> 137: 
>> 138:     public static <T> Tensor<T> execute(MethodHandles.Lookup l, OnnxFunction<Tensor<T>> codeLambda, Arena sessionArena) {
> 
> Can we make the `codeLambda` be the last parameter? Then the arguments are more easily identifiable, with the "configuration" stuff prefixed.

Fixed here: https://github.com/openjdk/babylon/pull/334

-------------

PR Review Comment: https://git.openjdk.org/babylon/pull/332#discussion_r1978830936

From mcimadamore at openjdk.org  Tue Mar  4 08:02:52 2025
From: mcimadamore at openjdk.org (Maurizio Cimadamore)
Date: Tue, 4 Mar 2025 08:02:52 GMT
Subject: git: openjdk/babylon: code-reflection: Tweak order of arena parameter
 in OnnxRuntime::execute
Message-ID: <aa3c07bf-9f1a-4c43-b74f-0d7500104962@openjdk.org>

Changeset: 78a993ca
Branch: code-reflection
Author:    Maurizio Cimadamore <mcimadamore at openjdk.org>
Date:      2025-03-04 08:00:41 +0000
URL:       https://git.openjdk.org/babylon/commit/78a993ca32681cd454370b653a0ae8028478d3ce

Tweak order of arena parameter in OnnxRuntime::execute

! cr-examples/onnx/src/main/java/oracle/code/onnx/OnnxRuntime.java
! cr-examples/onnx/src/test/java/oracle/code/onnx/mnist/MNISTDemo.java


From mcimadamore at openjdk.org  Tue Mar  4 08:03:29 2025
From: mcimadamore at openjdk.org (Maurizio Cimadamore)
Date: Tue, 4 Mar 2025 08:03:29 GMT
Subject: [code-reflection] Integrated: Tweak order of arena parameter in
 OnnxRuntime::execute
Message-ID: <MyajHY61-4TFPZHNbUNlU47c4xK0SFrEJyoNDkijWKM=.95a371ed-36b0-4473-a1c5-7674e3a5fa3e@github.com>

As the subject say, this is a trivial change to make the code lambda the last parameter of the `OnnxRuntime::execute` methods.

-------------

Commit messages:
 - Tweak order of arena parameter in OnnxRuntime::execute

Changes: https://git.openjdk.org/babylon/pull/334/files
  Webrev: https://webrevs.openjdk.org/?repo=babylon&pr=334&range=00
  Stats: 4 lines in 2 files changed: 0 ins; 0 del; 4 mod
  Patch: https://git.openjdk.org/babylon/pull/334.diff
  Fetch: git fetch https://git.openjdk.org/babylon.git pull/334/head:pull/334

PR: https://git.openjdk.org/babylon/pull/334

From mcimadamore at openjdk.org  Tue Mar  4 08:03:29 2025
From: mcimadamore at openjdk.org (Maurizio Cimadamore)
Date: Tue, 4 Mar 2025 08:03:29 GMT
Subject: [code-reflection] Integrated: Tweak order of arena parameter in
 OnnxRuntime::execute
In-Reply-To: <MyajHY61-4TFPZHNbUNlU47c4xK0SFrEJyoNDkijWKM=.95a371ed-36b0-4473-a1c5-7674e3a5fa3e@github.com>
References: <MyajHY61-4TFPZHNbUNlU47c4xK0SFrEJyoNDkijWKM=.95a371ed-36b0-4473-a1c5-7674e3a5fa3e@github.com>
Message-ID: <sf7DnXOybGflFJpAkSqgz5_l7ewCS1MXXwWBv5kg-NA=.17d7cbfb-e98c-4562-958e-6a75f89f4b02@github.com>

On Tue, 4 Mar 2025 07:58:09 GMT, Maurizio Cimadamore <mcimadamore at openjdk.org> wrote:

> As the subject say, this is a trivial change to make the code lambda the last parameter of the `OnnxRuntime::execute` methods.

This pull request has now been integrated.

Changeset: 78a993ca
Author:    Maurizio Cimadamore <mcimadamore at openjdk.org>
URL:       https://git.openjdk.org/babylon/commit/78a993ca32681cd454370b653a0ae8028478d3ce
Stats:     4 lines in 2 files changed: 0 ins; 0 del; 4 mod

Tweak order of arena parameter in OnnxRuntime::execute

-------------

PR: https://git.openjdk.org/babylon/pull/334

From mcimadamore at openjdk.org  Tue Mar  4 11:20:53 2025
From: mcimadamore at openjdk.org (Maurizio Cimadamore)
Date: Tue, 4 Mar 2025 11:20:53 GMT
Subject: [code-reflection] RFR: Add tensor factories that accept an external
 arena
Message-ID: <gqy2qvHR0a0ZtxL0KCESB0Fo5YQwsXmOMYZc7mhd37Y=.380b225e-3829-419d-ab6c-bd06cc1f3185@github.com>

This PR adds three overloaded `Tensor::ofShape` factories which take an external arena. The existing factories have been rewired to call the new ones with a new automatic arena.

This allows us to make a small improvement in the MNISTDemo, where we can now allocate a tensor whose lifetime is managed by the arena that is also used to manage the lifetime of all the resources created within onnx session.

-------------

Commit messages:
 - Initial push

Changes: https://git.openjdk.org/babylon/pull/335/files
  Webrev: https://webrevs.openjdk.org/?repo=babylon&pr=335&range=00
  Stats: 21 lines in 2 files changed: 15 ins; 2 del; 4 mod
  Patch: https://git.openjdk.org/babylon/pull/335.diff
  Fetch: git fetch https://git.openjdk.org/babylon.git pull/335/head:pull/335

PR: https://git.openjdk.org/babylon/pull/335

From gfrost at openjdk.org  Tue Mar  4 11:22:58 2025
From: gfrost at openjdk.org (Gary Frost)
Date: Tue, 4 Mar 2025 11:22:58 GMT
Subject: git: openjdk/babylon: code-reflection: Hat sync jextract and hat nbody
 and life demos
Message-ID: <bed757e3-b77c-4401-aba1-defa7a5dd84b@openjdk.org>

Changeset: 17b1bcd9
Branch: code-reflection
Author:    Gary Frost <gfrost at openjdk.org>
Date:      2025-03-04 11:21:35 +0000
URL:       https://git.openjdk.org/babylon/commit/17b1bcd964959bb7dcf6c3dd1eba2863103adde0

Hat sync jextract and hat nbody and life demos

! hat/backends/ffi/opencl/cpp/opencl_backend.cpp
! hat/backends/ffi/opencl/src/main/java/hat/backend/ffi/OpenCLBackend.java
! hat/backends/ffi/shared/include/shared.h
! hat/bld
! hat/examples/life/src/main/java/life/Main.java
- hat/examples/life/src/main/java/life/Main.java.no
! hat/examples/life/src/main/java/life/Viewer.java
- hat/examples/life/src/main/java/life/Viewer.java.no
- hat/examples/nbody/src/main/java/nbody/CLWrap.java
- hat/examples/nbody/src/main/java/nbody/GLWrap.java
! hat/examples/nbody/src/main/java/nbody/Main.java
- hat/examples/nbody/src/main/java/nbody/Main.java.no
! hat/examples/nbody/src/main/java/nbody/NBodyGLWindow.java
! hat/examples/nbody/src/main/java/nbody/opencl/NBody.java
! hat/hat/src/main/java/hat/buffer/Buffer.java
! hat/hat/src/main/java/hat/ifacemapper/SegmentMapper.java
! hat/intellij/.idea/misc.xml
! hat/intellij/clwrap.iml
! hat/intellij/life.iml
! hat/wrap/clwrap/src/main/java/wrap/clwrap/CLPlatform.java
! hat/wrap/clwrap/src/main/java/wrap/clwrap/CLWrapComputeContext.java
! hat/wrap/wrap/src/main/java/wrap/Sequence.java


From gfrost at openjdk.org  Tue Mar  4 11:24:24 2025
From: gfrost at openjdk.org (Gary Frost)
Date: Tue, 4 Mar 2025 11:24:24 GMT
Subject: [code-reflection] Integrated: Hat sync jextract and hat nbody and life
 demos
Message-ID: <4j5uMSKEm5gBaN_D5Hp0CYew8Fkg6zIyY_uGlFwZQOg=.831121bb-a655-4b26-93e0-52c6407a3052@github.com>

Trying to merge the jextract and HAT versions of game of life (and nbody) into single version.

This is step 1.

-------------

Commit messages:
 - whitespace
 - nbody and life demos now allow jextract and hat versions

Changes: https://git.openjdk.org/babylon/pull/336/files
  Webrev: https://webrevs.openjdk.org/?repo=babylon&pr=336&range=00
  Stats: 2778 lines in 22 files changed: 564 ins; 1902 del; 312 mod
  Patch: https://git.openjdk.org/babylon/pull/336.diff
  Fetch: git fetch https://git.openjdk.org/babylon.git pull/336/head:pull/336

PR: https://git.openjdk.org/babylon/pull/336

From gfrost at openjdk.org  Tue Mar  4 11:24:24 2025
From: gfrost at openjdk.org (Gary Frost)
Date: Tue, 4 Mar 2025 11:24:24 GMT
Subject: [code-reflection] Integrated: Hat sync jextract and hat nbody and life
 demos
In-Reply-To: <4j5uMSKEm5gBaN_D5Hp0CYew8Fkg6zIyY_uGlFwZQOg=.831121bb-a655-4b26-93e0-52c6407a3052@github.com>
References: <4j5uMSKEm5gBaN_D5Hp0CYew8Fkg6zIyY_uGlFwZQOg=.831121bb-a655-4b26-93e0-52c6407a3052@github.com>
Message-ID: <gPI7xgS0TU98JeYZV74PUcNS7ek55vigkLfKmASWq_c=.510b6111-376f-4777-b945-eacd70752197@github.com>

On Tue, 4 Mar 2025 11:18:28 GMT, Gary Frost <gfrost at openjdk.org> wrote:

> Trying to merge the jextract and HAT versions of game of life (and nbody) into single version.
> 
> This is step 1.

This pull request has now been integrated.

Changeset: 17b1bcd9
Author:    Gary Frost <gfrost at openjdk.org>
URL:       https://git.openjdk.org/babylon/commit/17b1bcd964959bb7dcf6c3dd1eba2863103adde0
Stats:     2778 lines in 22 files changed: 564 ins; 1902 del; 312 mod

Hat sync jextract and hat nbody and life demos

-------------

PR: https://git.openjdk.org/babylon/pull/336

From asotona at openjdk.org  Tue Mar  4 12:30:23 2025
From: asotona at openjdk.org (Adam Sotona)
Date: Tue, 4 Mar 2025 12:30:23 GMT
Subject: [code-reflection] RFR: Explicit Arena passing to Tensor construction
 and Session execution
Message-ID: <cDmBlH4c7cgE7faVIwtu_jlQPNe2iLd02rsV8ZcKfH8=.114e41bd-59a5-4078-8f29-bd7f7996e496@github.com>

Holding `Arena` instances in `OnnxRuntime` and `Session` still cause memory leaks.
This patch allows to pass explicit arena for `Tensor` construction, `Session` construction and  each `Session` run.

-------------

Commit messages:
 - Merge remote-tracking branch 'babylon/code-reflection' into arenas-cont
 - passing Arena instances to tensor construction and Session run

Changes: https://git.openjdk.org/babylon/pull/337/files
  Webrev: https://webrevs.openjdk.org/?repo=babylon&pr=337&range=00
  Stats: 142 lines in 6 files changed: 43 ins; 13 del; 86 mod
  Patch: https://git.openjdk.org/babylon/pull/337.diff
  Fetch: git fetch https://git.openjdk.org/babylon.git pull/337/head:pull/337

PR: https://git.openjdk.org/babylon/pull/337

From asotona at openjdk.org  Tue Mar  4 12:37:18 2025
From: asotona at openjdk.org (Adam Sotona)
Date: Tue, 4 Mar 2025 12:37:18 GMT
Subject: [code-reflection] RFR: Explicit Arena passing to Tensor
 construction and Session execution [v2]
In-Reply-To: <cDmBlH4c7cgE7faVIwtu_jlQPNe2iLd02rsV8ZcKfH8=.114e41bd-59a5-4078-8f29-bd7f7996e496@github.com>
References: <cDmBlH4c7cgE7faVIwtu_jlQPNe2iLd02rsV8ZcKfH8=.114e41bd-59a5-4078-8f29-bd7f7996e496@github.com>
Message-ID: <y1PSEiG46wTAVeRQ88WVtYzr4986Xi25gfzShIdZdtw=.5e6a3fe2-8d6d-456d-9c1f-725b2e519dff@github.com>

> Holding `Arena` instances in `OnnxRuntime` and `Session` still cause memory leaks.
> This patch allows to pass explicit arena for `Tensor` construction, `Session` construction and  each `Session` run.

Adam Sotona has updated the pull request incrementally with one additional commit since the last revision:

  using Arena.ofConfined() instance for each individual test in CNNTest

-------------

Changes:
  - all: https://git.openjdk.org/babylon/pull/337/files
  - new: https://git.openjdk.org/babylon/pull/337/files/58db6d00..3d33898d

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=babylon&pr=337&range=01
 - incr: https://webrevs.openjdk.org/?repo=babylon&pr=337&range=00-01

  Stats: 45 lines in 1 file changed: 6 ins; 0 del; 39 mod
  Patch: https://git.openjdk.org/babylon/pull/337.diff
  Fetch: git fetch https://git.openjdk.org/babylon.git pull/337/head:pull/337

PR: https://git.openjdk.org/babylon/pull/337

From asotona at openjdk.org  Tue Mar  4 13:00:47 2025
From: asotona at openjdk.org (Adam Sotona)
Date: Tue, 4 Mar 2025 13:00:47 GMT
Subject: [code-reflection] RFR: Explicit Arena passing to Tensor
 construction and Session execution [v3]
In-Reply-To: <cDmBlH4c7cgE7faVIwtu_jlQPNe2iLd02rsV8ZcKfH8=.114e41bd-59a5-4078-8f29-bd7f7996e496@github.com>
References: <cDmBlH4c7cgE7faVIwtu_jlQPNe2iLd02rsV8ZcKfH8=.114e41bd-59a5-4078-8f29-bd7f7996e496@github.com>
Message-ID: <YrMtbpvy8o-IsbI1q6UVmQsU2V3UUp204JV77UESoXo=.9548adb3-0cde-4f02-b514-ef96a5597ed2@github.com>

> Holding `Arena` instances in `OnnxRuntime` and `Session` still cause memory leaks.
> This patch allows to pass explicit arena for `Tensor` construction, `Session` construction and  each `Session` run.

Adam Sotona has updated the pull request incrementally with one additional commit since the last revision:

  more explicit arena use

-------------

Changes:
  - all: https://git.openjdk.org/babylon/pull/337/files
  - new: https://git.openjdk.org/babylon/pull/337/files/3d33898d..9bd3711d

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=babylon&pr=337&range=02
 - incr: https://webrevs.openjdk.org/?repo=babylon&pr=337&range=01-02

  Stats: 5 lines in 2 files changed: 1 ins; 0 del; 4 mod
  Patch: https://git.openjdk.org/babylon/pull/337.diff
  Fetch: git fetch https://git.openjdk.org/babylon.git pull/337/head:pull/337

PR: https://git.openjdk.org/babylon/pull/337

From asotona at openjdk.org  Tue Mar  4 13:07:30 2025
From: asotona at openjdk.org (Adam Sotona)
Date: Tue, 4 Mar 2025 13:07:30 GMT
Subject: [code-reflection] RFR: Explicit Arena passing to Tensor
 construction and Session execution [v4]
In-Reply-To: <cDmBlH4c7cgE7faVIwtu_jlQPNe2iLd02rsV8ZcKfH8=.114e41bd-59a5-4078-8f29-bd7f7996e496@github.com>
References: <cDmBlH4c7cgE7faVIwtu_jlQPNe2iLd02rsV8ZcKfH8=.114e41bd-59a5-4078-8f29-bd7f7996e496@github.com>
Message-ID: <XevOScaN4QksXNyvCdLBcluh8BPLvH4ez33_ZY15PGI=.e86ee060-b876-47bd-9e0c-081415795912@github.com>

> Holding `Arena` instances in `OnnxRuntime` and `Session` still cause memory leaks.
> This patch allows to pass explicit arena for `Tensor` construction, `Session` construction and  each `Session` run.

Adam Sotona has updated the pull request incrementally with one additional commit since the last revision:

  releasing tensors

-------------

Changes:
  - all: https://git.openjdk.org/babylon/pull/337/files
  - new: https://git.openjdk.org/babylon/pull/337/files/9bd3711d..52f18047

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=babylon&pr=337&range=03
 - incr: https://webrevs.openjdk.org/?repo=babylon&pr=337&range=02-03

  Stats: 7 lines in 2 files changed: 6 ins; 0 del; 1 mod
  Patch: https://git.openjdk.org/babylon/pull/337.diff
  Fetch: git fetch https://git.openjdk.org/babylon.git pull/337/head:pull/337

PR: https://git.openjdk.org/babylon/pull/337

From asotona at openjdk.org  Tue Mar  4 13:29:32 2025
From: asotona at openjdk.org (Adam Sotona)
Date: Tue, 4 Mar 2025 13:29:32 GMT
Subject: [code-reflection] RFR: Explicit Arena passing to Tensor
 construction and Session execution [v5]
In-Reply-To: <cDmBlH4c7cgE7faVIwtu_jlQPNe2iLd02rsV8ZcKfH8=.114e41bd-59a5-4078-8f29-bd7f7996e496@github.com>
References: <cDmBlH4c7cgE7faVIwtu_jlQPNe2iLd02rsV8ZcKfH8=.114e41bd-59a5-4078-8f29-bd7f7996e496@github.com>
Message-ID: <1JEtc7c5yg2dNN4lDstMMo2GRp05fpPuTHt_TpaCCBU=.40e6d157-2e4d-4118-af19-62942c46ed9d@github.com>

> Holding `Arena` instances in `OnnxRuntime` and `Session` still cause memory leaks.
> This patch allows to pass explicit arena for `Tensor` construction, `Session` construction and  each `Session` run.

Adam Sotona has updated the pull request incrementally with one additional commit since the last revision:

  tensor construction moved to runtime

-------------

Changes:
  - all: https://git.openjdk.org/babylon/pull/337/files
  - new: https://git.openjdk.org/babylon/pull/337/files/52f18047..6be08566

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=babylon&pr=337&range=04
 - incr: https://webrevs.openjdk.org/?repo=babylon&pr=337&range=03-04

  Stats: 13 lines in 2 files changed: 2 ins; 4 del; 7 mod
  Patch: https://git.openjdk.org/babylon/pull/337.diff
  Fetch: git fetch https://git.openjdk.org/babylon.git pull/337/head:pull/337

PR: https://git.openjdk.org/babylon/pull/337

From asotona at openjdk.org  Tue Mar  4 13:34:15 2025
From: asotona at openjdk.org (Adam Sotona)
Date: Tue, 4 Mar 2025 13:34:15 GMT
Subject: [code-reflection] RFR: Explicit Arena passing to Tensor
 construction and Session execution [v6]
In-Reply-To: <cDmBlH4c7cgE7faVIwtu_jlQPNe2iLd02rsV8ZcKfH8=.114e41bd-59a5-4078-8f29-bd7f7996e496@github.com>
References: <cDmBlH4c7cgE7faVIwtu_jlQPNe2iLd02rsV8ZcKfH8=.114e41bd-59a5-4078-8f29-bd7f7996e496@github.com>
Message-ID: <Xhsch2_OAam7Cdw260RTUZc4TVveFdnFbOfKb7PHEh0=.3ffdda80-7cf7-4125-82ab-d5b4fa60046c@github.com>

> Holding `Arena` instances in `OnnxRuntime` and `Session` still cause memory leaks.
> This patch allows to pass explicit arena for `Tensor` construction, `Session` construction and  each `Session` run.

Adam Sotona has updated the pull request incrementally with one additional commit since the last revision:

  CNNTest cleanup

-------------

Changes:
  - all: https://git.openjdk.org/babylon/pull/337/files
  - new: https://git.openjdk.org/babylon/pull/337/files/6be08566..b974d53c

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=babylon&pr=337&range=05
 - incr: https://webrevs.openjdk.org/?repo=babylon&pr=337&range=04-05

  Stats: 34 lines in 1 file changed: 3 ins; 2 del; 29 mod
  Patch: https://git.openjdk.org/babylon/pull/337.diff
  Fetch: git fetch https://git.openjdk.org/babylon.git pull/337/head:pull/337

PR: https://git.openjdk.org/babylon/pull/337

From asotona at openjdk.org  Tue Mar  4 13:46:35 2025
From: asotona at openjdk.org (Adam Sotona)
Date: Tue, 4 Mar 2025 13:46:35 GMT
Subject: [code-reflection] RFR: Explicit Arena passing to Tensor
 construction and Session execution [v7]
In-Reply-To: <cDmBlH4c7cgE7faVIwtu_jlQPNe2iLd02rsV8ZcKfH8=.114e41bd-59a5-4078-8f29-bd7f7996e496@github.com>
References: <cDmBlH4c7cgE7faVIwtu_jlQPNe2iLd02rsV8ZcKfH8=.114e41bd-59a5-4078-8f29-bd7f7996e496@github.com>
Message-ID: <M90seTHqxu6PTjTFtZ9D35J_qfCOCn7pgLBvyBymHYs=.d3e4b270-e93e-4e0d-a38b-0982e0c0d907@github.com>

> Holding `Arena` instances in `OnnxRuntime` and `Session` still cause memory leaks.
> This patch allows to pass explicit arena for `Tensor` construction, `Session` construction and  each `Session` run.
> Improved handling of `Session` life cycle allows to cache live `Session` instead of its protobuf model.

Adam Sotona has updated the pull request incrementally with one additional commit since the last revision:

  Caching live sessions insted of proto models

-------------

Changes:
  - all: https://git.openjdk.org/babylon/pull/337/files
  - new: https://git.openjdk.org/babylon/pull/337/files/b974d53c..b97afd8d

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=babylon&pr=337&range=06
 - incr: https://webrevs.openjdk.org/?repo=babylon&pr=337&range=05-06

  Stats: 16 lines in 1 file changed: 2 ins; 6 del; 8 mod
  Patch: https://git.openjdk.org/babylon/pull/337.diff
  Fetch: git fetch https://git.openjdk.org/babylon.git pull/337/head:pull/337

PR: https://git.openjdk.org/babylon/pull/337

From mcimadamore at openjdk.org  Tue Mar  4 13:59:09 2025
From: mcimadamore at openjdk.org (Maurizio Cimadamore)
Date: Tue, 4 Mar 2025 13:59:09 GMT
Subject: [code-reflection] Withdrawn: Add tensor factories that accept an
 external arena
In-Reply-To: <gqy2qvHR0a0ZtxL0KCESB0Fo5YQwsXmOMYZc7mhd37Y=.380b225e-3829-419d-ab6c-bd06cc1f3185@github.com>
References: <gqy2qvHR0a0ZtxL0KCESB0Fo5YQwsXmOMYZc7mhd37Y=.380b225e-3829-419d-ab6c-bd06cc1f3185@github.com>
Message-ID: <BSSITm2LkjxCZov65oJf-Ul4PxwMcanditRgZpdpqqA=.f35476ad-7666-4208-95a4-19c5452910db@github.com>

On Tue, 4 Mar 2025 11:16:03 GMT, Maurizio Cimadamore <mcimadamore at openjdk.org> wrote:

> This PR adds three overloaded `Tensor::ofShape` factories which take an external arena. The existing factories have been rewired to call the new ones with a new automatic arena.
> 
> This allows us to make a small improvement in the MNISTDemo, where we can now allocate a tensor whose lifetime is managed by the arena that is also used to manage the lifetime of all the resources created within onnx session.

This pull request has been closed without being integrated.

-------------

PR: https://git.openjdk.org/babylon/pull/335

From mcimadamore at openjdk.org  Tue Mar  4 13:59:09 2025
From: mcimadamore at openjdk.org (Maurizio Cimadamore)
Date: Tue, 4 Mar 2025 13:59:09 GMT
Subject: [code-reflection] RFR: Add tensor factories that accept an
 external arena
In-Reply-To: <gqy2qvHR0a0ZtxL0KCESB0Fo5YQwsXmOMYZc7mhd37Y=.380b225e-3829-419d-ab6c-bd06cc1f3185@github.com>
References: <gqy2qvHR0a0ZtxL0KCESB0Fo5YQwsXmOMYZc7mhd37Y=.380b225e-3829-419d-ab6c-bd06cc1f3185@github.com>
Message-ID: <T3zemZ9GKH6HnG-EVuH19WGA8Ouvf54X4ek2FsyDXmk=.f153b887-51f4-427d-92e7-5c2a6a2f6ba3@github.com>

On Tue, 4 Mar 2025 11:16:03 GMT, Maurizio Cimadamore <mcimadamore at openjdk.org> wrote:

> This PR adds three overloaded `Tensor::ofShape` factories which take an external arena. The existing factories have been rewired to call the new ones with a new automatic arena.
> 
> This allows us to make a small improvement in the MNISTDemo, where we can now allocate a tensor whose lifetime is managed by the arena that is also used to manage the lifetime of all the resources created within onnx session.

Closing as this work is included here: https://github.com/openjdk/babylon/pull/337

-------------

PR Comment: https://git.openjdk.org/babylon/pull/335#issuecomment-2697694257

From mcimadamore at openjdk.org  Tue Mar  4 14:03:04 2025
From: mcimadamore at openjdk.org (Maurizio Cimadamore)
Date: Tue, 4 Mar 2025 14:03:04 GMT
Subject: [code-reflection] RFR: Explicit Arena passing to Tensor
 construction and Session execution [v2]
In-Reply-To: <y1PSEiG46wTAVeRQ88WVtYzr4986Xi25gfzShIdZdtw=.5e6a3fe2-8d6d-456d-9c1f-725b2e519dff@github.com>
References: <cDmBlH4c7cgE7faVIwtu_jlQPNe2iLd02rsV8ZcKfH8=.114e41bd-59a5-4078-8f29-bd7f7996e496@github.com>
 <y1PSEiG46wTAVeRQ88WVtYzr4986Xi25gfzShIdZdtw=.5e6a3fe2-8d6d-456d-9c1f-725b2e519dff@github.com>
Message-ID: <X47Lt776YTM53NvJUUcJ1bu1ydAR1o628RsfISbnuc8=.ba0b4805-befa-4194-bd7e-fd110173c488@github.com>

On Tue, 4 Mar 2025 12:37:18 GMT, Adam Sotona <asotona at openjdk.org> wrote:

>> Holding `Arena` instances in `OnnxRuntime` and `Session` still cause memory leaks.
>> This patch allows to pass explicit arena for `Tensor` construction, `Session` construction and  each `Session` run.
>> Improved handling of `Session` life cycle allows to cache live `Session` instead of its protobuf model.
>
> Adam Sotona has updated the pull request incrementally with one additional commit since the last revision:
> 
>   using Arena.ofConfined() instance for each individual test in CNNTest

cr-examples/onnx/src/main/java/oracle/code/onnx/OnnxRuntime.java line 315:

> 313:         private final MemorySegment sessionAddress;
> 314: 
> 315:         private Session(Arena arena, MemorySegment sessionAddress) {

I don't think this is an improvement? The idea behind the current code is that  a session has a lifetime (managed by an arena) -- and all the resources allocated during `run` are associated to that same lifetime.

In the current code, `arena` is passed as parameter to both the constructor and `run` -- which means you could pass two different arenas, and that would result in subtle use-after-free issues.

-------------

PR Review Comment: https://git.openjdk.org/babylon/pull/337#discussion_r1979505779

From asotona at openjdk.org  Tue Mar  4 14:09:10 2025
From: asotona at openjdk.org (Adam Sotona)
Date: Tue, 4 Mar 2025 14:09:10 GMT
Subject: [code-reflection] RFR: Explicit Arena passing to Tensor
 construction and Session execution [v2]
In-Reply-To: <R1tcr3GxrjBawtiNBBZ1j-SqkMsJ3T8LtHz7gwMgq0E=.941815e1-78fd-4321-989a-9820916fac23@github.com>
References: <cDmBlH4c7cgE7faVIwtu_jlQPNe2iLd02rsV8ZcKfH8=.114e41bd-59a5-4078-8f29-bd7f7996e496@github.com>
 <y1PSEiG46wTAVeRQ88WVtYzr4986Xi25gfzShIdZdtw=.5e6a3fe2-8d6d-456d-9c1f-725b2e519dff@github.com>
 <X47Lt776YTM53NvJUUcJ1bu1ydAR1o628RsfISbnuc8=.ba0b4805-befa-4194-bd7e-fd110173c488@github.com>
 <R1tcr3GxrjBawtiNBBZ1j-SqkMsJ3T8LtHz7gwMgq0E=.941815e1-78fd-4321-989a-9820916fac23@github.com>
Message-ID: <PhMlb3M4mxcqLGkTRaiOWinl5n7oaC4R7wJ524vzay8=.7a177f95-4ad8-444a-9479-89e1693705aa@github.com>

On Tue, 4 Mar 2025 14:04:52 GMT, Maurizio Cimadamore <mcimadamore at openjdk.org> wrote:

>> cr-examples/onnx/src/main/java/oracle/code/onnx/OnnxRuntime.java line 315:
>> 
>>> 313:         private final MemorySegment sessionAddress;
>>> 314: 
>>> 315:         private Session(Arena arena, MemorySegment sessionAddress) {
>> 
>> I don't think this is an improvement? The idea behind the current code is that  a session has a lifetime (managed by an arena) -- and all the resources allocated during `run` are associated to that same lifetime.
>> 
>> In the current code, `arena` is passed as parameter to both the constructor and `run` -- which means you could pass two different arenas, and that would result in subtle use-after-free issues.
>
> (I see that this is done to cache the session --- but IMHO this feels like the caching tail is wagging the API dog a bit).

That is the intention. If we want to cache sessions and allow multiple executions, they must be constructed with a different arena than each of the execution. Execution has different lifecycle than the session.

-------------

PR Review Comment: https://git.openjdk.org/babylon/pull/337#discussion_r1979526913

From mcimadamore at openjdk.org  Tue Mar  4 14:09:10 2025
From: mcimadamore at openjdk.org (Maurizio Cimadamore)
Date: Tue, 4 Mar 2025 14:09:10 GMT
Subject: [code-reflection] RFR: Explicit Arena passing to Tensor
 construction and Session execution [v2]
In-Reply-To: <X47Lt776YTM53NvJUUcJ1bu1ydAR1o628RsfISbnuc8=.ba0b4805-befa-4194-bd7e-fd110173c488@github.com>
References: <cDmBlH4c7cgE7faVIwtu_jlQPNe2iLd02rsV8ZcKfH8=.114e41bd-59a5-4078-8f29-bd7f7996e496@github.com>
 <y1PSEiG46wTAVeRQ88WVtYzr4986Xi25gfzShIdZdtw=.5e6a3fe2-8d6d-456d-9c1f-725b2e519dff@github.com>
 <X47Lt776YTM53NvJUUcJ1bu1ydAR1o628RsfISbnuc8=.ba0b4805-befa-4194-bd7e-fd110173c488@github.com>
Message-ID: <R1tcr3GxrjBawtiNBBZ1j-SqkMsJ3T8LtHz7gwMgq0E=.941815e1-78fd-4321-989a-9820916fac23@github.com>

On Tue, 4 Mar 2025 13:58:46 GMT, Maurizio Cimadamore <mcimadamore at openjdk.org> wrote:

>> Adam Sotona has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   using Arena.ofConfined() instance for each individual test in CNNTest
>
> cr-examples/onnx/src/main/java/oracle/code/onnx/OnnxRuntime.java line 315:
> 
>> 313:         private final MemorySegment sessionAddress;
>> 314: 
>> 315:         private Session(Arena arena, MemorySegment sessionAddress) {
> 
> I don't think this is an improvement? The idea behind the current code is that  a session has a lifetime (managed by an arena) -- and all the resources allocated during `run` are associated to that same lifetime.
> 
> In the current code, `arena` is passed as parameter to both the constructor and `run` -- which means you could pass two different arenas, and that would result in subtle use-after-free issues.

(I see that this is done to cache the session --- but IMHO this feels like the caching tail is wagging the API dog a bit).

-------------

PR Review Comment: https://git.openjdk.org/babylon/pull/337#discussion_r1979523003

From mcimadamore at openjdk.org  Tue Mar  4 14:40:04 2025
From: mcimadamore at openjdk.org (Maurizio Cimadamore)
Date: Tue, 4 Mar 2025 14:40:04 GMT
Subject: [code-reflection] RFR: Explicit Arena passing to Tensor
 construction and Session execution [v2]
In-Reply-To: <PhMlb3M4mxcqLGkTRaiOWinl5n7oaC4R7wJ524vzay8=.7a177f95-4ad8-444a-9479-89e1693705aa@github.com>
References: <cDmBlH4c7cgE7faVIwtu_jlQPNe2iLd02rsV8ZcKfH8=.114e41bd-59a5-4078-8f29-bd7f7996e496@github.com>
 <y1PSEiG46wTAVeRQ88WVtYzr4986Xi25gfzShIdZdtw=.5e6a3fe2-8d6d-456d-9c1f-725b2e519dff@github.com>
 <X47Lt776YTM53NvJUUcJ1bu1ydAR1o628RsfISbnuc8=.ba0b4805-befa-4194-bd7e-fd110173c488@github.com>
 <R1tcr3GxrjBawtiNBBZ1j-SqkMsJ3T8LtHz7gwMgq0E=.941815e1-78fd-4321-989a-9820916fac23@github.com>
 <PhMlb3M4mxcqLGkTRaiOWinl5n7oaC4R7wJ524vzay8=.7a177f95-4ad8-444a-9479-89e1693705aa@github.com>
Message-ID: <mDVzDaVVmaZscMoaRxq2bi2yYhmq1xh6XhLM0JyaYQA=.bbf9729b-da7b-47ae-8a76-12ed2276d1ef@github.com>

On Tue, 4 Mar 2025 14:06:22 GMT, Adam Sotona <asotona at openjdk.org> wrote:

>> (I see that this is done to cache the session --- but IMHO this feels like the caching tail is wagging the API dog a bit).
>
> That is the intention. If we want to cache sessions and allow multiple executions, they must be constructed with a different arena than each of the execution. Execution has different lifecycle than the session.

Ok -- since you have a situation of dependent lifetimes (e.g. you cannot afford the session arena to be closed _while_ the `run` method is executing), I'd suggest to _always_ create the session using an automatic arena. Then having a lifetime dependency becomes non-problematic, because the `Session` object you call `run` on will always keep the session address alive. But if we leave to users the choice of which arena to pick for the session, they could shoot themselves in the foot, by picking e.g. a shared arena and then closing it _while_ `run` is executing.

-------------

PR Review Comment: https://git.openjdk.org/babylon/pull/337#discussion_r1979589144

From asotona at openjdk.org  Tue Mar  4 14:40:04 2025
From: asotona at openjdk.org (Adam Sotona)
Date: Tue, 4 Mar 2025 14:40:04 GMT
Subject: [code-reflection] RFR: Explicit Arena passing to Tensor
 construction and Session execution [v2]
In-Reply-To: <mDVzDaVVmaZscMoaRxq2bi2yYhmq1xh6XhLM0JyaYQA=.bbf9729b-da7b-47ae-8a76-12ed2276d1ef@github.com>
References: <cDmBlH4c7cgE7faVIwtu_jlQPNe2iLd02rsV8ZcKfH8=.114e41bd-59a5-4078-8f29-bd7f7996e496@github.com>
 <y1PSEiG46wTAVeRQ88WVtYzr4986Xi25gfzShIdZdtw=.5e6a3fe2-8d6d-456d-9c1f-725b2e519dff@github.com>
 <X47Lt776YTM53NvJUUcJ1bu1ydAR1o628RsfISbnuc8=.ba0b4805-befa-4194-bd7e-fd110173c488@github.com>
 <R1tcr3GxrjBawtiNBBZ1j-SqkMsJ3T8LtHz7gwMgq0E=.941815e1-78fd-4321-989a-9820916fac23@github.com>
 <PhMlb3M4mxcqLGkTRaiOWinl5n7oaC4R7wJ524vzay8=.7a177f95-4ad8-444a-9479-89e1693705aa@github.com>
 <mDVzDaVVmaZscMoaRxq2bi2yYhmq1xh6XhLM0JyaYQA=.bbf9729b-da7b-47ae-8a76-12ed2276d1ef@github.com>
Message-ID: <8hC24PjPXKZVl-MqhG7qxZSc7p2V-7pVHr6LBKgjXMw=.261b9d75-29b2-48aa-b082-1037920b99e7@github.com>

On Tue, 4 Mar 2025 14:35:23 GMT, Maurizio Cimadamore <mcimadamore at openjdk.org> wrote:

>> That is the intention. If we want to cache sessions and allow multiple executions, they must be constructed with a different arena than each of the execution. Execution has different lifecycle than the session.
>
> Ok -- since you have a situation of dependent lifetimes (e.g. you cannot afford the session arena to be closed _while_ the `run` method is executing), I'd suggest to _always_ create the session using an automatic arena. Then having a lifetime dependency becomes non-problematic, because the `Session` object you call `run` on will always keep the session address alive. But if we leave to users the choice of which arena to pick for the session, they could shoot themselves in the foot, by picking e.g. a shared arena and then closing it _while_ `run` is executing.

I don't see a problem here. Users can also close a stream while reading it.

-------------

PR Review Comment: https://git.openjdk.org/babylon/pull/337#discussion_r1979593310

From mcimadamore at openjdk.org  Tue Mar  4 14:50:13 2025
From: mcimadamore at openjdk.org (Maurizio Cimadamore)
Date: Tue, 4 Mar 2025 14:50:13 GMT
Subject: [code-reflection] RFR: Explicit Arena passing to Tensor
 construction and Session execution [v7]
In-Reply-To: <M90seTHqxu6PTjTFtZ9D35J_qfCOCn7pgLBvyBymHYs=.d3e4b270-e93e-4e0d-a38b-0982e0c0d907@github.com>
References: <cDmBlH4c7cgE7faVIwtu_jlQPNe2iLd02rsV8ZcKfH8=.114e41bd-59a5-4078-8f29-bd7f7996e496@github.com>
 <M90seTHqxu6PTjTFtZ9D35J_qfCOCn7pgLBvyBymHYs=.d3e4b270-e93e-4e0d-a38b-0982e0c0d907@github.com>
Message-ID: <iOLaCm9qGGaboyZUWFv0zoIRKAhDJy0JTnAQ9u8IkMw=.9e34cd4d-6df3-416b-a6ba-d32fb3daf396@github.com>

On Tue, 4 Mar 2025 13:46:35 GMT, Adam Sotona <asotona at openjdk.org> wrote:

>> Holding `Arena` instances in `OnnxRuntime` and `Session` still cause memory leaks.
>> This patch allows to pass explicit arena for `Tensor` construction, `Session` construction and  each `Session` run.
>> Improved handling of `Session` life cycle allows to cache live `Session` instead of its protobuf model.
>
> Adam Sotona has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Caching live sessions insted of proto models

cr-examples/onnx/src/main/java/oracle/code/onnx/OnnxRuntime.java line 362:

> 360:     }
> 361: 
> 362:     public MemorySegment createTensor(Arena arena, MemorySegment flatData, Tensor.ElementType elementType, long[] shape) {

good catch!

cr-examples/onnx/src/main/java/oracle/code/onnx/OnnxRuntime.java line 379:

> 377: 
> 378:     public long[] tensorShape(MemorySegment tensorAddr) {
> 379:         try (var arena = Arena.ofConfined()) {

Also, good catch!

-------------

PR Review Comment: https://git.openjdk.org/babylon/pull/337#discussion_r1979612096
PR Review Comment: https://git.openjdk.org/babylon/pull/337#discussion_r1979612427

From mcimadamore at openjdk.org  Tue Mar  4 14:54:12 2025
From: mcimadamore at openjdk.org (Maurizio Cimadamore)
Date: Tue, 4 Mar 2025 14:54:12 GMT
Subject: [code-reflection] RFR: Explicit Arena passing to Tensor
 construction and Session execution [v2]
In-Reply-To: <8hC24PjPXKZVl-MqhG7qxZSc7p2V-7pVHr6LBKgjXMw=.261b9d75-29b2-48aa-b082-1037920b99e7@github.com>
References: <cDmBlH4c7cgE7faVIwtu_jlQPNe2iLd02rsV8ZcKfH8=.114e41bd-59a5-4078-8f29-bd7f7996e496@github.com>
 <y1PSEiG46wTAVeRQ88WVtYzr4986Xi25gfzShIdZdtw=.5e6a3fe2-8d6d-456d-9c1f-725b2e519dff@github.com>
 <X47Lt776YTM53NvJUUcJ1bu1ydAR1o628RsfISbnuc8=.ba0b4805-befa-4194-bd7e-fd110173c488@github.com>
 <R1tcr3GxrjBawtiNBBZ1j-SqkMsJ3T8LtHz7gwMgq0E=.941815e1-78fd-4321-989a-9820916fac23@github.com>
 <PhMlb3M4mxcqLGkTRaiOWinl5n7oaC4R7wJ524vzay8=.7a177f95-4ad8-444a-9479-89e1693705aa@github.com>
 <mDVzDaVVmaZscMoaRxq2bi2yYhmq1xh6XhLM0JyaYQA=.bbf9729b-da7b-47ae-8a76-12ed2276d1ef@github.com>
 <8hC24PjPXKZVl-MqhG7qxZSc7p2V-7pVHr6LBKgjXMw=.261b9d75-29b2-48aa-b082-1037920b99e7@github.com>
Message-ID: <jmk4PiJ0KxtprtlbNr0lc99tPCfOgPvg0oHQKskouik=.a2b322bb-b11a-412f-9cba-cda198769c17@github.com>

On Tue, 4 Mar 2025 14:37:45 GMT, Adam Sotona <asotona at openjdk.org> wrote:

> I don't see a problem here. Users can also close a stream while reading it.

Yes, and that would cause an exception -- not a VM crash.

-------------

PR Review Comment: https://git.openjdk.org/babylon/pull/337#discussion_r1979619104

From asotona at openjdk.org  Tue Mar  4 15:11:04 2025
From: asotona at openjdk.org (Adam Sotona)
Date: Tue, 4 Mar 2025 15:11:04 GMT
Subject: [code-reflection] RFR: Explicit Arena passing to Tensor
 construction and Session execution [v2]
In-Reply-To: <jmk4PiJ0KxtprtlbNr0lc99tPCfOgPvg0oHQKskouik=.a2b322bb-b11a-412f-9cba-cda198769c17@github.com>
References: <cDmBlH4c7cgE7faVIwtu_jlQPNe2iLd02rsV8ZcKfH8=.114e41bd-59a5-4078-8f29-bd7f7996e496@github.com>
 <y1PSEiG46wTAVeRQ88WVtYzr4986Xi25gfzShIdZdtw=.5e6a3fe2-8d6d-456d-9c1f-725b2e519dff@github.com>
 <X47Lt776YTM53NvJUUcJ1bu1ydAR1o628RsfISbnuc8=.ba0b4805-befa-4194-bd7e-fd110173c488@github.com>
 <R1tcr3GxrjBawtiNBBZ1j-SqkMsJ3T8LtHz7gwMgq0E=.941815e1-78fd-4321-989a-9820916fac23@github.com>
 <PhMlb3M4mxcqLGkTRaiOWinl5n7oaC4R7wJ524vzay8=.7a177f95-4ad8-444a-9479-89e1693705aa@github.com>
 <mDVzDaVVmaZscMoaRxq2bi2yYhmq1xh6XhLM0JyaYQA=.bbf9729b-da7b-47ae-8a76-12ed2276d1ef@github.com>
 <8hC24PjPXKZVl-MqhG7qxZSc7p2V-7pVHr6LBKgjXMw=.261b9d75-29b2-48aa-b082-1037920b99e7@github.com>
 <jmk4PiJ0KxtprtlbNr0lc99tPCfOgPvg0oHQKskouik=.a2b322bb-b11a-412f-9cba-cda198769c17@github.com>
Message-ID: <ZbEaJnVuCMU8yi836eRiXiTbec4Q2g37CPYKjp5bLRg=.1bd105a1-56a7-4260-8e67-9aa4f6ebc7b0@github.com>

On Tue, 4 Mar 2025 14:51:19 GMT, Maurizio Cimadamore <mcimadamore at openjdk.org> wrote:

>> I don't see a problem here. Users can also close a stream while reading it.
>
>> I don't see a problem here. Users can also close a stream while reading it.
> 
> Yes, and that would cause an exception -- not a VM crash.

Following exception is thrown when you run a session constructed with already closed arena: 

java.lang.AssertionError: should not reach here
	at oracle.code.onnx.foreign.OrtApi$SessionGetInputCount.invoke(OrtApi.java:1891)


However yes, VM crashes are more frequent when playing with Onnx.

-------------

PR Review Comment: https://git.openjdk.org/babylon/pull/337#discussion_r1979652139

From mcimadamore at openjdk.org  Tue Mar  4 15:42:17 2025
From: mcimadamore at openjdk.org (Maurizio Cimadamore)
Date: Tue, 4 Mar 2025 15:42:17 GMT
Subject: [code-reflection] RFR: Explicit Arena passing to Tensor
 construction and Session execution [v2]
In-Reply-To: <ZbEaJnVuCMU8yi836eRiXiTbec4Q2g37CPYKjp5bLRg=.1bd105a1-56a7-4260-8e67-9aa4f6ebc7b0@github.com>
References: <cDmBlH4c7cgE7faVIwtu_jlQPNe2iLd02rsV8ZcKfH8=.114e41bd-59a5-4078-8f29-bd7f7996e496@github.com>
 <y1PSEiG46wTAVeRQ88WVtYzr4986Xi25gfzShIdZdtw=.5e6a3fe2-8d6d-456d-9c1f-725b2e519dff@github.com>
 <X47Lt776YTM53NvJUUcJ1bu1ydAR1o628RsfISbnuc8=.ba0b4805-befa-4194-bd7e-fd110173c488@github.com>
 <R1tcr3GxrjBawtiNBBZ1j-SqkMsJ3T8LtHz7gwMgq0E=.941815e1-78fd-4321-989a-9820916fac23@github.com>
 <PhMlb3M4mxcqLGkTRaiOWinl5n7oaC4R7wJ524vzay8=.7a177f95-4ad8-444a-9479-89e1693705aa@github.com>
 <mDVzDaVVmaZscMoaRxq2bi2yYhmq1xh6XhLM0JyaYQA=.bbf9729b-da7b-47ae-8a76-12ed2276d1ef@github.com>
 <8hC24PjPXKZVl-MqhG7qxZSc7p2V-7pVHr6LBKgjXMw=.261b9d75-29b2-48aa-b082-1037920b99e7@github.com>
 <jmk4PiJ0KxtprtlbNr0lc99tPCfOgPvg0oHQKskouik=.a2b322bb-b11a-412f-9cba-cda198769c17@github.com>
 <ZbEaJnVuCMU8yi836eRiXiTbec4Q2g37CPYKjp5bLRg=.1bd105a1-56a7-4260-8e67-9aa4f6ebc7b0@github.com>
Message-ID: <blTOJDfVfEuOq7MRaBhwty0XG5jNnpIRxWXmmGk1NyE=.032b95a5-8441-4465-af32-7c633f07542b@github.com>

On Tue, 4 Mar 2025 15:08:07 GMT, Adam Sotona <asotona at openjdk.org> wrote:

>>> I don't see a problem here. Users can also close a stream while reading it.
>> 
>> Yes, and that would cause an exception -- not a VM crash.
>
> Following exception is thrown when you run a session constructed with already closed arena: 
> 
> java.lang.AssertionError: should not reach here
> 	at oracle.code.onnx.foreign.OrtApi$SessionGetInputCount.invoke(OrtApi.java:1891)
> ...
> Caused by: java.lang.IllegalStateException: Already closed
> 	at java.base/jdk.internal.foreign.MemorySessionImpl.alreadyClosed(MemorySessionImpl.java:305)
> 
> 
> However yes, VM crashes are more frequent when playing with Onnx.

I think what saves us from crashes in the current code is the the fact that `sessionAddress` needs to be passed to the various onnx API calls. And, since `sessionAddress` points at the session arena, the `Linker` will not allow that arena to be closed until the downcall has finished running (this is part of the contract specified in tthe Linker API). Which probably avoids the worst cases.

As long as the lifetimes of the results of `run` are completely independent from that of the session (in the sense that it is ok for such results to "outlive" the session), then it should be ok -- but I couldn't find any guidance in the onnx API docs. The risk is that some of those results holds off to some memory of the session, and when the session is released, the result will point to already freed memory -- but I don't know if this could happen.

-------------

PR Review Comment: https://git.openjdk.org/babylon/pull/337#discussion_r1979708930

From asotona at openjdk.org  Tue Mar  4 18:29:17 2025
From: asotona at openjdk.org (Adam Sotona)
Date: Tue, 4 Mar 2025 18:29:17 GMT
Subject: [code-reflection] RFR: Explicit Arena passing to Tensor
 construction and Session execution [v2]
In-Reply-To: <blTOJDfVfEuOq7MRaBhwty0XG5jNnpIRxWXmmGk1NyE=.032b95a5-8441-4465-af32-7c633f07542b@github.com>
References: <cDmBlH4c7cgE7faVIwtu_jlQPNe2iLd02rsV8ZcKfH8=.114e41bd-59a5-4078-8f29-bd7f7996e496@github.com>
 <y1PSEiG46wTAVeRQ88WVtYzr4986Xi25gfzShIdZdtw=.5e6a3fe2-8d6d-456d-9c1f-725b2e519dff@github.com>
 <X47Lt776YTM53NvJUUcJ1bu1ydAR1o628RsfISbnuc8=.ba0b4805-befa-4194-bd7e-fd110173c488@github.com>
 <R1tcr3GxrjBawtiNBBZ1j-SqkMsJ3T8LtHz7gwMgq0E=.941815e1-78fd-4321-989a-9820916fac23@github.com>
 <PhMlb3M4mxcqLGkTRaiOWinl5n7oaC4R7wJ524vzay8=.7a177f95-4ad8-444a-9479-89e1693705aa@github.com>
 <mDVzDaVVmaZscMoaRxq2bi2yYhmq1xh6XhLM0JyaYQA=.bbf9729b-da7b-47ae-8a76-12ed2276d1ef@github.com>
 <8hC24PjPXKZVl-MqhG7qxZSc7p2V-7pVHr6LBKgjXMw=.261b9d75-29b2-48aa-b082-1037920b99e7@github.com>
 <jmk4PiJ0KxtprtlbNr0lc99tPCfOgPvg0oHQKskouik=.a2b322bb-b11a-412f-9cba-cda198769c17@github.com>
 <ZbEaJnVuCMU8yi836eRiXiTbec4Q2g37CPYKjp5bLRg=.1bd105a1-56a7-4260-8e67-9aa4f6ebc7b0@github.com>
 <blTOJDfVfEuOq7MRaBhwty0XG5jNnpIRxWXmmGk1NyE=.032b95a5-8441-4465-af32-7c633f07542b@github.com>
Message-ID: <uzFae1DXz6l02Pv-OhwgVzZwYeoC87PiDpAlVEzhVnA=.e9f1e5e5-8178-4892-912d-ed489202412a@github.com>

On Tue, 4 Mar 2025 15:38:18 GMT, Maurizio Cimadamore <mcimadamore at openjdk.org> wrote:

>> Following exception is thrown when you run a session constructed with already closed arena: 
>> 
>> java.lang.AssertionError: should not reach here
>> 	at oracle.code.onnx.foreign.OrtApi$SessionGetInputCount.invoke(OrtApi.java:1891)
>> ...
>> Caused by: java.lang.IllegalStateException: Already closed
>> 	at java.base/jdk.internal.foreign.MemorySessionImpl.alreadyClosed(MemorySessionImpl.java:305)
>> 
>> 
>> However yes, VM crashes are more frequent when playing with Onnx.
>
> I think what saves us from crashes in the current code is the the fact that `sessionAddress` needs to be passed to the various onnx API calls. And, since `sessionAddress` points at the session arena, the `Linker` will not allow that arena to be closed until the downcall has finished running (this is part of the contract specified in tthe Linker API). Which probably avoids the worst cases.
> 
> As long as the lifetimes of the results of `run` are completely independent from that of the session (in the sense that it is ok for such results to "outlive" the session), then it should be ok -- but I couldn't find any guidance in the onnx API docs. The risk is that some of those results holds off to some memory of the session, and when the session is released, the result will point to already freed memory -- but I don't know if this could happen.

It can happen for tensors and their data. I found no better prevention than use individual auto arena for each tensor, which is obviously a performance anti pattern.

-------------

PR Review Comment: https://git.openjdk.org/babylon/pull/337#discussion_r1979992361

From asotona at openjdk.org  Tue Mar  4 18:33:25 2025
From: asotona at openjdk.org (Adam Sotona)
Date: Tue, 4 Mar 2025 18:33:25 GMT
Subject: [code-reflection] RFR: Explicit Arena passing to Tensor
 construction and Session execution [v2]
In-Reply-To: <uzFae1DXz6l02Pv-OhwgVzZwYeoC87PiDpAlVEzhVnA=.e9f1e5e5-8178-4892-912d-ed489202412a@github.com>
References: <cDmBlH4c7cgE7faVIwtu_jlQPNe2iLd02rsV8ZcKfH8=.114e41bd-59a5-4078-8f29-bd7f7996e496@github.com>
 <y1PSEiG46wTAVeRQ88WVtYzr4986Xi25gfzShIdZdtw=.5e6a3fe2-8d6d-456d-9c1f-725b2e519dff@github.com>
 <X47Lt776YTM53NvJUUcJ1bu1ydAR1o628RsfISbnuc8=.ba0b4805-befa-4194-bd7e-fd110173c488@github.com>
 <R1tcr3GxrjBawtiNBBZ1j-SqkMsJ3T8LtHz7gwMgq0E=.941815e1-78fd-4321-989a-9820916fac23@github.com>
 <PhMlb3M4mxcqLGkTRaiOWinl5n7oaC4R7wJ524vzay8=.7a177f95-4ad8-444a-9479-89e1693705aa@github.com>
 <mDVzDaVVmaZscMoaRxq2bi2yYhmq1xh6XhLM0JyaYQA=.bbf9729b-da7b-47ae-8a76-12ed2276d1ef@github.com>
 <8hC24PjPXKZVl-MqhG7qxZSc7p2V-7pVHr6LBKgjXMw=.261b9d75-29b2-48aa-b082-1037920b99e7@github.com>
 <jmk4PiJ0KxtprtlbNr0lc99tPCfOgPvg0oHQKskouik=.a2b322bb-b11a-412f-9cba-cda198769c17@github.com>
 <ZbEaJnVuCMU8yi836eRiXiTbec4Q2g37CPYKjp5bLRg=.1bd105a1-56a7-4260-8e67-9aa4f6ebc7b0@github.com>
 <blTOJDfVfEuOq7MRaBhwty0XG5jNnpIRxWXmmGk1NyE=.032b95a5-8441-4465-af32-7c633f07542b@github.com>
 <uzFae1DXz6l02Pv-OhwgVzZwYeoC87PiDpA
 lVEzhVnA=.e9f1e5e5-8178-4892-912d-ed489202412a@github.com>
Message-ID: <0QokKYvip6ocYBplTzzGdTRNpmr6ADxVmipdBqa_Rus=.88d36eee-c7c9-4ac9-b31d-69d6873b777c@github.com>

On Tue, 4 Mar 2025 18:26:19 GMT, Adam Sotona <asotona at openjdk.org> wrote:

>> I think what saves us from crashes in the current code is the the fact that `sessionAddress` needs to be passed to the various onnx API calls. And, since `sessionAddress` points at the session arena, the `Linker` will not allow that arena to be closed until the downcall has finished running (this is part of the contract specified in tthe Linker API). Which probably avoids the worst cases.
>> 
>> As long as the lifetimes of the results of `run` are completely independent from that of the session (in the sense that it is ok for such results to "outlive" the session), then it should be ok -- but I couldn't find any guidance in the onnx API docs. The risk is that some of those results holds off to some memory of the session, and when the session is released, the result will point to already freed memory -- but I don't know if this could happen.
>
> It can happen for tensors and their data. I found no better prevention than use individual auto arena for each tensor, which is obviously a performance anti pattern.

Is there a way to allocate an individually GCed memory segment without a cost of creating the whole auto arena (and similarly an individually freed MS without a new confined arena cost)?

-------------

PR Review Comment: https://git.openjdk.org/babylon/pull/337#discussion_r1979997987

From mcimadamore at openjdk.org  Tue Mar  4 19:13:15 2025
From: mcimadamore at openjdk.org (Maurizio Cimadamore)
Date: Tue, 4 Mar 2025 19:13:15 GMT
Subject: [code-reflection] RFR: Explicit Arena passing to Tensor
 construction and Session execution [v2]
In-Reply-To: <0QokKYvip6ocYBplTzzGdTRNpmr6ADxVmipdBqa_Rus=.88d36eee-c7c9-4ac9-b31d-69d6873b777c@github.com>
References: <cDmBlH4c7cgE7faVIwtu_jlQPNe2iLd02rsV8ZcKfH8=.114e41bd-59a5-4078-8f29-bd7f7996e496@github.com>
 <y1PSEiG46wTAVeRQ88WVtYzr4986Xi25gfzShIdZdtw=.5e6a3fe2-8d6d-456d-9c1f-725b2e519dff@github.com>
 <X47Lt776YTM53NvJUUcJ1bu1ydAR1o628RsfISbnuc8=.ba0b4805-befa-4194-bd7e-fd110173c488@github.com>
 <R1tcr3GxrjBawtiNBBZ1j-SqkMsJ3T8LtHz7gwMgq0E=.941815e1-78fd-4321-989a-9820916fac23@github.com>
 <PhMlb3M4mxcqLGkTRaiOWinl5n7oaC4R7wJ524vzay8=.7a177f95-4ad8-444a-9479-89e1693705aa@github.com>
 <mDVzDaVVmaZscMoaRxq2bi2yYhmq1xh6XhLM0JyaYQA=.bbf9729b-da7b-47ae-8a76-12ed2276d1ef@github.com>
 <8hC24PjPXKZVl-MqhG7qxZSc7p2V-7pVHr6LBKgjXMw=.261b9d75-29b2-48aa-b082-1037920b99e7@github.com>
 <jmk4PiJ0KxtprtlbNr0lc99tPCfOgPvg0oHQKskouik=.a2b322bb-b11a-412f-9cba-cda198769c17@github.com>
 <ZbEaJnVuCMU8yi836eRiXiTbec4Q2g37CPYKjp5bLRg=.1bd105a1-56a7-4260-8e67-9aa4f6ebc7b0@github.com>
 <blTOJDfVfEuOq7MRaBhwty0XG5jNnpIRxWXmmGk1NyE=.032b95a5-8441-4465-af32-7c633f07542b@github.com>
 <uzFae1DXz6l02Pv-OhwgVzZwYeoC87PiDpA
 lVEzhVnA=.e9f1e5e5-8178-4892-912d-ed489202412a@github.com>
 <0QokKYvip6ocYBplTzzGdTRNpmr6ADxVmipdBqa_Rus=.88d36eee-c7c9-4ac9-b31d-69d6873b777c@github.com>
Message-ID: <zew8_YEwFWIE-kO8p3Hl_0t8Im30w_oO6AS27-0s3eo=.8c9f14a7-e26d-41b3-b2ff-f31cf0cb2ad6@github.com>

On Tue, 4 Mar 2025 18:30:36 GMT, Adam Sotona <asotona at openjdk.org> wrote:

> Is there a way to allocate an individually GCed memory segment without a cost of creating the whole auto arena (and similarly an individually freed MS without a new confined arena cost)?

no

-------------

PR Review Comment: https://git.openjdk.org/babylon/pull/337#discussion_r1980079044

From asotona at openjdk.org  Wed Mar  5 08:20:06 2025
From: asotona at openjdk.org (Adam Sotona)
Date: Wed, 5 Mar 2025 08:20:06 GMT
Subject: [code-reflection] RFR: Explicit Arena passing to Tensor
 construction and Session execution [v2]
In-Reply-To: <zew8_YEwFWIE-kO8p3Hl_0t8Im30w_oO6AS27-0s3eo=.8c9f14a7-e26d-41b3-b2ff-f31cf0cb2ad6@github.com>
References: <cDmBlH4c7cgE7faVIwtu_jlQPNe2iLd02rsV8ZcKfH8=.114e41bd-59a5-4078-8f29-bd7f7996e496@github.com>
 <y1PSEiG46wTAVeRQ88WVtYzr4986Xi25gfzShIdZdtw=.5e6a3fe2-8d6d-456d-9c1f-725b2e519dff@github.com>
 <X47Lt776YTM53NvJUUcJ1bu1ydAR1o628RsfISbnuc8=.ba0b4805-befa-4194-bd7e-fd110173c488@github.com>
 <R1tcr3GxrjBawtiNBBZ1j-SqkMsJ3T8LtHz7gwMgq0E=.941815e1-78fd-4321-989a-9820916fac23@github.com>
 <PhMlb3M4mxcqLGkTRaiOWinl5n7oaC4R7wJ524vzay8=.7a177f95-4ad8-444a-9479-89e1693705aa@github.com>
 <mDVzDaVVmaZscMoaRxq2bi2yYhmq1xh6XhLM0JyaYQA=.bbf9729b-da7b-47ae-8a76-12ed2276d1ef@github.com>
 <8hC24PjPXKZVl-MqhG7qxZSc7p2V-7pVHr6LBKgjXMw=.261b9d75-29b2-48aa-b082-1037920b99e7@github.com>
 <jmk4PiJ0KxtprtlbNr0lc99tPCfOgPvg0oHQKskouik=.a2b322bb-b11a-412f-9cba-cda198769c17@github.com>
 <ZbEaJnVuCMU8yi836eRiXiTbec4Q2g37CPYKjp5bLRg=.1bd105a1-56a7-4260-8e67-9aa4f6ebc7b0@github.com>
 <blTOJDfVfEuOq7MRaBhwty0XG5jNnpIRxWXmmGk1NyE=.032b95a5-8441-4465-af32-7c633f07542b@github.com>
 <uzFae1DXz6l02Pv-OhwgVzZwYeoC87PiDpA
 lVEzhVnA=.e9f1e5e5-8178-4892-912d-ed489202412a@github.com>
 <0QokKYvip6ocYBplTzzGdTRNpmr6ADxVmipdBqa_Rus=.88d36eee-c7c9-4ac9-b31d-69d6873b777c@github.com>
 <zew8_YEwFWIE-kO8p3Hl_0t8Im30w_oO6AS27-0s3eo=.8c9f14a7-e26d-41b3-b2ff-f31cf0cb2ad6@github.com>
Message-ID: <HszYfUG92IRO0YxyKAa5Le5F7z9D1gWqUbXfH2rTXHg=.64b9b666-4b3c-485b-a105-fcfdd1619c25@github.com>

On Tue, 4 Mar 2025 19:09:58 GMT, Maurizio Cimadamore <mcimadamore at openjdk.org> wrote:

>> Is there a way to allocate an individually GCed memory segment without a cost of creating the whole auto arena (and similarly an individually freed MS without a new confined arena cost)?
>
>> Is there a way to allocate an individually GCed memory segment without a cost of creating the whole auto arena (and similarly an individually freed MS without a new confined arena cost)?
> 
> no

In such case there are only two options:
1. Always create a new auto arena so each tensor, session and session result has its own life cycle.
2. Let user decide about the life cycle by passing arena to each tensor and session construction and session run.

This PR enables #2, with default to #1 for tensors construction.

-------------

PR Review Comment: https://git.openjdk.org/babylon/pull/337#discussion_r1980904454

From asotona at openjdk.org  Wed Mar  5 09:01:39 2025
From: asotona at openjdk.org (Adam Sotona)
Date: Wed, 5 Mar 2025 09:01:39 GMT
Subject: git: openjdk/babylon: code-reflection: Explicit Arena passing to
 Tensor construction and Session execution
Message-ID: <bf016a70-4031-4ce5-bdff-87833a26d5bd@openjdk.org>

Changeset: 72a9d699
Branch: code-reflection
Author:    Adam Sotona <asotona at openjdk.org>
Date:      2025-03-05 08:59:33 +0000
URL:       https://git.openjdk.org/babylon/commit/72a9d699761e524a2bf83551612b6d5e7ed6edbe

Explicit Arena passing to Tensor construction and Session execution

! cr-examples/onnx/src/main/java/oracle/code/onnx/OnnxInterpreter.java
! cr-examples/onnx/src/main/java/oracle/code/onnx/OnnxRuntime.java
! cr-examples/onnx/src/main/java/oracle/code/onnx/Tensor.java
! cr-examples/onnx/src/test/java/oracle/code/onnx/CNNTest.java
! cr-examples/onnx/src/test/java/oracle/code/onnx/RuntimeTest.java
! cr-examples/onnx/src/test/java/oracle/code/onnx/mnist/MNISTDemo.java


From asotona at openjdk.org  Wed Mar  5 09:02:08 2025
From: asotona at openjdk.org (Adam Sotona)
Date: Wed, 5 Mar 2025 09:02:08 GMT
Subject: [code-reflection] RFR: Explicit Arena passing to Tensor
 construction and Session execution [v7]
In-Reply-To: <M90seTHqxu6PTjTFtZ9D35J_qfCOCn7pgLBvyBymHYs=.d3e4b270-e93e-4e0d-a38b-0982e0c0d907@github.com>
References: <cDmBlH4c7cgE7faVIwtu_jlQPNe2iLd02rsV8ZcKfH8=.114e41bd-59a5-4078-8f29-bd7f7996e496@github.com>
 <M90seTHqxu6PTjTFtZ9D35J_qfCOCn7pgLBvyBymHYs=.d3e4b270-e93e-4e0d-a38b-0982e0c0d907@github.com>
Message-ID: <W2aRh_uqK0lrBVfQ3WOjaCJxsEayzUlf4fvu_VBR-ZU=.fc6f8604-d675-40d9-bbfd-0b1190477a05@github.com>

On Tue, 4 Mar 2025 13:46:35 GMT, Adam Sotona <asotona at openjdk.org> wrote:

>> Holding `Arena` instances in `OnnxRuntime` and `Session` still cause memory leaks.
>> This patch allows to pass explicit arena for `Tensor` construction, `Session` construction and  each `Session` run.
>> Improved handling of `Session` life cycle allows to cache live `Session` instead of its protobuf model.
>
> Adam Sotona has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Caching live sessions insted of proto models

Thanks for all the comments, let's move forward and continue in the follow-up discussions.

-------------

PR Comment: https://git.openjdk.org/babylon/pull/337#issuecomment-2700274743

From asotona at openjdk.org  Wed Mar  5 09:02:09 2025
From: asotona at openjdk.org (Adam Sotona)
Date: Wed, 5 Mar 2025 09:02:09 GMT
Subject: [code-reflection] Integrated: Explicit Arena passing to Tensor
 construction and Session execution
In-Reply-To: <cDmBlH4c7cgE7faVIwtu_jlQPNe2iLd02rsV8ZcKfH8=.114e41bd-59a5-4078-8f29-bd7f7996e496@github.com>
References: <cDmBlH4c7cgE7faVIwtu_jlQPNe2iLd02rsV8ZcKfH8=.114e41bd-59a5-4078-8f29-bd7f7996e496@github.com>
Message-ID: <Z63uhguTIp8H1VVv1Ts2r3-a6bRni-q3FzQgonf0aZU=.0591532e-7698-4e02-9203-11a0dd7c5261@github.com>

On Tue, 4 Mar 2025 12:25:47 GMT, Adam Sotona <asotona at openjdk.org> wrote:

> Holding `Arena` instances in `OnnxRuntime` and `Session` still cause memory leaks.
> This patch allows to pass explicit arena for `Tensor` construction, `Session` construction and  each `Session` run.
> Improved handling of `Session` life cycle allows to cache live `Session` instead of its protobuf model.

This pull request has now been integrated.

Changeset: 72a9d699
Author:    Adam Sotona <asotona at openjdk.org>
URL:       https://git.openjdk.org/babylon/commit/72a9d699761e524a2bf83551612b6d5e7ed6edbe
Stats:     211 lines in 6 files changed: 58 ins; 20 del; 133 mod

Explicit Arena passing to Tensor construction and Session execution

-------------

PR: https://git.openjdk.org/babylon/pull/337

From mcimadamore at openjdk.org  Wed Mar  5 11:08:06 2025
From: mcimadamore at openjdk.org (Maurizio Cimadamore)
Date: Wed, 5 Mar 2025 11:08:06 GMT
Subject: [code-reflection] RFR: Explicit Arena passing to Tensor
 construction and Session execution [v2]
In-Reply-To: <HszYfUG92IRO0YxyKAa5Le5F7z9D1gWqUbXfH2rTXHg=.64b9b666-4b3c-485b-a105-fcfdd1619c25@github.com>
References: <cDmBlH4c7cgE7faVIwtu_jlQPNe2iLd02rsV8ZcKfH8=.114e41bd-59a5-4078-8f29-bd7f7996e496@github.com>
 <y1PSEiG46wTAVeRQ88WVtYzr4986Xi25gfzShIdZdtw=.5e6a3fe2-8d6d-456d-9c1f-725b2e519dff@github.com>
 <X47Lt776YTM53NvJUUcJ1bu1ydAR1o628RsfISbnuc8=.ba0b4805-befa-4194-bd7e-fd110173c488@github.com>
 <R1tcr3GxrjBawtiNBBZ1j-SqkMsJ3T8LtHz7gwMgq0E=.941815e1-78fd-4321-989a-9820916fac23@github.com>
 <PhMlb3M4mxcqLGkTRaiOWinl5n7oaC4R7wJ524vzay8=.7a177f95-4ad8-444a-9479-89e1693705aa@github.com>
 <mDVzDaVVmaZscMoaRxq2bi2yYhmq1xh6XhLM0JyaYQA=.bbf9729b-da7b-47ae-8a76-12ed2276d1ef@github.com>
 <8hC24PjPXKZVl-MqhG7qxZSc7p2V-7pVHr6LBKgjXMw=.261b9d75-29b2-48aa-b082-1037920b99e7@github.com>
 <jmk4PiJ0KxtprtlbNr0lc99tPCfOgPvg0oHQKskouik=.a2b322bb-b11a-412f-9cba-cda198769c17@github.com>
 <ZbEaJnVuCMU8yi836eRiXiTbec4Q2g37CPYKjp5bLRg=.1bd105a1-56a7-4260-8e67-9aa4f6ebc7b0@github.com>
 <blTOJDfVfEuOq7MRaBhwty0XG5jNnpIRxWXmmGk1NyE=.032b95a5-8441-4465-af32-7c633f07542b@github.com>
 <uzFae1DXz6l02Pv-OhwgVzZwYeoC87PiDpA
 lVEzhVnA=.e9f1e5e5-8178-4892-912d-ed489202412a@github.com>
 <0QokKYvip6ocYBplTzzGdTRNpmr6ADxVmipdBqa_Rus=.88d36eee-c7c9-4ac9-b31d-69d6873b777c@github.com>
 <zew8_YEwFWIE-kO8p3Hl_0t8Im30w_oO6AS27-0s3eo=.8c9f14a7-e26d-41b3-b2ff-f31cf0cb2ad6@github.com>
 <HszYfUG92IRO0YxyKAa5Le5F7z9D1gWqUbXfH2rTXHg=.64b9b666-4b3c-485b-a105-fcfdd1619c25@github.com>
Message-ID: <B-8nAu4CPepqUhrAKKC4el_579pqgs6_ZVOcMKxzuDA=.b459f436-1572-4f29-ba5b-40af919c98be@github.com>

On Wed, 5 Mar 2025 08:16:57 GMT, Adam Sotona <asotona at openjdk.org> wrote:

>>> Is there a way to allocate an individually GCed memory segment without a cost of creating the whole auto arena (and similarly an individually freed MS without a new confined arena cost)?
>> 
>> no
>
> In such case there are only two options:
> 1. Always create a new auto arena so each tensor, session and session result has its own life cycle.
> 2. Let user decide about the life cycle by passing arena to each tensor and session construction and session run.
> 
> This PR enables #2, with default to #1 for tensors construction.

Even assuming we had some way to allocate an automatic segment w/o an arena, you would still be creating a segment backed by a new GC-managed lifetime. The arena creation only makes it explicit when you are creating such a lifetime.

For automatic segments, we could even provide such an arena-free factory -- we considered it in the past, but pruned it as it didn't really add much (creating an automatic segment is still an expensive operation, as objects have to be registered against a `Cleaner` object -- this is typically a problem for `ByteBuffer`s).

For non-automatic arenas this is not an option, for several reasons:
* an arena is a *capability* -- only a client with an arena can deallocate memory. If you have a `free` segment on a memory segment, suddenly you realize that you can no longer pass a segment you created safely to an API -- you always have to defend against that API potentially closing that segment, by calling `free`. We wrestled with this problem for a long time, by allowing defensive, non-closeable views of segments -- it really didn't look like a great world to live in.
* There is a natural "one to many" relationship between lifetime and segments. It is very common to have many segments share the same lifetime. This is is a good way to discipline the use of off-heap memory -- as any time you have segments with different lifetimes interacting with each other, you can end up with a potential use-after-free issue. This is covered in more details [here](https://cr.openjdk.org/~mcimadamore/panama/why_lifetimes.html)
* The performance model of shared arenas, naturally leans towards a model where you group segments together. That is, allowing safe memory release of individual memory segments would be just too expensive to support.
* It is always possible to "drop down a level" and have something more similar to `malloc`/`free` -- by using `Linker` + `reinterpret` -- an example can be found [here](https://github.com/openjdk/jdk/pull/19308/files#diff-e1f7dec1e858c9df738ebebd894dbd10a2885e81264ece400db3620a56f33c3eR58)

> In such case there are only two options:

I believe the key question here is which lifetimes we want developers to control. The more I look at this, the more it seems to me that when creating envs, sessions or global tensors you don't really care about "deterministic deallocation", so using an automatic lifetime is a good choice.

But, when you run a model, you likely need a lot of transient resources, and it would be nice to be able to control the lifetime of those resources. 

This leads, I think, to a model/API where e.g. a session can be created w/o an arena (an automatic arena will be picked for you). The session will let you allocate session-wide tensors -- which will stay alive as long as the session is alive.

In order to run a model, you have to get some kind of `AutoCloseable` object from the session. This object would be backed by a confined arena, and would also act as a tensor allocator -- but this time, the lifetime of these tensors would be tied to that of the confined arena (in a way, the lifetime of this `AutoCloseable` is "nested" in the lifetime of the session). Example usage:


OnnxSession session = OnnxSession.of();
Tensor<Float> session_tensor = session.allocateTensor(...);
...
try (RunContext runContext = session.newRunContext()) {
    Tensor<Float> run_tensor = runContext.allocateTensor(...);
    Tensor<Float> 'result' = runContext.run(... session_tensor ... run_tensor ... );
    // can use 'result' here
} // both run_tensor and result are deallocated here


(the names are not great and are for illustrative purposes only -- it is possible that instead of exposing `RunContext` as an `AutoCloseable` it might make sense to expose it as a `run` function taking a lambda expression on `OnnxSession` -- but the essence is the same).

IMHO the above strikes a nice balance between deterministic and non-deterministic deallocation, and the resulting API doesn't seem too complex to use (no arenas need to be created or be passed anywhere). It is of course only one of the many possible ways to model some of the capabilities of ONNX with an higher-level Java API -- there are probably other (and better) options.

-------------

PR Review Comment: https://git.openjdk.org/babylon/pull/337#discussion_r1981179834

From mabbay at openjdk.org  Thu Mar  6 03:31:55 2025
From: mabbay at openjdk.org (Mourad Abbay)
Date: Thu, 6 Mar 2025 03:31:55 GMT
Subject: [code-reflection] RFR: Support storing the code that builds the
 code model [v16]
In-Reply-To: <dexv8sQV_NcJsJIgMopd_M4wSWoZya9dU0ACwDtEL0E=.4ac0b80f-24fb-440f-a18d-bbfd07a681bc@github.com>
References: <dexv8sQV_NcJsJIgMopd_M4wSWoZya9dU0ACwDtEL0E=.4ac0b80f-24fb-440f-a18d-bbfd07a681bc@github.com>
Message-ID: <5m0eK2G7oC2pkyRcyNfgMmNaPbCYftd4x1QbXZbY8FI=.726954fc-129d-494e-b70e-6cedcc8322eb@github.com>

> In this PR we allow the user to decide how to store the code model. The first option is `TEXT`, if this is selected, we store the textual representation of the code model. The second option is `CODE_BUILDR`, if this is selected, we store the code that builds the code model. All work done here is around the second option, because the first option is already supported.

Mourad Abbay has updated the pull request incrementally with one additional commit since the last revision:

  Fix the remaining compiler tests failures

-------------

Changes:
  - all: https://git.openjdk.org/babylon/pull/305/files
  - new: https://git.openjdk.org/babylon/pull/305/files/cd143c37..ee0f906e

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=babylon&pr=305&range=15
 - incr: https://webrevs.openjdk.org/?repo=babylon&pr=305&range=14-15

  Stats: 21 lines in 3 files changed: 14 ins; 2 del; 5 mod
  Patch: https://git.openjdk.org/babylon/pull/305.diff
  Fetch: git fetch https://git.openjdk.org/babylon.git pull/305/head:pull/305

PR: https://git.openjdk.org/babylon/pull/305

From mabbay at openjdk.org  Thu Mar  6 15:49:32 2025
From: mabbay at openjdk.org (Mourad Abbay)
Date: Thu, 6 Mar 2025 15:49:32 GMT
Subject: [code-reflection] RFR: Support storing the code that builds the
 code model [v17]
In-Reply-To: <dexv8sQV_NcJsJIgMopd_M4wSWoZya9dU0ACwDtEL0E=.4ac0b80f-24fb-440f-a18d-bbfd07a681bc@github.com>
References: <dexv8sQV_NcJsJIgMopd_M4wSWoZya9dU0ACwDtEL0E=.4ac0b80f-24fb-440f-a18d-bbfd07a681bc@github.com>
Message-ID: <zKGCPAn3OsUu7bDP2cRAvYFeH4AMT1QPTyFssveYE7A=.956f33f9-80d7-4ab1-ad2b-d952a5f94bca@github.com>

> In this PR we allow the user to decide how to store the code model. The first option is `TEXT`, if this is selected, we store the textual representation of the code model. The second option is `CODE_BUILDR`, if this is selected, we store the code that builds the code model. All work done here is around the second option, because the first option is already supported.

Mourad Abbay has updated the pull request incrementally with one additional commit since the last revision:

  Load OpFactory and TypeElementFactory before invocation of opMethod only if the opMethod has there params.

-------------

Changes:
  - all: https://git.openjdk.org/babylon/pull/305/files
  - new: https://git.openjdk.org/babylon/pull/305/files/ee0f906e..0e49cceb

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=babylon&pr=305&range=16
 - incr: https://webrevs.openjdk.org/?repo=babylon&pr=305&range=15-16

  Stats: 9 lines in 1 file changed: 3 ins; 0 del; 6 mod
  Patch: https://git.openjdk.org/babylon/pull/305.diff
  Fetch: git fetch https://git.openjdk.org/babylon.git pull/305/head:pull/305

PR: https://git.openjdk.org/babylon/pull/305

From asotona at openjdk.org  Thu Mar  6 22:16:07 2025
From: asotona at openjdk.org (Adam Sotona)
Date: Thu, 6 Mar 2025 22:16:07 GMT
Subject: [code-reflection] RFR: Initializers prototype implementation
Message-ID: <PbOTudNh1S71G01Fy430DZRhha5EWUKSNkrFQOyxvRk=.f177bd0f-8883-47ac-b7f5-6dec86e5484c@github.com>

Initializers prototype implementation

-------------

Commit messages:
 - Initializers implementation

Changes: https://git.openjdk.org/babylon/pull/339/files
  Webrev: https://webrevs.openjdk.org/?repo=babylon&pr=339&range=00
  Stats: 141 lines in 6 files changed: 110 ins; 10 del; 21 mod
  Patch: https://git.openjdk.org/babylon/pull/339.diff
  Fetch: git fetch https://git.openjdk.org/babylon.git pull/339/head:pull/339

PR: https://git.openjdk.org/babylon/pull/339

From asotona at openjdk.org  Thu Mar  6 23:40:49 2025
From: asotona at openjdk.org (Adam Sotona)
Date: Thu, 6 Mar 2025 23:40:49 GMT
Subject: [code-reflection] RFR: Initializers prototype implementation [v2]
In-Reply-To: <PbOTudNh1S71G01Fy430DZRhha5EWUKSNkrFQOyxvRk=.f177bd0f-8883-47ac-b7f5-6dec86e5484c@github.com>
References: <PbOTudNh1S71G01Fy430DZRhha5EWUKSNkrFQOyxvRk=.f177bd0f-8883-47ac-b7f5-6dec86e5484c@github.com>
Message-ID: <l49gGsrs43IkHLWcATeH0SdSjaK18FJ0_kBYuIvJe9s=.c2997fd5-746e-4c3b-9e8b-2039fbb978fc@github.com>

> Initializers prototype implementation

Adam Sotona has updated the pull request incrementally with one additional commit since the last revision:

  static fields initializers

-------------

Changes:
  - all: https://git.openjdk.org/babylon/pull/339/files
  - new: https://git.openjdk.org/babylon/pull/339/files/4a89c557..5a31ed8c

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=babylon&pr=339&range=01
 - incr: https://webrevs.openjdk.org/?repo=babylon&pr=339&range=00-01

  Stats: 61 lines in 3 files changed: 16 ins; 28 del; 17 mod
  Patch: https://git.openjdk.org/babylon/pull/339.diff
  Fetch: git fetch https://git.openjdk.org/babylon.git pull/339/head:pull/339

PR: https://git.openjdk.org/babylon/pull/339

From asotona at openjdk.org  Fri Mar  7 00:10:08 2025
From: asotona at openjdk.org (Adam Sotona)
Date: Fri, 7 Mar 2025 00:10:08 GMT
Subject: [code-reflection] Withdrawn: Initializers prototype implementation
In-Reply-To: <PbOTudNh1S71G01Fy430DZRhha5EWUKSNkrFQOyxvRk=.f177bd0f-8883-47ac-b7f5-6dec86e5484c@github.com>
References: <PbOTudNh1S71G01Fy430DZRhha5EWUKSNkrFQOyxvRk=.f177bd0f-8883-47ac-b7f5-6dec86e5484c@github.com>
Message-ID: <GjDJNAcKvkY-2LcGECahEGi9UwWcT8JsN3d2ibb-Pl0=.c57d17fd-e438-4a72-a07a-eeec4e387518@github.com>

On Thu, 6 Mar 2025 22:11:03 GMT, Adam Sotona <asotona at openjdk.org> wrote:

> Initializers prototype implementation

This pull request has been closed without being integrated.

-------------

PR: https://git.openjdk.org/babylon/pull/339

From asotona at openjdk.org  Fri Mar  7 11:43:24 2025
From: asotona at openjdk.org (Adam Sotona)
Date: Fri, 7 Mar 2025 11:43:24 GMT
Subject: [code-reflection] RFR: Initializers through arguments
Message-ID: <Nqa8WhMnqKKw0I6sEMTp0yzm7BMXzFgb1LWYsm5kLn0=.ed95950f-8aa2-4d28-b898-2bb53247d60d@github.com>

Initializers through arguments implementation.
This PR is chained to ongoing work on `If` and `Loop` due to conflicting changes in `OnnxProtoBuilder`.

Input image reshape has to be removed from the CNN model due to unexpected exceptions in `Conv` op in relation with initializers.

-------------

Commit messages:
 - MNISTDemo using initializers
 - removed obsolete test
 - re-enable tests
 - Removed Reshape from CNNTest to avoid model Conv exception when used with initialized weights
 - initializers through arguments - work in progress
 - initializers as parameters prefix
 - removed debug print
 - improved SimpleTest
 - Fixed captured tensors in If op model
 - sealed ExplicitOnnxOps
 - ... and 16 more: https://git.openjdk.org/babylon/compare/72a9d699...5843369d

Changes: https://git.openjdk.org/babylon/pull/340/files
  Webrev: https://webrevs.openjdk.org/?repo=babylon&pr=340&range=00
  Stats: 767 lines in 12 files changed: 534 ins; 181 del; 52 mod
  Patch: https://git.openjdk.org/babylon/pull/340.diff
  Fetch: git fetch https://git.openjdk.org/babylon.git pull/340/head:pull/340

PR: https://git.openjdk.org/babylon/pull/340

From asotona at openjdk.org  Fri Mar  7 11:47:49 2025
From: asotona at openjdk.org (Adam Sotona)
Date: Fri, 7 Mar 2025 11:47:49 GMT
Subject: [code-reflection] RFR: Initializers through arguments [v2]
In-Reply-To: <Nqa8WhMnqKKw0I6sEMTp0yzm7BMXzFgb1LWYsm5kLn0=.ed95950f-8aa2-4d28-b898-2bb53247d60d@github.com>
References: <Nqa8WhMnqKKw0I6sEMTp0yzm7BMXzFgb1LWYsm5kLn0=.ed95950f-8aa2-4d28-b898-2bb53247d60d@github.com>
Message-ID: <aPc3uIBr2iszbEIJuzkw9iGxKLLPJ6tAd2RkDfrIQZM=.c716b17c-153d-40df-ad61-490c3826103c@github.com>

> Initializers through arguments implementation.
> This PR is chained to ongoing work on `If` and `Loop` due to conflicting changes in `OnnxProtoBuilder`.
> 
> Input image reshape has to be removed from the CNN model due to unexpected exceptions in `Conv` op in relation with initializers.

Adam Sotona has updated the pull request incrementally with one additional commit since the last revision:

  removed obsolete imports and comment

-------------

Changes:
  - all: https://git.openjdk.org/babylon/pull/340/files
  - new: https://git.openjdk.org/babylon/pull/340/files/5843369d..a796b230

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=babylon&pr=340&range=01
 - incr: https://webrevs.openjdk.org/?repo=babylon&pr=340&range=00-01

  Stats: 10 lines in 3 files changed: 0 ins; 10 del; 0 mod
  Patch: https://git.openjdk.org/babylon/pull/340.diff
  Fetch: git fetch https://git.openjdk.org/babylon.git pull/340/head:pull/340

PR: https://git.openjdk.org/babylon/pull/340

From gfrost at openjdk.org  Fri Mar  7 13:36:12 2025
From: gfrost at openjdk.org (Gary Frost)
Date: Fri, 7 Mar 2025 13:36:12 GMT
Subject: git: openjdk/babylon: code-reflection: Hat cleanup extracted examples
Message-ID: <204f7854-25a7-40a3-9926-7457c028ee10@openjdk.org>

Changeset: 15aa7b2d
Branch: code-reflection
Author:    Gary Frost <gfrost at openjdk.org>
Date:      2025-03-07 13:34:27 +0000
URL:       https://git.openjdk.org/babylon/commit/15aa7b2dc71b85b39cbb5a32d598a76fc63752bd

Hat cleanup extracted examples

! hat/backends/ffi/opencl/CMakeLists.txt
! hat/backends/ffi/opencl/cpp/opencl_backend.cpp
+ hat/backends/ffi/opencl/cpp/opencl_backend_queue.cpp
! hat/backends/ffi/opencl/include/opencl_backend.h
! hat/backends/ffi/shared/include/shared.h
! hat/bld
! hat/bldr/Bldr.java
! hat/env.bash
! hat/examples/experiments/src/main/java/experiments/MinBufferTest.java
! hat/examples/life/src/main/java/life/Main.java
! hat/examples/nbody/src/main/java/nbody/Main.java
+ hat/examples/nbody/src/main/java/nbody/Mode.java
! hat/examples/nbody/src/main/java/nbody/NBodyGLWindow.java
+ hat/examples/nbody/src/main/java/nbody/Universe.java
- hat/examples/nbody/src/main/java/nbody/opencl/NBody.java
+ hat/examples/nbody/src/main/java/nbody/opencl/OpenCLNBodyGLWindow.java
! hat/hat/src/main/java/hat/backend/ffi/C99FFIBackend.java
! hat/hat/src/main/java/hat/backend/jextracted/C99JExtractedBackend.java
! hat/hat/src/main/java/hat/buffer/ArgArray.java
! hat/hat/src/main/java/hat/callgraph/ComputeEntrypoint.java
! hat/hat/src/main/java/hat/callgraph/Entrypoint.java
! hat/hat/src/main/java/hat/callgraph/KernelEntrypoint.java
! hat/intellij/.idea/misc.xml
! hat/intellij/heal.iml
! hat/wrap/clwrap/src/main/java/wrap/clwrap/CLPlatform.java
! hat/wrap/wrap/src/main/java/wrap/ArenaHolder.java
! hat/wrap/wrap/src/main/java/wrap/Wrap.java


From gfrost at openjdk.org  Fri Mar  7 13:37:50 2025
From: gfrost at openjdk.org (Gary Frost)
Date: Fri, 7 Mar 2025 13:37:50 GMT
Subject: [code-reflection] Integrated: Hat cleanup extracted examples
Message-ID: <z_kle_ukiDeQyBzcOugi1WTeZbBsikMme8Ejvx-04Eg=.f88f6988-3375-4564-b343-1c7c2792f55a@github.com>

Cleaning up examples and adding profiling to OpenCL ffi backend.  So we can track buffer transfer costs
Actual implementation of code to minimize buffer transfers is next...

-------------

Commit messages:
 - whitespace and license fixes
 - let env.bash try to find jextract
 - Enqueue markers on compute start and end and on enter leave of kernel dispatch
 - Enqueue markers on compute start and end
 - Fix Bldr.java (mac osx version) and cleaned headers
 - We seem to be successfully marking buffer arg access types RO,RW and WO.
 - We seem to be successfully marking buffer arg access types RO,RW and WO.
 - Accept Short, Long and Double args
 - Nbody OpenCL mode uses HAT buffer (We need solution for OpenCL4)

Changes: https://git.openjdk.org/babylon/pull/341/files
  Webrev: https://webrevs.openjdk.org/?repo=babylon&pr=341&range=00
  Stats: 1616 lines in 27 files changed: 908 ins; 640 del; 68 mod
  Patch: https://git.openjdk.org/babylon/pull/341.diff
  Fetch: git fetch https://git.openjdk.org/babylon.git pull/341/head:pull/341

PR: https://git.openjdk.org/babylon/pull/341

From gfrost at openjdk.org  Fri Mar  7 13:37:52 2025
From: gfrost at openjdk.org (Gary Frost)
Date: Fri, 7 Mar 2025 13:37:52 GMT
Subject: [code-reflection] Integrated: Hat cleanup extracted examples
In-Reply-To: <z_kle_ukiDeQyBzcOugi1WTeZbBsikMme8Ejvx-04Eg=.f88f6988-3375-4564-b343-1c7c2792f55a@github.com>
References: <z_kle_ukiDeQyBzcOugi1WTeZbBsikMme8Ejvx-04Eg=.f88f6988-3375-4564-b343-1c7c2792f55a@github.com>
Message-ID: <VPAcjyAgOhuphivdCZeNSMNlgXFTSzvCjEspOKd3XAc=.b6dc8334-5bc1-46a2-ad83-5397b5f351aa@github.com>

On Fri, 7 Mar 2025 13:32:38 GMT, Gary Frost <gfrost at openjdk.org> wrote:

> Cleaning up examples and adding profiling to OpenCL ffi backend.  So we can track buffer transfer costs
> Actual implementation of code to minimize buffer transfers is next...

This pull request has now been integrated.

Changeset: 15aa7b2d
Author:    Gary Frost <gfrost at openjdk.org>
URL:       https://git.openjdk.org/babylon/commit/15aa7b2dc71b85b39cbb5a32d598a76fc63752bd
Stats:     1616 lines in 27 files changed: 908 ins; 640 del; 68 mod

Hat cleanup extracted examples

-------------

PR: https://git.openjdk.org/babylon/pull/341

From asotona at openjdk.org  Fri Mar  7 17:45:48 2025
From: asotona at openjdk.org (Adam Sotona)
Date: Fri, 7 Mar 2025 17:45:48 GMT
Subject: [code-reflection] RFR: Initializers through arguments [v3]
In-Reply-To: <Nqa8WhMnqKKw0I6sEMTp0yzm7BMXzFgb1LWYsm5kLn0=.ed95950f-8aa2-4d28-b898-2bb53247d60d@github.com>
References: <Nqa8WhMnqKKw0I6sEMTp0yzm7BMXzFgb1LWYsm5kLn0=.ed95950f-8aa2-4d28-b898-2bb53247d60d@github.com>
Message-ID: <iCyb_TC72AXsd6HX2upoodG8Gni5Earit1xnt9xsP38=.41525f97-6a75-4b2e-b7b3-d3ead31f36c5@github.com>

> Initializers through arguments implementation.
> This PR is chained to ongoing work on `If` and `Loop` due to conflicting changes in `OnnxProtoBuilder`.
> 
> Input image reshape has to be removed from the CNN model due to unexpected exceptions in `Conv` op in relation with initializers.

Adam Sotona has updated the pull request incrementally with one additional commit since the last revision:

  minor demo rearrangement

-------------

Changes:
  - all: https://git.openjdk.org/babylon/pull/340/files
  - new: https://git.openjdk.org/babylon/pull/340/files/a796b230..dbf678a3

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=babylon&pr=340&range=02
 - incr: https://webrevs.openjdk.org/?repo=babylon&pr=340&range=01-02

  Stats: 78 lines in 2 files changed: 51 ins; 26 del; 1 mod
  Patch: https://git.openjdk.org/babylon/pull/340.diff
  Fetch: git fetch https://git.openjdk.org/babylon.git pull/340/head:pull/340

PR: https://git.openjdk.org/babylon/pull/340

From gfrost at openjdk.org  Sun Mar  9 16:24:22 2025
From: gfrost at openjdk.org (Gary Frost)
Date: Sun, 9 Mar 2025 16:24:22 GMT
Subject: git: openjdk/babylon: code-reflection: Hat implement buffer
 minimization
Message-ID: <9fda7121-5ef4-4fba-b2ac-aee001a38c2a@openjdk.org>

Changeset: 08c3b3b2
Branch: code-reflection
Author:    Gary Frost <gfrost at openjdk.org>
Date:      2025-03-09 16:22:37 +0000
URL:       https://git.openjdk.org/babylon/commit/08c3b3b2b1d5a298eb238a07d83cf1bfad9b4bca

Hat implement buffer minimization

! hat/backends/ffi/opencl/cpp/opencl_backend.cpp
! hat/backends/ffi/opencl/cpp/opencl_backend_queue.cpp
! hat/backends/ffi/opencl/include/opencl_backend.h
! hat/bld
! hat/docs/hat-01-03-building-hat.md
! hat/examples/life/src/main/java/life/Main.java
! hat/examples/life/src/main/java/life/Viewer.java
= hat/hat-core/.gitignore
= hat/hat-core/pom.xml
= hat/hat-core/src/main/java/hat/Accelerator.java
= hat/hat-core/src/main/java/hat/ComputeContext.java
= hat/hat-core/src/main/java/hat/KernelContext.java
= hat/hat-core/src/main/java/hat/NDRange.java
= hat/hat-core/src/main/java/hat/OpsAndTypes.java
= hat/hat-core/src/main/java/hat/backend/Backend.java
= hat/hat-core/src/main/java/hat/backend/BackendAdaptor.java
= hat/hat-core/src/main/java/hat/backend/DebugBackend.java
= hat/hat-core/src/main/java/hat/backend/codebuilders/C99HATComputeBuilder.java
= hat/hat-core/src/main/java/hat/backend/codebuilders/C99HATKernelBuilder.java
= hat/hat-core/src/main/java/hat/backend/codebuilders/HATCodeBuilder.java
= hat/hat-core/src/main/java/hat/backend/codebuilders/HATCodeBuilderWithContext.java
= hat/hat-core/src/main/java/hat/backend/ffi/C99FFIBackend.java
= hat/hat-core/src/main/java/hat/backend/ffi/FFIBackend.java
= hat/hat-core/src/main/java/hat/backend/ffi/FFIBackendDriver.java
= hat/hat-core/src/main/java/hat/backend/ffi/FFILib.java
= hat/hat-core/src/main/java/hat/backend/java/JavaBackend.java
= hat/hat-core/src/main/java/hat/backend/java/JavaMultiThreadedBackend.java
= hat/hat-core/src/main/java/hat/backend/java/JavaSequentialBackend.java
= hat/hat-core/src/main/java/hat/backend/java/WorkStealer.java
= hat/hat-core/src/main/java/hat/backend/jextracted/C99JExtractedBackend.java
= hat/hat-core/src/main/java/hat/backend/jextracted/JExtractedBackend.java
= hat/hat-core/src/main/java/hat/backend/jextracted/JExtractedBackendDriver.java
= hat/hat-core/src/main/java/hat/buffer/ArgArray.java
= hat/hat-core/src/main/java/hat/buffer/Buffer.java
= hat/hat-core/src/main/java/hat/buffer/BufferAllocator.java
= hat/hat-core/src/main/java/hat/buffer/BufferTracker.java
= hat/hat-core/src/main/java/hat/buffer/ChessState.java
= hat/hat-core/src/main/java/hat/buffer/F32Array.java
= hat/hat-core/src/main/java/hat/buffer/F32Array2D.java
= hat/hat-core/src/main/java/hat/buffer/ImageIfaceBuffer.java
= hat/hat-core/src/main/java/hat/buffer/KernelContext.java
= hat/hat-core/src/main/java/hat/buffer/S08x3RGBImage.java
= hat/hat-core/src/main/java/hat/buffer/S32Array.java
= hat/hat-core/src/main/java/hat/buffer/S32Array2D.java
= hat/hat-core/src/main/java/hat/buffer/S32RGBAImage.java
= hat/hat-core/src/main/java/hat/buffer/SchemaBuilder.java
= hat/hat-core/src/main/java/hat/buffer/U16GreyImage.java
= hat/hat-core/src/main/java/hat/callgraph/CallGraph.java
= hat/hat-core/src/main/java/hat/callgraph/ComputeCallGraph.java
= hat/hat-core/src/main/java/hat/callgraph/ComputeEntrypoint.java
= hat/hat-core/src/main/java/hat/callgraph/Entrypoint.java
= hat/hat-core/src/main/java/hat/callgraph/KernelCallGraph.java
= hat/hat-core/src/main/java/hat/callgraph/KernelEntrypoint.java
= hat/hat-core/src/main/java/hat/ifacemapper/AbstractSegmentMapper.java
= hat/hat-core/src/main/java/hat/ifacemapper/BoundSchema.java
= hat/hat-core/src/main/java/hat/ifacemapper/ByteCodeGenerator.java
= hat/hat-core/src/main/java/hat/ifacemapper/MappableIface.java
= hat/hat-core/src/main/java/hat/ifacemapper/MapperCache.java
= hat/hat-core/src/main/java/hat/ifacemapper/MapperUtil.java
= hat/hat-core/src/main/java/hat/ifacemapper/Schema.java
= hat/hat-core/src/main/java/hat/ifacemapper/SegmentInterfaceMapper.java
= hat/hat-core/src/main/java/hat/ifacemapper/SegmentMapper.java
= hat/hat-core/src/main/java/hat/ifacemapper/accessor/AccessorInfo.java
= hat/hat-core/src/main/java/hat/ifacemapper/accessor/Accessors.java
= hat/hat-core/src/main/java/hat/ifacemapper/accessor/ArrayInfo.java
= hat/hat-core/src/main/java/hat/ifacemapper/accessor/Cardinality.java
= hat/hat-core/src/main/java/hat/ifacemapper/accessor/LayoutInfo.java
= hat/hat-core/src/main/java/hat/ifacemapper/accessor/ScalarInfo.java
= hat/hat-core/src/main/java/hat/ifacemapper/accessor/ValueType.java
= hat/hat-core/src/main/java/hat/ifacemapper/component/Util.java
= hat/hat-core/src/main/java/hat/opcodebuilders/OpCodeBuilder.java
= hat/hat-core/src/main/java/hat/opcodebuilders/StyledOpCodeBuilder.java
= hat/hat-core/src/main/java/hat/optools/BinaryArithmeticOrLogicOperation.java
= hat/hat-core/src/main/java/hat/optools/BinaryLogicalOpWrapper.java
= hat/hat-core/src/main/java/hat/optools/BinaryOpWrapper.java
= hat/hat-core/src/main/java/hat/optools/BinaryTestOpWrapper.java
= hat/hat-core/src/main/java/hat/optools/BlockWrapper.java
= hat/hat-core/src/main/java/hat/optools/BodyWrapper.java
= hat/hat-core/src/main/java/hat/optools/CodeElementWrapper.java
= hat/hat-core/src/main/java/hat/optools/ConstantOpWrapper.java
= hat/hat-core/src/main/java/hat/optools/ConvOpWrapper.java
= hat/hat-core/src/main/java/hat/optools/FieldAccessOpWrapper.java
= hat/hat-core/src/main/java/hat/optools/FieldLoadOpWrapper.java
= hat/hat-core/src/main/java/hat/optools/FieldStoreOpWrapper.java
= hat/hat-core/src/main/java/hat/optools/ForOpWrapper.java
= hat/hat-core/src/main/java/hat/optools/FuncCallOpWrapper.java
= hat/hat-core/src/main/java/hat/optools/FuncOpWrapper.java
= hat/hat-core/src/main/java/hat/optools/IfOpWrapper.java
= hat/hat-core/src/main/java/hat/optools/InvokeOpWrapper.java
= hat/hat-core/src/main/java/hat/optools/JavaBreakOpWrapper.java
= hat/hat-core/src/main/java/hat/optools/JavaContinueOpWrapper.java
= hat/hat-core/src/main/java/hat/optools/JavaLabeledOpWrapper.java
= hat/hat-core/src/main/java/hat/optools/LambdaOpWrapper.java
= hat/hat-core/src/main/java/hat/optools/LoadOpWrapper.java
= hat/hat-core/src/main/java/hat/optools/LogicalOpWrapper.java
= hat/hat-core/src/main/java/hat/optools/LoopOpWrapper.java
= hat/hat-core/src/main/java/hat/optools/ModuleOpWrapper.java
= hat/hat-core/src/main/java/hat/optools/OpWrapper.java
= hat/hat-core/src/main/java/hat/optools/ReturnOpWrapper.java
= hat/hat-core/src/main/java/hat/optools/RootSet.java
= hat/hat-core/src/main/java/hat/optools/StoreOpWrapper.java
= hat/hat-core/src/main/java/hat/optools/StructuralOpWrapper.java
= hat/hat-core/src/main/java/hat/optools/TernaryOpWrapper.java
= hat/hat-core/src/main/java/hat/optools/TupleOpWrapper.java
= hat/hat-core/src/main/java/hat/optools/UnaryArithmeticOrLogicOpWrapper.java
= hat/hat-core/src/main/java/hat/optools/UnaryOpWrapper.java
= hat/hat-core/src/main/java/hat/optools/VarAccessOpWrapper.java
= hat/hat-core/src/main/java/hat/optools/VarDeclarationOpWrapper.java
= hat/hat-core/src/main/java/hat/optools/VarFuncDeclarationOpWrapper.java
= hat/hat-core/src/main/java/hat/optools/VarLoadOpWrapper.java
= hat/hat-core/src/main/java/hat/optools/VarOpWrapper.java
= hat/hat-core/src/main/java/hat/optools/VarStoreOpWrapper.java
= hat/hat-core/src/main/java/hat/optools/WhileOpWrapper.java
= hat/hat-core/src/main/java/hat/optools/YieldOpWrapper.java
= hat/hat-core/src/main/java/hat/text/CodeBuilder.java
= hat/hat-core/src/main/java/hat/text/JavaCodeBuilder.java
= hat/hat-core/src/main/java/hat/text/TerminalColors.java
= hat/hat-core/src/main/java/hat/text/TextBuilder.java
= hat/hat-core/src/main/java/hat/util/Result.java
= hat/hat-core/src/main/java/hat/util/StreamCounter.java
+ hat/hat-core/src/main/java/hat/util/ui/SevenSegmentDisplay.java
= hat/hat-core/src/main/test/hat/CustomOpTest.java
= hat/hat-core/src/main/test/hat/SquaresTest.java
= hat/hat-core/src/main/test/logger/HATTestNGLogger.java
- hat/hatless-examples/nbody/.gitignore
- hat/hatless-examples/nbody/src/main/java/nbody/CLWrap.java
- hat/hatless-examples/nbody/src/main/java/nbody/GLWrap.java
- hat/hatless-examples/nbody/src/main/java/nbody/Main.java
- hat/hatless-examples/nbody/src/main/resources/particle.png
- hat/hatlessrun
- hat/hatrun.bash
! hat/intellij/.idea/compiler.xml
! hat/intellij/.idea/modules.xml
! hat/intellij/backend_ffi_cuda.iml
! hat/intellij/backend_ffi_mock.iml
! hat/intellij/backend_ffi_opencl.iml
! hat/intellij/backend_ffi_ptx.iml
! hat/intellij/backend_ffi_shared.iml
! hat/intellij/backend_ffi_spirv.iml
! hat/intellij/backend_java_mt.iml
! hat/intellij/backend_java_seq.iml
! hat/intellij/backend_jextracted_opencl.iml
! hat/intellij/backend_jextracted_shared.iml
! hat/intellij/blackscholes.iml
! hat/intellij/chess.iml
! hat/intellij/clwrap.iml
! hat/intellij/experiments.iml
+ hat/intellij/hat-core.iml
- hat/intellij/hat.iml
! hat/intellij/heal.iml
! hat/intellij/life.iml
! hat/intellij/mandel.iml
! hat/intellij/nbody.iml
! hat/intellij/squares.iml
! hat/intellij/view.iml
! hat/intellij/violajones.iml
- hat/licensecheck.bash


From gfrost at openjdk.org  Sun Mar  9 16:26:18 2025
From: gfrost at openjdk.org (Gary Frost)
Date: Sun, 9 Mar 2025 16:26:18 GMT
Subject: [code-reflection] Integrated: Hat implement buffer minimization
Message-ID: <W96fzSGJVDQ0CnNng5dMjENadSUQEBBMakB1WaqRCSE=.e6cd9f32-2140-4259-a803-412dffb36284@github.com>

Looks like lots of changes, but mainly moved files from hat/hat to hat/hat-core.  

This free's up the shell script 'namespace' so I can add `hat` as a script for builds and sanity checks. 

Got rid of bash files for sanity checking and for building.  We should be using `java @bldr/bld`

Synced docs with these changes. 

The life demo now correctly displays Generations Per Second and more code is shared between the jextract and hat execution paths.

-------------

Commit messages:
 - moved hat -> hat-core dir (to free up hat as a command name)  and modified iml, bld and docs
 - Moved SevenSegmentDisplay to hat.util.ui (so we can use it in other examples)
 - Finally we have correct numbers
 - Add seven seg displays to life
 - We can now add string markers to the profile.

Changes: https://git.openjdk.org/babylon/pull/342/files
  Webrev: https://webrevs.openjdk.org/?repo=babylon&pr=342&range=00
  Stats: 2020 lines in 157 files changed: 373 ins; 1448 del; 199 mod
  Patch: https://git.openjdk.org/babylon/pull/342.diff
  Fetch: git fetch https://git.openjdk.org/babylon.git pull/342/head:pull/342

PR: https://git.openjdk.org/babylon/pull/342

From gfrost at openjdk.org  Sun Mar  9 16:26:20 2025
From: gfrost at openjdk.org (Gary Frost)
Date: Sun, 9 Mar 2025 16:26:20 GMT
Subject: [code-reflection] Integrated: Hat implement buffer minimization
In-Reply-To: <W96fzSGJVDQ0CnNng5dMjENadSUQEBBMakB1WaqRCSE=.e6cd9f32-2140-4259-a803-412dffb36284@github.com>
References: <W96fzSGJVDQ0CnNng5dMjENadSUQEBBMakB1WaqRCSE=.e6cd9f32-2140-4259-a803-412dffb36284@github.com>
Message-ID: <kthO7kroObmVkcefP7eJw0BlJWx1UnM2a4TSyV59xdk=.343bc126-f27b-447c-92bc-4913395d8a87@github.com>

On Sun, 9 Mar 2025 16:20:46 GMT, Gary Frost <gfrost at openjdk.org> wrote:

> Looks like lots of changes, but mainly moved files from hat/hat to hat/hat-core.  
> 
> This free's up the shell script 'namespace' so I can add `hat` as a script for builds and sanity checks. 
> 
> Got rid of bash files for sanity checking and for building.  We should be using `java @bldr/bld`
> 
> Synced docs with these changes. 
> 
> The life demo now correctly displays Generations Per Second and more code is shared between the jextract and hat execution paths.

This pull request has now been integrated.

Changeset: 08c3b3b2
Author:    Gary Frost <gfrost at openjdk.org>
URL:       https://git.openjdk.org/babylon/commit/08c3b3b2b1d5a298eb238a07d83cf1bfad9b4bca
Stats:     2020 lines in 157 files changed: 373 ins; 1448 del; 199 mod

Hat implement buffer minimization

-------------

PR: https://git.openjdk.org/babylon/pull/342

From asotona at openjdk.org  Sun Mar  9 21:44:57 2025
From: asotona at openjdk.org (Adam Sotona)
Date: Sun, 9 Mar 2025 21:44:57 GMT
Subject: [code-reflection] RFR: Initializers through arguments [v4]
In-Reply-To: <Nqa8WhMnqKKw0I6sEMTp0yzm7BMXzFgb1LWYsm5kLn0=.ed95950f-8aa2-4d28-b898-2bb53247d60d@github.com>
References: <Nqa8WhMnqKKw0I6sEMTp0yzm7BMXzFgb1LWYsm5kLn0=.ed95950f-8aa2-4d28-b898-2bb53247d60d@github.com>
Message-ID: <aSqOSkx2rG4UBSHcLDsTGodTNlDIAHshxc8e5ZgXcnI=.ef296b2f-44cb-4c18-a4a5-d519255a453e@github.com>

> Initializers through arguments implementation.
> This PR is chained to ongoing work on `If` and `Loop` due to conflicting changes in `OnnxProtoBuilder`.
> 
> Input image reshape has to be removed from the CNN model due to unexpected exceptions in `Conv` op in relation with initializers.

Adam Sotona has updated the pull request incrementally with one additional commit since the last revision:

  initializers from fields

-------------

Changes:
  - all: https://git.openjdk.org/babylon/pull/340/files
  - new: https://git.openjdk.org/babylon/pull/340/files/dbf678a3..acc21440

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=babylon&pr=340&range=03
 - incr: https://webrevs.openjdk.org/?repo=babylon&pr=340&range=02-03

  Stats: 348 lines in 10 files changed: 150 ins; 173 del; 25 mod
  Patch: https://git.openjdk.org/babylon/pull/340.diff
  Fetch: git fetch https://git.openjdk.org/babylon.git pull/340/head:pull/340

PR: https://git.openjdk.org/babylon/pull/340

From asotona at openjdk.org  Mon Mar 10 06:35:01 2025
From: asotona at openjdk.org (Adam Sotona)
Date: Mon, 10 Mar 2025 06:35:01 GMT
Subject: [code-reflection] RFR: Initializers through arguments [v5]
In-Reply-To: <Nqa8WhMnqKKw0I6sEMTp0yzm7BMXzFgb1LWYsm5kLn0=.ed95950f-8aa2-4d28-b898-2bb53247d60d@github.com>
References: <Nqa8WhMnqKKw0I6sEMTp0yzm7BMXzFgb1LWYsm5kLn0=.ed95950f-8aa2-4d28-b898-2bb53247d60d@github.com>
Message-ID: <7qZUmNnfaj2vnoWLkjjpETC2eiciA_iXz_6lBYadl_0=.75d25782-36e1-4186-87cc-285b81940873@github.com>

> Initializers through arguments implementation.
> This PR is chained to ongoing work on `If` and `Loop` due to conflicting changes in `OnnxProtoBuilder`.
> 
> Input image reshape has to be removed from the CNN model due to unexpected exceptions in `Conv` op in relation with initializers.

Adam Sotona has updated the pull request incrementally with one additional commit since the last revision:

  raw implemenation of instance methods and instance initializers

-------------

Changes:
  - all: https://git.openjdk.org/babylon/pull/340/files
  - new: https://git.openjdk.org/babylon/pull/340/files/acc21440..384ee276

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=babylon&pr=340&range=04
 - incr: https://webrevs.openjdk.org/?repo=babylon&pr=340&range=03-04

  Stats: 87 lines in 8 files changed: 40 ins; 4 del; 43 mod
  Patch: https://git.openjdk.org/babylon/pull/340.diff
  Fetch: git fetch https://git.openjdk.org/babylon.git pull/340/head:pull/340

PR: https://git.openjdk.org/babylon/pull/340

From asotona at openjdk.org  Mon Mar 10 07:21:51 2025
From: asotona at openjdk.org (Adam Sotona)
Date: Mon, 10 Mar 2025 07:21:51 GMT
Subject: [code-reflection] RFR: Initializers through arguments [v6]
In-Reply-To: <Nqa8WhMnqKKw0I6sEMTp0yzm7BMXzFgb1LWYsm5kLn0=.ed95950f-8aa2-4d28-b898-2bb53247d60d@github.com>
References: <Nqa8WhMnqKKw0I6sEMTp0yzm7BMXzFgb1LWYsm5kLn0=.ed95950f-8aa2-4d28-b898-2bb53247d60d@github.com>
Message-ID: <b6n7EIQ1tC1EVuz4dGmPBAYjFvlfkyZqaoiiNayvHXE=.8637709b-ef33-49a9-977b-66402b7b99e2@github.com>

> Initializers through arguments implementation.
> This PR is chained to ongoing work on `If` and `Loop` due to conflicting changes in `OnnxProtoBuilder`.
> 
> Input image reshape has to be removed from the CNN model due to unexpected exceptions in `Conv` op in relation with initializers.

Adam Sotona has updated the pull request incrementally with one additional commit since the last revision:

  Minor demo cleanup

-------------

Changes:
  - all: https://git.openjdk.org/babylon/pull/340/files
  - new: https://git.openjdk.org/babylon/pull/340/files/384ee276..35dfef7b

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=babylon&pr=340&range=05
 - incr: https://webrevs.openjdk.org/?repo=babylon&pr=340&range=04-05

  Stats: 18 lines in 2 files changed: 9 ins; 5 del; 4 mod
  Patch: https://git.openjdk.org/babylon/pull/340.diff
  Fetch: git fetch https://git.openjdk.org/babylon.git pull/340/head:pull/340

PR: https://git.openjdk.org/babylon/pull/340

From asotona at openjdk.org  Mon Mar 10 07:27:55 2025
From: asotona at openjdk.org (Adam Sotona)
Date: Mon, 10 Mar 2025 07:27:55 GMT
Subject: [code-reflection] RFR: Initializers through arguments [v7]
In-Reply-To: <Nqa8WhMnqKKw0I6sEMTp0yzm7BMXzFgb1LWYsm5kLn0=.ed95950f-8aa2-4d28-b898-2bb53247d60d@github.com>
References: <Nqa8WhMnqKKw0I6sEMTp0yzm7BMXzFgb1LWYsm5kLn0=.ed95950f-8aa2-4d28-b898-2bb53247d60d@github.com>
Message-ID: <fmrFwMJlpjy0awdiTVFD_uIOmPs1-CQuMRKxDi-mU5E=.c16d01c3-1955-4480-8bf2-06cccb9400e8@github.com>

> Initializers through arguments implementation.
> This PR is chained to ongoing work on `If` and `Loop` due to conflicting changes in `OnnxProtoBuilder`.
> 
> Input image reshape has to be removed from the CNN model due to unexpected exceptions in `Conv` op in relation with initializers.

Adam Sotona has updated the pull request incrementally with one additional commit since the last revision:

  MNISTDemoUI rename

-------------

Changes:
  - all: https://git.openjdk.org/babylon/pull/340/files
  - new: https://git.openjdk.org/babylon/pull/340/files/35dfef7b..08bd6315

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=babylon&pr=340&range=06
 - incr: https://webrevs.openjdk.org/?repo=babylon&pr=340&range=05-06

  Stats: 3 lines in 1 file changed: 0 ins; 0 del; 3 mod
  Patch: https://git.openjdk.org/babylon/pull/340.diff
  Fetch: git fetch https://git.openjdk.org/babylon.git pull/340/head:pull/340

PR: https://git.openjdk.org/babylon/pull/340

From asotona at openjdk.org  Mon Mar 10 08:08:10 2025
From: asotona at openjdk.org (Adam Sotona)
Date: Mon, 10 Mar 2025 08:08:10 GMT
Subject: [code-reflection] RFR: Initializers through arguments [v8]
In-Reply-To: <Nqa8WhMnqKKw0I6sEMTp0yzm7BMXzFgb1LWYsm5kLn0=.ed95950f-8aa2-4d28-b898-2bb53247d60d@github.com>
References: <Nqa8WhMnqKKw0I6sEMTp0yzm7BMXzFgb1LWYsm5kLn0=.ed95950f-8aa2-4d28-b898-2bb53247d60d@github.com>
Message-ID: <u5g3lBaaOgNoxwRVSNTfAyzQD-XO_KhHj7bGc3o9q3g=.dc51964e-629d-4261-ad96-bd7395eea868@github.com>

> Initializers through arguments implementation.
> This PR is chained to ongoing work on `If` and `Loop` due to conflicting changes in `OnnxProtoBuilder`.
> 
> Input image reshape has to be removed from the CNN model due to unexpected exceptions in `Conv` op in relation with initializers.

Adam Sotona has updated the pull request incrementally with one additional commit since the last revision:

  nit change

-------------

Changes:
  - all: https://git.openjdk.org/babylon/pull/340/files
  - new: https://git.openjdk.org/babylon/pull/340/files/08bd6315..d885df19

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=babylon&pr=340&range=07
 - incr: https://webrevs.openjdk.org/?repo=babylon&pr=340&range=06-07

  Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod
  Patch: https://git.openjdk.org/babylon/pull/340.diff
  Fetch: git fetch https://git.openjdk.org/babylon.git pull/340/head:pull/340

PR: https://git.openjdk.org/babylon/pull/340

From gfrost at openjdk.org  Mon Mar 10 09:45:22 2025
From: gfrost at openjdk.org (Gary Frost)
Date: Mon, 10 Mar 2025 09:45:22 GMT
Subject: git: openjdk/babylon: code-reflection: Updated UIs for heal and
 violajones to use seven seg display widget
Message-ID: <7f426829-ce3f-43cf-b191-976d65511626@openjdk.org>

Changeset: c5ba520f
Branch: code-reflection
Author:    Gary Frost <gfrost at openjdk.org>
Date:      2025-03-10 09:44:14 +0000
URL:       https://git.openjdk.org/babylon/commit/c5ba520f76ef6c8124df80c970b585a7c06b1090

Updated UIs for heal and violajones to use seven seg display widget

! hat/examples/heal/src/main/java/heal/Compute.java
! hat/examples/heal/src/main/java/heal/Viewer.java
! hat/examples/violajones/src/main/java/violajones/HaarViewer.java
! hat/examples/violajones/src/main/java/violajones/Main.java
! hat/examples/violajones/src/main/java/violajones/ViolaJonesCoreCompute.java
! hat/examples/violajones/src/main/java/violajones/attic/ViolaJones.java


From gfrost at openjdk.org  Mon Mar 10 09:47:40 2025
From: gfrost at openjdk.org (Gary Frost)
Date: Mon, 10 Mar 2025 09:47:40 GMT
Subject: [code-reflection] Integrated: Updated UIs for heal and violajones to
 use seven seg display widget
In-Reply-To: <J82ACXCKm743tKCaVpwcSvS73GPiS2a_sgOst42nCLQ=.947084d9-e3a7-4054-bc61-54a3ae74872d@github.com>
References: <J82ACXCKm743tKCaVpwcSvS73GPiS2a_sgOst42nCLQ=.947084d9-e3a7-4054-bc61-54a3ae74872d@github.com>
Message-ID: <2tTpMAWOsgIYLDJqtz7mB78LEqJ_8FsSXzmaZYzv074=.59a2316b-d70e-4b1a-b6d2-37f3b0cad05a@github.com>

On Mon, 10 Mar 2025 09:42:04 GMT, Gary Frost <gfrost at openjdk.org> wrote:

> For some consistency (prep for J1 demos) I am trying to use some common layouts for example UI's
> 
> Here we us use the seven seg widget (included in life example)  for violajones and heal examples.

This pull request has now been integrated.

Changeset: c5ba520f
Author:    Gary Frost <gfrost at openjdk.org>
URL:       https://git.openjdk.org/babylon/commit/c5ba520f76ef6c8124df80c970b585a7c06b1090
Stats:     59 lines in 6 files changed: 31 ins; 4 del; 24 mod

Updated UIs for heal and violajones to use seven seg display widget

-------------

PR: https://git.openjdk.org/babylon/pull/343

From gfrost at openjdk.org  Mon Mar 10 09:47:40 2025
From: gfrost at openjdk.org (Gary Frost)
Date: Mon, 10 Mar 2025 09:47:40 GMT
Subject: [code-reflection] Integrated: Updated UIs for heal and violajones to
 use seven seg display widget
Message-ID: <J82ACXCKm743tKCaVpwcSvS73GPiS2a_sgOst42nCLQ=.947084d9-e3a7-4054-bc61-54a3ae74872d@github.com>

For some consistency (prep for J1 demos) I am trying to use some common layouts for example UI's

Here we us use the seven seg widget (included in life example)  for violajones and heal examples.

-------------

Commit messages:
 - Updated UIs for heal and violajones to use seven seg display widget

Changes: https://git.openjdk.org/babylon/pull/343/files
  Webrev: https://webrevs.openjdk.org/?repo=babylon&pr=343&range=00
  Stats: 59 lines in 6 files changed: 31 ins; 4 del; 24 mod
  Patch: https://git.openjdk.org/babylon/pull/343.diff
  Fetch: git fetch https://git.openjdk.org/babylon.git pull/343/head:pull/343

PR: https://git.openjdk.org/babylon/pull/343

From gfrost at openjdk.org  Mon Mar 10 14:16:16 2025
From: gfrost at openjdk.org (Gary Frost)
Date: Mon, 10 Mar 2025 14:16:16 GMT
Subject: git: openjdk/babylon: code-reflection: synced up maven and bld
 artifact names
Message-ID: <89bb66c5-b0e8-4458-a5b7-baec7c316cf2@openjdk.org>

Changeset: 2224a53d
Branch: code-reflection
Author:    Gary Frost <gfrost at openjdk.org>
Date:      2025-03-10 14:13:36 +0000
URL:       https://git.openjdk.org/babylon/commit/2224a53da6ba9a65db4fb268a58f8ba25536aedd

synced up maven and bld artifact names

! hat/backends/ffi/cuda/pom.xml
! hat/backends/ffi/hip/pom.xml
! hat/backends/ffi/mock/pom.xml
! hat/backends/ffi/opencl/pom.xml
! hat/backends/ffi/pom.xml
! hat/backends/ffi/ptx/pom.xml
! hat/backends/ffi/shared/pom.xml
! hat/backends/ffi/spirv/pom.xml
! hat/backends/java/mt/pom.xml
! hat/backends/java/pom.xml
! hat/backends/java/seq/pom.xml
! hat/backends/pom.xml
! hat/bld
! hat/bldr/Bldr.java
! hat/examples/blackscholes/pom.xml
! hat/examples/experiments/pom.xml
! hat/examples/heal/pom.xml
! hat/examples/life/pom.xml
! hat/examples/mandel/pom.xml
! hat/examples/pom.xml
! hat/examples/squares/pom.xml
! hat/examples/violajones/pom.xml
! hat/extractions/opencl/pom.xml
! hat/extractions/opengl/pom.xml
! hat/extractions/pom.xml
! hat/hat-core/pom.xml
! hat/hatrun
! hat/maven-build.bash
! hat/mkpoms
! hat/pom.xml
+ hat/wrap/clwrap/pom.xml
+ hat/wrap/glwrap/pom.xml
+ hat/wrap/pom.xml
+ hat/wrap/wrap/pom.xml


From gfrost at openjdk.org  Mon Mar 10 14:17:17 2025
From: gfrost at openjdk.org (Gary Frost)
Date: Mon, 10 Mar 2025 14:17:17 GMT
Subject: [code-reflection] Integrated: synced up maven and bld artifact names
Message-ID: <L0Z0EVGxIjPegiisZ4SrYsWAVJv0MxyFyzn4IgOnnmo=.923e5837-e554-4d20-9c85-14317930cec0@github.com>

Maven and bldr artifact names had diverged. This fixes that issue. 

To test 

rm -rf stage build
java @bldr/bld
java @bldr/hatrun ffi-opencl heal
# confirm heal runs
rm -rf stage build
bash maven-build.bash
java @bldr/hatrun ffi-opencl heal
# confirm heal runs

-------------

Commit messages:
 - synced up maven and bld artifact names

Changes: https://git.openjdk.org/babylon/pull/344/files
  Webrev: https://webrevs.openjdk.org/?repo=babylon&pr=344&range=00
  Stats: 469 lines in 34 files changed: 350 ins; 67 del; 52 mod
  Patch: https://git.openjdk.org/babylon/pull/344.diff
  Fetch: git fetch https://git.openjdk.org/babylon.git pull/344/head:pull/344

PR: https://git.openjdk.org/babylon/pull/344

From gfrost at openjdk.org  Mon Mar 10 14:17:18 2025
From: gfrost at openjdk.org (Gary Frost)
Date: Mon, 10 Mar 2025 14:17:18 GMT
Subject: [code-reflection] Integrated: synced up maven and bld artifact names
In-Reply-To: <L0Z0EVGxIjPegiisZ4SrYsWAVJv0MxyFyzn4IgOnnmo=.923e5837-e554-4d20-9c85-14317930cec0@github.com>
References: <L0Z0EVGxIjPegiisZ4SrYsWAVJv0MxyFyzn4IgOnnmo=.923e5837-e554-4d20-9c85-14317930cec0@github.com>
Message-ID: <zmCLwTrqtaArdkm8Msu5YNtxsaSTxT-Id8V5ulYRojI=.9bd1143a-5649-4e50-9742-dbd96f9ba155@github.com>

On Mon, 10 Mar 2025 14:11:30 GMT, Gary Frost <gfrost at openjdk.org> wrote:

> Maven and bldr artifact names had diverged. This fixes that issue. 
> 
> To test 
> 
> rm -rf stage build
> java @bldr/bld
> java @bldr/hatrun ffi-opencl heal
> # confirm heal runs
> rm -rf stage build
> bash maven-build.bash
> java @bldr/hatrun ffi-opencl heal
> # confirm heal runs

This pull request has now been integrated.

Changeset: 2224a53d
Author:    Gary Frost <gfrost at openjdk.org>
URL:       https://git.openjdk.org/babylon/commit/2224a53da6ba9a65db4fb268a58f8ba25536aedd
Stats:     469 lines in 34 files changed: 350 ins; 67 del; 52 mod

synced up maven and bld artifact names

-------------

PR: https://git.openjdk.org/babylon/pull/344

From asotona at openjdk.org  Mon Mar 10 14:46:08 2025
From: asotona at openjdk.org (Adam Sotona)
Date: Mon, 10 Mar 2025 14:46:08 GMT
Subject: [code-reflection] RFR: Initializers through arguments [v9]
In-Reply-To: <Nqa8WhMnqKKw0I6sEMTp0yzm7BMXzFgb1LWYsm5kLn0=.ed95950f-8aa2-4d28-b898-2bb53247d60d@github.com>
References: <Nqa8WhMnqKKw0I6sEMTp0yzm7BMXzFgb1LWYsm5kLn0=.ed95950f-8aa2-4d28-b898-2bb53247d60d@github.com>
Message-ID: <bWKkbV98f-XmEPKGtw6s8us8LsuzwCH_hUsi3HIlRoQ=.8aa571c4-5fed-48fa-baff-c5757c2e4f36@github.com>

> Initializers through arguments implementation.
> This PR is chained to ongoing work on `If` and `Loop` due to conflicting changes in `OnnxProtoBuilder`.
> 
> Input image reshape has to be removed from the CNN model due to unexpected exceptions in `Conv` op in relation with initializers.

Adam Sotona has updated the pull request incrementally with one additional commit since the last revision:

  OnnxProtoBuilder and OnnxProtoPrinter implemented support for packed data

-------------

Changes:
  - all: https://git.openjdk.org/babylon/pull/340/files
  - new: https://git.openjdk.org/babylon/pull/340/files/d885df19..5a0213f9

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=babylon&pr=340&range=08
 - incr: https://webrevs.openjdk.org/?repo=babylon&pr=340&range=07-08

  Stats: 135 lines in 2 files changed: 89 ins; 1 del; 45 mod
  Patch: https://git.openjdk.org/babylon/pull/340.diff
  Fetch: git fetch https://git.openjdk.org/babylon.git pull/340/head:pull/340

PR: https://git.openjdk.org/babylon/pull/340

From asotona at openjdk.org  Mon Mar 10 18:23:59 2025
From: asotona at openjdk.org (Adam Sotona)
Date: Mon, 10 Mar 2025 18:23:59 GMT
Subject: [code-reflection] RFR: Initializers through arguments [v10]
In-Reply-To: <Nqa8WhMnqKKw0I6sEMTp0yzm7BMXzFgb1LWYsm5kLn0=.ed95950f-8aa2-4d28-b898-2bb53247d60d@github.com>
References: <Nqa8WhMnqKKw0I6sEMTp0yzm7BMXzFgb1LWYsm5kLn0=.ed95950f-8aa2-4d28-b898-2bb53247d60d@github.com>
Message-ID: <LOb-bjwbzJZ0IXLgIyMuLzeEo50gqnJ0itdMrxNFQ38=.1798fa5b-bf3d-439c-b222-bbd6afcc4b04@github.com>

> Initializers through arguments implementation.
> This PR is chained to ongoing work on `If` and `Loop` due to conflicting changes in `OnnxProtoBuilder`.
> 
> Input image reshape has to be removed from the CNN model due to unexpected exceptions in `Conv` op in relation with initializers.

Adam Sotona has updated the pull request incrementally with one additional commit since the last revision:

  added DEBUG switch to OnnxRuntime with models print and export
  fixed protobuf input and output names matching Onnx model
  re-enabled some tests
  tests disabling DEBUG switch

-------------

Changes:
  - all: https://git.openjdk.org/babylon/pull/340/files
  - new: https://git.openjdk.org/babylon/pull/340/files/5a0213f9..ee1fbafb

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=babylon&pr=340&range=09
 - incr: https://webrevs.openjdk.org/?repo=babylon&pr=340&range=08-09

  Stats: 30 lines in 5 files changed: 27 ins; 0 del; 3 mod
  Patch: https://git.openjdk.org/babylon/pull/340.diff
  Fetch: git fetch https://git.openjdk.org/babylon.git pull/340/head:pull/340

PR: https://git.openjdk.org/babylon/pull/340

From asotona at openjdk.org  Mon Mar 10 18:58:36 2025
From: asotona at openjdk.org (Adam Sotona)
Date: Mon, 10 Mar 2025 18:58:36 GMT
Subject: git: openjdk/babylon: code-reflection: Initializers through arguments
Message-ID: <05beb594-064c-4745-bf99-76c588bef9fc@openjdk.org>

Changeset: 683c318a
Branch: code-reflection
Author:    Adam Sotona <asotona at openjdk.org>
Date:      2025-03-10 18:57:49 +0000
URL:       https://git.openjdk.org/babylon/commit/683c318aa6354670167b6c1ec23def7c5b1897c7

Initializers through arguments

! cr-examples/onnx/src/main/java/oracle/code/onnx/ExplicitOnnxOperators.java
+ cr-examples/onnx/src/main/java/oracle/code/onnx/LambdaToFunc.java
! cr-examples/onnx/src/main/java/oracle/code/onnx/OnnxProtoBuilder.java
! cr-examples/onnx/src/main/java/oracle/code/onnx/OnnxProtoPrinter.java
! cr-examples/onnx/src/main/java/oracle/code/onnx/OnnxRuntime.java
! cr-examples/onnx/src/main/java/oracle/code/onnx/Tensor.java
! cr-examples/onnx/src/main/java/oracle/code/onnx/compiler/OnnxPartialEvaluator.java
! cr-examples/onnx/src/main/java/oracle/code/onnx/compiler/OnnxTransformer.java
+ cr-examples/onnx/src/main/java/oracle/code/onnx/ir/ExplicitOnnxOps.java
! cr-examples/onnx/src/main/java/oracle/code/onnx/ir/OnnxOps.java
! cr-examples/onnx/src/test/java/oracle/code/onnx/CNNTest.java
! cr-examples/onnx/src/test/java/oracle/code/onnx/RuntimeTest.java
! cr-examples/onnx/src/test/java/oracle/code/onnx/SimpleTest.java
! cr-examples/onnx/src/test/java/oracle/code/onnx/mnist/MNISTDemo.java
- cr-examples/onnx/src/test/java/oracle/code/onnx/mnist/MNISTDemoUI.java
+ cr-examples/onnx/src/test/java/oracle/code/onnx/mnist/MNISTModel.java


From asotona at openjdk.org  Mon Mar 10 19:01:03 2025
From: asotona at openjdk.org (Adam Sotona)
Date: Mon, 10 Mar 2025 19:01:03 GMT
Subject: [code-reflection] RFR: Initializers through arguments [v11]
In-Reply-To: <Nqa8WhMnqKKw0I6sEMTp0yzm7BMXzFgb1LWYsm5kLn0=.ed95950f-8aa2-4d28-b898-2bb53247d60d@github.com>
References: <Nqa8WhMnqKKw0I6sEMTp0yzm7BMXzFgb1LWYsm5kLn0=.ed95950f-8aa2-4d28-b898-2bb53247d60d@github.com>
Message-ID: <JRWfndshS_GANiWsPhMx5_VLI8pysD5Kn-ni0mWI46U=.0bad8b5d-ca33-4971-8ff2-9ddf14441a81@github.com>

> Initializers through arguments implementation.
> This PR is chained to ongoing work on `If` and `Loop` due to conflicting changes in `OnnxProtoBuilder`.
> 
> Input image reshape has to be removed from the CNN model due to unexpected exceptions in `Conv` op in relation with initializers.

Adam Sotona has updated the pull request incrementally with one additional commit since the last revision:

  nit change

-------------

Changes:
  - all: https://git.openjdk.org/babylon/pull/340/files
  - new: https://git.openjdk.org/babylon/pull/340/files/ee1fbafb..001dea5b

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=babylon&pr=340&range=10
 - incr: https://webrevs.openjdk.org/?repo=babylon&pr=340&range=09-10

  Stats: 3 lines in 1 file changed: 0 ins; 2 del; 1 mod
  Patch: https://git.openjdk.org/babylon/pull/340.diff
  Fetch: git fetch https://git.openjdk.org/babylon.git pull/340/head:pull/340

PR: https://git.openjdk.org/babylon/pull/340

From asotona at openjdk.org  Mon Mar 10 19:01:04 2025
From: asotona at openjdk.org (Adam Sotona)
Date: Mon, 10 Mar 2025 19:01:04 GMT
Subject: [code-reflection] Integrated: Initializers through arguments
In-Reply-To: <Nqa8WhMnqKKw0I6sEMTp0yzm7BMXzFgb1LWYsm5kLn0=.ed95950f-8aa2-4d28-b898-2bb53247d60d@github.com>
References: <Nqa8WhMnqKKw0I6sEMTp0yzm7BMXzFgb1LWYsm5kLn0=.ed95950f-8aa2-4d28-b898-2bb53247d60d@github.com>
Message-ID: <zup0uoAazvk97oI6995MCNfHoTwIC2IMRIbkrOv6-M8=.90fe0d63-7e8f-4d38-a685-816e367f9ff1@github.com>

On Fri, 7 Mar 2025 08:30:16 GMT, Adam Sotona <asotona at openjdk.org> wrote:

> Initializers through arguments implementation.
> This PR is chained to ongoing work on `If` and `Loop` due to conflicting changes in `OnnxProtoBuilder`.
> 
> Input image reshape has to be removed from the CNN model due to unexpected exceptions in `Conv` op in relation with initializers.

This pull request has now been integrated.

Changeset: 683c318a
Author:    Adam Sotona <asotona at openjdk.org>
URL:       https://git.openjdk.org/babylon/commit/683c318aa6354670167b6c1ec23def7c5b1897c7
Stats:     1224 lines in 16 files changed: 773 ins; 275 del; 176 mod

Initializers through arguments

-------------

PR: https://git.openjdk.org/babylon/pull/340

From psandoz at openjdk.org  Tue Mar 11 00:39:05 2025
From: psandoz at openjdk.org (Paul Sandoz)
Date: Tue, 11 Mar 2025 00:39:05 GMT
Subject: git: openjdk/babylon: code-reflection: Use debug system property
Message-ID: <9dcd0ef0-05cf-49a6-b11f-d2c4c30cdb49@openjdk.org>

Changeset: 85fafbce
Branch: code-reflection
Author:    Paul Sandoz <psandoz at openjdk.org>
Date:      2025-03-11 00:36:29 +0000
URL:       https://git.openjdk.org/babylon/commit/85fafbcea7de015a057abb9a2091ebddb043b08a

Use debug system property

! cr-examples/onnx/src/main/java/oracle/code/onnx/LambdaToFunc.java
! cr-examples/onnx/src/main/java/oracle/code/onnx/OnnxRuntime.java
! cr-examples/onnx/src/main/java/oracle/code/onnx/compiler/OnnxTransformer.java
! cr-examples/onnx/src/test/java/oracle/code/onnx/CNNTest.java
! cr-examples/onnx/src/test/java/oracle/code/onnx/RuntimeTest.java
! cr-examples/onnx/src/test/java/oracle/code/onnx/SimpleTest.java
! cr-examples/onnx/src/test/java/oracle/code/onnx/mnist/MNISTModel.java


From psandoz at openjdk.org  Tue Mar 11 00:39:17 2025
From: psandoz at openjdk.org (Paul Sandoz)
Date: Tue, 11 Mar 2025 00:39:17 GMT
Subject: [code-reflection] Integrated: Use debug system property
Message-ID: <rl_78q1S4UV46kSOVWDr4me4UNpQ1OvXSVlo17a4oFg=.f832e0a8-7230-401f-b312-b9ab078e2778@github.com>

Use a system property for the debug value. When set also output the Java code model that is transformed to an ONNX code model. Tweak the example code+comments.

-------------

Commit messages:
 - Use debug system property.

Changes: https://git.openjdk.org/babylon/pull/345/files
  Webrev: https://webrevs.openjdk.org/?repo=babylon&pr=345&range=00
  Stats: 53 lines in 7 files changed: 24 ins; 13 del; 16 mod
  Patch: https://git.openjdk.org/babylon/pull/345.diff
  Fetch: git fetch https://git.openjdk.org/babylon.git pull/345/head:pull/345

PR: https://git.openjdk.org/babylon/pull/345

From psandoz at openjdk.org  Tue Mar 11 00:39:17 2025
From: psandoz at openjdk.org (Paul Sandoz)
Date: Tue, 11 Mar 2025 00:39:17 GMT
Subject: [code-reflection] Integrated: Use debug system property
In-Reply-To: <rl_78q1S4UV46kSOVWDr4me4UNpQ1OvXSVlo17a4oFg=.f832e0a8-7230-401f-b312-b9ab078e2778@github.com>
References: <rl_78q1S4UV46kSOVWDr4me4UNpQ1OvXSVlo17a4oFg=.f832e0a8-7230-401f-b312-b9ab078e2778@github.com>
Message-ID: <8XR_-olR4r5FxMvijHDqqAXIt4P5FP76NKRuNjnZhjk=.606832d0-825c-42a8-a58b-f47bf0ae40d5@github.com>

On Tue, 11 Mar 2025 00:34:15 GMT, Paul Sandoz <psandoz at openjdk.org> wrote:

> Use a system property for the debug value. When set also output the Java code model that is transformed to an ONNX code model. Tweak the example code+comments.

This pull request has now been integrated.

Changeset: 85fafbce
Author:    Paul Sandoz <psandoz at openjdk.org>
URL:       https://git.openjdk.org/babylon/commit/85fafbcea7de015a057abb9a2091ebddb043b08a
Stats:     53 lines in 7 files changed: 24 ins; 13 del; 16 mod

Use debug system property

-------------

PR: https://git.openjdk.org/babylon/pull/345

From gfrost at openjdk.org  Tue Mar 11 17:58:35 2025
From: gfrost at openjdk.org (Gary Frost)
Date: Tue, 11 Mar 2025 17:58:35 GMT
Subject: git: openjdk/babylon: code-reflection: Hat UI updates and iml fixes
Message-ID: <c165caab-2198-4881-8fce-4edd7b628c27@openjdk.org>

Changeset: 05ae51c3
Branch: code-reflection
Author:    Gary Frost <gfrost at openjdk.org>
Date:      2025-03-11 17:57:42 +0000
URL:       https://git.openjdk.org/babylon/commit/05ae51c33e0f6bf9423ed2e88c0fe64494b80366

Hat UI updates and iml fixes

! hat/backends/ffi/cuda/pom.xml
! hat/backends/ffi/mock/pom.xml
! hat/backends/ffi/opencl/CMakeLists.txt
! hat/backends/ffi/opencl/cpp/opencl_backend.cpp
+ hat/backends/ffi/opencl/cpp/opencl_backend_kernel_dispatch.cpp
! hat/backends/ffi/opencl/include/opencl_backend.h
! hat/backends/ffi/opencl/pom.xml
! hat/backends/ffi/opencl/src/main/java/hat/backend/ffi/OpenCLBackend.java
! hat/backends/ffi/pom.xml
! hat/backends/ffi/ptx/pom.xml
! hat/backends/ffi/shared/include/shared.h
! hat/docs/hat-00.md
! hat/docs/hat-01-01-project-layout.md
! hat/docs/hat-01-02-building-babylon.md
! hat/docs/hat-01-03-building-hat-with-maven.md
! hat/docs/hat-01-03-building-hat.md
! hat/docs/hat-01-04-intellij.md
! hat/docs/hat-03-programming-model.md
! hat/docs/hat-04-01-interface-mapping.md
! hat/docs/hat-04-02-cascade-interface-mapping.md
! hat/docs/hat-05-accelerator-compute.md
! hat/docs/hat-06-kernel-analysis.md
+ hat/docs/hat-minimizing-buffer-transfers.md
! hat/docs/hat-notes-and-links.md
! hat/examples/experiments/src/main/java/experiments/MinBufferTest.java
! hat/examples/heal/src/main/java/heal/Viewer.java
! hat/examples/life/src/main/java/life/Main.java
! hat/examples/life/src/main/java/life/Viewer.java
- hat/examples/violajones/src/main/java/violajones/HaarViewer.java
! hat/examples/violajones/src/main/java/violajones/Main.java
+ hat/examples/violajones/src/main/java/violajones/Viewer.java
! hat/examples/violajones/src/main/java/violajones/attic/ViolaJones.java
! hat/hat-core/src/main/java/hat/buffer/Buffer.java
! hat/hat-core/src/main/java/hat/buffer/S32Array.java
! hat/hat-core/src/main/java/hat/ifacemapper/SegmentMapper.java
! hat/hat-core/src/main/java/hat/util/ui/SevenSegmentDisplay.java
! hat/intellij/backend_jextracted_opencl.iml
! hat/intellij/clwrap.iml
! hat/intellij/glwrap.iml
! hat/intellij/nbody.iml
! hat/maven-build.bash


From gfrost at openjdk.org  Tue Mar 11 18:04:39 2025
From: gfrost at openjdk.org (Gary Frost)
Date: Tue, 11 Mar 2025 18:04:39 GMT
Subject: [code-reflection] Integrated: Hat UI updates and iml fixes
Message-ID: <hVPvJgiRzDplVYXUUDCZCj4-0-1Ml5k-UajtW-4P7_g=.d9234ee1-c3b8-46f2-bdf5-c290887f1d4d@github.com>

Lots of small changes. 

Some cleanup of ui's for demos. 

.iml changes (intellij) to align with move of hat -> hat-core

.pom changes (allowing maven to build hat-core, backends and examples) 

Added bit checks in ndrange to avoid buffer moves.

-------------

Commit messages:
 - whitespace
 - Updates to poms and ui's to align with J1 demos.
 - UI fixes and .iml changes

Changes: https://git.openjdk.org/babylon/pull/346/files
  Webrev: https://webrevs.openjdk.org/?repo=babylon&pr=346&range=00
  Stats: 1619 lines in 41 files changed: 998 ins; 523 del; 98 mod
  Patch: https://git.openjdk.org/babylon/pull/346.diff
  Fetch: git fetch https://git.openjdk.org/babylon.git pull/346/head:pull/346

PR: https://git.openjdk.org/babylon/pull/346

From gfrost at openjdk.org  Tue Mar 11 18:04:40 2025
From: gfrost at openjdk.org (Gary Frost)
Date: Tue, 11 Mar 2025 18:04:40 GMT
Subject: [code-reflection] Integrated: Hat UI updates and iml fixes
In-Reply-To: <hVPvJgiRzDplVYXUUDCZCj4-0-1Ml5k-UajtW-4P7_g=.d9234ee1-c3b8-46f2-bdf5-c290887f1d4d@github.com>
References: <hVPvJgiRzDplVYXUUDCZCj4-0-1Ml5k-UajtW-4P7_g=.d9234ee1-c3b8-46f2-bdf5-c290887f1d4d@github.com>
Message-ID: <Fptq2_pZFR4eHgGNSYIT2GVlY1hvBCLuHMhr2vKVOfM=.dcf73f2a-35bd-418d-afc9-f0aefb7df5e1@github.com>

On Tue, 11 Mar 2025 17:55:20 GMT, Gary Frost <gfrost at openjdk.org> wrote:

> Lots of small changes. 
> 
> Some cleanup of ui's for demos. 
> 
> .iml changes (intellij) to align with move of hat -> hat-core
> 
> .pom changes (allowing maven to build hat-core, backends and examples) 
> 
> Added bit checks in ndrange to avoid buffer moves.

This pull request has now been integrated.

Changeset: 05ae51c3
Author:    Gary Frost <gfrost at openjdk.org>
URL:       https://git.openjdk.org/babylon/commit/05ae51c33e0f6bf9423ed2e88c0fe64494b80366
Stats:     1619 lines in 41 files changed: 998 ins; 523 del; 98 mod

Hat UI updates and iml fixes

-------------

PR: https://git.openjdk.org/babylon/pull/346

From asotona at openjdk.org  Wed Mar 12 06:52:27 2025
From: asotona at openjdk.org (Adam Sotona)
Date: Wed, 12 Mar 2025 06:52:27 GMT
Subject: [code-reflection] RFR: Shape auto-dimension calculation from data size
Message-ID: <lgAUaGJjmNSY73uWNaQ5jQ0Fk8GSTPHN9pTgqt9vYVA=.ba483d8b-fdb8-4681-a005-39f675abf500@github.com>

Implemented auto-dimension calculation from data size.

Shaped tensors construction required manual shape calculation based on the actual tensor data.
This patch allows to set one of the shape dimensions to -1 and its size is automatically calculated from the actual tensor data size.

-------------

Commit messages:
 - Shape auto-dimension calculation from data size

Changes: https://git.openjdk.org/babylon/pull/347/files
  Webrev: https://webrevs.openjdk.org/?repo=babylon&pr=347&range=00
  Stats: 32 lines in 2 files changed: 29 ins; 1 del; 2 mod
  Patch: https://git.openjdk.org/babylon/pull/347.diff
  Fetch: git fetch https://git.openjdk.org/babylon.git pull/347/head:pull/347

PR: https://git.openjdk.org/babylon/pull/347

From gfrost at openjdk.org  Wed Mar 12 11:10:40 2025
From: gfrost at openjdk.org (Gary Frost)
Date: Wed, 12 Mar 2025 11:10:40 GMT
Subject: git: openjdk/babylon: code-reflection: Hat performance minimize buffer
 copy tweaking
Message-ID: <14822e8b-3ba8-4f9d-ad27-05f43fc34fe0@openjdk.org>

Changeset: 73274d57
Branch: code-reflection
Author:    Gary Frost <gfrost at openjdk.org>
Date:      2025-03-12 11:08:28 +0000
URL:       https://git.openjdk.org/babylon/commit/73274d571642db29c42e2e4cc97f995b010b8f10

Hat performance minimize buffer copy tweaking

! hat/backends/ffi/opencl/cpp/opencl_backend.cpp
! hat/backends/ffi/opencl/cpp/opencl_backend_kernel_dispatch.cpp
! hat/backends/ffi/opencl/include/opencl_backend.h
+ hat/backends/ffi/opencl/src/main/java/hat/backend/ffi/Config.java
! hat/backends/ffi/opencl/src/main/java/hat/backend/ffi/OpenCLBackend.java
! hat/examples/experiments/src/main/java/experiments/Mesh.java
! hat/examples/experiments/src/main/java/experiments/MinBufferTest.java
! hat/examples/life/src/main/java/life/Viewer.java
! hat/examples/mandel/src/main/java/mandel/Main.java
! hat/examples/mandel/src/main/java/mandel/Viewer.java
! hat/examples/nbody/src/main/java/nbody/opencl/OpenCLNBodyGLWindow.java


From gfrost at openjdk.org  Wed Mar 12 11:11:30 2025
From: gfrost at openjdk.org (Gary Frost)
Date: Wed, 12 Mar 2025 11:11:30 GMT
Subject: [code-reflection] Integrated: Hat performance minimize buffer copy
 tweaking
Message-ID: <cws2QboCgW9J9Lbi5BawDtT4oXCMsdV7ENyJMxJlXOA=.5ed8bdcd-feb2-401c-b9ff-976a0dcaa2d3@github.com>

Buffer marking seems to work

Get perf improvement on game of life (x2.5) mandel (x0.25 ish) and suprisingly on violajones. 

To compare with and without use env vars 


HAT=GPU,MINIMIZE_COPIES java @bldr/hatrun ffi-opencl life

vs 

HAT=GPU java @bldr/hatrun ffi-opencl life

-------------

Commit messages:
 - Buffer marking seems to work.
 - Added seven seg to mandel
 - Added TRACE_ENQUEUES to allow us to show the effect of MINIMIZE_COPIES

Changes: https://git.openjdk.org/babylon/pull/348/files
  Webrev: https://webrevs.openjdk.org/?repo=babylon&pr=348&range=00
  Stats: 653 lines in 11 files changed: 370 ins; 224 del; 59 mod
  Patch: https://git.openjdk.org/babylon/pull/348.diff
  Fetch: git fetch https://git.openjdk.org/babylon.git pull/348/head:pull/348

PR: https://git.openjdk.org/babylon/pull/348

From gfrost at openjdk.org  Wed Mar 12 11:11:31 2025
From: gfrost at openjdk.org (Gary Frost)
Date: Wed, 12 Mar 2025 11:11:31 GMT
Subject: [code-reflection] Integrated: Hat performance minimize buffer copy
 tweaking
In-Reply-To: <cws2QboCgW9J9Lbi5BawDtT4oXCMsdV7ENyJMxJlXOA=.5ed8bdcd-feb2-401c-b9ff-976a0dcaa2d3@github.com>
References: <cws2QboCgW9J9Lbi5BawDtT4oXCMsdV7ENyJMxJlXOA=.5ed8bdcd-feb2-401c-b9ff-976a0dcaa2d3@github.com>
Message-ID: <ftaeweTEEY_IL0bgTk58gNXL7MseJ3jWpf3uJrxS1eI=.203204da-99d9-426f-adb9-c6aeeeef8fc0@github.com>

On Wed, 12 Mar 2025 11:06:09 GMT, Gary Frost <gfrost at openjdk.org> wrote:

> Buffer marking seems to work
> 
> Get perf improvement on game of life (x2.5) mandel (x0.25 ish) and suprisingly on violajones. 
> 
> To compare with and without use env vars 
> 
> 
> HAT=GPU,MINIMIZE_COPIES java @bldr/hatrun ffi-opencl life
> 
> vs 
> 
> HAT=GPU java @bldr/hatrun ffi-opencl life

This pull request has now been integrated.

Changeset: 73274d57
Author:    Gary Frost <gfrost at openjdk.org>
URL:       https://git.openjdk.org/babylon/commit/73274d571642db29c42e2e4cc97f995b010b8f10
Stats:     653 lines in 11 files changed: 370 ins; 224 del; 59 mod

Hat performance minimize buffer copy tweaking

-------------

PR: https://git.openjdk.org/babylon/pull/348

From gfrost at openjdk.org  Wed Mar 12 14:08:12 2025
From: gfrost at openjdk.org (Gary Frost)
Date: Wed, 12 Mar 2025 14:08:12 GMT
Subject: git: openjdk/babylon: code-reflection: fixup hatrun for life
 dependencies, allow nbody to default to HAT
Message-ID: <8f8302cc-70c3-4a43-a785-50f6269a91c6@openjdk.org>

Changeset: 001f3573
Branch: code-reflection
Author:    Gary Frost <gfrost at openjdk.org>
Date:      2025-03-12 14:07:05 +0000
URL:       https://git.openjdk.org/babylon/commit/001f3573afb3ce423a78cf108f51175b808f0a1e

fixup hatrun for life dependencies, allow nbody to default to HAT

! hat/examples/nbody/src/main/java/nbody/Main.java
! hat/hatrun


From gfrost at openjdk.org  Wed Mar 12 14:09:58 2025
From: gfrost at openjdk.org (Gary Frost)
Date: Wed, 12 Mar 2025 14:09:58 GMT
Subject: [code-reflection] Integrated: fixup hatrun for life dependencies,
 allow nbody to default to HAT
Message-ID: <iJK7hnTRAzwrJB0itZX3DJ9xTY0eI0aJqwyC10aqL4s=.6d2aeaab-2711-4021-bf22-b5078a73d9ab@github.com>

Switched nbody and life examples to use HAT (instead of jextracted opencl) for J1

Also fixed up hatrun if we specified `java-mt`.   It still needed OpenCL wrapper.

-------------

Commit messages:
 - fixup hatrun for life dependencies, allow nbody to default to HAT

Changes: https://git.openjdk.org/babylon/pull/349/files
  Webrev: https://webrevs.openjdk.org/?repo=babylon&pr=349&range=00
  Stats: 5 lines in 2 files changed: 4 ins; 0 del; 1 mod
  Patch: https://git.openjdk.org/babylon/pull/349.diff
  Fetch: git fetch https://git.openjdk.org/babylon.git pull/349/head:pull/349

PR: https://git.openjdk.org/babylon/pull/349

From gfrost at openjdk.org  Wed Mar 12 14:09:58 2025
From: gfrost at openjdk.org (Gary Frost)
Date: Wed, 12 Mar 2025 14:09:58 GMT
Subject: [code-reflection] Integrated: fixup hatrun for life dependencies,
 allow nbody to default to HAT
In-Reply-To: <iJK7hnTRAzwrJB0itZX3DJ9xTY0eI0aJqwyC10aqL4s=.6d2aeaab-2711-4021-bf22-b5078a73d9ab@github.com>
References: <iJK7hnTRAzwrJB0itZX3DJ9xTY0eI0aJqwyC10aqL4s=.6d2aeaab-2711-4021-bf22-b5078a73d9ab@github.com>
Message-ID: <bXHQWpP6OIgvwqkhAu42IJ6U2A0kKobsMPCODVTsT4c=.dc55d779-f273-4a7a-a354-a5a9f7175f14@github.com>

On Wed, 12 Mar 2025 14:04:25 GMT, Gary Frost <gfrost at openjdk.org> wrote:

> Switched nbody and life examples to use HAT (instead of jextracted opencl) for J1
> 
> Also fixed up hatrun if we specified `java-mt`.   It still needed OpenCL wrapper.

This pull request has now been integrated.

Changeset: 001f3573
Author:    Gary Frost <gfrost at openjdk.org>
URL:       https://git.openjdk.org/babylon/commit/001f3573afb3ce423a78cf108f51175b808f0a1e
Stats:     5 lines in 2 files changed: 4 ins; 0 del; 1 mod

fixup hatrun for life dependencies, allow nbody to default to HAT

-------------

PR: https://git.openjdk.org/babylon/pull/349

From gfrost at openjdk.org  Wed Mar 12 16:21:13 2025
From: gfrost at openjdk.org (Gary Frost)
Date: Wed, 12 Mar 2025 16:21:13 GMT
Subject: git: openjdk/babylon: code-reflection: Removed annotation references
 ValueBased and Stable and cleaned bld =?UTF-8?B?YeKApg==?=
Message-ID: <9af2c02c-a1a8-40a3-bdde-18b3fd243126@openjdk.org>

Changeset: bc191fec
Branch: code-reflection
Author:    Gary Frost <gfrost at openjdk.org>
Date:      2025-03-12 16:19:58 +0000
URL:       https://git.openjdk.org/babylon/commit/bc191fecebce29df7762f28b9b2b81c054ff6277

Removed annotation references ValueBased and Stable and cleaned bld a?

! hat/backends/ffi/opencl/cpp/info.cpp
! hat/backends/ffi/opencl/cpp/opencl_backend.cpp
! hat/backends/ffi/opencl/include/opencl_backend.h
! hat/backends/ffi/opencl/src/main/java/hat/backend/ffi/Config.java
! hat/backends/ffi/opencl/src/main/java/hat/backend/ffi/OpenCLBackend.java
! hat/bld
+ hat/bldr/tst
! hat/hat-core/src/main/java/hat/ifacemapper/AbstractSegmentMapper.java
! hat/hat-core/src/main/java/hat/ifacemapper/ByteCodeGenerator.java
! hat/hat-core/src/main/java/hat/ifacemapper/MapperCache.java
! hat/hat-core/src/main/java/hat/ifacemapper/SegmentInterfaceMapper.java
! hat/hat-core/src/main/java/hat/ifacemapper/accessor/Accessors.java
! hat/hatrun
! hat/pom.xml
+ hat/tst/MinBufferTest.java


From gfrost at openjdk.org  Wed Mar 12 16:23:38 2025
From: gfrost at openjdk.org (Gary Frost)
Date: Wed, 12 Mar 2025 16:23:38 GMT
Subject: [code-reflection] Integrated: Removed annotation references ValueBased
 and Stable and cleaned bld =?UTF-8?B?YeKApg==?=
Message-ID: <SwXiJImW2H8ovaz0Dq7KG2y9X-yGMp4-7FQ5JJdDLws=.ff4255a5-1661-4f42-be2d-79f0abaa2e6f@github.com>

Chasing an issue with runnig scripts Paul suggested we can skip teh @Stable and @ValueBased annotations which will allow us to simplify the build and run scripts.  

This PR has the changes to support this.  

Also simplified the passing of configBits to ffi-backend.

-------------

Commit messages:
 - Removed annotation references ValueBased and Stable and cleaned bld and run scrips

Changes: https://git.openjdk.org/babylon/pull/350/files
  Webrev: https://webrevs.openjdk.org/?repo=babylon&pr=350&range=00
  Stats: 205 lines in 15 files changed: 121 ins; 0 del; 84 mod
  Patch: https://git.openjdk.org/babylon/pull/350.diff
  Fetch: git fetch https://git.openjdk.org/babylon.git pull/350/head:pull/350

PR: https://git.openjdk.org/babylon/pull/350

From gfrost at openjdk.org  Wed Mar 12 16:23:38 2025
From: gfrost at openjdk.org (Gary Frost)
Date: Wed, 12 Mar 2025 16:23:38 GMT
Subject: [code-reflection] Integrated: Removed annotation references ValueBased
 and Stable and cleaned bld =?UTF-8?B?YeKApg==?=
In-Reply-To: <SwXiJImW2H8ovaz0Dq7KG2y9X-yGMp4-7FQ5JJdDLws=.ff4255a5-1661-4f42-be2d-79f0abaa2e6f@github.com>
References: <SwXiJImW2H8ovaz0Dq7KG2y9X-yGMp4-7FQ5JJdDLws=.ff4255a5-1661-4f42-be2d-79f0abaa2e6f@github.com>
Message-ID: <yBiWEkpNsedn8SmoL2_evexMocH9NBz0e-CbCc-YwOM=.70e10c32-227c-4e86-9e33-96358b9956b4@github.com>

On Wed, 12 Mar 2025 16:17:44 GMT, Gary Frost <gfrost at openjdk.org> wrote:

> Chasing an issue with runnig scripts Paul suggested we can skip teh @Stable and @ValueBased annotations which will allow us to simplify the build and run scripts.  
> 
> This PR has the changes to support this.  
> 
> Also simplified the passing of configBits to ffi-backend.

This pull request has now been integrated.

Changeset: bc191fec
Author:    Gary Frost <gfrost at openjdk.org>
URL:       https://git.openjdk.org/babylon/commit/bc191fecebce29df7762f28b9b2b81c054ff6277
Stats:     205 lines in 15 files changed: 121 ins; 0 del; 84 mod

Removed annotation references ValueBased and Stable and cleaned bld a?

-------------

PR: https://git.openjdk.org/babylon/pull/350

From mabbay at openjdk.org  Wed Mar 12 17:49:43 2025
From: mabbay at openjdk.org (Mourad Abbay)
Date: Wed, 12 Mar 2025 17:49:43 GMT
Subject: [code-reflection] RFR: Represents literals of type byte/short as
 permitted in the Java language
Message-ID: <zbICBVLiPQJAIloGr_22qhNGXZA--UdjKLeoCH_bmBU=.b9a87fd0-5918-4ef7-93d2-b2103a6427ea@github.com>

Before this PR, we had literals of type byte/short in the code model, but that's not permitted by the Java language. In Java, a literals of type byte/short is a literal of type int + conversion to the type byte/short.

-------------

Commit messages:
 - Represents literals of type byte/short as permitted in the Java language.

Changes: https://git.openjdk.org/babylon/pull/351/files
  Webrev: https://webrevs.openjdk.org/?repo=babylon&pr=351&range=00
  Stats: 9 lines in 1 file changed: 1 ins; 4 del; 4 mod
  Patch: https://git.openjdk.org/babylon/pull/351.diff
  Fetch: git fetch https://git.openjdk.org/babylon.git pull/351/head:pull/351

PR: https://git.openjdk.org/babylon/pull/351

From mabbay at openjdk.org  Wed Mar 12 17:58:59 2025
From: mabbay at openjdk.org (Mourad Abbay)
Date: Wed, 12 Mar 2025 17:58:59 GMT
Subject: [code-reflection] RFR: Represents literals of type byte/short as
 permitted in the Java language [v2]
In-Reply-To: <zbICBVLiPQJAIloGr_22qhNGXZA--UdjKLeoCH_bmBU=.b9a87fd0-5918-4ef7-93d2-b2103a6427ea@github.com>
References: <zbICBVLiPQJAIloGr_22qhNGXZA--UdjKLeoCH_bmBU=.b9a87fd0-5918-4ef7-93d2-b2103a6427ea@github.com>
Message-ID: <K78qZW9TaOK4qOwPZaVFn9fSqEIY3EfmZgKZdRiLyoo=.d64aff9a-3d4b-4073-a53c-731b59eceac4@github.com>

> Before this PR, we had literals of type byte/short in the code model, but that's not permitted by the Java language. In Java, a literals of type byte/short is a literal of type int + conversion to the type byte/short.

Mourad Abbay has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains two additional commits since the last revision:

 - Merge branch 'code-reflection' into fix-literal-type
 - Represents literals of type byte/short as permitted in the Java language.

-------------

Changes:
  - all: https://git.openjdk.org/babylon/pull/351/files
  - new: https://git.openjdk.org/babylon/pull/351/files/ec41ee8d..41cb5acf

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=babylon&pr=351&range=01
 - incr: https://webrevs.openjdk.org/?repo=babylon&pr=351&range=00-01

  Stats: 10673 lines in 269 files changed: 4831 ins; 5080 del; 762 mod
  Patch: https://git.openjdk.org/babylon/pull/351.diff
  Fetch: git fetch https://git.openjdk.org/babylon.git pull/351/head:pull/351

PR: https://git.openjdk.org/babylon/pull/351

From mabbay at openjdk.org  Wed Mar 12 18:00:05 2025
From: mabbay at openjdk.org (Mourad Abbay)
Date: Wed, 12 Mar 2025 18:00:05 GMT
Subject: [code-reflection] RFR: Support storing the code that builds the
 code model [v18]
In-Reply-To: <dexv8sQV_NcJsJIgMopd_M4wSWoZya9dU0ACwDtEL0E=.4ac0b80f-24fb-440f-a18d-bbfd07a681bc@github.com>
References: <dexv8sQV_NcJsJIgMopd_M4wSWoZya9dU0ACwDtEL0E=.4ac0b80f-24fb-440f-a18d-bbfd07a681bc@github.com>
Message-ID: <rLtW9Wv92MNgUIywKaiPdoBs8O5-5EXNlu17J8tRflk=.3e0aa614-55db-4e86-a5a0-99e8e51758d8@github.com>

> In this PR we allow the user to decide how to store the code model. The first option is `TEXT`, if this is selected, we store the textual representation of the code model. The second option is `CODE_BUILDR`, if this is selected, we store the code that builds the code model. All work done here is around the second option, because the first option is already supported.

Mourad Abbay has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains 20 additional commits since the last revision:

 - Merge branch 'code-reflection' into code-model-storage-option
 - Load OpFactory and TypeElementFactory before invocation of opMethod only if the opMethod has there params.
 - Fix the remaining compiler tests failures
 - Fix some of the test failures (3 remains)
 - Fix the remaining test failures of SwitchExpressionTest2
 - Fix almost all test failures of SwitchExpressionTest2 (one remaining)
 - Pass arrayType instead of eleType in OpBuilder.buildArray
 - Ensure that block params are inserted in the correct order
 - Add missing imports
 - Merge branch 'code-reflection' into code-model-storage-option
 - ... and 10 more: https://git.openjdk.org/babylon/compare/ddc18534...966005ce

-------------

Changes:
  - all: https://git.openjdk.org/babylon/pull/305/files
  - new: https://git.openjdk.org/babylon/pull/305/files/0e49cceb..966005ce

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=babylon&pr=305&range=17
 - incr: https://webrevs.openjdk.org/?repo=babylon&pr=305&range=16-17

  Stats: 24999 lines in 272 files changed: 18963 ins; 5180 del; 856 mod
  Patch: https://git.openjdk.org/babylon/pull/305.diff
  Fetch: git fetch https://git.openjdk.org/babylon.git pull/305/head:pull/305

PR: https://git.openjdk.org/babylon/pull/305

From mabbay at openjdk.org  Wed Mar 12 18:00:06 2025
From: mabbay at openjdk.org (Mourad Abbay)
Date: Wed, 12 Mar 2025 18:00:06 GMT
Subject: [code-reflection] RFR: Support storing the code that builds the
 code model [v17]
In-Reply-To: <zKGCPAn3OsUu7bDP2cRAvYFeH4AMT1QPTyFssveYE7A=.956f33f9-80d7-4ab1-ad2b-d952a5f94bca@github.com>
References: <dexv8sQV_NcJsJIgMopd_M4wSWoZya9dU0ACwDtEL0E=.4ac0b80f-24fb-440f-a18d-bbfd07a681bc@github.com>
 <zKGCPAn3OsUu7bDP2cRAvYFeH4AMT1QPTyFssveYE7A=.956f33f9-80d7-4ab1-ad2b-d952a5f94bca@github.com>
Message-ID: <Q_9-oGU4GmF4GJdvjXcmXPPUaG29impaLEi5_HaS76A=.73170342-2145-4fae-9712-8326db79452b@github.com>

On Thu, 6 Mar 2025 15:49:32 GMT, Mourad Abbay <mabbay at openjdk.org> wrote:

>> In this PR we allow the user to decide how to store the code model. The first option is `TEXT`, if this is selected, we store the textual representation of the code model. The second option is `CODE_BUILDR`, if this is selected, we store the code that builds the code model. All work done here is around the second option, because the first option is already supported.
>
> Mourad Abbay has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Load OpFactory and TypeElementFactory before invocation of opMethod only if the opMethod has there params.

#351 need to be merged before this PR

-------------

PR Comment: https://git.openjdk.org/babylon/pull/305#issuecomment-2718679832

From gfrost at openjdk.org  Wed Mar 12 18:17:59 2025
From: gfrost at openjdk.org (Gary Frost)
Date: Wed, 12 Mar 2025 18:17:59 GMT
Subject: git: openjdk/babylon: code-reflection: Removed unneeded
 MethodHandles.lookup()
Message-ID: <4eb5a832-a262-449e-b350-cb77f70f7755@openjdk.org>

Changeset: db60cab9
Branch: code-reflection
Author:    Gary Frost <gfrost at openjdk.org>
Date:      2025-03-12 18:17:26 +0000
URL:       https://git.openjdk.org/babylon/commit/db60cab9b9f6a58796a0c09c625f73a41e7d45c3

Removed unneeded MethodHandles.lookup()

! hat/backends/ffi/cuda/src/main/java/hat/backend/ffi/CudaHatKernelBuilder.java
! hat/backends/ffi/opencl/src/main/java/hat/backend/ffi/OpenCLHatKernelBuilder.java
! hat/backends/ffi/ptx/src/main/java/hat/backend/ffi/PTXBackend.java
! hat/backends/jextracted/opencl/src/main/java/hat/backend/jextracted/OpenCLHatKernelBuilder.java
! hat/examples/experiments/src/main/java/experiments/InvokeToPtr.java
! hat/examples/experiments/src/main/java/experiments/LayoutExample.java
! hat/examples/experiments/src/main/java/experiments/PointyHatArray.java
! hat/examples/experiments/src/main/java/experiments/PrePostInc.java
! hat/examples/experiments/src/main/java/experiments/QuotedTest.java
! hat/examples/experiments/src/main/java/experiments/RawLayout.java
! hat/examples/squares/src/main/java/squares/Main.java
! hat/hat-core/src/main/java/hat/Accelerator.java
! hat/hat-core/src/main/java/hat/ComputeContext.java
! hat/hat-core/src/main/java/hat/OpsAndTypes.java
! hat/hat-core/src/main/java/hat/backend/DebugBackend.java
! hat/hat-core/src/main/java/hat/backend/codebuilders/C99HATComputeBuilder.java
! hat/hat-core/src/main/java/hat/backend/codebuilders/C99HATKernelBuilder.java
! hat/hat-core/src/main/java/hat/backend/codebuilders/HATCodeBuilder.java
! hat/hat-core/src/main/java/hat/backend/codebuilders/HATCodeBuilderWithContext.java
! hat/hat-core/src/main/java/hat/backend/ffi/FFIBackend.java
! hat/hat-core/src/main/java/hat/backend/jextracted/JExtractedBackend.java
! hat/hat-core/src/main/java/hat/callgraph/ComputeCallGraph.java
! hat/hat-core/src/main/java/hat/callgraph/KernelCallGraph.java
! hat/hat-core/src/main/java/hat/optools/BinaryArithmeticOrLogicOperation.java
! hat/hat-core/src/main/java/hat/optools/BinaryLogicalOpWrapper.java
! hat/hat-core/src/main/java/hat/optools/BinaryOpWrapper.java
! hat/hat-core/src/main/java/hat/optools/BinaryTestOpWrapper.java
! hat/hat-core/src/main/java/hat/optools/ConstantOpWrapper.java
! hat/hat-core/src/main/java/hat/optools/ConvOpWrapper.java
! hat/hat-core/src/main/java/hat/optools/FieldAccessOpWrapper.java
! hat/hat-core/src/main/java/hat/optools/FieldLoadOpWrapper.java
! hat/hat-core/src/main/java/hat/optools/FieldStoreOpWrapper.java
! hat/hat-core/src/main/java/hat/optools/ForOpWrapper.java
! hat/hat-core/src/main/java/hat/optools/FuncCallOpWrapper.java
! hat/hat-core/src/main/java/hat/optools/FuncOpWrapper.java
! hat/hat-core/src/main/java/hat/optools/IfOpWrapper.java
! hat/hat-core/src/main/java/hat/optools/InvokeOpWrapper.java
! hat/hat-core/src/main/java/hat/optools/JavaBreakOpWrapper.java
! hat/hat-core/src/main/java/hat/optools/JavaContinueOpWrapper.java
! hat/hat-core/src/main/java/hat/optools/JavaLabeledOpWrapper.java
! hat/hat-core/src/main/java/hat/optools/LambdaOpWrapper.java
! hat/hat-core/src/main/java/hat/optools/LogicalOpWrapper.java
! hat/hat-core/src/main/java/hat/optools/LoopOpWrapper.java
! hat/hat-core/src/main/java/hat/optools/ModuleOpWrapper.java
! hat/hat-core/src/main/java/hat/optools/OpWrapper.java
! hat/hat-core/src/main/java/hat/optools/ReturnOpWrapper.java
! hat/hat-core/src/main/java/hat/optools/StructuralOpWrapper.java
! hat/hat-core/src/main/java/hat/optools/TernaryOpWrapper.java
! hat/hat-core/src/main/java/hat/optools/TupleOpWrapper.java
! hat/hat-core/src/main/java/hat/optools/UnaryArithmeticOrLogicOpWrapper.java
! hat/hat-core/src/main/java/hat/optools/UnaryOpWrapper.java
! hat/hat-core/src/main/java/hat/optools/VarAccessOpWrapper.java
! hat/hat-core/src/main/java/hat/optools/VarDeclarationOpWrapper.java
! hat/hat-core/src/main/java/hat/optools/VarFuncDeclarationOpWrapper.java
! hat/hat-core/src/main/java/hat/optools/VarLoadOpWrapper.java
! hat/hat-core/src/main/java/hat/optools/VarOpWrapper.java
! hat/hat-core/src/main/java/hat/optools/VarStoreOpWrapper.java
! hat/hat-core/src/main/java/hat/optools/WhileOpWrapper.java
! hat/hat-core/src/main/java/hat/optools/YieldOpWrapper.java
! hat/hat-core/src/main/test/hat/SquaresTest.java


From gfrost at openjdk.org  Wed Mar 12 18:20:19 2025
From: gfrost at openjdk.org (Gary Frost)
Date: Wed, 12 Mar 2025 18:20:19 GMT
Subject: [code-reflection] Integrated: Removed unneeded MethodHandles.lookup()
Message-ID: <fZRVWIh0BjGXb3bTBAUxmHte0gEng2CWNaMENKAkZWg=.387db8a2-f882-4dfa-b4db-43e245cd0b73@github.com>

Exception  running script indicated mayble issues usig local MethodHandles.lookup() calls. 

Purged the code to reuse the accelerator lookup (from the app). 

Lots of code changes. ;)

-------------

Commit messages:
 - Removed unneeded MethodHandles.lookup()

Changes: https://git.openjdk.org/babylon/pull/352/files
  Webrev: https://webrevs.openjdk.org/?repo=babylon&pr=352&range=00
  Stats: 308 lines in 60 files changed: 91 ins; 1 del; 216 mod
  Patch: https://git.openjdk.org/babylon/pull/352.diff
  Fetch: git fetch https://git.openjdk.org/babylon.git pull/352/head:pull/352

PR: https://git.openjdk.org/babylon/pull/352

From mcimadamore at openjdk.org  Wed Mar 12 18:20:14 2025
From: mcimadamore at openjdk.org (Maurizio Cimadamore)
Date: Wed, 12 Mar 2025 18:20:14 GMT
Subject: [code-reflection] RFR: Represents literals of type byte/short as
 permitted in the Java language [v2]
In-Reply-To: <K78qZW9TaOK4qOwPZaVFn9fSqEIY3EfmZgKZdRiLyoo=.d64aff9a-3d4b-4073-a53c-731b59eceac4@github.com>
References: <zbICBVLiPQJAIloGr_22qhNGXZA--UdjKLeoCH_bmBU=.b9a87fd0-5918-4ef7-93d2-b2103a6427ea@github.com>
 <K78qZW9TaOK4qOwPZaVFn9fSqEIY3EfmZgKZdRiLyoo=.d64aff9a-3d4b-4073-a53c-731b59eceac4@github.com>
Message-ID: <qNoBRk1NCyfF3BQuhsI_JizAMcXwQUH6hyt4G_6zbdk=.9d5a4aab-4358-43f8-b8f1-66c1e7d04cd7@github.com>

On Wed, 12 Mar 2025 17:58:59 GMT, Mourad Abbay <mabbay at openjdk.org> wrote:

>> Before this PR, we had literals of type byte/short in the code model, but that's not permitted by the Java language. In Java, a literals of type byte/short is a literal of type int + conversion to the type byte/short.
>
> Mourad Abbay has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains two additional commits since the last revision:
> 
>  - Merge branch 'code-reflection' into fix-literal-type
>  - Represents literals of type byte/short as permitted in the Java language.

src/jdk.incubator.code/share/classes/jdk/incubator/code/internal/ReflectMethods.java line 2332:

> 2330:                     // Capture applying rhs and operation
> 2331:                     Function<Value, Value> scanRhs = (lhs) -> {
> 2332:                         Value one = convert(append(numericOneValue(tree.type)), types.unboxedType(tree.type));

I don't get the call to `types.uboxedType` -- `tree.type` is passed to `numericOneValue` which seems to assume that the type is a primitive type, right?

-------------

PR Review Comment: https://git.openjdk.org/babylon/pull/351#discussion_r1992055213

From mcimadamore at openjdk.org  Wed Mar 12 18:20:14 2025
From: mcimadamore at openjdk.org (Maurizio Cimadamore)
Date: Wed, 12 Mar 2025 18:20:14 GMT
Subject: [code-reflection] RFR: Represents literals of type byte/short as
 permitted in the Java language [v2]
In-Reply-To: <qNoBRk1NCyfF3BQuhsI_JizAMcXwQUH6hyt4G_6zbdk=.9d5a4aab-4358-43f8-b8f1-66c1e7d04cd7@github.com>
References: <zbICBVLiPQJAIloGr_22qhNGXZA--UdjKLeoCH_bmBU=.b9a87fd0-5918-4ef7-93d2-b2103a6427ea@github.com>
 <K78qZW9TaOK4qOwPZaVFn9fSqEIY3EfmZgKZdRiLyoo=.d64aff9a-3d4b-4073-a53c-731b59eceac4@github.com>
 <qNoBRk1NCyfF3BQuhsI_JizAMcXwQUH6hyt4G_6zbdk=.9d5a4aab-4358-43f8-b8f1-66c1e7d04cd7@github.com>
Message-ID: <544IY_GmFyTb3NrhziDihxplxCVKlPhejYHhVrUBd0U=.8224766a-dfb6-4cb3-8c7f-f953c4ab1b91@github.com>

On Wed, 12 Mar 2025 18:15:37 GMT, Maurizio Cimadamore <mcimadamore at openjdk.org> wrote:

>> Mourad Abbay has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains two additional commits since the last revision:
>> 
>>  - Merge branch 'code-reflection' into fix-literal-type
>>  - Represents literals of type byte/short as permitted in the Java language.
>
> src/jdk.incubator.code/share/classes/jdk/incubator/code/internal/ReflectMethods.java line 2332:
> 
>> 2330:                     // Capture applying rhs and operation
>> 2331:                     Function<Value, Value> scanRhs = (lhs) -> {
>> 2332:                         Value one = convert(append(numericOneValue(tree.type)), types.unboxedType(tree.type));
> 
> I don't get the call to `types.uboxedType` -- `tree.type` is passed to `numericOneValue` which seems to assume that the type is a primitive type, right?

Ah ok - I see:

                case CLASS -> numericOneValue(types.unboxedType(t));


So, `numericOneValue` also does unboxing. I think this code would be clearer by doing the unboxing first -- and then pass the unboxed type to both `numericOneValue` and `convert` (and then, we could remove the unboxing path from `numericOneValue` - and make that consistent with `defaultValue`).

-------------

PR Review Comment: https://git.openjdk.org/babylon/pull/351#discussion_r1992057794

From gfrost at openjdk.org  Wed Mar 12 18:20:19 2025
From: gfrost at openjdk.org (Gary Frost)
Date: Wed, 12 Mar 2025 18:20:19 GMT
Subject: [code-reflection] Integrated: Removed unneeded MethodHandles.lookup()
In-Reply-To: <fZRVWIh0BjGXb3bTBAUxmHte0gEng2CWNaMENKAkZWg=.387db8a2-f882-4dfa-b4db-43e245cd0b73@github.com>
References: <fZRVWIh0BjGXb3bTBAUxmHte0gEng2CWNaMENKAkZWg=.387db8a2-f882-4dfa-b4db-43e245cd0b73@github.com>
Message-ID: <NKe8nutznqieJ-TK6d6DLnX4MZ58YuiFTlFRIAlQg3Y=.bd010870-3d72-4a18-9ba1-255079e58ee0@github.com>

On Wed, 12 Mar 2025 18:16:03 GMT, Gary Frost <gfrost at openjdk.org> wrote:

> Exception  running script indicated mayble issues usig local MethodHandles.lookup() calls. 
> 
> Purged the code to reuse the accelerator lookup (from the app). 
> 
> Lots of code changes. ;)

This pull request has now been integrated.

Changeset: db60cab9
Author:    Gary Frost <gfrost at openjdk.org>
URL:       https://git.openjdk.org/babylon/commit/db60cab9b9f6a58796a0c09c625f73a41e7d45c3
Stats:     308 lines in 60 files changed: 91 ins; 1 del; 216 mod

Removed unneeded MethodHandles.lookup()

-------------

PR: https://git.openjdk.org/babylon/pull/352

From mcimadamore at openjdk.org  Wed Mar 12 18:24:11 2025
From: mcimadamore at openjdk.org (Maurizio Cimadamore)
Date: Wed, 12 Mar 2025 18:24:11 GMT
Subject: [code-reflection] RFR: Represents literals of type byte/short as
 permitted in the Java language [v2]
In-Reply-To: <K78qZW9TaOK4qOwPZaVFn9fSqEIY3EfmZgKZdRiLyoo=.d64aff9a-3d4b-4073-a53c-731b59eceac4@github.com>
References: <zbICBVLiPQJAIloGr_22qhNGXZA--UdjKLeoCH_bmBU=.b9a87fd0-5918-4ef7-93d2-b2103a6427ea@github.com>
 <K78qZW9TaOK4qOwPZaVFn9fSqEIY3EfmZgKZdRiLyoo=.d64aff9a-3d4b-4073-a53c-731b59eceac4@github.com>
Message-ID: <hfBIclD05wJpbOd6uyTB-OBNLBg9ptKAAp5__LPDzec=.aa294ec8-2e38-455b-9639-e9f5b3e013c6@github.com>

On Wed, 12 Mar 2025 17:58:59 GMT, Mourad Abbay <mabbay at openjdk.org> wrote:

>> Before this PR, we had literals of type byte/short in the code model, but that's not permitted by the Java language. In Java, a literals of type byte/short is a literal of type int + conversion to the type byte/short.
>
> Mourad Abbay has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains two additional commits since the last revision:
> 
>  - Merge branch 'code-reflection' into fix-literal-type
>  - Represents literals of type byte/short as permitted in the Java language.

Changes look good - I left some comments to improve the code clarity. I think this PR also needs a test of the model that comes out of javac.

-------------

PR Review: https://git.openjdk.org/babylon/pull/351#pullrequestreview-2679481548

From mcimadamore at openjdk.org  Wed Mar 12 18:24:11 2025
From: mcimadamore at openjdk.org (Maurizio Cimadamore)
Date: Wed, 12 Mar 2025 18:24:11 GMT
Subject: [code-reflection] RFR: Represents literals of type byte/short as
 permitted in the Java language [v2]
In-Reply-To: <544IY_GmFyTb3NrhziDihxplxCVKlPhejYHhVrUBd0U=.8224766a-dfb6-4cb3-8c7f-f953c4ab1b91@github.com>
References: <zbICBVLiPQJAIloGr_22qhNGXZA--UdjKLeoCH_bmBU=.b9a87fd0-5918-4ef7-93d2-b2103a6427ea@github.com>
 <K78qZW9TaOK4qOwPZaVFn9fSqEIY3EfmZgKZdRiLyoo=.d64aff9a-3d4b-4073-a53c-731b59eceac4@github.com>
 <qNoBRk1NCyfF3BQuhsI_JizAMcXwQUH6hyt4G_6zbdk=.9d5a4aab-4358-43f8-b8f1-66c1e7d04cd7@github.com>
 <544IY_GmFyTb3NrhziDihxplxCVKlPhejYHhVrUBd0U=.8224766a-dfb6-4cb3-8c7f-f953c4ab1b91@github.com>
Message-ID: <7tl7L9EbdrcijdFbQpFXVtF22jlqy0lqTnVkN4btVec=.e5fdf384-1694-40c6-a3d5-6eca37b401c5@github.com>

On Wed, 12 Mar 2025 18:17:04 GMT, Maurizio Cimadamore <mcimadamore at openjdk.org> wrote:

> and make that consistent with `defaultValue`).

Strike that -- `defaultValue` also allows `nulls`. Anyway -- `numericOneValue` should only work on primitives, I think.

-------------

PR Review Comment: https://git.openjdk.org/babylon/pull/351#discussion_r1992064231

From mabbay at openjdk.org  Wed Mar 12 18:38:30 2025
From: mabbay at openjdk.org (Mourad Abbay)
Date: Wed, 12 Mar 2025 18:38:30 GMT
Subject: [code-reflection] RFR: Represents literals of type byte/short as
 permitted in the Java language [v3]
In-Reply-To: <zbICBVLiPQJAIloGr_22qhNGXZA--UdjKLeoCH_bmBU=.b9a87fd0-5918-4ef7-93d2-b2103a6427ea@github.com>
References: <zbICBVLiPQJAIloGr_22qhNGXZA--UdjKLeoCH_bmBU=.b9a87fd0-5918-4ef7-93d2-b2103a6427ea@github.com>
Message-ID: <kyYj0qq2990G3zeFIBgxqEjxqd-ujBImxdfUEHpYxnk=.74adcda0-25f6-4999-9c05-2b4f58faf38a@github.com>

> Before this PR, we had literals of type byte/short in the code model, but that's not permitted by the Java language. In Java, a literals of type byte/short is a literal of type int + conversion to the type byte/short.

Mourad Abbay has updated the pull request incrementally with one additional commit since the last revision:

  Pass unboxed type to numericOneValue

-------------

Changes:
  - all: https://git.openjdk.org/babylon/pull/351/files
  - new: https://git.openjdk.org/babylon/pull/351/files/41cb5acf..c9326ccb

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=babylon&pr=351&range=02
 - incr: https://webrevs.openjdk.org/?repo=babylon&pr=351&range=01-02

  Stats: 3 lines in 1 file changed: 1 ins; 1 del; 1 mod
  Patch: https://git.openjdk.org/babylon/pull/351.diff
  Fetch: git fetch https://git.openjdk.org/babylon.git pull/351/head:pull/351

PR: https://git.openjdk.org/babylon/pull/351

From mabbay at openjdk.org  Wed Mar 12 23:08:37 2025
From: mabbay at openjdk.org (Mourad Abbay)
Date: Wed, 12 Mar 2025 23:08:37 GMT
Subject: [code-reflection] RFR: Represents literals of type byte/short as
 permitted in the Java language [v4]
In-Reply-To: <zbICBVLiPQJAIloGr_22qhNGXZA--UdjKLeoCH_bmBU=.b9a87fd0-5918-4ef7-93d2-b2103a6427ea@github.com>
References: <zbICBVLiPQJAIloGr_22qhNGXZA--UdjKLeoCH_bmBU=.b9a87fd0-5918-4ef7-93d2-b2103a6427ea@github.com>
Message-ID: <3QCW0HinX6I9ASKSm7pnY978Ovf8TUePZz0-7mY5Ik0=.393b4b44-3382-41f6-8550-48ad7698c57a@github.com>

> Before this PR, we had literals of type byte/short in the code model, but that's not permitted by the Java language. In Java, a literals of type byte/short is a literal of type int + conversion to the type byte/short.

Mourad Abbay has updated the pull request incrementally with one additional commit since the last revision:

  Add tests to make sure literals of type byte/short are created the way Java language permit.

-------------

Changes:
  - all: https://git.openjdk.org/babylon/pull/351/files
  - new: https://git.openjdk.org/babylon/pull/351/files/c9326ccb..c6614d18

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=babylon&pr=351&range=03
 - incr: https://webrevs.openjdk.org/?repo=babylon&pr=351&range=02-03

  Stats: 116 lines in 2 files changed: 116 ins; 0 del; 0 mod
  Patch: https://git.openjdk.org/babylon/pull/351.diff
  Fetch: git fetch https://git.openjdk.org/babylon.git pull/351/head:pull/351

PR: https://git.openjdk.org/babylon/pull/351

From mabbay at openjdk.org  Wed Mar 12 23:14:02 2025
From: mabbay at openjdk.org (Mourad Abbay)
Date: Wed, 12 Mar 2025 23:14:02 GMT
Subject: [code-reflection] RFR: Represents literals of type byte/short as
 permitted in the Java language [v2]
In-Reply-To: <hfBIclD05wJpbOd6uyTB-OBNLBg9ptKAAp5__LPDzec=.aa294ec8-2e38-455b-9639-e9f5b3e013c6@github.com>
References: <zbICBVLiPQJAIloGr_22qhNGXZA--UdjKLeoCH_bmBU=.b9a87fd0-5918-4ef7-93d2-b2103a6427ea@github.com>
 <K78qZW9TaOK4qOwPZaVFn9fSqEIY3EfmZgKZdRiLyoo=.d64aff9a-3d4b-4073-a53c-731b59eceac4@github.com>
 <hfBIclD05wJpbOd6uyTB-OBNLBg9ptKAAp5__LPDzec=.aa294ec8-2e38-455b-9639-e9f5b3e013c6@github.com>
Message-ID: <0qOgjmClA0AfgV9ZEOo5Hrre7SQ0oyzgSMQKne1LiGo=.0c9a763c-a905-47f4-b578-4575688594eb@github.com>

On Wed, 12 Mar 2025 18:21:19 GMT, Maurizio Cimadamore <mcimadamore at openjdk.org> wrote:

>> Mourad Abbay has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains two additional commits since the last revision:
>> 
>>  - Merge branch 'code-reflection' into fix-literal-type
>>  - Represents literals of type byte/short as permitted in the Java language.
>
> Changes look good - I left some comments to improve the code clarity. I think this PR also needs a test of the model that comes out of javac.

@mcimadamore I only added tests that cover the change we did to the compiler code. Cases like `byte/short x = some_literal` are already covered in `ConstantsTest`.

-------------

PR Comment: https://git.openjdk.org/babylon/pull/351#issuecomment-2719323001

From mabbay at openjdk.org  Wed Mar 12 23:24:14 2025
From: mabbay at openjdk.org (Mourad Abbay)
Date: Wed, 12 Mar 2025 23:24:14 GMT
Subject: [code-reflection] RFR: Represents literals of type byte/short as
 permitted in the Java language [v2]
In-Reply-To: <7tl7L9EbdrcijdFbQpFXVtF22jlqy0lqTnVkN4btVec=.e5fdf384-1694-40c6-a3d5-6eca37b401c5@github.com>
References: <zbICBVLiPQJAIloGr_22qhNGXZA--UdjKLeoCH_bmBU=.b9a87fd0-5918-4ef7-93d2-b2103a6427ea@github.com>
 <K78qZW9TaOK4qOwPZaVFn9fSqEIY3EfmZgKZdRiLyoo=.d64aff9a-3d4b-4073-a53c-731b59eceac4@github.com>
 <qNoBRk1NCyfF3BQuhsI_JizAMcXwQUH6hyt4G_6zbdk=.9d5a4aab-4358-43f8-b8f1-66c1e7d04cd7@github.com>
 <544IY_GmFyTb3NrhziDihxplxCVKlPhejYHhVrUBd0U=.8224766a-dfb6-4cb3-8c7f-f953c4ab1b91@github.com>
 <7tl7L9EbdrcijdFbQpFXVtF22jlqy0lqTnVkN4btVec=.e5fdf384-1694-40c6-a3d5-6eca37b401c5@github.com>
Message-ID: <5jyoDeyvjIVJifSc1_kNgMYaLZBXt_iVVToBdpjevm4=.7299b768-a70b-434a-b00d-188c00320439@github.com>

On Wed, 12 Mar 2025 18:20:32 GMT, Maurizio Cimadamore <mcimadamore at openjdk.org> wrote:

>> Ah ok - I see:
>> 
>>                 case CLASS -> numericOneValue(types.unboxedType(t));
>> 
>> 
>> So, `numericOneValue` also does unboxing. I think this code would be clearer by doing the unboxing first -- and then pass the unboxed type to both `numericOneValue` and `convert` (and then, we could remove the unboxing path from `numericOneValue` - and make that consistent with `defaultValue`).
>
>> and make that consistent with `defaultValue`).
> 
> Strike that -- `defaultValue` also allows `nulls`. Anyway -- `numericOneValue` should only work on primitives, I think.

For `defaultValue`, the passed type can be primitive or reference. For `numericOneValue` we can only accept a numerical primitive type, as value 1 doesn't make any senses for other types.

-------------

PR Review Comment: https://git.openjdk.org/babylon/pull/351#discussion_r1992421421

From mcimadamore at openjdk.org  Thu Mar 13 10:04:26 2025
From: mcimadamore at openjdk.org (Maurizio Cimadamore)
Date: Thu, 13 Mar 2025 10:04:26 GMT
Subject: [code-reflection] RFR: Represents literals of type byte/short as
 permitted in the Java language [v4]
In-Reply-To: <3QCW0HinX6I9ASKSm7pnY978Ovf8TUePZz0-7mY5Ik0=.393b4b44-3382-41f6-8550-48ad7698c57a@github.com>
References: <zbICBVLiPQJAIloGr_22qhNGXZA--UdjKLeoCH_bmBU=.b9a87fd0-5918-4ef7-93d2-b2103a6427ea@github.com>
 <3QCW0HinX6I9ASKSm7pnY978Ovf8TUePZz0-7mY5Ik0=.393b4b44-3382-41f6-8550-48ad7698c57a@github.com>
Message-ID: <pYuIUalwjxbs-4iQNHW_b9wktm2q5Vaz-BiCXwemSNQ=.7aa094cf-6faf-4c99-b4b5-a68348ec50a0@github.com>

On Wed, 12 Mar 2025 23:08:37 GMT, Mourad Abbay <mabbay at openjdk.org> wrote:

>> Before this PR, we had literals of type byte/short in the code model, but that's not permitted by the Java language. In Java, a literals of type byte/short is a literal of type int + conversion to the type byte/short.
>
> Mourad Abbay has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Add tests to make sure literals of type byte/short are created the way Java language permit.

Very good - thanks for adding the tests!

-------------

Marked as reviewed by mcimadamore (Reviewer).

PR Review: https://git.openjdk.org/babylon/pull/351#pullrequestreview-2681254467

From gfrost at openjdk.org  Thu Mar 13 12:58:24 2025
From: gfrost at openjdk.org (Gary Frost)
Date: Thu, 13 Mar 2025 12:58:24 GMT
Subject: git: openjdk/babylon: code-reflection: More lookup cleanup
Message-ID: <8a0ca55a-249e-4581-a017-94a4957b6165@openjdk.org>

Changeset: 226b61b8
Branch: code-reflection
Author:    Gary Frost <gfrost at openjdk.org>
Date:      2025-03-13 12:57:03 +0000
URL:       https://git.openjdk.org/babylon/commit/226b61b815dd25846a07c9d4a750c9045258993d

More lookup cleanup

! hat/backends/ffi/cuda/src/main/java/hat/backend/ffi/CudaHatKernelBuilder.java
! hat/backends/ffi/opencl/src/main/java/hat/backend/ffi/OpenCLHatKernelBuilder.java
! hat/backends/ffi/ptx/src/main/java/hat/backend/ffi/PTXBackend.java
! hat/backends/ffi/ptx/src/main/java/hat/backend/ffi/PTXCodeBuilder.java
! hat/backends/jextracted/opencl/src/main/java/hat/backend/jextracted/OpenCLHatKernelBuilder.java
! hat/examples/experiments/src/main/java/experiments/Chess.java
! hat/examples/experiments/src/main/java/experiments/LambdaTest.java
! hat/examples/experiments/src/main/java/experiments/QuotedTest.java
! hat/examples/violajones/src/main/java/violajones/attic/ViolaJonesRaw.java
! hat/hat-core/src/main/java/hat/Accelerator.java
! hat/hat-core/src/main/java/hat/ComputeContext.java
! hat/hat-core/src/main/java/hat/OpsAndTypes.java
! hat/hat-core/src/main/java/hat/backend/codebuilders/C99HATComputeBuilder.java
! hat/hat-core/src/main/java/hat/backend/codebuilders/C99HATKernelBuilder.java
! hat/hat-core/src/main/java/hat/backend/codebuilders/HATCodeBuilder.java
! hat/hat-core/src/main/java/hat/backend/codebuilders/HATCodeBuilderWithContext.java
! hat/hat-core/src/main/java/hat/backend/ffi/FFIBackend.java
! hat/hat-core/src/main/java/hat/backend/jextracted/JExtractedBackend.java
! hat/hat-core/src/main/java/hat/callgraph/ComputeCallGraph.java
! hat/hat-core/src/main/java/hat/callgraph/KernelCallGraph.java
! hat/hat-core/src/main/java/hat/optools/BinaryArithmeticOrLogicOperation.java
! hat/hat-core/src/main/java/hat/optools/BinaryLogicalOpWrapper.java
! hat/hat-core/src/main/java/hat/optools/BinaryOpWrapper.java
! hat/hat-core/src/main/java/hat/optools/BinaryTestOpWrapper.java
! hat/hat-core/src/main/java/hat/optools/ConstantOpWrapper.java
! hat/hat-core/src/main/java/hat/optools/ConvOpWrapper.java
! hat/hat-core/src/main/java/hat/optools/FieldAccessOpWrapper.java
! hat/hat-core/src/main/java/hat/optools/FieldLoadOpWrapper.java
! hat/hat-core/src/main/java/hat/optools/FieldStoreOpWrapper.java
! hat/hat-core/src/main/java/hat/optools/ForOpWrapper.java
! hat/hat-core/src/main/java/hat/optools/FuncCallOpWrapper.java
! hat/hat-core/src/main/java/hat/optools/FuncOpWrapper.java
! hat/hat-core/src/main/java/hat/optools/IfOpWrapper.java
! hat/hat-core/src/main/java/hat/optools/InvokeOpWrapper.java
! hat/hat-core/src/main/java/hat/optools/JavaBreakOpWrapper.java
! hat/hat-core/src/main/java/hat/optools/JavaContinueOpWrapper.java
! hat/hat-core/src/main/java/hat/optools/JavaLabeledOpWrapper.java
! hat/hat-core/src/main/java/hat/optools/LambdaOpWrapper.java
! hat/hat-core/src/main/java/hat/optools/LogicalOpWrapper.java
! hat/hat-core/src/main/java/hat/optools/LoopOpWrapper.java
! hat/hat-core/src/main/java/hat/optools/ModuleOpWrapper.java
! hat/hat-core/src/main/java/hat/optools/OpWrapper.java
! hat/hat-core/src/main/java/hat/optools/ReturnOpWrapper.java
! hat/hat-core/src/main/java/hat/optools/StructuralOpWrapper.java
! hat/hat-core/src/main/java/hat/optools/TernaryOpWrapper.java
! hat/hat-core/src/main/java/hat/optools/TupleOpWrapper.java
! hat/hat-core/src/main/java/hat/optools/UnaryArithmeticOrLogicOpWrapper.java
! hat/hat-core/src/main/java/hat/optools/UnaryOpWrapper.java
! hat/hat-core/src/main/java/hat/optools/VarAccessOpWrapper.java
! hat/hat-core/src/main/java/hat/optools/VarDeclarationOpWrapper.java
! hat/hat-core/src/main/java/hat/optools/VarFuncDeclarationOpWrapper.java
! hat/hat-core/src/main/java/hat/optools/VarLoadOpWrapper.java
! hat/hat-core/src/main/java/hat/optools/VarOpWrapper.java
! hat/hat-core/src/main/java/hat/optools/VarStoreOpWrapper.java
! hat/hat-core/src/main/java/hat/optools/WhileOpWrapper.java
! hat/hat-core/src/main/java/hat/optools/YieldOpWrapper.java


From gfrost at openjdk.org  Thu Mar 13 12:59:39 2025
From: gfrost at openjdk.org (Gary Frost)
Date: Thu, 13 Mar 2025 12:59:39 GMT
Subject: [code-reflection] Integrated: More lookup cleanup
Message-ID: <2oYbAvxzUo3wKlkjz66zE_sPqF7Tdv3vZpnnND84oyY=.61a85296-5c4e-469c-b9de-6b10b18861e2@github.com>

After previous PR I realised that we were passing lookups, which could be extracted from context. 

This PR cleans that up. 

Also I made the parametr order consistant across opWrappers.

-------------

Commit messages:
 - More lookup cleanup

Changes: https://git.openjdk.org/babylon/pull/354/files
  Webrev: https://webrevs.openjdk.org/?repo=babylon&pr=354&range=00
  Stats: 262 lines in 56 files changed: 15 ins; 15 del; 232 mod
  Patch: https://git.openjdk.org/babylon/pull/354.diff
  Fetch: git fetch https://git.openjdk.org/babylon.git pull/354/head:pull/354

PR: https://git.openjdk.org/babylon/pull/354

From gfrost at openjdk.org  Thu Mar 13 12:59:39 2025
From: gfrost at openjdk.org (Gary Frost)
Date: Thu, 13 Mar 2025 12:59:39 GMT
Subject: [code-reflection] Integrated: More lookup cleanup
In-Reply-To: <2oYbAvxzUo3wKlkjz66zE_sPqF7Tdv3vZpnnND84oyY=.61a85296-5c4e-469c-b9de-6b10b18861e2@github.com>
References: <2oYbAvxzUo3wKlkjz66zE_sPqF7Tdv3vZpnnND84oyY=.61a85296-5c4e-469c-b9de-6b10b18861e2@github.com>
Message-ID: <yuFFdMD45aJ09J1-gBZm_dUB8HDrHRAU6w_KwV5nU_g=.9ee66a67-f7e2-4f52-9c56-cacbbb215d1a@github.com>

On Thu, 13 Mar 2025 12:54:49 GMT, Gary Frost <gfrost at openjdk.org> wrote:

> After previous PR I realised that we were passing lookups, which could be extracted from context. 
> 
> This PR cleans that up. 
> 
> Also I made the parametr order consistant across opWrappers.

This pull request has now been integrated.

Changeset: 226b61b8
Author:    Gary Frost <gfrost at openjdk.org>
URL:       https://git.openjdk.org/babylon/commit/226b61b815dd25846a07c9d4a750c9045258993d
Stats:     262 lines in 56 files changed: 15 ins; 15 del; 232 mod

More lookup cleanup

-------------

PR: https://git.openjdk.org/babylon/pull/354

From mcimadamore at openjdk.org  Thu Mar 13 13:20:15 2025
From: mcimadamore at openjdk.org (Maurizio Cimadamore)
Date: Thu, 13 Mar 2025 13:20:15 GMT
Subject: [code-reflection] RFR: Support storing the code that builds the
 code model [v18]
In-Reply-To: <rLtW9Wv92MNgUIywKaiPdoBs8O5-5EXNlu17J8tRflk=.3e0aa614-55db-4e86-a5a0-99e8e51758d8@github.com>
References: <dexv8sQV_NcJsJIgMopd_M4wSWoZya9dU0ACwDtEL0E=.4ac0b80f-24fb-440f-a18d-bbfd07a681bc@github.com>
 <rLtW9Wv92MNgUIywKaiPdoBs8O5-5EXNlu17J8tRflk=.3e0aa614-55db-4e86-a5a0-99e8e51758d8@github.com>
Message-ID: <23hI2EWkXS2p8UsVzNiJ-nlSKnWXSquYQTbcg4nXyHA=.a2b260e2-906e-4f70-82d6-d52b8750568d@github.com>

On Wed, 12 Mar 2025 18:00:05 GMT, Mourad Abbay <mabbay at openjdk.org> wrote:

>> In this PR we allow the user to decide how to store the code model. The first option is `TEXT`, if this is selected, we store the textual representation of the code model. The second option is `CODE_BUILDR`, if this is selected, we store the code that builds the code model. All work done here is around the second option, because the first option is already supported.
>
> Mourad Abbay has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains 20 additional commits since the last revision:
> 
>  - Merge branch 'code-reflection' into code-model-storage-option
>  - Load OpFactory and TypeElementFactory before invocation of opMethod only if the opMethod has there params.
>  - Fix the remaining compiler tests failures
>  - Fix some of the test failures (3 remains)
>  - Fix the remaining test failures of SwitchExpressionTest2
>  - Fix almost all test failures of SwitchExpressionTest2 (one remaining)
>  - Pass arrayType instead of eleType in OpBuilder.buildArray
>  - Ensure that block params are inserted in the correct order
>  - Add missing imports
>  - Merge branch 'code-reflection' into code-model-storage-option
>  - ... and 10 more: https://git.openjdk.org/babylon/compare/71c99f4b...966005ce

I've verified that all tests pass, except the two test which fail with a compiler crash because of #351.

-------------

PR Comment: https://git.openjdk.org/babylon/pull/305#issuecomment-2721231441

From mcimadamore at openjdk.org  Thu Mar 13 13:31:18 2025
From: mcimadamore at openjdk.org (Maurizio Cimadamore)
Date: Thu, 13 Mar 2025 13:31:18 GMT
Subject: [code-reflection] RFR: Support storing the code that builds the
 code model [v18]
In-Reply-To: <rLtW9Wv92MNgUIywKaiPdoBs8O5-5EXNlu17J8tRflk=.3e0aa614-55db-4e86-a5a0-99e8e51758d8@github.com>
References: <dexv8sQV_NcJsJIgMopd_M4wSWoZya9dU0ACwDtEL0E=.4ac0b80f-24fb-440f-a18d-bbfd07a681bc@github.com>
 <rLtW9Wv92MNgUIywKaiPdoBs8O5-5EXNlu17J8tRflk=.3e0aa614-55db-4e86-a5a0-99e8e51758d8@github.com>
Message-ID: <UZ2Z1W9PyhkEv4ucTxh-zfBISS__-0xiY5PGL4p67qw=.9a8dfbfd-3809-4fba-891f-543bead72d6b@github.com>

On Wed, 12 Mar 2025 18:00:05 GMT, Mourad Abbay <mabbay at openjdk.org> wrote:

>> In this PR we allow the user to decide how to store the code model. The first option is `TEXT`, if this is selected, we store the textual representation of the code model. The second option is `CODE_BUILDR`, if this is selected, we store the code that builds the code model. All work done here is around the second option, because the first option is already supported.
>
> Mourad Abbay has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains 20 additional commits since the last revision:
> 
>  - Merge branch 'code-reflection' into code-model-storage-option
>  - Load OpFactory and TypeElementFactory before invocation of opMethod only if the opMethod has there params.
>  - Fix the remaining compiler tests failures
>  - Fix some of the test failures (3 remains)
>  - Fix the remaining test failures of SwitchExpressionTest2
>  - Fix almost all test failures of SwitchExpressionTest2 (one remaining)
>  - Pass arrayType instead of eleType in OpBuilder.buildArray
>  - Ensure that block params are inserted in the correct order
>  - Add missing imports
>  - Merge branch 'code-reflection' into code-model-storage-option
>  - ... and 10 more: https://git.openjdk.org/babylon/compare/40db887c...966005ce

src/java.base/share/classes/java/lang/invoke/InnerClassLambdaMetafactory.java line 453:

> 451:         }
> 452:         if (quotableOpGetterInfo.getMethodType().parameterList().equals(List.of(CodeReflectionSupport.OP_FACTORY_CLASS,
> 453:                 CodeReflectionSupport.TYPE_ELEMENT_FACTORY_CLASS))) {

If these fields are public/static, why don't we load them from inside the compiler-generated method, instead of injecting them here?

src/jdk.incubator.code/share/classes/jdk/incubator/code/Op.java line 544:

> 542:                 opMethod = method.getDeclaringClass().getDeclaredMethod(opMethodName, OpFactory.class,
> 543:                         TypeElementFactory.class);
> 544:                 args = new Object[] {ExtendedOp.FACTORY, CoreTypeFactory.CORE_TYPE_FACTORY};

Same questions -- why are these fields injected into the code -- instead of being emitted into the compiler-generated code? (keeping these no-arg methods would simplify this PR)

-------------

PR Review Comment: https://git.openjdk.org/babylon/pull/305#discussion_r1993522330
PR Review Comment: https://git.openjdk.org/babylon/pull/305#discussion_r1993523632

From mcimadamore at openjdk.org  Thu Mar 13 14:12:21 2025
From: mcimadamore at openjdk.org (Maurizio Cimadamore)
Date: Thu, 13 Mar 2025 14:12:21 GMT
Subject: [code-reflection] RFR: Support storing the code that builds the
 code model [v18]
In-Reply-To: <rLtW9Wv92MNgUIywKaiPdoBs8O5-5EXNlu17J8tRflk=.3e0aa614-55db-4e86-a5a0-99e8e51758d8@github.com>
References: <dexv8sQV_NcJsJIgMopd_M4wSWoZya9dU0ACwDtEL0E=.4ac0b80f-24fb-440f-a18d-bbfd07a681bc@github.com>
 <rLtW9Wv92MNgUIywKaiPdoBs8O5-5EXNlu17J8tRflk=.3e0aa614-55db-4e86-a5a0-99e8e51758d8@github.com>
Message-ID: <yc9uuZ_qzf1TwSQ5ppEY8BNA97TpgRwTLKSHfPYeLF8=.464efbf8-0bed-4540-a40f-e9ac2dc99d2c@github.com>

On Wed, 12 Mar 2025 18:00:05 GMT, Mourad Abbay <mabbay at openjdk.org> wrote:

>> In this PR we allow the user to decide how to store the code model. The first option is `TEXT`, if this is selected, we store the textual representation of the code model. The second option is `CODE_BUILDR`, if this is selected, we store the code that builds the code model. All work done here is around the second option, because the first option is already supported.
>
> Mourad Abbay has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains 20 additional commits since the last revision:
> 
>  - Merge branch 'code-reflection' into code-model-storage-option
>  - Load OpFactory and TypeElementFactory before invocation of opMethod only if the opMethod has there params.
>  - Fix the remaining compiler tests failures
>  - Fix some of the test failures (3 remains)
>  - Fix the remaining test failures of SwitchExpressionTest2
>  - Fix almost all test failures of SwitchExpressionTest2 (one remaining)
>  - Pass arrayType instead of eleType in OpBuilder.buildArray
>  - Ensure that block params are inserted in the correct order
>  - Add missing imports
>  - Merge branch 'code-reflection' into code-model-storage-option
>  - ... and 10 more: https://git.openjdk.org/babylon/compare/e12c5274...966005ce

test/jdk/java/lang/reflect/code/TestAddVarsWhenNecessary.java line 17:

> 15:  * @run junit TestAddVarsWhenNecessary
> 16:  */
> 17: public class TestAddVarsWhenNecessary {

Is this still needed?

-------------

PR Review Comment: https://git.openjdk.org/babylon/pull/305#discussion_r1993613761

From mcimadamore at openjdk.org  Thu Mar 13 14:32:10 2025
From: mcimadamore at openjdk.org (Maurizio Cimadamore)
Date: Thu, 13 Mar 2025 14:32:10 GMT
Subject: [code-reflection] RFR: Support storing the code that builds the
 code model [v18]
In-Reply-To: <rLtW9Wv92MNgUIywKaiPdoBs8O5-5EXNlu17J8tRflk=.3e0aa614-55db-4e86-a5a0-99e8e51758d8@github.com>
References: <dexv8sQV_NcJsJIgMopd_M4wSWoZya9dU0ACwDtEL0E=.4ac0b80f-24fb-440f-a18d-bbfd07a681bc@github.com>
 <rLtW9Wv92MNgUIywKaiPdoBs8O5-5EXNlu17J8tRflk=.3e0aa614-55db-4e86-a5a0-99e8e51758d8@github.com>
Message-ID: <fz8Kzl5g5McZSRm9TMVLMzYUIdGHREPgher1NlND_s0=.bf74f1db-6407-4ace-a198-5a4a662d8bc5@github.com>

On Wed, 12 Mar 2025 18:00:05 GMT, Mourad Abbay <mabbay at openjdk.org> wrote:

>> In this PR we allow the user to decide how to store the code model. The first option is `TEXT`, if this is selected, we store the textual representation of the code model. The second option is `CODE_BUILDR`, if this is selected, we store the code that builds the code model. All work done here is around the second option, because the first option is already supported.
>
> Mourad Abbay has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains 20 additional commits since the last revision:
> 
>  - Merge branch 'code-reflection' into code-model-storage-option
>  - Load OpFactory and TypeElementFactory before invocation of opMethod only if the opMethod has there params.
>  - Fix the remaining compiler tests failures
>  - Fix some of the test failures (3 remains)
>  - Fix the remaining test failures of SwitchExpressionTest2
>  - Fix almost all test failures of SwitchExpressionTest2 (one remaining)
>  - Pass arrayType instead of eleType in OpBuilder.buildArray
>  - Ensure that block params are inserted in the correct order
>  - Add missing imports
>  - Merge branch 'code-reflection' into code-model-storage-option
>  - ... and 10 more: https://git.openjdk.org/babylon/compare/c9c0f0fa...966005ce

src/jdk.incubator.code/share/classes/jdk/incubator/code/internal/CodeModelToAST.java line 222:

> 220:         // TODO add VarOps in OpBuilder
> 221:         funcOp = addVarsWhenNecessary(funcOp);
> 222:         funcOp.writeTo(System.out);

This should be removed?

-------------

PR Review Comment: https://git.openjdk.org/babylon/pull/305#discussion_r1993656341

From mcimadamore at openjdk.org  Thu Mar 13 14:40:21 2025
From: mcimadamore at openjdk.org (Maurizio Cimadamore)
Date: Thu, 13 Mar 2025 14:40:21 GMT
Subject: [code-reflection] RFR: Support storing the code that builds the
 code model [v18]
In-Reply-To: <rLtW9Wv92MNgUIywKaiPdoBs8O5-5EXNlu17J8tRflk=.3e0aa614-55db-4e86-a5a0-99e8e51758d8@github.com>
References: <dexv8sQV_NcJsJIgMopd_M4wSWoZya9dU0ACwDtEL0E=.4ac0b80f-24fb-440f-a18d-bbfd07a681bc@github.com>
 <rLtW9Wv92MNgUIywKaiPdoBs8O5-5EXNlu17J8tRflk=.3e0aa614-55db-4e86-a5a0-99e8e51758d8@github.com>
Message-ID: <ftnpqrjPGs9YPOQm7MbEV8PZb2dGx92ga2-340cOuh0=.8a2f956a-f277-4009-b461-05e9d88363f5@github.com>

On Wed, 12 Mar 2025 18:00:05 GMT, Mourad Abbay <mabbay at openjdk.org> wrote:

>> In this PR we allow the user to decide how to store the code model. The first option is `TEXT`, if this is selected, we store the textual representation of the code model. The second option is `CODE_BUILDR`, if this is selected, we store the code that builds the code model. All work done here is around the second option, because the first option is already supported.
>
> Mourad Abbay has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains 20 additional commits since the last revision:
> 
>  - Merge branch 'code-reflection' into code-model-storage-option
>  - Load OpFactory and TypeElementFactory before invocation of opMethod only if the opMethod has there params.
>  - Fix the remaining compiler tests failures
>  - Fix some of the test failures (3 remains)
>  - Fix the remaining test failures of SwitchExpressionTest2
>  - Fix almost all test failures of SwitchExpressionTest2 (one remaining)
>  - Pass arrayType instead of eleType in OpBuilder.buildArray
>  - Ensure that block params are inserted in the correct order
>  - Add missing imports
>  - Merge branch 'code-reflection' into code-model-storage-option
>  - ... and 10 more: https://git.openjdk.org/babylon/compare/8b5654da...966005ce

src/jdk.incubator.code/share/classes/jdk/incubator/code/internal/CodeModelToAST.java line 151:

> 149:                     yield na;
> 150:                 }
> 151:                 var ownerType = typeElementToType(newOp.constructorType().returnType());

There is a strange asymmetry in the model between constructors and methods. Methods have a descriptor, constructors do not. Also, for methods you know from the op if it is varargs, but for constructors this info is not available. My feeling is that (at least coming at this from javac), trying to lump array and class creation under one op is probably why these asymmetries crop up.

This has nothing to do with this PR of course -- but I suspect that if you had a varargs constructor call this code would not work.

-------------

PR Review Comment: https://git.openjdk.org/babylon/pull/305#discussion_r1993672762

From mcimadamore at openjdk.org  Thu Mar 13 14:43:18 2025
From: mcimadamore at openjdk.org (Maurizio Cimadamore)
Date: Thu, 13 Mar 2025 14:43:18 GMT
Subject: [code-reflection] RFR: Support storing the code that builds the
 code model [v18]
In-Reply-To: <rLtW9Wv92MNgUIywKaiPdoBs8O5-5EXNlu17J8tRflk=.3e0aa614-55db-4e86-a5a0-99e8e51758d8@github.com>
References: <dexv8sQV_NcJsJIgMopd_M4wSWoZya9dU0ACwDtEL0E=.4ac0b80f-24fb-440f-a18d-bbfd07a681bc@github.com>
 <rLtW9Wv92MNgUIywKaiPdoBs8O5-5EXNlu17J8tRflk=.3e0aa614-55db-4e86-a5a0-99e8e51758d8@github.com>
Message-ID: <ieI_WwBslJzVLxiNH_lKEwpStmUw_ITELaj5LFXTtxs=.c9565238-3b7e-42ab-997f-455fb061f807@github.com>

On Wed, 12 Mar 2025 18:00:05 GMT, Mourad Abbay <mabbay at openjdk.org> wrote:

>> In this PR we allow the user to decide how to store the code model. The first option is `TEXT`, if this is selected, we store the textual representation of the code model. The second option is `CODE_BUILDR`, if this is selected, we store the code that builds the code model. All work done here is around the second option, because the first option is already supported.
>
> Mourad Abbay has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains 20 additional commits since the last revision:
> 
>  - Merge branch 'code-reflection' into code-model-storage-option
>  - Load OpFactory and TypeElementFactory before invocation of opMethod only if the opMethod has there params.
>  - Fix the remaining compiler tests failures
>  - Fix some of the test failures (3 remains)
>  - Fix the remaining test failures of SwitchExpressionTest2
>  - Fix almost all test failures of SwitchExpressionTest2 (one remaining)
>  - Pass arrayType instead of eleType in OpBuilder.buildArray
>  - Ensure that block params are inserted in the correct order
>  - Add missing imports
>  - Merge branch 'code-reflection' into code-model-storage-option
>  - ... and 10 more: https://git.openjdk.org/babylon/compare/2e04b839...966005ce

src/jdk.incubator.code/share/classes/jdk/incubator/code/internal/CodeModelToAST.java line 145:

> 143:                 if (newOp.resultType() instanceof ArrayType at) {
> 144:                     var elemType = treeMaker.Ident(typeElementToType(at.componentType()).tsym);
> 145:                     // @@@ we assume one dimension

note: `ArrayType` has a `dimensions()` method -- so in principle you could use that and keep fetching dimension operands (and update the type of the array to be built)

-------------

PR Review Comment: https://git.openjdk.org/babylon/pull/305#discussion_r1993679515

From mcimadamore at openjdk.org  Thu Mar 13 14:48:15 2025
From: mcimadamore at openjdk.org (Maurizio Cimadamore)
Date: Thu, 13 Mar 2025 14:48:15 GMT
Subject: [code-reflection] RFR: Support storing the code that builds the
 code model [v18]
In-Reply-To: <rLtW9Wv92MNgUIywKaiPdoBs8O5-5EXNlu17J8tRflk=.3e0aa614-55db-4e86-a5a0-99e8e51758d8@github.com>
References: <dexv8sQV_NcJsJIgMopd_M4wSWoZya9dU0ACwDtEL0E=.4ac0b80f-24fb-440f-a18d-bbfd07a681bc@github.com>
 <rLtW9Wv92MNgUIywKaiPdoBs8O5-5EXNlu17J8tRflk=.3e0aa614-55db-4e86-a5a0-99e8e51758d8@github.com>
Message-ID: <EtpyFI5TXv5K0vKcAsucd7PBtFRQRwaEk-vo_xQXDSg=.60a9033a-a7a3-40a8-abe5-de1331f93627@github.com>

On Wed, 12 Mar 2025 18:00:05 GMT, Mourad Abbay <mabbay at openjdk.org> wrote:

>> In this PR we allow the user to decide how to store the code model. The first option is `TEXT`, if this is selected, we store the textual representation of the code model. The second option is `CODE_BUILDR`, if this is selected, we store the code that builds the code model. All work done here is around the second option, because the first option is already supported.
>
> Mourad Abbay has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains 20 additional commits since the last revision:
> 
>  - Merge branch 'code-reflection' into code-model-storage-option
>  - Load OpFactory and TypeElementFactory before invocation of opMethod only if the opMethod has there params.
>  - Fix the remaining compiler tests failures
>  - Fix some of the test failures (3 remains)
>  - Fix the remaining test failures of SwitchExpressionTest2
>  - Fix almost all test failures of SwitchExpressionTest2 (one remaining)
>  - Pass arrayType instead of eleType in OpBuilder.buildArray
>  - Ensure that block params are inserted in the correct order
>  - Add missing imports
>  - Merge branch 'code-reflection' into code-model-storage-option
>  - ... and 10 more: https://git.openjdk.org/babylon/compare/54ef7f9c...966005ce

src/jdk.incubator.code/share/classes/jdk/incubator/code/internal/CodeModelToAST.java line 160:

> 158:                 var argTypes = new ListBuffer<Type>();
> 159:                 for (Value operand : newOp.operands()) {
> 160:                     argTypes.add(typeElementToType(operand.type()));

This seems wrong -- and is related to the point I made above -- the type of the operands here are the types of the actual arguments of the constructor calls. So, if you have:

class Foo {
     Foo(String string) { ... }
 }
 ```
 
 the method type of the constructor call has to say `Foo(String)`. But if you "infer" the type of the constructor argument from the actual argument, if the call is like this:
 
 ```
 new Foo(null)
 ```
 
We should probably use the op's `constructorType` to derive the type of the invoked constructor? (You use that to determine the constructor's owner type)

-------------

PR Review Comment: https://git.openjdk.org/babylon/pull/305#discussion_r1993690501

From mcimadamore at openjdk.org  Thu Mar 13 15:04:23 2025
From: mcimadamore at openjdk.org (Maurizio Cimadamore)
Date: Thu, 13 Mar 2025 15:04:23 GMT
Subject: [code-reflection] RFR: Support storing the code that builds the
 code model [v18]
In-Reply-To: <rLtW9Wv92MNgUIywKaiPdoBs8O5-5EXNlu17J8tRflk=.3e0aa614-55db-4e86-a5a0-99e8e51758d8@github.com>
References: <dexv8sQV_NcJsJIgMopd_M4wSWoZya9dU0ACwDtEL0E=.4ac0b80f-24fb-440f-a18d-bbfd07a681bc@github.com>
 <rLtW9Wv92MNgUIywKaiPdoBs8O5-5EXNlu17J8tRflk=.3e0aa614-55db-4e86-a5a0-99e8e51758d8@github.com>
Message-ID: <V8PvxUNR984P--XPe6Pm3Mc8OvwrLObPEBQpfpJBf2E=.7409e575-468a-4541-ad5c-bf1d06008be3@github.com>

On Wed, 12 Mar 2025 18:00:05 GMT, Mourad Abbay <mabbay at openjdk.org> wrote:

>> In this PR we allow the user to decide how to store the code model. The first option is `TEXT`, if this is selected, we store the textual representation of the code model. The second option is `CODE_BUILDR`, if this is selected, we store the code that builds the code model. All work done here is around the second option, because the first option is already supported.
>
> Mourad Abbay has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains 20 additional commits since the last revision:
> 
>  - Merge branch 'code-reflection' into code-model-storage-option
>  - Load OpFactory and TypeElementFactory before invocation of opMethod only if the opMethod has there params.
>  - Fix the remaining compiler tests failures
>  - Fix some of the test failures (3 remains)
>  - Fix the remaining test failures of SwitchExpressionTest2
>  - Fix almost all test failures of SwitchExpressionTest2 (one remaining)
>  - Pass arrayType instead of eleType in OpBuilder.buildArray
>  - Ensure that block params are inserted in the correct order
>  - Add missing imports
>  - Merge branch 'code-reflection' into code-model-storage-option
>  - ... and 10 more: https://git.openjdk.org/babylon/compare/1fa13c4a...966005ce

src/jdk.incubator.code/share/classes/jdk/incubator/code/internal/CodeModelToAST.java line 239:

> 237:     }
> 238: 
> 239:     public static CoreOp.FuncOp addVarsWhenNecessary(CoreOp.FuncOp funcOp) {

Maybe I'm missing something obvious. But if you have something like:

func @"f" ()java.util.Map -> {
    %0 : java.util.Map<java.lang.Integer, java.lang.Integer> = new @"func<java.util.HashMap>";
    %1 : int = constant @"1";
    %2 : int = constant @"2";
    %3 : java.lang.Object = invoke %0 %1 %2 @"java.util.Map::put(java.lang.Object, java.lang.Object)java.lang.Object";
    return %0;
}


Can't we translate as:


Map $0 = new HashMap();
int $1 = 1;
int $2 = 3;    
Object $3 = $0.put($1, $2);
return $0;


Note how this translation is very 1-1 and direct, and doesn't require any adaptation step in the model. We just emit a new local variable for each op we see in the body -- and then refer to operands via their corresponding (locally declared) variable.

-------------

PR Review Comment: https://git.openjdk.org/babylon/pull/305#discussion_r1993725923

From mcimadamore at openjdk.org  Thu Mar 13 15:13:25 2025
From: mcimadamore at openjdk.org (Maurizio Cimadamore)
Date: Thu, 13 Mar 2025 15:13:25 GMT
Subject: [code-reflection] RFR: Support storing the code that builds the
 code model [v18]
In-Reply-To: <rLtW9Wv92MNgUIywKaiPdoBs8O5-5EXNlu17J8tRflk=.3e0aa614-55db-4e86-a5a0-99e8e51758d8@github.com>
References: <dexv8sQV_NcJsJIgMopd_M4wSWoZya9dU0ACwDtEL0E=.4ac0b80f-24fb-440f-a18d-bbfd07a681bc@github.com>
 <rLtW9Wv92MNgUIywKaiPdoBs8O5-5EXNlu17J8tRflk=.3e0aa614-55db-4e86-a5a0-99e8e51758d8@github.com>
Message-ID: <JuZpObuzFUKUoRyO3UBQZpcjNAhzwhLIvh3B7-uoK74=.c1d4aa04-085a-4281-8b90-910b160fd9c8@github.com>

On Wed, 12 Mar 2025 18:00:05 GMT, Mourad Abbay <mabbay at openjdk.org> wrote:

>> In this PR we allow the user to decide how to store the code model. The first option is `TEXT`, if this is selected, we store the textual representation of the code model. The second option is `CODE_BUILDR`, if this is selected, we store the code that builds the code model. All work done here is around the second option, because the first option is already supported.
>
> Mourad Abbay has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains 20 additional commits since the last revision:
> 
>  - Merge branch 'code-reflection' into code-model-storage-option
>  - Load OpFactory and TypeElementFactory before invocation of opMethod only if the opMethod has there params.
>  - Fix the remaining compiler tests failures
>  - Fix some of the test failures (3 remains)
>  - Fix the remaining test failures of SwitchExpressionTest2
>  - Fix almost all test failures of SwitchExpressionTest2 (one remaining)
>  - Pass arrayType instead of eleType in OpBuilder.buildArray
>  - Ensure that block params are inserted in the correct order
>  - Add missing imports
>  - Merge branch 'code-reflection' into code-model-storage-option
>  - ... and 10 more: https://git.openjdk.org/babylon/compare/29930c4b...966005ce

src/jdk.incubator.code/share/classes/jdk/incubator/code/internal/CodeModelToAST.java line 117:

> 115:         var restype = typeElementToType(invokeOp.resultType());
> 116:         var type = new Type.MethodType(paramTypes, restype, List.nil(), syms.methodClass);
> 117:         var methodSym = new Symbol.MethodSymbol(flags, name, type,

Same as for field access (see below) -- we should not create a new method symbol from scratch but "resolve" a method in a given class symbol (using `Resolve::findInternalMethod`)

src/jdk.incubator.code/share/classes/jdk/incubator/code/internal/CodeModelToAST.java line 192:

> 190:                 var type = typeElementToType(fieldLoadOp.resultType());
> 191:                 var owner = typeElementToType(fieldLoadOp.fieldDescriptor().refType());
> 192:                 var sym = new Symbol.VarSymbol(flags, name, type, owner.tsym);

This seems wrong -- the accessed field symbol should be findable in the scope of the class in which the field is declared. It seems to me that we need some piece of code that turns a method/field descriptor into a java symbol. Look at `Lower::lookupMethod` which uses `Resolve::resolveInternalMethod` -- I think here we should do that too.

-------------

PR Review Comment: https://git.openjdk.org/babylon/pull/305#discussion_r1993741896
PR Review Comment: https://git.openjdk.org/babylon/pull/305#discussion_r1993736120

From mcimadamore at openjdk.org  Thu Mar 13 15:13:25 2025
From: mcimadamore at openjdk.org (Maurizio Cimadamore)
Date: Thu, 13 Mar 2025 15:13:25 GMT
Subject: [code-reflection] RFR: Support storing the code that builds the
 code model [v18]
In-Reply-To: <JuZpObuzFUKUoRyO3UBQZpcjNAhzwhLIvh3B7-uoK74=.c1d4aa04-085a-4281-8b90-910b160fd9c8@github.com>
References: <dexv8sQV_NcJsJIgMopd_M4wSWoZya9dU0ACwDtEL0E=.4ac0b80f-24fb-440f-a18d-bbfd07a681bc@github.com>
 <rLtW9Wv92MNgUIywKaiPdoBs8O5-5EXNlu17J8tRflk=.3e0aa614-55db-4e86-a5a0-99e8e51758d8@github.com>
 <JuZpObuzFUKUoRyO3UBQZpcjNAhzwhLIvh3B7-uoK74=.c1d4aa04-085a-4281-8b90-910b160fd9c8@github.com>
Message-ID: <hD63TlJ-G8jkbGGmJ85CNk7F-o9rdoytopiZpd21tgw=.5ffa9624-df81-4002-9e80-14efee391388@github.com>

On Thu, 13 Mar 2025 15:09:06 GMT, Maurizio Cimadamore <mcimadamore at openjdk.org> wrote:

>> Mourad Abbay has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains 20 additional commits since the last revision:
>> 
>>  - Merge branch 'code-reflection' into code-model-storage-option
>>  - Load OpFactory and TypeElementFactory before invocation of opMethod only if the opMethod has there params.
>>  - Fix the remaining compiler tests failures
>>  - Fix some of the test failures (3 remains)
>>  - Fix the remaining test failures of SwitchExpressionTest2
>>  - Fix almost all test failures of SwitchExpressionTest2 (one remaining)
>>  - Pass arrayType instead of eleType in OpBuilder.buildArray
>>  - Ensure that block params are inserted in the correct order
>>  - Add missing imports
>>  - Merge branch 'code-reflection' into code-model-storage-option
>>  - ... and 10 more: https://git.openjdk.org/babylon/compare/29930c4b...966005ce
>
> src/jdk.incubator.code/share/classes/jdk/incubator/code/internal/CodeModelToAST.java line 117:
> 
>> 115:         var restype = typeElementToType(invokeOp.resultType());
>> 116:         var type = new Type.MethodType(paramTypes, restype, List.nil(), syms.methodClass);
>> 117:         var methodSym = new Symbol.MethodSymbol(flags, name, type,
> 
> Same as for field access (see below) -- we should not create a new method symbol from scratch but "resolve" a method in a given class symbol (using `Resolve::findInternalMethod`)

I suggest to add two helper methods -- one takes a FieldDescriptor and returns a VarSymbol (using `Resolve::findFieldInternal` -- the other takes a MethodDescriptor and returns a MethodSymbol (using `Resolve::findMethodIntenal`. Then you rewrite the code here (and in field access) to use these methods.

-------------

PR Review Comment: https://git.openjdk.org/babylon/pull/305#discussion_r1993745510

From mcimadamore at openjdk.org  Thu Mar 13 15:19:13 2025
From: mcimadamore at openjdk.org (Maurizio Cimadamore)
Date: Thu, 13 Mar 2025 15:19:13 GMT
Subject: [code-reflection] RFR: Support storing the code that builds the
 code model [v18]
In-Reply-To: <V8PvxUNR984P--XPe6Pm3Mc8OvwrLObPEBQpfpJBf2E=.7409e575-468a-4541-ad5c-bf1d06008be3@github.com>
References: <dexv8sQV_NcJsJIgMopd_M4wSWoZya9dU0ACwDtEL0E=.4ac0b80f-24fb-440f-a18d-bbfd07a681bc@github.com>
 <rLtW9Wv92MNgUIywKaiPdoBs8O5-5EXNlu17J8tRflk=.3e0aa614-55db-4e86-a5a0-99e8e51758d8@github.com>
 <V8PvxUNR984P--XPe6Pm3Mc8OvwrLObPEBQpfpJBf2E=.7409e575-468a-4541-ad5c-bf1d06008be3@github.com>
Message-ID: <OvhU6gyOr8IAyPV3vm9-l5lGeu8omMcuGcSYXkW2w-4=.15cee4a4-d18f-40b4-b50c-cc34fdb762aa@github.com>

On Thu, 13 Mar 2025 15:01:55 GMT, Maurizio Cimadamore <mcimadamore at openjdk.org> wrote:

>> Mourad Abbay has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains 20 additional commits since the last revision:
>> 
>>  - Merge branch 'code-reflection' into code-model-storage-option
>>  - Load OpFactory and TypeElementFactory before invocation of opMethod only if the opMethod has there params.
>>  - Fix the remaining compiler tests failures
>>  - Fix some of the test failures (3 remains)
>>  - Fix the remaining test failures of SwitchExpressionTest2
>>  - Fix almost all test failures of SwitchExpressionTest2 (one remaining)
>>  - Pass arrayType instead of eleType in OpBuilder.buildArray
>>  - Ensure that block params are inserted in the correct order
>>  - Add missing imports
>>  - Merge branch 'code-reflection' into code-model-storage-option
>>  - ... and 10 more: https://git.openjdk.org/babylon/compare/07cefe5c...966005ce
>
> src/jdk.incubator.code/share/classes/jdk/incubator/code/internal/CodeModelToAST.java line 239:
> 
>> 237:     }
>> 238: 
>> 239:     public static CoreOp.FuncOp addVarsWhenNecessary(CoreOp.FuncOp funcOp) {
> 
> Maybe I'm missing something obvious. But if you have something like:
> 
> func @"f" ()java.util.Map -> {
>     %0 : java.util.Map<java.lang.Integer, java.lang.Integer> = new @"func<java.util.HashMap>";
>     %1 : int = constant @"1";
>     %2 : int = constant @"2";
>     %3 : java.lang.Object = invoke %0 %1 %2 @"java.util.Map::put(java.lang.Object, java.lang.Object)java.lang.Object";
>     return %0;
> }
> 
> 
> Can't we translate as:
> 
> 
> Map $0 = new HashMap();
> int $1 = 1;
> int $2 = 3;    
> Object $3 = $0.put($1, $2);
> return $0;
> 
> 
> Note how this translation is very 1-1 and direct, and doesn't require any adaptation step in the model. We just emit a new local variable for each op we see in the body -- and then refer to operands via their corresponding (locally declared) variable.

(of course we'd only do the above translation for ops that have a non-void result type)

-------------

PR Review Comment: https://git.openjdk.org/babylon/pull/305#discussion_r1993755791

From gfrost at openjdk.org  Thu Mar 13 15:26:37 2025
From: gfrost at openjdk.org (Gary Frost)
Date: Thu, 13 Mar 2025 15:26:37 GMT
Subject: [code-reflection] Integrated: We should be able to select
 platform/device via env var
Message-ID: <JaEzV4X8-4HhnMk7c_t5WqiOo7E-HyKHZ4jNiBjyJeo=.43a4f779-ae13-46dd-9eff-dcbc10d6e9ce@github.com>

We can request specific opencl platform and device

Default is platfom:0 device:0

We can pass via HAT env var

HAT=PLATFORM:0,DEVICE:0 java @bldr/hatrun ffi-opencl life

-------------

Commit messages:
 - We should be able to select platform/device via env var

Changes: https://git.openjdk.org/babylon/pull/355/files
  Webrev: https://webrevs.openjdk.org/?repo=babylon&pr=355&range=00
  Stats: 638 lines in 9 files changed: 227 ins; 373 del; 38 mod
  Patch: https://git.openjdk.org/babylon/pull/355.diff
  Fetch: git fetch https://git.openjdk.org/babylon.git pull/355/head:pull/355

PR: https://git.openjdk.org/babylon/pull/355

From gfrost at openjdk.org  Thu Mar 13 15:26:38 2025
From: gfrost at openjdk.org (Gary Frost)
Date: Thu, 13 Mar 2025 15:26:38 GMT
Subject: [code-reflection] Integrated: We should be able to select
 platform/device via env var
In-Reply-To: <JaEzV4X8-4HhnMk7c_t5WqiOo7E-HyKHZ4jNiBjyJeo=.43a4f779-ae13-46dd-9eff-dcbc10d6e9ce@github.com>
References: <JaEzV4X8-4HhnMk7c_t5WqiOo7E-HyKHZ4jNiBjyJeo=.43a4f779-ae13-46dd-9eff-dcbc10d6e9ce@github.com>
Message-ID: <84Qf1_DszlUTrWg-13_lCcYSc96dlU5V1XvtDZZfmfo=.4759d34e-a97b-4da5-a116-b3fc758ece01@github.com>

On Thu, 13 Mar 2025 15:21:51 GMT, Gary Frost <gfrost at openjdk.org> wrote:

> We can request specific opencl platform and device
> 
> Default is platfom:0 device:0
> 
> We can pass via HAT env var
> 
> HAT=PLATFORM:0,DEVICE:0 java @bldr/hatrun ffi-opencl life

This pull request has now been integrated.

Changeset: f93580a0
Author:    Gary Frost <gfrost at openjdk.org>
URL:       https://git.openjdk.org/babylon/commit/f93580a0797179a39bc000f9ed56322e09d34185
Stats:     638 lines in 9 files changed: 227 ins; 373 del; 38 mod

We should be able to select platform/device via env var

-------------

PR: https://git.openjdk.org/babylon/pull/355

From gfrost at openjdk.org  Thu Mar 13 15:27:13 2025
From: gfrost at openjdk.org (Gary Frost)
Date: Thu, 13 Mar 2025 15:27:13 GMT
Subject: git: openjdk/babylon: code-reflection: We should be able to select
 platform/device via env var
Message-ID: <d73bb6f3-d0ba-467d-bee3-06307b241b1a@openjdk.org>

Changeset: f93580a0
Branch: code-reflection
Author:    Gary Frost <gfrost at openjdk.org>
Date:      2025-03-13 15:23:24 +0000
URL:       https://git.openjdk.org/babylon/commit/f93580a0797179a39bc000f9ed56322e09d34185

We should be able to select platform/device via env var

! hat/backends/ffi/opencl/cpp/info.cpp
! hat/backends/ffi/opencl/cpp/opencl_backend.cpp
! hat/backends/ffi/opencl/include/opencl_backend.h
- hat/backends/ffi/opencl/src/main/java/hat/backend/ffi/Config.java
! hat/backends/ffi/opencl/src/main/java/hat/backend/ffi/OpenCLBackend.java
+ hat/backends/ffi/opencl/src/main/java/hat/backend/ffi/OpenCLConfig.java
! hat/examples/experiments/src/main/java/experiments/Mesh.java
! hat/examples/experiments/src/main/java/experiments/MinBufferTest.java
! hat/examples/nbody/src/main/java/nbody/opencl/OpenCLNBodyGLWindow.java


From gfrost at openjdk.org  Thu Mar 13 16:17:36 2025
From: gfrost at openjdk.org (Gary Frost)
Date: Thu, 13 Mar 2025 16:17:36 GMT
Subject: git: openjdk/babylon: code-reflection: Fixed bug selecting opencl
 device
Message-ID: <3df2bdb3-0995-4d1a-bee6-ddb269255a87@openjdk.org>

Changeset: e4ebcbb9
Branch: code-reflection
Author:    Gary Frost <gfrost at openjdk.org>
Date:      2025-03-13 16:15:52 +0000
URL:       https://git.openjdk.org/babylon/commit/e4ebcbb9f26e09d250444aaf24908a9e5f8b823a

Fixed bug selecting opencl device

! hat/backends/ffi/opencl/cpp/opencl_backend.cpp


From gfrost at openjdk.org  Thu Mar 13 16:21:46 2025
From: gfrost at openjdk.org (Gary Frost)
Date: Thu, 13 Mar 2025 16:21:46 GMT
Subject: [code-reflection] Integrated: Fixed bug selecting opencl device
Message-ID: <YGo-Ih_pfzV6oOf7kktSX7Yf0FgyYGKjHjr00dnlE8Q=.1ea6c3df-5f85-46de-ab3f-fe08d9e6a92d@github.com>

Found a bug on older mac trying to select opencl device. 

Was always selcting device 0  ;

-------------

Commit messages:
 - Fixed bug selecting opencl device

Changes: https://git.openjdk.org/babylon/pull/356/files
  Webrev: https://webrevs.openjdk.org/?repo=babylon&pr=356&range=00
  Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod
  Patch: https://git.openjdk.org/babylon/pull/356.diff
  Fetch: git fetch https://git.openjdk.org/babylon.git pull/356/head:pull/356

PR: https://git.openjdk.org/babylon/pull/356

From gfrost at openjdk.org  Thu Mar 13 16:21:47 2025
From: gfrost at openjdk.org (Gary Frost)
Date: Thu, 13 Mar 2025 16:21:47 GMT
Subject: [code-reflection] Integrated: Fixed bug selecting opencl device
In-Reply-To: <YGo-Ih_pfzV6oOf7kktSX7Yf0FgyYGKjHjr00dnlE8Q=.1ea6c3df-5f85-46de-ab3f-fe08d9e6a92d@github.com>
References: <YGo-Ih_pfzV6oOf7kktSX7Yf0FgyYGKjHjr00dnlE8Q=.1ea6c3df-5f85-46de-ab3f-fe08d9e6a92d@github.com>
Message-ID: <5EwQBiXf8eJHHD5qT5iP1EqbnPNmPvOrsg5tM3wGyn4=.457aa275-eb09-49e7-8e29-f40a89162216@github.com>

On Thu, 13 Mar 2025 16:13:55 GMT, Gary Frost <gfrost at openjdk.org> wrote:

> Found a bug on older mac trying to select opencl device. 
> 
> Was always selcting device 0  ;

This pull request has now been integrated.

Changeset: e4ebcbb9
Author:    Gary Frost <gfrost at openjdk.org>
URL:       https://git.openjdk.org/babylon/commit/e4ebcbb9f26e09d250444aaf24908a9e5f8b823a
Stats:     1 line in 1 file changed: 0 ins; 0 del; 1 mod

Fixed bug selecting opencl device

-------------

PR: https://git.openjdk.org/babylon/pull/356

From gfrost at openjdk.org  Thu Mar 13 16:38:23 2025
From: gfrost at openjdk.org (Gary Frost)
Date: Thu, 13 Mar 2025 16:38:23 GMT
Subject: git: openjdk/babylon: code-reflection: Fixed bug selecting opencl
 device - really this time
Message-ID: <a8ba1f67-34f8-4cc1-83d5-015d1fe130df@openjdk.org>

Changeset: b65b7ceb
Branch: code-reflection
Author:    Gary Frost <gfrost at openjdk.org>
Date:      2025-03-13 16:36:13 +0000
URL:       https://git.openjdk.org/babylon/commit/b65b7cebdc660ab7681647f19d4a9bdd2d6509eb

Fixed bug selecting opencl device - really this time

! hat/backends/ffi/opencl/cpp/opencl_backend.cpp


From gfrost at openjdk.org  Thu Mar 13 16:39:39 2025
From: gfrost at openjdk.org (Gary Frost)
Date: Thu, 13 Mar 2025 16:39:39 GMT
Subject: [code-reflection] Integrated: Fixed bug selecting opencl device -
 really this time
Message-ID: <gI5dVSrJSNd084J2eWGDMFAt5F6BdLs19m6Hj3AziCE=.6cbc8b2c-ce35-4465-9efe-24816455b56f@github.com>

Apologies. 

Last fix was not complete.  This should fix device selection

-------------

Commit messages:
 - Fixed bug selecting opencl device - really this time

Changes: https://git.openjdk.org/babylon/pull/357/files
  Webrev: https://webrevs.openjdk.org/?repo=babylon&pr=357&range=00
  Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod
  Patch: https://git.openjdk.org/babylon/pull/357.diff
  Fetch: git fetch https://git.openjdk.org/babylon.git pull/357/head:pull/357

PR: https://git.openjdk.org/babylon/pull/357

From gfrost at openjdk.org  Thu Mar 13 16:39:39 2025
From: gfrost at openjdk.org (Gary Frost)
Date: Thu, 13 Mar 2025 16:39:39 GMT
Subject: [code-reflection] Integrated: Fixed bug selecting opencl device -
 really this time
In-Reply-To: <gI5dVSrJSNd084J2eWGDMFAt5F6BdLs19m6Hj3AziCE=.6cbc8b2c-ce35-4465-9efe-24816455b56f@github.com>
References: <gI5dVSrJSNd084J2eWGDMFAt5F6BdLs19m6Hj3AziCE=.6cbc8b2c-ce35-4465-9efe-24816455b56f@github.com>
Message-ID: <nfLw3pv2fTN9mYjECwxfKpccncvPHF0X0D38RLrHsRg=.2e915910-ae05-491c-86a6-a3a515c3e73c@github.com>

On Thu, 13 Mar 2025 16:33:55 GMT, Gary Frost <gfrost at openjdk.org> wrote:

> Apologies. 
> 
> Last fix was not complete.  This should fix device selection

This pull request has now been integrated.

Changeset: b65b7ceb
Author:    Gary Frost <gfrost at openjdk.org>
URL:       https://git.openjdk.org/babylon/commit/b65b7cebdc660ab7681647f19d4a9bdd2d6509eb
Stats:     1 line in 1 file changed: 0 ins; 0 del; 1 mod

Fixed bug selecting opencl device - really this time

-------------

PR: https://git.openjdk.org/babylon/pull/357

From asotona at openjdk.org  Thu Mar 13 17:16:49 2025
From: asotona at openjdk.org (Adam Sotona)
Date: Thu, 13 Mar 2025 17:16:49 GMT
Subject: [code-reflection] RFR: ExplicitOnnxOps.If implementation of captured
 values and initializers
Message-ID: <FRtr7UEqDyj-b0wRtas8cd5vqIS8dQ3vjzXY9P1unWY=.03959780-6ce3-472e-a121-77b9efcbc469@github.com>

Deconstruction of `OnnxTransformer` into multiple steps with support for captured values and initializers in nested bodies of `If` op.

-------------

Commit messages:
 - nit change
 - fixed initializers referenced from nested bodies
 - OnnxTransformer and OnnxPartialEvaluator refactoring to support If operator with nested bodies - work in progress
 - OnnxTransformer refactoring
 - if - work in progress

Changes: https://git.openjdk.org/babylon/pull/353/files
  Webrev: https://webrevs.openjdk.org/?repo=babylon&pr=353&range=00
  Stats: 557 lines in 8 files changed: 177 ins; 284 del; 96 mod
  Patch: https://git.openjdk.org/babylon/pull/353.diff
  Fetch: git fetch https://git.openjdk.org/babylon.git pull/353/head:pull/353

PR: https://git.openjdk.org/babylon/pull/353

From asotona at openjdk.org  Thu Mar 13 17:22:31 2025
From: asotona at openjdk.org (Adam Sotona)
Date: Thu, 13 Mar 2025 17:22:31 GMT
Subject: [code-reflection] RFR: ExplicitOnnxOps.If implementation of
 captured values and initializers [v2]
In-Reply-To: <FRtr7UEqDyj-b0wRtas8cd5vqIS8dQ3vjzXY9P1unWY=.03959780-6ce3-472e-a121-77b9efcbc469@github.com>
References: <FRtr7UEqDyj-b0wRtas8cd5vqIS8dQ3vjzXY9P1unWY=.03959780-6ce3-472e-a121-77b9efcbc469@github.com>
Message-ID: <fi8HUoaWQaiNuDOjyLO2JnalNQg0GnKY3pLUY_i0FXo=.79677dd9-890e-40ac-aaf1-c3eb9ac0e7b4@github.com>

> Deconstruction of `OnnxTransformer` into multiple steps with support for captured values and initializers in nested bodies of `If` op.

Adam Sotona has updated the pull request incrementally with one additional commit since the last revision:

  improved test

-------------

Changes:
  - all: https://git.openjdk.org/babylon/pull/353/files
  - new: https://git.openjdk.org/babylon/pull/353/files/8aa07ecc..c135238e

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=babylon&pr=353&range=01
 - incr: https://webrevs.openjdk.org/?repo=babylon&pr=353&range=00-01

  Stats: 21 lines in 1 file changed: 14 ins; 2 del; 5 mod
  Patch: https://git.openjdk.org/babylon/pull/353.diff
  Fetch: git fetch https://git.openjdk.org/babylon.git pull/353/head:pull/353

PR: https://git.openjdk.org/babylon/pull/353

From asotona at openjdk.org  Fri Mar 14 15:15:13 2025
From: asotona at openjdk.org (Adam Sotona)
Date: Fri, 14 Mar 2025 15:15:13 GMT
Subject: git: openjdk/babylon: code-reflection: ExplicitOnnxOps.If
 implementation of captured values and initializers
Message-ID: <cdc4dd1f-4c69-45d8-adf9-64130936bac1@openjdk.org>

Changeset: e1d4ad80
Branch: code-reflection
Author:    Adam Sotona <asotona at openjdk.org>
Date:      2025-03-14 15:11:43 +0000
URL:       https://git.openjdk.org/babylon/commit/e1d4ad8030acdcd1082e059080d979eb7e8ffdb8

ExplicitOnnxOps.If implementation of captured values and initializers

- cr-examples/onnx/src/main/java/oracle/code/onnx/LambdaToFunc.java
! cr-examples/onnx/src/main/java/oracle/code/onnx/OnnxProtoBuilder.java
! cr-examples/onnx/src/main/java/oracle/code/onnx/OnnxRuntime.java
! cr-examples/onnx/src/main/java/oracle/code/onnx/compiler/OnnxPartialEvaluator.java
! cr-examples/onnx/src/main/java/oracle/code/onnx/compiler/OnnxTransformer.java
! cr-examples/onnx/src/test/java/oracle/code/onnx/CNNTest.java
! cr-examples/onnx/src/test/java/oracle/code/onnx/SimpleTest.java
! src/jdk.incubator.code/share/classes/jdk/incubator/code/Block.java


From asotona at openjdk.org  Fri Mar 14 15:15:27 2025
From: asotona at openjdk.org (Adam Sotona)
Date: Fri, 14 Mar 2025 15:15:27 GMT
Subject: [code-reflection] Integrated: ExplicitOnnxOps.If implementation of
 captured values and initializers
In-Reply-To: <FRtr7UEqDyj-b0wRtas8cd5vqIS8dQ3vjzXY9P1unWY=.03959780-6ce3-472e-a121-77b9efcbc469@github.com>
References: <FRtr7UEqDyj-b0wRtas8cd5vqIS8dQ3vjzXY9P1unWY=.03959780-6ce3-472e-a121-77b9efcbc469@github.com>
Message-ID: <PewtKhCLRHyYpFoKy-9diZN4Pyg29U46OjAsSZ8iGRs=.aa6c4fbe-3dac-4632-9435-913a49fe9e9b@github.com>

On Wed, 12 Mar 2025 19:13:29 GMT, Adam Sotona <asotona at openjdk.org> wrote:

> Deconstruction of `OnnxTransformer` into multiple steps with support for captured values and initializers in nested bodies of `If` op.

This pull request has now been integrated.

Changeset: e1d4ad80
Author:    Adam Sotona <asotona at openjdk.org>
URL:       https://git.openjdk.org/babylon/commit/e1d4ad8030acdcd1082e059080d979eb7e8ffdb8
Stats:     569 lines in 8 files changed: 189 ins; 284 del; 96 mod

ExplicitOnnxOps.If implementation of captured values and initializers

-------------

PR: https://git.openjdk.org/babylon/pull/353

From asotona at openjdk.org  Fri Mar 14 19:33:53 2025
From: asotona at openjdk.org (Adam Sotona)
Date: Fri, 14 Mar 2025 19:33:53 GMT
Subject: [code-reflection] RFR: Revert ExplicitOnnxOps.If implementation of
 captured values and initializers
Message-ID: <QVFjTDwbe-CuUATMgMgSK7i65iZVe6QVMexbBD6akbs=.0e747d67-7d6f-4a4c-b407-d8073338f89a@github.com>

This reverts commit e1d4ad8030acdcd1082e059080d979eb7e8ffdb8.

-------------

Commit messages:
 - Revert "ExplicitOnnxOps.If implementation of captured values and initializers"

Changes: https://git.openjdk.org/babylon/pull/358/files
  Webrev: https://webrevs.openjdk.org/?repo=babylon&pr=358&range=00
  Stats: 547 lines in 8 files changed: 262 ins; 167 del; 118 mod
  Patch: https://git.openjdk.org/babylon/pull/358.diff
  Fetch: git fetch https://git.openjdk.org/babylon.git pull/358/head:pull/358

PR: https://git.openjdk.org/babylon/pull/358

From asotona at openjdk.org  Fri Mar 14 19:38:12 2025
From: asotona at openjdk.org (Adam Sotona)
Date: Fri, 14 Mar 2025 19:38:12 GMT
Subject: git: openjdk/babylon: code-reflection: Revert ExplicitOnnxOps.If
 implementation of captured values and initializers
Message-ID: <348f99aa-7ea4-43bd-b398-e43a3fd82fdd@openjdk.org>

Changeset: 150f09cd
Branch: code-reflection
Author:    Adam Sotona <asotona at openjdk.org>
Date:      2025-03-14 19:36:01 +0000
URL:       https://git.openjdk.org/babylon/commit/150f09cddf365b0a0bef5244c507ebcf49b5a26d

Revert ExplicitOnnxOps.If implementation of captured values and initializers

+ cr-examples/onnx/src/main/java/oracle/code/onnx/LambdaToFunc.java
! cr-examples/onnx/src/main/java/oracle/code/onnx/OnnxProtoBuilder.java
! cr-examples/onnx/src/main/java/oracle/code/onnx/OnnxRuntime.java
! cr-examples/onnx/src/main/java/oracle/code/onnx/compiler/OnnxPartialEvaluator.java
! cr-examples/onnx/src/main/java/oracle/code/onnx/compiler/OnnxTransformer.java
! cr-examples/onnx/src/test/java/oracle/code/onnx/CNNTest.java
! cr-examples/onnx/src/test/java/oracle/code/onnx/SimpleTest.java
! src/jdk.incubator.code/share/classes/jdk/incubator/code/Block.java


From asotona at openjdk.org  Fri Mar 14 19:39:05 2025
From: asotona at openjdk.org (Adam Sotona)
Date: Fri, 14 Mar 2025 19:39:05 GMT
Subject: [code-reflection] Integrated: Revert ExplicitOnnxOps.If implementation
 of captured values and initializers
In-Reply-To: <QVFjTDwbe-CuUATMgMgSK7i65iZVe6QVMexbBD6akbs=.0e747d67-7d6f-4a4c-b407-d8073338f89a@github.com>
References: <QVFjTDwbe-CuUATMgMgSK7i65iZVe6QVMexbBD6akbs=.0e747d67-7d6f-4a4c-b407-d8073338f89a@github.com>
Message-ID: <oi6UVnqmST755PriTWuu-ctsArldf2klIkvHvJf8Q5o=.f8355b85-d22a-4884-a438-cec9ef1f5ddf@github.com>

On Fri, 14 Mar 2025 19:28:37 GMT, Adam Sotona <asotona at openjdk.org> wrote:

> This reverts commit e1d4ad8030acdcd1082e059080d979eb7e8ffdb8.

This pull request has now been integrated.

Changeset: 150f09cd
Author:    Adam Sotona <asotona at openjdk.org>
URL:       https://git.openjdk.org/babylon/commit/150f09cddf365b0a0bef5244c507ebcf49b5a26d
Stats:     547 lines in 8 files changed: 262 ins; 167 del; 118 mod

Revert ExplicitOnnxOps.If implementation of captured values and initializers

-------------

PR: https://git.openjdk.org/babylon/pull/358

From gfrost at openjdk.org  Sat Mar 15 01:04:45 2025
From: gfrost at openjdk.org (Gary Frost)
Date: Sat, 15 Mar 2025 01:04:45 GMT
Subject: git: openjdk/babylon: code-reflection: We are switching to use hat
 subdir (instead of bldr) for scripts
Message-ID: <cfe0254b-abda-46e2-8e88-0d7ca96ece15@openjdk.org>

Changeset: a3bd45c5
Branch: code-reflection
Author:    Gary Frost <gfrost at openjdk.org>
Date:      2025-03-15 01:02:38 +0000
URL:       https://git.openjdk.org/babylon/commit/a3bd45c570a94ee4dcd11fbe54d47f60b57a823d

We are switching to use hat subdir (instead of bldr) for scripts

! hat/backends/ffi/cuda/src/main/java/hat/backend/ffi/CudaBackend.java
! hat/backends/ffi/mock/src/main/java/hat/backend/ffi/MockBackend.java
! hat/backends/ffi/opencl/src/main/java/hat/backend/ffi/OpenCLBackend.java
! hat/backends/ffi/opencl/src/main/java/hat/backend/ffi/OpenCLConfig.java
! hat/backends/ffi/ptx/src/main/java/hat/backend/ffi/PTXBackend.java
! hat/backends/jextracted/opencl/src/main/java/hat/backend/jextracted/OpenCLBackend.java
! hat/docs/hat-01-01-project-layout.md
! hat/docs/hat-01-03-building-hat-with-maven.md
! hat/docs/hat-01-03-building-hat.md
! hat/examples/experiments/src/main/java/experiments/Mesh.java
! hat/examples/experiments/src/main/java/experiments/MinBufferTest.java
! hat/examples/life/src/main/java/life/Main.java
! hat/hat-core/src/main/java/hat/backend/ffi/FFIBackend.java
= hat/hat/Script.java
+ hat/hat/bld
+ hat/hat/bld.java
+ hat/hat/clean
= hat/hat/clean.java
+ hat/hat/mkpoms
= hat/hat/mkpoms.java
+ hat/hat/run
+ hat/hat/run-ffi-opencl-life
= hat/hat/run.java
+ hat/hat/sanity
= hat/hat/sanity.java
= hat/hat/scriptformat.xml
= hat/hat/tst
! hat/intellij/.idea/compiler.xml
! hat/intellij/.idea/modules.xml
! hat/intellij/hat-core.iml


From gfrost at openjdk.org  Sat Mar 15 01:06:03 2025
From: gfrost at openjdk.org (Gary Frost)
Date: Sat, 15 Mar 2025 01:06:03 GMT
Subject: [code-reflection] Integrated: We are switching to use hat subdir
 (instead of bldr) for scripts
Message-ID: <jBckVJYHlKefQn-ss5vRTte7PIVGE4Ml8s2WMeUEx2M=.8c6e96f5-926d-4928-a2d6-c9e34f465ec1@github.com>

Previously we used scripts in bldr  subdir for various tasks.  This made sense when it was a bld script ;)  but not so muhc for running, sanity checks etc.  

Now we have switched to using the hat subdir. 

So previously 

java @bldr/sanity
java @bldr/bld
java @bldr/hatrun ffi-opencl life 


Now 

java @hat/sanity
java @hat/bld
java @hat/run ffi-opencl life

-------------

Commit messages:
 - We are switching to use hat subdir (instead of bldr) for scripts

Changes: https://git.openjdk.org/babylon/pull/360/files
  Webrev: https://webrevs.openjdk.org/?repo=babylon&pr=360&range=00
  Stats: 480 lines in 30 files changed: 379 ins; 13 del; 88 mod
  Patch: https://git.openjdk.org/babylon/pull/360.diff
  Fetch: git fetch https://git.openjdk.org/babylon.git pull/360/head:pull/360

PR: https://git.openjdk.org/babylon/pull/360

From gfrost at openjdk.org  Sat Mar 15 01:06:05 2025
From: gfrost at openjdk.org (Gary Frost)
Date: Sat, 15 Mar 2025 01:06:05 GMT
Subject: [code-reflection] Integrated: We are switching to use hat subdir
 (instead of bldr) for scripts
In-Reply-To: <jBckVJYHlKefQn-ss5vRTte7PIVGE4Ml8s2WMeUEx2M=.8c6e96f5-926d-4928-a2d6-c9e34f465ec1@github.com>
References: <jBckVJYHlKefQn-ss5vRTte7PIVGE4Ml8s2WMeUEx2M=.8c6e96f5-926d-4928-a2d6-c9e34f465ec1@github.com>
Message-ID: <PGv7H6wcnVS2kp4VrsAEja52vX0D1gtL5fKROY4-Udk=.a4018c18-e6f7-45ca-a633-667a75318529@github.com>

On Sat, 15 Mar 2025 01:00:46 GMT, Gary Frost <gfrost at openjdk.org> wrote:

> Previously we used scripts in bldr  subdir for various tasks.  This made sense when it was a bld script ;)  but not so muhc for running, sanity checks etc.  
> 
> Now we have switched to using the hat subdir. 
> 
> So previously 
> 
> java @bldr/sanity
> java @bldr/bld
> java @bldr/hatrun ffi-opencl life 
> 
> 
> Now 
> 
> java @hat/sanity
> java @hat/bld
> java @hat/run ffi-opencl life

This pull request has now been integrated.

Changeset: a3bd45c5
Author:    Gary Frost <gfrost at openjdk.org>
URL:       https://git.openjdk.org/babylon/commit/a3bd45c570a94ee4dcd11fbe54d47f60b57a823d
Stats:     480 lines in 30 files changed: 379 ins; 13 del; 88 mod

We are switching to use hat subdir (instead of bldr) for scripts

-------------

PR: https://git.openjdk.org/babylon/pull/360

From gfrost at openjdk.org  Sat Mar 15 01:19:51 2025
From: gfrost at openjdk.org (Gary Frost)
Date: Sat, 15 Mar 2025 01:19:51 GMT
Subject: git: openjdk/babylon: code-reflection: Finalize switch to using hat
 subdir insteaf of bldr subdir
Message-ID: <e29c61f1-c726-4c29-b80f-4f9c8151c3c6@openjdk.org>

Changeset: 5856c8d3
Branch: code-reflection
Author:    Gary Frost <gfrost at openjdk.org>
Date:      2025-03-15 01:17:04 +0000
URL:       https://git.openjdk.org/babylon/commit/5856c8d39a19ce3e29fa3b8b140f827262b88ac6

Finalize switch to using hat subdir insteaf of bldr subdir

- hat/bld
- hat/bldr/.gitignore
- hat/bldr/Bldr.java
- hat/bldr/args
- hat/bldr/bld
- hat/bldr/clean
- hat/bldr/hatlessrun
- hat/bldr/hatrun
- hat/bldr/java
- hat/bldr/mkpoms
- hat/bldr/opencl
- hat/bldr/sanity
- hat/bldr/scriptformat.xml
- hat/bldr/tst
- hat/clean
! hat/hat/mkpoms.java
- hat/hatrun
+ hat/intellij/hat.iml
- hat/mkpoms
- hat/tst/MinBufferTest.java


From gfrost at openjdk.org  Sat Mar 15 01:20:36 2025
From: gfrost at openjdk.org (Gary Frost)
Date: Sat, 15 Mar 2025 01:20:36 GMT
Subject: [code-reflection] Integrated: Finalize switch to using hat subdir
 insteaf of bldr subdir
Message-ID: <JoY7LT1nSxM7-v5aOGtte5W51CI9uIcScrE426kHkCg=.42069f67-c273-49bf-ae7e-a387ccd6d018@github.com>

Finalized move to using hat subdir for scripts rather than bldr subdir

-------------

Commit messages:
 - Finalize switch to using hat subdir insteaf of bldr subdir

Changes: https://git.openjdk.org/babylon/pull/361/files
  Webrev: https://webrevs.openjdk.org/?repo=babylon&pr=361&range=00
  Stats: 5400 lines in 20 files changed: 12 ins; 5378 del; 10 mod
  Patch: https://git.openjdk.org/babylon/pull/361.diff
  Fetch: git fetch https://git.openjdk.org/babylon.git pull/361/head:pull/361

PR: https://git.openjdk.org/babylon/pull/361

From gfrost at openjdk.org  Sat Mar 15 01:20:37 2025
From: gfrost at openjdk.org (Gary Frost)
Date: Sat, 15 Mar 2025 01:20:37 GMT
Subject: [code-reflection] Integrated: Finalize switch to using hat subdir
 insteaf of bldr subdir
In-Reply-To: <JoY7LT1nSxM7-v5aOGtte5W51CI9uIcScrE426kHkCg=.42069f67-c273-49bf-ae7e-a387ccd6d018@github.com>
References: <JoY7LT1nSxM7-v5aOGtte5W51CI9uIcScrE426kHkCg=.42069f67-c273-49bf-ae7e-a387ccd6d018@github.com>
Message-ID: <jnXSKa096XOujiEP3tg817aqH7hwP6nb5s60X4RGQOI=.42024872-7bd8-4cfc-bfae-e39b5040821f@github.com>

On Sat, 15 Mar 2025 01:15:04 GMT, Gary Frost <gfrost at openjdk.org> wrote:

> Finalized move to using hat subdir for scripts rather than bldr subdir

This pull request has now been integrated.

Changeset: 5856c8d3
Author:    Gary Frost <gfrost at openjdk.org>
URL:       https://git.openjdk.org/babylon/commit/5856c8d39a19ce3e29fa3b8b140f827262b88ac6
Stats:     5400 lines in 20 files changed: 12 ins; 5378 del; 10 mod

Finalize switch to using hat subdir insteaf of bldr subdir

-------------

PR: https://git.openjdk.org/babylon/pull/361

From asotona at openjdk.org  Sat Mar 15 01:22:30 2025
From: asotona at openjdk.org (Adam Sotona)
Date: Sat, 15 Mar 2025 01:22:30 GMT
Subject: [code-reflection] RFR: ExplicitOnnxOps.If implementation of captured
 values and initializers
Message-ID: <9pGY7kCbBiYjrFHzelwweejUpCFEzlEQeXLLR0HUYuU=.21c0b04d-42a1-4cd0-a185-d2e2c296384b@github.com>

ExplicitOnnxOps.If implementation of captured values and initializers - another attempt to do it right

-------------

Commit messages:
 - removed debug line
 - Fix of the OnnxTransformer
 - Re-application of the problematic ExplicitOnnxOps.If implementation of captured values and initializers

Changes: https://git.openjdk.org/babylon/pull/359/files
  Webrev: https://webrevs.openjdk.org/?repo=babylon&pr=359&range=00
  Stats: 558 lines in 7 files changed: 182 ins; 285 del; 91 mod
  Patch: https://git.openjdk.org/babylon/pull/359.diff
  Fetch: git fetch https://git.openjdk.org/babylon.git pull/359/head:pull/359

PR: https://git.openjdk.org/babylon/pull/359

From mabbay at openjdk.org  Sat Mar 15 02:29:05 2025
From: mabbay at openjdk.org (Mourad Abbay)
Date: Sat, 15 Mar 2025 02:29:05 GMT
Subject: [code-reflection] RFR: Support storing the code that builds the
 code model [v18]
In-Reply-To: <23hI2EWkXS2p8UsVzNiJ-nlSKnWXSquYQTbcg4nXyHA=.a2b260e2-906e-4f70-82d6-d52b8750568d@github.com>
References: <dexv8sQV_NcJsJIgMopd_M4wSWoZya9dU0ACwDtEL0E=.4ac0b80f-24fb-440f-a18d-bbfd07a681bc@github.com>
 <rLtW9Wv92MNgUIywKaiPdoBs8O5-5EXNlu17J8tRflk=.3e0aa614-55db-4e86-a5a0-99e8e51758d8@github.com>
 <23hI2EWkXS2p8UsVzNiJ-nlSKnWXSquYQTbcg4nXyHA=.a2b260e2-906e-4f70-82d6-d52b8750568d@github.com>
Message-ID: <1IWfCOyiFaE_quT8UXfTpunJF9GlLsCvKtJaSZS1JEs=.2529482d-0064-4c51-8dd8-99181ee391d8@github.com>

On Thu, 13 Mar 2025 13:17:47 GMT, Maurizio Cimadamore <mcimadamore at openjdk.org> wrote:

> I've verified that all tests pass, except the two test which fail with a compiler crash because of #351.

If we merge 351 into this, those two tests will pass.

-------------

PR Comment: https://git.openjdk.org/babylon/pull/305#issuecomment-2726131079

From mabbay at openjdk.org  Sat Mar 15 02:41:08 2025
From: mabbay at openjdk.org (Mourad Abbay)
Date: Sat, 15 Mar 2025 02:41:08 GMT
Subject: [code-reflection] RFR: Support storing the code that builds the
 code model [v18]
In-Reply-To: <UZ2Z1W9PyhkEv4ucTxh-zfBISS__-0xiY5PGL4p67qw=.9a8dfbfd-3809-4fba-891f-543bead72d6b@github.com>
References: <dexv8sQV_NcJsJIgMopd_M4wSWoZya9dU0ACwDtEL0E=.4ac0b80f-24fb-440f-a18d-bbfd07a681bc@github.com>
 <rLtW9Wv92MNgUIywKaiPdoBs8O5-5EXNlu17J8tRflk=.3e0aa614-55db-4e86-a5a0-99e8e51758d8@github.com>
 <UZ2Z1W9PyhkEv4ucTxh-zfBISS__-0xiY5PGL4p67qw=.9a8dfbfd-3809-4fba-891f-543bead72d6b@github.com>
Message-ID: <_vyINIwx1fIfdBtHZYt9VgABP7AIE8Q2bcsFaprRJAc=.2b84048a-69b1-4576-aca0-86caf814b280@github.com>

On Thu, 13 Mar 2025 13:27:28 GMT, Maurizio Cimadamore <mcimadamore at openjdk.org> wrote:

>> Mourad Abbay has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains 20 additional commits since the last revision:
>> 
>>  - Merge branch 'code-reflection' into code-model-storage-option
>>  - Load OpFactory and TypeElementFactory before invocation of opMethod only if the opMethod has there params.
>>  - Fix the remaining compiler tests failures
>>  - Fix some of the test failures (3 remains)
>>  - Fix the remaining test failures of SwitchExpressionTest2
>>  - Fix almost all test failures of SwitchExpressionTest2 (one remaining)
>>  - Pass arrayType instead of eleType in OpBuilder.buildArray
>>  - Ensure that block params are inserted in the correct order
>>  - Add missing imports
>>  - Merge branch 'code-reflection' into code-model-storage-option
>>  - ... and 10 more: https://git.openjdk.org/babylon/compare/c9d98dd5...966005ce
>
> src/java.base/share/classes/java/lang/invoke/InnerClassLambdaMetafactory.java line 453:
> 
>> 451:         }
>> 452:         if (quotableOpGetterInfo.getMethodType().parameterList().equals(List.of(CodeReflectionSupport.OP_FACTORY_CLASS,
>> 453:                 CodeReflectionSupport.TYPE_ELEMENT_FACTORY_CLASS))) {
> 
> If these fields are public/static, why don't we load them from inside the compiler-generated method, instead of injecting them here?

I am assuming by the compiler-generated method you mean `opMethod`. The `opMethod` has two params (OpFactory, TypeElementFactory), so loading these fields is necessary before invoking it.

-------------

PR Review Comment: https://git.openjdk.org/babylon/pull/305#discussion_r1996522797

From mabbay at openjdk.org  Sat Mar 15 02:45:04 2025
From: mabbay at openjdk.org (Mourad Abbay)
Date: Sat, 15 Mar 2025 02:45:04 GMT
Subject: [code-reflection] RFR: Support storing the code that builds the
 code model [v18]
In-Reply-To: <_vyINIwx1fIfdBtHZYt9VgABP7AIE8Q2bcsFaprRJAc=.2b84048a-69b1-4576-aca0-86caf814b280@github.com>
References: <dexv8sQV_NcJsJIgMopd_M4wSWoZya9dU0ACwDtEL0E=.4ac0b80f-24fb-440f-a18d-bbfd07a681bc@github.com>
 <rLtW9Wv92MNgUIywKaiPdoBs8O5-5EXNlu17J8tRflk=.3e0aa614-55db-4e86-a5a0-99e8e51758d8@github.com>
 <UZ2Z1W9PyhkEv4ucTxh-zfBISS__-0xiY5PGL4p67qw=.9a8dfbfd-3809-4fba-891f-543bead72d6b@github.com>
 <_vyINIwx1fIfdBtHZYt9VgABP7AIE8Q2bcsFaprRJAc=.2b84048a-69b1-4576-aca0-86caf814b280@github.com>
Message-ID: <8vDjuCkLBgFFnPK_tYycS420iPcfeMK9JqtsStKzTp8=.66587d94-c595-4d24-9c1d-b787867e71b3@github.com>

On Sat, 15 Mar 2025 02:38:37 GMT, Mourad Abbay <mabbay at openjdk.org> wrote:

>> src/java.base/share/classes/java/lang/invoke/InnerClassLambdaMetafactory.java line 453:
>> 
>>> 451:         }
>>> 452:         if (quotableOpGetterInfo.getMethodType().parameterList().equals(List.of(CodeReflectionSupport.OP_FACTORY_CLASS,
>>> 453:                 CodeReflectionSupport.TYPE_ELEMENT_FACTORY_CLASS))) {
>> 
>> If these fields are public/static, why don't we load them from inside the compiler-generated method, instead of injecting them here?
>
> I am assuming by the compiler-generated method you mean `opMethod`. The `opMethod` has two params (OpFactory, TypeElementFactory), so loading these fields is necessary before invoking it.

We can turn opMethod into one with no param and load these fields inside its body. But this removes the flexibility we have now (i.e. we can choose the OpFactory and TypeElementFactory we want to pass).

-------------

PR Review Comment: https://git.openjdk.org/babylon/pull/305#discussion_r1996523587

From mabbay at openjdk.org  Sat Mar 15 02:53:03 2025
From: mabbay at openjdk.org (Mourad Abbay)
Date: Sat, 15 Mar 2025 02:53:03 GMT
Subject: [code-reflection] RFR: Support storing the code that builds the
 code model [v18]
In-Reply-To: <fz8Kzl5g5McZSRm9TMVLMzYUIdGHREPgher1NlND_s0=.bf74f1db-6407-4ace-a198-5a4a662d8bc5@github.com>
References: <dexv8sQV_NcJsJIgMopd_M4wSWoZya9dU0ACwDtEL0E=.4ac0b80f-24fb-440f-a18d-bbfd07a681bc@github.com>
 <rLtW9Wv92MNgUIywKaiPdoBs8O5-5EXNlu17J8tRflk=.3e0aa614-55db-4e86-a5a0-99e8e51758d8@github.com>
 <fz8Kzl5g5McZSRm9TMVLMzYUIdGHREPgher1NlND_s0=.bf74f1db-6407-4ace-a198-5a4a662d8bc5@github.com>
Message-ID: <aFPas3hq19r_N9bOarGsj35ATQOEwiCEjQxSCN2Q-kk=.40043f20-2a85-426d-b546-abee7464bb41@github.com>

On Thu, 13 Mar 2025 14:29:17 GMT, Maurizio Cimadamore <mcimadamore at openjdk.org> wrote:

>> Mourad Abbay has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains 20 additional commits since the last revision:
>> 
>>  - Merge branch 'code-reflection' into code-model-storage-option
>>  - Load OpFactory and TypeElementFactory before invocation of opMethod only if the opMethod has there params.
>>  - Fix the remaining compiler tests failures
>>  - Fix some of the test failures (3 remains)
>>  - Fix the remaining test failures of SwitchExpressionTest2
>>  - Fix almost all test failures of SwitchExpressionTest2 (one remaining)
>>  - Pass arrayType instead of eleType in OpBuilder.buildArray
>>  - Ensure that block params are inserted in the correct order
>>  - Add missing imports
>>  - Merge branch 'code-reflection' into code-model-storage-option
>>  - ... and 10 more: https://git.openjdk.org/babylon/compare/9300a6df...966005ce
>
> src/jdk.incubator.code/share/classes/jdk/incubator/code/internal/CodeModelToAST.java line 222:
> 
>> 220:         // TODO add VarOps in OpBuilder
>> 221:         funcOp = addVarsWhenNecessary(funcOp);
>> 222:         funcOp.writeTo(System.out);
> 
> This should be removed?

`addVarsWhenNecessary` is used to add necessary VarOp in the code model before transformation to AST. For example if an op result is used more than once, we introduce a VarOp to hold the value, because that's what the equivalent Java code will have. This makes the transformation from code model to AST a bit easier.
We also use it to in case we want an Op (e.g. InvokeOp) to be appended right away to the method we are building. As you now the block of a method is a list of statements, some operations maps to an expression, so they won't be appended until needed.

-------------

PR Review Comment: https://git.openjdk.org/babylon/pull/305#discussion_r1996526001

From mabbay at openjdk.org  Sat Mar 15 02:53:03 2025
From: mabbay at openjdk.org (Mourad Abbay)
Date: Sat, 15 Mar 2025 02:53:03 GMT
Subject: [code-reflection] RFR: Support storing the code that builds the
 code model [v18]
In-Reply-To: <aFPas3hq19r_N9bOarGsj35ATQOEwiCEjQxSCN2Q-kk=.40043f20-2a85-426d-b546-abee7464bb41@github.com>
References: <dexv8sQV_NcJsJIgMopd_M4wSWoZya9dU0ACwDtEL0E=.4ac0b80f-24fb-440f-a18d-bbfd07a681bc@github.com>
 <rLtW9Wv92MNgUIywKaiPdoBs8O5-5EXNlu17J8tRflk=.3e0aa614-55db-4e86-a5a0-99e8e51758d8@github.com>
 <fz8Kzl5g5McZSRm9TMVLMzYUIdGHREPgher1NlND_s0=.bf74f1db-6407-4ace-a198-5a4a662d8bc5@github.com>
 <aFPas3hq19r_N9bOarGsj35ATQOEwiCEjQxSCN2Q-kk=.40043f20-2a85-426d-b546-abee7464bb41@github.com>
Message-ID: <PF-vWBtGLibTbYhRg0BvK-Au3ibIl6TQJfspXNr_O0U=.848ee0b6-dbae-472c-82a9-64536499d4db@github.com>

On Sat, 15 Mar 2025 02:48:49 GMT, Mourad Abbay <mabbay at openjdk.org> wrote:

>> src/jdk.incubator.code/share/classes/jdk/incubator/code/internal/CodeModelToAST.java line 222:
>> 
>>> 220:         // TODO add VarOps in OpBuilder
>>> 221:         funcOp = addVarsWhenNecessary(funcOp);
>>> 222:         funcOp.writeTo(System.out);
>> 
>> This should be removed?
>
> `addVarsWhenNecessary` is used to add necessary VarOp in the code model before transformation to AST. For example if an op result is used more than once, we introduce a VarOp to hold the value, because that's what the equivalent Java code will have. This makes the transformation from code model to AST a bit easier.
> We also use it to in case we want an Op (e.g. InvokeOp) to be appended right away to the method we are building. As you now the block of a method is a list of statements, some operations maps to an expression, so they won't be appended until needed.

Normally, the OpBuilder should produces a model that contains the VarOp when they should be. But I decided to have a seperate transformation as a quick fix. Later I will change the OpBuilder.

-------------

PR Review Comment: https://git.openjdk.org/babylon/pull/305#discussion_r1996526723

From gfrost at openjdk.org  Sat Mar 15 17:17:20 2025
From: gfrost at openjdk.org (Gary Frost)
Date: Sat, 15 Mar 2025 17:17:20 GMT
Subject: git: openjdk/babylon: code-reflection: Hat fix min buffer state
 machine
Message-ID: <726e30f0-844c-40a9-b454-7795f71c8b46@openjdk.org>

Changeset: a26f48c6
Branch: code-reflection
Author:    Gary Frost <gfrost at openjdk.org>
Date:      2025-03-15 17:15:18 +0000
URL:       https://git.openjdk.org/babylon/commit/a26f48c6a1d67e48532a2671d7560641efb92432

Hat fix min buffer state machine

! hat/backends/ffi/opencl/CMakeLists.txt
! hat/backends/ffi/opencl/cpp/opencl_backend.cpp
! hat/backends/ffi/opencl/cpp/opencl_backend_kernel_dispatch.cpp
! hat/backends/ffi/opencl/include/opencl_backend.h
! hat/backends/ffi/opencl/src/main/java/hat/backend/ffi/OpenCLBackend.java
! hat/backends/ffi/opencl/src/main/java/hat/backend/ffi/OpenCLConfig.java
! hat/backends/ffi/shared/cpp/shared.cpp
! hat/backends/ffi/shared/include/shared.h
! hat/examples/life/src/main/java/life/Main.java
! hat/examples/life/src/main/java/life/Viewer.java
! hat/examples/nbody/src/main/java/nbody/opencl/OpenCLNBodyGLWindow.java
! hat/hat-core/src/main/java/hat/backend/ffi/FFIBackendDriver.java
! hat/hat-core/src/main/java/hat/buffer/Buffer.java
+ hat/hat-core/src/main/java/hat/ifacemapper/BufferState.java
! hat/hat-core/src/main/java/hat/ifacemapper/SegmentMapper.java
! hat/hat/Script.java
! hat/hat/mkpoms.java
+ hat/hat/run-java
+ hat/hat/run-opencl
! hat/wrap/clwrap/src/main/java/wrap/clwrap/CLPlatform.java


From gfrost at openjdk.org  Sat Mar 15 17:18:24 2025
From: gfrost at openjdk.org (Gary Frost)
Date: Sat, 15 Mar 2025 17:18:24 GMT
Subject: [code-reflection] Integrated: Hat fix min buffer state machine
Message-ID: <JDgFVuNQKnhwBJWvOUTJUboIFu9h5M12Ji83zj6qHhQ=.0b9c53c7-89ba-4ccf-98bb-1bf7142aac6c@github.com>

Swapped out the bitfield based states for buffer management to one using a state machine. 

Seems to be working.

-------------

Commit messages:
 - State machine to avoid buffer transfers seems to be working
 - Added state to buffer bits
 - Should copy can access config directly
 - Moved shouldCopy (to/from) into Buffer
 - We have added the real length (minus the status bits) to the status bits.

Changes: https://git.openjdk.org/babylon/pull/362/files
  Webrev: https://webrevs.openjdk.org/?repo=babylon&pr=362&range=00
  Stats: 962 lines in 20 files changed: 584 ins; 296 del; 82 mod
  Patch: https://git.openjdk.org/babylon/pull/362.diff
  Fetch: git fetch https://git.openjdk.org/babylon.git pull/362/head:pull/362

PR: https://git.openjdk.org/babylon/pull/362

From gfrost at openjdk.org  Sat Mar 15 17:18:25 2025
From: gfrost at openjdk.org (Gary Frost)
Date: Sat, 15 Mar 2025 17:18:25 GMT
Subject: [code-reflection] Integrated: Hat fix min buffer state machine
In-Reply-To: <JDgFVuNQKnhwBJWvOUTJUboIFu9h5M12Ji83zj6qHhQ=.0b9c53c7-89ba-4ccf-98bb-1bf7142aac6c@github.com>
References: <JDgFVuNQKnhwBJWvOUTJUboIFu9h5M12Ji83zj6qHhQ=.0b9c53c7-89ba-4ccf-98bb-1bf7142aac6c@github.com>
Message-ID: <7wutNkKd2-68tPkxkTNtqGOfKKGLYtEu9s50Fxc86Eo=.20ef278f-fabe-4295-be97-c1a19946d2b7@github.com>

On Sat, 15 Mar 2025 17:12:27 GMT, Gary Frost <gfrost at openjdk.org> wrote:

> Swapped out the bitfield based states for buffer management to one using a state machine. 
> 
> Seems to be working.

This pull request has now been integrated.

Changeset: a26f48c6
Author:    Gary Frost <gfrost at openjdk.org>
URL:       https://git.openjdk.org/babylon/commit/a26f48c6a1d67e48532a2671d7560641efb92432
Stats:     962 lines in 20 files changed: 584 ins; 296 del; 82 mod

Hat fix min buffer state machine

-------------

PR: https://git.openjdk.org/babylon/pull/362

From gfrost at openjdk.org  Mon Mar 17 16:32:11 2025
From: gfrost at openjdk.org (Gary Frost)
Date: Mon, 17 Mar 2025 16:32:11 GMT
Subject: [code-reflection] Integrated: updates demos for j1
Message-ID: <VKxmUwtIhP8oriXS_U0esfGDoHvkBe97qCyGU5yDEgY=.e54f4979-0786-4791-806b-7f98dde97339@github.com>

Some ui tweaks. So the life demo matches presentation.

-------------

Commit messages:
 - updates demos for j1

Changes: https://git.openjdk.org/babylon/pull/364/files
  Webrev: https://webrevs.openjdk.org/?repo=babylon&pr=364&range=00
  Stats: 88 lines in 4 files changed: 36 ins; 33 del; 19 mod
  Patch: https://git.openjdk.org/babylon/pull/364.diff
  Fetch: git fetch https://git.openjdk.org/babylon.git pull/364/head:pull/364

PR: https://git.openjdk.org/babylon/pull/364

From gfrost at openjdk.org  Mon Mar 17 16:32:11 2025
From: gfrost at openjdk.org (Gary Frost)
Date: Mon, 17 Mar 2025 16:32:11 GMT
Subject: [code-reflection] Integrated: updates demos for j1
In-Reply-To: <VKxmUwtIhP8oriXS_U0esfGDoHvkBe97qCyGU5yDEgY=.e54f4979-0786-4791-806b-7f98dde97339@github.com>
References: <VKxmUwtIhP8oriXS_U0esfGDoHvkBe97qCyGU5yDEgY=.e54f4979-0786-4791-806b-7f98dde97339@github.com>
Message-ID: <Bj5O0JZGt4wXP8SL2ZlBC6FBVADOfGLHqQbkuCW3Dsk=.82937d73-12e7-4994-bbd9-fac2554f085f@github.com>

On Mon, 17 Mar 2025 16:17:39 GMT, Gary Frost <gfrost at openjdk.org> wrote:

> Some ui tweaks. So the life demo matches presentation.

This pull request has now been integrated.

Changeset: 28179a67
Author:    Gary Frost <gfrost at openjdk.org>
URL:       https://git.openjdk.org/babylon/commit/28179a6715279d475a5957a1b56cc6babc4e163f
Stats:     88 lines in 4 files changed: 36 ins; 33 del; 19 mod

updates demos for j1

-------------

PR: https://git.openjdk.org/babylon/pull/364

From gfrost at openjdk.org  Mon Mar 17 16:36:52 2025
From: gfrost at openjdk.org (Gary Frost)
Date: Mon, 17 Mar 2025 16:36:52 GMT
Subject: git: openjdk/babylon: code-reflection: updates demos for j1
Message-ID: <b1fcca52-aa99-4330-81cf-7c819d091ea7@openjdk.org>

Changeset: 28179a67
Branch: code-reflection
Author:    Gary Frost <gfrost at openjdk.org>
Date:      2025-03-17 16:24:24 +0000
URL:       https://git.openjdk.org/babylon/commit/28179a6715279d475a5957a1b56cc6babc4e163f

updates demos for j1

! hat/examples/life/src/main/java/life/Main.java
! hat/examples/life/src/main/java/life/Viewer.java
! hat/examples/nbody/src/main/java/nbody/opencl/OpenCLNBodyGLWindow.java
! hat/hat-core/src/main/java/hat/Accelerator.java


From psandoz at openjdk.org  Mon Mar 17 16:52:27 2025
From: psandoz at openjdk.org (Paul Sandoz)
Date: Mon, 17 Mar 2025 16:52:27 GMT
Subject: git: openjdk/babylon: code-reflection: Rearrange the source for better
 display.
Message-ID: <1a9b1707-4389-482c-b8bd-033aa9cedd42@openjdk.org>

Changeset: 88441d85
Branch: code-reflection
Author:    Paul Sandoz <psandoz at openjdk.org>
Date:      2025-03-17 16:49:45 +0000
URL:       https://git.openjdk.org/babylon/commit/88441d85eedd7426829620dfd33587713d10a869

Rearrange the source for better display.

! cr-examples/onnx/src/test/java/oracle/code/onnx/mnist/MNISTModel.java


From psandoz at openjdk.org  Mon Mar 17 16:54:35 2025
From: psandoz at openjdk.org (Paul Sandoz)
Date: Mon, 17 Mar 2025 16:54:35 GMT
Subject: [code-reflection] Integrated: Rearrange the source for better display.
Message-ID: <xZgk1Jslicu0ffE0EdlVMO-6qPiYU6v_2DPWidh0HZo=.a1bfda28-6d6f-4b2f-ad1f-e7ec4e2f436b@github.com>

Rearrange the source for better display.

-------------

Commit messages:
 - Rearrange the source for better display.

Changes: https://git.openjdk.org/babylon/pull/365/files
  Webrev: https://webrevs.openjdk.org/?repo=babylon&pr=365&range=00
  Stats: 37 lines in 1 file changed: 19 ins; 16 del; 2 mod
  Patch: https://git.openjdk.org/babylon/pull/365.diff
  Fetch: git fetch https://git.openjdk.org/babylon.git pull/365/head:pull/365

PR: https://git.openjdk.org/babylon/pull/365

From psandoz at openjdk.org  Mon Mar 17 16:54:35 2025
From: psandoz at openjdk.org (Paul Sandoz)
Date: Mon, 17 Mar 2025 16:54:35 GMT
Subject: [code-reflection] Integrated: Rearrange the source for better
 display.
In-Reply-To: <xZgk1Jslicu0ffE0EdlVMO-6qPiYU6v_2DPWidh0HZo=.a1bfda28-6d6f-4b2f-ad1f-e7ec4e2f436b@github.com>
References: <xZgk1Jslicu0ffE0EdlVMO-6qPiYU6v_2DPWidh0HZo=.a1bfda28-6d6f-4b2f-ad1f-e7ec4e2f436b@github.com>
Message-ID: <lOH3srbeG2bPBN-hJZzxy_Q5BnKYjfC9M92uNt7zb8M=.079f9c70-72cf-4c49-989a-e4ffee8b1353@github.com>

On Mon, 17 Mar 2025 16:30:54 GMT, Paul Sandoz <psandoz at openjdk.org> wrote:

> Rearrange the source for better display.

prod the bots

-------------

PR Comment: https://git.openjdk.org/babylon/pull/365#issuecomment-2730153140

From psandoz at openjdk.org  Mon Mar 17 16:54:35 2025
From: psandoz at openjdk.org (Paul Sandoz)
Date: Mon, 17 Mar 2025 16:54:35 GMT
Subject: [code-reflection] Integrated: Rearrange the source for better display.
In-Reply-To: <xZgk1Jslicu0ffE0EdlVMO-6qPiYU6v_2DPWidh0HZo=.a1bfda28-6d6f-4b2f-ad1f-e7ec4e2f436b@github.com>
References: <xZgk1Jslicu0ffE0EdlVMO-6qPiYU6v_2DPWidh0HZo=.a1bfda28-6d6f-4b2f-ad1f-e7ec4e2f436b@github.com>
Message-ID: <raX0gMT6czQA1yYzZ1_ZN70n8ZHm5RNuTeGQsbbLr5c=.5891dacf-ba31-40fb-9061-a80d25c4f167@github.com>

On Mon, 17 Mar 2025 16:30:54 GMT, Paul Sandoz <psandoz at openjdk.org> wrote:

> Rearrange the source for better display.

This pull request has now been integrated.

Changeset: 88441d85
Author:    Paul Sandoz <psandoz at openjdk.org>
URL:       https://git.openjdk.org/babylon/commit/88441d85eedd7426829620dfd33587713d10a869
Stats:     37 lines in 1 file changed: 19 ins; 16 del; 2 mod

Rearrange the source for better display.

-------------

PR: https://git.openjdk.org/babylon/pull/365

From mabbay at openjdk.org  Wed Mar 19 03:08:24 2025
From: mabbay at openjdk.org (Mourad Abbay)
Date: Wed, 19 Mar 2025 03:08:24 GMT
Subject: [code-reflection] RFR: Support storing the code that builds the
 code model [v18]
In-Reply-To: <ftnpqrjPGs9YPOQm7MbEV8PZb2dGx92ga2-340cOuh0=.8a2f956a-f277-4009-b461-05e9d88363f5@github.com>
References: <dexv8sQV_NcJsJIgMopd_M4wSWoZya9dU0ACwDtEL0E=.4ac0b80f-24fb-440f-a18d-bbfd07a681bc@github.com>
 <rLtW9Wv92MNgUIywKaiPdoBs8O5-5EXNlu17J8tRflk=.3e0aa614-55db-4e86-a5a0-99e8e51758d8@github.com>
 <ftnpqrjPGs9YPOQm7MbEV8PZb2dGx92ga2-340cOuh0=.8a2f956a-f277-4009-b461-05e9d88363f5@github.com>
Message-ID: <WbrP4ERmNzNCkHNRQvAr5Ff72Z5mX3nk1-KVNeyMRZc=.31ae7834-839d-4cf6-9577-ac5d5c6ba901@github.com>

On Thu, 13 Mar 2025 14:37:38 GMT, Maurizio Cimadamore <mcimadamore at openjdk.org> wrote:

>> Mourad Abbay has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains 20 additional commits since the last revision:
>> 
>>  - Merge branch 'code-reflection' into code-model-storage-option
>>  - Load OpFactory and TypeElementFactory before invocation of opMethod only if the opMethod has there params.
>>  - Fix the remaining compiler tests failures
>>  - Fix some of the test failures (3 remains)
>>  - Fix the remaining test failures of SwitchExpressionTest2
>>  - Fix almost all test failures of SwitchExpressionTest2 (one remaining)
>>  - Pass arrayType instead of eleType in OpBuilder.buildArray
>>  - Ensure that block params are inserted in the correct order
>>  - Add missing imports
>>  - Merge branch 'code-reflection' into code-model-storage-option
>>  - ... and 10 more: https://git.openjdk.org/babylon/compare/0f2868bc...966005ce
>
> src/jdk.incubator.code/share/classes/jdk/incubator/code/internal/CodeModelToAST.java line 151:
> 
>> 149:                     yield na;
>> 150:                 }
>> 151:                 var ownerType = typeElementToType(newOp.constructorType().returnType());
> 
> There is a strange asymmetry in the model between constructors and methods. Methods have a descriptor, constructors do not. Also, for methods you know from the op if it is varargs, but for constructors this info is not available. My feeling is that (at least coming at this from javac), trying to lump array and class creation under one op is probably why these asymmetries crop up.
> 
> This has nothing to do with this PR of course -- but I suspect that if you had a varargs constructor call this code would not work.

I agree on all points. We can have a `new.array` operation and then model class creation as an `invoke` op with method name being "init" ?

-------------

PR Review Comment: https://git.openjdk.org/babylon/pull/305#discussion_r2002333689

From mabbay at openjdk.org  Wed Mar 19 03:20:56 2025
From: mabbay at openjdk.org (Mourad Abbay)
Date: Wed, 19 Mar 2025 03:20:56 GMT
Subject: [code-reflection] RFR: Support storing the code that builds the
 code model [v19]
In-Reply-To: <dexv8sQV_NcJsJIgMopd_M4wSWoZya9dU0ACwDtEL0E=.4ac0b80f-24fb-440f-a18d-bbfd07a681bc@github.com>
References: <dexv8sQV_NcJsJIgMopd_M4wSWoZya9dU0ACwDtEL0E=.4ac0b80f-24fb-440f-a18d-bbfd07a681bc@github.com>
Message-ID: <NtS2fc1trqX8NhsMhmOP8P8YyFfi3MiK5xHHdjh9yBE=.abe3926b-3c52-46f1-93e6-45bacf2fda53@github.com>

> In this PR we allow the user to decide how to store the code model. The first option is `TEXT`, if this is selected, we store the textual representation of the code model. The second option is `CODE_BUILDR`, if this is selected, we store the code that builds the code model. All work done here is around the second option, because the first option is already supported.

Mourad Abbay has updated the pull request incrementally with one additional commit since the last revision:

  Don't rely on the assumption of 1d array

-------------

Changes:
  - all: https://git.openjdk.org/babylon/pull/305/files
  - new: https://git.openjdk.org/babylon/pull/305/files/966005ce..95f227ee

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=babylon&pr=305&range=18
 - incr: https://webrevs.openjdk.org/?repo=babylon&pr=305&range=17-18

  Stats: 5 lines in 1 file changed: 2 ins; 0 del; 3 mod
  Patch: https://git.openjdk.org/babylon/pull/305.diff
  Fetch: git fetch https://git.openjdk.org/babylon.git pull/305/head:pull/305

PR: https://git.openjdk.org/babylon/pull/305

From mabbay at openjdk.org  Wed Mar 19 03:20:58 2025
From: mabbay at openjdk.org (Mourad Abbay)
Date: Wed, 19 Mar 2025 03:20:58 GMT
Subject: [code-reflection] RFR: Support storing the code that builds the
 code model [v18]
In-Reply-To: <ieI_WwBslJzVLxiNH_lKEwpStmUw_ITELaj5LFXTtxs=.c9565238-3b7e-42ab-997f-455fb061f807@github.com>
References: <dexv8sQV_NcJsJIgMopd_M4wSWoZya9dU0ACwDtEL0E=.4ac0b80f-24fb-440f-a18d-bbfd07a681bc@github.com>
 <rLtW9Wv92MNgUIywKaiPdoBs8O5-5EXNlu17J8tRflk=.3e0aa614-55db-4e86-a5a0-99e8e51758d8@github.com>
 <ieI_WwBslJzVLxiNH_lKEwpStmUw_ITELaj5LFXTtxs=.c9565238-3b7e-42ab-997f-455fb061f807@github.com>
Message-ID: <rQgubeg2Sc7zezEv_TQRYDfqigJ5OVNz6uvQxsr76j8=.ab4b96cc-16c4-4c60-8457-2979c665df09@github.com>

On Thu, 13 Mar 2025 14:40:31 GMT, Maurizio Cimadamore <mcimadamore at openjdk.org> wrote:

>> Mourad Abbay has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains 20 additional commits since the last revision:
>> 
>>  - Merge branch 'code-reflection' into code-model-storage-option
>>  - Load OpFactory and TypeElementFactory before invocation of opMethod only if the opMethod has there params.
>>  - Fix the remaining compiler tests failures
>>  - Fix some of the test failures (3 remains)
>>  - Fix the remaining test failures of SwitchExpressionTest2
>>  - Fix almost all test failures of SwitchExpressionTest2 (one remaining)
>>  - Pass arrayType instead of eleType in OpBuilder.buildArray
>>  - Ensure that block params are inserted in the correct order
>>  - Add missing imports
>>  - Merge branch 'code-reflection' into code-model-storage-option
>>  - ... and 10 more: https://git.openjdk.org/babylon/compare/91305e8c...966005ce
>
> src/jdk.incubator.code/share/classes/jdk/incubator/code/internal/CodeModelToAST.java line 145:
> 
>> 143:                 if (newOp.resultType() instanceof ArrayType at) {
>> 144:                     var elemType = treeMaker.Ident(typeElementToType(at.componentType()).tsym);
>> 145:                     // @@@ we assume one dimension
> 
> note: `ArrayType` has a `dimensions()` method -- so in principle you could use that and keep fetching dimension operands (and update the type of the array to be built)

I updated the code.

-------------

PR Review Comment: https://git.openjdk.org/babylon/pull/305#discussion_r2002341204

From mcimadamore at openjdk.org  Wed Mar 19 09:25:25 2025
From: mcimadamore at openjdk.org (Maurizio Cimadamore)
Date: Wed, 19 Mar 2025 09:25:25 GMT
Subject: [code-reflection] RFR: Support storing the code that builds the
 code model [v18]
In-Reply-To: <8vDjuCkLBgFFnPK_tYycS420iPcfeMK9JqtsStKzTp8=.66587d94-c595-4d24-9c1d-b787867e71b3@github.com>
References: <dexv8sQV_NcJsJIgMopd_M4wSWoZya9dU0ACwDtEL0E=.4ac0b80f-24fb-440f-a18d-bbfd07a681bc@github.com>
 <rLtW9Wv92MNgUIywKaiPdoBs8O5-5EXNlu17J8tRflk=.3e0aa614-55db-4e86-a5a0-99e8e51758d8@github.com>
 <UZ2Z1W9PyhkEv4ucTxh-zfBISS__-0xiY5PGL4p67qw=.9a8dfbfd-3809-4fba-891f-543bead72d6b@github.com>
 <_vyINIwx1fIfdBtHZYt9VgABP7AIE8Q2bcsFaprRJAc=.2b84048a-69b1-4576-aca0-86caf814b280@github.com>
 <8vDjuCkLBgFFnPK_tYycS420iPcfeMK9JqtsStKzTp8=.66587d94-c595-4d24-9c1d-b787867e71b3@github.com>
Message-ID: <4Q4bmumCJYqik0o1Zh0bh3KslXYfls-6L_s_v2l44Y4=.1d61c2e5-3c19-40cb-9dbf-b362616f4f8d@github.com>

On Sat, 15 Mar 2025 02:42:01 GMT, Mourad Abbay <mabbay at openjdk.org> wrote:

> We can turn opMethod into one with no param and load these fields inside its body.

That is what I was asking about -- while I agree what we have seems more flexible, we don't seem to use this flexibility anywhere -- as the method that turns a `Quotable` into a `Quoted` works with hardwired op and type factories. But maybe this has been already discussed -- in which case it's fine.

>> `addVarsWhenNecessary` is used to add necessary VarOp in the code model before transformation to AST. For example if an op result is used more than once, we introduce a VarOp to hold the value, because that's what the equivalent Java code will have. This makes the transformation from code model to AST a bit easier.
>> We also use it to in case we want an Op (e.g. InvokeOp) to be appended right away to the method we are building. As you now the block of a method is a list of statements, some operations maps to an expression, so they won't be appended until needed. So by putting the result of an op in a VarOp, that VarOp will be mapped to a statement which causes the tree of the op to be added immediately.
>> For example an `InvokeOp` to `Block.Builder::op` is something we want to be transformed to a `JCTree` and added immediately to the method's block, so that we preserve the order of the operations as in the original code model.
>
> Normally, the OpBuilder should produces a model that contains the VarOp when they should be. But I decided to have a seperate transformation as a quick fix. Later I will change the OpBuilder.

Apologies -- I meant whether the `writeTo` should be removed :-)

-------------

PR Review Comment: https://git.openjdk.org/babylon/pull/305#discussion_r2002857554
PR Review Comment: https://git.openjdk.org/babylon/pull/305#discussion_r2002850721

From mcimadamore at openjdk.org  Wed Mar 19 09:25:25 2025
From: mcimadamore at openjdk.org (Maurizio Cimadamore)
Date: Wed, 19 Mar 2025 09:25:25 GMT
Subject: [code-reflection] RFR: Support storing the code that builds the
 code model [v18]
In-Reply-To: <4Q4bmumCJYqik0o1Zh0bh3KslXYfls-6L_s_v2l44Y4=.1d61c2e5-3c19-40cb-9dbf-b362616f4f8d@github.com>
References: <dexv8sQV_NcJsJIgMopd_M4wSWoZya9dU0ACwDtEL0E=.4ac0b80f-24fb-440f-a18d-bbfd07a681bc@github.com>
 <rLtW9Wv92MNgUIywKaiPdoBs8O5-5EXNlu17J8tRflk=.3e0aa614-55db-4e86-a5a0-99e8e51758d8@github.com>
 <UZ2Z1W9PyhkEv4ucTxh-zfBISS__-0xiY5PGL4p67qw=.9a8dfbfd-3809-4fba-891f-543bead72d6b@github.com>
 <_vyINIwx1fIfdBtHZYt9VgABP7AIE8Q2bcsFaprRJAc=.2b84048a-69b1-4576-aca0-86caf814b280@github.com>
 <8vDjuCkLBgFFnPK_tYycS420iPcfeMK9JqtsStKzTp8=.66587d94-c595-4d24-9c1d-b787867e71b3@github.com>
 <4Q4bmumCJYqik0o1Zh0bh3KslXYfls-6L_s_v2l44Y4=.1d61c2e5-3c19-40cb-9dbf-b362616f4f8d@github.com>
Message-ID: <bhrfaSv-BcS4ylQIM_WbwPzDnPfY12Fb4MHV_LWWZwk=.4dfad721-6331-4640-8019-ff4d7985b027@github.com>

On Wed, 19 Mar 2025 09:19:45 GMT, Maurizio Cimadamore <mcimadamore at openjdk.org> wrote:

>> Normally, the OpBuilder should produces a model that contains the VarOp when they should be. But I decided to have a seperate transformation as a quick fix. Later I will change the OpBuilder.
>
> Apologies -- I meant whether the `writeTo` should be removed :-)

> But I decided to have a seperate transformation as a quick fix. Later I will change the OpBuilder.

As stated in another comment, I'm not sure changing the op builder is needed -- as the javac code can easily workaround what the op builder does (e.g. what it does doesn't seem obviously wrong to me). But if the decision is to fix op builder that's fine too

-------------

PR Review Comment: https://git.openjdk.org/babylon/pull/305#discussion_r2002853522

From mcimadamore at openjdk.org  Wed Mar 19 09:29:22 2025
From: mcimadamore at openjdk.org (Maurizio Cimadamore)
Date: Wed, 19 Mar 2025 09:29:22 GMT
Subject: [code-reflection] RFR: Support storing the code that builds the
 code model [v18]
In-Reply-To: <WbrP4ERmNzNCkHNRQvAr5Ff72Z5mX3nk1-KVNeyMRZc=.31ae7834-839d-4cf6-9577-ac5d5c6ba901@github.com>
References: <dexv8sQV_NcJsJIgMopd_M4wSWoZya9dU0ACwDtEL0E=.4ac0b80f-24fb-440f-a18d-bbfd07a681bc@github.com>
 <rLtW9Wv92MNgUIywKaiPdoBs8O5-5EXNlu17J8tRflk=.3e0aa614-55db-4e86-a5a0-99e8e51758d8@github.com>
 <ftnpqrjPGs9YPOQm7MbEV8PZb2dGx92ga2-340cOuh0=.8a2f956a-f277-4009-b461-05e9d88363f5@github.com>
 <WbrP4ERmNzNCkHNRQvAr5Ff72Z5mX3nk1-KVNeyMRZc=.31ae7834-839d-4cf6-9577-ac5d5c6ba901@github.com>
Message-ID: <QQ-_TRdoc-nUMms-gcrcM0GQ7WCnAnqQ5KriOmzfCRY=.161b6225-32a5-4e00-ab35-81e3c2a8be30@github.com>

On Wed, 19 Mar 2025 03:04:46 GMT, Mourad Abbay <mabbay at openjdk.org> wrote:

> I agree on all points. We can have a `new.array` operation and then model class creation as an `invoke` op with method name being "init" ?

not sure I follow -- I was talking about class construction -- arrays are a bit special (e.g. there's no array constructor descriptor, unlike for classes). But a constructor call does share a lot of similarities with a method call, and there should be reusable code between method and constructors -- as it happens inside the javac code.

-------------

PR Review Comment: https://git.openjdk.org/babylon/pull/305#discussion_r2002865321

From gfrost at openjdk.org  Thu Mar 20 15:17:13 2025
From: gfrost at openjdk.org (Gary Frost)
Date: Thu, 20 Mar 2025 15:17:13 GMT
Subject: git: openjdk/babylon: code-reflection: Hat replace bitfield min
 bufffer with state machine
Message-ID: <a5a9f966-0a0e-4a39-898f-a021bfa07f33@openjdk.org>

Changeset: 9c42d66f
Branch: code-reflection
Author:    Gary Frost <gfrost at openjdk.org>
Date:      2025-03-20 15:15:03 +0000
URL:       https://git.openjdk.org/babylon/commit/9c42d66f66baac4b099d3107352daa5e43e2ea5d

Hat replace bitfield min bufffer with state machine

! hat/backends/ffi/opencl/cpp/opencl_backend.cpp
! hat/backends/ffi/opencl/cpp/opencl_backend_kernel_dispatch.cpp
! hat/backends/ffi/opencl/include/opencl_backend.h
! hat/backends/ffi/opencl/src/main/java/hat/backend/ffi/OpenCLBackend.java
! hat/backends/ffi/opencl/src/main/java/hat/backend/ffi/OpenCLConfig.java
! hat/backends/ffi/shared/include/shared.h
! hat/examples/life/src/main/java/life/Main.java
! hat/examples/nbody/src/main/java/nbody/opencl/OpenCLNBodyGLWindow.java
! hat/hat-core/src/main/java/hat/backend/ffi/FFIBackend.java
! hat/hat-core/src/main/java/hat/buffer/Buffer.java
! hat/hat-core/src/main/java/hat/ifacemapper/BufferState.java
! hat/hat-core/src/main/java/hat/ifacemapper/MappableIface.java
! hat/hat-core/src/main/java/hat/ifacemapper/SegmentMapper.java
+ hat/hat/debug
! hat/wrap/clwrap/src/main/java/wrap/clwrap/CLPlatform.java


From gfrost at openjdk.org  Thu Mar 20 15:18:02 2025
From: gfrost at openjdk.org (Gary Frost)
Date: Thu, 20 Mar 2025 15:18:02 GMT
Subject: [code-reflection] Integrated: Hat replace bitfield min bufffer with
 state machine
Message-ID: <_9Kuf8ijmfE10O-myAXuiX0YtTc5S8auV2KLJqcsp3I=.edada571-bc74-4066-b540-5f4300b6f650@github.com>

The previous bitfield based state info replaced by a simpler state machine. 

Also here we are able to minimize copies when buffers 'escape'.  If the escaping function annotates its buffer arg RO,RW or WO then HAT can avoid at leas one copy.

-------------

Commit messages:
 - Injection of pre/post access methods now snoops access annotations
 - removed bit state information from BufferState. We use state machine now

Changes: https://git.openjdk.org/babylon/pull/366/files
  Webrev: https://webrevs.openjdk.org/?repo=babylon&pr=366&range=00
  Stats: 674 lines in 15 files changed: 149 ins; 408 del; 117 mod
  Patch: https://git.openjdk.org/babylon/pull/366.diff
  Fetch: git fetch https://git.openjdk.org/babylon.git pull/366/head:pull/366

PR: https://git.openjdk.org/babylon/pull/366

From gfrost at openjdk.org  Thu Mar 20 15:18:02 2025
From: gfrost at openjdk.org (Gary Frost)
Date: Thu, 20 Mar 2025 15:18:02 GMT
Subject: [code-reflection] Integrated: Hat replace bitfield min bufffer with
 state machine
In-Reply-To: <_9Kuf8ijmfE10O-myAXuiX0YtTc5S8auV2KLJqcsp3I=.edada571-bc74-4066-b540-5f4300b6f650@github.com>
References: <_9Kuf8ijmfE10O-myAXuiX0YtTc5S8auV2KLJqcsp3I=.edada571-bc74-4066-b540-5f4300b6f650@github.com>
Message-ID: <n-9qDJLvHSoyr9bkHdj1JfAEg6skVslWd_l6o_wyD04=.30e7e91a-5849-4e8a-9eef-7cfca813d927@github.com>

On Thu, 20 Mar 2025 15:13:06 GMT, Gary Frost <gfrost at openjdk.org> wrote:

> The previous bitfield based state info replaced by a simpler state machine. 
> 
> Also here we are able to minimize copies when buffers 'escape'.  If the escaping function annotates its buffer arg RO,RW or WO then HAT can avoid at leas one copy.

This pull request has now been integrated.

Changeset: 9c42d66f
Author:    Gary Frost <gfrost at openjdk.org>
URL:       https://git.openjdk.org/babylon/commit/9c42d66f66baac4b099d3107352daa5e43e2ea5d
Stats:     674 lines in 15 files changed: 149 ins; 408 del; 117 mod

Hat replace bitfield min bufffer with state machine

-------------

PR: https://git.openjdk.org/babylon/pull/366

From asotona at openjdk.org  Thu Mar 20 15:54:51 2025
From: asotona at openjdk.org (Adam Sotona)
Date: Thu, 20 Mar 2025 15:54:51 GMT
Subject: [code-reflection] RFR: ExplicitOnnxOps.If implementation of
 captured values and initializers [v2]
In-Reply-To: <9pGY7kCbBiYjrFHzelwweejUpCFEzlEQeXLLR0HUYuU=.21c0b04d-42a1-4cd0-a185-d2e2c296384b@github.com>
References: <9pGY7kCbBiYjrFHzelwweejUpCFEzlEQeXLLR0HUYuU=.21c0b04d-42a1-4cd0-a185-d2e2c296384b@github.com>
Message-ID: <ZbYhPNvPIiuO1qNGV7Tb8C90wi3A0iaqgxeDUsEkjq4=.b8e62d9e-a4a4-402f-b512-782be049b064@github.com>

> ExplicitOnnxOps.If implementation of captured values and initializers - another attempt to do it right

Adam Sotona has updated the pull request incrementally with one additional commit since the last revision:

  added minimal support for variadic parameter + SimpleTest::testConcat

-------------

Changes:
  - all: https://git.openjdk.org/babylon/pull/359/files
  - new: https://git.openjdk.org/babylon/pull/359/files/100683d4..69f9d1ab

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=babylon&pr=359&range=01
 - incr: https://webrevs.openjdk.org/?repo=babylon&pr=359&range=00-01

  Stats: 37 lines in 3 files changed: 34 ins; 0 del; 3 mod
  Patch: https://git.openjdk.org/babylon/pull/359.diff
  Fetch: git fetch https://git.openjdk.org/babylon.git pull/359/head:pull/359

PR: https://git.openjdk.org/babylon/pull/359

From asotona at openjdk.org  Thu Mar 20 18:10:59 2025
From: asotona at openjdk.org (Adam Sotona)
Date: Thu, 20 Mar 2025 18:10:59 GMT
Subject: [code-reflection] RFR: ExplicitOnnxOps.If implementation of
 captured values and initializers [v3]
In-Reply-To: <9pGY7kCbBiYjrFHzelwweejUpCFEzlEQeXLLR0HUYuU=.21c0b04d-42a1-4cd0-a185-d2e2c296384b@github.com>
References: <9pGY7kCbBiYjrFHzelwweejUpCFEzlEQeXLLR0HUYuU=.21c0b04d-42a1-4cd0-a185-d2e2c296384b@github.com>
Message-ID: <3VF2CeFRPKRTnWSupfqLpj6XcOsiETd58EwJfN_IbwY=.5c6dd285-015b-44fe-a6d1-5a7b43a98f5b@github.com>

> ExplicitOnnxOps.If implementation of captured values and initializers - another attempt to do it right

Adam Sotona has updated the pull request incrementally with one additional commit since the last revision:

  added raw support for variadic (list) results + SimpleTest::testSplit

-------------

Changes:
  - all: https://git.openjdk.org/babylon/pull/359/files
  - new: https://git.openjdk.org/babylon/pull/359/files/69f9d1ab..09e4a482

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=babylon&pr=359&range=02
 - incr: https://webrevs.openjdk.org/?repo=babylon&pr=359&range=01-02

  Stats: 59 lines in 5 files changed: 54 ins; 0 del; 5 mod
  Patch: https://git.openjdk.org/babylon/pull/359.diff
  Fetch: git fetch https://git.openjdk.org/babylon.git pull/359/head:pull/359

PR: https://git.openjdk.org/babylon/pull/359

From asotona at openjdk.org  Thu Mar 20 18:27:37 2025
From: asotona at openjdk.org (Adam Sotona)
Date: Thu, 20 Mar 2025 18:27:37 GMT
Subject: [code-reflection] RFR: ExplicitOnnxOps.If implementation of
 captured values and initializers [v4]
In-Reply-To: <9pGY7kCbBiYjrFHzelwweejUpCFEzlEQeXLLR0HUYuU=.21c0b04d-42a1-4cd0-a185-d2e2c296384b@github.com>
References: <9pGY7kCbBiYjrFHzelwweejUpCFEzlEQeXLLR0HUYuU=.21c0b04d-42a1-4cd0-a185-d2e2c296384b@github.com>
Message-ID: <fnf-q1RjarRhr44uIVQQIVz9DHi9gqag38cziaEKYR8=.b154e30b-e80f-4eb8-9cd3-03723077b39a@github.com>

> ExplicitOnnxOps.If implementation of captured values and initializers - another attempt to do it right

Adam Sotona has updated the pull request incrementally with one additional commit since the last revision:

  OxplicitOnnxOperators::If turned into variadic (list)

-------------

Changes:
  - all: https://git.openjdk.org/babylon/pull/359/files
  - new: https://git.openjdk.org/babylon/pull/359/files/09e4a482..f3f73bdd

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=babylon&pr=359&range=03
 - incr: https://webrevs.openjdk.org/?repo=babylon&pr=359&range=02-03

  Stats: 20 lines in 3 files changed: 6 ins; 1 del; 13 mod
  Patch: https://git.openjdk.org/babylon/pull/359.diff
  Fetch: git fetch https://git.openjdk.org/babylon.git pull/359/head:pull/359

PR: https://git.openjdk.org/babylon/pull/359

From asotona at openjdk.org  Fri Mar 21 10:58:35 2025
From: asotona at openjdk.org (Adam Sotona)
Date: Fri, 21 Mar 2025 10:58:35 GMT
Subject: [code-reflection] RFR: ExplicitOnnxOps.If implementation of
 captured values and initializers [v5]
In-Reply-To: <9pGY7kCbBiYjrFHzelwweejUpCFEzlEQeXLLR0HUYuU=.21c0b04d-42a1-4cd0-a185-d2e2c296384b@github.com>
References: <9pGY7kCbBiYjrFHzelwweejUpCFEzlEQeXLLR0HUYuU=.21c0b04d-42a1-4cd0-a185-d2e2c296384b@github.com>
Message-ID: <CdZeMIEM5SQ5V4bJtmN5COCxdg-rNPgDrnUW3HwoedY=.b46a0e6c-2cc9-4010-add2-67aefe0c0765@github.com>

> ExplicitOnnxOps.If implementation of captured values and initializers - another attempt to do it right

Adam Sotona has updated the pull request incrementally with one additional commit since the last revision:

  first draft of Loop op - work in progress

-------------

Changes:
  - all: https://git.openjdk.org/babylon/pull/359/files
  - new: https://git.openjdk.org/babylon/pull/359/files/f3f73bdd..9e5cf88a

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=babylon&pr=359&range=04
 - incr: https://webrevs.openjdk.org/?repo=babylon&pr=359&range=03-04

  Stats: 181 lines in 3 files changed: 177 ins; 3 del; 1 mod
  Patch: https://git.openjdk.org/babylon/pull/359.diff
  Fetch: git fetch https://git.openjdk.org/babylon.git pull/359/head:pull/359

PR: https://git.openjdk.org/babylon/pull/359

From asotona at openjdk.org  Mon Mar 24 13:25:10 2025
From: asotona at openjdk.org (Adam Sotona)
Date: Mon, 24 Mar 2025 13:25:10 GMT
Subject: [code-reflection] RFR: ExplicitOnnxOps.If implementation of
 captured values and initializers [v6]
In-Reply-To: <9pGY7kCbBiYjrFHzelwweejUpCFEzlEQeXLLR0HUYuU=.21c0b04d-42a1-4cd0-a185-d2e2c296384b@github.com>
References: <9pGY7kCbBiYjrFHzelwweejUpCFEzlEQeXLLR0HUYuU=.21c0b04d-42a1-4cd0-a185-d2e2c296384b@github.com>
Message-ID: <SIHcpBsCpQtxpRf3cQ_ji-0BaFhvTdMP732b517CGqU=.38ab6c86-1b5a-43a7-af06-8c1e1ebd0ed9@github.com>

> ExplicitOnnxOps.If implementation of captured values and initializers - another attempt to do it right
> 
> + raw minimal support of variadic inputs and outputs
> + continuation work on If and Loop

Adam Sotona has updated the pull request incrementally with one additional commit since the last revision:

  Loop op - work in progress

-------------

Changes:
  - all: https://git.openjdk.org/babylon/pull/359/files
  - new: https://git.openjdk.org/babylon/pull/359/files/9e5cf88a..15edc4de

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=babylon&pr=359&range=05
 - incr: https://webrevs.openjdk.org/?repo=babylon&pr=359&range=04-05

  Stats: 15 lines in 4 files changed: 6 ins; 0 del; 9 mod
  Patch: https://git.openjdk.org/babylon/pull/359.diff
  Fetch: git fetch https://git.openjdk.org/babylon.git pull/359/head:pull/359

PR: https://git.openjdk.org/babylon/pull/359

From asotona at openjdk.org  Mon Mar 24 17:20:03 2025
From: asotona at openjdk.org (Adam Sotona)
Date: Mon, 24 Mar 2025 17:20:03 GMT
Subject: git: openjdk/babylon: code-reflection: ExplicitOnnxOps.If
 implementation of captured values and initializers
Message-ID: <98e0d800-41c3-4611-863d-bb19d7d85bbe@openjdk.org>

Changeset: 15fa02fc
Branch: code-reflection
Author:    Adam Sotona <asotona at openjdk.org>
Date:      2025-03-24 17:18:18 +0000
URL:       https://git.openjdk.org/babylon/commit/15fa02fc338e7746093ea253f513da393cfa3492

ExplicitOnnxOps.If implementation of captured values and initializers

! cr-examples/onnx/src/main/java/oracle/code/onnx/ExplicitOnnxOperators.java
- cr-examples/onnx/src/main/java/oracle/code/onnx/LambdaToFunc.java
! cr-examples/onnx/src/main/java/oracle/code/onnx/OnnxInterpreter.java
! cr-examples/onnx/src/main/java/oracle/code/onnx/OnnxProtoBuilder.java
! cr-examples/onnx/src/main/java/oracle/code/onnx/OnnxRuntime.java
! cr-examples/onnx/src/main/java/oracle/code/onnx/compiler/OnnxPartialEvaluator.java
! cr-examples/onnx/src/main/java/oracle/code/onnx/compiler/OnnxTransformer.java
! cr-examples/onnx/src/main/java/oracle/code/onnx/ir/ExplicitOnnxOps.java
! cr-examples/onnx/src/test/java/oracle/code/onnx/CNNTest.java
! cr-examples/onnx/src/test/java/oracle/code/onnx/SimpleTest.java


From asotona at openjdk.org  Mon Mar 24 17:21:21 2025
From: asotona at openjdk.org (Adam Sotona)
Date: Mon, 24 Mar 2025 17:21:21 GMT
Subject: [code-reflection] Integrated: ExplicitOnnxOps.If implementation of
 captured values and initializers
In-Reply-To: <9pGY7kCbBiYjrFHzelwweejUpCFEzlEQeXLLR0HUYuU=.21c0b04d-42a1-4cd0-a185-d2e2c296384b@github.com>
References: <9pGY7kCbBiYjrFHzelwweejUpCFEzlEQeXLLR0HUYuU=.21c0b04d-42a1-4cd0-a185-d2e2c296384b@github.com>
Message-ID: <chr9kA4cDYOuKfDmJo1l0yFfzMYOTQi6blNrpS3aqG8=.de0ad630-de1c-4384-8fc5-ecafe55b3cbe@github.com>

On Fri, 14 Mar 2025 19:41:31 GMT, Adam Sotona <asotona at openjdk.org> wrote:

> ExplicitOnnxOps.If implementation of captured values and initializers - another attempt to do it right
> 
> + raw minimal support of variadic inputs and outputs
> + continuation work on If and Loop

This pull request has now been integrated.

Changeset: 15fa02fc
Author:    Adam Sotona <asotona at openjdk.org>
URL:       https://git.openjdk.org/babylon/commit/15fa02fc338e7746093ea253f513da393cfa3492
Stats:     827 lines in 10 files changed: 441 ins; 271 del; 115 mod

ExplicitOnnxOps.If implementation of captured values and initializers

-------------

PR: https://git.openjdk.org/babylon/pull/359

From psandoz at openjdk.org  Mon Mar 24 21:00:27 2025
From: psandoz at openjdk.org (Paul Sandoz)
Date: Mon, 24 Mar 2025 21:00:27 GMT
Subject: [code-reflection] RFR: Represents literals of type byte/short as
 permitted in the Java language [v4]
In-Reply-To: <3QCW0HinX6I9ASKSm7pnY978Ovf8TUePZz0-7mY5Ik0=.393b4b44-3382-41f6-8550-48ad7698c57a@github.com>
References: <zbICBVLiPQJAIloGr_22qhNGXZA--UdjKLeoCH_bmBU=.b9a87fd0-5918-4ef7-93d2-b2103a6427ea@github.com>
 <3QCW0HinX6I9ASKSm7pnY978Ovf8TUePZz0-7mY5Ik0=.393b4b44-3382-41f6-8550-48ad7698c57a@github.com>
Message-ID: <HUKVdpCkTer5OFouP5ab_PcHbXG2PtHqhM7KEO_xzjs=.24a86194-9f40-40c0-bb6e-dafc9ce90787@github.com>

On Wed, 12 Mar 2025 23:08:37 GMT, Mourad Abbay <mabbay at openjdk.org> wrote:

>> Before this PR, we had literals of type byte/short in the code model, but that's not permitted by the Java language. In Java, a literals of type byte/short is a literal of type int + conversion to the type byte/short.
>
> Mourad Abbay has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Add tests to make sure literals of type byte/short are created the way Java language permit.

Looks good.
@grfrost this might impact C99 gen (see example tests, you could check if the variable's initial value is the result of a primitive conversion op and check if value to be converted is the result of a constant op)

-------------

Marked as reviewed by psandoz (Lead).

PR Review: https://git.openjdk.org/babylon/pull/351#pullrequestreview-2711663439

From asotona at openjdk.org  Tue Mar 25 10:44:07 2025
From: asotona at openjdk.org (Adam Sotona)
Date: Tue, 25 Mar 2025 10:44:07 GMT
Subject: git: openjdk/babylon: code-reflection: Shape auto-dimension
 calculation from data size
Message-ID: <d4aea9ec-3428-4fea-915f-fb6e9fbf7e3d@openjdk.org>

Changeset: 82e9f905
Branch: code-reflection
Author:    Adam Sotona <asotona at openjdk.org>
Date:      2025-03-25 10:41:54 +0000
URL:       https://git.openjdk.org/babylon/commit/82e9f90562b8b40e52f0feb766bef3d9f9fcad84

Shape auto-dimension calculation from data size

! cr-examples/onnx/src/main/java/oracle/code/onnx/OnnxRuntime.java
! cr-examples/onnx/src/test/java/oracle/code/onnx/CNNTest.java


From asotona at openjdk.org  Tue Mar 25 10:44:19 2025
From: asotona at openjdk.org (Adam Sotona)
Date: Tue, 25 Mar 2025 10:44:19 GMT
Subject: [code-reflection] Integrated: Shape auto-dimension calculation from
 data size
In-Reply-To: <lgAUaGJjmNSY73uWNaQ5jQ0Fk8GSTPHN9pTgqt9vYVA=.ba483d8b-fdb8-4681-a005-39f675abf500@github.com>
References: <lgAUaGJjmNSY73uWNaQ5jQ0Fk8GSTPHN9pTgqt9vYVA=.ba483d8b-fdb8-4681-a005-39f675abf500@github.com>
Message-ID: <GXn8qzR4uz2O9pefDBi3wlYBjntj7DBTiXrjEcMpG1A=.76265339-f8e8-44b3-8fb1-31defb2f1575@github.com>

On Wed, 12 Mar 2025 06:47:15 GMT, Adam Sotona <asotona at openjdk.org> wrote:

> Implemented auto-dimension calculation from data size.
> 
> Shaped tensors construction required manual shape calculation based on the actual tensor data.
> This patch allows to set one of the shape dimensions to -1 and its size is automatically calculated from the actual tensor data size.

This pull request has now been integrated.

Changeset: 82e9f905
Author:    Adam Sotona <asotona at openjdk.org>
URL:       https://git.openjdk.org/babylon/commit/82e9f90562b8b40e52f0feb766bef3d9f9fcad84
Stats:     32 lines in 2 files changed: 29 ins; 1 del; 2 mod

Shape auto-dimension calculation from data size

-------------

PR: https://git.openjdk.org/babylon/pull/347

From psandoz at openjdk.org  Tue Mar 25 19:57:34 2025
From: psandoz at openjdk.org (Paul Sandoz)
Date: Tue, 25 Mar 2025 19:57:34 GMT
Subject: [code-reflection] RFR: Support storing the code that builds the
 code model [v19]
In-Reply-To: <NtS2fc1trqX8NhsMhmOP8P8YyFfi3MiK5xHHdjh9yBE=.abe3926b-3c52-46f1-93e6-45bacf2fda53@github.com>
References: <dexv8sQV_NcJsJIgMopd_M4wSWoZya9dU0ACwDtEL0E=.4ac0b80f-24fb-440f-a18d-bbfd07a681bc@github.com>
 <NtS2fc1trqX8NhsMhmOP8P8YyFfi3MiK5xHHdjh9yBE=.abe3926b-3c52-46f1-93e6-45bacf2fda53@github.com>
Message-ID: <Qk-nzXd0aOkab15M-x_Exrk1QQ4PMnJpzsyK3lqwulc=.1dcce742-da2c-4d09-bc50-8044525e6962@github.com>

On Wed, 19 Mar 2025 03:20:56 GMT, Mourad Abbay <mabbay at openjdk.org> wrote:

>> In this PR we allow the user to decide how to store the code model. The first option is `TEXT`, if this is selected, we store the textual representation of the code model. The second option is `CODE_BUILDR`, if this is selected, we store the code that builds the code model. All work done here is around the second option, because the first option is already supported.
>
> Mourad Abbay has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Don't rely on the assumption of 1d array

I cannot tell what may be left to address for Maurizio's comments and what we can defer. If Maurizio is ok to integrate then i am ok.

It seems we can follow up later on the following:
- avoiding the insertion of Vars
- uniformity of method signatures that produce code models
- adjust the new operation to support an invocation descriptor attribute, similar to the invoke operation.

-------------

PR Review: https://git.openjdk.org/babylon/pull/305#pullrequestreview-2714997295

From psandoz at openjdk.org  Tue Mar 25 19:57:34 2025
From: psandoz at openjdk.org (Paul Sandoz)
Date: Tue, 25 Mar 2025 19:57:34 GMT
Subject: [code-reflection] RFR: Support storing the code that builds the
 code model [v18]
In-Reply-To: <4Q4bmumCJYqik0o1Zh0bh3KslXYfls-6L_s_v2l44Y4=.1d61c2e5-3c19-40cb-9dbf-b362616f4f8d@github.com>
References: <dexv8sQV_NcJsJIgMopd_M4wSWoZya9dU0ACwDtEL0E=.4ac0b80f-24fb-440f-a18d-bbfd07a681bc@github.com>
 <rLtW9Wv92MNgUIywKaiPdoBs8O5-5EXNlu17J8tRflk=.3e0aa614-55db-4e86-a5a0-99e8e51758d8@github.com>
 <UZ2Z1W9PyhkEv4ucTxh-zfBISS__-0xiY5PGL4p67qw=.9a8dfbfd-3809-4fba-891f-543bead72d6b@github.com>
 <_vyINIwx1fIfdBtHZYt9VgABP7AIE8Q2bcsFaprRJAc=.2b84048a-69b1-4576-aca0-86caf814b280@github.com>
 <8vDjuCkLBgFFnPK_tYycS420iPcfeMK9JqtsStKzTp8=.66587d94-c595-4d24-9c1d-b787867e71b3@github.com>
 <4Q4bmumCJYqik0o1Zh0bh3KslXYfls-6L_s_v2l44Y4=.1d61c2e5-3c19-40cb-9dbf-b362616f4f8d@github.com>
Message-ID: <Co5ZOdqnd-iJO8wEQmXgZaHKi5j6fAaNDVU02m7KmXY=.1538f80f-559b-4a80-b89f-191240601068@github.com>

On Wed, 19 Mar 2025 09:22:59 GMT, Maurizio Cimadamore <mcimadamore at openjdk.org> wrote:

>> We can turn opMethod into one with no param and load these fields inside its body. But this removes the flexibility we have now (i.e. we can choose the OpFactory and TypeElementFactory we want to pass).
>
>> We can turn opMethod into one with no param and load these fields inside its body.
> 
> That is what I was asking about -- while I agree what we have seems more flexible, we don't seem to use this flexibility anywhere -- as the method that turns a `Quotable` into a `Quoted` works with hardwired op and type factories. But maybe this has been already discussed -- in which case it's fine.

Alternatively use a consistent the method signature with the parameters even if they are not used. Uniformity in the method signature is i think a nice property we should aim for, but could be followed up later.

-------------

PR Review Comment: https://git.openjdk.org/babylon/pull/305#discussion_r2012825819

From psandoz at openjdk.org  Tue Mar 25 19:57:34 2025
From: psandoz at openjdk.org (Paul Sandoz)
Date: Tue, 25 Mar 2025 19:57:34 GMT
Subject: [code-reflection] RFR: Support storing the code that builds the
 code model [v18]
In-Reply-To: <bhrfaSv-BcS4ylQIM_WbwPzDnPfY12Fb4MHV_LWWZwk=.4dfad721-6331-4640-8019-ff4d7985b027@github.com>
References: <dexv8sQV_NcJsJIgMopd_M4wSWoZya9dU0ACwDtEL0E=.4ac0b80f-24fb-440f-a18d-bbfd07a681bc@github.com>
 <rLtW9Wv92MNgUIywKaiPdoBs8O5-5EXNlu17J8tRflk=.3e0aa614-55db-4e86-a5a0-99e8e51758d8@github.com>
 <UZ2Z1W9PyhkEv4ucTxh-zfBISS__-0xiY5PGL4p67qw=.9a8dfbfd-3809-4fba-891f-543bead72d6b@github.com>
 <_vyINIwx1fIfdBtHZYt9VgABP7AIE8Q2bcsFaprRJAc=.2b84048a-69b1-4576-aca0-86caf814b280@github.com>
 <8vDjuCkLBgFFnPK_tYycS420iPcfeMK9JqtsStKzTp8=.66587d94-c595-4d24-9c1d-b787867e71b3@github.com>
 <4Q4bmumCJYqik0o1Zh0bh3KslXYfls-6L_s_v2l44Y4=.1d61c2e5-3c19-40cb-9dbf-b362616f4f8d@github.com>
 <bhrfaSv-BcS4ylQIM_WbwPzDnPfY12Fb4MHV_LWWZwk=.4dfad721-6331-4640-8019-ff4d7985b027@github.com>
Message-ID: <Ptvm5TNnKmcKCPZilRqSTjdPEpvngV1zE6jR6A8h-6c=.928f224a-5ae5-42b6-9db4-299c5a28996c@github.com>

On Wed, 19 Mar 2025 09:21:03 GMT, Maurizio Cimadamore <mcimadamore at openjdk.org> wrote:

>> Apologies -- I meant whether the `writeTo` should be removed :-)
>
>> But I decided to have a seperate transformation as a quick fix. Later I will change the OpBuilder.
> 
> As stated in another comment, I'm not sure changing the op builder is needed -- as the javac code can easily workaround what the op builder does (e.g. what it does doesn't seem obviously wrong to me). But if the decision is to fix op builder that's fine too

It would be nice to directly avoid changing OpBuilder (investing in a SSA to Var transformation, using liveness analysis, is probably the general right approach), but there might be a more focused approach to generating the AST so we can avoid this.

-------------

PR Review Comment: https://git.openjdk.org/babylon/pull/305#discussion_r2012835536

From mcimadamore at openjdk.org  Wed Mar 26 09:53:31 2025
From: mcimadamore at openjdk.org (Maurizio Cimadamore)
Date: Wed, 26 Mar 2025 09:53:31 GMT
Subject: [code-reflection] RFR: Support storing the code that builds the
 code model [v18]
In-Reply-To: <Co5ZOdqnd-iJO8wEQmXgZaHKi5j6fAaNDVU02m7KmXY=.1538f80f-559b-4a80-b89f-191240601068@github.com>
References: <dexv8sQV_NcJsJIgMopd_M4wSWoZya9dU0ACwDtEL0E=.4ac0b80f-24fb-440f-a18d-bbfd07a681bc@github.com>
 <rLtW9Wv92MNgUIywKaiPdoBs8O5-5EXNlu17J8tRflk=.3e0aa614-55db-4e86-a5a0-99e8e51758d8@github.com>
 <UZ2Z1W9PyhkEv4ucTxh-zfBISS__-0xiY5PGL4p67qw=.9a8dfbfd-3809-4fba-891f-543bead72d6b@github.com>
 <_vyINIwx1fIfdBtHZYt9VgABP7AIE8Q2bcsFaprRJAc=.2b84048a-69b1-4576-aca0-86caf814b280@github.com>
 <8vDjuCkLBgFFnPK_tYycS420iPcfeMK9JqtsStKzTp8=.66587d94-c595-4d24-9c1d-b787867e71b3@github.com>
 <4Q4bmumCJYqik0o1Zh0bh3KslXYfls-6L_s_v2l44Y4=.1d61c2e5-3c19-40cb-9dbf-b362616f4f8d@github.com>
 <Co5ZOdqnd-iJO8wEQmXgZaHKi5j6fAaNDVU02m7KmXY=.1538f80f-559b-4a80-b89f-191240601068@github.com>
Message-ID: <XFN40-2Q20lDayUZFl6PJ6hgVEihmxyD6QTB97b3RZI=.1f826ce4-c42e-4b42-91fe-25efc96aad75@github.com>

On Tue, 25 Mar 2025 19:44:58 GMT, Paul Sandoz <psandoz at openjdk.org> wrote:

>>> We can turn opMethod into one with no param and load these fields inside its body.
>> 
>> That is what I was asking about -- while I agree what we have seems more flexible, we don't seem to use this flexibility anywhere -- as the method that turns a `Quotable` into a `Quoted` works with hardwired op and type factories. But maybe this has been already discussed -- in which case it's fine.
>
> Alternatively use a consistent the method signature with the parameters even if they are not used. Uniformity in the method signature is i think a nice property we should aim for, but could be followed up later.

Yep - I agree we can follow up later.

-------------

PR Review Comment: https://git.openjdk.org/babylon/pull/305#discussion_r2013748080

From mcimadamore at openjdk.org  Wed Mar 26 09:53:31 2025
From: mcimadamore at openjdk.org (Maurizio Cimadamore)
Date: Wed, 26 Mar 2025 09:53:31 GMT
Subject: [code-reflection] RFR: Support storing the code that builds the
 code model [v18]
In-Reply-To: <hD63TlJ-G8jkbGGmJ85CNk7F-o9rdoytopiZpd21tgw=.5ffa9624-df81-4002-9e80-14efee391388@github.com>
References: <dexv8sQV_NcJsJIgMopd_M4wSWoZya9dU0ACwDtEL0E=.4ac0b80f-24fb-440f-a18d-bbfd07a681bc@github.com>
 <rLtW9Wv92MNgUIywKaiPdoBs8O5-5EXNlu17J8tRflk=.3e0aa614-55db-4e86-a5a0-99e8e51758d8@github.com>
 <JuZpObuzFUKUoRyO3UBQZpcjNAhzwhLIvh3B7-uoK74=.c1d4aa04-085a-4281-8b90-910b160fd9c8@github.com>
 <hD63TlJ-G8jkbGGmJ85CNk7F-o9rdoytopiZpd21tgw=.5ffa9624-df81-4002-9e80-14efee391388@github.com>
Message-ID: <UdL-YWpMQMDlG0uTP_qhib-UE-AAehPrlsFwEf4lA6o=.43e55e87-92c1-49d2-8b85-3c1b7c30232a@github.com>

On Thu, 13 Mar 2025 15:10:51 GMT, Maurizio Cimadamore <mcimadamore at openjdk.org> wrote:

>> src/jdk.incubator.code/share/classes/jdk/incubator/code/internal/CodeModelToAST.java line 117:
>> 
>>> 115:         var restype = typeElementToType(invokeOp.resultType());
>>> 116:         var type = new Type.MethodType(paramTypes, restype, List.nil(), syms.methodClass);
>>> 117:         var methodSym = new Symbol.MethodSymbol(flags, name, type,
>> 
>> Same as for field access (see below) -- we should not create a new method symbol from scratch but "resolve" a method in a given class symbol (using `Resolve::findInternalMethod`)
>
> I suggest to add two helper methods -- one takes a FieldDescriptor and returns a VarSymbol (using `Resolve::findFieldInternal` -- the other takes a MethodDescriptor and returns a MethodSymbol (using `Resolve::findMethodIntenal`. Then you rewrite the code here (and in field access) to use these methods.

This is related to the one above for var symbol -- creating method/var symbols directly in the backend is not the way to do things -- we should lean on `Resolve` instead. It would be nice to fix these -- but we can also take another pass at it later if required.

>> src/jdk.incubator.code/share/classes/jdk/incubator/code/internal/CodeModelToAST.java line 239:
>> 
>>> 237:     }
>>> 238: 
>>> 239:     public static CoreOp.FuncOp addVarsWhenNecessary(CoreOp.FuncOp funcOp) {
>> 
>> Maybe I'm missing something obvious. But if you have something like:
>> 
>> func @"f" ()java.util.Map -> {
>>     %0 : java.util.Map<java.lang.Integer, java.lang.Integer> = new @"func<java.util.HashMap>";
>>     %1 : int = constant @"1";
>>     %2 : int = constant @"2";
>>     %3 : java.lang.Object = invoke %0 %1 %2 @"java.util.Map::put(java.lang.Object, java.lang.Object)java.lang.Object";
>>     return %0;
>> }
>> 
>> 
>> Can't we translate as:
>> 
>> 
>> Map $0 = new HashMap();
>> int $1 = 1;
>> int $2 = 3;    
>> Object $3 = $0.put($1, $2);
>> return $0;
>> 
>> 
>> Note how this translation is very 1-1 and direct, and doesn't require any adaptation step in the model. We just emit a new local variable for each op we see in the body -- and then refer to operands via their corresponding (locally declared) variable.
>
> (of course we'd only do the above translation for ops that have a non-void result type)

We can follow up on this improved translation at a later point.

-------------

PR Review Comment: https://git.openjdk.org/babylon/pull/305#discussion_r2013758784
PR Review Comment: https://git.openjdk.org/babylon/pull/305#discussion_r2013751016

From mcimadamore at openjdk.org  Wed Mar 26 09:53:31 2025
From: mcimadamore at openjdk.org (Maurizio Cimadamore)
Date: Wed, 26 Mar 2025 09:53:31 GMT
Subject: [code-reflection] RFR: Support storing the code that builds the
 code model [v18]
In-Reply-To: <QQ-_TRdoc-nUMms-gcrcM0GQ7WCnAnqQ5KriOmzfCRY=.161b6225-32a5-4e00-ab35-81e3c2a8be30@github.com>
References: <dexv8sQV_NcJsJIgMopd_M4wSWoZya9dU0ACwDtEL0E=.4ac0b80f-24fb-440f-a18d-bbfd07a681bc@github.com>
 <rLtW9Wv92MNgUIywKaiPdoBs8O5-5EXNlu17J8tRflk=.3e0aa614-55db-4e86-a5a0-99e8e51758d8@github.com>
 <ftnpqrjPGs9YPOQm7MbEV8PZb2dGx92ga2-340cOuh0=.8a2f956a-f277-4009-b461-05e9d88363f5@github.com>
 <WbrP4ERmNzNCkHNRQvAr5Ff72Z5mX3nk1-KVNeyMRZc=.31ae7834-839d-4cf6-9577-ac5d5c6ba901@github.com>
 <QQ-_TRdoc-nUMms-gcrcM0GQ7WCnAnqQ5KriOmzfCRY=.161b6225-32a5-4e00-ab35-81e3c2a8be30@github.com>
Message-ID: <hf-qSnNJwBGOdRy3w3xfEOHhDEU4cJPgV0_Y4u09RW8=.0dd11079-ca99-4532-9fcc-4065a6378b3e@github.com>

On Wed, 19 Mar 2025 09:26:36 GMT, Maurizio Cimadamore <mcimadamore at openjdk.org> wrote:

>> I agree on all points. We can have a `new.array` operation and then model class creation as an `invoke` op with method name being "init" ?
>
>> I agree on all points. We can have a `new.array` operation and then model class creation as an `invoke` op with method name being "init" ?
> 
> not sure I follow -- I was talking about class construction -- arrays are a bit special (e.g. there's no array constructor descriptor, unlike for classes). But a constructor call does share a lot of similarities with a method call, and there should be reusable code between method and constructors -- as it happens inside the javac code.

This can be deferred until we have a better op to model instance creation expression.

-------------

PR Review Comment: https://git.openjdk.org/babylon/pull/305#discussion_r2013749979

From mcimadamore at openjdk.org  Wed Mar 26 09:53:34 2025
From: mcimadamore at openjdk.org (Maurizio Cimadamore)
Date: Wed, 26 Mar 2025 09:53:34 GMT
Subject: [code-reflection] RFR: Support storing the code that builds the
 code model [v18]
In-Reply-To: <EtpyFI5TXv5K0vKcAsucd7PBtFRQRwaEk-vo_xQXDSg=.60a9033a-a7a3-40a8-abe5-de1331f93627@github.com>
References: <dexv8sQV_NcJsJIgMopd_M4wSWoZya9dU0ACwDtEL0E=.4ac0b80f-24fb-440f-a18d-bbfd07a681bc@github.com>
 <rLtW9Wv92MNgUIywKaiPdoBs8O5-5EXNlu17J8tRflk=.3e0aa614-55db-4e86-a5a0-99e8e51758d8@github.com>
 <EtpyFI5TXv5K0vKcAsucd7PBtFRQRwaEk-vo_xQXDSg=.60a9033a-a7a3-40a8-abe5-de1331f93627@github.com>
Message-ID: <_Gbrv1DjyKxDc2ZMZFgIkHxx7hsgFB7lyBxEMWStuI8=.d6d92694-68c2-4fe3-8d5e-b6e48d76f825@github.com>

On Thu, 13 Mar 2025 14:45:51 GMT, Maurizio Cimadamore <mcimadamore at openjdk.org> wrote:

>> Mourad Abbay has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains 20 additional commits since the last revision:
>> 
>>  - Merge branch 'code-reflection' into code-model-storage-option
>>  - Load OpFactory and TypeElementFactory before invocation of opMethod only if the opMethod has there params.
>>  - Fix the remaining compiler tests failures
>>  - Fix some of the test failures (3 remains)
>>  - Fix the remaining test failures of SwitchExpressionTest2
>>  - Fix almost all test failures of SwitchExpressionTest2 (one remaining)
>>  - Pass arrayType instead of eleType in OpBuilder.buildArray
>>  - Ensure that block params are inserted in the correct order
>>  - Add missing imports
>>  - Merge branch 'code-reflection' into code-model-storage-option
>>  - ... and 10 more: https://git.openjdk.org/babylon/compare/a405674e...966005ce
>
> src/jdk.incubator.code/share/classes/jdk/incubator/code/internal/CodeModelToAST.java line 160:
> 
>> 158:                 var argTypes = new ListBuffer<Type>();
>> 159:                 for (Value operand : newOp.operands()) {
>> 160:                     argTypes.add(typeElementToType(operand.type()));
> 
> This seems wrong -- and is related to the point I made above -- the type of the operands here are the types of the actual arguments of the constructor calls. So, if you have:
> 
> class Foo {
>      Foo(String string) { ... }
>  }
>  ```
>  
>  the method type of the constructor call has to say `Foo(String)`. But if you "infer" the type of the constructor argument from the actual argument, if the call is like this:
>  
>  ```
>  new Foo(null)
>  ```
>  
> We should probably use the op's `constructorType` to derive the type of the invoked constructor? (You use that to determine the constructor's owner type)

This can be deferred until we have a better op to model instance creation expression.

> src/jdk.incubator.code/share/classes/jdk/incubator/code/internal/CodeModelToAST.java line 192:
> 
>> 190:                 var type = typeElementToType(fieldLoadOp.resultType());
>> 191:                 var owner = typeElementToType(fieldLoadOp.fieldDescriptor().refType());
>> 192:                 var sym = new Symbol.VarSymbol(flags, name, type, owner.tsym);
> 
> This seems wrong -- the accessed field symbol should be findable in the scope of the class in which the field is declared. It seems to me that we need some piece of code that turns a method/field descriptor into a java symbol. Look at `Lower::lookupMethod` which uses `Resolve::resolveInternalMethod` -- I think here we should do that too.

This is borderline, but again, we can leave it as is for now (the risk is that if we have a bad model javac will just emit a reference to some non-existent field, which will likely cause non-sensical bytecode to be emitted).

-------------

PR Review Comment: https://git.openjdk.org/babylon/pull/305#discussion_r2013741734
PR Review Comment: https://git.openjdk.org/babylon/pull/305#discussion_r2013755517

From mcimadamore at openjdk.org  Wed Mar 26 09:53:34 2025
From: mcimadamore at openjdk.org (Maurizio Cimadamore)
Date: Wed, 26 Mar 2025 09:53:34 GMT
Subject: [code-reflection] RFR: Support storing the code that builds the
 code model [v18]
In-Reply-To: <Ptvm5TNnKmcKCPZilRqSTjdPEpvngV1zE6jR6A8h-6c=.928f224a-5ae5-42b6-9db4-299c5a28996c@github.com>
References: <dexv8sQV_NcJsJIgMopd_M4wSWoZya9dU0ACwDtEL0E=.4ac0b80f-24fb-440f-a18d-bbfd07a681bc@github.com>
 <rLtW9Wv92MNgUIywKaiPdoBs8O5-5EXNlu17J8tRflk=.3e0aa614-55db-4e86-a5a0-99e8e51758d8@github.com>
 <UZ2Z1W9PyhkEv4ucTxh-zfBISS__-0xiY5PGL4p67qw=.9a8dfbfd-3809-4fba-891f-543bead72d6b@github.com>
 <_vyINIwx1fIfdBtHZYt9VgABP7AIE8Q2bcsFaprRJAc=.2b84048a-69b1-4576-aca0-86caf814b280@github.com>
 <8vDjuCkLBgFFnPK_tYycS420iPcfeMK9JqtsStKzTp8=.66587d94-c595-4d24-9c1d-b787867e71b3@github.com>
 <4Q4bmumCJYqik0o1Zh0bh3KslXYfls-6L_s_v2l44Y4=.1d61c2e5-3c19-40cb-9dbf-b362616f4f8d@github.com>
 <bhrfaSv-BcS4ylQIM_WbwPzDnPfY12Fb4MHV_LWWZwk=.4dfad721-6331-4640-8019-ff4d7985b027@github.com>
 <Ptvm5TNnKmcKCPZilRqSTjdPEpvngV1zE6jR6A8h-6c=.928f224a-5ae5-42b6-9db4-299c5a28996c@github.com>
Message-ID: <6PCDNoOumuqiSvaApWaoEtYNguV2CEQVOy-x2x5aEP8=.a2a342b4-a57f-4d5f-98d7-ab088ec830ff@github.com>

On Tue, 25 Mar 2025 19:50:18 GMT, Paul Sandoz <psandoz at openjdk.org> wrote:

>>> But I decided to have a seperate transformation as a quick fix. Later I will change the OpBuilder.
>> 
>> As stated in another comment, I'm not sure changing the op builder is needed -- as the javac code can easily workaround what the op builder does (e.g. what it does doesn't seem obviously wrong to me). But if the decision is to fix op builder that's fine too
>
> It would be nice to directly avoid changing OpBuilder (investing in a SSA to Var transformation, using liveness analysis, is probably the general right approach), but there might be a more focused approach to generating the AST so we can avoid this.

Thanks for confirming @PaulSandoz -- we can defer this improvement to a follow up change -- and keep the var-adding transformation in this PR for now.

-------------

PR Review Comment: https://git.openjdk.org/babylon/pull/305#discussion_r2013743930