From psandoz at openjdk.org  Mon Feb  3 23:00:58 2025
From: psandoz at openjdk.org (Paul Sandoz)
Date: Mon, 3 Feb 2025 23:00:58 GMT
Subject: git: openjdk/babylon: code-reflection: Onnx
Message-ID: <223a7b08-b74b-4e58-8e4e-3a394a2227b2@openjdk.org>

Changeset: 3b874574
Branch: code-reflection
Author:    Paul Sandoz <psandoz at openjdk.org>
Date:      2025-02-03 22:59:02 +0000
URL:       https://git.openjdk.org/babylon/commit/3b87457432d4cfb147960c2edb2eabd25e50f6f1

Onnx

+ cr-examples/onnx/.gitignore
+ cr-examples/onnx/opgen/onnxOpSchemaToJSON.py
+ cr-examples/onnx/opgen/pom.xml
+ cr-examples/onnx/opgen/src/main/java/oracle/code/json/Json.java
+ cr-examples/onnx/opgen/src/main/java/oracle/code/json/JsonArray.java
+ cr-examples/onnx/opgen/src/main/java/oracle/code/json/JsonArrayImpl.java
+ cr-examples/onnx/opgen/src/main/java/oracle/code/json/JsonBoolean.java
+ cr-examples/onnx/opgen/src/main/java/oracle/code/json/JsonBooleanImpl.java
+ cr-examples/onnx/opgen/src/main/java/oracle/code/json/JsonDocumentInfo.java
+ cr-examples/onnx/opgen/src/main/java/oracle/code/json/JsonGenerator.java
+ cr-examples/onnx/opgen/src/main/java/oracle/code/json/JsonNull.java
+ cr-examples/onnx/opgen/src/main/java/oracle/code/json/JsonNullImpl.java
+ cr-examples/onnx/opgen/src/main/java/oracle/code/json/JsonNumber.java
+ cr-examples/onnx/opgen/src/main/java/oracle/code/json/JsonNumberImpl.java
+ cr-examples/onnx/opgen/src/main/java/oracle/code/json/JsonObject.java
+ cr-examples/onnx/opgen/src/main/java/oracle/code/json/JsonObjectImpl.java
+ cr-examples/onnx/opgen/src/main/java/oracle/code/json/JsonParseException.java
+ cr-examples/onnx/opgen/src/main/java/oracle/code/json/JsonParser.java
+ cr-examples/onnx/opgen/src/main/java/oracle/code/json/JsonString.java
+ cr-examples/onnx/opgen/src/main/java/oracle/code/json/JsonStringImpl.java
+ cr-examples/onnx/opgen/src/main/java/oracle/code/json/JsonValue.java
+ cr-examples/onnx/opgen/src/main/java/oracle/code/json/JsonValueImpl.java
+ cr-examples/onnx/opgen/src/main/java/oracle/code/onnx/OpSchema.java
+ cr-examples/onnx/opgen/src/main/java/oracle/code/onnx/Tensor.java
+ cr-examples/onnx/opgen/src/main/java/oracle/code/onnx/opgen/IndentWriter.java
+ cr-examples/onnx/opgen/src/main/java/oracle/code/onnx/opgen/OpGen.java
+ cr-examples/onnx/opgen/src/main/java/oracle/code/onnx/opgen/OpSchemaParser.java
+ cr-examples/onnx/opgen/src/main/java/oracle/code/onnx/opgen/OperatorGen.java
+ cr-examples/onnx/pom.xml
+ cr-examples/onnx/src/main/java/oracle/code/onnx/ExplicitOnnxOperators.java
+ cr-examples/onnx/src/main/java/oracle/code/onnx/OnnxInterpreter.java
+ cr-examples/onnx/src/main/java/oracle/code/onnx/OnnxNumber.java
+ cr-examples/onnx/src/main/java/oracle/code/onnx/OnnxOperators.java
+ cr-examples/onnx/src/main/java/oracle/code/onnx/Tensor.java
+ cr-examples/onnx/src/main/java/oracle/code/onnx/ir/OnnxOp.java
+ cr-examples/onnx/src/main/java/oracle/code/onnx/ir/OnnxOps.java
+ cr-examples/onnx/src/main/java/oracle/code/onnx/ir/OnnxType.java
+ cr-examples/onnx/src/test/java/oracle/code/onnx/CNNTest.java


From psandoz at openjdk.org  Mon Feb  3 23:04:38 2025
From: psandoz at openjdk.org (Paul Sandoz)
Date: Mon, 3 Feb 2025 23:04:38 GMT
Subject: [code-reflection] Integrated: Onnx
Message-ID: <Nz971Xc9kXQ7Zbu_B7O2hEBqUjzKHGSfgnhsIqLpMAo=.2ad88a90-1b17-4ac5-826b-3d5ed96d5304@github.com>

Initial work on a Java ONNX script prototype, a Java equivalent of (Python) [ONNX script](https://github.com/microsoft/onnxscript). The intention is to enable developers to author ONNX functions using a subset of Java and enable easy interaction between Java and the ONNX Runtime (ORT).

Here are the main aspects:
- A Java ONNX function is a Java method whose implementation contains a restricted set of expressions and statements, primarily consisting of invocation expressions to the Java ONNX operator API.
- A Java ONNX function is identified for code reflection so its Java code model can be accessed and translated to an ONNX code model consisting of operations instantiated from the Java ONNX IR API that model ONNX operators.
- A Java ONNX model is exportable to an ONNX model, in protobuf, which can be executed by the ONNX runtime. The Foriegn Function and Memory API is used to interoperate with the ONNX runtime.
- In addition the Java ONNX function may be invoked. Each invocation to a method of the Java ONNX operator API is interpreted dynamically using the ONNX runtime. This is useful for debugging. 

This PR contains the Java ONNX operator API and Java ONNX IR API. Both APIs are generated from a representation of the ONNX schema, which contains a description of each operator supported by ONNX. A JSON document representing the ONNX schema is generated from a Python script. The JSON document is parsed in Java into a simple in-memory representation and that representation is traversed to generate the source code of both APIs. The Java ONNX IR API contains sufficient schema information to reflect over ONNX operators.

A simple test, a CNN implementation, is contributed by @Craigacp. This will initially serve as the primary test case for inference on MNIST data, with weights/biases extracted from an ONNX model exported from an equivalent model trained in Python. Over time we will add further examples.

-------------

Commit messages:
 - Remove trailing white space.
 - Remove trailing white space
 - Update gitingore
 - Add JSON API and impl
 - Update JSON API.
 - License
 - Expected model output for CNN test
 - Code model of CNN example.
 - Fix types.
 - Optional outputs
 - ... and 13 more: https://git.openjdk.org/babylon/compare/79cd5cf3...28fcc3ea

Changes: https://git.openjdk.org/babylon/pull/306/files
  Webrev: https://webrevs.openjdk.org/?repo=babylon&pr=306&range=00
  Stats: 38221 lines in 38 files changed: 38221 ins; 0 del; 0 mod
  Patch: https://git.openjdk.org/babylon/pull/306.diff
  Fetch: git fetch https://git.openjdk.org/babylon.git pull/306/head:pull/306

PR: https://git.openjdk.org/babylon/pull/306

From psandoz at openjdk.org  Mon Feb  3 23:04:39 2025
From: psandoz at openjdk.org (Paul Sandoz)
Date: Mon, 3 Feb 2025 23:04:39 GMT
Subject: [code-reflection] Integrated: Onnx
In-Reply-To: <Nz971Xc9kXQ7Zbu_B7O2hEBqUjzKHGSfgnhsIqLpMAo=.2ad88a90-1b17-4ac5-826b-3d5ed96d5304@github.com>
References: <Nz971Xc9kXQ7Zbu_B7O2hEBqUjzKHGSfgnhsIqLpMAo=.2ad88a90-1b17-4ac5-826b-3d5ed96d5304@github.com>
Message-ID: <0_-zqulZ3WuSoFZbmOMoEOVpMGubcFVGgig3qHy3zf4=.b3ef3fdf-c3fb-4d87-8b22-b53afbfaf064@github.com>

On Mon, 3 Feb 2025 22:32:51 GMT, Paul Sandoz <psandoz at openjdk.org> wrote:

> Initial work on a Java ONNX script prototype, a Java equivalent of (Python) [ONNX script](https://github.com/microsoft/onnxscript). The intention is to enable developers to author ONNX functions using a subset of Java and enable easy interaction between Java and the ONNX Runtime (ORT).
> 
> Here are the main aspects:
> - A Java ONNX function is a Java method whose implementation contains a restricted set of expressions and statements, primarily consisting of invocation expressions to the Java ONNX operator API.
> - A Java ONNX function is identified for code reflection so its Java code model can be accessed and translated to an ONNX code model consisting of operations instantiated from the Java ONNX IR API that model ONNX operators.
> - A Java ONNX model is exportable to an ONNX model, in protobuf, which can be executed by the ONNX runtime. The Foriegn Function and Memory API is used to interoperate with the ONNX runtime.
> - In addition the Java ONNX function may be invoked. Each invocation to a method of the Java ONNX operator API is interpreted dynamically using the ONNX runtime. This is useful for debugging. 
> 
> This PR contains the Java ONNX operator API and Java ONNX IR API. Both APIs are generated from a representation of the ONNX schema, which contains a description of each operator supported by ONNX. A JSON document representing the ONNX schema is generated from a Python script. The JSON document is parsed in Java into a simple in-memory representation and that representation is traversed to generate the source code of both APIs. The Java ONNX IR API contains sufficient schema information to reflect over ONNX operators.
> 
> A simple test, a CNN implementation, is contributed by @Craigacp. This will initially serve as the primary test case for inference on MNIST data, with weights/biases extracted from an ONNX model exported from an equivalent model trained in Python. Over time we will add further examples.

This pull request has now been integrated.

Changeset: 3b874574
Author:    Paul Sandoz <psandoz at openjdk.org>
URL:       https://git.openjdk.org/babylon/commit/3b87457432d4cfb147960c2edb2eabd25e50f6f1
Stats:     38221 lines in 38 files changed: 38221 ins; 0 del; 0 mod

Onnx

-------------

PR: https://git.openjdk.org/babylon/pull/306

From psandoz at openjdk.org  Wed Feb  5 23:35:49 2025
From: psandoz at openjdk.org (Paul Sandoz)
Date: Wed, 5 Feb 2025 23:35:49 GMT
Subject: [code-reflection] RFR: Very basic ONNX transformer
Message-ID: <FVX-DJSO7xE0lEzbJJ6fN--VO7_uGtoABKdZHmkDJnw=.5e3ea88f-6a7a-4efb-a5a0-37d9541a2e8c@github.com>

An ONNX transformer that transforms the Java code model of a Java ONNX function to an ONNX code model, from which we can create an ONNX model. This is just the first attempt and it will likely take many iterations to get this right.

There are three stages to the transformation:
1. Partially evaluate the Java code model, using a simple interpreter. Any operation that depends on a function input parameter, directly or indirectly, is ignored. This leaves operations that depend on constants, directly or indirectly, and since attributes are constants we can evaluate them and associate a list of attributes with each invocation to an ONNX operator.
2. Transform the Java code model to an ONNX model, only transforming operations whose results have not been evaluated namely all operations that depend, directly or indirectly on function inputs. An invocation to an ONNX operator is transformed to an instance of the corresponding ONNX operation, and corresponding attributes evaluated in the first step are passed directly as arguments to the operation factory method.
3. Clean up the ONNX model, transforming to pure SSA, and removing any operation whose result is unused.

So far we avoided any need to compute tensor shape information and we may not need it, although it would be nice to have if we could defer to the ONNX runtime to compute for each operation.

Currently we assume that the Java ONNX function accepts weights as parameters. This simplifies the transformation and modeling. When we export the ONNX code model a prefix of tensors can be given for the weights. We can revisit this modeling later on and improve the developer experience.

This PR also adjusts the mapping of ONNX attribute type to Java class. `INT` should map to `long`, and `INTS` to `long[]`.

-------------

Commit messages:
 - Very basic ONNX transformer.

Changes: https://git.openjdk.org/babylon/pull/308/files
  Webrev: https://webrevs.openjdk.org/?repo=babylon&pr=308&range=00
  Stats: 2090 lines in 8 files changed: 897 ins; 22 del; 1171 mod
  Patch: https://git.openjdk.org/babylon/pull/308.diff
  Fetch: git fetch https://git.openjdk.org/babylon.git pull/308/head:pull/308

PR: https://git.openjdk.org/babylon/pull/308

From psandoz at openjdk.org  Wed Feb  5 23:39:49 2025
From: psandoz at openjdk.org (Paul Sandoz)
Date: Wed, 5 Feb 2025 23:39:49 GMT
Subject: git: openjdk/babylon: code-reflection: Very basic ONNX transformer
Message-ID: <9f3a4a95-0cc0-42ed-b1db-b4fabd4f7817@openjdk.org>

Changeset: 408e7b56
Branch: code-reflection
Author:    Paul Sandoz <psandoz at openjdk.org>
Date:      2025-02-05 23:37:25 +0000
URL:       https://git.openjdk.org/babylon/commit/408e7b56db440349a7d11871a84a91a372b75f8f

Very basic ONNX transformer

! cr-examples/onnx/opgen/src/main/java/oracle/code/onnx/OpSchema.java
! cr-examples/onnx/src/main/java/oracle/code/onnx/ExplicitOnnxOperators.java
! cr-examples/onnx/src/main/java/oracle/code/onnx/OnnxOperators.java
= cr-examples/onnx/src/main/java/oracle/code/onnx/compiler/InvokableLeafOps.java
+ cr-examples/onnx/src/main/java/oracle/code/onnx/compiler/OnnxPartialEvaluator.java
+ cr-examples/onnx/src/main/java/oracle/code/onnx/compiler/OnnxTransformer.java
! cr-examples/onnx/src/main/java/oracle/code/onnx/ir/OnnxOps.java
! cr-examples/onnx/src/test/java/oracle/code/onnx/CNNTest.java


From psandoz at openjdk.org  Wed Feb  5 23:40:39 2025
From: psandoz at openjdk.org (Paul Sandoz)
Date: Wed, 5 Feb 2025 23:40:39 GMT
Subject: [code-reflection] Integrated: Very basic ONNX transformer
In-Reply-To: <FVX-DJSO7xE0lEzbJJ6fN--VO7_uGtoABKdZHmkDJnw=.5e3ea88f-6a7a-4efb-a5a0-37d9541a2e8c@github.com>
References: <FVX-DJSO7xE0lEzbJJ6fN--VO7_uGtoABKdZHmkDJnw=.5e3ea88f-6a7a-4efb-a5a0-37d9541a2e8c@github.com>
Message-ID: <NQFgMXHuutk-OUlA7QX_2UKSqfPWpdXNYsWqtO5GdM8=.efb38148-1ff4-4577-b331-6d9711155c99@github.com>

On Wed, 5 Feb 2025 23:18:14 GMT, Paul Sandoz <psandoz at openjdk.org> wrote:

> An ONNX transformer that transforms the Java code model of a Java ONNX function to an ONNX code model, from which we can create an ONNX model. This is just the first attempt and it will likely take many iterations to get this right.
> 
> There are three stages to the transformation:
> 1. Partially evaluate the Java code model, using a simple interpreter. Any operation that depends on a function input parameter, directly or indirectly, is ignored. This leaves operations that depend on constants, directly or indirectly, and since attributes are constants we can evaluate them and associate a list of attributes with each invocation to an ONNX operator.
> 2. Transform the Java code model to an ONNX model, only transforming operations whose results have not been evaluated namely all operations that depend, directly or indirectly on function inputs. An invocation to an ONNX operator is transformed to an instance of the corresponding ONNX operation, and corresponding attributes evaluated in the first step are passed directly as arguments to the operation factory method.
> 3. Clean up the ONNX model, transforming to pure SSA, and removing any operation whose result is unused.
> 
> So far we avoided any need to compute tensor shape information and we may not need it, although it would be nice to have if we could defer to the ONNX runtime to compute for each operation.
> 
> Currently we assume that the Java ONNX function accepts weights as parameters. This simplifies the transformation and modeling. When we export the ONNX code model a prefix of tensors can be given for the weights. We can revisit this modeling later on and improve the developer experience.
> 
> This PR also adjusts the mapping of ONNX attribute type to Java class. `INT` should map to `long`, and `INTS` to `long[]`.

This pull request has now been integrated.

Changeset: 408e7b56
Author:    Paul Sandoz <psandoz at openjdk.org>
URL:       https://git.openjdk.org/babylon/commit/408e7b56db440349a7d11871a84a91a372b75f8f
Stats:     2090 lines in 8 files changed: 897 ins; 22 del; 1171 mod

Very basic ONNX transformer

-------------

PR: https://git.openjdk.org/babylon/pull/308

From gfrost at openjdk.org  Thu Feb  6 10:54:05 2025
From: gfrost at openjdk.org (Gary Frost)
Date: Thu, 6 Feb 2025 10:54:05 GMT
Subject: git: openjdk/babylon: code-reflection: Cleanup of HAT env.bash,
 bld and hatrun scripts
Message-ID: <6657bde6-8b37-41b2-9dec-b4b56a5d6226@openjdk.org>

Changeset: 1a6ea358
Branch: code-reflection
Author:    Gary Frost <gfrost at openjdk.org>
Date:      2025-02-06 10:52:09 +0000
URL:       https://git.openjdk.org/babylon/commit/1a6ea35817fb72cd113c216c10efe2b2e151b7a4

Cleanup of HAT env.bash, bld and hatrun scripts

! hat/env.bash
! hat/hatrun
! hat/intellij/.idea/compiler.xml


From gfrost at openjdk.org  Thu Feb  6 10:56:14 2025
From: gfrost at openjdk.org (Gary Frost)
Date: Thu, 6 Feb 2025 10:56:14 GMT
Subject: [code-reflection] Integrated: Cleanup of HAT env.bash,
 bld and hatrun scripts
Message-ID: <mcVFmyu-m88fhf74jJd_x_oOC16ocLzaALVj6n-nbAw=.dba10c51-d483-413c-ba70-dab67a93fcb2@github.com>

Added a jextract check in env.bash 

hatrun's usage now shows way to specify backend (i.e. ffi-opencl vs just opencl) in preparation for jextracted backends

-------------

Commit messages:
 - Cleanup of HAT env.bash, bld and hatrun scripts

Changes: https://git.openjdk.org/babylon/pull/309/files
  Webrev: https://webrevs.openjdk.org/?repo=babylon&pr=309&range=00
  Stats: 24 lines in 3 files changed: 11 ins; 1 del; 12 mod
  Patch: https://git.openjdk.org/babylon/pull/309.diff
  Fetch: git fetch https://git.openjdk.org/babylon.git pull/309/head:pull/309

PR: https://git.openjdk.org/babylon/pull/309

From gfrost at openjdk.org  Thu Feb  6 10:56:15 2025
From: gfrost at openjdk.org (Gary Frost)
Date: Thu, 6 Feb 2025 10:56:15 GMT
Subject: [code-reflection] Integrated: Cleanup of HAT env.bash,
 bld and hatrun scripts
In-Reply-To: <mcVFmyu-m88fhf74jJd_x_oOC16ocLzaALVj6n-nbAw=.dba10c51-d483-413c-ba70-dab67a93fcb2@github.com>
References: <mcVFmyu-m88fhf74jJd_x_oOC16ocLzaALVj6n-nbAw=.dba10c51-d483-413c-ba70-dab67a93fcb2@github.com>
Message-ID: <PCQtsERKq1dvdoXPL724WZJw02PhozuJiNwMKGRpTCI=.0f7b815a-b4c9-4421-95e4-1e4a6d60c8b9@github.com>

On Thu, 6 Feb 2025 10:49:34 GMT, Gary Frost <gfrost at openjdk.org> wrote:

> Added a jextract check in env.bash 
> 
> hatrun's usage now shows way to specify backend (i.e. ffi-opencl vs just opencl) in preparation for jextracted backends

This pull request has now been integrated.

Changeset: 1a6ea358
Author:    Gary Frost <gfrost at openjdk.org>
URL:       https://git.openjdk.org/babylon/commit/1a6ea35817fb72cd113c216c10efe2b2e151b7a4
Stats:     24 lines in 3 files changed: 11 ins; 1 del; 12 mod

Cleanup of HAT env.bash, bld and hatrun scripts

-------------

PR: https://git.openjdk.org/babylon/pull/309

From duke at openjdk.org  Thu Feb  6 20:30:40 2025
From: duke at openjdk.org (duke)
Date: Thu, 6 Feb 2025 20:30:40 GMT
Subject: [code-reflection] Withdrawn: Integrate HIP backend to bldr
 infrastructure.
In-Reply-To: <FeTaN009eviFNSrX4weilMJnL0bcrwJzDVhSXn5F3eA=.972f1d8f-338f-479b-be76-7cc5d88422f1@github.com>
References: <FeTaN009eviFNSrX4weilMJnL0bcrwJzDVhSXn5F3eA=.972f1d8f-338f-479b-be76-7cc5d88422f1@github.com>
Message-ID: <mrR82QsI_XuZluXypdGda3Lj3DpzfLeu8MRMCGSsSnU=.d3bf30f7-cf2e-4ae2-9f23-6d819feac4b1@github.com>

On Thu, 12 Dec 2024 18:18:39 GMT, iotamudelta <duke at openjdk.org> wrote:

> jextract fails in the presence of HIP on OpenCL/OpenGL - disable those for now if HIP is present.
> 
> While there, catch up to the reflection include change.

This pull request has been closed without being integrated.

-------------

PR: https://git.openjdk.org/babylon/pull/296

From psandoz at openjdk.org  Thu Feb  6 22:53:38 2025
From: psandoz at openjdk.org (Paul Sandoz)
Date: Thu, 6 Feb 2025 22:53:38 GMT
Subject: git: openjdk/babylon: code-reflection: Ensure array attributes are
 serialized correctly
Message-ID: <81ccdcfb-3f30-4d1d-bead-d8ee443c02ed@openjdk.org>

Changeset: 8026db04
Branch: code-reflection
Author:    Paul Sandoz <psandoz at openjdk.org>
Date:      2025-02-06 22:51:41 +0000
URL:       https://git.openjdk.org/babylon/commit/8026db04318deda1272431bd47ff333c4af16e5c

Ensure array attributes are serialized correctly

! src/jdk.incubator.code/share/classes/jdk/incubator/code/writer/OpWriter.java
+ test/jdk/java/lang/reflect/code/writer/TestAttributeSerialization.java


From psandoz at openjdk.org  Thu Feb  6 22:55:07 2025
From: psandoz at openjdk.org (Paul Sandoz)
Date: Thu, 6 Feb 2025 22:55:07 GMT
Subject: [code-reflection] Integrated: Ensure array attributes are serialized
 correctly
Message-ID: <t15zxEJ_tQUnqWfurJ6Iwf85VQU-7Sy0-AKsltCyZQI=.3c1481ae-bbfd-441a-8561-d53c1e10bb0f@github.com>

Operation attributes that are instances of arrays are not serialized correctly, Array::toString prints out the object identity and not the contents.

-------------

Commit messages:
 - Ensure array attributes are serialized correctly.

Changes: https://git.openjdk.org/babylon/pull/310/files
  Webrev: https://webrevs.openjdk.org/?repo=babylon&pr=310&range=00
  Stats: 133 lines in 2 files changed: 125 ins; 3 del; 5 mod
  Patch: https://git.openjdk.org/babylon/pull/310.diff
  Fetch: git fetch https://git.openjdk.org/babylon.git pull/310/head:pull/310

PR: https://git.openjdk.org/babylon/pull/310

From psandoz at openjdk.org  Thu Feb  6 22:55:08 2025
From: psandoz at openjdk.org (Paul Sandoz)
Date: Thu, 6 Feb 2025 22:55:08 GMT
Subject: [code-reflection] Integrated: Ensure array attributes are serialized
 correctly
In-Reply-To: <t15zxEJ_tQUnqWfurJ6Iwf85VQU-7Sy0-AKsltCyZQI=.3c1481ae-bbfd-441a-8561-d53c1e10bb0f@github.com>
References: <t15zxEJ_tQUnqWfurJ6Iwf85VQU-7Sy0-AKsltCyZQI=.3c1481ae-bbfd-441a-8561-d53c1e10bb0f@github.com>
Message-ID: <nz8QMv2QvR3r-Effi23ByA2voLKphQazLtrO5qbx6Pc=.a7716c40-8f0d-4bd0-9e87-4cf8f0fbf626@github.com>

On Thu, 6 Feb 2025 22:46:32 GMT, Paul Sandoz <psandoz at openjdk.org> wrote:

> Operation attributes that are instances of arrays are not serialized correctly, Array::toString prints out the object identity and not the contents.

This pull request has now been integrated.

Changeset: 8026db04
Author:    Paul Sandoz <psandoz at openjdk.org>
URL:       https://git.openjdk.org/babylon/commit/8026db04318deda1272431bd47ff333c4af16e5c
Stats:     133 lines in 2 files changed: 125 ins; 3 del; 5 mod

Ensure array attributes are serialized correctly

-------------

PR: https://git.openjdk.org/babylon/pull/310

From asotona at openjdk.org  Sun Feb  9 09:28:18 2025
From: asotona at openjdk.org (Adam Sotona)
Date: Sun, 9 Feb 2025 09:28:18 GMT
Subject: [code-reflection] RFR: ONNX FFM Runtime initial work
Message-ID: <f3CBMcuRX8reQzHRmd_9eGaN8jEh4iJeoIayQB9ffe0=.ea5873ca-fbd3-41cf-8baa-91fdf3443947@github.com>

This is initial work on ONNX FFM runtime with very raw connection with OnnxInterpreter and Tensor.

It is a rebase of https://github.com/PaulSandoz/babylon/pull/1

-------------

Commit messages:
 - minor OnnxProtoBuilder change
 - minor OnnxProtoBuilder change
 - unused imports
 - Implementation of dedicated OnnxProtoBuilder
 - OnnxRuntime work in progress
 - OnnxRuntime work in progress
 - separated RuntimeTest
 - ONNX FFM Runtime initial work

Changes: https://git.openjdk.org/babylon/pull/311/files
  Webrev: https://webrevs.openjdk.org/?repo=babylon&pr=311&range=00
  Stats: 936 lines in 7 files changed: 931 ins; 4 del; 1 mod
  Patch: https://git.openjdk.org/babylon/pull/311.diff
  Fetch: git fetch https://git.openjdk.org/babylon.git pull/311/head:pull/311

PR: https://git.openjdk.org/babylon/pull/311

From asotona at openjdk.org  Sun Feb  9 14:26:14 2025
From: asotona at openjdk.org (Adam Sotona)
Date: Sun, 9 Feb 2025 14:26:14 GMT
Subject: [code-reflection] RFR: ONNX FFM Runtime initial work [v2]
In-Reply-To: <f3CBMcuRX8reQzHRmd_9eGaN8jEh4iJeoIayQB9ffe0=.ea5873ca-fbd3-41cf-8baa-91fdf3443947@github.com>
References: <f3CBMcuRX8reQzHRmd_9eGaN8jEh4iJeoIayQB9ffe0=.ea5873ca-fbd3-41cf-8baa-91fdf3443947@github.com>
Message-ID: <34AvWquJq3yYta5_uhFpdbXBX5tfjuGC2t-Ff2ebx24=.ebc12a88-e36e-47c7-bf01-af9fca634d57@github.com>

> This is initial work on ONNX FFM runtime with very raw connection with OnnxInterpreter and Tensor.
> 
> It is a rebase of https://github.com/PaulSandoz/babylon/pull/1

Adam Sotona has updated the pull request incrementally with one additional commit since the last revision:

  OnnxRuntime use OnnxSchema - work in progress

-------------

Changes:
  - all: https://git.openjdk.org/babylon/pull/311/files
  - new: https://git.openjdk.org/babylon/pull/311/files/a4fd150b..6e5e6188

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=babylon&pr=311&range=01
 - incr: https://webrevs.openjdk.org/?repo=babylon&pr=311&range=00-01

  Stats: 66 lines in 4 files changed: 22 ins; 6 del; 38 mod
  Patch: https://git.openjdk.org/babylon/pull/311.diff
  Fetch: git fetch https://git.openjdk.org/babylon.git pull/311/head:pull/311

PR: https://git.openjdk.org/babylon/pull/311

From asotona at openjdk.org  Sun Feb  9 14:31:14 2025
From: asotona at openjdk.org (Adam Sotona)
Date: Sun, 9 Feb 2025 14:31:14 GMT
Subject: [code-reflection] RFR: ONNX FFM Runtime initial work [v3]
In-Reply-To: <f3CBMcuRX8reQzHRmd_9eGaN8jEh4iJeoIayQB9ffe0=.ea5873ca-fbd3-41cf-8baa-91fdf3443947@github.com>
References: <f3CBMcuRX8reQzHRmd_9eGaN8jEh4iJeoIayQB9ffe0=.ea5873ca-fbd3-41cf-8baa-91fdf3443947@github.com>
Message-ID: <kpTYerPqazGPqWWvAhZF65dzeWFvanGbIXPEeU3CCiM=.9b9ac033-59fb-48c2-8e1f-0670990cfa6a@github.com>

> This is initial work on ONNX FFM runtime with very raw connection with OnnxInterpreter and Tensor.
> 
> It is a rebase of https://github.com/PaulSandoz/babylon/pull/1

Adam Sotona has updated the pull request incrementally with one additional commit since the last revision:

  minor rename

-------------

Changes:
  - all: https://git.openjdk.org/babylon/pull/311/files
  - new: https://git.openjdk.org/babylon/pull/311/files/6e5e6188..3184267f

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=babylon&pr=311&range=02
 - incr: https://webrevs.openjdk.org/?repo=babylon&pr=311&range=01-02

  Stats: 4 lines in 3 files changed: 0 ins; 0 del; 4 mod
  Patch: https://git.openjdk.org/babylon/pull/311.diff
  Fetch: git fetch https://git.openjdk.org/babylon.git pull/311/head:pull/311

PR: https://git.openjdk.org/babylon/pull/311

From juan.fumero at manchester.ac.uk  Mon Feb 10 08:23:55 2025
From: juan.fumero at manchester.ac.uk (Juan Fumero)
Date: Mon, 10 Feb 2025 08:23:55 +0000
Subject: [blog] Early Performance Findings: Babylon/HAT and TornadoVM
Message-ID: <CWLP265MB2308E6F012C3A5465DF66BE0C2F22@CWLP265MB2308.GBRP265.PROD.OUTLOOK.COM>

Hi all,
    I've written an article comparing some performance of Babylon/HAT against TornadoVM. Since this touches on GPU utilization and performance, I thought it might be of interest to the GPU folks here. The article also includes some constructive feedback on both technologies.
You can find the article here: https://jjfumero.github.io/posts/2025/02/07/babylon-and-tornadovm
Juan

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/babylon-dev/attachments/20250210/83f60cf5/attachment.htm>

From mcimadamore at openjdk.org  Mon Feb 10 11:14:51 2025
From: mcimadamore at openjdk.org (Maurizio Cimadamore)
Date: Mon, 10 Feb 2025 11:14:51 GMT
Subject: [code-reflection] RFR: ONNX FFM Runtime initial work [v3]
In-Reply-To: <kpTYerPqazGPqWWvAhZF65dzeWFvanGbIXPEeU3CCiM=.9b9ac033-59fb-48c2-8e1f-0670990cfa6a@github.com>
References: <f3CBMcuRX8reQzHRmd_9eGaN8jEh4iJeoIayQB9ffe0=.ea5873ca-fbd3-41cf-8baa-91fdf3443947@github.com>
 <kpTYerPqazGPqWWvAhZF65dzeWFvanGbIXPEeU3CCiM=.9b9ac033-59fb-48c2-8e1f-0670990cfa6a@github.com>
Message-ID: <xBkv07wM2UWETsNNuX87rwcoGRxMMSq43pKM8wBif5A=.be5e6c2e-a4c9-4831-bcc5-3a608f076bc3@github.com>

On Sun, 9 Feb 2025 14:31:14 GMT, Adam Sotona <asotona at openjdk.org> wrote:

>> This is initial work on ONNX FFM runtime with very raw connection with OnnxInterpreter and Tensor.
>> 
>> It is a rebase of https://github.com/PaulSandoz/babylon/pull/1
>
> Adam Sotona has updated the pull request incrementally with one additional commit since the last revision:
> 
>   minor rename

I've tried to run jextract on the onnx header here:
https://github.com/microsoft/onnxruntime/blob/main/include/onnxruntime/core/session/onnxruntime_c_api.h

It seems to work correctly and to generate the bindings for the onnx runtime. Of course, this is not a classic API -- in the sense that the functions you want to call are defined inside a struct (OrtAPI). But, you can use a couple of jextract-generated function to get a pointer to that API struct:

1. `OrtGetApiBase`  (in `onnxruntime_c_api_h`)
2. `GetApi` (in OrtApiBase)

Note: the latter returns a function pointer, so in order to get the API you need to "invoke" the pointer. To do that, there's a GetAPI functional interface generated by jextract for this function pointer -- it has a static invoke method:


        public static MemorySegment invoke(MemorySegment funcPtr,int _x0) {


So, if you pass this static method the segment you got in (2), you should get the desired API back.

>From that point on, you use the same trick: that is, the OrtAPI struct is basically a struct containing a bunch of function pointers. So, to call a function in that API you need to:

1. get the memory segment for the field corresponding to the function you want to call
2. call the static `invoke` method on the generated functional interface modelling that function pointer

For instance, for `AllocatorGetInfo`, the OrtApi class has this getter:


/**
     * Getter for field:
     * {@snippet lang=c :
     * OrtStatusPtr (*AllocatorGetInfo)(const OrtAllocator *, const struct OrtMemoryInfo **)
     * }
     */
    public static MemorySegment AllocatorGetInfo(MemorySegment struct) { .. }


You call that (`struct` here is the reference to the OrtApi struct). And then you pass the returned segment (a function pointer) to this:


/**
     * {@snippet lang=c :
     * OrtStatusPtr (*AllocatorGetInfo)(const OrtAllocator *, const struct OrtMemoryInfo **)
     * }
     */
    public static class AllocatorGetInfo {
        /**
         * Invoke the upcall stub {@code funcPtr}, with given parameters
         */
        public static MemorySegment invoke(MemorySegment funcPtr,MemorySegment _x0, MemorySegment _x1) { ... }
    }


The nice thing about this is that you don't need to hardwire "magic numbers" as you have done in your implementation. You can leave the mechanical part to jextract, and focus on what API you really want to expose (your `Environment` class).

-------------

PR Comment: https://git.openjdk.org/babylon/pull/311#issuecomment-2647671328

From mcimadamore at openjdk.org  Mon Feb 10 12:15:38 2025
From: mcimadamore at openjdk.org (Maurizio Cimadamore)
Date: Mon, 10 Feb 2025 12:15:38 GMT
Subject: [code-reflection] RFR: Support storing the code that builds the
 code model [v2]
In-Reply-To: <W5OPqovjc4c2-wXuuqT9GveOC9Kv9ooDa8-_A9t9-VM=.683bf92e-19f0-4d26-805f-78b64f72c084@github.com>
References: <dexv8sQV_NcJsJIgMopd_M4wSWoZya9dU0ACwDtEL0E=.4ac0b80f-24fb-440f-a18d-bbfd07a681bc@github.com>
 <W5OPqovjc4c2-wXuuqT9GveOC9Kv9ooDa8-_A9t9-VM=.683bf92e-19f0-4d26-805f-78b64f72c084@github.com>
Message-ID: <0ug_-F7Oz_TImedL2gFJ5Fl9h82MZvcXO5J1kFuySww=.bafa9bed-e0ae-4263-97af-4ac54b3a20a0@github.com>

On Wed, 22 Jan 2025 17:25:58 GMT, Mourad Abbay <mabbay at openjdk.org> wrote:

>> In this PR we allow the user to decide how to store the code model. The first option is `TEXT`, if this is selected, we store the textual representation of the code model. The second option is `CODE_BUILDR`, if this is selected, we store the code that builds the code model. All work done here is around the second option, because the first option is already supported.
>
> Mourad Abbay has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Make TEXT the default storage mechanism for code models

Marked as reviewed by mcimadamore (Reviewer).

src/jdk.incubator.code/share/classes/jdk/incubator/code/internal/ReflectMethods.java line 159:

> 157:                         options.isSet(G_CUSTOM, "lines");
> 158:         String cmso = options.get("codeModelStorageOption");
> 159:         codeModelStorageOption = CodeModelStorageOption.valueOf(cmso != null ? cmso : CodeModelStorageOption.TEXT.name());

If you want you can define the parsing code here as a static method in the CodeModelStorageOption enum, for better encapsulation.

src/jdk.incubator.code/share/classes/jdk/incubator/code/internal/ReflectMethods.java line 400:

> 398: 
> 399:     private JCMethodDecl opMethodDecl(Name methodName, CoreOp.FuncOp op, CodeModelStorageOption codeModelStorageOption) {
> 400:         if (CodeModelStorageOption.TEXT.equals(codeModelStorageOption)) {

Maybe we can a switch on enum? That should allow you to get rid of the `else` part (as switching on enum constants is "exhaustive")

-------------

PR Review: https://git.openjdk.org/babylon/pull/305#pullrequestreview-2605575969
PR Review Comment: https://git.openjdk.org/babylon/pull/305#discussion_r1948946427
PR Review Comment: https://git.openjdk.org/babylon/pull/305#discussion_r1948948497

From asotona at openjdk.org  Mon Feb 10 12:16:23 2025
From: asotona at openjdk.org (Adam Sotona)
Date: Mon, 10 Feb 2025 12:16:23 GMT
Subject: [code-reflection] RFR: ONNX FFM Runtime initial work [v3]
In-Reply-To: <kpTYerPqazGPqWWvAhZF65dzeWFvanGbIXPEeU3CCiM=.9b9ac033-59fb-48c2-8e1f-0670990cfa6a@github.com>
References: <f3CBMcuRX8reQzHRmd_9eGaN8jEh4iJeoIayQB9ffe0=.ea5873ca-fbd3-41cf-8baa-91fdf3443947@github.com>
 <kpTYerPqazGPqWWvAhZF65dzeWFvanGbIXPEeU3CCiM=.9b9ac033-59fb-48c2-8e1f-0670990cfa6a@github.com>
Message-ID: <14L6YG3LcTf2VAVyoSW8fJQWms2e2ufQMkdT1qnIdaA=.ec3009bc-bf0c-418b-a0fa-84e73746fe8b@github.com>

On Sun, 9 Feb 2025 14:31:14 GMT, Adam Sotona <asotona at openjdk.org> wrote:

>> This is initial work on ONNX FFM runtime with very raw connection with OnnxInterpreter and Tensor.
>> 
>> It is a rebase of https://github.com/PaulSandoz/babylon/pull/1
>
> Adam Sotona has updated the pull request incrementally with one additional commit since the last revision:
> 
>   minor rename

I also tried jextract and it generated ~45k of source code, which suppose to be wrapped in another API layer.
My first attempt was to make the test working with jextracted code and simplify, however I got lost pretty soon.

-------------

PR Comment: https://git.openjdk.org/babylon/pull/311#issuecomment-2647813317

From mcimadamore at openjdk.org  Mon Feb 10 12:31:24 2025
From: mcimadamore at openjdk.org (Maurizio Cimadamore)
Date: Mon, 10 Feb 2025 12:31:24 GMT
Subject: [code-reflection] RFR: ONNX FFM Runtime initial work [v3]
In-Reply-To: <14L6YG3LcTf2VAVyoSW8fJQWms2e2ufQMkdT1qnIdaA=.ec3009bc-bf0c-418b-a0fa-84e73746fe8b@github.com>
References: <f3CBMcuRX8reQzHRmd_9eGaN8jEh4iJeoIayQB9ffe0=.ea5873ca-fbd3-41cf-8baa-91fdf3443947@github.com>
 <kpTYerPqazGPqWWvAhZF65dzeWFvanGbIXPEeU3CCiM=.9b9ac033-59fb-48c2-8e1f-0670990cfa6a@github.com>
 <14L6YG3LcTf2VAVyoSW8fJQWms2e2ufQMkdT1qnIdaA=.ec3009bc-bf0c-418b-a0fa-84e73746fe8b@github.com>
Message-ID: <omTwxHV40wDo8gaAnnMAfuEZzW6EVVC6iNZZfjD9Aig=.1ecf7c9a-e9ac-4ecf-b182-16d9f77e7614@github.com>

On Mon, 10 Feb 2025 12:14:12 GMT, Adam Sotona <asotona at openjdk.org> wrote:

> I also tried jextract and it generated ~45k of source code, which suppose to be wrapped in another API layer. My first attempt was to make the test working with jextracted code and simplify, however I got lost pretty soon.

Jextract allows to filter -- however the main OrtApi class alone is ~30K LoC. That said, you generate it once, and you check it in. Your approach surely leads to more succinct code - but stuff like this:


createEnv = handle( 3, JAVA_INT, ADDRESS, ADDRESS);


Seems questionable, as you have now encoded the offset of the function pointer in your high-level API. If the header changes, you'll need to update these numbers (whereas if `Environment` depends on jextract code, you just re-run jextract, and everything else will probably stay the same).

I'm not sure the size of the generated code should be a concern here? (Also note that jextract will only force loading of classes that are referenced by your main application).

-------------

PR Comment: https://git.openjdk.org/babylon/pull/311#issuecomment-2647846329

From mcimadamore at openjdk.org  Mon Feb 10 12:34:29 2025
From: mcimadamore at openjdk.org (Maurizio Cimadamore)
Date: Mon, 10 Feb 2025 12:34:29 GMT
Subject: [code-reflection] RFR: ONNX FFM Runtime initial work [v3]
In-Reply-To: <omTwxHV40wDo8gaAnnMAfuEZzW6EVVC6iNZZfjD9Aig=.1ecf7c9a-e9ac-4ecf-b182-16d9f77e7614@github.com>
References: <f3CBMcuRX8reQzHRmd_9eGaN8jEh4iJeoIayQB9ffe0=.ea5873ca-fbd3-41cf-8baa-91fdf3443947@github.com>
 <kpTYerPqazGPqWWvAhZF65dzeWFvanGbIXPEeU3CCiM=.9b9ac033-59fb-48c2-8e1f-0670990cfa6a@github.com>
 <14L6YG3LcTf2VAVyoSW8fJQWms2e2ufQMkdT1qnIdaA=.ec3009bc-bf0c-418b-a0fa-84e73746fe8b@github.com>
 <omTwxHV40wDo8gaAnnMAfuEZzW6EVVC6iNZZfjD9Aig=.1ecf7c9a-e9ac-4ecf-b182-16d9f77e7614@github.com>
Message-ID: <bOqHNyO-XhQb4GSo_tLkWqLRP2s3-4-G2zMLN0FqbUM=.68867000-fc76-4b2a-8de7-573970a07e22@github.com>

On Mon, 10 Feb 2025 12:28:42 GMT, Maurizio Cimadamore <mcimadamore at openjdk.org> wrote:

> Jextract allows to filter -- however the main OrtApi class alone is ~30K LoC.

The reason jextract filtering is not very effective here is that these are not simple functions - but function pointers fields in a struct -- and jextract doesn't have option to "omit" struct fields.

-------------

PR Comment: https://git.openjdk.org/babylon/pull/311#issuecomment-2647853763

From pminborg at openjdk.org  Mon Feb 10 12:58:35 2025
From: pminborg at openjdk.org (Per Minborg)
Date: Mon, 10 Feb 2025 12:58:35 GMT
Subject: [code-reflection] RFR: ONNX FFM Runtime initial work [v3]
In-Reply-To: <kpTYerPqazGPqWWvAhZF65dzeWFvanGbIXPEeU3CCiM=.9b9ac033-59fb-48c2-8e1f-0670990cfa6a@github.com>
References: <f3CBMcuRX8reQzHRmd_9eGaN8jEh4iJeoIayQB9ffe0=.ea5873ca-fbd3-41cf-8baa-91fdf3443947@github.com>
 <kpTYerPqazGPqWWvAhZF65dzeWFvanGbIXPEeU3CCiM=.9b9ac033-59fb-48c2-8e1f-0670990cfa6a@github.com>
Message-ID: <amiwNfz2y34g8tLWi5amY4sf9L7skPnk36eDto3XVOo=.374a48dc-60a7-43f5-a271-f134673b32f1@github.com>

On Sun, 9 Feb 2025 14:31:14 GMT, Adam Sotona <asotona at openjdk.org> wrote:

>> This is initial work on ONNX FFM runtime with very raw connection with OnnxInterpreter and Tensor.
>> 
>> It is a rebase of https://github.com/PaulSandoz/babylon/pull/1
>
> Adam Sotona has updated the pull request incrementally with one additional commit since the last revision:
> 
>   minor rename

cr-examples/onnx/src/main/java/oracle/code/onnx/OnnxRuntime.java line 68:

> 66:     private final SymbolLookup  library;
> 67:     private final MemorySegment runtimeAddress, ret;
> 68:     private final MethodHandle  allocatorGetInfo,

Can these be `static` for better performance?

-------------

PR Review Comment: https://git.openjdk.org/babylon/pull/311#discussion_r1949008459

From mcimadamore at openjdk.org  Mon Feb 10 13:12:20 2025
From: mcimadamore at openjdk.org (Maurizio Cimadamore)
Date: Mon, 10 Feb 2025 13:12:20 GMT
Subject: [code-reflection] RFR: ONNX FFM Runtime initial work [v3]
In-Reply-To: <kpTYerPqazGPqWWvAhZF65dzeWFvanGbIXPEeU3CCiM=.9b9ac033-59fb-48c2-8e1f-0670990cfa6a@github.com>
References: <f3CBMcuRX8reQzHRmd_9eGaN8jEh4iJeoIayQB9ffe0=.ea5873ca-fbd3-41cf-8baa-91fdf3443947@github.com>
 <kpTYerPqazGPqWWvAhZF65dzeWFvanGbIXPEeU3CCiM=.9b9ac033-59fb-48c2-8e1f-0670990cfa6a@github.com>
Message-ID: <Gg0WUmtOKkGViUndxk5QzomA6EuRTaIEvX92os2pPz4=.547365ba-8d6b-4b0c-83bd-b07382442ae0@github.com>

On Sun, 9 Feb 2025 14:31:14 GMT, Adam Sotona <asotona at openjdk.org> wrote:

>> This is initial work on ONNX FFM runtime with very raw connection with OnnxInterpreter and Tensor.
>> 
>> It is a rebase of https://github.com/PaulSandoz/babylon/pull/1
>
> Adam Sotona has updated the pull request incrementally with one additional commit since the last revision:
> 
>   minor rename

cr-examples/onnx/src/main/java/oracle/code/onnx/OnnxRuntime.java line 153:

> 151:     }
> 152: 
> 153:     public Environment createEnv(LoggingLevel logLevel, String logId) {

There's some lifetime here to deal with... createEnv has a releaseEnv method, which is not dealt with here. Perhaps it would be convenient to associate the returned env segment with an automatic arena (Arena::ofAuto) that will be kept alive as long as the Environment instance is reachable which is, I think, what you want.

-------------

PR Review Comment: https://git.openjdk.org/babylon/pull/311#discussion_r1949026219

From mcimadamore at openjdk.org  Mon Feb 10 13:21:24 2025
From: mcimadamore at openjdk.org (Maurizio Cimadamore)
Date: Mon, 10 Feb 2025 13:21:24 GMT
Subject: [code-reflection] RFR: ONNX FFM Runtime initial work [v3]
In-Reply-To: <kpTYerPqazGPqWWvAhZF65dzeWFvanGbIXPEeU3CCiM=.9b9ac033-59fb-48c2-8e1f-0670990cfa6a@github.com>
References: <f3CBMcuRX8reQzHRmd_9eGaN8jEh4iJeoIayQB9ffe0=.ea5873ca-fbd3-41cf-8baa-91fdf3443947@github.com>
 <kpTYerPqazGPqWWvAhZF65dzeWFvanGbIXPEeU3CCiM=.9b9ac033-59fb-48c2-8e1f-0670990cfa6a@github.com>
Message-ID: <CIjN6E_NfdW82CBRujkutypvRgCvsaG28X497rXnHNU=.24543ca3-1def-4636-9e35-708ec8a3b3e8@github.com>

On Sun, 9 Feb 2025 14:31:14 GMT, Adam Sotona <asotona at openjdk.org> wrote:

>> This is initial work on ONNX FFM runtime with very raw connection with OnnxInterpreter and Tensor.
>> 
>> It is a rebase of https://github.com/PaulSandoz/babylon/pull/1
>
> Adam Sotona has updated the pull request incrementally with one additional commit since the last revision:
> 
>   minor rename

cr-examples/onnx/src/main/java/oracle/code/onnx/OnnxRuntime.java line 214:

> 212:         }
> 213: 
> 214:         public final class Session implements AutoCloseable {

There's another lifetime here - that of the session. I notice few things here:
* you expect access to be single-threaded (otherwise, reusing the `ret` output segment will backfire)
* you expect the session to be no longer used after the close method on session is closed

Both aspects could be modeled by having `Session` carry a new confined arena, and reinterpret the session segment to that arena (so that access is thread-confined, and only possible while the session is alive). When the session is closed, you  should call the confined arena close. You can also register (while reinterpreting) a manual cleanup action, to be invoked on the session segment when the confined arena is closed (e.g. to call `releaseSession`).

This will guarantee that no method on session can be called _after_ the session has been closed (note any such call will likely result in a JVM crash because of use-after-free, so that seems valuable?)

cr-examples/onnx/src/main/java/oracle/code/onnx/OnnxRuntime.java line 224:

> 222:             public int getNumberOfInputs() {
> 223:                 try {
> 224:                     return retInt(sessionGetInputCount.invokeExact(sessionAddress, ret));

I note the use of this shared `ret` output segment. We plan to add more capabilities to FFM to address the case of recyclable allocation -- this will help code like this to avoid the shared segment, at the same time while avoiding the cost of a malloc per call.

cr-examples/onnx/src/main/java/oracle/code/onnx/OnnxRuntime.java line 524:

> 522: 
> 523:     private void checkStatus(Object res) {
> 524:         if (!res.equals(MemorySegment.NULL) && res instanceof MemorySegment status) {

This is another example of "hardwired logic". There's a struct definition for this:


struct OrtStatus {
  OrtErrorCode code;
  char msg[1];  // a null-terminated string
};


And the code is optimistically assuming that this layout will hold (across different versions of the API and across different platforms). Ok for a prototype of course, but that's another thing jextract might help you with.

-------------

PR Review Comment: https://git.openjdk.org/babylon/pull/311#discussion_r1949032243
PR Review Comment: https://git.openjdk.org/babylon/pull/311#discussion_r1949038116
PR Review Comment: https://git.openjdk.org/babylon/pull/311#discussion_r1949035987

From gfrost at openjdk.org  Mon Feb 10 13:50:25 2025
From: gfrost at openjdk.org (Gary Frost)
Date: Mon, 10 Feb 2025 13:50:25 GMT
Subject: git: openjdk/babylon: code-reflection: Hat cleanup jextraction
Message-ID: <627803f3-9b76-40e5-b52b-4141430d7783@openjdk.org>

Changeset: c2fc40f7
Branch: code-reflection
Author:    Gary Frost <gfrost at openjdk.org>
Date:      2025-02-10 13:48:22 +0000
URL:       https://git.openjdk.org/babylon/commit/c2fc40f7fab420f1775e9d5b4772ae90e68f4f68

Hat cleanup jextraction

! hat/bld
! hat/bldr/Bldr.java
! hat/examples/experiments/src/main/java/experiments/spirv/Bad.java
! hat/examples/nbody/src/main/java/nbody/GLWrap.java
! hat/examples/nbody/src/main/java/nbody/Main.java
! hat/hatrun
! hat/intellij/backend_jextracted_opencl.iml
! hat/intellij/nbody.iml
! hat/sanity


From gfrost at openjdk.org  Mon Feb 10 13:51:09 2025
From: gfrost at openjdk.org (Gary Frost)
Date: Mon, 10 Feb 2025 13:51:09 GMT
Subject: [code-reflection] Integrated: Hat cleanup jextraction
Message-ID: <EDZcRYzwKdgT6cU5zFtTCsmL7QzEfwVjypUnDWV42SA=.8f87452e-6e2f-42ee-b626-ae6f7834a0e6@github.com>

The bldr script jextraction capability handling is still 'evolving' in preparation for jextract based opencl and cuda backends. 

Here we simplified the script by allowing jextract to accept the opencl/gl capabilities and set relevant options.

The nbody example used CLwrap and GLwrap (requiring jextractions).  I changed the header-name to opengl_h from the default glut_h.  

I cleaned up sanity script as well.

-------------

Commit messages:
 - Fixed up sanity script and removed whitespace
 - Fixed up sanity script
 - Cleaning up jextractions in build. GLWrap now uses opengl_h (instead of glut_h)

Changes: https://git.openjdk.org/babylon/pull/312/files
  Webrev: https://webrevs.openjdk.org/?repo=babylon&pr=312&range=00
  Stats: 1149 lines in 9 files changed: 470 ins; 122 del; 557 mod
  Patch: https://git.openjdk.org/babylon/pull/312.diff
  Fetch: git fetch https://git.openjdk.org/babylon.git pull/312/head:pull/312

PR: https://git.openjdk.org/babylon/pull/312

From gfrost at openjdk.org  Mon Feb 10 13:51:09 2025
From: gfrost at openjdk.org (Gary Frost)
Date: Mon, 10 Feb 2025 13:51:09 GMT
Subject: [code-reflection] Integrated: Hat cleanup jextraction
In-Reply-To: <EDZcRYzwKdgT6cU5zFtTCsmL7QzEfwVjypUnDWV42SA=.8f87452e-6e2f-42ee-b626-ae6f7834a0e6@github.com>
References: <EDZcRYzwKdgT6cU5zFtTCsmL7QzEfwVjypUnDWV42SA=.8f87452e-6e2f-42ee-b626-ae6f7834a0e6@github.com>
Message-ID: <YKXsbAViLZ6jLMzwzxusdq7NeqVa2r_XqXV7goPcAVM=.ca58fd51-820e-4293-9a3b-33e3b5de04bf@github.com>

On Mon, 10 Feb 2025 13:40:12 GMT, Gary Frost <gfrost at openjdk.org> wrote:

> The bldr script jextraction capability handling is still 'evolving' in preparation for jextract based opencl and cuda backends. 
> 
> Here we simplified the script by allowing jextract to accept the opencl/gl capabilities and set relevant options.
> 
> The nbody example used CLwrap and GLwrap (requiring jextractions).  I changed the header-name to opengl_h from the default glut_h.  
> 
> I cleaned up sanity script as well.

This pull request has now been integrated.

Changeset: c2fc40f7
Author:    Gary Frost <gfrost at openjdk.org>
URL:       https://git.openjdk.org/babylon/commit/c2fc40f7fab420f1775e9d5b4772ae90e68f4f68
Stats:     1149 lines in 9 files changed: 470 ins; 122 del; 557 mod

Hat cleanup jextraction

-------------

PR: https://git.openjdk.org/babylon/pull/312

From asotona at openjdk.org  Mon Feb 10 13:53:26 2025
From: asotona at openjdk.org (Adam Sotona)
Date: Mon, 10 Feb 2025 13:53:26 GMT
Subject: [code-reflection] RFR: ONNX FFM Runtime initial work [v3]
In-Reply-To: <Gg0WUmtOKkGViUndxk5QzomA6EuRTaIEvX92os2pPz4=.547365ba-8d6b-4b0c-83bd-b07382442ae0@github.com>
References: <f3CBMcuRX8reQzHRmd_9eGaN8jEh4iJeoIayQB9ffe0=.ea5873ca-fbd3-41cf-8baa-91fdf3443947@github.com>
 <kpTYerPqazGPqWWvAhZF65dzeWFvanGbIXPEeU3CCiM=.9b9ac033-59fb-48c2-8e1f-0670990cfa6a@github.com>
 <Gg0WUmtOKkGViUndxk5QzomA6EuRTaIEvX92os2pPz4=.547365ba-8d6b-4b0c-83bd-b07382442ae0@github.com>
Message-ID: <_NHE4gI6NHyarfthWRT17sUJjX9_DtGchU5Vsp6HNkM=.bbcc7514-f68d-43af-b5a0-5e5997dfd99b@github.com>

On Mon, 10 Feb 2025 13:09:17 GMT, Maurizio Cimadamore <mcimadamore at openjdk.org> wrote:

>> Adam Sotona has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   minor rename
>
> cr-examples/onnx/src/main/java/oracle/code/onnx/OnnxRuntime.java line 153:
> 
>> 151:     }
>> 152: 
>> 153:     public Environment createEnv(LoggingLevel logLevel, String logId) {
> 
> There's some lifetime here to deal with... createEnv has a releaseEnv method, which is not dealt with here. Perhaps it would be convenient to associate the returned env segment with an automatic arena (Arena::ofAuto) that will be kept alive as long as the Environment instance is reachable which is, I think, what you want.

Yes, there are definitely many places for improvements. This is just a first shot to make it working ASAP.
I expect many refactoring rounds.

-------------

PR Review Comment: https://git.openjdk.org/babylon/pull/311#discussion_r1949092878

From asotona at openjdk.org  Mon Feb 10 14:01:25 2025
From: asotona at openjdk.org (Adam Sotona)
Date: Mon, 10 Feb 2025 14:01:25 GMT
Subject: [code-reflection] RFR: ONNX FFM Runtime initial work [v3]
In-Reply-To: <CIjN6E_NfdW82CBRujkutypvRgCvsaG28X497rXnHNU=.24543ca3-1def-4636-9e35-708ec8a3b3e8@github.com>
References: <f3CBMcuRX8reQzHRmd_9eGaN8jEh4iJeoIayQB9ffe0=.ea5873ca-fbd3-41cf-8baa-91fdf3443947@github.com>
 <kpTYerPqazGPqWWvAhZF65dzeWFvanGbIXPEeU3CCiM=.9b9ac033-59fb-48c2-8e1f-0670990cfa6a@github.com>
 <CIjN6E_NfdW82CBRujkutypvRgCvsaG28X497rXnHNU=.24543ca3-1def-4636-9e35-708ec8a3b3e8@github.com>
Message-ID: <hwlPutr71LDSkZMQZrzhWcoTE_t3CzF9jtVREWYs4CA=.a0786719-2a3e-4120-ba00-ba04d88e28cc@github.com>

On Mon, 10 Feb 2025 13:13:33 GMT, Maurizio Cimadamore <mcimadamore at openjdk.org> wrote:

>> Adam Sotona has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   minor rename
>
> cr-examples/onnx/src/main/java/oracle/code/onnx/OnnxRuntime.java line 214:
> 
>> 212:         }
>> 213: 
>> 214:         public final class Session implements AutoCloseable {
> 
> There's another lifetime here - that of the session. I notice few things here:
> * you expect access to be single-threaded (otherwise, reusing the `ret` output segment will backfire)
> * you expect the session to be no longer used after the close method on session is closed
> 
> Both aspects could be modeled by having `Session` carry a new confined arena, and reinterpret the session segment to that arena (so that access is thread-confined, and only possible while the session is alive). When the session is closed, you  should call the confined arena close. You can also register (while reinterpreting) a manual cleanup action, to be invoked on the session segment when the confined arena is closed (e.g. to call `releaseSession`).
> 
> This will guarantee that no method on session can be called _after_ the session has been closed (note any such call will likely result in a JVM crash because of use-after-free, so that seems valuable?)

Yes, both assumptions are right.
Single `ret` design has been selected for simplicity and there are no constraints checked, so double-close will throw or maybe even kill JVM.
I have some ideas how to cache sessions for models and auto-close them with the environment, plus some other performance improvements.
However with Paul we agreed it is not a #1 priority for now. The design should stay simple for easy refactoring as the API will settle down with the use cases.

-------------

PR Review Comment: https://git.openjdk.org/babylon/pull/311#discussion_r1949104293

From asotona at openjdk.org  Mon Feb 10 14:05:28 2025
From: asotona at openjdk.org (Adam Sotona)
Date: Mon, 10 Feb 2025 14:05:28 GMT
Subject: [code-reflection] RFR: ONNX FFM Runtime initial work [v3]
In-Reply-To: <CIjN6E_NfdW82CBRujkutypvRgCvsaG28X497rXnHNU=.24543ca3-1def-4636-9e35-708ec8a3b3e8@github.com>
References: <f3CBMcuRX8reQzHRmd_9eGaN8jEh4iJeoIayQB9ffe0=.ea5873ca-fbd3-41cf-8baa-91fdf3443947@github.com>
 <kpTYerPqazGPqWWvAhZF65dzeWFvanGbIXPEeU3CCiM=.9b9ac033-59fb-48c2-8e1f-0670990cfa6a@github.com>
 <CIjN6E_NfdW82CBRujkutypvRgCvsaG28X497rXnHNU=.24543ca3-1def-4636-9e35-708ec8a3b3e8@github.com>
Message-ID: <aYM_MGD_pVFo8bVnF_mwYM0MxizWMZu6hnnHtdtQods=.f6faebe5-7343-495e-b0d6-5d074d5d8b54@github.com>

On Mon, 10 Feb 2025 13:16:10 GMT, Maurizio Cimadamore <mcimadamore at openjdk.org> wrote:

>> Adam Sotona has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   minor rename
>
> cr-examples/onnx/src/main/java/oracle/code/onnx/OnnxRuntime.java line 524:
> 
>> 522: 
>> 523:     private void checkStatus(Object res) {
>> 524:         if (!res.equals(MemorySegment.NULL) && res instanceof MemorySegment status) {
> 
> This is another example of "hardwired logic". There's a struct definition for this:
> 
> 
> struct OrtStatus {
>   OrtErrorCode code;
>   char msg[1];  // a null-terminated string
> };
> 
> 
> And the code is optimistically assuming that this layout will hold (across different versions of the API and across different platforms). Ok for a prototype of course, but that's another thing jextract might help you with.

I agree the code is hard-wired to the ONNX library, however any significant changes in the API (more than reordering of the functions in the struct) will break the compatibility anyway. 
Jextracted code is also hard-wired to the specific library version.

-------------

PR Review Comment: https://git.openjdk.org/babylon/pull/311#discussion_r1949115159

From asotona at openjdk.org  Mon Feb 10 14:10:25 2025
From: asotona at openjdk.org (Adam Sotona)
Date: Mon, 10 Feb 2025 14:10:25 GMT
Subject: [code-reflection] RFR: ONNX FFM Runtime initial work [v3]
In-Reply-To: <CIjN6E_NfdW82CBRujkutypvRgCvsaG28X497rXnHNU=.24543ca3-1def-4636-9e35-708ec8a3b3e8@github.com>
References: <f3CBMcuRX8reQzHRmd_9eGaN8jEh4iJeoIayQB9ffe0=.ea5873ca-fbd3-41cf-8baa-91fdf3443947@github.com>
 <kpTYerPqazGPqWWvAhZF65dzeWFvanGbIXPEeU3CCiM=.9b9ac033-59fb-48c2-8e1f-0670990cfa6a@github.com>
 <CIjN6E_NfdW82CBRujkutypvRgCvsaG28X497rXnHNU=.24543ca3-1def-4636-9e35-708ec8a3b3e8@github.com>
Message-ID: <wvwwsE32s4wvAqVOmHSep-OBeohFtNeOMTaeBBzaNFs=.2f45c038-ea4c-45a4-9974-e63d251705a4@github.com>

On Mon, 10 Feb 2025 13:17:36 GMT, Maurizio Cimadamore <mcimadamore at openjdk.org> wrote:

>> Adam Sotona has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   minor rename
>
> cr-examples/onnx/src/main/java/oracle/code/onnx/OnnxRuntime.java line 224:
> 
>> 222:             public int getNumberOfInputs() {
>> 223:                 try {
>> 224:                     return retInt(sessionGetInputCount.invokeExact(sessionAddress, ret));
> 
> I note the use of this shared `ret` output segment. We plan to add more capabilities to FFM to address the case of recyclable allocation -- this will help code like this to avoid the shared segment, at the same time while avoiding the cost of a malloc per call.

That would be very helpful.
Truly I haven't found any FFM hints which type of arena is best for specific case.
My guess is that garbage-collected allocations may be better than creating and disposing individual arena for each allocation (I've seen a lot of objects constructed).
Simple manual disposal of a memory segment might be a step forward?

-------------

PR Review Comment: https://git.openjdk.org/babylon/pull/311#discussion_r1949123217

From asotona at openjdk.org  Mon Feb 10 14:10:25 2025
From: asotona at openjdk.org (Adam Sotona)
Date: Mon, 10 Feb 2025 14:10:25 GMT
Subject: [code-reflection] RFR: ONNX FFM Runtime initial work [v3]
In-Reply-To: <wvwwsE32s4wvAqVOmHSep-OBeohFtNeOMTaeBBzaNFs=.2f45c038-ea4c-45a4-9974-e63d251705a4@github.com>
References: <f3CBMcuRX8reQzHRmd_9eGaN8jEh4iJeoIayQB9ffe0=.ea5873ca-fbd3-41cf-8baa-91fdf3443947@github.com>
 <kpTYerPqazGPqWWvAhZF65dzeWFvanGbIXPEeU3CCiM=.9b9ac033-59fb-48c2-8e1f-0670990cfa6a@github.com>
 <CIjN6E_NfdW82CBRujkutypvRgCvsaG28X497rXnHNU=.24543ca3-1def-4636-9e35-708ec8a3b3e8@github.com>
 <wvwwsE32s4wvAqVOmHSep-OBeohFtNeOMTaeBBzaNFs=.2f45c038-ea4c-45a4-9974-e63d251705a4@github.com>
Message-ID: <cZzPW5LsNR_LokPowQB3umju1m9loN9k5OzHsrs6Q1w=.892d9b61-5cc9-45ff-9a28-1409b4f755ea@github.com>

On Mon, 10 Feb 2025 14:06:45 GMT, Adam Sotona <asotona at openjdk.org> wrote:

>> cr-examples/onnx/src/main/java/oracle/code/onnx/OnnxRuntime.java line 224:
>> 
>>> 222:             public int getNumberOfInputs() {
>>> 223:                 try {
>>> 224:                     return retInt(sessionGetInputCount.invokeExact(sessionAddress, ret));
>> 
>> I note the use of this shared `ret` output segment. We plan to add more capabilities to FFM to address the case of recyclable allocation -- this will help code like this to avoid the shared segment, at the same time while avoiding the cost of a malloc per call.
>
> That would be very helpful.
> Truly I haven't found any FFM hints which type of arena is best for specific case.
> My guess is that garbage-collected allocations may be better than creating and disposing individual arena for each allocation (I've seen a lot of objects constructed).
> Simple manual disposal of a memory segment might be a step forward?

Finally I've decided to re-use the `ret` as the thread-safety is not yet a problem.

-------------

PR Review Comment: https://git.openjdk.org/babylon/pull/311#discussion_r1949125220

From asotona at openjdk.org  Mon Feb 10 14:14:22 2025
From: asotona at openjdk.org (Adam Sotona)
Date: Mon, 10 Feb 2025 14:14:22 GMT
Subject: [code-reflection] RFR: ONNX FFM Runtime initial work [v3]
In-Reply-To: <kpTYerPqazGPqWWvAhZF65dzeWFvanGbIXPEeU3CCiM=.9b9ac033-59fb-48c2-8e1f-0670990cfa6a@github.com>
References: <f3CBMcuRX8reQzHRmd_9eGaN8jEh4iJeoIayQB9ffe0=.ea5873ca-fbd3-41cf-8baa-91fdf3443947@github.com>
 <kpTYerPqazGPqWWvAhZF65dzeWFvanGbIXPEeU3CCiM=.9b9ac033-59fb-48c2-8e1f-0670990cfa6a@github.com>
Message-ID: <m464H7CJN8N9k70tcfuda9_mkrGJzzMh4cOgBC23XhY=.4638f538-e123-46bc-80b5-3f4ee4559efe@github.com>

On Sun, 9 Feb 2025 14:31:14 GMT, Adam Sotona <asotona at openjdk.org> wrote:

>> This is initial work on ONNX FFM runtime with very raw connection with OnnxInterpreter and Tensor.
>> 
>> It is a rebase of https://github.com/PaulSandoz/babylon/pull/1
>
> Adam Sotona has updated the pull request incrementally with one additional commit since the last revision:
> 
>   minor rename

I've been hesitating to use jextracted code vs. struct magic offsets. As the generated code is really huge and I'm unable to orientate in it, vs. there is just a single struct I'm interested in (and I can grep the offset : function name list).

-------------

PR Comment: https://git.openjdk.org/babylon/pull/311#issuecomment-2648110648

From asotona at openjdk.org  Mon Feb 10 14:17:43 2025
From: asotona at openjdk.org (Adam Sotona)
Date: Mon, 10 Feb 2025 14:17:43 GMT
Subject: [code-reflection] RFR: ONNX FFM Runtime initial work [v3]
In-Reply-To: <kpTYerPqazGPqWWvAhZF65dzeWFvanGbIXPEeU3CCiM=.9b9ac033-59fb-48c2-8e1f-0670990cfa6a@github.com>
References: <f3CBMcuRX8reQzHRmd_9eGaN8jEh4iJeoIayQB9ffe0=.ea5873ca-fbd3-41cf-8baa-91fdf3443947@github.com>
 <kpTYerPqazGPqWWvAhZF65dzeWFvanGbIXPEeU3CCiM=.9b9ac033-59fb-48c2-8e1f-0670990cfa6a@github.com>
Message-ID: <AnyfNZO2EpgZ_3dzuuqDpAiQpJ6QFCrFih3_B1MIgyg=.2fd2721e-8430-4e4c-867c-ced5c48fa06d@github.com>

On Sun, 9 Feb 2025 14:31:14 GMT, Adam Sotona <asotona at openjdk.org> wrote:

>> This is initial work on ONNX FFM runtime with very raw connection with OnnxInterpreter and Tensor.
>> 
>> It is a rebase of https://github.com/PaulSandoz/babylon/pull/1
>
> Adam Sotona has updated the pull request incrementally with one additional commit since the last revision:
> 
>   minor rename

The same applies to `OnnxProtoBuilder`, the generated source is huuuge, hard to read and depending on an external library. 
While a simple grep + regexp replace can produce the code I used.

-------------

PR Comment: https://git.openjdk.org/babylon/pull/311#issuecomment-2648118246

From asotona at openjdk.org  Mon Feb 10 14:21:21 2025
From: asotona at openjdk.org (Adam Sotona)
Date: Mon, 10 Feb 2025 14:21:21 GMT
Subject: [code-reflection] RFR: ONNX FFM Runtime initial work [v3]
In-Reply-To: <amiwNfz2y34g8tLWi5amY4sf9L7skPnk36eDto3XVOo=.374a48dc-60a7-43f5-a271-f134673b32f1@github.com>
References: <f3CBMcuRX8reQzHRmd_9eGaN8jEh4iJeoIayQB9ffe0=.ea5873ca-fbd3-41cf-8baa-91fdf3443947@github.com>
 <kpTYerPqazGPqWWvAhZF65dzeWFvanGbIXPEeU3CCiM=.9b9ac033-59fb-48c2-8e1f-0670990cfa6a@github.com>
 <amiwNfz2y34g8tLWi5amY4sf9L7skPnk36eDto3XVOo=.374a48dc-60a7-43f5-a271-f134673b32f1@github.com>
Message-ID: <0C6eit0XhgEoQo1eiqskpIGCcGqlpsZhaYz1-egSLSI=.d94d5823-7ed1-4af4-8845-61358215087f@github.com>

On Mon, 10 Feb 2025 12:56:07 GMT, Per Minborg <pminborg at openjdk.org> wrote:

>> Adam Sotona has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   minor rename
>
> cr-examples/onnx/src/main/java/oracle/code/onnx/OnnxRuntime.java line 68:
> 
>> 66:     private final SymbolLookup  library;
>> 67:     private final MemorySegment runtimeAddress, ret;
>> 68:     private final MethodHandle  allocatorGetInfo,
> 
> Can these be `static` for better performance?

No with the current API design. The memory segment of the instance of obtained by `OrtGetApiBase()->GetApi((uint32_t)apiVersion)`

-------------

PR Review Comment: https://git.openjdk.org/babylon/pull/311#discussion_r1949143800

From asotona at openjdk.org  Mon Feb 10 14:34:30 2025
From: asotona at openjdk.org (Adam Sotona)
Date: Mon, 10 Feb 2025 14:34:30 GMT
Subject: [code-reflection] RFR: ONNX FFM Runtime initial work [v3]
In-Reply-To: <kpTYerPqazGPqWWvAhZF65dzeWFvanGbIXPEeU3CCiM=.9b9ac033-59fb-48c2-8e1f-0670990cfa6a@github.com>
References: <f3CBMcuRX8reQzHRmd_9eGaN8jEh4iJeoIayQB9ffe0=.ea5873ca-fbd3-41cf-8baa-91fdf3443947@github.com>
 <kpTYerPqazGPqWWvAhZF65dzeWFvanGbIXPEeU3CCiM=.9b9ac033-59fb-48c2-8e1f-0670990cfa6a@github.com>
Message-ID: <AMkKlrFDNm85AfBUSZWbh4KHFjZAsuYYMfxHDq2Kygo=.79ad971d-b945-4b35-8015-da770a81e53b@github.com>

On Sun, 9 Feb 2025 14:31:14 GMT, Adam Sotona <asotona at openjdk.org> wrote:

>> This is initial work on ONNX FFM runtime with very raw connection with OnnxInterpreter and Tensor.
>> 
>> It is a rebase of https://github.com/PaulSandoz/babylon/pull/1
>
> Adam Sotona has updated the pull request incrementally with one additional commit since the last revision:
> 
>   minor rename

The doors to make the API more layered and rebase it on the generated code (for both `jextract` and  `protoc`) are still open.

However I don't prefer to start a prototype of relatively simple functionality by dependency on ~90k lines of generated and unreadable sources + external library and tools. I think we need something lightweight to start playing with.

-------------

PR Comment: https://git.openjdk.org/babylon/pull/311#issuecomment-2648174547

From mcimadamore at openjdk.org  Mon Feb 10 14:40:25 2025
From: mcimadamore at openjdk.org (Maurizio Cimadamore)
Date: Mon, 10 Feb 2025 14:40:25 GMT
Subject: [code-reflection] RFR: ONNX FFM Runtime initial work [v3]
In-Reply-To: <kpTYerPqazGPqWWvAhZF65dzeWFvanGbIXPEeU3CCiM=.9b9ac033-59fb-48c2-8e1f-0670990cfa6a@github.com>
References: <f3CBMcuRX8reQzHRmd_9eGaN8jEh4iJeoIayQB9ffe0=.ea5873ca-fbd3-41cf-8baa-91fdf3443947@github.com>
 <kpTYerPqazGPqWWvAhZF65dzeWFvanGbIXPEeU3CCiM=.9b9ac033-59fb-48c2-8e1f-0670990cfa6a@github.com>
Message-ID: <IhLkxCjAl6H2p1vtu7atf8UaiZ57A8txDqCUYn80G8Q=.97d53239-4aa4-4d25-89c3-150bcf761828@github.com>

On Sun, 9 Feb 2025 14:31:14 GMT, Adam Sotona <asotona at openjdk.org> wrote:

>> This is initial work on ONNX FFM runtime with very raw connection with OnnxInterpreter and Tensor.
>> 
>> It is a rebase of https://github.com/PaulSandoz/babylon/pull/1
>
> Adam Sotona has updated the pull request incrementally with one additional commit since the last revision:
> 
>   minor rename

Did a quick experiment with jextract. I can write code like this:


        var apiBase = OrtGetApiBase();
        System.out.println(apiBase);
        var version = OrtApiBase.GetVersionString.invoke(OrtApiBase.GetVersionString(apiBase));
        System.out.println(version.getString(0));
        var api = OrtApiBase.GetApi.invoke(OrtApiBase.GetApi(apiBase), 1);
        System.out.println(api); // prints 1.20.1
        MemorySegment buf = Arena.ofAuto().allocate(ValueLayout.ADDRESS);
        var res = OrtApi.CreateSessionOptions.invoke(OrtApi.CreateSessionOptions(api), buf);
        var sessionOptions = buf.get(ValueLayout.ADDRESS, 0);
        System.out.println(sessionOptions);


While not super-compact (due to the way the API is defined, via function pointers stored in structs), this is fairly mechanical, and should provide a good foundation for your `Environment` API.

I note something that jextract could deal with slightly better - when invoking a function pointer on a struct, the `invoke` method takes the function pointer address (as that form is more primitive) - but in this case it would be handy if the method took a pointer to the base struct, so that this code:


        var version = OrtApiBase.GetVersionString.invoke(OrtApiBase.GetVersionString(apiBase));


Could become just:


        var version = OrtApiBase.GetVersionString.invoke(apiBase);


(after all jextract knows where that function pointer is, given a pointer to the base struct).

-------------

PR Comment: https://git.openjdk.org/babylon/pull/311#issuecomment-2648192400

From mcimadamore at openjdk.org  Mon Feb 10 14:56:34 2025
From: mcimadamore at openjdk.org (Maurizio Cimadamore)
Date: Mon, 10 Feb 2025 14:56:34 GMT
Subject: [code-reflection] RFR: ONNX FFM Runtime initial work [v3]
In-Reply-To: <AMkKlrFDNm85AfBUSZWbh4KHFjZAsuYYMfxHDq2Kygo=.79ad971d-b945-4b35-8015-da770a81e53b@github.com>
References: <f3CBMcuRX8reQzHRmd_9eGaN8jEh4iJeoIayQB9ffe0=.ea5873ca-fbd3-41cf-8baa-91fdf3443947@github.com>
 <kpTYerPqazGPqWWvAhZF65dzeWFvanGbIXPEeU3CCiM=.9b9ac033-59fb-48c2-8e1f-0670990cfa6a@github.com>
 <AMkKlrFDNm85AfBUSZWbh4KHFjZAsuYYMfxHDq2Kygo=.79ad971d-b945-4b35-8015-da770a81e53b@github.com>
Message-ID: <0uFFsHKFq3pJGyom1QqxvK7sJZqp1RK0rnMr0S3MO5Y=.86267761-24b5-4162-8d02-e1ecaf0b2493@github.com>

On Mon, 10 Feb 2025 14:32:07 GMT, Adam Sotona <asotona at openjdk.org> wrote:

> However I don't prefer to start a prototype of relatively simple functionality by dependency on ~90k lines of generated and unreadable sources + external library and tools. I think we need something lightweight to start playing with.

You are of course free to do as you please. I suppose the definition of "lightweight" is what I'm objecting to. Sure, your code is more compact, but in terms of getting started quickly, I think the jextract version is superior. I literally got the example I linked [here](https://github.com/openjdk/babylon/pull/311#issuecomment-2648192400) up and running in 5 minutes. In my view the generated code is... generated code. We will add more filtering option to address this specific use case (jextract design center is not -- yet -- around APIs that are tables of function pointers). But if you are in prototype mode, whether it's 100 LoC, 1K or 100K, does it matter? I'd still rely on a battle-tested tool to generate all the required offsets/API points from me.

-------------

PR Comment: https://git.openjdk.org/babylon/pull/311#issuecomment-2648248483

From asotona at openjdk.org  Mon Feb 10 15:17:25 2025
From: asotona at openjdk.org (Adam Sotona)
Date: Mon, 10 Feb 2025 15:17:25 GMT
Subject: [code-reflection] RFR: ONNX FFM Runtime initial work [v3]
In-Reply-To: <kpTYerPqazGPqWWvAhZF65dzeWFvanGbIXPEeU3CCiM=.9b9ac033-59fb-48c2-8e1f-0670990cfa6a@github.com>
References: <f3CBMcuRX8reQzHRmd_9eGaN8jEh4iJeoIayQB9ffe0=.ea5873ca-fbd3-41cf-8baa-91fdf3443947@github.com>
 <kpTYerPqazGPqWWvAhZF65dzeWFvanGbIXPEeU3CCiM=.9b9ac033-59fb-48c2-8e1f-0670990cfa6a@github.com>
Message-ID: <Y1dNyLD7Gn-PGEGJIV8f8wvh1Fy7kQwHJSfX8UV9Ks0=.de13bfc3-5bf5-4eab-be7f-b1ddc58b3c37@github.com>

On Sun, 9 Feb 2025 14:31:14 GMT, Adam Sotona <asotona at openjdk.org> wrote:

>> This is initial work on ONNX FFM runtime with very raw connection with OnnxInterpreter and Tensor.
>> 
>> It is a rebase of https://github.com/PaulSandoz/babylon/pull/1
>
> Adam Sotona has updated the pull request incrementally with one additional commit since the last revision:
> 
>   minor rename

In order to use the code generated by `jextract` you need to know exactly how `jextract` works. There are thousands of methods like:

        /**
         * Invoke the upcall stub {@code funcPtr}, with given parameters
         */
        public static MemorySegment invoke(MemorySegment funcPtr,MemorySegment _x0, MemorySegment _x1, int _x2, MemorySegment _x3, MemorySegment _x4) {

and I have no idea how they are related to the original struct and what are the parameters. 
I gave it a try with the original header file open on one monitor and the generated code on the other and I was not able to find any entry point and had no idea how to compose the invoke calls.

I'm sorry but this generated code may be helpful only when you exactly know how `jextract` works.

-------------

PR Comment: https://git.openjdk.org/babylon/pull/311#issuecomment-2648331610

From mcimadamore at openjdk.org  Mon Feb 10 15:22:27 2025
From: mcimadamore at openjdk.org (Maurizio Cimadamore)
Date: Mon, 10 Feb 2025 15:22:27 GMT
Subject: [code-reflection] RFR: ONNX FFM Runtime initial work [v3]
In-Reply-To: <Y1dNyLD7Gn-PGEGJIV8f8wvh1Fy7kQwHJSfX8UV9Ks0=.de13bfc3-5bf5-4eab-be7f-b1ddc58b3c37@github.com>
References: <f3CBMcuRX8reQzHRmd_9eGaN8jEh4iJeoIayQB9ffe0=.ea5873ca-fbd3-41cf-8baa-91fdf3443947@github.com>
 <kpTYerPqazGPqWWvAhZF65dzeWFvanGbIXPEeU3CCiM=.9b9ac033-59fb-48c2-8e1f-0670990cfa6a@github.com>
 <Y1dNyLD7Gn-PGEGJIV8f8wvh1Fy7kQwHJSfX8UV9Ks0=.de13bfc3-5bf5-4eab-be7f-b1ddc58b3c37@github.com>
Message-ID: <AfNALc7JS_g0FMdiB-u1lUGI7YsFCupjXTqwvwG422Q=.38cbb757-fc6e-4324-815e-abbea62303c6@github.com>

On Mon, 10 Feb 2025 15:14:41 GMT, Adam Sotona <asotona at openjdk.org> wrote:

> I'm sorry but this generated code may be helpful only when you exactly know how `jextract` works.

Sure - you might find this helpful
https://github.com/openjdk/jextract/blob/master/doc/GUIDE.md

-------------

PR Comment: https://git.openjdk.org/babylon/pull/311#issuecomment-2648355290

From gfrost at openjdk.org  Mon Feb 10 15:42:26 2025
From: gfrost at openjdk.org (Gary Frost)
Date: Mon, 10 Feb 2025 15:42:26 GMT
Subject: git: openjdk/babylon: code-reflection: Added inversion of control
 pattern so capabilities can coordinate with jextract
Message-ID: <48c7e63a-3714-4325-b000-d1d3c60f21d4@openjdk.org>

Changeset: 432d13eb
Branch: code-reflection
Author:    Gary Frost <gfrost at openjdk.org>
Date:      2025-02-10 15:40:31 +0000
URL:       https://git.openjdk.org/babylon/commit/432d13ebb69d7cb4e170caa70b49ac33fddc85a9

Added inversion of control pattern so capabilities can coordinate with jextract

! hat/bld
! hat/bldr/Bldr.java


From gfrost at openjdk.org  Mon Feb 10 15:43:37 2025
From: gfrost at openjdk.org (Gary Frost)
Date: Mon, 10 Feb 2025 15:43:37 GMT
Subject: [code-reflection] Integrated: Added inversion of control pattern so
 capabilities can coordinate with jextract
Message-ID: <KwpRIhnUoArkU9qX7ac82-y3I0luJG6l4AwjkGfuyBw=.4f4bafdb-fd0d-4ad6-83be-82170d45d748@github.com>

simplified bld script, by allowing  'cababilities' implementing 'jextractable' to coordinate directly with jextract.

-------------

Commit messages:
 - Added inversion of control pattern so capabilities can coordinate with jextract

Changes: https://git.openjdk.org/babylon/pull/313/files
  Webrev: https://webrevs.openjdk.org/?repo=babylon&pr=313&range=00
  Stats: 67 lines in 2 files changed: 36 ins; 25 del; 6 mod
  Patch: https://git.openjdk.org/babylon/pull/313.diff
  Fetch: git fetch https://git.openjdk.org/babylon.git pull/313/head:pull/313

PR: https://git.openjdk.org/babylon/pull/313

From gfrost at openjdk.org  Mon Feb 10 15:43:37 2025
From: gfrost at openjdk.org (Gary Frost)
Date: Mon, 10 Feb 2025 15:43:37 GMT
Subject: [code-reflection] Integrated: Added inversion of control pattern so
 capabilities can coordinate with jextract
In-Reply-To: <KwpRIhnUoArkU9qX7ac82-y3I0luJG6l4AwjkGfuyBw=.4f4bafdb-fd0d-4ad6-83be-82170d45d748@github.com>
References: <KwpRIhnUoArkU9qX7ac82-y3I0luJG6l4AwjkGfuyBw=.4f4bafdb-fd0d-4ad6-83be-82170d45d748@github.com>
Message-ID: <IiIQcafanTcKjOmHJ7ZFE3a-MGqu29_MW5MsGpt4k9g=.786f462d-080b-4b68-b4fd-a46e6c6935c7@github.com>

On Mon, 10 Feb 2025 15:38:23 GMT, Gary Frost <gfrost at openjdk.org> wrote:

> simplified bld script, by allowing  'cababilities' implementing 'jextractable' to coordinate directly with jextract.

This pull request has now been integrated.

Changeset: 432d13eb
Author:    Gary Frost <gfrost at openjdk.org>
URL:       https://git.openjdk.org/babylon/commit/432d13ebb69d7cb4e170caa70b49ac33fddc85a9
Stats:     67 lines in 2 files changed: 36 ins; 25 del; 6 mod

Added inversion of control pattern so capabilities can coordinate with jextract

-------------

PR: https://git.openjdk.org/babylon/pull/313

From duke at openjdk.org  Mon Feb 10 19:07:50 2025
From: duke at openjdk.org (Adam Pocock)
Date: Mon, 10 Feb 2025 19:07:50 GMT
Subject: [code-reflection] RFR: Updating the ONNX CNNTest
Message-ID: <Ks6GEU2O7N4uBbkVSNd_5pRdiPvn16xJcddIpLNCom8=.dc8880ad-603c-43cf-89eb-649b34c2c3f0@github.com>

- Adds a Python script which trains a small LeNet style CNN on MNIST, exporting it to ONNX format.
- Updates the CNNTest to match the trained CNN.
- Added support for loading in the ONNX model, extracting the weights and saving them out in serialized Java records (this adds protobuf-java as a dependency along with the compiled ONNX protobuf).

The compiled ONNX protobuf is Apache 2.0 licensed, and the Python script is adapted from a PyTorch tutorial which is BSD licensed.

-------------

Commit messages:
 - Removing more whitespace generated by protoc.
 - Removing whitespace from generated protobuf.
 - Remove extraneous whitespace
 - CNNTest shape & flatten fix.
 - Adding a Python script to generate a CNN, updating the CNN code model.

Changes: https://git.openjdk.org/babylon/pull/307/files
  Webrev: https://webrevs.openjdk.org/?repo=babylon&pr=307&range=00
  Stats: 47795 lines in 6 files changed: 47687 ins; 33 del; 75 mod
  Patch: https://git.openjdk.org/babylon/pull/307.diff
  Fetch: git fetch https://git.openjdk.org/babylon.git pull/307/head:pull/307

PR: https://git.openjdk.org/babylon/pull/307

From psandoz at openjdk.org  Mon Feb 10 19:26:30 2025
From: psandoz at openjdk.org (Paul Sandoz)
Date: Mon, 10 Feb 2025 19:26:30 GMT
Subject: [code-reflection] RFR: Updating the ONNX CNNTest
In-Reply-To: <Ks6GEU2O7N4uBbkVSNd_5pRdiPvn16xJcddIpLNCom8=.dc8880ad-603c-43cf-89eb-649b34c2c3f0@github.com>
References: <Ks6GEU2O7N4uBbkVSNd_5pRdiPvn16xJcddIpLNCom8=.dc8880ad-603c-43cf-89eb-649b34c2c3f0@github.com>
Message-ID: <HemovMxaEXVYX-HH7RxT98VdOt1_qcc6x-Df0vF0Tl8=.c3fe44cc-a589-42c1-b5b7-6598694f4baf@github.com>

On Wed, 5 Feb 2025 16:09:36 GMT, Adam Pocock <duke at openjdk.org> wrote:

> - Adds a Python script which trains a small LeNet style CNN on MNIST, exporting it to ONNX format.
> - Updates the CNNTest to match the trained CNN.
> - Added support for loading in the ONNX model, extracting the weights and saving them out in serialized Java records (this adds protobuf-java as a dependency along with the compiled ONNX protobuf).
> 
> The compiled ONNX protobuf is Apache 2.0 licensed, and the Python script is adapted from a PyTorch tutorial which is BSD licensed.

Marked as reviewed by psandoz (Lead).

-------------

PR Review: https://git.openjdk.org/babylon/pull/307#pullrequestreview-2606936502

From duke at openjdk.org  Mon Feb 10 19:40:33 2025
From: duke at openjdk.org (duke)
Date: Mon, 10 Feb 2025 19:40:33 GMT
Subject: [code-reflection] RFR: Updating the ONNX CNNTest
In-Reply-To: <Ks6GEU2O7N4uBbkVSNd_5pRdiPvn16xJcddIpLNCom8=.dc8880ad-603c-43cf-89eb-649b34c2c3f0@github.com>
References: <Ks6GEU2O7N4uBbkVSNd_5pRdiPvn16xJcddIpLNCom8=.dc8880ad-603c-43cf-89eb-649b34c2c3f0@github.com>
Message-ID: <EspSVMsdRY-q8Q2Dl4W0HuFSq-4spAe3_hx5uHZydmQ=.93ef3f2c-9677-4d17-8335-9fd6449f30ea@github.com>

On Wed, 5 Feb 2025 16:09:36 GMT, Adam Pocock <duke at openjdk.org> wrote:

> - Adds a Python script which trains a small LeNet style CNN on MNIST, exporting it to ONNX format.
> - Updates the CNNTest to match the trained CNN.
> - Added support for loading in the ONNX model, extracting the weights and saving them out in serialized Java records (this adds protobuf-java as a dependency along with the compiled ONNX protobuf).
> 
> The compiled ONNX protobuf is Apache 2.0 licensed, and the Python script is adapted from a PyTorch tutorial which is BSD licensed.

@Craigacp 
Your change (at version 89e445a76f1de5ac3bf89a4a858dc794196f2ee5) is now ready to be sponsored by a Committer.

-------------

PR Comment: https://git.openjdk.org/babylon/pull/307#issuecomment-2649056577

From psandoz at openjdk.org  Mon Feb 10 19:42:18 2025
From: psandoz at openjdk.org (Paul Sandoz)
Date: Mon, 10 Feb 2025 19:42:18 GMT
Subject: git: openjdk/babylon: code-reflection: Updating the ONNX CNNTest
Message-ID: <f833b472-9f18-4308-82dc-9af30def9df1@openjdk.org>

Changeset: adbb7a81
Branch: code-reflection
Author:    Adam Pocock <adam.pocock at oracle.com>
Committer: Paul Sandoz <psandoz at openjdk.org>
Date:      2025-02-10 19:40:46 +0000
URL:       https://git.openjdk.org/babylon/commit/adbb7a819c52121da58754ba222015422c7a38b5

Updating the ONNX CNNTest

Reviewed-by: psandoz

! cr-examples/onnx/pom.xml
+ cr-examples/onnx/src/test/java/onnx/NOTICE.md
+ cr-examples/onnx/src/test/java/onnx/OnnxMl.java
! cr-examples/onnx/src/test/java/oracle/code/onnx/CNNTest.java
+ cr-examples/onnx/src/test/resources/oracle/code/onnx/lenet-torchscript.onnx
+ cr-examples/onnx/src/test/resources/oracle/code/onnx/lenet.py


From duke at openjdk.org  Mon Feb 10 19:43:31 2025
From: duke at openjdk.org (Adam Pocock)
Date: Mon, 10 Feb 2025 19:43:31 GMT
Subject: [code-reflection] Integrated: Updating the ONNX CNNTest
In-Reply-To: <Ks6GEU2O7N4uBbkVSNd_5pRdiPvn16xJcddIpLNCom8=.dc8880ad-603c-43cf-89eb-649b34c2c3f0@github.com>
References: <Ks6GEU2O7N4uBbkVSNd_5pRdiPvn16xJcddIpLNCom8=.dc8880ad-603c-43cf-89eb-649b34c2c3f0@github.com>
Message-ID: <TvnCBob7RLUorHZUEXYFbxqT1682ymgiBLONbHkyM0Y=.9621c8ae-6444-4ba3-a134-bc109e7b4262@github.com>

On Wed, 5 Feb 2025 16:09:36 GMT, Adam Pocock <duke at openjdk.org> wrote:

> - Adds a Python script which trains a small LeNet style CNN on MNIST, exporting it to ONNX format.
> - Updates the CNNTest to match the trained CNN.
> - Added support for loading in the ONNX model, extracting the weights and saving them out in serialized Java records (this adds protobuf-java as a dependency along with the compiled ONNX protobuf).
> 
> The compiled ONNX protobuf is Apache 2.0 licensed, and the Python script is adapted from a PyTorch tutorial which is BSD licensed.

This pull request has now been integrated.

Changeset: adbb7a81
Author:    Adam Pocock <adam.pocock at oracle.com>
Committer: Paul Sandoz <psandoz at openjdk.org>
URL:       https://git.openjdk.org/babylon/commit/adbb7a819c52121da58754ba222015422c7a38b5
Stats:     47795 lines in 6 files changed: 47687 ins; 33 del; 75 mod

Updating the ONNX CNNTest

Reviewed-by: psandoz

-------------

PR: https://git.openjdk.org/babylon/pull/307

From mabbay at openjdk.org  Mon Feb 10 19:58:11 2025
From: mabbay at openjdk.org (Mourad Abbay)
Date: Mon, 10 Feb 2025 19:58:11 GMT
Subject: [code-reflection] RFR: Support storing the code that builds the
 code model [v3]
In-Reply-To: <dexv8sQV_NcJsJIgMopd_M4wSWoZya9dU0ACwDtEL0E=.4ac0b80f-24fb-440f-a18d-bbfd07a681bc@github.com>
References: <dexv8sQV_NcJsJIgMopd_M4wSWoZya9dU0ACwDtEL0E=.4ac0b80f-24fb-440f-a18d-bbfd07a681bc@github.com>
Message-ID: <gllpucB_MfObCpo_pzgLQpmEsQW_E6HduIpWz3kVDzQ=.beb5df92-520b-4f15-8ae2-f9935b42733c@github.com>

> In this PR we allow the user to decide how to store the code model. The first option is `TEXT`, if this is selected, we store the textual representation of the code model. The second option is `CODE_BUILDR`, if this is selected, we store the code that builds the code model. All work done here is around the second option, because the first option is already supported.

Mourad Abbay has updated the pull request incrementally with one additional commit since the last revision:

  Support storing code that builds code model

-------------

Changes:
  - all: https://git.openjdk.org/babylon/pull/305/files
  - new: https://git.openjdk.org/babylon/pull/305/files/4cdc021e..c3a71f94

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=babylon&pr=305&range=02
 - incr: https://webrevs.openjdk.org/?repo=babylon&pr=305&range=01-02

  Stats: 461 lines in 4 files changed: 444 ins; 0 del; 17 mod
  Patch: https://git.openjdk.org/babylon/pull/305.diff
  Fetch: git fetch https://git.openjdk.org/babylon.git pull/305/head:pull/305

PR: https://git.openjdk.org/babylon/pull/305

From mabbay at openjdk.org  Mon Feb 10 20:14:41 2025
From: mabbay at openjdk.org (Mourad Abbay)
Date: Mon, 10 Feb 2025 20:14:41 GMT
Subject: [code-reflection] RFR: Support storing the code that builds the
 code model [v4]
In-Reply-To: <dexv8sQV_NcJsJIgMopd_M4wSWoZya9dU0ACwDtEL0E=.4ac0b80f-24fb-440f-a18d-bbfd07a681bc@github.com>
References: <dexv8sQV_NcJsJIgMopd_M4wSWoZya9dU0ACwDtEL0E=.4ac0b80f-24fb-440f-a18d-bbfd07a681bc@github.com>
Message-ID: <Hr4Gxe9EusVvZ1qApsBkhSmkC3yDO0OWh1gCKUrWahE=.87e363c8-4ddc-4941-a5a1-2c047acfe993@github.com>

> In this PR we allow the user to decide how to store the code model. The first option is `TEXT`, if this is selected, we store the textual representation of the code model. The second option is `CODE_BUILDR`, if this is selected, we store the code that builds the code model. All work done here is around the second option, because the first option is already supported.

Mourad Abbay has updated the pull request incrementally with two additional commits since the last revision:

 - Add a program to test storing code that builds code model
 - Add JavaType -> Type mapping for Block.Parameter

-------------

Changes:
  - all: https://git.openjdk.org/babylon/pull/305/files
  - new: https://git.openjdk.org/babylon/pull/305/files/c3a71f94..d4d3a25c

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=babylon&pr=305&range=03
 - incr: https://webrevs.openjdk.org/?repo=babylon&pr=305&range=02-03

  Stats: 13 lines in 3 files changed: 12 ins; 0 del; 1 mod
  Patch: https://git.openjdk.org/babylon/pull/305.diff
  Fetch: git fetch https://git.openjdk.org/babylon.git pull/305/head:pull/305

PR: https://git.openjdk.org/babylon/pull/305

From mabbay at openjdk.org  Mon Feb 10 20:36:20 2025
From: mabbay at openjdk.org (Mourad Abbay)
Date: Mon, 10 Feb 2025 20:36:20 GMT
Subject: [code-reflection] RFR: Support storing the code that builds the
 code model [v4]
In-Reply-To: <Hr4Gxe9EusVvZ1qApsBkhSmkC3yDO0OWh1gCKUrWahE=.87e363c8-4ddc-4941-a5a1-2c047acfe993@github.com>
References: <dexv8sQV_NcJsJIgMopd_M4wSWoZya9dU0ACwDtEL0E=.4ac0b80f-24fb-440f-a18d-bbfd07a681bc@github.com>
 <Hr4Gxe9EusVvZ1qApsBkhSmkC3yDO0OWh1gCKUrWahE=.87e363c8-4ddc-4941-a5a1-2c047acfe993@github.com>
Message-ID: <bBiFmSRBPNpLFI9d2ul7vbRendpiBoVq-dAkyeTfuIc=.1abe186c-3d8d-4ac5-b478-f949cd508c86@github.com>

On Mon, 10 Feb 2025 20:14:41 GMT, Mourad Abbay <mabbay at openjdk.org> wrote:

>> In this PR we allow the user to decide how to store the code model. The first option is `TEXT`, if this is selected, we store the textual representation of the code model. The second option is `CODE_BUILDR`, if this is selected, we store the code that builds the code model. All work done here is around the second option, because the first option is already supported.
>
> Mourad Abbay has updated the pull request incrementally with two additional commits since the last revision:
> 
>  - Add a program to test storing code that builds code model
>  - Add JavaType -> Type mapping for Block.Parameter

To test this you can compile the program `TestStoringCodeModelBuilder.java` with the following configuration:

- Program: `com.sun.tools.javac.Main`

- Arguments: `-XDdumpIR -XDcodeModelStorageOption=CODE_BUILDER src/jdk.incubator.code/share/classes/jdk/incubator/code/internal/TestStoringCodeModelBuilder.java`

The compilation will fail with the error: `no enclosing instance of type TypeElementFactory is in scope`

-------------

PR Comment: https://git.openjdk.org/babylon/pull/305#issuecomment-2649173489

From mabbay at openjdk.org  Mon Feb 10 20:43:45 2025
From: mabbay at openjdk.org (Mourad Abbay)
Date: Mon, 10 Feb 2025 20:43:45 GMT
Subject: [code-reflection] RFR: Support storing the code that builds the
 code model [v5]
In-Reply-To: <dexv8sQV_NcJsJIgMopd_M4wSWoZya9dU0ACwDtEL0E=.4ac0b80f-24fb-440f-a18d-bbfd07a681bc@github.com>
References: <dexv8sQV_NcJsJIgMopd_M4wSWoZya9dU0ACwDtEL0E=.4ac0b80f-24fb-440f-a18d-bbfd07a681bc@github.com>
Message-ID: <FgDnvkwGHuUoUz_0n2uSmW_5kmjai1Z0WNcBiI-zH5M=.7e7a6d5c-1561-4b3a-8eba-9a296033d9bc@github.com>

> In this PR we allow the user to decide how to store the code model. The first option is `TEXT`, if this is selected, we store the textual representation of the code model. The second option is `CODE_BUILDR`, if this is selected, we store the code that builds the code model. All work done here is around the second option, because the first option is already supported.

Mourad Abbay has updated the pull request incrementally with one additional commit since the last revision:

  Define parsing code in the enum CodeModelStorageOption

-------------

Changes:
  - all: https://git.openjdk.org/babylon/pull/305/files
  - new: https://git.openjdk.org/babylon/pull/305/files/d4d3a25c..16b16087

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=babylon&pr=305&range=04
 - incr: https://webrevs.openjdk.org/?repo=babylon&pr=305&range=03-04

  Stats: 10 lines in 1 file changed: 6 ins; 1 del; 3 mod
  Patch: https://git.openjdk.org/babylon/pull/305.diff
  Fetch: git fetch https://git.openjdk.org/babylon.git pull/305/head:pull/305

PR: https://git.openjdk.org/babylon/pull/305

From duke at openjdk.org  Mon Feb 10 21:57:23 2025
From: duke at openjdk.org (Adam Pocock)
Date: Mon, 10 Feb 2025 21:57:23 GMT
Subject: [code-reflection] RFR: ONNX FFM Runtime initial work [v3]
In-Reply-To: <_NHE4gI6NHyarfthWRT17sUJjX9_DtGchU5Vsp6HNkM=.bbcc7514-f68d-43af-b5a0-5e5997dfd99b@github.com>
References: <f3CBMcuRX8reQzHRmd_9eGaN8jEh4iJeoIayQB9ffe0=.ea5873ca-fbd3-41cf-8baa-91fdf3443947@github.com>
 <kpTYerPqazGPqWWvAhZF65dzeWFvanGbIXPEeU3CCiM=.9b9ac033-59fb-48c2-8e1f-0670990cfa6a@github.com>
 <Gg0WUmtOKkGViUndxk5QzomA6EuRTaIEvX92os2pPz4=.547365ba-8d6b-4b0c-83bd-b07382442ae0@github.com>
 <_NHE4gI6NHyarfthWRT17sUJjX9_DtGchU5Vsp6HNkM=.bbcc7514-f68d-43af-b5a0-5e5997dfd99b@github.com>
Message-ID: <OU2R0c0HLCcCO0xxaUagjpSx72wxQIVeLUj5YyXYxEc=.cf75d5f5-8732-4970-8bad-ba6712c3b569@github.com>

On Mon, 10 Feb 2025 13:51:10 GMT, Adam Sotona <asotona at openjdk.org> wrote:

>> cr-examples/onnx/src/main/java/oracle/code/onnx/OnnxRuntime.java line 153:
>> 
>>> 151:     }
>>> 152: 
>>> 153:     public Environment createEnv(LoggingLevel logLevel, String logId) {
>> 
>> There's some lifetime here to deal with... createEnv has a releaseEnv method, which is not dealt with here. Perhaps it would be convenient to associate the returned env segment with an automatic arena (Arena::ofAuto) that will be kept alive as long as the Environment instance is reachable which is, I think, what you want.
>
> Yes, there are definitely many places for improvements. This is just a first shot to make it working ASAP.
> I expect many refactoring rounds.

`OrtEnv` must be instantiated at most once in the lifetime of a process, if you close it and recreate it then you get mysterious crashes. That's why the Java API has a singleton for it which registers a JVM shutdown hook to close it. That has other problems as the hook runs concurrently with daemon threads, but I couldn't figure out a way around it.

-------------

PR Review Comment: https://git.openjdk.org/babylon/pull/311#discussion_r1949951800

From asotona at openjdk.org  Tue Feb 11 08:16:26 2025
From: asotona at openjdk.org (Adam Sotona)
Date: Tue, 11 Feb 2025 08:16:26 GMT
Subject: [code-reflection] RFR: ONNX FFM Runtime initial work [v3]
In-Reply-To: <OU2R0c0HLCcCO0xxaUagjpSx72wxQIVeLUj5YyXYxEc=.cf75d5f5-8732-4970-8bad-ba6712c3b569@github.com>
References: <f3CBMcuRX8reQzHRmd_9eGaN8jEh4iJeoIayQB9ffe0=.ea5873ca-fbd3-41cf-8baa-91fdf3443947@github.com>
 <kpTYerPqazGPqWWvAhZF65dzeWFvanGbIXPEeU3CCiM=.9b9ac033-59fb-48c2-8e1f-0670990cfa6a@github.com>
 <Gg0WUmtOKkGViUndxk5QzomA6EuRTaIEvX92os2pPz4=.547365ba-8d6b-4b0c-83bd-b07382442ae0@github.com>
 <_NHE4gI6NHyarfthWRT17sUJjX9_DtGchU5Vsp6HNkM=.bbcc7514-f68d-43af-b5a0-5e5997dfd99b@github.com>
 <OU2R0c0HLCcCO0xxaUagjpSx72wxQIVeLUj5YyXYxEc=.cf75d5f5-8732-4970-8bad-ba6712c3b569@github.com>
Message-ID: <C_xed3OvBsaAzr8Ah_-E8QhRJ-g-8f-vRHE2AtPmyIY=.05d33ad9-ff2d-4882-ba78-b948e5b4ec7d@github.com>

On Mon, 10 Feb 2025 21:54:42 GMT, Adam Pocock <duke at openjdk.org> wrote:

>> Yes, there are definitely many places for improvements. This is just a first shot to make it working ASAP.
>> I expect many refactoring rounds.
>
> `OrtEnv` must be instantiated at most once in the lifetime of a process, if you close it and recreate it then you get mysterious crashes. That's why the Java API has a singleton for it which registers a JVM shutdown hook to close it. That has other problems as the hook runs concurrently with daemon threads, but I couldn't figure out a way around it.

Thank you!

-------------

PR Review Comment: https://git.openjdk.org/babylon/pull/311#discussion_r1950401796

From gfrost at openjdk.org  Tue Feb 11 10:16:52 2025
From: gfrost at openjdk.org (Gary Frost)
Date: Tue, 11 Feb 2025 10:16:52 GMT
Subject: git: openjdk/babylon: code-reflection: Hat jextracted backend infra
Message-ID: <d833c52d-3a25-4456-bf50-9b8559832b00@openjdk.org>

Changeset: 8cde02f9
Branch: code-reflection
Author:    Gary Frost <gfrost at openjdk.org>
Date:      2025-02-11 10:14:28 +0000
URL:       https://git.openjdk.org/babylon/commit/8cde02f9cea452e750fd783cdd71f9c85e60388b

Hat jextracted backend infra

! hat/backends/jextracted/opencl/src/main/java/hat/backend/jextracted/CLWrap.java
+ hat/backends/jextracted/shared/src/main/java/hat/backend/jextracted/SoWeCanBuild.java
! hat/bld
! hat/intellij/.idea/compiler.xml
! hat/intellij/.idea/modules.xml
! hat/intellij/backend_jextracted_opencl.iml
+ hat/intellij/clwrap.iml
+ hat/intellij/cuwrap.iml
+ hat/intellij/glwrap.iml
+ hat/intellij/wrap.iml
! hat/sanity
+ hat/wrap/clwrap/src/main/java/wrap/clwrap/CLPlatform.java
+ hat/wrap/clwrap/src/main/java/wrap/clwrap/CLStatusPtr.java
+ hat/wrap/clwrap/src/main/java/wrap/clwrap/ComputeContext.java
+ hat/wrap/clwrap/src/main/java/wrap/clwrap/Squares.java
+ hat/wrap/glwrap/src/main/java/wrap/glwrap/Fonts.java
+ hat/wrap/glwrap/src/main/java/wrap/glwrap/GLCallbackEventHandler.java
+ hat/wrap/glwrap/src/main/java/wrap/glwrap/GLEventHandler.java
+ hat/wrap/glwrap/src/main/java/wrap/glwrap/GLFuncEventHandler.java
+ hat/wrap/glwrap/src/main/java/wrap/glwrap/GLStatusPtr.java
+ hat/wrap/glwrap/src/main/java/wrap/glwrap/GLTexture.java
+ hat/wrap/glwrap/src/main/java/wrap/glwrap/GLWindow.java
+ hat/wrap/wrap/src/main/java/wrap/ArenaHolder.java
+ hat/wrap/wrap/src/main/java/wrap/LayoutBuilder.java
+ hat/wrap/wrap/src/main/java/wrap/Scalar.java
+ hat/wrap/wrap/src/main/java/wrap/Sequence.java
+ hat/wrap/wrap/src/main/java/wrap/Wrap.java


From gfrost at openjdk.org  Tue Feb 11 10:18:36 2025
From: gfrost at openjdk.org (Gary Frost)
Date: Tue, 11 Feb 2025 10:18:36 GMT
Subject: [code-reflection] Integrated: Hat jextracted backend infra
In-Reply-To: <48qWgXn27Q0WBp4QsOZdxSXitdVzEMEYjaOAeQE9C6U=.ff1c1b77-2cef-47bc-823e-eb5e25eba8ca@github.com>
References: <48qWgXn27Q0WBp4QsOZdxSXitdVzEMEYjaOAeQE9C6U=.ff1c1b77-2cef-47bc-823e-eb5e25eba8ca@github.com>
Message-ID: <QSlxOOBmkRaYuly2oXyuia48mEDULWK9dXmGbj8ZJiE=.7bdea995-d0f4-48a8-9ec9-56e7b27c7107@github.com>

On Tue, 11 Feb 2025 10:11:44 GMT, Gary Frost <gfrost at openjdk.org> wrote:

> More jextract infra. 
> 
> Specifically created wrappers (cuwrap, clwrap and glwrap).  These wrap jextracted CUDA, OpenCL and OpenGL. providing a consistent API across for jextracted backends to build against as well as allowing us to include non hat examples (just using these wrappers)

This pull request has now been integrated.

Changeset: 8cde02f9
Author:    Gary Frost <gfrost at openjdk.org>
URL:       https://git.openjdk.org/babylon/commit/8cde02f9cea452e750fd783cdd71f9c85e60388b
Stats:     2063 lines in 27 files changed: 2060 ins; 0 del; 3 mod

Hat jextracted backend infra

-------------

PR: https://git.openjdk.org/babylon/pull/314

From gfrost at openjdk.org  Tue Feb 11 10:18:36 2025
From: gfrost at openjdk.org (Gary Frost)
Date: Tue, 11 Feb 2025 10:18:36 GMT
Subject: [code-reflection] Integrated: Hat jextracted backend infra
Message-ID: <48qWgXn27Q0WBp4QsOZdxSXitdVzEMEYjaOAeQE9C6U=.ff1c1b77-2cef-47bc-823e-eb5e25eba8ca@github.com>

More jextract infra. 

Specifically created wrappers (cuwrap, clwrap and glwrap).  These wrap jextracted CUDA, OpenCL and OpenGL. providing a consistent API across for jextracted backends to build against as well as allowing us to include non hat examples (just using these wrappers)

-------------

Commit messages:
 - Added missing licenses
 - created clwrap,cuwrap,glwrap and wrap (shared) subprojects
 - Initial code (and bld changes) for jextracted backend infra

Changes: https://git.openjdk.org/babylon/pull/314/files
  Webrev: https://webrevs.openjdk.org/?repo=babylon&pr=314&range=00
  Stats: 2063 lines in 27 files changed: 2060 ins; 0 del; 3 mod
  Patch: https://git.openjdk.org/babylon/pull/314.diff
  Fetch: git fetch https://git.openjdk.org/babylon.git pull/314/head:pull/314

PR: https://git.openjdk.org/babylon/pull/314

From gary.frost at oracle.com  Tue Feb 11 10:25:17 2025
From: gary.frost at oracle.com (Gary Frost)
Date: Tue, 11 Feb 2025 10:25:17 +0000
Subject: [blog] Early Performance Findings: Babylon/HAT and TornadoVM
In-Reply-To: <CWLP265MB2308E6F012C3A5465DF66BE0C2F22@CWLP265MB2308.GBRP265.PROD.OUTLOOK.COM>
References: <CWLP265MB2308E6F012C3A5465DF66BE0C2F22@CWLP265MB2308.GBRP265.PROD.OUTLOOK.COM>
Message-ID: <MN2PR10MB4365FFCB20AE1CDC1B33DD8DFCFD2@MN2PR10MB4365.namprd10.prod.outlook.com>

Thanks for the link Juan.

This is a nice writeup, and I think offers a pretty succinct view of the current state of play.

Whilst I believe it is pretty early for us to chase perf comparisons, I think this puts a nice stake in the ground as we prepare to implement our algorithm for automatically avoiding unnecessary buffer transfers.

Re 2D and 3D ranges and for mechanism to select specific GPU's.  These are all in the 'backlog' :) thanks for keeping us honest.

I look forward to continued HAT + TornadoVM collaboration in the future.  I think the iface-mapper is very likely to be an early collaboration point.

Gary


________________________________
From: babylon-dev <babylon-dev-retn at openjdk.org> on behalf of Juan Fumero <juan.fumero at manchester.ac.uk>
Sent: Monday, February 10, 2025 8:23 AM
To: babylon-dev at openjdk.org <babylon-dev at openjdk.org>
Subject: [blog] Early Performance Findings: Babylon/HAT and TornadoVM

Hi all,
    I've written an article comparing some performance of Babylon/HAT against TornadoVM. Since this touches on GPU utilization and performance, I thought it might be of interest to the GPU folks here. The article also includes some constructive feedback on both technologies.
You can find the article here: https://jjfumero.github.io/posts/2025/02/07/babylon-and-tornadovm
Juan

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/babylon-dev/attachments/20250211/d1a28bdb/attachment.htm>

From asotona at openjdk.org  Tue Feb 11 10:27:01 2025
From: asotona at openjdk.org (Adam Sotona)
Date: Tue, 11 Feb 2025 10:27:01 GMT
Subject: [code-reflection] RFR: ONNX FFM Runtime initial work [v4]
In-Reply-To: <f3CBMcuRX8reQzHRmd_9eGaN8jEh4iJeoIayQB9ffe0=.ea5873ca-fbd3-41cf-8baa-91fdf3443947@github.com>
References: <f3CBMcuRX8reQzHRmd_9eGaN8jEh4iJeoIayQB9ffe0=.ea5873ca-fbd3-41cf-8baa-91fdf3443947@github.com>
Message-ID: <1zEmpVkGsHrzU4qz_CZ2N8IGMTpPaRIfBqTg2B3IfRw=.5ea8b3f5-fc19-45ae-86bf-afe81aa0d24c@github.com>

> This is initial work on ONNX FFM runtime with very raw connection with OnnxInterpreter and Tensor.
> 
> It is a rebase of https://github.com/PaulSandoz/babylon/pull/1

Adam Sotona has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 12 commits:

 - Merge remote-tracking branch 'babylon/code-reflection' into onnx-runtime-initial
   
 - Merged OnnxRuntime with OnnxRuntime.Environment and made singleton with shutdown hook
   
 - minor rename
   
 - OnnxRuntime use OnnxSchema - work in progress
   
 - minor OnnxProtoBuilder change
   
 - minor OnnxProtoBuilder change
   
 - unused imports
   
 - Implementation of dedicated OnnxProtoBuilder
   
 - OnnxRuntime work in progress
   
 - OnnxRuntime work in progress
   
 - ... and 2 more: https://git.openjdk.org/babylon/compare/8cde02f9...ed4b8178

-------------

Changes: https://git.openjdk.org/babylon/pull/311/files
  Webrev: https://webrevs.openjdk.org/?repo=babylon&pr=311&range=03
  Stats: 942 lines in 7 files changed: 936 ins; 4 del; 2 mod
  Patch: https://git.openjdk.org/babylon/pull/311.diff
  Fetch: git fetch https://git.openjdk.org/babylon.git pull/311/head:pull/311

PR: https://git.openjdk.org/babylon/pull/311

From asotona at openjdk.org  Tue Feb 11 13:48:41 2025
From: asotona at openjdk.org (Adam Sotona)
Date: Tue, 11 Feb 2025 13:48:41 GMT
Subject: [code-reflection] RFR: ONNX FFM Runtime initial work [v5]
In-Reply-To: <f3CBMcuRX8reQzHRmd_9eGaN8jEh4iJeoIayQB9ffe0=.ea5873ca-fbd3-41cf-8baa-91fdf3443947@github.com>
References: <f3CBMcuRX8reQzHRmd_9eGaN8jEh4iJeoIayQB9ffe0=.ea5873ca-fbd3-41cf-8baa-91fdf3443947@github.com>
Message-ID: <qeqfBgeBTu8pJSozKbbmyUUFIlE-X0G9FBFpmse71dE=.e9060119-a64d-44cb-b77d-558ab67bfab2@github.com>

> This is initial work on ONNX FFM runtime with very raw connection with OnnxInterpreter and Tensor.
> 
> It is a rebase of https://github.com/PaulSandoz/babylon/pull/1

Adam Sotona has updated the pull request incrementally with two additional commits since the last revision:

 - runtime getting tensor element type from the first input value
   
 - interpreter returns array of tensors for multiple outputs

-------------

Changes:
  - all: https://git.openjdk.org/babylon/pull/311/files
  - new: https://git.openjdk.org/babylon/pull/311/files/ed4b8178..f4d30408

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=babylon&pr=311&range=04
 - incr: https://webrevs.openjdk.org/?repo=babylon&pr=311&range=03-04

  Stats: 11 lines in 2 files changed: 6 ins; 0 del; 5 mod
  Patch: https://git.openjdk.org/babylon/pull/311.diff
  Fetch: git fetch https://git.openjdk.org/babylon.git pull/311/head:pull/311

PR: https://git.openjdk.org/babylon/pull/311

From asotona at openjdk.org  Tue Feb 11 14:12:50 2025
From: asotona at openjdk.org (Adam Sotona)
Date: Tue, 11 Feb 2025 14:12:50 GMT
Subject: [code-reflection] RFR: ONNX FFM Runtime initial work [v6]
In-Reply-To: <f3CBMcuRX8reQzHRmd_9eGaN8jEh4iJeoIayQB9ffe0=.ea5873ca-fbd3-41cf-8baa-91fdf3443947@github.com>
References: <f3CBMcuRX8reQzHRmd_9eGaN8jEh4iJeoIayQB9ffe0=.ea5873ca-fbd3-41cf-8baa-91fdf3443947@github.com>
Message-ID: <koHIKhu4vp8SCumskksNsvmM3myDBwjh76YnqPCmIJw=.3c8a1135-5b1a-4985-a323-f23fc12c6f59@github.com>

> This is initial work on ONNX FFM runtime with very raw connection with OnnxInterpreter and Tensor.
> 
> It is a rebase of https://github.com/PaulSandoz/babylon/pull/1

Adam Sotona has updated the pull request incrementally with one additional commit since the last revision:

  working with flat tensors by default

-------------

Changes:
  - all: https://git.openjdk.org/babylon/pull/311/files
  - new: https://git.openjdk.org/babylon/pull/311/files/f4d30408..7f44d909

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=babylon&pr=311&range=05
 - incr: https://webrevs.openjdk.org/?repo=babylon&pr=311&range=04-05

  Stats: 15 lines in 2 files changed: 0 ins; 6 del; 9 mod
  Patch: https://git.openjdk.org/babylon/pull/311.diff
  Fetch: git fetch https://git.openjdk.org/babylon.git pull/311/head:pull/311

PR: https://git.openjdk.org/babylon/pull/311

From asotona at openjdk.org  Tue Feb 11 16:57:15 2025
From: asotona at openjdk.org (Adam Sotona)
Date: Tue, 11 Feb 2025 16:57:15 GMT
Subject: [code-reflection] RFR: ONNX FFM Runtime initial work [v7]
In-Reply-To: <f3CBMcuRX8reQzHRmd_9eGaN8jEh4iJeoIayQB9ffe0=.ea5873ca-fbd3-41cf-8baa-91fdf3443947@github.com>
References: <f3CBMcuRX8reQzHRmd_9eGaN8jEh4iJeoIayQB9ffe0=.ea5873ca-fbd3-41cf-8baa-91fdf3443947@github.com>
Message-ID: <pwts4ZIsv5nnC2K99osv4-3bpjxW2vSh5NrDylinR-s=.cec69e17-e433-4ab4-8c43-c8230d075d8d@github.com>

> This is initial work on ONNX FFM runtime with very raw connection with OnnxInterpreter and Tensor.
> 
> It is a rebase of https://github.com/PaulSandoz/babylon/pull/1

Adam Sotona has updated the pull request incrementally with one additional commit since the last revision:

  OnnxProtoBuilder and OnnxRuntime work in progress

-------------

Changes:
  - all: https://git.openjdk.org/babylon/pull/311/files
  - new: https://git.openjdk.org/babylon/pull/311/files/7f44d909..d67f5e88

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=babylon&pr=311&range=06
 - incr: https://webrevs.openjdk.org/?repo=babylon&pr=311&range=05-06

  Stats: 91 lines in 6 files changed: 47 ins; 17 del; 27 mod
  Patch: https://git.openjdk.org/babylon/pull/311.diff
  Fetch: git fetch https://git.openjdk.org/babylon.git pull/311/head:pull/311

PR: https://git.openjdk.org/babylon/pull/311

From asotona at openjdk.org  Tue Feb 11 17:52:42 2025
From: asotona at openjdk.org (Adam Sotona)
Date: Tue, 11 Feb 2025 17:52:42 GMT
Subject: [code-reflection] RFR: ONNX FFM Runtime initial work [v8]
In-Reply-To: <f3CBMcuRX8reQzHRmd_9eGaN8jEh4iJeoIayQB9ffe0=.ea5873ca-fbd3-41cf-8baa-91fdf3443947@github.com>
References: <f3CBMcuRX8reQzHRmd_9eGaN8jEh4iJeoIayQB9ffe0=.ea5873ca-fbd3-41cf-8baa-91fdf3443947@github.com>
Message-ID: <mbpQRgx_ixeKV4BjLeSNedwpS8oH0SjxaJmDUZA9iT4=.4ed6d62b-ed3b-448c-859f-c3a328ad3692@github.com>

> This is initial work on ONNX FFM runtime with very raw connection with OnnxInterpreter and Tensor.
> 
> It is a rebase of https://github.com/PaulSandoz/babylon/pull/1

Adam Sotona has updated the pull request incrementally with one additional commit since the last revision:

  OnnxProtoBuilder resolution of output tensor types from scheme + shape fix and test

-------------

Changes:
  - all: https://git.openjdk.org/babylon/pull/311/files
  - new: https://git.openjdk.org/babylon/pull/311/files/d67f5e88..d5113ede

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=babylon&pr=311&range=07
 - incr: https://webrevs.openjdk.org/?repo=babylon&pr=311&range=06-07

  Stats: 85 lines in 4 files changed: 73 ins; 2 del; 10 mod
  Patch: https://git.openjdk.org/babylon/pull/311.diff
  Fetch: git fetch https://git.openjdk.org/babylon.git pull/311/head:pull/311

PR: https://git.openjdk.org/babylon/pull/311

From mabbay at openjdk.org  Tue Feb 11 20:15:06 2025
From: mabbay at openjdk.org (Mourad Abbay)
Date: Tue, 11 Feb 2025 20:15:06 GMT
Subject: [code-reflection] RFR: Support storing the code that builds the
 code model [v6]
In-Reply-To: <dexv8sQV_NcJsJIgMopd_M4wSWoZya9dU0ACwDtEL0E=.4ac0b80f-24fb-440f-a18d-bbfd07a681bc@github.com>
References: <dexv8sQV_NcJsJIgMopd_M4wSWoZya9dU0ACwDtEL0E=.4ac0b80f-24fb-440f-a18d-bbfd07a681bc@github.com>
Message-ID: <bzLQ1zOmdgAkK8AnkzhtzS28SaySIZpOnARbcLVgiYQ=.6f0c8491-2b78-46ad-846c-6c871c7931d9@github.com>

> In this PR we allow the user to decide how to store the code model. The first option is `TEXT`, if this is selected, we store the textual representation of the code model. The second option is `CODE_BUILDR`, if this is selected, we store the code that builds the code model. All work done here is around the second option, because the first option is already supported.

Mourad Abbay has updated the pull request incrementally with two additional commits since the last revision:

 - Reformat code
 - Apply fixes from Maurizio

-------------

Changes:
  - all: https://git.openjdk.org/babylon/pull/305/files
  - new: https://git.openjdk.org/babylon/pull/305/files/16b16087..21c12cb3

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=babylon&pr=305&range=05
 - incr: https://webrevs.openjdk.org/?repo=babylon&pr=305&range=04-05

  Stats: 16 lines in 1 file changed: 2 ins; 6 del; 8 mod
  Patch: https://git.openjdk.org/babylon/pull/305.diff
  Fetch: git fetch https://git.openjdk.org/babylon.git pull/305/head:pull/305

PR: https://git.openjdk.org/babylon/pull/305

From asotona at openjdk.org  Tue Feb 11 20:38:34 2025
From: asotona at openjdk.org (Adam Sotona)
Date: Tue, 11 Feb 2025 20:38:34 GMT
Subject: [code-reflection] RFR: ONNX FFM Runtime initial work [v9]
In-Reply-To: <f3CBMcuRX8reQzHRmd_9eGaN8jEh4iJeoIayQB9ffe0=.ea5873ca-fbd3-41cf-8baa-91fdf3443947@github.com>
References: <f3CBMcuRX8reQzHRmd_9eGaN8jEh4iJeoIayQB9ffe0=.ea5873ca-fbd3-41cf-8baa-91fdf3443947@github.com>
Message-ID: <rECL8ryYDJOVhuKbG02yPvqkI9FGZqJUJcRIZG3i8nc=.2ce19000-3f31-4f5f-be4d-b46ece739a19@github.com>

> This is initial work on ONNX FFM runtime with very raw connection with OnnxInterpreter and Tensor.
> 
> It is a rebase of https://github.com/PaulSandoz/babylon/pull/1

Adam Sotona has updated the pull request incrementally with one additional commit since the last revision:

  OnnxProtoBuilder and OnnxRuntime simplifications

-------------

Changes:
  - all: https://git.openjdk.org/babylon/pull/311/files
  - new: https://git.openjdk.org/babylon/pull/311/files/d5113ede..3d44e6a6

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=babylon&pr=311&range=08
 - incr: https://webrevs.openjdk.org/?repo=babylon&pr=311&range=07-08

  Stats: 71 lines in 4 files changed: 4 ins; 55 del; 12 mod
  Patch: https://git.openjdk.org/babylon/pull/311.diff
  Fetch: git fetch https://git.openjdk.org/babylon.git pull/311/head:pull/311

PR: https://git.openjdk.org/babylon/pull/311

From psandoz at openjdk.org  Tue Feb 11 22:13:48 2025
From: psandoz at openjdk.org (Paul Sandoz)
Date: Tue, 11 Feb 2025 22:13:48 GMT
Subject: git: openjdk/babylon: code-reflection: Remove redundant type variables
Message-ID: <92e1e851-55b8-4820-a9c9-d1bfe69b946f@openjdk.org>

Changeset: 7ce041b3
Branch: code-reflection
Author:    Paul Sandoz <psandoz at openjdk.org>
Date:      2025-02-11 22:11:30 +0000
URL:       https://git.openjdk.org/babylon/commit/7ce041b3ca8e95cbc195bd5c2adc4157a8a932c8

Remove redundant type variables

! cr-examples/onnx/opgen/src/main/java/oracle/code/onnx/opgen/OperatorGen.java
! cr-examples/onnx/src/main/java/oracle/code/onnx/OnnxOperators.java


From psandoz at openjdk.org  Tue Feb 11 22:14:55 2025
From: psandoz at openjdk.org (Paul Sandoz)
Date: Tue, 11 Feb 2025 22:14:55 GMT
Subject: [code-reflection] Integrated: Remove redundant type variables
Message-ID: <0eWzCOw72w8wmtlN-olY5CE9Fb1iXMug3CxICOJrnuw=.8d82477f-3ca6-4627-8dce-aaec6102ecd1@github.com>

Remove redundant type variables from the ONNX operator API.

-------------

Commit messages:
 - Remove redundant type variables.

Changes: https://git.openjdk.org/babylon/pull/315/files
  Webrev: https://webrevs.openjdk.org/?repo=babylon&pr=315&range=00
  Stats: 117 lines in 2 files changed: 24 ins; 19 del; 74 mod
  Patch: https://git.openjdk.org/babylon/pull/315.diff
  Fetch: git fetch https://git.openjdk.org/babylon.git pull/315/head:pull/315

PR: https://git.openjdk.org/babylon/pull/315

From psandoz at openjdk.org  Tue Feb 11 22:14:56 2025
From: psandoz at openjdk.org (Paul Sandoz)
Date: Tue, 11 Feb 2025 22:14:56 GMT
Subject: [code-reflection] Integrated: Remove redundant type variables
In-Reply-To: <0eWzCOw72w8wmtlN-olY5CE9Fb1iXMug3CxICOJrnuw=.8d82477f-3ca6-4627-8dce-aaec6102ecd1@github.com>
References: <0eWzCOw72w8wmtlN-olY5CE9Fb1iXMug3CxICOJrnuw=.8d82477f-3ca6-4627-8dce-aaec6102ecd1@github.com>
Message-ID: <ehuhLf6TKANAHJtwbnJ2wVRP1rJESv5U2Nw9x0b11eU=.869617f3-773f-4a4b-8b61-a0a6ebf1d021@github.com>

On Tue, 11 Feb 2025 22:07:16 GMT, Paul Sandoz <psandoz at openjdk.org> wrote:

> Remove redundant type variables from the ONNX operator API.

This pull request has now been integrated.

Changeset: 7ce041b3
Author:    Paul Sandoz <psandoz at openjdk.org>
URL:       https://git.openjdk.org/babylon/commit/7ce041b3ca8e95cbc195bd5c2adc4157a8a932c8
Stats:     117 lines in 2 files changed: 24 ins; 19 del; 74 mod

Remove redundant type variables

-------------

PR: https://git.openjdk.org/babylon/pull/315

From asotona at openjdk.org  Tue Feb 11 22:24:09 2025
From: asotona at openjdk.org (Adam Sotona)
Date: Tue, 11 Feb 2025 22:24:09 GMT
Subject: [code-reflection] RFR: ONNX FFM Runtime initial work [v10]
In-Reply-To: <f3CBMcuRX8reQzHRmd_9eGaN8jEh4iJeoIayQB9ffe0=.ea5873ca-fbd3-41cf-8baa-91fdf3443947@github.com>
References: <f3CBMcuRX8reQzHRmd_9eGaN8jEh4iJeoIayQB9ffe0=.ea5873ca-fbd3-41cf-8baa-91fdf3443947@github.com>
Message-ID: <SfT6vAI3Q7PNZlR3VeekvRDS84cg4D-LhHTFRFF_EJA=.7af72223-0df8-4fe6-9a3f-aadd00630353@github.com>

> This is initial work on ONNX FFM runtime with very raw connection with OnnxInterpreter and Tensor.
> 
> It is a rebase of https://github.com/PaulSandoz/babylon/pull/1

Adam Sotona has updated the pull request incrementally with one additional commit since the last revision:

  initial draft of OnnxProtoBuilder::buildFuncModel and OnnxRuntime::runFunc

-------------

Changes:
  - all: https://git.openjdk.org/babylon/pull/311/files
  - new: https://git.openjdk.org/babylon/pull/311/files/3d44e6a6..f6143f68

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=babylon&pr=311&range=09
 - incr: https://webrevs.openjdk.org/?repo=babylon&pr=311&range=08-09

  Stats: 137 lines in 3 files changed: 114 ins; 0 del; 23 mod
  Patch: https://git.openjdk.org/babylon/pull/311.diff
  Fetch: git fetch https://git.openjdk.org/babylon.git pull/311/head:pull/311

PR: https://git.openjdk.org/babylon/pull/311

From asotona at openjdk.org  Wed Feb 12 07:09:52 2025
From: asotona at openjdk.org (Adam Sotona)
Date: Wed, 12 Feb 2025 07:09:52 GMT
Subject: [code-reflection] RFR: ONNX FFM Runtime initial work [v11]
In-Reply-To: <f3CBMcuRX8reQzHRmd_9eGaN8jEh4iJeoIayQB9ffe0=.ea5873ca-fbd3-41cf-8baa-91fdf3443947@github.com>
References: <f3CBMcuRX8reQzHRmd_9eGaN8jEh4iJeoIayQB9ffe0=.ea5873ca-fbd3-41cf-8baa-91fdf3443947@github.com>
Message-ID: <Voi5CZ_H_bmt-da8vB-mWofChRhp_NrVTvJoSCBz0eE=.9c81b315-a9eb-49c7-9766-25ddc16d98c7@github.com>

> This is initial work on ONNX FFM runtime with very raw connection with OnnxInterpreter and Tensor.
> 
> It is a rebase of https://github.com/PaulSandoz/babylon/pull/1

Adam Sotona has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 20 commits:

 - Merge remote-tracking branch 'babylon/code-reflection' into onnx-runtime-initial
 - initial draft of OnnxProtoBuilder::buildFuncModel and OnnxRuntime::runFunc
   
 - OnnxProtoBuilder and OnnxRuntime simplifications
   
 - OnnxProtoBuilder resolution of output tensor types from scheme + shape fix and test
   
 - OnnxProtoBuilder and OnnxRuntime work in progress
   
 - working with flat tensors by default
   
 - runtime getting tensor element type from the first input value
   
 - interpreter returns array of tensors for multiple outputs
   
 - Merge remote-tracking branch 'babylon/code-reflection' into onnx-runtime-initial
   
 - Merged OnnxRuntime with OnnxRuntime.Environment and made singleton with shutdown hook
   
 - ... and 10 more: https://git.openjdk.org/babylon/compare/7ce041b3...f89647d7

-------------

Changes: https://git.openjdk.org/babylon/pull/311/files
  Webrev: https://webrevs.openjdk.org/?repo=babylon&pr=311&range=10
  Stats: 1106 lines in 8 files changed: 1100 ins; 4 del; 2 mod
  Patch: https://git.openjdk.org/babylon/pull/311.diff
  Fetch: git fetch https://git.openjdk.org/babylon.git pull/311/head:pull/311

PR: https://git.openjdk.org/babylon/pull/311

From asotona at openjdk.org  Wed Feb 12 07:13:25 2025
From: asotona at openjdk.org (Adam Sotona)
Date: Wed, 12 Feb 2025 07:13:25 GMT
Subject: git: openjdk/babylon: code-reflection: ONNX FFM Runtime initial work
Message-ID: <cea9757d-08ce-467a-a200-3dde3d54ef4a@openjdk.org>

Changeset: 1f9416b7
Branch: code-reflection
Author:    Adam Sotona <asotona at openjdk.org>
Date:      2025-02-12 07:11:25 +0000
URL:       https://git.openjdk.org/babylon/commit/1f9416b75e82f69ae3361adb59b72b90a1f5b455

ONNX FFM Runtime initial work

! cr-examples/onnx/pom.xml
! cr-examples/onnx/src/main/java/oracle/code/onnx/OnnxInterpreter.java
+ cr-examples/onnx/src/main/java/oracle/code/onnx/OnnxProtoBuilder.java
+ cr-examples/onnx/src/main/java/oracle/code/onnx/OnnxRuntime.java
! cr-examples/onnx/src/main/java/oracle/code/onnx/Tensor.java
! cr-examples/onnx/src/main/java/oracle/code/onnx/ir/OnnxOp.java
+ cr-examples/onnx/src/test/java/oracle/code/onnx/RuntimeTest.java
+ cr-examples/onnx/src/test/java/oracle/code/onnx/SimpleTest.java


From asotona at openjdk.org  Wed Feb 12 07:14:28 2025
From: asotona at openjdk.org (Adam Sotona)
Date: Wed, 12 Feb 2025 07:14:28 GMT
Subject: [code-reflection] RFR: ONNX FFM Runtime initial work [v11]
In-Reply-To: <Voi5CZ_H_bmt-da8vB-mWofChRhp_NrVTvJoSCBz0eE=.9c81b315-a9eb-49c7-9766-25ddc16d98c7@github.com>
References: <f3CBMcuRX8reQzHRmd_9eGaN8jEh4iJeoIayQB9ffe0=.ea5873ca-fbd3-41cf-8baa-91fdf3443947@github.com>
 <Voi5CZ_H_bmt-da8vB-mWofChRhp_NrVTvJoSCBz0eE=.9c81b315-a9eb-49c7-9766-25ddc16d98c7@github.com>
Message-ID: <C9QBEfnDLwD-L-YExQQlzlGdr0DpaCXZQA2SWcpmeTE=.ab7a8e9e-045f-44d0-90da-0f0dde5964e7@github.com>

On Wed, 12 Feb 2025 07:09:52 GMT, Adam Sotona <asotona at openjdk.org> wrote:

>> This is initial work on ONNX FFM runtime with very raw connection with OnnxInterpreter and Tensor.
>> 
>> It is a rebase of https://github.com/PaulSandoz/babylon/pull/1
>
> Adam Sotona has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 20 commits:
> 
>  - Merge remote-tracking branch 'babylon/code-reflection' into onnx-runtime-initial
>  - initial draft of OnnxProtoBuilder::buildFuncModel and OnnxRuntime::runFunc
>    
>  - OnnxProtoBuilder and OnnxRuntime simplifications
>    
>  - OnnxProtoBuilder resolution of output tensor types from scheme + shape fix and test
>    
>  - OnnxProtoBuilder and OnnxRuntime work in progress
>    
>  - working with flat tensors by default
>    
>  - runtime getting tensor element type from the first input value
>    
>  - interpreter returns array of tensors for multiple outputs
>    
>  - Merge remote-tracking branch 'babylon/code-reflection' into onnx-runtime-initial
>    
>  - Merged OnnxRuntime with OnnxRuntime.Environment and made singleton with shutdown hook
>    
>  - ... and 10 more: https://git.openjdk.org/babylon/compare/7ce041b3...f89647d7

Thank you for all the comments and reviews!

I'll integrate this PR and let's continue with a new one.

-------------

PR Comment: https://git.openjdk.org/babylon/pull/311#issuecomment-2652839757

From asotona at openjdk.org  Wed Feb 12 07:14:29 2025
From: asotona at openjdk.org (Adam Sotona)
Date: Wed, 12 Feb 2025 07:14:29 GMT
Subject: [code-reflection] Integrated: ONNX FFM Runtime initial work
In-Reply-To: <f3CBMcuRX8reQzHRmd_9eGaN8jEh4iJeoIayQB9ffe0=.ea5873ca-fbd3-41cf-8baa-91fdf3443947@github.com>
References: <f3CBMcuRX8reQzHRmd_9eGaN8jEh4iJeoIayQB9ffe0=.ea5873ca-fbd3-41cf-8baa-91fdf3443947@github.com>
Message-ID: <QXiVc4OewHTuwo46cC-54wTJkC1_rOGtNJH8rI1dpLw=.c1826a73-b1c3-46dc-83d0-802c9863b0f9@github.com>

On Sun, 9 Feb 2025 09:23:38 GMT, Adam Sotona <asotona at openjdk.org> wrote:

> This is initial work on ONNX FFM runtime with very raw connection with OnnxInterpreter and Tensor.
> 
> It is a rebase of https://github.com/PaulSandoz/babylon/pull/1

This pull request has now been integrated.

Changeset: 1f9416b7
Author:    Adam Sotona <asotona at openjdk.org>
URL:       https://git.openjdk.org/babylon/commit/1f9416b75e82f69ae3361adb59b72b90a1f5b455
Stats:     1106 lines in 8 files changed: 1100 ins; 4 del; 2 mod

ONNX FFM Runtime initial work

-------------

PR: https://git.openjdk.org/babylon/pull/311

From pminborg at openjdk.org  Wed Feb 12 08:51:29 2025
From: pminborg at openjdk.org (Per Minborg)
Date: Wed, 12 Feb 2025 08:51:29 GMT
Subject: [code-reflection] RFR: ONNX FFM Runtime initial work [v3]
In-Reply-To: <0C6eit0XhgEoQo1eiqskpIGCcGqlpsZhaYz1-egSLSI=.d94d5823-7ed1-4af4-8845-61358215087f@github.com>
References: <f3CBMcuRX8reQzHRmd_9eGaN8jEh4iJeoIayQB9ffe0=.ea5873ca-fbd3-41cf-8baa-91fdf3443947@github.com>
 <kpTYerPqazGPqWWvAhZF65dzeWFvanGbIXPEeU3CCiM=.9b9ac033-59fb-48c2-8e1f-0670990cfa6a@github.com>
 <amiwNfz2y34g8tLWi5amY4sf9L7skPnk36eDto3XVOo=.374a48dc-60a7-43f5-a271-f134673b32f1@github.com>
 <0C6eit0XhgEoQo1eiqskpIGCcGqlpsZhaYz1-egSLSI=.d94d5823-7ed1-4af4-8845-61358215087f@github.com>
Message-ID: <1CWbIdt7fOjGh6vypaK9Nc1TroFuo7xXdHK9M2d7NTo=.f6d180fa-6539-423f-b690-b177e39cb13d@github.com>

On Mon, 10 Feb 2025 14:18:49 GMT, Adam Sotona <asotona at openjdk.org> wrote:

>> cr-examples/onnx/src/main/java/oracle/code/onnx/OnnxRuntime.java line 68:
>> 
>>> 66:     private final SymbolLookup  library;
>>> 67:     private final MemorySegment runtimeAddress, ret;
>>> 68:     private final MethodHandle  allocatorGetInfo,
>> 
>> Can these be `static` for better performance?
>
> No with the current API design. The memory segment of the instance of obtained by `OrtGetApiBase()->GetApi((uint32_t)apiVersion)`

Can they be @Stable ?

-------------

PR Review Comment: https://git.openjdk.org/babylon/pull/311#discussion_r1952219386

From psandoz at openjdk.org  Wed Feb 12 15:12:38 2025
From: psandoz at openjdk.org (Paul Sandoz)
Date: Wed, 12 Feb 2025 15:12:38 GMT
Subject: [code-reflection] RFR: ONNX FFM Runtime initial work [v3]
In-Reply-To: <1CWbIdt7fOjGh6vypaK9Nc1TroFuo7xXdHK9M2d7NTo=.f6d180fa-6539-423f-b690-b177e39cb13d@github.com>
References: <f3CBMcuRX8reQzHRmd_9eGaN8jEh4iJeoIayQB9ffe0=.ea5873ca-fbd3-41cf-8baa-91fdf3443947@github.com>
 <kpTYerPqazGPqWWvAhZF65dzeWFvanGbIXPEeU3CCiM=.9b9ac033-59fb-48c2-8e1f-0670990cfa6a@github.com>
 <amiwNfz2y34g8tLWi5amY4sf9L7skPnk36eDto3XVOo=.374a48dc-60a7-43f5-a271-f134673b32f1@github.com>
 <0C6eit0XhgEoQo1eiqskpIGCcGqlpsZhaYz1-egSLSI=.d94d5823-7ed1-4af4-8845-61358215087f@github.com>
 <1CWbIdt7fOjGh6vypaK9Nc1TroFuo7xXdHK9M2d7NTo=.f6d180fa-6539-423f-b690-b177e39cb13d@github.com>
Message-ID: <L9FEtmDbCU9RFmNSUEOiNwnYE5vZdgbE6AAadS5tCvI=.3a5b10b9-54cc-4ffb-908c-65dbd18e5725@github.com>

On Wed, 12 Feb 2025 08:48:36 GMT, Per Minborg <pminborg at openjdk.org> wrote:

>> No with the current API design. The memory segment of the instance of obtained by `OrtGetApiBase()->GetApi((uint32_t)apiVersion)`
>
> Can they be @Stable ?

We can make things more optimal (if possible) later on, after we get things working.

-------------

PR Review Comment: https://git.openjdk.org/babylon/pull/311#discussion_r1952847821

From asotona at openjdk.org  Wed Feb 12 17:49:57 2025
From: asotona at openjdk.org (Adam Sotona)
Date: Wed, 12 Feb 2025 17:49:57 GMT
Subject: [code-reflection] RFR: OnnxRuntime and OnnxProtoBuilder continuation
Message-ID: <0UVt3G80W3Rotrsjj3HsrHZZC8BTDOlnd3ZDWtbXYKk=.45c7437b-0e3c-45ea-b52e-49bae61e2a80@github.com>

OnnxRuntime and OnnxProtoBuilder continuation

-------------

Commit messages:
 - improved SimpleTest to cover constants encoding in protobuf models
 - modeling of attributes (for constant op)
 - fixed protobuf encoding of longs
 - adding attributes to model building - work in progress
 - OnnxProtoBuilder comments + SimpleTest consolidation
 - added OnnxElementType::id, simplified OnnxProtoBuilder::buildFuncModel and consolidated RuntimeTest and SimnpleTest

Changes: https://git.openjdk.org/babylon/pull/316/files
  Webrev: https://webrevs.openjdk.org/?repo=babylon&pr=316&range=00
  Stats: 358 lines in 7 files changed: 223 ins; 91 del; 44 mod
  Patch: https://git.openjdk.org/babylon/pull/316.diff
  Fetch: git fetch https://git.openjdk.org/babylon.git pull/316/head:pull/316

PR: https://git.openjdk.org/babylon/pull/316

From duke at openjdk.org  Thu Feb 13 06:35:24 2025
From: duke at openjdk.org (Juan Fumero)
Date: Thu, 13 Feb 2025 06:35:24 GMT
Subject: [code-reflection] RFR: 1D Matrix Multiplication example for HAT
 [v2]
In-Reply-To: <QV4ZYVPuadGjonbLCZGvWBuGoZStuD5olcgoFQZVihw=.95e0d7b7-2a79-4b37-86c2-a7939259cdba@github.com>
References: <ZpTxbD2nAktHoH-AnjleYaCKBKDDlwZxhWV-blDbavg=.64e1317f-b888-4b9c-97d3-83c649081aab@github.com>
 <QV4ZYVPuadGjonbLCZGvWBuGoZStuD5olcgoFQZVihw=.95e0d7b7-2a79-4b37-86c2-a7939259cdba@github.com>
Message-ID: <WHOew3I33V79PuXZ-T39AqyggsaW_y6pJm5SIS3JkAQ=.503b84c3-0ccc-4a0b-967d-1dcb90935f6d@github.com>

On Sat, 18 Jan 2025 06:27:18 GMT, Juan Fumero <duke at openjdk.org> wrote:

>> Add new example for 1D Matrix Multiplication in HAT.
>> 
>> ### How to test?
>> 
>> 
>> ## Compile 
>> java --add-modules jdk.incubator.code --enable-preview --source 24 bld
>> 
>> ## Run with the OpenCL Backend
>> java @bldr/hatrun ffi-opencl matmul  
>> 
>> ## Run with the CUDA Backend
>> java @bldr/hatrun ffi-ptx matmul 
>> 
>> 
>> #### Note that the generated kernel for OpenCL contains a race condition:
>> 
>> 
>> __kernel void matrixMultiplyKernel(
>>     __global KernelContext_t *kc, __global F32Array_t* matrixA, __global F32Array_t* matrixB, __global F32Array_t* matrixC, int size
>> ){
>>     kc->x=get_global_id(0);                   //  << Shared struct across all threads to store the thread-id 
>>     if(kc->x<kc->maxX){
>>         for(int j = 0; j<size; j=j+1){
>>             float acc = (float)0;
>>             for(int k = 0; k<size; k=k+1){
>>                 acc=acc+matrixA->array[(long)(kc->x*size+k)]*matrixB->array[(long)(k*size+j)];
>>             }
>>             matrixC->array[(long)(kc->x*size+j)]=acc;
>>         }
>>     }
>>     return;
>> }
>> 
>> 
>> After applying a patch provided by Gary Frost to solve the race condition, it works. 
>> 
>> Patch:
>> 
>> 
>> diff --git a/hat/hat/src/main/java/hat/backend/c99codebuilders/C99HatKernelBuilder.java b/hat/hat/src/main/java/hat/backend/c99codebuilders/C99HatKernelBuilder.java
>> index ade90914d7e..2719fed31ed 100644
>> --- a/hat/hat/src/main/java/hat/backend/c99codebuilders/C99HatKernelBuilder.java
>> +++ b/hat/hat/src/main/java/hat/backend/c99codebuilders/C99HatKernelBuilder.java
>> @@ -26,7 +26,6 @@
>>  
>>  
>>  import hat.buffer.Buffer;
>> -import hat.buffer.KernelContext;
>>  import hat.callgraph.KernelCallGraph;
>>  import hat.callgraph.KernelEntrypoint;
>>  import hat.optools.FuncOpWrapper;
>> @@ -72,9 +71,13 @@ T typedefStructOrUnion(boolean isStruct, String name, Consumer<T> consumer) {
>>  
>>  
>>      public final T scope() {
>> -        return
>> -                identifier("kc").rarrow().identifier("x").equals().globalId().semicolon().nl();
>> -                //.identifier("kc").rarrow().identifier("maxX").equals().globalSize().semicolon().nl();
>> +
>> +        identifier("KernelContext_t").space().identifier("mine").semicolon().nl();
>> +        identifier("KernelContext_t").asterisk().space().identifier("kc").equals().ampersand().identifier("mine").semicolon().nl();
>> +        identifier("kc").rarrow().identifier("x").equals().globalId().semicolon().nl();
>> +        identifier("kc").rarrow().identifier("maxX").equals().identifie...
>
> Juan Fumero has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 11 commits:
> 
>  - Merge branch 'code-reflection' into dev/examples
>  - Minor fix seq-comparison code
>  - Merge with latest develop
>  - Merge branch 'code-reflection' into dev/examples
>  - Merge branch 'code-reflection' into dev/examples
>  - Merge branch 'code-reflection' into dev/examples
>  - MatrixMult example moved to matmul directory
>  - Merge branch 'code-reflection' into dev/examples
>  - Precision control error down to 1%
>  - Matrix-Multiplication checks
>  - ... and 1 more: https://git.openjdk.org/babylon/compare/ee3da036...cd3c7ce9

Pending for review

-------------

PR Comment: https://git.openjdk.org/babylon/pull/276#issuecomment-2655657381

From asotona at openjdk.org  Thu Feb 13 08:28:45 2025
From: asotona at openjdk.org (Adam Sotona)
Date: Thu, 13 Feb 2025 08:28:45 GMT
Subject: [code-reflection] RFR: OnnxRuntime and OnnxProtoBuilder
 continuation [v2]
In-Reply-To: <0UVt3G80W3Rotrsjj3HsrHZZC8BTDOlnd3ZDWtbXYKk=.45c7437b-0e3c-45ea-b52e-49bae61e2a80@github.com>
References: <0UVt3G80W3Rotrsjj3HsrHZZC8BTDOlnd3ZDWtbXYKk=.45c7437b-0e3c-45ea-b52e-49bae61e2a80@github.com>
Message-ID: <fxUnvLJ-eFKMNGHJKgFuGzPmKEx6-xnR2HpeRx_bEjM=.de7d89a1-df03-4b32-9ca1-2ff046327af5@github.com>

> OnnxRuntime and OnnxProtoBuilder continuation

Adam Sotona has updated the pull request incrementally with one additional commit since the last revision:

  OnnxProtoBuilder work in progress

-------------

Changes:
  - all: https://git.openjdk.org/babylon/pull/316/files
  - new: https://git.openjdk.org/babylon/pull/316/files/f24519f6..2585c841

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=babylon&pr=316&range=01
 - incr: https://webrevs.openjdk.org/?repo=babylon&pr=316&range=00-01

  Stats: 51 lines in 1 file changed: 30 ins; 16 del; 5 mod
  Patch: https://git.openjdk.org/babylon/pull/316.diff
  Fetch: git fetch https://git.openjdk.org/babylon.git pull/316/head:pull/316

PR: https://git.openjdk.org/babylon/pull/316

From asotona at openjdk.org  Thu Feb 13 09:17:00 2025
From: asotona at openjdk.org (Adam Sotona)
Date: Thu, 13 Feb 2025 09:17:00 GMT
Subject: [code-reflection] RFR: OnnxRuntime and OnnxProtoBuilder
 continuation [v3]
In-Reply-To: <0UVt3G80W3Rotrsjj3HsrHZZC8BTDOlnd3ZDWtbXYKk=.45c7437b-0e3c-45ea-b52e-49bae61e2a80@github.com>
References: <0UVt3G80W3Rotrsjj3HsrHZZC8BTDOlnd3ZDWtbXYKk=.45c7437b-0e3c-45ea-b52e-49bae61e2a80@github.com>
Message-ID: <zsANbrPpuFjymGS4sM7PEMHCMsHBXbb-UavyABKEIGg=.75763f22-18f6-4ce4-a2d4-f5b905ebe7b0@github.com>

> OnnxRuntime and OnnxProtoBuilder continuation

Adam Sotona has updated the pull request incrementally with one additional commit since the last revision:

  hack to workaround TupleLoadOp in the CNN example

-------------

Changes:
  - all: https://git.openjdk.org/babylon/pull/316/files
  - new: https://git.openjdk.org/babylon/pull/316/files/2585c841..f9b575e8

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=babylon&pr=316&range=02
 - incr: https://webrevs.openjdk.org/?repo=babylon&pr=316&range=01-02

  Stats: 8 lines in 1 file changed: 4 ins; 3 del; 1 mod
  Patch: https://git.openjdk.org/babylon/pull/316.diff
  Fetch: git fetch https://git.openjdk.org/babylon.git pull/316/head:pull/316

PR: https://git.openjdk.org/babylon/pull/316

From asotona at openjdk.org  Thu Feb 13 12:23:46 2025
From: asotona at openjdk.org (Adam Sotona)
Date: Thu, 13 Feb 2025 12:23:46 GMT
Subject: [code-reflection] RFR: OnnxRuntime and OnnxProtoBuilder
 continuation [v4]
In-Reply-To: <0UVt3G80W3Rotrsjj3HsrHZZC8BTDOlnd3ZDWtbXYKk=.45c7437b-0e3c-45ea-b52e-49bae61e2a80@github.com>
References: <0UVt3G80W3Rotrsjj3HsrHZZC8BTDOlnd3ZDWtbXYKk=.45c7437b-0e3c-45ea-b52e-49bae61e2a80@github.com>
Message-ID: <ebhlSrOuCd9uds4WPL3Ow57u1v3mhlxNRSbG9GlQiBk=.6bf5a6cc-3a3a-487a-b463-3c5f07ce916f@github.com>

> OnnxRuntime and OnnxProtoBuilder continuation

Adam Sotona has updated the pull request incrementally with one additional commit since the last revision:

  fixed scalars creation + tests

-------------

Changes:
  - all: https://git.openjdk.org/babylon/pull/316/files
  - new: https://git.openjdk.org/babylon/pull/316/files/f9b575e8..cc9ab739

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=babylon&pr=316&range=03
 - incr: https://webrevs.openjdk.org/?repo=babylon&pr=316&range=02-03

  Stats: 66 lines in 4 files changed: 53 ins; 1 del; 12 mod
  Patch: https://git.openjdk.org/babylon/pull/316.diff
  Fetch: git fetch https://git.openjdk.org/babylon.git pull/316/head:pull/316

PR: https://git.openjdk.org/babylon/pull/316

From gfrost at openjdk.org  Thu Feb 13 13:55:48 2025
From: gfrost at openjdk.org (Gary Frost)
Date: Thu, 13 Feb 2025 13:55:48 GMT
Subject: git: openjdk/babylon: code-reflection: Hat mark buffers
Message-ID: <b7ccac15-b18d-42d1-80b8-0d6815df07a0@openjdk.org>

Changeset: ca6bbc21
Branch: code-reflection
Author:    Gary Frost <gfrost at openjdk.org>
Date:      2025-02-13 13:55:22 +0000
URL:       https://git.openjdk.org/babylon/commit/ca6bbc211795bee5d87b08a514d4f8457c0af714

Hat mark buffers

! hat/backends/ffi/cuda/cpp/cuda_backend.cpp
! hat/backends/ffi/opencl/cpp/opencl_backend.cpp
! hat/backends/ffi/shared/include/shared.h
! hat/examples/life/src/main/java/life/Main.java
! hat/examples/mandel/src/main/java/mandel/Main.java
! hat/examples/violajones/src/main/java/violajones/ifaces/Cascade.java
! hat/examples/violajones/src/main/java/violajones/ifaces/ResultTable.java
! hat/hat/src/main/java/hat/ComputeContext.java
! hat/hat/src/main/java/hat/backend/codebuilders/HATCodeBuilderWithContext.java
! hat/hat/src/main/java/hat/backend/ffi/C99FFIBackend.java
! hat/hat/src/main/java/hat/backend/ffi/FFIBackend.java
! hat/hat/src/main/java/hat/backend/jextracted/C99JExtractedBackend.java
! hat/hat/src/main/java/hat/backend/jextracted/JExtractedBackend.java
- hat/hat/src/main/java/hat/buffer/After.java
! hat/hat/src/main/java/hat/buffer/ArgArray.java
- hat/hat/src/main/java/hat/buffer/Atomic.java
- hat/hat/src/main/java/hat/buffer/BoundBy.java
! hat/hat/src/main/java/hat/buffer/Buffer.java
! hat/hat/src/main/java/hat/buffer/F32Array.java
! hat/hat/src/main/java/hat/buffer/F32Array2D.java
- hat/hat/src/main/java/hat/buffer/Length.java
- hat/hat/src/main/java/hat/buffer/Pad.java
- hat/hat/src/main/java/hat/buffer/SelectedBy.java
! hat/hat/src/main/java/hat/callgraph/KernelCallGraph.java
! hat/hat/src/main/java/hat/ifacemapper/MappableIface.java
! hat/hat/src/main/java/hat/ifacemapper/SegmentMapper.java
! hat/hat/src/main/java/hat/optools/FieldLoadOpWrapper.java
! hat/hat/src/main/java/hat/optools/InvokeOpWrapper.java
! hat/hat/src/main/java/hat/optools/OpWrapper.java
! hat/hat/src/main/java/hat/optools/VarOpWrapper.java


From gfrost at openjdk.org  Thu Feb 13 13:58:44 2025
From: gfrost at openjdk.org (Gary Frost)
Date: Thu, 13 Feb 2025 13:58:44 GMT
Subject: [code-reflection] Integrated: Hat mark buffers
Message-ID: <od-9heX1S5QMIOAm-eIR3IsiENjO581QMKWzj3hpLLE=.86c549c5-2fe8-4684-ad1a-c62cfd9fb38a@github.com>

Ungoing changes. As we add ability to avoid unnecessary buffer copies via buffer marking. 

Moved info previously in the ArgSled (from Java to FFI code) into the buffers themselves. 

At present we still copy all in and out of kernel dispatches.

-------------

Commit messages:
 - Ungoing transition to avoiding copies via buffer marking
 - Added Buffer.Tail for accessing mark bits.
 - Removed buffer marking from sled, now the buffers themselves are marked
 - Add 256 bit marker tail to each segment buffer

Changes: https://git.openjdk.org/babylon/pull/317/files
  Webrev: https://webrevs.openjdk.org/?repo=babylon&pr=317&range=00
  Stats: 596 lines in 30 files changed: 224 ins; 282 del; 90 mod
  Patch: https://git.openjdk.org/babylon/pull/317.diff
  Fetch: git fetch https://git.openjdk.org/babylon.git pull/317/head:pull/317

PR: https://git.openjdk.org/babylon/pull/317

From gfrost at openjdk.org  Thu Feb 13 13:58:44 2025
From: gfrost at openjdk.org (Gary Frost)
Date: Thu, 13 Feb 2025 13:58:44 GMT
Subject: [code-reflection] Integrated: Hat mark buffers
In-Reply-To: <od-9heX1S5QMIOAm-eIR3IsiENjO581QMKWzj3hpLLE=.86c549c5-2fe8-4684-ad1a-c62cfd9fb38a@github.com>
References: <od-9heX1S5QMIOAm-eIR3IsiENjO581QMKWzj3hpLLE=.86c549c5-2fe8-4684-ad1a-c62cfd9fb38a@github.com>
Message-ID: <Clb8i2L3aSs1dtewmKcXe1cPa8eF5wa5Cr9lNqG_MHU=.767a6422-22c8-495d-be3b-f43eeb38c1ee@github.com>

On Thu, 13 Feb 2025 13:53:33 GMT, Gary Frost <gfrost at openjdk.org> wrote:

> Ungoing changes. As we add ability to avoid unnecessary buffer copies via buffer marking. 
> 
> Moved info previously in the ArgSled (from Java to FFI code) into the buffers themselves. 
> 
> At present we still copy all in and out of kernel dispatches.

This pull request has now been integrated.

Changeset: ca6bbc21
Author:    Gary Frost <gfrost at openjdk.org>
URL:       https://git.openjdk.org/babylon/commit/ca6bbc211795bee5d87b08a514d4f8457c0af714
Stats:     596 lines in 30 files changed: 224 ins; 282 del; 90 mod

Hat mark buffers

-------------

PR: https://git.openjdk.org/babylon/pull/317

From asotona at openjdk.org  Thu Feb 13 14:30:12 2025
From: asotona at openjdk.org (Adam Sotona)
Date: Thu, 13 Feb 2025 14:30:12 GMT
Subject: [code-reflection] RFR: OnnxRuntime and OnnxProtoBuilder
 continuation [v5]
In-Reply-To: <0UVt3G80W3Rotrsjj3HsrHZZC8BTDOlnd3ZDWtbXYKk=.45c7437b-0e3c-45ea-b52e-49bae61e2a80@github.com>
References: <0UVt3G80W3Rotrsjj3HsrHZZC8BTDOlnd3ZDWtbXYKk=.45c7437b-0e3c-45ea-b52e-49bae61e2a80@github.com>
Message-ID: <dS9TEVJq7jHXfQdGNO8EPKtDfXdTLzp1sMGIsF6NZms=.26e74ab8-7730-4287-be9e-1a1d34e45215@github.com>

> OnnxRuntime and OnnxProtoBuilder continuation

Adam Sotona has updated the pull request incrementally with two additional commits since the last revision:

 - initial implementation of optional inputs
   
 - removed debug prints

-------------

Changes:
  - all: https://git.openjdk.org/babylon/pull/316/files
  - new: https://git.openjdk.org/babylon/pull/316/files/cc9ab739..8da4431a

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=babylon&pr=316&range=04
 - incr: https://webrevs.openjdk.org/?repo=babylon&pr=316&range=03-04

  Stats: 35 lines in 5 files changed: 14 ins; 2 del; 19 mod
  Patch: https://git.openjdk.org/babylon/pull/316.diff
  Fetch: git fetch https://git.openjdk.org/babylon.git pull/316/head:pull/316

PR: https://git.openjdk.org/babylon/pull/316

From asotona at openjdk.org  Thu Feb 13 14:36:40 2025
From: asotona at openjdk.org (Adam Sotona)
Date: Thu, 13 Feb 2025 14:36:40 GMT
Subject: [code-reflection] RFR: OnnxRuntime and OnnxProtoBuilder
 continuation [v6]
In-Reply-To: <0UVt3G80W3Rotrsjj3HsrHZZC8BTDOlnd3ZDWtbXYKk=.45c7437b-0e3c-45ea-b52e-49bae61e2a80@github.com>
References: <0UVt3G80W3Rotrsjj3HsrHZZC8BTDOlnd3ZDWtbXYKk=.45c7437b-0e3c-45ea-b52e-49bae61e2a80@github.com>
Message-ID: <zSghmiPMeD0yVRJwp5_Zw6BsmLSpsSSYGWc0KEPN-SA=.e16f9a53-3289-4f38-a8d5-64b10360aaf9@github.com>

> OnnxRuntime and OnnxProtoBuilder continuation

Adam Sotona has updated the pull request incrementally with one additional commit since the last revision:

  preparation of CNNTest

-------------

Changes:
  - all: https://git.openjdk.org/babylon/pull/316/files
  - new: https://git.openjdk.org/babylon/pull/316/files/8da4431a..35dab48d

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=babylon&pr=316&range=05
 - incr: https://webrevs.openjdk.org/?repo=babylon&pr=316&range=04-05

  Stats: 7 lines in 1 file changed: 7 ins; 0 del; 0 mod
  Patch: https://git.openjdk.org/babylon/pull/316.diff
  Fetch: git fetch https://git.openjdk.org/babylon.git pull/316/head:pull/316

PR: https://git.openjdk.org/babylon/pull/316

From asotona at openjdk.org  Thu Feb 13 19:03:01 2025
From: asotona at openjdk.org (Adam Sotona)
Date: Thu, 13 Feb 2025 19:03:01 GMT
Subject: [code-reflection] RFR: OnnxRuntime and OnnxProtoBuilder
 continuation [v7]
In-Reply-To: <0UVt3G80W3Rotrsjj3HsrHZZC8BTDOlnd3ZDWtbXYKk=.45c7437b-0e3c-45ea-b52e-49bae61e2a80@github.com>
References: <0UVt3G80W3Rotrsjj3HsrHZZC8BTDOlnd3ZDWtbXYKk=.45c7437b-0e3c-45ea-b52e-49bae61e2a80@github.com>
Message-ID: <kTL0sSz4LMLY6uT3bgQ-vNr-l894zMpDxn3orQXOz0I=.4b22f18a-1504-4b8c-b76f-df3489b79e55@github.com>

> OnnxRuntime and OnnxProtoBuilder continuation

Adam Sotona has updated the pull request incrementally with one additional commit since the last revision:

  CNNTest loading weights and passign through with empty image

-------------

Changes:
  - all: https://git.openjdk.org/babylon/pull/316/files
  - new: https://git.openjdk.org/babylon/pull/316/files/35dab48d..a2345f3f

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=babylon&pr=316&range=06
 - incr: https://webrevs.openjdk.org/?repo=babylon&pr=316&range=05-06

  Stats: 28 lines in 2 files changed: 16 ins; 2 del; 10 mod
  Patch: https://git.openjdk.org/babylon/pull/316.diff
  Fetch: git fetch https://git.openjdk.org/babylon.git pull/316/head:pull/316

PR: https://git.openjdk.org/babylon/pull/316

From asotona at openjdk.org  Fri Feb 14 00:08:38 2025
From: asotona at openjdk.org (Adam Sotona)
Date: Fri, 14 Feb 2025 00:08:38 GMT
Subject: [code-reflection] RFR: OnnxRuntime and OnnxProtoBuilder
 continuation [v8]
In-Reply-To: <0UVt3G80W3Rotrsjj3HsrHZZC8BTDOlnd3ZDWtbXYKk=.45c7437b-0e3c-45ea-b52e-49bae61e2a80@github.com>
References: <0UVt3G80W3Rotrsjj3HsrHZZC8BTDOlnd3ZDWtbXYKk=.45c7437b-0e3c-45ea-b52e-49bae61e2a80@github.com>
Message-ID: <PeqlOiA6j-lL6MR9RP2FwDUWLJeZc2mx-IaGUddRlbQ=.5bb35afa-0f91-4fac-81e5-48052d1cd2e7@github.com>

> OnnxRuntime and OnnxProtoBuilder continuation

Adam Sotona has updated the pull request incrementally with one additional commit since the last revision:

  OnnxRuntime cleanup

-------------

Changes:
  - all: https://git.openjdk.org/babylon/pull/316/files
  - new: https://git.openjdk.org/babylon/pull/316/files/a2345f3f..1f8a417b

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=babylon&pr=316&range=07
 - incr: https://webrevs.openjdk.org/?repo=babylon&pr=316&range=06-07

  Stats: 226 lines in 6 files changed: 8 ins; 147 del; 71 mod
  Patch: https://git.openjdk.org/babylon/pull/316.diff
  Fetch: git fetch https://git.openjdk.org/babylon.git pull/316/head:pull/316

PR: https://git.openjdk.org/babylon/pull/316

From duke at openjdk.org  Fri Feb 14 02:36:24 2025
From: duke at openjdk.org (duke)
Date: Fri, 14 Feb 2025 02:36:24 GMT
Subject: [code-reflection] Withdrawn: Support for Generalized Buffer Type in
 Hat SPIR-V Backend
In-Reply-To: <iciWNN7LHxLpsttDq-x-pqDfWHCPA4Xw0_qvz5DnY6w=.5c8b1360-e5d9-4b75-8093-c943b1518c06@github.com>
References: <iciWNN7LHxLpsttDq-x-pqDfWHCPA4Xw0_qvz5DnY6w=.5c8b1360-e5d9-4b75-8093-c943b1518c06@github.com>
Message-ID: <MeyObq0ualWEClKewBbkmBRyjagMav7bUVN5-BcZ6lg=.d8ee776b-cbca-44d0-a57f-3d03368dd278@github.com>

On Mon, 16 Dec 2024 22:02:23 GMT, hanklo6 <duke at openjdk.org> wrote:

> This PR introduces support for generalized buffer by creating SPIRV type structure and arrays in the generated module. Most of the implementation references the OpenCL Hat builder. The SPIR-V module assumes the same memory layout as the java buffer object, provided there is not additional customized padding between fields. Also, generated dependent functions from callgraph instead of using code model to resolve the issue where private functions can't be accessed.
> 
> The blackscholes, mandel, and violajones examples now run successfully. However, the violajones example still gets wrong value when accessing `threshold` from `stage` inside `isAFaceStage` function, so I currently update the condition to `sumOfThisStage > threshold`.
> 
> For machines that don't have `float64` capability, enable `FP64` emulation by setting the environment variable: `OverrideDefaultFP64Settings=1` and `IGC_EnableDPEmulation=1` before running. For more details: https://github.com/intel/compute-runtime/blob/master/opencl/doc/FAQ.md#feature-double-precision-emulation-fp64

This pull request has been closed without being integrated.

-------------

PR: https://git.openjdk.org/babylon/pull/298

From asotona at openjdk.org  Fri Feb 14 11:51:05 2025
From: asotona at openjdk.org (Adam Sotona)
Date: Fri, 14 Feb 2025 11:51:05 GMT
Subject: [code-reflection] RFR: OnnxRuntime and OnnxProtoBuilder
 continuation [v9]
In-Reply-To: <0UVt3G80W3Rotrsjj3HsrHZZC8BTDOlnd3ZDWtbXYKk=.45c7437b-0e3c-45ea-b52e-49bae61e2a80@github.com>
References: <0UVt3G80W3Rotrsjj3HsrHZZC8BTDOlnd3ZDWtbXYKk=.45c7437b-0e3c-45ea-b52e-49bae61e2a80@github.com>
Message-ID: <4bToeIdEOjXxQGaccpRqikUC0iHsZAgNCfV2zcqvN8U=.baa21d61-ae96-4317-994c-43beffea8ff7@github.com>

> OnnxRuntime and OnnxProtoBuilder continuation

Adam Sotona has updated the pull request incrementally with one additional commit since the last revision:

  implemented CNNTest

-------------

Changes:
  - all: https://git.openjdk.org/babylon/pull/316/files
  - new: https://git.openjdk.org/babylon/pull/316/files/1f8a417b..94d6a121

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=babylon&pr=316&range=08
 - incr: https://webrevs.openjdk.org/?repo=babylon&pr=316&range=07-08

  Stats: 140 lines in 6 files changed: 111 ins; 26 del; 3 mod
  Patch: https://git.openjdk.org/babylon/pull/316.diff
  Fetch: git fetch https://git.openjdk.org/babylon.git pull/316/head:pull/316

PR: https://git.openjdk.org/babylon/pull/316

From gfrost at openjdk.org  Fri Feb 14 15:37:35 2025
From: gfrost at openjdk.org (Gary Frost)
Date: Fri, 14 Feb 2025 15:37:35 GMT
Subject: [code-reflection] RFR: Hat mark buffers 1
Message-ID: <_rL_aBfVoB5IxpniphOT0FX2DKJqvcanpdxU_oFAQK8=.0dbc5ef1-1829-4c76-b995-14da051e4277@github.com>

Ongoing work for moving towards buffer marking to reduce unnecessary copies. 

Here mostly infrastructural 

We moved buffer status data from the ArgSled to a trailing BufferState section at the end of each Buffer. 
The native buffers now are kept in this buffer state section along with marker bits (host dirty, gpu dirty) 
OpenCL + Mock + Spirv native backends now have event queues separated from the Kernel (in prep for the buffer to outlive the kernel dispatch)
Changes tested on OpenCL (and Mock). 
CUDA Backends very likely broken.  I will come back to those. 
HIP also.  I need to work with AMD to sync the OpenCL changes to HIP backend. 

Buffers are still copied in and out on each dispatch.

-------------

Commit messages:
 - OpenCL Mock and Spirv implementations now have isolated queues.
 - We need to move Queue out of Program because buffers now are allocated a the backend level
 - Cleaned up MemorySegment State (java side) using a MemoryLayout.
 - Move vendorPtr from ArgSled to tail.
 - Allow annotations for RW,RO and WO
 - Added Buffer.Tail for accessing mark bits.

Changes: https://git.openjdk.org/babylon/pull/318/files
  Webrev: https://webrevs.openjdk.org/?repo=babylon&pr=318&range=00
  Stats: 503 lines in 14 files changed: 216 ins; 186 del; 101 mod
  Patch: https://git.openjdk.org/babylon/pull/318.diff
  Fetch: git fetch https://git.openjdk.org/babylon.git pull/318/head:pull/318

PR: https://git.openjdk.org/babylon/pull/318

From gfrost at openjdk.org  Fri Feb 14 15:37:35 2025
From: gfrost at openjdk.org (Gary Frost)
Date: Fri, 14 Feb 2025 15:37:35 GMT
Subject: [code-reflection] RFR: Hat mark buffers 1
In-Reply-To: <_rL_aBfVoB5IxpniphOT0FX2DKJqvcanpdxU_oFAQK8=.0dbc5ef1-1829-4c76-b995-14da051e4277@github.com>
References: <_rL_aBfVoB5IxpniphOT0FX2DKJqvcanpdxU_oFAQK8=.0dbc5ef1-1829-4c76-b995-14da051e4277@github.com>
Message-ID: <fR7kMbMWpycaZnYtHZXCLAkh186dZ6OIzmG9eK158R0=.d3013dd7-14e0-4c24-88d9-1bfd0a949bb4@github.com>

On Fri, 14 Feb 2025 15:33:15 GMT, Gary Frost <gfrost at openjdk.org> wrote:

> Ongoing work for moving towards buffer marking to reduce unnecessary copies. 
> 
> Here mostly infrastructural 
> 
> We moved buffer status data from the ArgSled to a trailing BufferState section at the end of each Buffer. 
> The native buffers now are kept in this buffer state section along with marker bits (host dirty, gpu dirty) 
> OpenCL + Mock + Spirv native backends now have event queues separated from the Kernel (in prep for the buffer to outlive the kernel dispatch)
> Changes tested on OpenCL (and Mock). 
> CUDA Backends very likely broken.  I will come back to those. 
> HIP also.  I need to work with AMD to sync the OpenCL changes to HIP backend. 
> 
> Buffers are still copied in and out on each dispatch.

/Integrate

-------------

PR Comment: https://git.openjdk.org/babylon/pull/318#issuecomment-2659641975

From gfrost at openjdk.org  Fri Feb 14 15:39:54 2025
From: gfrost at openjdk.org (Gary Frost)
Date: Fri, 14 Feb 2025 15:39:54 GMT
Subject: git: openjdk/babylon: code-reflection: Hat mark buffers 1
Message-ID: <6af514c2-a520-415c-9722-0a87f7aa418f@openjdk.org>

Changeset: 14b3744a
Branch: code-reflection
Author:    Gary Frost <gfrost at openjdk.org>
Date:      2025-02-14 15:38:48 +0000
URL:       https://git.openjdk.org/babylon/commit/14b3744ae151968fef69688f444c6e0fd47542d4

Hat mark buffers 1

! hat/backends/ffi/mock/cpp/mock_backend.cpp
! hat/backends/ffi/opencl/cpp/opencl_backend.cpp
! hat/backends/ffi/opencl/include/opencl_backend.h
! hat/backends/ffi/shared/cpp/shared.cpp
! hat/backends/ffi/shared/include/shared.h
! hat/backends/ffi/spirv/cpp/spirv_backend.cpp
! hat/examples/violajones/src/main/java/violajones/ifaces/Cascade.java
! hat/hat/src/main/java/hat/ComputeContext.java
! hat/hat/src/main/java/hat/buffer/ArgArray.java
! hat/hat/src/main/java/hat/buffer/Buffer.java
! hat/hat/src/main/java/hat/callgraph/CallGraph.java
! hat/hat/src/main/java/hat/callgraph/KernelCallGraph.java
! hat/hat/src/main/java/hat/ifacemapper/SegmentMapper.java
! hat/intellij/.idea/compiler.xml


From gfrost at openjdk.org  Fri Feb 14 15:41:20 2025
From: gfrost at openjdk.org (Gary Frost)
Date: Fri, 14 Feb 2025 15:41:20 GMT
Subject: [code-reflection] Integrated: Hat mark buffers 1
In-Reply-To: <_rL_aBfVoB5IxpniphOT0FX2DKJqvcanpdxU_oFAQK8=.0dbc5ef1-1829-4c76-b995-14da051e4277@github.com>
References: <_rL_aBfVoB5IxpniphOT0FX2DKJqvcanpdxU_oFAQK8=.0dbc5ef1-1829-4c76-b995-14da051e4277@github.com>
Message-ID: <8vGVf6Lrhk2BSoD4HrmH5gENO3KT_UCNfZz1B4inVTQ=.1cac612c-ac20-401f-9317-73ab3fe5f25d@github.com>

On Fri, 14 Feb 2025 15:33:15 GMT, Gary Frost <gfrost at openjdk.org> wrote:

> Ongoing work for moving towards buffer marking to reduce unnecessary copies. 
> 
> Here mostly infrastructural 
> 
> We moved buffer status data from the ArgSled to a trailing BufferState section at the end of each Buffer. 
> The native buffers now are kept in this buffer state section along with marker bits (host dirty, gpu dirty) 
> OpenCL + Mock + Spirv native backends now have event queues separated from the Kernel (in prep for the buffer to outlive the kernel dispatch)
> Changes tested on OpenCL (and Mock). 
> CUDA Backends very likely broken.  I will come back to those. 
> HIP also.  I need to work with AMD to sync the OpenCL changes to HIP backend. 
> 
> Buffers are still copied in and out on each dispatch.

This pull request has now been integrated.

Changeset: 14b3744a
Author:    Gary Frost <gfrost at openjdk.org>
URL:       https://git.openjdk.org/babylon/commit/14b3744ae151968fef69688f444c6e0fd47542d4
Stats:     503 lines in 14 files changed: 216 ins; 186 del; 101 mod

Hat mark buffers 1

-------------

PR: https://git.openjdk.org/babylon/pull/318

From asotona at openjdk.org  Fri Feb 14 18:23:05 2025
From: asotona at openjdk.org (Adam Sotona)
Date: Fri, 14 Feb 2025 18:23:05 GMT
Subject: [code-reflection] RFR: OnnxRuntime and OnnxProtoBuilder
 continuation [v10]
In-Reply-To: <0UVt3G80W3Rotrsjj3HsrHZZC8BTDOlnd3ZDWtbXYKk=.45c7437b-0e3c-45ea-b52e-49bae61e2a80@github.com>
References: <0UVt3G80W3Rotrsjj3HsrHZZC8BTDOlnd3ZDWtbXYKk=.45c7437b-0e3c-45ea-b52e-49bae61e2a80@github.com>
Message-ID: <zSYYM80OUbQhwIDBllJ0An6pGXPNlhF7NgkdEek9e9I=.487aebbe-41e8-4812-a630-febfa1071eb4@github.com>

> OnnxRuntime and OnnxProtoBuilder continuation

Adam Sotona has updated the pull request incrementally with one additional commit since the last revision:

  Implement simple Swing UI frontend to CNNTest

-------------

Changes:
  - all: https://git.openjdk.org/babylon/pull/316/files
  - new: https://git.openjdk.org/babylon/pull/316/files/94d6a121..f5c8f78f

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=babylon&pr=316&range=09
 - incr: https://webrevs.openjdk.org/?repo=babylon&pr=316&range=08-09

  Stats: 72 lines in 1 file changed: 64 ins; 0 del; 8 mod
  Patch: https://git.openjdk.org/babylon/pull/316.diff
  Fetch: git fetch https://git.openjdk.org/babylon.git pull/316/head:pull/316

PR: https://git.openjdk.org/babylon/pull/316

From asotona at openjdk.org  Sat Feb 15 20:20:32 2025
From: asotona at openjdk.org (Adam Sotona)
Date: Sat, 15 Feb 2025 20:20:32 GMT
Subject: [code-reflection] RFR: OnnxRuntime and OnnxProtoBuilder
 continuation [v11]
In-Reply-To: <0UVt3G80W3Rotrsjj3HsrHZZC8BTDOlnd3ZDWtbXYKk=.45c7437b-0e3c-45ea-b52e-49bae61e2a80@github.com>
References: <0UVt3G80W3Rotrsjj3HsrHZZC8BTDOlnd3ZDWtbXYKk=.45c7437b-0e3c-45ea-b52e-49bae61e2a80@github.com>
Message-ID: <OgkPCBj2V_hUcXMkn9408A8SsHGL6hKN0jR96hDlrOg=.5f3b82f1-c2b6-42c4-b0a4-0739f93e08e5@github.com>

> OnnxRuntime and OnnxProtoBuilder continuation

Adam Sotona has updated the pull request incrementally with three additional commits since the last revision:

 - Standalone DigitRecognizer implementation
   
 - Tensor and CNNTest cleanup
   
 - Tensor cleanup

-------------

Changes:
  - all: https://git.openjdk.org/babylon/pull/316/files
  - new: https://git.openjdk.org/babylon/pull/316/files/f5c8f78f..1da306b9

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=babylon&pr=316&range=10
 - incr: https://webrevs.openjdk.org/?repo=babylon&pr=316&range=09-10

  Stats: 330 lines in 17 files changed: 193 ins; 99 del; 38 mod
  Patch: https://git.openjdk.org/babylon/pull/316.diff
  Fetch: git fetch https://git.openjdk.org/babylon.git pull/316/head:pull/316

PR: https://git.openjdk.org/babylon/pull/316

From asotona at openjdk.org  Sat Feb 15 20:26:48 2025
From: asotona at openjdk.org (Adam Sotona)
Date: Sat, 15 Feb 2025 20:26:48 GMT
Subject: [code-reflection] RFR: OnnxRuntime and OnnxProtoBuilder
 continuation [v12]
In-Reply-To: <0UVt3G80W3Rotrsjj3HsrHZZC8BTDOlnd3ZDWtbXYKk=.45c7437b-0e3c-45ea-b52e-49bae61e2a80@github.com>
References: <0UVt3G80W3Rotrsjj3HsrHZZC8BTDOlnd3ZDWtbXYKk=.45c7437b-0e3c-45ea-b52e-49bae61e2a80@github.com>
Message-ID: <sqmLbjsVYJCAx_Be_-3DsgZGpQrkVy0VILDQOADLRP8=.84468bf2-47b5-4702-913c-8b28d9cbb475@github.com>

> OnnxRuntime and OnnxProtoBuilder continuation

Adam Sotona has updated the pull request incrementally with one additional commit since the last revision:

  obsolete import

-------------

Changes:
  - all: https://git.openjdk.org/babylon/pull/316/files
  - new: https://git.openjdk.org/babylon/pull/316/files/1da306b9..91ee7b24

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=babylon&pr=316&range=11
 - incr: https://webrevs.openjdk.org/?repo=babylon&pr=316&range=10-11

  Stats: 1 line in 1 file changed: 0 ins; 1 del; 0 mod
  Patch: https://git.openjdk.org/babylon/pull/316.diff
  Fetch: git fetch https://git.openjdk.org/babylon.git pull/316/head:pull/316

PR: https://git.openjdk.org/babylon/pull/316

From asotona at openjdk.org  Sat Feb 15 20:34:17 2025
From: asotona at openjdk.org (Adam Sotona)
Date: Sat, 15 Feb 2025 20:34:17 GMT
Subject: [code-reflection] RFR: OnnxRuntime and OnnxProtoBuilder
 continuation [v13]
In-Reply-To: <0UVt3G80W3Rotrsjj3HsrHZZC8BTDOlnd3ZDWtbXYKk=.45c7437b-0e3c-45ea-b52e-49bae61e2a80@github.com>
References: <0UVt3G80W3Rotrsjj3HsrHZZC8BTDOlnd3ZDWtbXYKk=.45c7437b-0e3c-45ea-b52e-49bae61e2a80@github.com>
Message-ID: <yxKVOiY2lkK-6iW7g-vbYW_V5jt6dM8jMmvFULDZFjo=.8d5c038e-44e5-4c0c-8fb9-889c2d7ae9be@github.com>

> OnnxRuntime and OnnxProtoBuilder continuation

Adam Sotona has updated the pull request incrementally with one additional commit since the last revision:

  updated .gitattributes

-------------

Changes:
  - all: https://git.openjdk.org/babylon/pull/316/files
  - new: https://git.openjdk.org/babylon/pull/316/files/91ee7b24..e9eac961

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=babylon&pr=316&range=12
 - incr: https://webrevs.openjdk.org/?repo=babylon&pr=316&range=11-12

  Stats: 3 lines in 1 file changed: 3 ins; 0 del; 0 mod
  Patch: https://git.openjdk.org/babylon/pull/316.diff
  Fetch: git fetch https://git.openjdk.org/babylon.git pull/316/head:pull/316

PR: https://git.openjdk.org/babylon/pull/316

From asotona at openjdk.org  Sat Feb 15 20:39:00 2025
From: asotona at openjdk.org (Adam Sotona)
Date: Sat, 15 Feb 2025 20:39:00 GMT
Subject: [code-reflection] RFR: OnnxRuntime and OnnxProtoBuilder
 continuation [v14]
In-Reply-To: <0UVt3G80W3Rotrsjj3HsrHZZC8BTDOlnd3ZDWtbXYKk=.45c7437b-0e3c-45ea-b52e-49bae61e2a80@github.com>
References: <0UVt3G80W3Rotrsjj3HsrHZZC8BTDOlnd3ZDWtbXYKk=.45c7437b-0e3c-45ea-b52e-49bae61e2a80@github.com>
Message-ID: <iLEug_Zrr4AL0uOstBfJXJ0QozoigBOXU-pr9WiZJkA=.3fceac51-9f60-4050-bde7-7996b2cdcbf9@github.com>

> OnnxRuntime and OnnxProtoBuilder continuation

Adam Sotona has updated the pull request incrementally with one additional commit since the last revision:

  Revert "updated .gitattributes"
  
  This reverts commit e9eac961661abf0ce5e35eb6ff182f77d05e6640.

-------------

Changes:
  - all: https://git.openjdk.org/babylon/pull/316/files
  - new: https://git.openjdk.org/babylon/pull/316/files/e9eac961..fbe0485e

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=babylon&pr=316&range=13
 - incr: https://webrevs.openjdk.org/?repo=babylon&pr=316&range=12-13

  Stats: 3 lines in 1 file changed: 0 ins; 3 del; 0 mod
  Patch: https://git.openjdk.org/babylon/pull/316.diff
  Fetch: git fetch https://git.openjdk.org/babylon.git pull/316/head:pull/316

PR: https://git.openjdk.org/babylon/pull/316

From asotona at openjdk.org  Mon Feb 17 08:58:04 2025
From: asotona at openjdk.org (Adam Sotona)
Date: Mon, 17 Feb 2025 08:58:04 GMT
Subject: [code-reflection] RFR: OnnxRuntime and OnnxProtoBuilder
 continuation [v15]
In-Reply-To: <0UVt3G80W3Rotrsjj3HsrHZZC8BTDOlnd3ZDWtbXYKk=.45c7437b-0e3c-45ea-b52e-49bae61e2a80@github.com>
References: <0UVt3G80W3Rotrsjj3HsrHZZC8BTDOlnd3ZDWtbXYKk=.45c7437b-0e3c-45ea-b52e-49bae61e2a80@github.com>
Message-ID: <e9gGJu1F35306gUWzipn73BCYMEoqll-8n-cydxlvH4=.f349091f-d21d-4c4f-b75d-218f9f58894f@github.com>

> OnnxRuntime and OnnxProtoBuilder continuation

Adam Sotona has updated the pull request incrementally with one additional commit since the last revision:

  Demo work in progress

-------------

Changes:
  - all: https://git.openjdk.org/babylon/pull/316/files
  - new: https://git.openjdk.org/babylon/pull/316/files/fbe0485e..9c8c9c75

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=babylon&pr=316&range=14
 - incr: https://webrevs.openjdk.org/?repo=babylon&pr=316&range=13-14

  Stats: 362 lines in 3 files changed: 187 ins; 175 del; 0 mod
  Patch: https://git.openjdk.org/babylon/pull/316.diff
  Fetch: git fetch https://git.openjdk.org/babylon.git pull/316/head:pull/316

PR: https://git.openjdk.org/babylon/pull/316

From asotona at openjdk.org  Mon Feb 17 09:01:48 2025
From: asotona at openjdk.org (Adam Sotona)
Date: Mon, 17 Feb 2025 09:01:48 GMT
Subject: git: openjdk/babylon: code-reflection: OnnxRuntime and
 OnnxProtoBuilder continuation
Message-ID: <d91d15e2-4e94-4746-ad5f-5dc889323b20@openjdk.org>

Changeset: 8b6b07fb
Branch: code-reflection
Author:    Adam Sotona <asotona at openjdk.org>
Date:      2025-02-17 09:00:25 +0000
URL:       https://git.openjdk.org/babylon/commit/8b6b07fbe5bd5874ec4a6ab58026bc697924b75e

OnnxRuntime and OnnxProtoBuilder continuation

+ cr-examples/onnx/README.md
! cr-examples/onnx/src/main/java/oracle/code/onnx/OnnxInterpreter.java
! cr-examples/onnx/src/main/java/oracle/code/onnx/OnnxProtoBuilder.java
! cr-examples/onnx/src/main/java/oracle/code/onnx/OnnxRuntime.java
! cr-examples/onnx/src/main/java/oracle/code/onnx/Tensor.java
! cr-examples/onnx/src/main/java/oracle/code/onnx/compiler/OnnxTransformer.java
! cr-examples/onnx/src/main/java/oracle/code/onnx/ir/OnnxOp.java
! cr-examples/onnx/src/main/java/oracle/code/onnx/ir/OnnxType.java
! cr-examples/onnx/src/test/java/oracle/code/onnx/CNNTest.java
+ cr-examples/onnx/src/test/java/oracle/code/onnx/MNISTDemo.java
! cr-examples/onnx/src/test/java/oracle/code/onnx/RuntimeTest.java
! cr-examples/onnx/src/test/java/oracle/code/onnx/SimpleTest.java
+ cr-examples/onnx/src/test/resources/oracle/code/onnx/conv1-bias-float
+ cr-examples/onnx/src/test/resources/oracle/code/onnx/conv1-weight-float
+ cr-examples/onnx/src/test/resources/oracle/code/onnx/conv2-bias-float
+ cr-examples/onnx/src/test/resources/oracle/code/onnx/conv2-weight-float
+ cr-examples/onnx/src/test/resources/oracle/code/onnx/fc1-bias-float
+ cr-examples/onnx/src/test/resources/oracle/code/onnx/fc1-weight-float
+ cr-examples/onnx/src/test/resources/oracle/code/onnx/fc2-bias-float
+ cr-examples/onnx/src/test/resources/oracle/code/onnx/fc2-weight-float
+ cr-examples/onnx/src/test/resources/oracle/code/onnx/fc3-bias-float
+ cr-examples/onnx/src/test/resources/oracle/code/onnx/fc3-weight-float
+ cr-examples/onnx/src/test/resources/oracle/code/onnx/images-ubyte
+ cr-examples/onnx/src/test/resources/oracle/code/onnx/labels-ubyte


From asotona at openjdk.org  Mon Feb 17 09:03:31 2025
From: asotona at openjdk.org (Adam Sotona)
Date: Mon, 17 Feb 2025 09:03:31 GMT
Subject: [code-reflection] Integrated: OnnxRuntime and OnnxProtoBuilder
 continuation
In-Reply-To: <0UVt3G80W3Rotrsjj3HsrHZZC8BTDOlnd3ZDWtbXYKk=.45c7437b-0e3c-45ea-b52e-49bae61e2a80@github.com>
References: <0UVt3G80W3Rotrsjj3HsrHZZC8BTDOlnd3ZDWtbXYKk=.45c7437b-0e3c-45ea-b52e-49bae61e2a80@github.com>
Message-ID: <S-h44CLUi1zII0H7Ma7Q-vQcSRM02Q3O7YtZ2PcT1-w=.4375af51-1453-4dbb-8f68-52a1206b763b@github.com>

On Wed, 12 Feb 2025 09:46:40 GMT, Adam Sotona <asotona at openjdk.org> wrote:

> OnnxRuntime and OnnxProtoBuilder continuation

This pull request has now been integrated.

Changeset: 8b6b07fb
Author:    Adam Sotona <asotona at openjdk.org>
URL:       https://git.openjdk.org/babylon/commit/8b6b07fbe5bd5874ec4a6ab58026bc697924b75e
Stats:     985 lines in 24 files changed: 611 ins; 264 del; 110 mod

OnnxRuntime and OnnxProtoBuilder continuation

-------------

PR: https://git.openjdk.org/babylon/pull/316

From asotona at openjdk.org  Mon Feb 17 09:32:03 2025
From: asotona at openjdk.org (Adam Sotona)
Date: Mon, 17 Feb 2025 09:32:03 GMT
Subject: [code-reflection] RFR: MNISTDemo fix
Message-ID: <iDRYcP66T1jfRwbGo9a8bViBMd80Gsqr86X3RtPiqTI=.651c10f0-5f55-47d6-8b51-6d3b4a23d99f@github.com>

MNISTDemo fix of the result numbers shown for the non-dominant digits

-------------

Commit messages:
 - MNISTDemo fix

Changes: https://git.openjdk.org/babylon/pull/319/files
  Webrev: https://webrevs.openjdk.org/?repo=babylon&pr=319&range=00
  Stats: 5 lines in 1 file changed: 0 ins; 0 del; 5 mod
  Patch: https://git.openjdk.org/babylon/pull/319.diff
  Fetch: git fetch https://git.openjdk.org/babylon.git pull/319/head:pull/319

PR: https://git.openjdk.org/babylon/pull/319

From asotona at openjdk.org  Mon Feb 17 09:36:40 2025
From: asotona at openjdk.org (Adam Sotona)
Date: Mon, 17 Feb 2025 09:36:40 GMT
Subject: git: openjdk/babylon: code-reflection: MNISTDemo fix
Message-ID: <cb0d3c1c-920a-431a-b775-dcb06b3e858e@openjdk.org>

Changeset: 6523c4fd
Branch: code-reflection
Author:    Adam Sotona <asotona at openjdk.org>
Date:      2025-02-17 09:35:41 +0000
URL:       https://git.openjdk.org/babylon/commit/6523c4fd137b9b3cde1f6ae4ff612d7e22e6ea73

MNISTDemo fix

! cr-examples/onnx/src/test/java/oracle/code/onnx/MNISTDemo.java


From asotona at openjdk.org  Mon Feb 17 09:38:35 2025
From: asotona at openjdk.org (Adam Sotona)
Date: Mon, 17 Feb 2025 09:38:35 GMT
Subject: [code-reflection] Integrated: MNISTDemo fix
In-Reply-To: <iDRYcP66T1jfRwbGo9a8bViBMd80Gsqr86X3RtPiqTI=.651c10f0-5f55-47d6-8b51-6d3b4a23d99f@github.com>
References: <iDRYcP66T1jfRwbGo9a8bViBMd80Gsqr86X3RtPiqTI=.651c10f0-5f55-47d6-8b51-6d3b4a23d99f@github.com>
Message-ID: <VNai9oM3y0HMulzMsUmoR0CE37Ehq0Wr7lk6uLc3xAo=.2de52725-023c-4caa-8c00-d058803603db@github.com>

On Mon, 17 Feb 2025 09:27:35 GMT, Adam Sotona <asotona at openjdk.org> wrote:

> MNISTDemo fix of the result numbers shown for the non-dominant digits

This pull request has now been integrated.

Changeset: 6523c4fd
Author:    Adam Sotona <asotona at openjdk.org>
URL:       https://git.openjdk.org/babylon/commit/6523c4fd137b9b3cde1f6ae4ff612d7e22e6ea73
Stats:     9 lines in 1 file changed: 0 ins; 3 del; 6 mod

MNISTDemo fix

-------------

PR: https://git.openjdk.org/babylon/pull/319

From asotona at openjdk.org  Mon Feb 17 09:38:35 2025
From: asotona at openjdk.org (Adam Sotona)
Date: Mon, 17 Feb 2025 09:38:35 GMT
Subject: [code-reflection] RFR: MNISTDemo fix [v2]
In-Reply-To: <iDRYcP66T1jfRwbGo9a8bViBMd80Gsqr86X3RtPiqTI=.651c10f0-5f55-47d6-8b51-6d3b4a23d99f@github.com>
References: <iDRYcP66T1jfRwbGo9a8bViBMd80Gsqr86X3RtPiqTI=.651c10f0-5f55-47d6-8b51-6d3b4a23d99f@github.com>
Message-ID: <QWCkNM2NqrhsO0ExA6ZldnWIi6JfDof4OBZX2NHkK6I=.4bb90c81-7aa0-4cc8-80ca-1f0ea8d96f8e@github.com>

> MNISTDemo fix of the result numbers shown for the non-dominant digits

Adam Sotona has updated the pull request incrementally with one additional commit since the last revision:

  MNISTDemo fix

-------------

Changes:
  - all: https://git.openjdk.org/babylon/pull/319/files
  - new: https://git.openjdk.org/babylon/pull/319/files/2b8aa420..11a4e1f3

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=babylon&pr=319&range=01
 - incr: https://webrevs.openjdk.org/?repo=babylon&pr=319&range=00-01

  Stats: 6 lines in 1 file changed: 0 ins; 3 del; 3 mod
  Patch: https://git.openjdk.org/babylon/pull/319.diff
  Fetch: git fetch https://git.openjdk.org/babylon.git pull/319/head:pull/319

PR: https://git.openjdk.org/babylon/pull/319

From gfrost at openjdk.org  Mon Feb 17 19:22:09 2025
From: gfrost at openjdk.org (Gary Frost)
Date: Mon, 17 Feb 2025 19:22:09 GMT
Subject: git: openjdk/babylon: code-reflection: Hat ongoing buffer marking
Message-ID: <73eb530d-a1dd-4eb5-b9db-ce1785a9fe8c@openjdk.org>

Changeset: 99339839
Branch: code-reflection
Author:    Gary Frost <gfrost at openjdk.org>
Date:      2025-02-17 19:20:10 +0000
URL:       https://git.openjdk.org/babylon/commit/993398392c121aaf278d6d035ebf86532381d6d3

Hat ongoing buffer marking

! hat/backends/ffi/cuda/cpp/info.cpp
! hat/backends/ffi/cuda/include/cuda_backend.h
! hat/backends/ffi/cuda/src/main/java/hat/backend/ffi/CudaBackend.java
! hat/backends/ffi/mock/cpp/mock_backend.cpp
! hat/backends/ffi/mock/src/main/java/hat/backend/ffi/MockBackend.java
! hat/backends/ffi/opencl/cpp/info.cpp
! hat/backends/ffi/opencl/cpp/opencl_backend.cpp
! hat/backends/ffi/opencl/include/opencl_backend.h
! hat/backends/ffi/opencl/src/main/java/hat/backend/ffi/OpenCLBackend.java
! hat/backends/ffi/ptx/cpp/ptx_backend.cpp
! hat/backends/ffi/ptx/include/ptx_backend.h
! hat/backends/ffi/ptx/src/main/java/hat/backend/ffi/PTXBackend.java
! hat/backends/ffi/shared/cpp/shared.cpp
! hat/backends/ffi/shared/include/shared.h
! hat/backends/ffi/spirv/cpp/spirv_backend.cpp
! hat/backends/jextracted/opencl/src/main/java/hat/backend/jextracted/OpenCLBackend.java
! hat/docs/hat-06-kernel-analysis.md
! hat/docs/hat-notes-and-links.md
! hat/examples/mandel/src/main/java/mandel/Main.java
! hat/hat/src/main/java/hat/Accelerator.java
! hat/hat/src/main/java/hat/ComputeContext.java
! hat/hat/src/main/java/hat/backend/ffi/C99FFIBackend.java
! hat/hat/src/main/java/hat/backend/ffi/FFIBackend.java
! hat/hat/src/main/java/hat/backend/ffi/FFIBackendDriver.java
! hat/hat/src/main/java/hat/backend/ffi/FFILib.java
! hat/hat/src/main/java/hat/backend/jextracted/JExtractedBackendDriver.java
- hat/hat/src/main/java/hat/buffer/BackendConfig.java
! hat/hat/src/main/java/hat/buffer/Buffer.java
+ hat/hat/src/main/java/hat/buffer/BufferTracker.java
! hat/hat/src/main/java/hat/ifacemapper/SegmentMapper.java


From gfrost at openjdk.org  Mon Feb 17 19:22:39 2025
From: gfrost at openjdk.org (Gary Frost)
Date: Mon, 17 Feb 2025 19:22:39 GMT
Subject: [code-reflection] Integrated: Hat ongoing buffer marking
Message-ID: <DlUKBpyq0WyXP4fqpR7wbr1NrJbYehiq3TCRL1OjLp0=.bd4237da-5165-4446-9e7a-6201d960c2f3@github.com>

Lots of changes here.  Specifically moving native backend data structures around. 

We now cache native side buffer info (say clmem for OpenCL) in the vendor ptr of the memory segment 'state' 

This has a substantial perf boost with OpenCL.  Which was suprising (mandel x1.5, gol also x1.5) 

Still defaulting to copying in and out for each kernel dispatch. 

Close to turning on min buffer options.

-------------

Commit messages:
 - clmem in vendorPtr.
 - Add profiling data
 - Gated fetch of dirty buffer from opencl ffi layer
 - Removed Queue from all but OpenCL
 - We'll use deviceDirty instead of gpuDirty
 - FFI:getBuffer now getBufferFromDeviceIfDirty()
 - Removed mode from BufferState.  Mode now conveyed via Backend Config
 - Removed mode from BufferState.  Mode now conveyed via Backend Config
 - We pass mode, platform and device selectors to backends.
 - We pass mode, platform and device selectors to backends.
 - ... and 6 more: https://git.openjdk.org/babylon/compare/14b3744a...e5e6ca56

Changes: https://git.openjdk.org/babylon/pull/321/files
  Webrev: https://webrevs.openjdk.org/?repo=babylon&pr=321&range=00
  Stats: 1037 lines in 30 files changed: 525 ins; 350 del; 162 mod
  Patch: https://git.openjdk.org/babylon/pull/321.diff
  Fetch: git fetch https://git.openjdk.org/babylon.git pull/321/head:pull/321

PR: https://git.openjdk.org/babylon/pull/321

From gfrost at openjdk.org  Mon Feb 17 19:22:40 2025
From: gfrost at openjdk.org (Gary Frost)
Date: Mon, 17 Feb 2025 19:22:40 GMT
Subject: [code-reflection] Integrated: Hat ongoing buffer marking
In-Reply-To: <DlUKBpyq0WyXP4fqpR7wbr1NrJbYehiq3TCRL1OjLp0=.bd4237da-5165-4446-9e7a-6201d960c2f3@github.com>
References: <DlUKBpyq0WyXP4fqpR7wbr1NrJbYehiq3TCRL1OjLp0=.bd4237da-5165-4446-9e7a-6201d960c2f3@github.com>
Message-ID: <UDZgoOc57IfRCCGFOn_hTeHwEJia13qeHk1XV84_mdg=.b375ba95-88a5-4071-ab0b-ecec2a755ab7@github.com>

On Mon, 17 Feb 2025 19:17:57 GMT, Gary Frost <gfrost at openjdk.org> wrote:

> Lots of changes here.  Specifically moving native backend data structures around. 
> 
> We now cache native side buffer info (say clmem for OpenCL) in the vendor ptr of the memory segment 'state' 
> 
> This has a substantial perf boost with OpenCL.  Which was suprising (mandel x1.5, gol also x1.5) 
> 
> Still defaulting to copying in and out for each kernel dispatch. 
> 
> Close to turning on min buffer options.

This pull request has now been integrated.

Changeset: 99339839
Author:    Gary Frost <gfrost at openjdk.org>
URL:       https://git.openjdk.org/babylon/commit/993398392c121aaf278d6d035ebf86532381d6d3
Stats:     1037 lines in 30 files changed: 525 ins; 350 del; 162 mod

Hat ongoing buffer marking

-------------

PR: https://git.openjdk.org/babylon/pull/321

From asotona at openjdk.org  Tue Feb 18 09:02:25 2025
From: asotona at openjdk.org (Adam Sotona)
Date: Tue, 18 Feb 2025 09:02:25 GMT
Subject: [code-reflection] RFR: OnnxRuntime and OnnxProtoBuilder continuation
Message-ID: <YJNigMWEGOBOmbnpDjbAG6fDvZ9-AZ0KJJN47w4jRHE=.3490ef8d-0743-45ff-b538-03a66b04ba99@github.com>

OnnxRuntime and OnnxProtoBuilder continuation

-------------

Commit messages:
 - CNNTest cleanup + weight tensors stored in little-endian for direct mapping
 - OnnxProtoPrinter implemenation
 - OnnxProtoBuilder support for tuple results + test

Changes: https://git.openjdk.org/babylon/pull/320/files
  Webrev: https://webrevs.openjdk.org/?repo=babylon&pr=320&range=00
  Stats: 469 lines in 26 files changed: 330 ins; 63 del; 76 mod
  Patch: https://git.openjdk.org/babylon/pull/320.diff
  Fetch: git fetch https://git.openjdk.org/babylon.git pull/320/head:pull/320

PR: https://git.openjdk.org/babylon/pull/320

From asotona at openjdk.org  Tue Feb 18 13:37:12 2025
From: asotona at openjdk.org (Adam Sotona)
Date: Tue, 18 Feb 2025 13:37:12 GMT
Subject: git: openjdk/babylon: code-reflection: OnnxRuntime and
 OnnxProtoBuilder continuation
Message-ID: <15623f88-b44c-470e-9fbe-87e0566fd024@openjdk.org>

Changeset: 416c84bb
Branch: code-reflection
Author:    Adam Sotona <asotona at openjdk.org>
Date:      2025-02-18 13:34:11 +0000
URL:       https://git.openjdk.org/babylon/commit/416c84bb4053314584252cb6bfa9e30f882a9f75

OnnxRuntime and OnnxProtoBuilder continuation

! cr-examples/onnx/src/main/java/oracle/code/onnx/OnnxProtoBuilder.java
+ cr-examples/onnx/src/main/java/oracle/code/onnx/OnnxProtoPrinter.java
! cr-examples/onnx/src/main/java/oracle/code/onnx/Tensor.java
! cr-examples/onnx/src/test/java/oracle/code/onnx/CNNTest.java
! cr-examples/onnx/src/test/java/oracle/code/onnx/MNISTDemo.java
! cr-examples/onnx/src/test/java/oracle/code/onnx/SimpleTest.java
- cr-examples/onnx/src/test/resources/oracle/code/onnx/conv1-bias-float
+ cr-examples/onnx/src/test/resources/oracle/code/onnx/conv1-bias-float-le
- cr-examples/onnx/src/test/resources/oracle/code/onnx/conv1-weight-float
+ cr-examples/onnx/src/test/resources/oracle/code/onnx/conv1-weight-float-le
- cr-examples/onnx/src/test/resources/oracle/code/onnx/conv2-bias-float
+ cr-examples/onnx/src/test/resources/oracle/code/onnx/conv2-bias-float-le
- cr-examples/onnx/src/test/resources/oracle/code/onnx/conv2-weight-float
+ cr-examples/onnx/src/test/resources/oracle/code/onnx/conv2-weight-float-le
- cr-examples/onnx/src/test/resources/oracle/code/onnx/fc1-bias-float
+ cr-examples/onnx/src/test/resources/oracle/code/onnx/fc1-bias-float-le
- cr-examples/onnx/src/test/resources/oracle/code/onnx/fc1-weight-float
+ cr-examples/onnx/src/test/resources/oracle/code/onnx/fc1-weight-float-le
- cr-examples/onnx/src/test/resources/oracle/code/onnx/fc2-bias-float
+ cr-examples/onnx/src/test/resources/oracle/code/onnx/fc2-bias-float-le
- cr-examples/onnx/src/test/resources/oracle/code/onnx/fc2-weight-float
+ cr-examples/onnx/src/test/resources/oracle/code/onnx/fc2-weight-float-le
- cr-examples/onnx/src/test/resources/oracle/code/onnx/fc3-bias-float
+ cr-examples/onnx/src/test/resources/oracle/code/onnx/fc3-bias-float-le
- cr-examples/onnx/src/test/resources/oracle/code/onnx/fc3-weight-float
+ cr-examples/onnx/src/test/resources/oracle/code/onnx/fc3-weight-float-le


From asotona at openjdk.org  Tue Feb 18 13:38:22 2025
From: asotona at openjdk.org (Adam Sotona)
Date: Tue, 18 Feb 2025 13:38:22 GMT
Subject: [code-reflection] Integrated: OnnxRuntime and OnnxProtoBuilder
 continuation
In-Reply-To: <YJNigMWEGOBOmbnpDjbAG6fDvZ9-AZ0KJJN47w4jRHE=.3490ef8d-0743-45ff-b538-03a66b04ba99@github.com>
References: <YJNigMWEGOBOmbnpDjbAG6fDvZ9-AZ0KJJN47w4jRHE=.3490ef8d-0743-45ff-b538-03a66b04ba99@github.com>
Message-ID: <gVNBIeTHW2CshIRKjDe6-RIZ5GDxEZ4wRecwGSWljUY=.3ea98467-6ed3-4d99-88c0-5724edb5a27d@github.com>

On Mon, 17 Feb 2025 13:21:19 GMT, Adam Sotona <asotona at openjdk.org> wrote:

> OnnxRuntime and OnnxProtoBuilder continuation

This pull request has now been integrated.

Changeset: 416c84bb
Author:    Adam Sotona <asotona at openjdk.org>
URL:       https://git.openjdk.org/babylon/commit/416c84bb4053314584252cb6bfa9e30f882a9f75
Stats:     465 lines in 26 files changed: 326 ins; 59 del; 80 mod

OnnxRuntime and OnnxProtoBuilder continuation

-------------

PR: https://git.openjdk.org/babylon/pull/320

From asotona at openjdk.org  Tue Feb 18 13:38:20 2025
From: asotona at openjdk.org (Adam Sotona)
Date: Tue, 18 Feb 2025 13:38:20 GMT
Subject: [code-reflection] RFR: OnnxRuntime and OnnxProtoBuilder
 continuation [v2]
In-Reply-To: <YJNigMWEGOBOmbnpDjbAG6fDvZ9-AZ0KJJN47w4jRHE=.3490ef8d-0743-45ff-b538-03a66b04ba99@github.com>
References: <YJNigMWEGOBOmbnpDjbAG6fDvZ9-AZ0KJJN47w4jRHE=.3490ef8d-0743-45ff-b538-03a66b04ba99@github.com>
Message-ID: <KlGXjZ4xDRG5HnP8vkjciW2AQjHEo_V2oSxThWujjbs=.127f4725-9770-4bca-ac8d-be8d716eecc5@github.com>

> OnnxRuntime and OnnxProtoBuilder continuation

Adam Sotona has updated the pull request incrementally with one additional commit since the last revision:

  minor CNNTest cleanup

-------------

Changes:
  - all: https://git.openjdk.org/babylon/pull/320/files
  - new: https://git.openjdk.org/babylon/pull/320/files/ab8daf4f..d01b7dda

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=babylon&pr=320&range=01
 - incr: https://webrevs.openjdk.org/?repo=babylon&pr=320&range=00-01

  Stats: 6 lines in 2 files changed: 0 ins; 0 del; 6 mod
  Patch: https://git.openjdk.org/babylon/pull/320.diff
  Fetch: git fetch https://git.openjdk.org/babylon.git pull/320/head:pull/320

PR: https://git.openjdk.org/babylon/pull/320

From mcimadamore at openjdk.org  Tue Feb 18 19:23:56 2025
From: mcimadamore at openjdk.org (Maurizio Cimadamore)
Date: Tue, 18 Feb 2025 19:23:56 GMT
Subject: git: openjdk/babylon: code-reflection: Make downcall method handles
 for ORT API functions static
Message-ID: <2e5ce270-b50c-4e40-8d53-2529aea1e44c@openjdk.org>

Changeset: 90be8529
Branch: code-reflection
Author:    Maurizio Cimadamore <mcimadamore at openjdk.org>
Date:      2025-02-18 18:54:47 +0000
URL:       https://git.openjdk.org/babylon/commit/90be85293126f550c8672adc706ff58d0c6309d3

Make downcall method handles for ORT API functions static

Reviewed-by: asotona

! cr-examples/onnx/src/main/java/oracle/code/onnx/OnnxRuntime.java


From mcimadamore at openjdk.org  Tue Feb 18 19:28:19 2025
From: mcimadamore at openjdk.org (Maurizio Cimadamore)
Date: Tue, 18 Feb 2025 19:28:19 GMT
Subject: [code-reflection] Integrated: Make downcall method handles for ORT API
 functions static
Message-ID: <fiNMWH6j15rUXnGKJ_t0ZxqQmVudOsEpZ3dkmMh0uNw=.0dfc3d2a-e242-4bf2-b79c-b1c3b6c1f2a5@github.com>

This PR tweaks the method handles in `OnnxRuntime` to be static and final.
The trick is to use a so called "virtual" downcall handle -- a downcall handle that takes its input address as an extra parameter.
Then, we can just adapt such a method handle to fetch the target function address from the incoming `runtimeAddress` parameter.
This way, all the method handles modelling functions in the ORT API can be made static, which unlocks better C2 optimizations.

Clients of such method handles have to be tweaked to pass in the additional leading `runtimeAddress` parameter.

-------------

Commit messages:
 - Move back the `handle` helper method to minimize diffs
 - Add missing leading parameters
 - Add commments
 - Make ORT downcall handles static

Changes: https://git.openjdk.org/babylon/pull/322/files
  Webrev: https://webrevs.openjdk.org/?repo=babylon&pr=322&range=00
  Stats: 72 lines in 1 file changed: 28 ins; 15 del; 29 mod
  Patch: https://git.openjdk.org/babylon/pull/322.diff
  Fetch: git fetch https://git.openjdk.org/babylon.git pull/322/head:pull/322

PR: https://git.openjdk.org/babylon/pull/322

From asotona at openjdk.org  Tue Feb 18 19:28:20 2025
From: asotona at openjdk.org (Adam Sotona)
Date: Tue, 18 Feb 2025 19:28:20 GMT
Subject: [code-reflection] Integrated: Make downcall method handles for ORT
 API functions static
In-Reply-To: <fiNMWH6j15rUXnGKJ_t0ZxqQmVudOsEpZ3dkmMh0uNw=.0dfc3d2a-e242-4bf2-b79c-b1c3b6c1f2a5@github.com>
References: <fiNMWH6j15rUXnGKJ_t0ZxqQmVudOsEpZ3dkmMh0uNw=.0dfc3d2a-e242-4bf2-b79c-b1c3b6c1f2a5@github.com>
Message-ID: <uVV4XZelCi2S30MS8TJ8PJR5qoEylF9EzjGhYhVEndQ=.63a73a67-bb41-4f08-bbbf-d74fcbef6dc9@github.com>

On Tue, 18 Feb 2025 15:15:26 GMT, Maurizio Cimadamore <mcimadamore at openjdk.org> wrote:

> This PR tweaks the method handles in `OnnxRuntime` to be static and final.
> The trick is to use a so called "virtual" downcall handle -- a downcall handle that takes its input address as an extra parameter.
> Then, we can just adapt such a method handle to fetch the target function address from the incoming `runtimeAddress` parameter.
> This way, all the method handles modelling functions in the ORT API can be made static, which unlocks better C2 optimizations.
> 
> Clients of such method handles have to be tweaked to pass in the additional leading `runtimeAddress` parameter.

It looks good to me.

I'm wondering if there are any guides what patterns can be better optimized.
OnnxRuntime is a singleton, so the optimization-wise differences between static final and singleton instance final are not clear to me.

Thanks!

-------------

Marked as reviewed by asotona (Reviewer).

PR Review: https://git.openjdk.org/babylon/pull/322#pullrequestreview-2624082892

From mcimadamore at openjdk.org  Tue Feb 18 19:28:25 2025
From: mcimadamore at openjdk.org (Maurizio Cimadamore)
Date: Tue, 18 Feb 2025 19:28:25 GMT
Subject: [code-reflection] Integrated: Make downcall method handles for ORT
 API functions static
In-Reply-To: <uVV4XZelCi2S30MS8TJ8PJR5qoEylF9EzjGhYhVEndQ=.63a73a67-bb41-4f08-bbbf-d74fcbef6dc9@github.com>
References: <fiNMWH6j15rUXnGKJ_t0ZxqQmVudOsEpZ3dkmMh0uNw=.0dfc3d2a-e242-4bf2-b79c-b1c3b6c1f2a5@github.com>
 <uVV4XZelCi2S30MS8TJ8PJR5qoEylF9EzjGhYhVEndQ=.63a73a67-bb41-4f08-bbbf-d74fcbef6dc9@github.com>
Message-ID: <lH6HGDGZTJ9xJmTJKV_VU0AqdCU45zQ_ju49UHMnSDQ=.4b6acb81-96a9-4e81-a7ba-edd896e42905@github.com>

On Tue, 18 Feb 2025 15:50:19 GMT, Adam Sotona <asotona at openjdk.org> wrote:

> It looks good to me.
> 
> I'm wondering if there are any guides what patterns can be better optimized. OnnxRuntime is a singleton, so the optimization-wise differences between static final and singleton instance final are not clear to me.
> 
> Thanks!

Ideally, having final instance fields in a singleton should work. However, plain final instance fields are not "trusted" by the JVM. So the chain of reference (singleton -> function method handle) is not treated as a "true constant" and, as such, the downcall method handle is not inlined in the caller.

Once [stable values](https://openjdk.org/jeps/502) land, this can be fixed: the singleton can be a `static final StableValue<ONNXRuntime>` which contains many `StableValue<MethodHandle>` - one per function (or better, by using a stable list of method handles). Since stable values are... stable, the JVM can trust them, meaning the chain of references is treated as a true constant.

-------------

PR Comment: https://git.openjdk.org/babylon/pull/322#issuecomment-2666383663

From mcimadamore at openjdk.org  Tue Feb 18 19:28:27 2025
From: mcimadamore at openjdk.org (Maurizio Cimadamore)
Date: Tue, 18 Feb 2025 19:28:27 GMT
Subject: [code-reflection] Integrated: Make downcall method handles for ORT
 API functions static
In-Reply-To: <L6bfWYgtZg4lTQUE1vcUzahrbILXyKqlVu_71qZCaCw=.a1259096-51f8-4aef-a706-eb6f2c006782@github.com>
References: <fiNMWH6j15rUXnGKJ_t0ZxqQmVudOsEpZ3dkmMh0uNw=.0dfc3d2a-e242-4bf2-b79c-b1c3b6c1f2a5@github.com>
 <L6bfWYgtZg4lTQUE1vcUzahrbILXyKqlVu_71qZCaCw=.a1259096-51f8-4aef-a706-eb6f2c006782@github.com>
Message-ID: <0ousl8evktoYc0Qovj7RZmj-0qsktPaKZgc2mNFNBQU=.5bf83494-4351-43ae-a053-d51f46068717@github.com>

On Tue, 18 Feb 2025 16:27:47 GMT, Hannes Greule <hgreule at openjdk.org> wrote:

> I don't know if there are downsides to those "virtual" downcall handles. Alternatively, would it make sense to use MutableCallSite? Afaik that should allow optimizations similar to static final fields but the actual targets could be resolved in the constructor still.

No downside -- in fact, regular downcall handles are just "virtual" downcall handle, where the first memory segment parameter is injected statically (e.g. using `insertArguments`). Internally, the downcall method handle machinery always model the target address as an input parameter, so there's no extra cost.

-------------

PR Comment: https://git.openjdk.org/babylon/pull/322#issuecomment-2666390523

From hgreule at openjdk.org  Tue Feb 18 19:28:23 2025
From: hgreule at openjdk.org (Hannes Greule)
Date: Tue, 18 Feb 2025 19:28:23 GMT
Subject: [code-reflection] Integrated: Make downcall method handles for ORT
 API functions static
In-Reply-To: <fiNMWH6j15rUXnGKJ_t0ZxqQmVudOsEpZ3dkmMh0uNw=.0dfc3d2a-e242-4bf2-b79c-b1c3b6c1f2a5@github.com>
References: <fiNMWH6j15rUXnGKJ_t0ZxqQmVudOsEpZ3dkmMh0uNw=.0dfc3d2a-e242-4bf2-b79c-b1c3b6c1f2a5@github.com>
Message-ID: <L6bfWYgtZg4lTQUE1vcUzahrbILXyKqlVu_71qZCaCw=.a1259096-51f8-4aef-a706-eb6f2c006782@github.com>

On Tue, 18 Feb 2025 15:15:26 GMT, Maurizio Cimadamore <mcimadamore at openjdk.org> wrote:

> This PR tweaks the method handles in `OnnxRuntime` to be static and final.
> The trick is to use a so called "virtual" downcall handle -- a downcall handle that takes its input address as an extra parameter.
> Then, we can just adapt such a method handle to fetch the target function address from the incoming `runtimeAddress` parameter.
> This way, all the method handles modelling functions in the ORT API can be made static, which unlocks better C2 optimizations.
> 
> Clients of such method handles have to be tweaked to pass in the additional leading `runtimeAddress` parameter.

I don't know if there are downsides to those "virtual" downcall handles. Alternatively, would it make sense to use MutableCallSite? Afaik that should allow optimizations similar to static final fields but the actual targets could be resolved in the constructor still.

-------------

PR Comment: https://git.openjdk.org/babylon/pull/322#issuecomment-2666222388

From hgreule at openjdk.org  Tue Feb 18 19:28:29 2025
From: hgreule at openjdk.org (Hannes Greule)
Date: Tue, 18 Feb 2025 19:28:29 GMT
Subject: [code-reflection] Integrated: Make downcall method handles for ORT
 API functions static
In-Reply-To: <0ousl8evktoYc0Qovj7RZmj-0qsktPaKZgc2mNFNBQU=.5bf83494-4351-43ae-a053-d51f46068717@github.com>
References: <fiNMWH6j15rUXnGKJ_t0ZxqQmVudOsEpZ3dkmMh0uNw=.0dfc3d2a-e242-4bf2-b79c-b1c3b6c1f2a5@github.com>
 <L6bfWYgtZg4lTQUE1vcUzahrbILXyKqlVu_71qZCaCw=.a1259096-51f8-4aef-a706-eb6f2c006782@github.com>
 <0ousl8evktoYc0Qovj7RZmj-0qsktPaKZgc2mNFNBQU=.5bf83494-4351-43ae-a053-d51f46068717@github.com>
Message-ID: <oKJwK2xPga3fPy2LG2G7-D9MP4rPJVldnaHchKSi60Y=.b07085b0-6aa8-4153-91bc-1b9036495a53@github.com>

On Tue, 18 Feb 2025 17:31:23 GMT, Maurizio Cimadamore <mcimadamore at openjdk.org> wrote:

> No downside -- in fact, regular downcall handles are just "virtual" downcall handle, where the first memory segment parameter is injected statically (e.g. using `insertArguments`). Internally, the downcall method handle machinery always model the target address as an input parameter, so there's no extra cost.

I see, thanks!

-------------

PR Comment: https://git.openjdk.org/babylon/pull/322#issuecomment-2666475783

From mcimadamore at openjdk.org  Tue Feb 18 19:28:30 2025
From: mcimadamore at openjdk.org (Maurizio Cimadamore)
Date: Tue, 18 Feb 2025 19:28:30 GMT
Subject: [code-reflection] Integrated: Make downcall method handles for ORT API
 functions static
In-Reply-To: <fiNMWH6j15rUXnGKJ_t0ZxqQmVudOsEpZ3dkmMh0uNw=.0dfc3d2a-e242-4bf2-b79c-b1c3b6c1f2a5@github.com>
References: <fiNMWH6j15rUXnGKJ_t0ZxqQmVudOsEpZ3dkmMh0uNw=.0dfc3d2a-e242-4bf2-b79c-b1c3b6c1f2a5@github.com>
Message-ID: <s9zoZ2xjHj917NY6ZZhP0rsNqDP2oRTPRDcnfxTGLDA=.2d55e177-c06e-48d0-8e7e-7084fafa5a6d@github.com>

On Tue, 18 Feb 2025 15:15:26 GMT, Maurizio Cimadamore <mcimadamore at openjdk.org> wrote:

> This PR tweaks the method handles in `OnnxRuntime` to be static and final.
> The trick is to use a so called "virtual" downcall handle -- a downcall handle that takes its input address as an extra parameter.
> Then, we can just adapt such a method handle to fetch the target function address from the incoming `runtimeAddress` parameter.
> This way, all the method handles modelling functions in the ORT API can be made static, which unlocks better C2 optimizations.
> 
> Clients of such method handles have to be tweaked to pass in the additional leading `runtimeAddress` parameter.

This pull request has now been integrated.

Changeset: 90be8529
Author:    Maurizio Cimadamore <mcimadamore at openjdk.org>
URL:       https://git.openjdk.org/babylon/commit/90be85293126f550c8672adc706ff58d0c6309d3
Stats:     72 lines in 1 file changed: 28 ins; 15 del; 29 mod

Make downcall method handles for ORT API functions static

Reviewed-by: asotona

-------------

PR: https://git.openjdk.org/babylon/pull/322

From mcimadamore at openjdk.org  Thu Feb 20 13:09:05 2025
From: mcimadamore at openjdk.org (Maurizio Cimadamore)
Date: Thu, 20 Feb 2025 13:09:05 GMT
Subject: [code-reflection] RFR: Split OnnxRuntime into high-level and low-level
 generated code
Message-ID: <NCddlDCBPiAuXrcBxlnjrpgLy3Z85M4GEOGqAxNbHsE=.8ca5799a-1f9e-4d46-89d2-ca655c4dc3c8@github.com>

This PR reorganizes the code in the OnnxRuntime class. It now contains an higher-level implementation of the onnx C API, which is separate from the low-level bindings. These bindings are defined in the new `foreign` package, and have been mechanically generated with a tweaked version of `jextract` which:
* omits some of the code that is not required (e.g. struct getters/setters)
* make the generated code more usable for onnx use case -- meaning each struct field should have an "invoker" accessor, which fetches the struct pointer and then invokes it

The generated code is still quite big (OrtApi struct contains 10K LoC). I can make the bindings smaller if we only want to focus on the handful of functions we use today -- but in its current form, the low-level bindings are "complete" and support the full C API.

The main changes in OnnxRuntime are as follows:
* calls to function pointers in OrtApi/OrtApiBase now are handled as calls to static wrappers generated by jextract.
* there's no need to try/catch and wrap exceptions because the generated code does that
* some of the explicit reinterpret calls with an explicit size have been replaced by generated `reinterpret` methods (which use the correct struct layout size)
* some of the constants (logging level, API version) are now taken from the generated code

When doing this exercise, I uncovered an issue with the MHs for a couple of functions (`ReleaseEnv` and `ReleaseSession`) being incorrect -- the underlying function pointers for these are void (so no result) and they don't accept a trailing `ret` segment. Also calling `checkStatus` on them makes no sense (again, because these functions don't return a status).

-------------

Commit messages:
 - Split OnnxRuntime into high-level and low-level generated code

Changes: https://git.openjdk.org/babylon/pull/323/files
  Webrev: https://webrevs.openjdk.org/?repo=babylon&pr=323&range=00
  Stats: 14395 lines in 4 files changed: 14189 ins; 158 del; 48 mod
  Patch: https://git.openjdk.org/babylon/pull/323.diff
  Fetch: git fetch https://git.openjdk.org/babylon.git pull/323/head:pull/323

PR: https://git.openjdk.org/babylon/pull/323

From asotona at openjdk.org  Thu Feb 20 18:38:44 2025
From: asotona at openjdk.org (Adam Sotona)
Date: Thu, 20 Feb 2025 18:38:44 GMT
Subject: [code-reflection] RFR: OnnxProtoBuilder simplifications
Message-ID: <d0zF2gB14DnDqVyj1SZxpf1n1Qes-zkGg_FP-oTb2nw=.1f73ebbd-8a7e-4c74-8006-398d44f19b6c@github.com>

OnnxProtoBuilder simplifications

-------------

Commit messages:
 - more simplification
 - OnnxProtoBuilder simplifications

Changes: https://git.openjdk.org/babylon/pull/324/files
  Webrev: https://webrevs.openjdk.org/?repo=babylon&pr=324&range=00
  Stats: 111 lines in 8 files changed: 33 ins; 44 del; 34 mod
  Patch: https://git.openjdk.org/babylon/pull/324.diff
  Fetch: git fetch https://git.openjdk.org/babylon.git pull/324/head:pull/324

PR: https://git.openjdk.org/babylon/pull/324

From gfrost at openjdk.org  Fri Feb 21 15:36:15 2025
From: gfrost at openjdk.org (Gary Frost)
Date: Fri, 21 Feb 2025 15:36:15 GMT
Subject: git: openjdk/babylon: code-reflection: Hat testing buffer marking 1
Message-ID: <cf46e187-0870-42d4-a762-0466f19737f7@openjdk.org>

Changeset: 576b09a9
Branch: code-reflection
Author:    Gary Frost <gfrost at openjdk.org>
Date:      2025-02-21 15:33:47 +0000
URL:       https://git.openjdk.org/babylon/commit/576b09a901e5d201ef9bdec0518918ee4b16f6ff

Hat testing buffer marking 1

! hat/backends/ffi/cuda/cpp/cuda_backend.cpp
! hat/backends/ffi/cuda/include/cuda_backend.h
! hat/backends/ffi/cuda/src/main/java/hat/backend/ffi/CudaBackend.java
! hat/backends/ffi/mock/cpp/mock_backend.cpp
! hat/backends/ffi/mock/src/main/java/hat/backend/ffi/MockBackend.java
! hat/backends/ffi/opencl/cpp/info.cpp
! hat/backends/ffi/opencl/cpp/opencl_backend.cpp
! hat/backends/ffi/opencl/include/opencl_backend.h
! hat/backends/ffi/opencl/src/main/java/hat/backend/ffi/OpenCLBackend.java
! hat/backends/ffi/ptx/cpp/ptx_backend.cpp
! hat/backends/ffi/ptx/include/ptx_backend.h
! hat/backends/ffi/ptx/src/main/java/hat/backend/ffi/PTXBackend.java
! hat/backends/ffi/shared/cpp/shared.cpp
! hat/backends/ffi/shared/include/shared.h
! hat/backends/ffi/spirv/cpp/spirv_backend.cpp
! hat/backends/jextracted/opencl/src/main/java/hat/backend/jextracted/OpenCLBackend.java
! hat/bld
! hat/docs/hat-05-accelerator-compute.md
! hat/docs/hat-06-kernel-analysis.md
! hat/examples/blackscholes/src/main/java/blackscholes/Main.java
! hat/examples/experiments/src/main/java/experiments/Mesh.java
+ hat/examples/experiments/src/main/java/experiments/MinBufferTest.java
- hat/examples/experiments/src/main/java/experiments/S32Array.java
! hat/examples/heal/src/main/java/heal/Compute.java
! hat/examples/life/src/main/java/life/Main.java
+ hat/examples/life/src/main/java/life/Main.java.no
! hat/examples/life/src/main/java/life/Viewer.java
= hat/examples/life/src/main/java/life/Viewer.java.no
! hat/examples/nbody/src/main/java/nbody/Main.java
= hat/examples/nbody/src/main/java/nbody/Main.java.no
+ hat/examples/nbody/src/main/java/nbody/NBodyGLWindow.java
+ hat/examples/nbody/src/main/java/nbody/opencl/NBody.java
! hat/examples/squares/src/main/java/squares/Main.java
! hat/examples/violajones/src/main/java/violajones/Main.java
! hat/examples/violajones/src/main/java/violajones/ViolaJonesCoreCompute.java
! hat/hat/src/main/java/hat/Accelerator.java
! hat/hat/src/main/java/hat/backend/ffi/C99FFIBackend.java
! hat/hat/src/main/java/hat/backend/ffi/FFIBackend.java
! hat/hat/src/main/java/hat/backend/ffi/FFIBackendDriver.java
! hat/hat/src/main/java/hat/buffer/ArgArray.java
! hat/hat/src/main/java/hat/buffer/Buffer.java
! hat/hat/src/main/java/hat/callgraph/KernelCallGraph.java
! hat/hatrun
! hat/intellij/.idea/compiler.xml
! hat/intellij/experiments.iml
! hat/intellij/life.iml
! hat/intellij/nbody.iml
! hat/wrap/clwrap/src/main/java/wrap/clwrap/CLPlatform.java
= hat/wrap/clwrap/src/main/java/wrap/clwrap/CLWrapComputeContext.java
! hat/wrap/clwrap/src/main/java/wrap/clwrap/Squares.java


From gfrost at openjdk.org  Fri Feb 21 15:36:38 2025
From: gfrost at openjdk.org (Gary Frost)
Date: Fri, 21 Feb 2025 15:36:38 GMT
Subject: [code-reflection] Integrated: Hat testing buffer marking 1
Message-ID: <qC8y3XB3IKXcOLDOFkCoIVrS8bhUnihJ7FnJba9mDGY=.054e489d-643c-4bc6-8136-c6be3472df39@github.com>

Step 1 using buffer marking to minimize copies. 

This works with most samples/examples.  We check markings at kernel dispatch time, and if a buffer is still 'device dirty' (meaning the last update to the segment was a kernel dispatch) then we avoid copying back to the device.  
This saves 15% or so  on violajones

Next step is not writing buffers after each kernel, and lazily pulling from the device if Java code requires the buffer. Plumbing for this is in place, but not turned on yet.

We also need to mark buffers based on kernel usage.  I planned to eventually statically analyze kernels.  At present I rely on annotations of kernel params @RO, @RW and @WO.

-------------

Commit messages:
 - License and whitespace
 - Phase one ok.
 - Phase one ok.
 - Phase one ok.
 - Phase one ok.
 - Phase one ok.
 - Phase one ok.
 - Allow opencl modes to be selected from env
 - Allow opencl modes to be selected from env
 - adding hat implementation to life example (it previously used raw extracted cl)
 - ... and 13 more: https://git.openjdk.org/babylon/compare/99339839...e51badd9

Changes: https://git.openjdk.org/babylon/pull/325/files
  Webrev: https://webrevs.openjdk.org/?repo=babylon&pr=325&range=00
  Stats: 3089 lines in 50 files changed: 1773 ins; 831 del; 485 mod
  Patch: https://git.openjdk.org/babylon/pull/325.diff
  Fetch: git fetch https://git.openjdk.org/babylon.git pull/325/head:pull/325

PR: https://git.openjdk.org/babylon/pull/325

From gfrost at openjdk.org  Fri Feb 21 15:36:40 2025
From: gfrost at openjdk.org (Gary Frost)
Date: Fri, 21 Feb 2025 15:36:40 GMT
Subject: [code-reflection] Integrated: Hat testing buffer marking 1
In-Reply-To: <qC8y3XB3IKXcOLDOFkCoIVrS8bhUnihJ7FnJba9mDGY=.054e489d-643c-4bc6-8136-c6be3472df39@github.com>
References: <qC8y3XB3IKXcOLDOFkCoIVrS8bhUnihJ7FnJba9mDGY=.054e489d-643c-4bc6-8136-c6be3472df39@github.com>
Message-ID: <BT5_FLbwhUfxhLszgixU8V0APdsZ_vj_ogZQDtUpdfk=.319b24ac-b620-4e4e-b7e0-c6c16d3d18f5@github.com>

On Fri, 21 Feb 2025 15:31:43 GMT, Gary Frost <gfrost at openjdk.org> wrote:

> Step 1 using buffer marking to minimize copies. 
> 
> This works with most samples/examples.  We check markings at kernel dispatch time, and if a buffer is still 'device dirty' (meaning the last update to the segment was a kernel dispatch) then we avoid copying back to the device.  
> This saves 15% or so  on violajones
> 
> Next step is not writing buffers after each kernel, and lazily pulling from the device if Java code requires the buffer. Plumbing for this is in place, but not turned on yet.
> 
> We also need to mark buffers based on kernel usage.  I planned to eventually statically analyze kernels.  At present I rely on annotations of kernel params @RO, @RW and @WO.

This pull request has now been integrated.

Changeset: 576b09a9
Author:    Gary Frost <gfrost at openjdk.org>
URL:       https://git.openjdk.org/babylon/commit/576b09a901e5d201ef9bdec0518918ee4b16f6ff
Stats:     3089 lines in 50 files changed: 1773 ins; 831 del; 485 mod

Hat testing buffer marking 1

-------------

PR: https://git.openjdk.org/babylon/pull/325

From asotona at openjdk.org  Fri Feb 21 15:47:27 2025
From: asotona at openjdk.org (Adam Sotona)
Date: Fri, 21 Feb 2025 15:47:27 GMT
Subject: [code-reflection] RFR: OnnxProtoBuilder simplifications [v2]
In-Reply-To: <d0zF2gB14DnDqVyj1SZxpf1n1Qes-zkGg_FP-oTb2nw=.1f73ebbd-8a7e-4c74-8006-398d44f19b6c@github.com>
References: <d0zF2gB14DnDqVyj1SZxpf1n1Qes-zkGg_FP-oTb2nw=.1f73ebbd-8a7e-4c74-8006-398d44f19b6c@github.com>
Message-ID: <5g4E4fbC-Lz4sbWMbpfDWb5GVf6k3r38vFRT3ysK_CU=.c6f36a2f-0d9a-49c4-8ba4-86e726e91ece@github.com>

> OnnxProtoBuilder simplifications

Adam Sotona has updated the pull request incrementally with one additional commit since the last revision:

  OnnxProtoBuilder refactored

-------------

Changes:
  - all: https://git.openjdk.org/babylon/pull/324/files
  - new: https://git.openjdk.org/babylon/pull/324/files/c9fc62cb..6bb2c106

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=babylon&pr=324&range=01
 - incr: https://webrevs.openjdk.org/?repo=babylon&pr=324&range=00-01

  Stats: 92 lines in 3 files changed: 39 ins; 39 del; 14 mod
  Patch: https://git.openjdk.org/babylon/pull/324.diff
  Fetch: git fetch https://git.openjdk.org/babylon.git pull/324/head:pull/324

PR: https://git.openjdk.org/babylon/pull/324

From asotona at openjdk.org  Fri Feb 21 15:51:43 2025
From: asotona at openjdk.org (Adam Sotona)
Date: Fri, 21 Feb 2025 15:51:43 GMT
Subject: [code-reflection] RFR: OnnxProtoBuilder simplifications [v3]
In-Reply-To: <d0zF2gB14DnDqVyj1SZxpf1n1Qes-zkGg_FP-oTb2nw=.1f73ebbd-8a7e-4c74-8006-398d44f19b6c@github.com>
References: <d0zF2gB14DnDqVyj1SZxpf1n1Qes-zkGg_FP-oTb2nw=.1f73ebbd-8a7e-4c74-8006-398d44f19b6c@github.com>
Message-ID: <a7P4FLT7VmqISQIilybcjzsuPOuxZdGuyztIuRygk3s=.734e7327-f37d-42d1-96c9-da1914991653@github.com>

> OnnxProtoBuilder simplifications

Adam Sotona has updated the pull request incrementally with one additional commit since the last revision:

  OnnxInterpreter patch related to optional inputs

-------------

Changes:
  - all: https://git.openjdk.org/babylon/pull/324/files
  - new: https://git.openjdk.org/babylon/pull/324/files/6bb2c106..3eb31c5a

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=babylon&pr=324&range=02
 - incr: https://webrevs.openjdk.org/?repo=babylon&pr=324&range=01-02

  Stats: 11 lines in 1 file changed: 2 ins; 8 del; 1 mod
  Patch: https://git.openjdk.org/babylon/pull/324.diff
  Fetch: git fetch https://git.openjdk.org/babylon.git pull/324/head:pull/324

PR: https://git.openjdk.org/babylon/pull/324

From asotona at openjdk.org  Fri Feb 21 15:54:54 2025
From: asotona at openjdk.org (Adam Sotona)
Date: Fri, 21 Feb 2025 15:54:54 GMT
Subject: git: openjdk/babylon: code-reflection: OnnxProtoBuilder
 simplifications
Message-ID: <6d99097b-4543-48f7-92ae-8ac922233226@openjdk.org>

Changeset: 22a2404c
Branch: code-reflection
Author:    Adam Sotona <asotona at openjdk.org>
Date:      2025-02-21 15:54:00 +0000
URL:       https://git.openjdk.org/babylon/commit/22a2404c7900641ea45a147a0ededa5c3cab20e0

OnnxProtoBuilder simplifications

! cr-examples/onnx/src/main/java/oracle/code/onnx/OnnxInterpreter.java
! cr-examples/onnx/src/main/java/oracle/code/onnx/OnnxProtoBuilder.java
! cr-examples/onnx/src/main/java/oracle/code/onnx/OnnxRuntime.java
! cr-examples/onnx/src/main/java/oracle/code/onnx/ir/OnnxOp.java
! cr-examples/onnx/src/test/java/oracle/code/onnx/CNNTest.java
! cr-examples/onnx/src/test/java/oracle/code/onnx/MNISTDemo.java
! cr-examples/onnx/src/test/java/oracle/code/onnx/RuntimeTest.java
! cr-examples/onnx/src/test/java/oracle/code/onnx/SimpleTest.java


From asotona at openjdk.org  Fri Feb 21 15:57:06 2025
From: asotona at openjdk.org (Adam Sotona)
Date: Fri, 21 Feb 2025 15:57:06 GMT
Subject: [code-reflection] Integrated: OnnxProtoBuilder simplifications
In-Reply-To: <d0zF2gB14DnDqVyj1SZxpf1n1Qes-zkGg_FP-oTb2nw=.1f73ebbd-8a7e-4c74-8006-398d44f19b6c@github.com>
References: <d0zF2gB14DnDqVyj1SZxpf1n1Qes-zkGg_FP-oTb2nw=.1f73ebbd-8a7e-4c74-8006-398d44f19b6c@github.com>
Message-ID: <SVGIgBoTsH94BROGGI5IcV5qOaf8CgX27iwxu85TBLU=.0424ffdd-3fcc-423f-b7ec-ddcd17e7e2e1@github.com>

On Thu, 20 Feb 2025 18:01:24 GMT, Adam Sotona <asotona at openjdk.org> wrote:

> OnnxProtoBuilder simplifications

This pull request has now been integrated.

Changeset: 22a2404c
Author:    Adam Sotona <asotona at openjdk.org>
URL:       https://git.openjdk.org/babylon/commit/22a2404c7900641ea45a147a0ededa5c3cab20e0
Stats:     155 lines in 8 files changed: 51 ins; 68 del; 36 mod

OnnxProtoBuilder simplifications

-------------

PR: https://git.openjdk.org/babylon/pull/324

From mabbay at openjdk.org  Sun Feb 23 20:26:54 2025
From: mabbay at openjdk.org (Mourad Abbay)
Date: Sun, 23 Feb 2025 20:26:54 GMT
Subject: [code-reflection] RFR: Support storing the code that builds the
 code model [v7]
In-Reply-To: <dexv8sQV_NcJsJIgMopd_M4wSWoZya9dU0ACwDtEL0E=.4ac0b80f-24fb-440f-a18d-bbfd07a681bc@github.com>
References: <dexv8sQV_NcJsJIgMopd_M4wSWoZya9dU0ACwDtEL0E=.4ac0b80f-24fb-440f-a18d-bbfd07a681bc@github.com>
Message-ID: <9p9_UIxsOsfpIra16AtpwKcnFryFQVRBHcYxfaJBmzg=.b37f1825-f869-4f01-b49b-b3b8734ea572@github.com>

> In this PR we allow the user to decide how to store the code model. The first option is `TEXT`, if this is selected, we store the textual representation of the code model. The second option is `CODE_BUILDR`, if this is selected, we store the code that builds the code model. All work done here is around the second option, because the first option is already supported.

Mourad Abbay has updated the pull request incrementally with one additional commit since the last revision:

  Support storing code that builds code model

-------------

Changes:
  - all: https://git.openjdk.org/babylon/pull/305/files
  - new: https://git.openjdk.org/babylon/pull/305/files/21c12cb3..86a3959d

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=babylon&pr=305&range=06
 - incr: https://webrevs.openjdk.org/?repo=babylon&pr=305&range=05-06

  Stats: 156 lines in 4 files changed: 45 ins; 96 del; 15 mod
  Patch: https://git.openjdk.org/babylon/pull/305.diff
  Fetch: git fetch https://git.openjdk.org/babylon.git pull/305/head:pull/305

PR: https://git.openjdk.org/babylon/pull/305

From mabbay at openjdk.org  Sun Feb 23 20:26:55 2025
From: mabbay at openjdk.org (Mourad Abbay)
Date: Sun, 23 Feb 2025 20:26:55 GMT
Subject: [code-reflection] RFR: Support storing the code that builds the
 code model [v6]
In-Reply-To: <bzLQ1zOmdgAkK8AnkzhtzS28SaySIZpOnARbcLVgiYQ=.6f0c8491-2b78-46ad-846c-6c871c7931d9@github.com>
References: <dexv8sQV_NcJsJIgMopd_M4wSWoZya9dU0ACwDtEL0E=.4ac0b80f-24fb-440f-a18d-bbfd07a681bc@github.com>
 <bzLQ1zOmdgAkK8AnkzhtzS28SaySIZpOnARbcLVgiYQ=.6f0c8491-2b78-46ad-846c-6c871c7931d9@github.com>
Message-ID: <5OAHDATruMRK-d3DRbdOjkd-t5V3n-eNjT3-Avht2cw=.c14156ce-88d4-4955-80ef-6f471a2206eb@github.com>

On Tue, 11 Feb 2025 20:15:06 GMT, Mourad Abbay <mabbay at openjdk.org> wrote:

>> In this PR we allow the user to decide how to store the code model. The first option is `TEXT`, if this is selected, we store the textual representation of the code model. The second option is `CODE_BUILDR`, if this is selected, we store the code that builds the code model. All work done here is around the second option, because the first option is already supported.
>
> Mourad Abbay has updated the pull request incrementally with two additional commits since the last revision:
> 
>  - Reformat code
>  - Apply fixes from Maurizio

I made CODE_BUILDER the default storage mechanism for code models, so you can test the capability by running the compiler tests.

-------------

PR Comment: https://git.openjdk.org/babylon/pull/305#issuecomment-2677095971

From mabbay at openjdk.org  Sun Feb 23 20:34:37 2025
From: mabbay at openjdk.org (Mourad Abbay)
Date: Sun, 23 Feb 2025 20:34:37 GMT
Subject: [code-reflection] RFR: Support storing the code that builds the
 code model [v8]
In-Reply-To: <dexv8sQV_NcJsJIgMopd_M4wSWoZya9dU0ACwDtEL0E=.4ac0b80f-24fb-440f-a18d-bbfd07a681bc@github.com>
References: <dexv8sQV_NcJsJIgMopd_M4wSWoZya9dU0ACwDtEL0E=.4ac0b80f-24fb-440f-a18d-bbfd07a681bc@github.com>
Message-ID: <dd_ixlnPLyglgglOaN5WCrkEl8Npli-dJvl6eW5TQp0=.c0d6d767-47f8-42dd-a97b-e431ff0f982f@github.com>

> In this PR we allow the user to decide how to store the code model. The first option is `TEXT`, if this is selected, we store the textual representation of the code model. The second option is `CODE_BUILDR`, if this is selected, we store the code that builds the code model. All work done here is around the second option, because the first option is already supported.

Mourad Abbay has updated the pull request incrementally with one additional commit since the last revision:

  Add comment

-------------

Changes:
  - all: https://git.openjdk.org/babylon/pull/305/files
  - new: https://git.openjdk.org/babylon/pull/305/files/86a3959d..cd302fc9

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=babylon&pr=305&range=07
 - incr: https://webrevs.openjdk.org/?repo=babylon&pr=305&range=06-07

  Stats: 2 lines in 1 file changed: 2 ins; 0 del; 0 mod
  Patch: https://git.openjdk.org/babylon/pull/305.diff
  Fetch: git fetch https://git.openjdk.org/babylon.git pull/305/head:pull/305

PR: https://git.openjdk.org/babylon/pull/305

From mabbay at openjdk.org  Mon Feb 24 17:45:03 2025
From: mabbay at openjdk.org (Mourad Abbay)
Date: Mon, 24 Feb 2025 17:45:03 GMT
Subject: [code-reflection] RFR: Support storing the code that builds the
 code model [v9]
In-Reply-To: <dexv8sQV_NcJsJIgMopd_M4wSWoZya9dU0ACwDtEL0E=.4ac0b80f-24fb-440f-a18d-bbfd07a681bc@github.com>
References: <dexv8sQV_NcJsJIgMopd_M4wSWoZya9dU0ACwDtEL0E=.4ac0b80f-24fb-440f-a18d-bbfd07a681bc@github.com>
Message-ID: <5FotwnFOc6WHkvwsriz8wwBTBDP23vZ1y-Wlz9atXxM=.aa256306-6521-4b92-b746-3fd075036322@github.com>

> In this PR we allow the user to decide how to store the code model. The first option is `TEXT`, if this is selected, we store the textual representation of the code model. The second option is `CODE_BUILDR`, if this is selected, we store the code that builds the code model. All work done here is around the second option, because the first option is already supported.

Mourad Abbay has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains 11 additional commits since the last revision:

 - Merge branch 'code-reflection' into code-model-storage-option
 - Add comment
 - Support storing code that builds code model
 - Reformat code
 - Apply fixes from Maurizio
 - Define parsing code in the enum CodeModelStorageOption
 - Add a program to test storing code that builds code model
 - Add JavaType -> Type mapping for Block.Parameter
 - Support storing code that builds code model
 - Make TEXT the default storage mechanism for code models
 - ... and 1 more: https://git.openjdk.org/babylon/compare/782b8e14...f8327908

-------------

Changes:
  - all: https://git.openjdk.org/babylon/pull/305/files
  - new: https://git.openjdk.org/babylon/pull/305/files/cd302fc9..f8327908

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=babylon&pr=305&range=08
 - incr: https://webrevs.openjdk.org/?repo=babylon&pr=305&range=07-08

  Stats: 96868 lines in 170 files changed: 94359 ins; 1437 del; 1072 mod
  Patch: https://git.openjdk.org/babylon/pull/305.diff
  Fetch: git fetch https://git.openjdk.org/babylon.git pull/305/head:pull/305

PR: https://git.openjdk.org/babylon/pull/305

From mcimadamore at openjdk.org  Mon Feb 24 19:22:10 2025
From: mcimadamore at openjdk.org (Maurizio Cimadamore)
Date: Mon, 24 Feb 2025 19:22:10 GMT
Subject: [code-reflection] RFR: Split OnnxRuntime into high-level and
 low-level generated code [v2]
In-Reply-To: <NCddlDCBPiAuXrcBxlnjrpgLy3Z85M4GEOGqAxNbHsE=.8ca5799a-1f9e-4d46-89d2-ca655c4dc3c8@github.com>
References: <NCddlDCBPiAuXrcBxlnjrpgLy3Z85M4GEOGqAxNbHsE=.8ca5799a-1f9e-4d46-89d2-ca655c4dc3c8@github.com>
Message-ID: <4vH4kKg9s994_XNuA5GigTPVS3gvRvX57n-X5pXdOn0=.9bba8146-ec02-4ca5-9e5b-e8be2041b58c@github.com>

> This PR reorganizes the code in the OnnxRuntime class. It now contains an higher-level implementation of the onnx C API, which is separate from the low-level bindings. These bindings are defined in the new `foreign` package, and have been mechanically generated with a tweaked version of `jextract` which:
> * omits some of the code that is not required (e.g. struct getters/setters)
> * make the generated code more usable for onnx use case -- meaning each struct field should have an "invoker" accessor, which fetches the struct pointer and then invokes it
> 
> The generated code is still quite big (OrtApi struct contains 10K LoC). I can make the bindings smaller if we only want to focus on the handful of functions we use today -- but in its current form, the low-level bindings are "complete" and support the full C API.
> 
> The main changes in OnnxRuntime are as follows:
> * calls to function pointers in OrtApi/OrtApiBase now are handled as calls to static wrappers generated by jextract.
> * there's no need to try/catch and wrap exceptions because the generated code does that
> * some of the explicit reinterpret calls with an explicit size have been replaced by generated `reinterpret` methods (which use the correct struct layout size)
> * some of the constants (logging level, API version) are now taken from the generated code
> 
> When doing this exercise, I uncovered an issue with the MHs for a couple of functions (`ReleaseEnv` and `ReleaseSession`) being incorrect -- the underlying function pointers for these are void (so no result) and they don't accept a trailing `ret` segment. Also calling `checkStatus` on them makes no sense (again, because these functions don't return a status).

Maurizio Cimadamore has updated the pull request incrementally with one additional commit since the last revision:

  Use jextract type constants
  Fix lifetime of output address params

-------------

Changes:
  - all: https://git.openjdk.org/babylon/pull/323/files
  - new: https://git.openjdk.org/babylon/pull/323/files/5d8f9b03..fe746c01

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=babylon&pr=323&range=01
 - incr: https://webrevs.openjdk.org/?repo=babylon&pr=323&range=00-01

  Stats: 25 lines in 1 file changed: 1 ins; 4 del; 20 mod
  Patch: https://git.openjdk.org/babylon/pull/323.diff
  Fetch: git fetch https://git.openjdk.org/babylon.git pull/323/head:pull/323

PR: https://git.openjdk.org/babylon/pull/323

From mcimadamore at openjdk.org  Mon Feb 24 19:22:10 2025
From: mcimadamore at openjdk.org (Maurizio Cimadamore)
Date: Mon, 24 Feb 2025 19:22:10 GMT
Subject: [code-reflection] RFR: Split OnnxRuntime into high-level and
 low-level generated code
In-Reply-To: <NCddlDCBPiAuXrcBxlnjrpgLy3Z85M4GEOGqAxNbHsE=.8ca5799a-1f9e-4d46-89d2-ca655c4dc3c8@github.com>
References: <NCddlDCBPiAuXrcBxlnjrpgLy3Z85M4GEOGqAxNbHsE=.8ca5799a-1f9e-4d46-89d2-ca655c4dc3c8@github.com>
Message-ID: <OBNw7IVTVQ2MzBco-zTSL6GmupXL7_VoLDHOeAfk498=.f2c9e281-8d8d-465b-8a52-110858d3c53d@github.com>

On Thu, 20 Feb 2025 13:03:41 GMT, Maurizio Cimadamore <mcimadamore at openjdk.org> wrote:

> This PR reorganizes the code in the OnnxRuntime class. It now contains an higher-level implementation of the onnx C API, which is separate from the low-level bindings. These bindings are defined in the new `foreign` package, and have been mechanically generated with a tweaked version of `jextract` which:
> * omits some of the code that is not required (e.g. struct getters/setters)
> * make the generated code more usable for onnx use case -- meaning each struct field should have an "invoker" accessor, which fetches the struct pointer and then invokes it
> 
> The generated code is still quite big (OrtApi struct contains 10K LoC). I can make the bindings smaller if we only want to focus on the handful of functions we use today -- but in its current form, the low-level bindings are "complete" and support the full C API.
> 
> The main changes in OnnxRuntime are as follows:
> * calls to function pointers in OrtApi/OrtApiBase now are handled as calls to static wrappers generated by jextract.
> * there's no need to try/catch and wrap exceptions because the generated code does that
> * some of the explicit reinterpret calls with an explicit size have been replaced by generated `reinterpret` methods (which use the correct struct layout size)
> * some of the constants (logging level, API version) are now taken from the generated code
> 
> When doing this exercise, I uncovered an issue with the MHs for a couple of functions (`ReleaseEnv` and `ReleaseSession`) being incorrect -- the underlying function pointers for these are void (so no result) and they don't accept a trailing `ret` segment. Also calling `checkStatus` on them makes no sense (again, because these functions don't return a status).

I've added some changes to make sure that the segments returned by the high-level API keep the automatic arena reachable. There are two places where I've added a `reinterpret`:
* `retAddr` -- as the output parameter `ret` is accessed, a segment is read (which will have no arena)
* `run` -- this also features output parameters (`outputs`) so we need to keep the segments read from these output parameters connected to the automatic arena

-------------

PR Comment: https://git.openjdk.org/babylon/pull/323#issuecomment-2679426768

From asotona at openjdk.org  Mon Feb 24 19:24:17 2025
From: asotona at openjdk.org (Adam Sotona)
Date: Mon, 24 Feb 2025 19:24:17 GMT
Subject: [code-reflection] RFR: MNISTDemo UI tweak
Message-ID: <g_PxSOEGTIw8ozJfVYW_NdZBclbJIeYm-ePREZ6JEXU=.1fe251d2-2637-45b3-9014-b794b682202b@github.com>

MNISTDemo UI adjusted.

![screenshot](https://github.com/user-attachments/assets/a9eaca28-13d7-45f4-9fb3-97f83bff7f38)

-------------

Commit messages:
 - MNISTDemo UI tweak

Changes: https://git.openjdk.org/babylon/pull/327/files
  Webrev: https://webrevs.openjdk.org/?repo=babylon&pr=327&range=00
  Stats: 13 lines in 1 file changed: 4 ins; 6 del; 3 mod
  Patch: https://git.openjdk.org/babylon/pull/327.diff
  Fetch: git fetch https://git.openjdk.org/babylon.git pull/327/head:pull/327

PR: https://git.openjdk.org/babylon/pull/327

From mcimadamore at openjdk.org  Mon Feb 24 19:29:45 2025
From: mcimadamore at openjdk.org (Maurizio Cimadamore)
Date: Mon, 24 Feb 2025 19:29:45 GMT
Subject: [code-reflection] RFR: Split OnnxRuntime into high-level and
 low-level generated code [v3]
In-Reply-To: <NCddlDCBPiAuXrcBxlnjrpgLy3Z85M4GEOGqAxNbHsE=.8ca5799a-1f9e-4d46-89d2-ca655c4dc3c8@github.com>
References: <NCddlDCBPiAuXrcBxlnjrpgLy3Z85M4GEOGqAxNbHsE=.8ca5799a-1f9e-4d46-89d2-ca655c4dc3c8@github.com>
Message-ID: <gorF4JXVYF8l52s8glOX8GEoC3lMfWxqqY6vLlUa_K8=.c0aa32d2-e373-44f3-b248-45b59e44d89b@github.com>

> This PR reorganizes the code in the OnnxRuntime class. It now contains an higher-level implementation of the onnx C API, which is separate from the low-level bindings. These bindings are defined in the new `foreign` package, and have been mechanically generated with a tweaked version of `jextract` which:
> * omits some of the code that is not required (e.g. struct getters/setters)
> * make the generated code more usable for onnx use case -- meaning each struct field should have an "invoker" accessor, which fetches the struct pointer and then invokes it
> 
> The generated code is still quite big (OrtApi struct contains 10K LoC). I can make the bindings smaller if we only want to focus on the handful of functions we use today -- but in its current form, the low-level bindings are "complete" and support the full C API.
> 
> The main changes in OnnxRuntime are as follows:
> * calls to function pointers in OrtApi/OrtApiBase now are handled as calls to static wrappers generated by jextract.
> * there's no need to try/catch and wrap exceptions because the generated code does that
> * some of the explicit reinterpret calls with an explicit size have been replaced by generated `reinterpret` methods (which use the correct struct layout size)
> * some of the constants (logging level, API version) are now taken from the generated code
> 
> When doing this exercise, I uncovered an issue with the MHs for a couple of functions (`ReleaseEnv` and `ReleaseSession`) being incorrect -- the underlying function pointers for these are void (so no result) and they don't accept a trailing `ret` segment. Also calling `checkStatus` on them makes no sense (again, because these functions don't return a status).

Maurizio Cimadamore has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains three commits:

 - Merge branch 'code-reflection' into onnx_jextract
 - Use jextract type constants
   Fix lifetime of output address params
 - Split OnnxRuntime into high-level and low-level generated code

-------------

Changes: https://git.openjdk.org/babylon/pull/323/files
  Webrev: https://webrevs.openjdk.org/?repo=babylon&pr=323&range=02
  Stats: 14414 lines in 4 files changed: 14189 ins; 158 del; 67 mod
  Patch: https://git.openjdk.org/babylon/pull/323.diff
  Fetch: git fetch https://git.openjdk.org/babylon.git pull/323/head:pull/323

PR: https://git.openjdk.org/babylon/pull/323

From mcimadamore at openjdk.org  Mon Feb 24 19:34:23 2025
From: mcimadamore at openjdk.org (Maurizio Cimadamore)
Date: Mon, 24 Feb 2025 19:34:23 GMT
Subject: git: openjdk/babylon: code-reflection: Split OnnxRuntime into
 high-level and low-level generated code
Message-ID: <622fa37c-d51f-4767-b6ff-33b12e2fca87@openjdk.org>

Changeset: 4752b37c
Branch: code-reflection
Author:    Maurizio Cimadamore <mcimadamore at openjdk.org>
Date:      2025-02-24 19:33:12 +0000
URL:       https://git.openjdk.org/babylon/commit/4752b37cb924f18dfe0c8d4b7e5553a06eac44dd

Split OnnxRuntime into high-level and low-level generated code

! cr-examples/onnx/src/main/java/oracle/code/onnx/OnnxRuntime.java
+ cr-examples/onnx/src/main/java/oracle/code/onnx/foreign/OrtApi.java
+ cr-examples/onnx/src/main/java/oracle/code/onnx/foreign/OrtApiBase.java
+ cr-examples/onnx/src/main/java/oracle/code/onnx/foreign/onnxruntime_c_api_h.java


From mcimadamore at openjdk.org  Mon Feb 24 19:36:21 2025
From: mcimadamore at openjdk.org (Maurizio Cimadamore)
Date: Mon, 24 Feb 2025 19:36:21 GMT
Subject: [code-reflection] Integrated: Split OnnxRuntime into high-level and
 low-level generated code
In-Reply-To: <NCddlDCBPiAuXrcBxlnjrpgLy3Z85M4GEOGqAxNbHsE=.8ca5799a-1f9e-4d46-89d2-ca655c4dc3c8@github.com>
References: <NCddlDCBPiAuXrcBxlnjrpgLy3Z85M4GEOGqAxNbHsE=.8ca5799a-1f9e-4d46-89d2-ca655c4dc3c8@github.com>
Message-ID: <yqLZLHDWY-7JKTI8itvU1ltFVNqyb1pt7TLXl141Mfk=.17ef2040-6f33-49b6-8d62-d02152ec1760@github.com>

On Thu, 20 Feb 2025 13:03:41 GMT, Maurizio Cimadamore <mcimadamore at openjdk.org> wrote:

> This PR reorganizes the code in the OnnxRuntime class. It now contains an higher-level implementation of the onnx C API, which is separate from the low-level bindings. These bindings are defined in the new `foreign` package, and have been mechanically generated with a tweaked version of `jextract` which:
> * omits some of the code that is not required (e.g. struct getters/setters)
> * make the generated code more usable for onnx use case -- meaning each struct field should have an "invoker" accessor, which fetches the struct pointer and then invokes it
> 
> The generated code is still quite big (OrtApi struct contains 10K LoC). I can make the bindings smaller if we only want to focus on the handful of functions we use today -- but in its current form, the low-level bindings are "complete" and support the full C API.
> 
> The main changes in OnnxRuntime are as follows:
> * calls to function pointers in OrtApi/OrtApiBase now are handled as calls to static wrappers generated by jextract.
> * there's no need to try/catch and wrap exceptions because the generated code does that
> * some of the explicit reinterpret calls with an explicit size have been replaced by generated `reinterpret` methods (which use the correct struct layout size)
> * some of the constants (logging level, API version) are now taken from the generated code
> 
> When doing this exercise, I uncovered an issue with the MHs for a couple of functions (`ReleaseEnv` and `ReleaseSession`) being incorrect -- the underlying function pointers for these are void (so no result) and they don't accept a trailing `ret` segment. Also calling `checkStatus` on them makes no sense (again, because these functions don't return a status).

This pull request has now been integrated.

Changeset: 4752b37c
Author:    Maurizio Cimadamore <mcimadamore at openjdk.org>
URL:       https://git.openjdk.org/babylon/commit/4752b37cb924f18dfe0c8d4b7e5553a06eac44dd
Stats:     14414 lines in 4 files changed: 14189 ins; 158 del; 67 mod

Split OnnxRuntime into high-level and low-level generated code

-------------

PR: https://git.openjdk.org/babylon/pull/323

From asotona at openjdk.org  Mon Feb 24 19:57:24 2025
From: asotona at openjdk.org (Adam Sotona)
Date: Mon, 24 Feb 2025 19:57:24 GMT
Subject: [code-reflection] RFR: MNISTDemo UI tweak [v2]
In-Reply-To: <g_PxSOEGTIw8ozJfVYW_NdZBclbJIeYm-ePREZ6JEXU=.1fe251d2-2637-45b3-9014-b794b682202b@github.com>
References: <g_PxSOEGTIw8ozJfVYW_NdZBclbJIeYm-ePREZ6JEXU=.1fe251d2-2637-45b3-9014-b794b682202b@github.com>
Message-ID: <l5IYSB1mCSAFTenvGwCVWcgO51uwqpsRKsUgplKQ7cs=.36830651-98f9-4de9-b011-c2890c48fb2a@github.com>

> MNISTDemo UI adjusted.
> 
> ![screenshot](https://github.com/user-attachments/assets/a9eaca28-13d7-45f4-9fb3-97f83bff7f38)

Adam Sotona has updated the pull request incrementally with one additional commit since the last revision:

  added colors

-------------

Changes:
  - all: https://git.openjdk.org/babylon/pull/327/files
  - new: https://git.openjdk.org/babylon/pull/327/files/d9d98e08..6799f648

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=babylon&pr=327&range=01
 - incr: https://webrevs.openjdk.org/?repo=babylon&pr=327&range=00-01

  Stats: 4 lines in 1 file changed: 3 ins; 0 del; 1 mod
  Patch: https://git.openjdk.org/babylon/pull/327.diff
  Fetch: git fetch https://git.openjdk.org/babylon.git pull/327/head:pull/327

PR: https://git.openjdk.org/babylon/pull/327

From psandoz at openjdk.org  Mon Feb 24 21:29:12 2025
From: psandoz at openjdk.org (Paul Sandoz)
Date: Mon, 24 Feb 2025 21:29:12 GMT
Subject: [code-reflection] RFR: MNISTDemo UI tweak [v2]
In-Reply-To: <l5IYSB1mCSAFTenvGwCVWcgO51uwqpsRKsUgplKQ7cs=.36830651-98f9-4de9-b011-c2890c48fb2a@github.com>
References: <g_PxSOEGTIw8ozJfVYW_NdZBclbJIeYm-ePREZ6JEXU=.1fe251d2-2637-45b3-9014-b794b682202b@github.com>
 <l5IYSB1mCSAFTenvGwCVWcgO51uwqpsRKsUgplKQ7cs=.36830651-98f9-4de9-b011-c2890c48fb2a@github.com>
Message-ID: <ZznTJRhlPAX60QlDEHeO35bGkb2xRY1VQquRNEmm4X8=.75ef21d3-a8f9-4fbc-9cce-c0c80d49ea18@github.com>

On Mon, 24 Feb 2025 19:57:24 GMT, Adam Sotona <asotona at openjdk.org> wrote:

>> MNISTDemo UI adjusted.
>> 
>> ![screenshot](https://github.com/user-attachments/assets/e0054126-45e5-4eec-bf2d-707475dfc5b1)
>
> Adam Sotona has updated the pull request incrementally with one additional commit since the last revision:
> 
>   added colors

Marked as reviewed by psandoz (Lead).

-------------

PR Review: https://git.openjdk.org/babylon/pull/327#pullrequestreview-2638477372

From asotona at openjdk.org  Tue Feb 25 07:15:35 2025
From: asotona at openjdk.org (Adam Sotona)
Date: Tue, 25 Feb 2025 07:15:35 GMT
Subject: git: openjdk/babylon: code-reflection: MNISTDemo UI tweak
Message-ID: <edc222ef-fb93-4b70-b1bb-9c692579c419@openjdk.org>

Changeset: 5a30fac0
Branch: code-reflection
Author:    Adam Sotona <asotona at openjdk.org>
Date:      2025-02-25 07:13:22 +0000
URL:       https://git.openjdk.org/babylon/commit/5a30fac048d76298c2a5cb5184c960d533fceee0

MNISTDemo UI tweak

Reviewed-by: psandoz

! cr-examples/onnx/src/test/java/oracle/code/onnx/MNISTDemo.java


From asotona at openjdk.org  Tue Feb 25 07:16:47 2025
From: asotona at openjdk.org (Adam Sotona)
Date: Tue, 25 Feb 2025 07:16:47 GMT
Subject: [code-reflection] RFR: MNISTDemo UI tweak [v3]
In-Reply-To: <g_PxSOEGTIw8ozJfVYW_NdZBclbJIeYm-ePREZ6JEXU=.1fe251d2-2637-45b3-9014-b794b682202b@github.com>
References: <g_PxSOEGTIw8ozJfVYW_NdZBclbJIeYm-ePREZ6JEXU=.1fe251d2-2637-45b3-9014-b794b682202b@github.com>
Message-ID: <YjtWsql_LkaR9Egfb67f1SZdGGZefy_tTGGvXk90_jU=.ba5d316c-ea8e-4ec1-aae6-37b6cadc5004@github.com>

> MNISTDemo UI adjusted.
> 
> ![screenshot](https://github.com/user-attachments/assets/e0054126-45e5-4eec-bf2d-707475dfc5b1)

Adam Sotona has updated the pull request incrementally with one additional commit since the last revision:

  Using MemorySegment instead of ByteBuffer

-------------

Changes:
  - all: https://git.openjdk.org/babylon/pull/327/files
  - new: https://git.openjdk.org/babylon/pull/327/files/6799f648..fbd4471d

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=babylon&pr=327&range=02
 - incr: https://webrevs.openjdk.org/?repo=babylon&pr=327&range=01-02

  Stats: 4 lines in 1 file changed: 1 ins; 1 del; 2 mod
  Patch: https://git.openjdk.org/babylon/pull/327.diff
  Fetch: git fetch https://git.openjdk.org/babylon.git pull/327/head:pull/327

PR: https://git.openjdk.org/babylon/pull/327

From asotona at openjdk.org  Tue Feb 25 07:16:48 2025
From: asotona at openjdk.org (Adam Sotona)
Date: Tue, 25 Feb 2025 07:16:48 GMT
Subject: [code-reflection] Integrated: MNISTDemo UI tweak
In-Reply-To: <g_PxSOEGTIw8ozJfVYW_NdZBclbJIeYm-ePREZ6JEXU=.1fe251d2-2637-45b3-9014-b794b682202b@github.com>
References: <g_PxSOEGTIw8ozJfVYW_NdZBclbJIeYm-ePREZ6JEXU=.1fe251d2-2637-45b3-9014-b794b682202b@github.com>
Message-ID: <I_19mg0X3wF4Q9ANjd9a7IxJuhp5yAh2sILRHSLeX-o=.9ce1b1d8-bc66-43b9-b859-648470370e4b@github.com>

On Mon, 24 Feb 2025 19:15:59 GMT, Adam Sotona <asotona at openjdk.org> wrote:

> MNISTDemo UI adjusted.
> 
> ![screenshot](https://github.com/user-attachments/assets/e0054126-45e5-4eec-bf2d-707475dfc5b1)

This pull request has now been integrated.

Changeset: 5a30fac0
Author:    Adam Sotona <asotona at openjdk.org>
URL:       https://git.openjdk.org/babylon/commit/5a30fac048d76298c2a5cb5184c960d533fceee0
Stats:     18 lines in 1 file changed: 6 ins; 5 del; 7 mod

MNISTDemo UI tweak

Reviewed-by: psandoz

-------------

PR: https://git.openjdk.org/babylon/pull/327

From asotona at openjdk.org  Tue Feb 25 15:01:49 2025
From: asotona at openjdk.org (Adam Sotona)
Date: Tue, 25 Feb 2025 15:01:49 GMT
Subject: [code-reflection] RFR: Onnx subgraphs - work in progress
Message-ID: <CYyR9Ypl-frjAUum631CITskwAXBevLVjgRbKqwmrdU=.026f0f2b-f941-45fc-bfe3-1c0f899912db@github.com>

Onnx subgraphs - work in progress

-------------

Commit messages:
 - implemented Onnx Loop test
 - OnnxProtoBuilder refactoring
 - reduced use of ByteBuffers and Tensor API refactoring
 - Merge branch 'code-reflection' into onnx-runtime-cont6
 - Onnx subgraphs - work in progress
 - Onnx subgraphs - work in progress

Changes: https://git.openjdk.org/babylon/pull/326/files
  Webrev: https://webrevs.openjdk.org/?repo=babylon&pr=326&range=00
  Stats: 184 lines in 8 files changed: 92 ins; 38 del; 54 mod
  Patch: https://git.openjdk.org/babylon/pull/326.diff
  Fetch: git fetch https://git.openjdk.org/babylon.git pull/326/head:pull/326

PR: https://git.openjdk.org/babylon/pull/326

From psandoz at openjdk.org  Tue Feb 25 17:28:11 2025
From: psandoz at openjdk.org (Paul Sandoz)
Date: Tue, 25 Feb 2025 17:28:11 GMT
Subject: [code-reflection] RFR: Support storing the code that builds the
 code model [v9]
In-Reply-To: <5FotwnFOc6WHkvwsriz8wwBTBDP23vZ1y-Wlz9atXxM=.aa256306-6521-4b92-b746-3fd075036322@github.com>
References: <dexv8sQV_NcJsJIgMopd_M4wSWoZya9dU0ACwDtEL0E=.4ac0b80f-24fb-440f-a18d-bbfd07a681bc@github.com>
 <5FotwnFOc6WHkvwsriz8wwBTBDP23vZ1y-Wlz9atXxM=.aa256306-6521-4b92-b746-3fd075036322@github.com>
Message-ID: <E6nT42hN5YYuWeELJWEE197WjAXqTK9z_Zov5oUA-SQ=.d3feb4f6-ddc7-4681-a6ce-a8d23dbf16e1@github.com>

On Mon, 24 Feb 2025 17:45:03 GMT, Mourad Abbay <mabbay at openjdk.org> wrote:

>> In this PR we allow the user to decide how to store the code model. The first option is `TEXT`, if this is selected, we store the textual representation of the code model. The second option is `CODE_BUILDR`, if this is selected, we store the code that builds the code model. All work done here is around the second option, because the first option is already supported.
>
> Mourad Abbay has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains 11 additional commits since the last revision:
> 
>  - Merge branch 'code-reflection' into code-model-storage-option
>  - Add comment
>  - Support storing code that builds code model
>  - Reformat code
>  - Apply fixes from Maurizio
>  - Define parsing code in the enum CodeModelStorageOption
>  - Add a program to test storing code that builds code model
>  - Add JavaType -> Type mapping for Block.Parameter
>  - Support storing code that builds code model
>  - Make TEXT the default storage mechanism for code models
>  - ... and 1 more: https://git.openjdk.org/babylon/compare/1158cd39...f8327908

Getting compiler errors due to lack of imports:

> === Output from failing command(s) repeated here ===
> * For target jdk_modules_jdk.incubator.code__the.jdk.incubator.code_batch:
> /Users/sandoz/Projects/jdk/babylon/src/jdk.incubator.code/share/classes/jdk/incubator/code/Op.java:540: error: cannot find symbol
>                 opMethod = method.getDeclaringClass().getDeclaredMethod(opMethodName, OpFactory.class,
>                                                                                       ^
>   symbol:   class OpFactory
>   location: class Op
> /Users/sandoz/Projects/jdk/babylon/src/jdk.incubator.code/share/classes/jdk/incubator/code/Op.java:542: error: cannot find symbol
>                 args = new Object[] {ExtendedOp.FACTORY, CoreTypeFactory.CORE_TYPE_FACTORY};
>                                                          ^
>   symbol:   variable CoreTypeFactory
>   location: class Op
> 2 errors
> 

After fixing those locally i get many test failures.

-------------

PR Comment: https://git.openjdk.org/babylon/pull/305#issuecomment-2682737088

From mabbay at openjdk.org  Tue Feb 25 17:42:58 2025
From: mabbay at openjdk.org (Mourad Abbay)
Date: Tue, 25 Feb 2025 17:42:58 GMT
Subject: [code-reflection] RFR: Support storing the code that builds the
 code model [v10]
In-Reply-To: <dexv8sQV_NcJsJIgMopd_M4wSWoZya9dU0ACwDtEL0E=.4ac0b80f-24fb-440f-a18d-bbfd07a681bc@github.com>
References: <dexv8sQV_NcJsJIgMopd_M4wSWoZya9dU0ACwDtEL0E=.4ac0b80f-24fb-440f-a18d-bbfd07a681bc@github.com>
Message-ID: <HYCazVQCWUB-oBKgIagYfAEyFMSz2qExK8l1arsOOG8=.817938e6-9af0-431c-a930-919830aabf1a@github.com>

> In this PR we allow the user to decide how to store the code model. The first option is `TEXT`, if this is selected, we store the textual representation of the code model. The second option is `CODE_BUILDR`, if this is selected, we store the code that builds the code model. All work done here is around the second option, because the first option is already supported.

Mourad Abbay has updated the pull request incrementally with one additional commit since the last revision:

  Add missing imports

-------------

Changes:
  - all: https://git.openjdk.org/babylon/pull/305/files
  - new: https://git.openjdk.org/babylon/pull/305/files/f8327908..e9fcd8c7

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=babylon&pr=305&range=09
 - incr: https://webrevs.openjdk.org/?repo=babylon&pr=305&range=08-09

  Stats: 2 lines in 1 file changed: 2 ins; 0 del; 0 mod
  Patch: https://git.openjdk.org/babylon/pull/305.diff
  Fetch: git fetch https://git.openjdk.org/babylon.git pull/305/head:pull/305

PR: https://git.openjdk.org/babylon/pull/305

From psandoz at openjdk.org  Tue Feb 25 19:14:26 2025
From: psandoz at openjdk.org (Paul Sandoz)
Date: Tue, 25 Feb 2025 19:14:26 GMT
Subject: [code-reflection] RFR: Use memory segment
Message-ID: <OE-H32XACfr1T25rewXZWhm6YmTlLh7GsVEYjzaHRjk=.1a655c9c-3822-4cb0-b0c5-ad0c37bc6e36@github.com>

Replace many uses of `ByteBuffer` use with idiomatic `MemorySegment` use. This also tends to make clearer the life cycle of segments. Generally in any ONNX APIs we design i think we should avoid exposing `*Buffer`. The "escape hatch" for accessing the data of a `Tensor` should be `MemorySegment`.

-------------

Commit messages:
 - Variable rename.
 - Use memory segment.

Changes: https://git.openjdk.org/babylon/pull/328/files
  Webrev: https://webrevs.openjdk.org/?repo=babylon&pr=328&range=00
  Stats: 84 lines in 6 files changed: 14 ins; 12 del; 58 mod
  Patch: https://git.openjdk.org/babylon/pull/328.diff
  Fetch: git fetch https://git.openjdk.org/babylon.git pull/328/head:pull/328

PR: https://git.openjdk.org/babylon/pull/328

From mcimadamore at openjdk.org  Wed Feb 26 10:18:20 2025
From: mcimadamore at openjdk.org (Maurizio Cimadamore)
Date: Wed, 26 Feb 2025 10:18:20 GMT
Subject: [code-reflection] RFR: Use memory segment
In-Reply-To: <OE-H32XACfr1T25rewXZWhm6YmTlLh7GsVEYjzaHRjk=.1a655c9c-3822-4cb0-b0c5-ad0c37bc6e36@github.com>
References: <OE-H32XACfr1T25rewXZWhm6YmTlLh7GsVEYjzaHRjk=.1a655c9c-3822-4cb0-b0c5-ad0c37bc6e36@github.com>
Message-ID: <G6efvGs9ZkZcGkOFEE9pzHl5eB-ju1Bf8eu3wesxe5w=.3246b616-0689-4648-9e63-f6a400ea1585@github.com>

On Tue, 25 Feb 2025 19:09:36 GMT, Paul Sandoz <psandoz at openjdk.org> wrote:

> Replace many uses of `ByteBuffer` use with idiomatic `MemorySegment` use. This also tends to make clearer the life cycle of segments. Generally in any ONNX APIs we design i think we should avoid exposing `*Buffer`. The "escape hatch" for accessing the data of a `Tensor` should be `MemorySegment`.

cr-examples/onnx/src/main/java/oracle/code/onnx/OnnxProtoBuilder.java line 320:

> 318:                 .buf.toByteArray();
> 319: //        OnnxProtoPrinter.printModel(ByteBuffer.wrap(bytes).order(ByteOrder.LITTLE_ENDIAN));
> 320:         return Arena.ofAuto().allocate(bytes.length).copyFrom(MemorySegment.ofArray(bytes)).asReadOnly();

I suggest using `allocateFrom(JAVA_BYTE, bytes)` to avoid the zeroing

-------------

PR Review Comment: https://git.openjdk.org/babylon/pull/328#discussion_r1971316325

From mcimadamore at openjdk.org  Wed Feb 26 10:23:08 2025
From: mcimadamore at openjdk.org (Maurizio Cimadamore)
Date: Wed, 26 Feb 2025 10:23:08 GMT
Subject: [code-reflection] RFR: Use memory segment
In-Reply-To: <62bPcZMfAUai8uQKGPzf_M4lli2drw7_tub_J4cOnPo=.fb0c3507-6ac8-4841-8081-58b19054483f@github.com>
References: <OE-H32XACfr1T25rewXZWhm6YmTlLh7GsVEYjzaHRjk=.1a655c9c-3822-4cb0-b0c5-ad0c37bc6e36@github.com>
 <62bPcZMfAUai8uQKGPzf_M4lli2drw7_tub_J4cOnPo=.fb0c3507-6ac8-4841-8081-58b19054483f@github.com>
Message-ID: <_HyR5NN7DJ6XLdMVu8zTea2ntiy-ybSBujSHJSXanLg=.c32179d2-3ef6-4fc1-b987-114e36aaa068@github.com>

On Wed, 26 Feb 2025 10:18:43 GMT, Maurizio Cimadamore <mcimadamore at openjdk.org> wrote:

>> Replace many uses of `ByteBuffer` use with idiomatic `MemorySegment` use. This also tends to make clearer the life cycle of segments. Generally in any ONNX APIs we design i think we should avoid exposing `*Buffer`. The "escape hatch" for accessing the data of a `Tensor` should be `MemorySegment`.
>
> cr-examples/onnx/src/main/java/oracle/code/onnx/OnnxProtoBuilder.java line 318:
> 
>> 316:                         .forEach(outputNames, (g, oName) -> g.output(new ValueInfoProto().name(oName))))
>> 317:                 .opset_import(new OperatorSetIdProto().version(OPSET_VERSION))
>> 318:                 .buf.toByteArray();
> 
> is `buf` a protobuf `ByteString` ? If so, note that ByteString supports `copyTo` a byte buffer. So a possibility would be to create the memory segment, then get a BB view of the segment, and then pass that view to the `copyTo` method. This should avoid the intermediate array creation (but cannot avoid the memory zeroing).

nvm - this seems to be an output stream

-------------

PR Review Comment: https://git.openjdk.org/babylon/pull/328#discussion_r1971324924

From mcimadamore at openjdk.org  Wed Feb 26 10:23:08 2025
From: mcimadamore at openjdk.org (Maurizio Cimadamore)
Date: Wed, 26 Feb 2025 10:23:08 GMT
Subject: [code-reflection] RFR: Use memory segment
In-Reply-To: <OE-H32XACfr1T25rewXZWhm6YmTlLh7GsVEYjzaHRjk=.1a655c9c-3822-4cb0-b0c5-ad0c37bc6e36@github.com>
References: <OE-H32XACfr1T25rewXZWhm6YmTlLh7GsVEYjzaHRjk=.1a655c9c-3822-4cb0-b0c5-ad0c37bc6e36@github.com>
Message-ID: <62bPcZMfAUai8uQKGPzf_M4lli2drw7_tub_J4cOnPo=.fb0c3507-6ac8-4841-8081-58b19054483f@github.com>

On Tue, 25 Feb 2025 19:09:36 GMT, Paul Sandoz <psandoz at openjdk.org> wrote:

> Replace many uses of `ByteBuffer` use with idiomatic `MemorySegment` use. This also tends to make clearer the life cycle of segments. Generally in any ONNX APIs we design i think we should avoid exposing `*Buffer`. The "escape hatch" for accessing the data of a `Tensor` should be `MemorySegment`.

Looks good -- I was similarly wondering why not using segments more directly, but I was not sure about the connection with the protobuf which I imagine is more BB-oriented.

cr-examples/onnx/src/main/java/oracle/code/onnx/OnnxProtoBuilder.java line 318:

> 316:                         .forEach(outputNames, (g, oName) -> g.output(new ValueInfoProto().name(oName))))
> 317:                 .opset_import(new OperatorSetIdProto().version(OPSET_VERSION))
> 318:                 .buf.toByteArray();

is `buf` a protobuf `ByteString` ? If so, note that ByteString supports `copyTo` a byte buffer. So a possibility would be to create the memory segment, then get a BB view of the segment, and then pass that view to the `copyTo` method. This should avoid the intermediate array creation (but cannot avoid the memory zeroing).

-------------

Marked as reviewed by mcimadamore (Reviewer).

PR Review: https://git.openjdk.org/babylon/pull/328#pullrequestreview-2643937822
PR Review Comment: https://git.openjdk.org/babylon/pull/328#discussion_r1971322247

From asotona at openjdk.org  Wed Feb 26 11:58:22 2025
From: asotona at openjdk.org (Adam Sotona)
Date: Wed, 26 Feb 2025 11:58:22 GMT
Subject: [code-reflection] RFR: Use memory segment
In-Reply-To: <OE-H32XACfr1T25rewXZWhm6YmTlLh7GsVEYjzaHRjk=.1a655c9c-3822-4cb0-b0c5-ad0c37bc6e36@github.com>
References: <OE-H32XACfr1T25rewXZWhm6YmTlLh7GsVEYjzaHRjk=.1a655c9c-3822-4cb0-b0c5-ad0c37bc6e36@github.com>
Message-ID: <vdve1fGk5BcAWVHH99H5_Gbl-SIrFUc3USQ1IoJs1Vo=.16c4a1cd-a535-48d6-b590-0c6151576eaa@github.com>

On Tue, 25 Feb 2025 19:09:36 GMT, Paul Sandoz <psandoz at openjdk.org> wrote:

> Replace many uses of `ByteBuffer` use with idiomatic `MemorySegment` use. This also tends to make clearer the life cycle of segments. Generally in any ONNX APIs we design i think we should avoid exposing `*Buffer`. The "escape hatch" for accessing the data of a `Tensor` should be `MemorySegment`.

I've already addressed some of the BB removals in https://github.com/openjdk/babylon/pull/326

-------------

PR Comment: https://git.openjdk.org/babylon/pull/328#issuecomment-2684749558

From asotona at openjdk.org  Wed Feb 26 12:34:47 2025
From: asotona at openjdk.org (Adam Sotona)
Date: Wed, 26 Feb 2025 12:34:47 GMT
Subject: [code-reflection] RFR: Use memory segment
In-Reply-To: <G6efvGs9ZkZcGkOFEE9pzHl5eB-ju1Bf8eu3wesxe5w=.3246b616-0689-4648-9e63-f6a400ea1585@github.com>
References: <OE-H32XACfr1T25rewXZWhm6YmTlLh7GsVEYjzaHRjk=.1a655c9c-3822-4cb0-b0c5-ad0c37bc6e36@github.com>
 <G6efvGs9ZkZcGkOFEE9pzHl5eB-ju1Bf8eu3wesxe5w=.3246b616-0689-4648-9e63-f6a400ea1585@github.com>
Message-ID: <veSTuWur-y5ky4P_zOrMrogWkGG_W9iimfPX1kWxalk=.f05eb1fb-9fc6-4ca4-91a5-59234fb53f33@github.com>

On Wed, 26 Feb 2025 10:15:00 GMT, Maurizio Cimadamore <mcimadamore at openjdk.org> wrote:

>> Replace many uses of `ByteBuffer` use with idiomatic `MemorySegment` use. This also tends to make clearer the life cycle of segments. Generally in any ONNX APIs we design i think we should avoid exposing `*Buffer`. The "escape hatch" for accessing the data of a `Tensor` should be `MemorySegment`.
>
> cr-examples/onnx/src/main/java/oracle/code/onnx/OnnxProtoBuilder.java line 320:
> 
>> 318:                 .buf.toByteArray();
>> 319: //        OnnxProtoPrinter.printModel(ByteBuffer.wrap(bytes).order(ByteOrder.LITTLE_ENDIAN));
>> 320:         return Arena.ofAuto().allocate(bytes.length).copyFrom(MemorySegment.ofArray(bytes)).asReadOnly();
> 
> I suggest using `allocateFrom(JAVA_BYTE, bytes)` to avoid the zeroing

allocateFrom creates on-heap memory segment

-------------

PR Review Comment: https://git.openjdk.org/babylon/pull/328#discussion_r1971504306

From mcimadamore at openjdk.org  Wed Feb 26 12:48:11 2025
From: mcimadamore at openjdk.org (Maurizio Cimadamore)
Date: Wed, 26 Feb 2025 12:48:11 GMT
Subject: [code-reflection] RFR: Use memory segment
In-Reply-To: <veSTuWur-y5ky4P_zOrMrogWkGG_W9iimfPX1kWxalk=.f05eb1fb-9fc6-4ca4-91a5-59234fb53f33@github.com>
References: <OE-H32XACfr1T25rewXZWhm6YmTlLh7GsVEYjzaHRjk=.1a655c9c-3822-4cb0-b0c5-ad0c37bc6e36@github.com>
 <G6efvGs9ZkZcGkOFEE9pzHl5eB-ju1Bf8eu3wesxe5w=.3246b616-0689-4648-9e63-f6a400ea1585@github.com>
 <veSTuWur-y5ky4P_zOrMrogWkGG_W9iimfPX1kWxalk=.f05eb1fb-9fc6-4ca4-91a5-59234fb53f33@github.com>
Message-ID: <XetmgOWirB_QcqQ-WYwdG1R1zKv4urGOAkYdAGWQfg4=.c0aeb0bc-c155-4827-a859-18e714f63e67@github.com>

On Wed, 26 Feb 2025 12:31:22 GMT, Adam Sotona <asotona at openjdk.org> wrote:

>> cr-examples/onnx/src/main/java/oracle/code/onnx/OnnxProtoBuilder.java line 320:
>> 
>>> 318:                 .buf.toByteArray();
>>> 319: //        OnnxProtoPrinter.printModel(ByteBuffer.wrap(bytes).order(ByteOrder.LITTLE_ENDIAN));
>>> 320:         return Arena.ofAuto().allocate(bytes.length).copyFrom(MemorySegment.ofArray(bytes)).asReadOnly();
>> 
>> I suggest using `allocateFrom(JAVA_BYTE, bytes)` to avoid the zeroing
>
> allocateFrom creates on-heap memory segment

No - `allocateFrom` creates a new off-heap segment when invoked on an `Arena`. `Arena` is a segment allocator that always allocates off-heap. The `allocateFrom` functions are explicitly designed for use cases when you want to allocate and copy existing data on top of the new allocated segment -- in which case the FFM knows that zeroing the allocated segment is redundant. I think that should be used here.

-------------

PR Review Comment: https://git.openjdk.org/babylon/pull/328#discussion_r1971524785

From mcimadamore at openjdk.org  Wed Feb 26 12:53:19 2025
From: mcimadamore at openjdk.org (Maurizio Cimadamore)
Date: Wed, 26 Feb 2025 12:53:19 GMT
Subject: [code-reflection] RFR: Use memory segment
In-Reply-To: <XetmgOWirB_QcqQ-WYwdG1R1zKv4urGOAkYdAGWQfg4=.c0aeb0bc-c155-4827-a859-18e714f63e67@github.com>
References: <OE-H32XACfr1T25rewXZWhm6YmTlLh7GsVEYjzaHRjk=.1a655c9c-3822-4cb0-b0c5-ad0c37bc6e36@github.com>
 <G6efvGs9ZkZcGkOFEE9pzHl5eB-ju1Bf8eu3wesxe5w=.3246b616-0689-4648-9e63-f6a400ea1585@github.com>
 <veSTuWur-y5ky4P_zOrMrogWkGG_W9iimfPX1kWxalk=.f05eb1fb-9fc6-4ca4-91a5-59234fb53f33@github.com>
 <XetmgOWirB_QcqQ-WYwdG1R1zKv4urGOAkYdAGWQfg4=.c0aeb0bc-c155-4827-a859-18e714f63e67@github.com>
Message-ID: <YcEa5VAmx8o_qRqT51FdZHPp0CNKOOdUrh_TkRGzqKk=.933b58ce-0063-4df6-8602-d730c12cb201@github.com>

On Wed, 26 Feb 2025 12:45:59 GMT, Maurizio Cimadamore <mcimadamore at openjdk.org> wrote:

> No - `allocateFrom` creates a new off-heap segment when invoked on an `Arena`. `Arena` is a segment allocator that always allocates off-heap. The `allocateFrom` functions are explicitly designed for use cases when you want to allocate and copy existing data on top of the new allocated segment -- in which case the FFM knows that zeroing the allocated segment is redundant. I think that should be used here.

Note: what I'm pointing out is that, performance-wise, `allocateFrom` is not the same as `allocate` + `copyFrom`. Internally, `allocateFrom` will still create a temporary on-heap view of the byte array (w/o allocating a new heap array though) -- but that will be passed to an allocation routine that does allocation + copy w/o zeroring. If you call `allocate` zeroing will happen no matter what.

-------------

PR Review Comment: https://git.openjdk.org/babylon/pull/328#discussion_r1971529704

From mcimadamore at openjdk.org  Wed Feb 26 12:53:19 2025
From: mcimadamore at openjdk.org (Maurizio Cimadamore)
Date: Wed, 26 Feb 2025 12:53:19 GMT
Subject: [code-reflection] RFR: Use memory segment
In-Reply-To: <YcEa5VAmx8o_qRqT51FdZHPp0CNKOOdUrh_TkRGzqKk=.933b58ce-0063-4df6-8602-d730c12cb201@github.com>
References: <OE-H32XACfr1T25rewXZWhm6YmTlLh7GsVEYjzaHRjk=.1a655c9c-3822-4cb0-b0c5-ad0c37bc6e36@github.com>
 <G6efvGs9ZkZcGkOFEE9pzHl5eB-ju1Bf8eu3wesxe5w=.3246b616-0689-4648-9e63-f6a400ea1585@github.com>
 <veSTuWur-y5ky4P_zOrMrogWkGG_W9iimfPX1kWxalk=.f05eb1fb-9fc6-4ca4-91a5-59234fb53f33@github.com>
 <XetmgOWirB_QcqQ-WYwdG1R1zKv4urGOAkYdAGWQfg4=.c0aeb0bc-c155-4827-a859-18e714f63e67@github.com>
 <YcEa5VAmx8o_qRqT51FdZHPp0CNKOOdUrh_TkRGzqKk=.933b58ce-0063-4df6-8602-d730c12cb201@github.com>
Message-ID: <Mb3ZcWwnGyu5UyKoSMyNBY6u1wvBzvEaG5WODZkKbIw=.53056212-974c-4622-ac29-495b910214ae@github.com>

On Wed, 26 Feb 2025 12:49:29 GMT, Maurizio Cimadamore <mcimadamore at openjdk.org> wrote:

>> No - `allocateFrom` creates a new off-heap segment when invoked on an `Arena`. `Arena` is a segment allocator that always allocates off-heap. The `allocateFrom` functions are explicitly designed for use cases when you want to allocate and copy existing data on top of the new allocated segment -- in which case the FFM knows that zeroing the allocated segment is redundant. I think that should be used here.
>
>> No - `allocateFrom` creates a new off-heap segment when invoked on an `Arena`. `Arena` is a segment allocator that always allocates off-heap. The `allocateFrom` functions are explicitly designed for use cases when you want to allocate and copy existing data on top of the new allocated segment -- in which case the FFM knows that zeroing the allocated segment is redundant. I think that should be used here.
> 
> Note: what I'm pointing out is that, performance-wise, `allocateFrom` is not the same as `allocate` + `copyFrom`. Internally, `allocateFrom` will still create a temporary on-heap view of the byte array (w/o allocating a new heap array though) -- but that will be passed to an allocation routine that does allocation + copy w/o zeroring. If you call `allocate` zeroing will happen no matter what.

And... final note -- ByteBuffer doesn't have an `allocateFrom` variant, so zeroing is unavoidable there -- but with segments we can do better.

-------------

PR Review Comment: https://git.openjdk.org/babylon/pull/328#discussion_r1971531526

From asotona at openjdk.org  Wed Feb 26 13:45:02 2025
From: asotona at openjdk.org (Adam Sotona)
Date: Wed, 26 Feb 2025 13:45:02 GMT
Subject: [code-reflection] RFR: Use memory segment
In-Reply-To: <Mb3ZcWwnGyu5UyKoSMyNBY6u1wvBzvEaG5WODZkKbIw=.53056212-974c-4622-ac29-495b910214ae@github.com>
References: <OE-H32XACfr1T25rewXZWhm6YmTlLh7GsVEYjzaHRjk=.1a655c9c-3822-4cb0-b0c5-ad0c37bc6e36@github.com>
 <G6efvGs9ZkZcGkOFEE9pzHl5eB-ju1Bf8eu3wesxe5w=.3246b616-0689-4648-9e63-f6a400ea1585@github.com>
 <veSTuWur-y5ky4P_zOrMrogWkGG_W9iimfPX1kWxalk=.f05eb1fb-9fc6-4ca4-91a5-59234fb53f33@github.com>
 <XetmgOWirB_QcqQ-WYwdG1R1zKv4urGOAkYdAGWQfg4=.c0aeb0bc-c155-4827-a859-18e714f63e67@github.com>
 <YcEa5VAmx8o_qRqT51FdZHPp0CNKOOdUrh_TkRGzqKk=.933b58ce-0063-4df6-8602-d730c12cb201@github.com>
 <Mb3ZcWwnGyu5UyKoSMyNBY6u1wvBzvEaG5WODZkKbIw=.53056212-974c-4622-ac29-495b910214ae@github.com>
Message-ID: <40xu9mNYmn9fWUdYc0wKGyDModDl_44sk79gVT75pyE=.3eaa47f7-baa8-4ac5-bd06-edb2ba8d9de5@github.com>

On Wed, 26 Feb 2025 12:50:50 GMT, Maurizio Cimadamore <mcimadamore at openjdk.org> wrote:

>>> No - `allocateFrom` creates a new off-heap segment when invoked on an `Arena`. `Arena` is a segment allocator that always allocates off-heap. The `allocateFrom` functions are explicitly designed for use cases when you want to allocate and copy existing data on top of the new allocated segment -- in which case the FFM knows that zeroing the allocated segment is redundant. I think that should be used here.
>> 
>> Note: what I'm pointing out is that, performance-wise, `allocateFrom` is not the same as `allocate` + `copyFrom`. Internally, `allocateFrom` will still create a temporary on-heap view of the byte array (w/o allocating a new heap array though) -- but that will be passed to an allocation routine that does allocation + copy w/o zeroring. If you call `allocate` zeroing will happen no matter what.
>
> And... final note -- ByteBuffer doesn't have an `allocateFrom` variant, so zeroing is unavoidable there -- but with segments we can do better.

Oh, I mixed it with MemorySegment::ofArray

-------------

PR Review Comment: https://git.openjdk.org/babylon/pull/328#discussion_r1971610700

From asotona at openjdk.org  Wed Feb 26 15:56:40 2025
From: asotona at openjdk.org (Adam Sotona)
Date: Wed, 26 Feb 2025 15:56:40 GMT
Subject: [code-reflection] RFR: Onnx subgraphs,
 lambda execution and BB reduction [v2]
In-Reply-To: <CYyR9Ypl-frjAUum631CITskwAXBevLVjgRbKqwmrdU=.026f0f2b-f941-45fc-bfe3-1c0f899912db@github.com>
References: <CYyR9Ypl-frjAUum631CITskwAXBevLVjgRbKqwmrdU=.026f0f2b-f941-45fc-bfe3-1c0f899912db@github.com>
Message-ID: <_qxCr7w4c7vR7bdSFPNIhtrtmwTUfXUsaRl_s_OF21A=.3a5d24e4-f850-4b4e-b898-a1fd0ece4929@github.com>

> Onnx subgraphs, lambda execution and BB reduction

Adam Sotona has updated the pull request incrementally with one additional commit since the last revision:

  implemented OnnxRuntime::execute(Quotable & Supplier<Tensor>)

-------------

Changes:
  - all: https://git.openjdk.org/babylon/pull/326/files
  - new: https://git.openjdk.org/babylon/pull/326/files/8b485862..84715103

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=babylon&pr=326&range=01
 - incr: https://webrevs.openjdk.org/?repo=babylon&pr=326&range=00-01

  Stats: 139 lines in 6 files changed: 40 ins; 22 del; 77 mod
  Patch: https://git.openjdk.org/babylon/pull/326.diff
  Fetch: git fetch https://git.openjdk.org/babylon.git pull/326/head:pull/326

PR: https://git.openjdk.org/babylon/pull/326

From asotona at openjdk.org  Wed Feb 26 16:42:29 2025
From: asotona at openjdk.org (Adam Sotona)
Date: Wed, 26 Feb 2025 16:42:29 GMT
Subject: [code-reflection] RFR: Onnx subgraphs,
 lambda execution and BB reduction [v3]
In-Reply-To: <CYyR9Ypl-frjAUum631CITskwAXBevLVjgRbKqwmrdU=.026f0f2b-f941-45fc-bfe3-1c0f899912db@github.com>
References: <CYyR9Ypl-frjAUum631CITskwAXBevLVjgRbKqwmrdU=.026f0f2b-f941-45fc-bfe3-1c0f899912db@github.com>
Message-ID: <F98Fd4cuvFmrV-Q02PWWoDh3AkeahpSHFXYsOUb6OQM=.6cb6f9fe-4b9a-430b-a772-83b7511c933a@github.com>

> Onnx subgraphs, lambda execution and BB reduction

Adam Sotona has updated the pull request incrementally with one additional commit since the last revision:

  removal of Tensor::asByteBuffer

-------------

Changes:
  - all: https://git.openjdk.org/babylon/pull/326/files
  - new: https://git.openjdk.org/babylon/pull/326/files/84715103..b55fc2b1

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=babylon&pr=326&range=02
 - incr: https://webrevs.openjdk.org/?repo=babylon&pr=326&range=01-02

  Stats: 54 lines in 3 files changed: 27 ins; 20 del; 7 mod
  Patch: https://git.openjdk.org/babylon/pull/326.diff
  Fetch: git fetch https://git.openjdk.org/babylon.git pull/326/head:pull/326

PR: https://git.openjdk.org/babylon/pull/326

From psandoz at openjdk.org  Wed Feb 26 17:06:11 2025
From: psandoz at openjdk.org (Paul Sandoz)
Date: Wed, 26 Feb 2025 17:06:11 GMT
Subject: [code-reflection] RFR: Use memory segment
In-Reply-To: <OE-H32XACfr1T25rewXZWhm6YmTlLh7GsVEYjzaHRjk=.1a655c9c-3822-4cb0-b0c5-ad0c37bc6e36@github.com>
References: <OE-H32XACfr1T25rewXZWhm6YmTlLh7GsVEYjzaHRjk=.1a655c9c-3822-4cb0-b0c5-ad0c37bc6e36@github.com>
Message-ID: <2KHckCKmQziamTNm8WJ0iFxG_nHGlGocfTGZ6pYpJ9Y=.0a1d7fbe-dd2a-4ffd-bbd3-c2fff251ffb2@github.com>

On Tue, 25 Feb 2025 19:09:36 GMT, Paul Sandoz <psandoz at openjdk.org> wrote:

> Replace many uses of `ByteBuffer` use with idiomatic `MemorySegment` use. This also tends to make clearer the life cycle of segments. Generally in any ONNX APIs we design i think we should avoid exposing `*Buffer`. The "escape hatch" for accessing the data of a `Tensor` should be `MemorySegment`.

Superseded by #326

-------------

PR Comment: https://git.openjdk.org/babylon/pull/328#issuecomment-2685671732

From psandoz at openjdk.org  Wed Feb 26 17:06:11 2025
From: psandoz at openjdk.org (Paul Sandoz)
Date: Wed, 26 Feb 2025 17:06:11 GMT
Subject: [code-reflection] Withdrawn: Use memory segment
In-Reply-To: <OE-H32XACfr1T25rewXZWhm6YmTlLh7GsVEYjzaHRjk=.1a655c9c-3822-4cb0-b0c5-ad0c37bc6e36@github.com>
References: <OE-H32XACfr1T25rewXZWhm6YmTlLh7GsVEYjzaHRjk=.1a655c9c-3822-4cb0-b0c5-ad0c37bc6e36@github.com>
Message-ID: <XHKdtpZnNY-NESPyLJFr3g1jHMtsmy8S4G2ULqC-AP8=.ccf1f8f6-cb8e-414e-b431-91ad7f300d63@github.com>

On Tue, 25 Feb 2025 19:09:36 GMT, Paul Sandoz <psandoz at openjdk.org> wrote:

> Replace many uses of `ByteBuffer` use with idiomatic `MemorySegment` use. This also tends to make clearer the life cycle of segments. Generally in any ONNX APIs we design i think we should avoid exposing `*Buffer`. The "escape hatch" for accessing the data of a `Tensor` should be `MemorySegment`.

This pull request has been closed without being integrated.

-------------

PR: https://git.openjdk.org/babylon/pull/328

From asotona at openjdk.org  Wed Feb 26 17:24:51 2025
From: asotona at openjdk.org (Adam Sotona)
Date: Wed, 26 Feb 2025 17:24:51 GMT
Subject: [code-reflection] RFR: Onnx subgraphs,
 lambda execution and BB removal [v4]
In-Reply-To: <CYyR9Ypl-frjAUum631CITskwAXBevLVjgRbKqwmrdU=.026f0f2b-f941-45fc-bfe3-1c0f899912db@github.com>
References: <CYyR9Ypl-frjAUum631CITskwAXBevLVjgRbKqwmrdU=.026f0f2b-f941-45fc-bfe3-1c0f899912db@github.com>
Message-ID: <38S4HYltQtbD3R9nlFsnAvyxS8lJTzFC76uPjbOKce8=.06179cc1-a371-41d8-8a0c-fc358155d27e@github.com>

> Onnx subgraphs, lambda execution and BB removal

Adam Sotona has updated the pull request incrementally with one additional commit since the last revision:

  more BB cleanup

-------------

Changes:
  - all: https://git.openjdk.org/babylon/pull/326/files
  - new: https://git.openjdk.org/babylon/pull/326/files/b55fc2b1..07202658

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=babylon&pr=326&range=03
 - incr: https://webrevs.openjdk.org/?repo=babylon&pr=326&range=02-03

  Stats: 37 lines in 2 files changed: 1 ins; 25 del; 11 mod
  Patch: https://git.openjdk.org/babylon/pull/326.diff
  Fetch: git fetch https://git.openjdk.org/babylon.git pull/326/head:pull/326

PR: https://git.openjdk.org/babylon/pull/326

From psandoz at openjdk.org  Wed Feb 26 17:34:07 2025
From: psandoz at openjdk.org (Paul Sandoz)
Date: Wed, 26 Feb 2025 17:34:07 GMT
Subject: [code-reflection] RFR: Onnx subgraphs,
 lambda execution and BB removal [v3]
In-Reply-To: <F98Fd4cuvFmrV-Q02PWWoDh3AkeahpSHFXYsOUb6OQM=.6cb6f9fe-4b9a-430b-a772-83b7511c933a@github.com>
References: <CYyR9Ypl-frjAUum631CITskwAXBevLVjgRbKqwmrdU=.026f0f2b-f941-45fc-bfe3-1c0f899912db@github.com>
 <F98Fd4cuvFmrV-Q02PWWoDh3AkeahpSHFXYsOUb6OQM=.6cb6f9fe-4b9a-430b-a772-83b7511c933a@github.com>
Message-ID: <8t0wJ9s9nDIpxVuKECamyZc9NXkMIsM0JF0VfMuxZ4U=.bd92b811-28c7-4731-9b08-3550d2d3f129@github.com>

On Wed, 26 Feb 2025 16:42:29 GMT, Adam Sotona <asotona at openjdk.org> wrote:

>> Onnx subgraphs, lambda execution and BB removal
>
> Adam Sotona has updated the pull request incrementally with one additional commit since the last revision:
> 
>   removal of Tensor::asByteBuffer

Nice cleanup. There is a bunch of raw type usage for `Tensor` we should cleanup.

cr-examples/onnx/src/test/java/oracle/code/onnx/MNISTDemo.java line 44:

> 42:     public static float[] loadConstant(String resource) {
> 43:         try {
> 44:             return MemorySegment.ofArray(MNISTDemo.class.getResourceAsStream(resource).readAllBytes())

We should use try-with-resources on `MNISTDemo.class.getResourceAsStream(resource)` so the resource is closed.

cr-examples/onnx/src/test/java/oracle/code/onnx/MNISTDemo.java line 52:

> 50: 
> 51:     public static Tensor<Float> cnn(Tensor<Float> inputImage) {
> 52:         return OnnxRuntime.execute(() -> {

I think it's preferable to eventually express the following:

OnnxRuntime.execute(() -> cnn(inputImage));

i.e. i would like to see that expression embedded in the drawing code.

This generally requires that we support invocations to other ONNX functions (or we inline when transforming), which we anyway should eventually work out.

For the case of a single method invocation it should be possible to take a short cut for now (for clues see `LambdaOp.methodReference` which we could refine). So i would recommend following up on the short cut and leaning into that so we can pull out the execute from the `cnn` method.

With this approach we still need to eventually workout the difference between initializers and inputs. I believe the former is the generally accepted way of representing constant input rather than more directly as constants in the graph.

-------------

PR Review: https://git.openjdk.org/babylon/pull/326#pullrequestreview-2645257454
PR Review Comment: https://git.openjdk.org/babylon/pull/326#discussion_r1972040058
PR Review Comment: https://git.openjdk.org/babylon/pull/326#discussion_r1972020840

From mabbay at openjdk.org  Wed Feb 26 18:06:44 2025
From: mabbay at openjdk.org (Mourad Abbay)
Date: Wed, 26 Feb 2025 18:06:44 GMT
Subject: [code-reflection] RFR: Support storing the code that builds the
 code model [v11]
In-Reply-To: <dexv8sQV_NcJsJIgMopd_M4wSWoZya9dU0ACwDtEL0E=.4ac0b80f-24fb-440f-a18d-bbfd07a681bc@github.com>
References: <dexv8sQV_NcJsJIgMopd_M4wSWoZya9dU0ACwDtEL0E=.4ac0b80f-24fb-440f-a18d-bbfd07a681bc@github.com>
Message-ID: <KT1Hs6fGZAH1tCOqT_MZ4DoaDBc0FkS7qFh4kKkAwL8=.13476382-ff9a-4f29-b1bf-8f0b8ebc71fe@github.com>

> In this PR we allow the user to decide how to store the code model. The first option is `TEXT`, if this is selected, we store the textual representation of the code model. The second option is `CODE_BUILDR`, if this is selected, we store the code that builds the code model. All work done here is around the second option, because the first option is already supported.

Mourad Abbay has updated the pull request incrementally with one additional commit since the last revision:

  Ensure that block params are inserted in the correct order

-------------

Changes:
  - all: https://git.openjdk.org/babylon/pull/305/files
  - new: https://git.openjdk.org/babylon/pull/305/files/e9fcd8c7..c2323661

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=babylon&pr=305&range=10
 - incr: https://webrevs.openjdk.org/?repo=babylon&pr=305&range=09-10

  Stats: 3 lines in 1 file changed: 2 ins; 0 del; 1 mod
  Patch: https://git.openjdk.org/babylon/pull/305.diff
  Fetch: git fetch https://git.openjdk.org/babylon.git pull/305/head:pull/305

PR: https://git.openjdk.org/babylon/pull/305

From asotona at openjdk.org  Wed Feb 26 18:42:01 2025
From: asotona at openjdk.org (Adam Sotona)
Date: Wed, 26 Feb 2025 18:42:01 GMT
Subject: [code-reflection] RFR: Onnx subgraphs,
 lambda execution and BB removal [v3]
In-Reply-To: <8t0wJ9s9nDIpxVuKECamyZc9NXkMIsM0JF0VfMuxZ4U=.bd92b811-28c7-4731-9b08-3550d2d3f129@github.com>
References: <CYyR9Ypl-frjAUum631CITskwAXBevLVjgRbKqwmrdU=.026f0f2b-f941-45fc-bfe3-1c0f899912db@github.com>
 <F98Fd4cuvFmrV-Q02PWWoDh3AkeahpSHFXYsOUb6OQM=.6cb6f9fe-4b9a-430b-a772-83b7511c933a@github.com>
 <8t0wJ9s9nDIpxVuKECamyZc9NXkMIsM0JF0VfMuxZ4U=.bd92b811-28c7-4731-9b08-3550d2d3f129@github.com>
Message-ID: <2MFL3z7W4-2OIqADGeCe3KpY31bP14sPtzHbkMgaiZE=.e73413a3-e93a-4e37-a177-c33e10f952d9@github.com>

On Wed, 26 Feb 2025 17:19:28 GMT, Paul Sandoz <psandoz at openjdk.org> wrote:

>> Adam Sotona has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   removal of Tensor::asByteBuffer
>
> cr-examples/onnx/src/test/java/oracle/code/onnx/MNISTDemo.java line 52:
> 
>> 50: 
>> 51:     public static Tensor<Float> cnn(Tensor<Float> inputImage) {
>> 52:         return OnnxRuntime.execute(() -> {
> 
> I think it's preferable to eventually express the following:
> 
> OnnxRuntime.execute(() -> cnn(inputImage));
> 
> i.e. i would like to see that expression embedded in the drawing code.
> 
> This generally requires that we support invocations to other ONNX functions (or we inline when transforming), which we anyway should eventually work out.
> 
> For the case of a single method invocation it should be possible to take a short cut for now (for clues see `LambdaOp.methodReference` which we could refine). So i would recommend following up on the short cut and leaning into that so we can pull out the execute from the `cnn` method.
> 
> With this approach we still need to eventually workout the difference between initializers and inputs. I believe the former is the generally accepted way of representing constant input rather than more directly as constants in the graph.

Initializers are now on top of my ToDo list. They are technically just differently serialized Constant ops. I'm still not sure how we should model them - maybe Initializer op?

-------------

PR Review Comment: https://git.openjdk.org/babylon/pull/326#discussion_r1972146598

From asotona at openjdk.org  Wed Feb 26 18:45:06 2025
From: asotona at openjdk.org (Adam Sotona)
Date: Wed, 26 Feb 2025 18:45:06 GMT
Subject: [code-reflection] RFR: Onnx subgraphs,
 lambda execution and BB removal [v3]
In-Reply-To: <2MFL3z7W4-2OIqADGeCe3KpY31bP14sPtzHbkMgaiZE=.e73413a3-e93a-4e37-a177-c33e10f952d9@github.com>
References: <CYyR9Ypl-frjAUum631CITskwAXBevLVjgRbKqwmrdU=.026f0f2b-f941-45fc-bfe3-1c0f899912db@github.com>
 <F98Fd4cuvFmrV-Q02PWWoDh3AkeahpSHFXYsOUb6OQM=.6cb6f9fe-4b9a-430b-a772-83b7511c933a@github.com>
 <8t0wJ9s9nDIpxVuKECamyZc9NXkMIsM0JF0VfMuxZ4U=.bd92b811-28c7-4731-9b08-3550d2d3f129@github.com>
 <2MFL3z7W4-2OIqADGeCe3KpY31bP14sPtzHbkMgaiZE=.e73413a3-e93a-4e37-a177-c33e10f952d9@github.com>
Message-ID: <u-_qrfGqF1aRgW5HaKgr943-lz-W0MBkZMhu3tPnHvg=.20368a53-f139-4cb6-9d47-7aea889e5060@github.com>

On Wed, 26 Feb 2025 18:39:09 GMT, Adam Sotona <asotona at openjdk.org> wrote:

>> cr-examples/onnx/src/test/java/oracle/code/onnx/MNISTDemo.java line 52:
>> 
>>> 50: 
>>> 51:     public static Tensor<Float> cnn(Tensor<Float> inputImage) {
>>> 52:         return OnnxRuntime.execute(() -> {
>> 
>> I think it's preferable to eventually express the following:
>> 
>> OnnxRuntime.execute(() -> cnn(inputImage));
>> 
>> i.e. i would like to see that expression embedded in the drawing code.
>> 
>> This generally requires that we support invocations to other ONNX functions (or we inline when transforming), which we anyway should eventually work out.
>> 
>> For the case of a single method invocation it should be possible to take a short cut for now (for clues see `LambdaOp.methodReference` which we could refine). So i would recommend following up on the short cut and leaning into that so we can pull out the execute from the `cnn` method.
>> 
>> With this approach we still need to eventually workout the difference between initializers and inputs. I believe the former is the generally accepted way of representing constant input rather than more directly as constants in the graph.
>
> Initializers are now on top of my ToDo list. They are technically just differently serialized Constant ops. I'm still not sure how we should model them - maybe Initializer op?

Method call in a lambda is another level of complexity. I guess OnnxTransformer should inline it?

-------------

PR Review Comment: https://git.openjdk.org/babylon/pull/326#discussion_r1972150336

From asotona at openjdk.org  Wed Feb 26 18:52:41 2025
From: asotona at openjdk.org (Adam Sotona)
Date: Wed, 26 Feb 2025 18:52:41 GMT
Subject: [code-reflection] RFR: Onnx subgraphs,
 lambda execution and BB removal [v5]
In-Reply-To: <CYyR9Ypl-frjAUum631CITskwAXBevLVjgRbKqwmrdU=.026f0f2b-f941-45fc-bfe3-1c0f899912db@github.com>
References: <CYyR9Ypl-frjAUum631CITskwAXBevLVjgRbKqwmrdU=.026f0f2b-f941-45fc-bfe3-1c0f899912db@github.com>
Message-ID: <I1WeURwrBImOuQ_RSP8nPNCzgjOeXGN8nv6rJ_OICx0=.9e2d5e4a-7b54-4436-ac6d-1415473da6af@github.com>

> Onnx subgraphs, lambda execution and BB removal

Adam Sotona has updated the pull request incrementally with one additional commit since the last revision:

  fixed try with resources in MNISTDemo

-------------

Changes:
  - all: https://git.openjdk.org/babylon/pull/326/files
  - new: https://git.openjdk.org/babylon/pull/326/files/07202658..7627ec00

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=babylon&pr=326&range=04
 - incr: https://webrevs.openjdk.org/?repo=babylon&pr=326&range=03-04

  Stats: 2 lines in 1 file changed: 0 ins; 0 del; 2 mod
  Patch: https://git.openjdk.org/babylon/pull/326.diff
  Fetch: git fetch https://git.openjdk.org/babylon.git pull/326/head:pull/326

PR: https://git.openjdk.org/babylon/pull/326

From asotona at openjdk.org  Wed Feb 26 18:59:16 2025
From: asotona at openjdk.org (Adam Sotona)
Date: Wed, 26 Feb 2025 18:59:16 GMT
Subject: git: openjdk/babylon: code-reflection: Onnx subgraphs,
 lambda execution and BB removal
Message-ID: <b7f7d78b-ce1d-42fd-b3ea-b88161e8fc29@openjdk.org>

Changeset: 80934b2b
Branch: code-reflection
Author:    Adam Sotona <asotona at openjdk.org>
Date:      2025-02-26 18:57:29 +0000
URL:       https://git.openjdk.org/babylon/commit/80934b2bd934523dd129fda6b38029a8956f8d51

Onnx subgraphs, lambda execution and BB removal

! cr-examples/onnx/src/main/java/oracle/code/onnx/OnnxInterpreter.java
! cr-examples/onnx/src/main/java/oracle/code/onnx/OnnxProtoBuilder.java
! cr-examples/onnx/src/main/java/oracle/code/onnx/OnnxRuntime.java
! cr-examples/onnx/src/main/java/oracle/code/onnx/Tensor.java
! cr-examples/onnx/src/test/java/oracle/code/onnx/CNNTest.java
! cr-examples/onnx/src/test/java/oracle/code/onnx/MNISTDemo.java
! cr-examples/onnx/src/test/java/oracle/code/onnx/RuntimeTest.java
! cr-examples/onnx/src/test/java/oracle/code/onnx/SimpleTest.java


From psandoz at openjdk.org  Wed Feb 26 19:15:06 2025
From: psandoz at openjdk.org (Paul Sandoz)
Date: Wed, 26 Feb 2025 19:15:06 GMT
Subject: [code-reflection] RFR: Onnx subgraphs,
 lambda execution and BB removal [v3]
In-Reply-To: <u-_qrfGqF1aRgW5HaKgr943-lz-W0MBkZMhu3tPnHvg=.20368a53-f139-4cb6-9d47-7aea889e5060@github.com>
References: <CYyR9Ypl-frjAUum631CITskwAXBevLVjgRbKqwmrdU=.026f0f2b-f941-45fc-bfe3-1c0f899912db@github.com>
 <F98Fd4cuvFmrV-Q02PWWoDh3AkeahpSHFXYsOUb6OQM=.6cb6f9fe-4b9a-430b-a772-83b7511c933a@github.com>
 <8t0wJ9s9nDIpxVuKECamyZc9NXkMIsM0JF0VfMuxZ4U=.bd92b811-28c7-4731-9b08-3550d2d3f129@github.com>
 <2MFL3z7W4-2OIqADGeCe3KpY31bP14sPtzHbkMgaiZE=.e73413a3-e93a-4e37-a177-c33e10f952d9@github.com>
 <u-_qrfGqF1aRgW5HaKgr943-lz-W0MBkZMhu3tPnHvg=.20368a53-f139-4cb6-9d47-7aea889e5060@github.com>
Message-ID: <qJ696QKu0b3-WkKRVslkomD6JPrD-fhvvWWihlQwVyE=.2d65271f-22bc-4169-822f-a288e6cea7b7@github.com>

On Wed, 26 Feb 2025 18:42:04 GMT, Adam Sotona <asotona at openjdk.org> wrote:

>> Initializers are now on top of my ToDo list. They are technically just differently serialized Constant ops. I'm still not sure how we should model them - maybe Initializer op?
>
> Method call in a lambda is another level of complexity. I guess OnnxTransformer should inline it?

> Initializers are now on top of my ToDo list. They are technically just differently serialized Constant ops. I'm still not sure how we should model them - maybe Initializer op?

My understanding is initializers allow for easier introspection of the ONNX model rather than analyzing the graph's constant operators and deciding which are relevant or not.

I was thinking we should avoid representing initializers explicitly in the ONNX code model, and instead a prefix of parameters can be identified as initializers (with their tensor values) when transforming to an ONNX model. I don't fully know how to connect this to the Java ONNX function and the `OnnxRuntime.execute(() -> ...);` since we cannot distinguish between various kinds of captured arguments. Perhaps we can leverage field access, representing the weights as fields (there could be a large amount of them for larger models). Alternatively, we can use structural quoting, target typing the lambda expression to quoted (no functional interface required) where the lambda's parameters correspond to initializers, captures to inputs, and passing the initializer values to the execute method.


> Method call in a lambda is another level of complexity. I guess OnnxTransformer should inline it?

Possibly, although i think the right approach is to leverage [ONNX functions](https://onnx.ai/onnx/intro/concepts.html#functions).

Still, i believe it would be worth supporting the short cut approach now as that shows where we are heading.

-------------

PR Review Comment: https://git.openjdk.org/babylon/pull/326#discussion_r1972236048

From psandoz at openjdk.org  Wed Feb 26 23:22:24 2025
From: psandoz at openjdk.org (Paul Sandoz)
Date: Wed, 26 Feb 2025 23:22:24 GMT
Subject: [code-reflection] RFR: Short cut for single method execution
Message-ID: <OUz4HBkZfhYFkfxiUNHAHgbxMh5cBm2itogEVms0FO4=.2f63e769-4724-4cb7-92c6-b72e0985df9c@github.com>

This allows us to retain the exact method declaration and simply wrap in a lambda (its like an optimization). The execute method requires the user pass the lookup, otherwise reflective access can fail. We can overload using the public lookup if need be later on.

-------------

Commit messages:
 - Short cut for single method execution

Changes: https://git.openjdk.org/babylon/pull/329/files
  Webrev: https://webrevs.openjdk.org/?repo=babylon&pr=329&range=00
  Stats: 198 lines in 2 files changed: 127 ins; 3 del; 68 mod
  Patch: https://git.openjdk.org/babylon/pull/329.diff
  Fetch: git fetch https://git.openjdk.org/babylon.git pull/329/head:pull/329

PR: https://git.openjdk.org/babylon/pull/329

From psandoz at openjdk.org  Thu Feb 27 00:24:04 2025
From: psandoz at openjdk.org (Paul Sandoz)
Date: Thu, 27 Feb 2025 00:24:04 GMT
Subject: [code-reflection] RFR: Support storing the code that builds the
 code model [v2]
In-Reply-To: <0ug_-F7Oz_TImedL2gFJ5Fl9h82MZvcXO5J1kFuySww=.bafa9bed-e0ae-4263-97af-4ac54b3a20a0@github.com>
References: <dexv8sQV_NcJsJIgMopd_M4wSWoZya9dU0ACwDtEL0E=.4ac0b80f-24fb-440f-a18d-bbfd07a681bc@github.com>
 <W5OPqovjc4c2-wXuuqT9GveOC9Kv9ooDa8-_A9t9-VM=.683bf92e-19f0-4d26-805f-78b64f72c084@github.com>
 <0ug_-F7Oz_TImedL2gFJ5Fl9h82MZvcXO5J1kFuySww=.bafa9bed-e0ae-4263-97af-4ac54b3a20a0@github.com>
Message-ID: <7B2GtbL9SNzt2h5qK4onfruWUPTFXAluIBIdh0dGk6Q=.b12f77c2-d888-407e-80ba-48b7dfda819e@github.com>

On Mon, 10 Feb 2025 12:11:51 GMT, Maurizio Cimadamore <mcimadamore at openjdk.org> wrote:

>> Mourad Abbay has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   Make TEXT the default storage mechanism for code models
>
> Marked as reviewed by mcimadamore (Reviewer).

@mcimadamore Mourad implemented a transformation of the code model that builds a code model that adds local variables for values with more than one use, which makes it easier to generate the AST nodes. Would use of the internal `LetExpr` help avoid such a transformation, if so we can consider that for follow on work.

-------------

PR Comment: https://git.openjdk.org/babylon/pull/305#issuecomment-2686501336

From psandoz at openjdk.org  Thu Feb 27 02:32:47 2025
From: psandoz at openjdk.org (Paul Sandoz)
Date: Thu, 27 Feb 2025 02:32:47 GMT
Subject: [code-reflection] RFR: Short cut for single method execution [v2]
In-Reply-To: <OUz4HBkZfhYFkfxiUNHAHgbxMh5cBm2itogEVms0FO4=.2f63e769-4724-4cb7-92c6-b72e0985df9c@github.com>
References: <OUz4HBkZfhYFkfxiUNHAHgbxMh5cBm2itogEVms0FO4=.2f63e769-4724-4cb7-92c6-b72e0985df9c@github.com>
Message-ID: <7LjNZAz08fgSOXmHCF5OaPxg0CLDQ_pftx9_WvZD48U=.69aac9cc-e0ca-42b7-b9bc-df5bde4f9fe9@github.com>

> This allows us to retain the exact method declaration and simply wrap in a lambda (its like an optimization). The execute method requires the user pass the lookup, otherwise reflective access can fail. We can overload using the public lookup if need be later on.

Paul Sandoz has updated the pull request incrementally with one additional commit since the last revision:

  Ensure arguments are mapped correctly.

-------------

Changes:
  - all: https://git.openjdk.org/babylon/pull/329/files
  - new: https://git.openjdk.org/babylon/pull/329/files/0780287e..f0f41e70

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=babylon&pr=329&range=01
 - incr: https://webrevs.openjdk.org/?repo=babylon&pr=329&range=00-01

  Stats: 20 lines in 1 file changed: 13 ins; 2 del; 5 mod
  Patch: https://git.openjdk.org/babylon/pull/329.diff
  Fetch: git fetch https://git.openjdk.org/babylon.git pull/329/head:pull/329

PR: https://git.openjdk.org/babylon/pull/329

From asotona at openjdk.org  Thu Feb 27 07:04:10 2025
From: asotona at openjdk.org (Adam Sotona)
Date: Thu, 27 Feb 2025 07:04:10 GMT
Subject: [code-reflection] RFR: Short cut for single method execution [v2]
In-Reply-To: <7LjNZAz08fgSOXmHCF5OaPxg0CLDQ_pftx9_WvZD48U=.69aac9cc-e0ca-42b7-b9bc-df5bde4f9fe9@github.com>
References: <OUz4HBkZfhYFkfxiUNHAHgbxMh5cBm2itogEVms0FO4=.2f63e769-4724-4cb7-92c6-b72e0985df9c@github.com>
 <7LjNZAz08fgSOXmHCF5OaPxg0CLDQ_pftx9_WvZD48U=.69aac9cc-e0ca-42b7-b9bc-df5bde4f9fe9@github.com>
Message-ID: <mRWZNQ4fo0AABsN-4nl_2IMNZnMbefLlf8bo5wIMTS4=.10207ea3-ab0e-4daa-a26e-22df0aeb58cf@github.com>

On Thu, 27 Feb 2025 02:32:47 GMT, Paul Sandoz <psandoz at openjdk.org> wrote:

>> This allows us to retain the exact method declaration and simply wrap in a lambda (its like an optimization). The execute method requires the user pass the lookup, otherwise reflective access can fail. We can overload using the public lookup if need be later on.
>
> Paul Sandoz has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Ensure arguments are mapped correctly.

Looks great.

-------------

Marked as reviewed by asotona (Reviewer).

PR Review: https://git.openjdk.org/babylon/pull/329#pullrequestreview-2646794299

From asotona at openjdk.org  Thu Feb 27 08:15:11 2025
From: asotona at openjdk.org (Adam Sotona)
Date: Thu, 27 Feb 2025 08:15:11 GMT
Subject: [code-reflection] RFR: Short cut for single method execution [v2]
In-Reply-To: <7LjNZAz08fgSOXmHCF5OaPxg0CLDQ_pftx9_WvZD48U=.69aac9cc-e0ca-42b7-b9bc-df5bde4f9fe9@github.com>
References: <OUz4HBkZfhYFkfxiUNHAHgbxMh5cBm2itogEVms0FO4=.2f63e769-4724-4cb7-92c6-b72e0985df9c@github.com>
 <7LjNZAz08fgSOXmHCF5OaPxg0CLDQ_pftx9_WvZD48U=.69aac9cc-e0ca-42b7-b9bc-df5bde4f9fe9@github.com>
Message-ID: <G1G0BGzsayDzDviRdSW9y2oVwN6X2jYWaRIYueN_J90=.b4cf9f3a-989e-4907-b9ea-426f4eed3d66@github.com>

On Thu, 27 Feb 2025 02:32:47 GMT, Paul Sandoz <psandoz at openjdk.org> wrote:

>> This allows us to retain the exact method declaration and simply wrap in a lambda (its like an optimization). The execute method requires the user pass the lookup, otherwise reflective access can fail. We can overload using the public lookup if need be later on.
>
> Paul Sandoz has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Ensure arguments are mapped correctly.

FYI: I've tried to add https://github.com/PaulSandoz/babylon/pull/2 to this PR to adjust tests
However for some reasons the PRs do not chain as in the openjdk/jdk

-------------

PR Comment: https://git.openjdk.org/babylon/pull/329#issuecomment-2687202808

From mcimadamore at openjdk.org  Thu Feb 27 10:25:03 2025
From: mcimadamore at openjdk.org (Maurizio Cimadamore)
Date: Thu, 27 Feb 2025 10:25:03 GMT
Subject: [code-reflection] RFR: Short cut for single method execution [v2]
In-Reply-To: <7LjNZAz08fgSOXmHCF5OaPxg0CLDQ_pftx9_WvZD48U=.69aac9cc-e0ca-42b7-b9bc-df5bde4f9fe9@github.com>
References: <OUz4HBkZfhYFkfxiUNHAHgbxMh5cBm2itogEVms0FO4=.2f63e769-4724-4cb7-92c6-b72e0985df9c@github.com>
 <7LjNZAz08fgSOXmHCF5OaPxg0CLDQ_pftx9_WvZD48U=.69aac9cc-e0ca-42b7-b9bc-df5bde4f9fe9@github.com>
Message-ID: <mb_Gn7F6BZAZPkAinXbIdh9CVuY5g4LKjstiCDQT82E=.772b2a59-d44a-45ef-9e0c-02259c75475f@github.com>

On Thu, 27 Feb 2025 02:32:47 GMT, Paul Sandoz <psandoz at openjdk.org> wrote:

>> This allows us to retain the exact method declaration and simply wrap in a lambda (its like an optimization). The execute method requires the user pass the lookup, otherwise reflective access can fail. We can overload using the public lookup if need be later on.
>
> Paul Sandoz has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Ensure arguments are mapped correctly.

Marked as reviewed by mcimadamore (Reviewer).

cr-examples/onnx/src/main/java/oracle/code/onnx/OnnxRuntime.java line 58:

> 56:     }
> 57: 
> 58:     public static <T> Tensor<T> execute(MethodHandles.Lookup l, OnnxFunction<Tensor<T>> codeLambda) {

The expectation seems to be that the input values are all tensors... maybe we can model them as explicit parameters in the `OnnxFunction` interface - e.g. something like:


interface OnnxFunction<T> {
     T apply(Tensor... tensors)
}


This would open the possibility to use method reference in the example -- but one problem is that this uses a raw type (Tensor).

Another option could be to model both the input and output generically:


interface OnnxFunction<T, U> {
     T apply(U u)
}


Where both T and U can be either:
* a single Tensor (of any type)
* a list/array of Tensors (again, of any type)

-------------

PR Review: https://git.openjdk.org/babylon/pull/329#pullrequestreview-2647304590
PR Review Comment: https://git.openjdk.org/babylon/pull/329#discussion_r1973294055

From mcimadamore at openjdk.org  Thu Feb 27 11:05:04 2025
From: mcimadamore at openjdk.org (Maurizio Cimadamore)
Date: Thu, 27 Feb 2025 11:05:04 GMT
Subject: [code-reflection] RFR: Support storing the code that builds the
 code model [v2]
In-Reply-To: <0ug_-F7Oz_TImedL2gFJ5Fl9h82MZvcXO5J1kFuySww=.bafa9bed-e0ae-4263-97af-4ac54b3a20a0@github.com>
References: <dexv8sQV_NcJsJIgMopd_M4wSWoZya9dU0ACwDtEL0E=.4ac0b80f-24fb-440f-a18d-bbfd07a681bc@github.com>
 <W5OPqovjc4c2-wXuuqT9GveOC9Kv9ooDa8-_A9t9-VM=.683bf92e-19f0-4d26-805f-78b64f72c084@github.com>
 <0ug_-F7Oz_TImedL2gFJ5Fl9h82MZvcXO5J1kFuySww=.bafa9bed-e0ae-4263-97af-4ac54b3a20a0@github.com>
Message-ID: <t4iEutU8Y14VzB0wQCZTGpDaKgH7PI_eWnj7FyvIjU4=.825c2167-583e-48f9-8da5-3b8194785670@github.com>

On Mon, 10 Feb 2025 12:11:51 GMT, Maurizio Cimadamore <mcimadamore at openjdk.org> wrote:

>> Mourad Abbay has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   Make TEXT the default storage mechanism for code models
>
> Marked as reviewed by mcimadamore (Reviewer).

> @mcimadamore Mourad implemented a transformation of the code model that builds a code model that adds local variables for values with more than one use, which makes it easier to generate the AST nodes. Would use of the internal `LetExpr` help avoid such a transformation, if so we can consider that for follow on work.

I'm not sure I have all the context here. The problem here seems to be when you have a value that is resulting from some potentially side-effect operation. E.g. like a method call:


%2 = bar(%1)


If `%2` is used multiple times, then javac has only one option -- that is, to hoist `%2` in a local variable, and then replaces all references to `%2` with references to the local variable. Inlining the call to `bar` at the use-site is not really an option, as that could change the semantics of the program.

Let expression nodes are useful when dealing with compact expressions. E.g.

`List l = let x = 42 in List.of(42)`

E.g. javac typically uses a let expression when it has to translate a single expression into something more complex, but it wants to do so by keeping the result as an expression (rather than turning the expression into a statement, which is not possible in all cases, such as in the case of a variable initializer).

It is true that what seems like a linear list of ops in a block can be modelled as something like more convoluted, like so:


let op1 = <op1 init> in
    let op2 = <op2 init> in
        let op3 = <op2 init> in
               ....
               <result>


This would mean to generate one let expression per op, where the "body" of the let expression is the remainder of the code model block. All this nesting is confusing, but is also avoidable -- a `LetExpr` node allows for more than one declaration for each body -- so you can translate the above as follows:


let (op1 = <op1 init> ;
     op2 = <op2 init> ;
     op3 = <op2 init>) in <result>


Doing something like this would probably avoid the need of generating extra local variables -- you now have one var declaration per op in the "statements" part of a `LetExpr`. It looks a bit odd -- visually -- that the body of the `LetExpr` is just the result of the code model block -- e.g. all the interesting part is in the setup code. But seems doable.

At the end of the day either adding extra variables (which can even be done as a pre-processing step, by javac), or using a more functional translation with `LetExpr` should work. 
             
P.S.
I looked at the code and, at least in some cases (`h`) the `TestAddVarsWhenNecessary` seems to add intermediate `Var` ops, but which seem redundant - as they are initialized with some function parameter. I'd expect javac to be able to deal with references to function parameters using a `JCIdent` pointing at the desired parameter.

-------------

PR Comment: https://git.openjdk.org/babylon/pull/305#issuecomment-2687617699

From mcimadamore at openjdk.org  Thu Feb 27 11:12:06 2025
From: mcimadamore at openjdk.org (Maurizio Cimadamore)
Date: Thu, 27 Feb 2025 11:12:06 GMT
Subject: [code-reflection] RFR: Support storing the code that builds the
 code model [v11]
In-Reply-To: <KT1Hs6fGZAH1tCOqT_MZ4DoaDBc0FkS7qFh4kKkAwL8=.13476382-ff9a-4f29-b1bf-8f0b8ebc71fe@github.com>
References: <dexv8sQV_NcJsJIgMopd_M4wSWoZya9dU0ACwDtEL0E=.4ac0b80f-24fb-440f-a18d-bbfd07a681bc@github.com>
 <KT1Hs6fGZAH1tCOqT_MZ4DoaDBc0FkS7qFh4kKkAwL8=.13476382-ff9a-4f29-b1bf-8f0b8ebc71fe@github.com>
Message-ID: <Eas1o5n8673YLGOG-QlBfILACmNxw2hR6c_t016fZ8M=.86455e20-d9e2-4036-9732-acf7406d0a98@github.com>

On Wed, 26 Feb 2025 18:06:44 GMT, Mourad Abbay <mabbay at openjdk.org> wrote:

>> In this PR we allow the user to decide how to store the code model. The first option is `TEXT`, if this is selected, we store the textual representation of the code model. The second option is `CODE_BUILDR`, if this is selected, we store the code that builds the code model. All work done here is around the second option, because the first option is already supported.
>
> Mourad Abbay has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Ensure that block params are inserted in the correct order

src/jdk.incubator.code/share/classes/jdk/incubator/code/internal/CodeModelToAST.java line 44:

> 42:     private Map<JavaType, Type> mappingFromJavaTypeToType() {
> 43:         Map<JavaType, Type> m = new HashMap<>();
> 44:         Symbol.ModuleSymbol jdk_incubator_code = syms.enterModule(names.jdk_incubator_code);

This is effectively an extension to javac's symbol table. My preference would be to:
* add whatever symbol/type you need in `CodeReflectionSymbol`
* then set up the translation map simply, like you do for primitive types -- e.g.

Map.entry(<type element>, <javac type>)

-------------

PR Review Comment: https://git.openjdk.org/babylon/pull/305#discussion_r1973372549

From mcimadamore at openjdk.org  Thu Feb 27 11:38:17 2025
From: mcimadamore at openjdk.org (Maurizio Cimadamore)
Date: Thu, 27 Feb 2025 11:38:17 GMT
Subject: [code-reflection] RFR: Support storing the code that builds the
 code model [v11]
In-Reply-To: <KT1Hs6fGZAH1tCOqT_MZ4DoaDBc0FkS7qFh4kKkAwL8=.13476382-ff9a-4f29-b1bf-8f0b8ebc71fe@github.com>
References: <dexv8sQV_NcJsJIgMopd_M4wSWoZya9dU0ACwDtEL0E=.4ac0b80f-24fb-440f-a18d-bbfd07a681bc@github.com>
 <KT1Hs6fGZAH1tCOqT_MZ4DoaDBc0FkS7qFh4kKkAwL8=.13476382-ff9a-4f29-b1bf-8f0b8ebc71fe@github.com>
Message-ID: <HmUBv-LGse8JEE2HejVDfmVyEk3kv7CN3BjsmreS-XM=.ef010813-60de-45f0-bbad-a4b168ddbe22@github.com>

On Wed, 26 Feb 2025 18:06:44 GMT, Mourad Abbay <mabbay at openjdk.org> wrote:

>> In this PR we allow the user to decide how to store the code model. The first option is `TEXT`, if this is selected, we store the textual representation of the code model. The second option is `CODE_BUILDR`, if this is selected, we store the code that builds the code model. All work done here is around the second option, because the first option is already supported.
>
> Mourad Abbay has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Ensure that block params are inserted in the correct order

src/jdk.incubator.code/share/classes/jdk/incubator/code/internal/CodeModelToAST.java line 102:

> 100:     }
> 101: 
> 102:     private JCTree invokeOpToJCMethodInvocation(CoreOp.InvokeOp invokeOp) {

This method has some issues -- but perhaps we can keep it for now.
The main problem is that we seem to copy types from the invoked method declaration and put them "as is" in the generated AST. This works only as long as the invoked method doesn't have any generic type. For instance, consider a call to `List::add`. The signature of this method accepts `X` which is a type parameter of `List<X>`. I believe the code you have copies the `X` from the declared signature and sticks it into the javac's `MethodType` instance attached to the method call. Since this method type is used to describe the use-site (the call), and not the declaration, the type seems incorrect. E.g. calling `List<String>:add` should have a type of `add(String)` not `add(X)`. (btw, a similar problem is present for field access, if the type of the accessed field is a type-variable -- so it's not just an issue with method calls).

The type of the invoked method/accessed field should always be adjusted with `Types.memberType` first. This method takes two parameters:

* the receiver type you are using to call the instance method/access the instance field
* the symbol you are trying to access

And returns the _instantiated_ type of that symbol at that receiver type. Examples:


memberType(List<String>, add(X)->void) -> (String)->void
memberType(List<Integer>, add(X)->void) -> (Integer)->void


Once you address this, there is still another problem with generic methods -- as methods can have their own type-variables too. To figure out what is the type to be replaced for these type-variables you generally need to run inference -- which seems way too much for what we're trying to do here. The issue here is that the code model you are processing doesn't expose these details, and so generating the AST needs to "trace back" whatever steps where done when generating this model -- which is an hard problem.

I hope we can get away with just working with erased types, and maybe insert type-conversion to convert the type of the invoked method/accessed field so that it matches the expected op result type.

src/jdk.incubator.code/share/classes/jdk/incubator/code/internal/CodeModelToAST.java line 128:

> 126:             var lastParam = invokeOp.invokeDescriptor().type().parameterTypes().getLast();
> 127:             Assert.check(lastParam instanceof ArrayType);
> 128:             methodInvocation.varargsElement = typeElementToType(((ArrayType) lastParam).componentType());

Same problem as described above -- here we're copying the vararg array at the declaration site into a type at the use-site -- for something like `List::of` this won't give the result you expect.

-------------

PR Review Comment: https://git.openjdk.org/babylon/pull/305#discussion_r1973406083
PR Review Comment: https://git.openjdk.org/babylon/pull/305#discussion_r1973407675

From mcimadamore at openjdk.org  Thu Feb 27 11:42:15 2025
From: mcimadamore at openjdk.org (Maurizio Cimadamore)
Date: Thu, 27 Feb 2025 11:42:15 GMT
Subject: [code-reflection] RFR: Support storing the code that builds the
 code model [v11]
In-Reply-To: <HmUBv-LGse8JEE2HejVDfmVyEk3kv7CN3BjsmreS-XM=.ef010813-60de-45f0-bbad-a4b168ddbe22@github.com>
References: <dexv8sQV_NcJsJIgMopd_M4wSWoZya9dU0ACwDtEL0E=.4ac0b80f-24fb-440f-a18d-bbfd07a681bc@github.com>
 <KT1Hs6fGZAH1tCOqT_MZ4DoaDBc0FkS7qFh4kKkAwL8=.13476382-ff9a-4f29-b1bf-8f0b8ebc71fe@github.com>
 <HmUBv-LGse8JEE2HejVDfmVyEk3kv7CN3BjsmreS-XM=.ef010813-60de-45f0-bbad-a4b168ddbe22@github.com>
Message-ID: <U-4AhyoQjGBNl-SDgBd32NMTM2esNcCRLuthwO9tUc8=.fe53345d-7673-4f0a-b907-1a6fb274a337@github.com>

On Thu, 27 Feb 2025 11:36:16 GMT, Maurizio Cimadamore <mcimadamore at openjdk.org> wrote:

>> Mourad Abbay has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   Ensure that block params are inserted in the correct order
>
> src/jdk.incubator.code/share/classes/jdk/incubator/code/internal/CodeModelToAST.java line 128:
> 
>> 126:             var lastParam = invokeOp.invokeDescriptor().type().parameterTypes().getLast();
>> 127:             Assert.check(lastParam instanceof ArrayType);
>> 128:             methodInvocation.varargsElement = typeElementToType(((ArrayType) lastParam).componentType());
> 
> Same problem as described above -- here we're copying the vararg array at the declaration site into a type at the use-site -- for something like `List::of` this won't give the result you expect.

Now -- while all the points I made above apply (e.g. if you were to try and type-check the generated AST using `Attr` you will get several errors), the saving grace here is that you are sending this tree into javac's backend anyway. And the backend plays a lot looser with types, only inserting casts where absolutely needed. Since you are using the op result type on the `MethodType` instance you are generating, I believe that should be enough for the backend to at least insert 90% of the type conversions that might be required because of erasure. So, in practice, even if incorrect, the code above might work fine.

-------------

PR Review Comment: https://git.openjdk.org/babylon/pull/305#discussion_r1973412610

From psandoz at openjdk.org  Thu Feb 27 16:42:12 2025
From: psandoz at openjdk.org (Paul Sandoz)
Date: Thu, 27 Feb 2025 16:42:12 GMT
Subject: git: openjdk/babylon: code-reflection: Short cut for single method
 execution
Message-ID: <fa40b226-a2c2-4f56-86f5-45d1b45e351b@openjdk.org>

Changeset: cd265d27
Branch: code-reflection
Author:    Paul Sandoz <psandoz at openjdk.org>
Date:      2025-02-27 16:40:16 +0000
URL:       https://git.openjdk.org/babylon/commit/cd265d2704e43faae95b70a03d698d4941792ea2

Short cut for single method execution

Reviewed-by: asotona, mcimadamore

! cr-examples/onnx/src/main/java/oracle/code/onnx/OnnxRuntime.java
! cr-examples/onnx/src/test/java/oracle/code/onnx/MNISTDemo.java


From psandoz at openjdk.org  Thu Feb 27 16:43:16 2025
From: psandoz at openjdk.org (Paul Sandoz)
Date: Thu, 27 Feb 2025 16:43:16 GMT
Subject: [code-reflection] Integrated: Short cut for single method execution
In-Reply-To: <OUz4HBkZfhYFkfxiUNHAHgbxMh5cBm2itogEVms0FO4=.2f63e769-4724-4cb7-92c6-b72e0985df9c@github.com>
References: <OUz4HBkZfhYFkfxiUNHAHgbxMh5cBm2itogEVms0FO4=.2f63e769-4724-4cb7-92c6-b72e0985df9c@github.com>
Message-ID: <ULY0iQqxC1fPfIunwTK1TmMUgiEu__uAUaB4WUgUKX0=.2b8ed39b-e109-4ddb-b512-e2ed244a6218@github.com>

On Wed, 26 Feb 2025 23:17:23 GMT, Paul Sandoz <psandoz at openjdk.org> wrote:

> This allows us to retain the exact method declaration and simply wrap in a lambda (its like an optimization). The execute method requires the user pass the lookup, otherwise reflective access can fail. We can overload using the public lookup if need be later on.

This pull request has now been integrated.

Changeset: cd265d27
Author:    Paul Sandoz <psandoz at openjdk.org>
URL:       https://git.openjdk.org/babylon/commit/cd265d2704e43faae95b70a03d698d4941792ea2
Stats:     209 lines in 2 files changed: 138 ins; 3 del; 68 mod

Short cut for single method execution

Reviewed-by: asotona, mcimadamore

-------------

PR: https://git.openjdk.org/babylon/pull/329

From asotona at openjdk.org  Thu Feb 27 16:56:34 2025
From: asotona at openjdk.org (Adam Sotona)
Date: Thu, 27 Feb 2025 16:56:34 GMT
Subject: [code-reflection] RFR: Adjusted tests to use execution of a method and
 implementation of the execution model cache
Message-ID: <fh6o1Sf1-Hu57rvlxCW4MmP5C9pMVPlAzO5ao6eOFFE=.2a9557c5-52b7-4070-8562-94dede52272b@github.com>

Adjusted tests to use execution of a method and implementation of the execution model cache.

-------------

Commit messages:
 - Merge remote-tracking branch 'babylon/code-reflection' into onnx-execute-cache
 - implemented operands shuffling + test
 - caching models instead of the session
 - primitive caching of the last session
 - adjusted tests to use execution of a method (no initializers yet)
 - Ensure arguments are mapped correctly.
 - Short cut for single method execution

Changes: https://git.openjdk.org/babylon/pull/331/files
  Webrev: https://webrevs.openjdk.org/?repo=babylon&pr=331&range=00
  Stats: 160 lines in 4 files changed: 70 ins; 45 del; 45 mod
  Patch: https://git.openjdk.org/babylon/pull/331.diff
  Fetch: git fetch https://git.openjdk.org/babylon.git pull/331/head:pull/331

PR: https://git.openjdk.org/babylon/pull/331

From psandoz at openjdk.org  Thu Feb 27 19:21:12 2025
From: psandoz at openjdk.org (Paul Sandoz)
Date: Thu, 27 Feb 2025 19:21:12 GMT
Subject: [code-reflection] RFR: Adjusted tests to use execution of a method
 and implementation of the execution model cache
In-Reply-To: <fh6o1Sf1-Hu57rvlxCW4MmP5C9pMVPlAzO5ao6eOFFE=.2a9557c5-52b7-4070-8562-94dede52272b@github.com>
References: <fh6o1Sf1-Hu57rvlxCW4MmP5C9pMVPlAzO5ao6eOFFE=.2a9557c5-52b7-4070-8562-94dede52272b@github.com>
Message-ID: <kJollT6OPI-6HqMcz8yTHd37EjFduGxBOfBq7wD_OuY=.9a2fae87-1b2b-4e1a-982d-b15856ff43ad@github.com>

On Thu, 27 Feb 2025 16:50:04 GMT, Adam Sotona <asotona at openjdk.org> wrote:

> Adjusted tests to use execution of a method and implementation of the execution model cache.

Very good.

cr-examples/onnx/src/main/java/oracle/code/onnx/OnnxRuntime.java line 60:

> 58:     record CachedModel(byte[] protoModel, int[] operandsMapping) {}
> 59: 
> 60:     private static final WeakHashMap<Class<?>, CachedModel> MODEL_CACHE = WeakHashMap.newWeakHashMap(10);

Using `ClassValue` is another option.

cr-examples/onnx/src/main/java/oracle/code/onnx/OnnxRuntime.java line 86:

> 84: 
> 85:                 var operands = iop.operands();
> 86:                 var captured = List.of(quotable.capturedValues().sequencedKeySet().toArray());

`var captured = quotable.capturedValues().sequencedKeySet().stream().toList();` ?

-------------

Marked as reviewed by psandoz (Lead).

PR Review: https://git.openjdk.org/babylon/pull/331#pullrequestreview-2648835318
PR Review Comment: https://git.openjdk.org/babylon/pull/331#discussion_r1974200239
PR Review Comment: https://git.openjdk.org/babylon/pull/331#discussion_r1974202463

From mabbay at openjdk.org  Fri Feb 28 00:07:37 2025
From: mabbay at openjdk.org (Mourad Abbay)
Date: Fri, 28 Feb 2025 00:07:37 GMT
Subject: [code-reflection] RFR: Support storing the code that builds the
 code model [v12]
In-Reply-To: <dexv8sQV_NcJsJIgMopd_M4wSWoZya9dU0ACwDtEL0E=.4ac0b80f-24fb-440f-a18d-bbfd07a681bc@github.com>
References: <dexv8sQV_NcJsJIgMopd_M4wSWoZya9dU0ACwDtEL0E=.4ac0b80f-24fb-440f-a18d-bbfd07a681bc@github.com>
Message-ID: <Jk__IsuOmruwzQg3sdGANbIsGShYDO7ODXXXrUEW1yg=.ce26faab-b229-4415-8457-2c65ecc8d852@github.com>

> In this PR we allow the user to decide how to store the code model. The first option is `TEXT`, if this is selected, we store the textual representation of the code model. The second option is `CODE_BUILDR`, if this is selected, we store the code that builds the code model. All work done here is around the second option, because the first option is already supported.

Mourad Abbay has updated the pull request incrementally with one additional commit since the last revision:

  Pass arrayType instead of eleType in OpBuilder.buildArray

-------------

Changes:
  - all: https://git.openjdk.org/babylon/pull/305/files
  - new: https://git.openjdk.org/babylon/pull/305/files/c2323661..a1d134c3

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=babylon&pr=305&range=11
 - incr: https://webrevs.openjdk.org/?repo=babylon&pr=305&range=10-11

  Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod
  Patch: https://git.openjdk.org/babylon/pull/305.diff
  Fetch: git fetch https://git.openjdk.org/babylon.git pull/305/head:pull/305

PR: https://git.openjdk.org/babylon/pull/305

From asotona at openjdk.org  Fri Feb 28 07:55:06 2025
From: asotona at openjdk.org (Adam Sotona)
Date: Fri, 28 Feb 2025 07:55:06 GMT
Subject: [code-reflection] RFR: Adjusted tests to use execution of a method
 and implementation of the execution model cache
In-Reply-To: <kJollT6OPI-6HqMcz8yTHd37EjFduGxBOfBq7wD_OuY=.9a2fae87-1b2b-4e1a-982d-b15856ff43ad@github.com>
References: <fh6o1Sf1-Hu57rvlxCW4MmP5C9pMVPlAzO5ao6eOFFE=.2a9557c5-52b7-4070-8562-94dede52272b@github.com>
 <kJollT6OPI-6HqMcz8yTHd37EjFduGxBOfBq7wD_OuY=.9a2fae87-1b2b-4e1a-982d-b15856ff43ad@github.com>
Message-ID: <_iuflwkfM_Mijonhyrg3vCqMnvD0eYy6dkQY1hD7mR0=.2f8c0746-4507-497e-814e-be945edeb4e2@github.com>

On Thu, 27 Feb 2025 19:17:00 GMT, Paul Sandoz <psandoz at openjdk.org> wrote:

>> Adjusted tests to use execution of a method and implementation of the execution model cache.
>
> cr-examples/onnx/src/main/java/oracle/code/onnx/OnnxRuntime.java line 60:
> 
>> 58:     record CachedModel(byte[] protoModel, int[] operandsMapping) {}
>> 59: 
>> 60:     private static final WeakHashMap<Class<?>, CachedModel> MODEL_CACHE = WeakHashMap.newWeakHashMap(10);
> 
> Using `ClassValue` is another option.

Unfortunately `ClassValue` does not support computation if absent (if more than just the bare class is needed). Maybe if compute returns null and the cached value is added later...?

-------------

PR Review Comment: https://git.openjdk.org/babylon/pull/331#discussion_r1974959775

From asotona at openjdk.org  Fri Feb 28 09:59:46 2025
From: asotona at openjdk.org (Adam Sotona)
Date: Fri, 28 Feb 2025 09:59:46 GMT
Subject: [code-reflection] RFR: Adjusted tests to use execution of a method
 and implementation of the execution model cache [v2]
In-Reply-To: <fh6o1Sf1-Hu57rvlxCW4MmP5C9pMVPlAzO5ao6eOFFE=.2a9557c5-52b7-4070-8562-94dede52272b@github.com>
References: <fh6o1Sf1-Hu57rvlxCW4MmP5C9pMVPlAzO5ao6eOFFE=.2a9557c5-52b7-4070-8562-94dede52272b@github.com>
Message-ID: <GzQ6cmCzw0MZANYL03vzPCWBXsvN37-0ffdZn3ChfkI=.e7308c28-4460-43f7-ad29-c096569da242@github.com>

> Adjusted tests to use execution of a method and implementation of the execution model cache.

Adam Sotona has updated the pull request incrementally with one additional commit since the last revision:

  recommended changes applied

-------------

Changes:
  - all: https://git.openjdk.org/babylon/pull/331/files
  - new: https://git.openjdk.org/babylon/pull/331/files/003b8ad3..5a05ed12

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=babylon&pr=331&range=01
 - incr: https://webrevs.openjdk.org/?repo=babylon&pr=331&range=00-01

  Stats: 30 lines in 1 file changed: 20 ins; 0 del; 10 mod
  Patch: https://git.openjdk.org/babylon/pull/331.diff
  Fetch: git fetch https://git.openjdk.org/babylon.git pull/331/head:pull/331

PR: https://git.openjdk.org/babylon/pull/331

From mcimadamore at openjdk.org  Fri Feb 28 09:59:47 2025
From: mcimadamore at openjdk.org (Maurizio Cimadamore)
Date: Fri, 28 Feb 2025 09:59:47 GMT
Subject: [code-reflection] RFR: Adjusted tests to use execution of a method
 and implementation of the execution model cache [v2]
In-Reply-To: <_iuflwkfM_Mijonhyrg3vCqMnvD0eYy6dkQY1hD7mR0=.2f8c0746-4507-497e-814e-be945edeb4e2@github.com>
References: <fh6o1Sf1-Hu57rvlxCW4MmP5C9pMVPlAzO5ao6eOFFE=.2a9557c5-52b7-4070-8562-94dede52272b@github.com>
 <kJollT6OPI-6HqMcz8yTHd37EjFduGxBOfBq7wD_OuY=.9a2fae87-1b2b-4e1a-982d-b15856ff43ad@github.com>
 <_iuflwkfM_Mijonhyrg3vCqMnvD0eYy6dkQY1hD7mR0=.2f8c0746-4507-497e-814e-be945edeb4e2@github.com>
Message-ID: <lBxmJshq_VXbLGrxBEiSxt4vR050p3RJ5owpajloVao=.ff691c8c-0db5-49ec-b412-fa6ce9acb603@github.com>

On Fri, 28 Feb 2025 07:51:05 GMT, Adam Sotona <asotona at openjdk.org> wrote:

>> cr-examples/onnx/src/main/java/oracle/code/onnx/OnnxRuntime.java line 60:
>> 
>>> 58:     record CachedModel(byte[] protoModel, int[] operandsMapping) {}
>>> 59: 
>>> 60:     private static final WeakHashMap<Class<?>, CachedModel> MODEL_CACHE = WeakHashMap.newWeakHashMap(10);
>> 
>> Using `ClassValue` is another option.
>
> Unfortunately `ClassValue` does not support computation if absent (if more than just the bare class is needed). Maybe if compute returns null and the cached value is added later...?

I believe `ClassValue` is similar to a thread local in how it's used -- e.g. you create an instance of it and stick it in a static final (the javadoc of `ClassValue` is useless in that respect):


ClassValue<CachedModel> CACHED_MODEL = new ClassValue<>() {
    @Override
    CachedModel computeValue() { <logic here> }
};


That should work here.

-------------

PR Review Comment: https://git.openjdk.org/babylon/pull/331#discussion_r1975129271

From asotona at openjdk.org  Fri Feb 28 09:59:48 2025
From: asotona at openjdk.org (Adam Sotona)
Date: Fri, 28 Feb 2025 09:59:48 GMT
Subject: [code-reflection] RFR: Adjusted tests to use execution of a method
 and implementation of the execution model cache [v2]
In-Reply-To: <lBxmJshq_VXbLGrxBEiSxt4vR050p3RJ5owpajloVao=.ff691c8c-0db5-49ec-b412-fa6ce9acb603@github.com>
References: <fh6o1Sf1-Hu57rvlxCW4MmP5C9pMVPlAzO5ao6eOFFE=.2a9557c5-52b7-4070-8562-94dede52272b@github.com>
 <kJollT6OPI-6HqMcz8yTHd37EjFduGxBOfBq7wD_OuY=.9a2fae87-1b2b-4e1a-982d-b15856ff43ad@github.com>
 <_iuflwkfM_Mijonhyrg3vCqMnvD0eYy6dkQY1hD7mR0=.2f8c0746-4507-497e-814e-be945edeb4e2@github.com>
 <lBxmJshq_VXbLGrxBEiSxt4vR050p3RJ5owpajloVao=.ff691c8c-0db5-49ec-b412-fa6ce9acb603@github.com>
Message-ID: <V4vVzxFaz4BAteJliy6dKPJ1aVusPSmZoJec_04cX_k=.e260e90f-730b-4571-9905-2383ff695552@github.com>

On Fri, 28 Feb 2025 09:51:35 GMT, Maurizio Cimadamore <mcimadamore at openjdk.org> wrote:

>> Unfortunately `ClassValue` does not support computation if absent (if more than just the bare class is needed). Maybe if compute returns null and the cached value is added later...?
>
> I believe `ClassValue` is similar to a thread local in how it's used -- e.g. you create an instance of it and stick it in a static final (the javadoc of `ClassValue` is useless in that respect):
> 
> 
> ClassValue<CachedModel> CACHED_MODEL = new ClassValue<>() {
>     @Override
>     CachedModel computeValue() { <logic here> }
> };
> 
> 
> That should work here.

Yes, I'm just having troubles to initiate the ClassValue when I need it (when I have additional arguments).
See my solution.

-------------

PR Review Comment: https://git.openjdk.org/babylon/pull/331#discussion_r1975137191

From asotona at openjdk.org  Fri Feb 28 10:04:01 2025
From: asotona at openjdk.org (Adam Sotona)
Date: Fri, 28 Feb 2025 10:04:01 GMT
Subject: [code-reflection] RFR: Adjusted tests to use execution of a method
 and implementation of the execution model cache [v2]
In-Reply-To: <V4vVzxFaz4BAteJliy6dKPJ1aVusPSmZoJec_04cX_k=.e260e90f-730b-4571-9905-2383ff695552@github.com>
References: <fh6o1Sf1-Hu57rvlxCW4MmP5C9pMVPlAzO5ao6eOFFE=.2a9557c5-52b7-4070-8562-94dede52272b@github.com>
 <kJollT6OPI-6HqMcz8yTHd37EjFduGxBOfBq7wD_OuY=.9a2fae87-1b2b-4e1a-982d-b15856ff43ad@github.com>
 <_iuflwkfM_Mijonhyrg3vCqMnvD0eYy6dkQY1hD7mR0=.2f8c0746-4507-497e-814e-be945edeb4e2@github.com>
 <lBxmJshq_VXbLGrxBEiSxt4vR050p3RJ5owpajloVao=.ff691c8c-0db5-49ec-b412-fa6ce9acb603@github.com>
 <V4vVzxFaz4BAteJliy6dKPJ1aVusPSmZoJec_04cX_k=.e260e90f-730b-4571-9905-2383ff695552@github.com>
Message-ID: <lqC1pv2FFKRX6Z7pO8HulNRXJFM_njcs1dETtJJgQg8=.5c085682-d2b0-4dd4-a73e-6531439e7679@github.com>

On Fri, 28 Feb 2025 09:56:07 GMT, Adam Sotona <asotona at openjdk.org> wrote:

>> I believe `ClassValue` is similar to a thread local in how it's used -- e.g. you create an instance of it and stick it in a static final (the javadoc of `ClassValue` is useless in that respect):
>> 
>> 
>> ClassValue<CachedModel> CACHED_MODEL = new ClassValue<>() {
>>     @Override
>>     CachedModel computeValue() { <logic here> }
>> };
>> 
>> 
>> That should work here.
>
> Yes, I'm just having troubles to initiate the ClassValue when I need it (when I have additional arguments).
> See my solution.

There is a design flaw in the ClassValue, the only way to initialize it is by ClassValue::computeValue(Class<?>).
It has been designed to attach some value to a class, however if your value needs some arguments to be computed, you need to attach them to the class, which you cannot. So it is a Catch-22.

-------------

PR Review Comment: https://git.openjdk.org/babylon/pull/331#discussion_r1975144551

From mcimadamore at openjdk.org  Fri Feb 28 10:14:05 2025
From: mcimadamore at openjdk.org (Maurizio Cimadamore)
Date: Fri, 28 Feb 2025 10:14:05 GMT
Subject: [code-reflection] RFR: Adjusted tests to use execution of a method
 and implementation of the execution model cache [v2]
In-Reply-To: <lqC1pv2FFKRX6Z7pO8HulNRXJFM_njcs1dETtJJgQg8=.5c085682-d2b0-4dd4-a73e-6531439e7679@github.com>
References: <fh6o1Sf1-Hu57rvlxCW4MmP5C9pMVPlAzO5ao6eOFFE=.2a9557c5-52b7-4070-8562-94dede52272b@github.com>
 <kJollT6OPI-6HqMcz8yTHd37EjFduGxBOfBq7wD_OuY=.9a2fae87-1b2b-4e1a-982d-b15856ff43ad@github.com>
 <_iuflwkfM_Mijonhyrg3vCqMnvD0eYy6dkQY1hD7mR0=.2f8c0746-4507-497e-814e-be945edeb4e2@github.com>
 <lBxmJshq_VXbLGrxBEiSxt4vR050p3RJ5owpajloVao=.ff691c8c-0db5-49ec-b412-fa6ce9acb603@github.com>
 <V4vVzxFaz4BAteJliy6dKPJ1aVusPSmZoJec_04cX_k=.e260e90f-730b-4571-9905-2383ff695552@github.com>
 <lqC1pv2FFKRX6Z7pO8HulNRXJFM_njcs1dETtJJgQg8=.5c085682-d2b0-4dd4-a73e-6531439e7679@github.com>
Message-ID: <h5bW5Y5UGZolBhAlwPZQXrnSPpT8qvfJMdEdKZtwGhk=.40c99e26-ba91-49d7-9b19-025a873c022b@github.com>

On Fri, 28 Feb 2025 10:01:25 GMT, Adam Sotona <asotona at openjdk.org> wrote:

>> Yes, I'm just having troubles to initiate the ClassValue when I need it (when I have additional arguments).
>> See my solution.
>
> There is a design flaw in the ClassValue, the only way to initialize it is by ClassValue::computeValue(Class<?>).
> It has been designed to attach some value to a class, however if your value needs some arguments to be computed, you need to attach them to the class, which you cannot. So it is a Catch-22.

Ah I see. I wouldn't say that's an issue with `ClassValue` -- but more of an issue with the fact that the class is not a valid key in your case (and `ClassValue` is better alternative to a  `Map<Class, V>`).

But here I wonder -- if the passed quotable instance does have an associated `Quoted` object -- doesn't it mean that it must be a class generated by the metafactory, and one that is specialized for that particular code model? E.g. I'm wondering if the class alone (after you verified there's a `Quoted` object in there) isn't already a good key?

-------------

PR Review Comment: https://git.openjdk.org/babylon/pull/331#discussion_r1975157497

From asotona at openjdk.org  Fri Feb 28 10:17:21 2025
From: asotona at openjdk.org (Adam Sotona)
Date: Fri, 28 Feb 2025 10:17:21 GMT
Subject: [code-reflection] RFR: Adjusted tests to use execution of a method
 and implementation of the execution model cache [v2]
In-Reply-To: <h5bW5Y5UGZolBhAlwPZQXrnSPpT8qvfJMdEdKZtwGhk=.40c99e26-ba91-49d7-9b19-025a873c022b@github.com>
References: <fh6o1Sf1-Hu57rvlxCW4MmP5C9pMVPlAzO5ao6eOFFE=.2a9557c5-52b7-4070-8562-94dede52272b@github.com>
 <kJollT6OPI-6HqMcz8yTHd37EjFduGxBOfBq7wD_OuY=.9a2fae87-1b2b-4e1a-982d-b15856ff43ad@github.com>
 <_iuflwkfM_Mijonhyrg3vCqMnvD0eYy6dkQY1hD7mR0=.2f8c0746-4507-497e-814e-be945edeb4e2@github.com>
 <lBxmJshq_VXbLGrxBEiSxt4vR050p3RJ5owpajloVao=.ff691c8c-0db5-49ec-b412-fa6ce9acb603@github.com>
 <V4vVzxFaz4BAteJliy6dKPJ1aVusPSmZoJec_04cX_k=.e260e90f-730b-4571-9905-2383ff695552@github.com>
 <lqC1pv2FFKRX6Z7pO8HulNRXJFM_njcs1dETtJJgQg8=.5c085682-d2b0-4dd4-a73e-6531439e7679@github.com>
 <h5bW5Y5UGZolBhAlwPZQXrnSPpT8qvfJMdEdKZtwGhk=.40c99e26-ba91-49d7-9b19-025a873c022b@github.com>
Message-ID: <fJp0uk-barf-PMT4J7onjAv0ZUN6N_Aj9oD9xDd1HUs=.2187cadb-a0cd-451f-b8e6-30435d0ebf0d@github.com>

On Fri, 28 Feb 2025 10:10:57 GMT, Maurizio Cimadamore <mcimadamore at openjdk.org> wrote:

>> There is a design flaw in the ClassValue, the only way to initialize it is by ClassValue::computeValue(Class<?>).
>> It has been designed to attach some value to a class, however if your value needs some arguments to be computed, you need to attach them to the class, which you cannot. So it is a Catch-22.
>
> Ah I see. I wouldn't say that's an issue with `ClassValue` -- but more of an issue with the fact that the class is not a valid key in your case (and `ClassValue` is better alternative to a  `Map<Class, V>`).
> 
> But here I wonder -- if the passed quotable instance does have an associated `Quoted` object -- doesn't it mean that it must be a class generated by the metafactory, and one that is specialized for that particular code model? E.g. I'm wondering if the class alone (after you verified there's a `Quoted` object in there) isn't already a good key?

I tried it and unfortunately a different instance of Quoted each time.

-------------

PR Review Comment: https://git.openjdk.org/babylon/pull/331#discussion_r1975162411

From asotona at openjdk.org  Fri Feb 28 10:27:21 2025
From: asotona at openjdk.org (Adam Sotona)
Date: Fri, 28 Feb 2025 10:27:21 GMT
Subject: [code-reflection] RFR: Adjusted tests to use execution of a method
 and implementation of the execution model cache [v2]
In-Reply-To: <fJp0uk-barf-PMT4J7onjAv0ZUN6N_Aj9oD9xDd1HUs=.2187cadb-a0cd-451f-b8e6-30435d0ebf0d@github.com>
References: <fh6o1Sf1-Hu57rvlxCW4MmP5C9pMVPlAzO5ao6eOFFE=.2a9557c5-52b7-4070-8562-94dede52272b@github.com>
 <kJollT6OPI-6HqMcz8yTHd37EjFduGxBOfBq7wD_OuY=.9a2fae87-1b2b-4e1a-982d-b15856ff43ad@github.com>
 <_iuflwkfM_Mijonhyrg3vCqMnvD0eYy6dkQY1hD7mR0=.2f8c0746-4507-497e-814e-be945edeb4e2@github.com>
 <lBxmJshq_VXbLGrxBEiSxt4vR050p3RJ5owpajloVao=.ff691c8c-0db5-49ec-b412-fa6ce9acb603@github.com>
 <V4vVzxFaz4BAteJliy6dKPJ1aVusPSmZoJec_04cX_k=.e260e90f-730b-4571-9905-2383ff695552@github.com>
 <lqC1pv2FFKRX6Z7pO8HulNRXJFM_njcs1dETtJJgQg8=.5c085682-d2b0-4dd4-a73e-6531439e7679@github.com>
 <h5bW5Y5UGZolBhAlwPZQXrnSPpT8qvfJMdEdKZtwGhk=.40c99e26-ba91-49d7-9b19-025a873c022b@github.com>
 <fJp0uk-barf-PMT4J7onjAv0ZUN6N_Aj9oD9xDd1HUs=.2187cadb-a0cd-451f-b8e6-30435d0ebf0d@github.com>
Message-ID: <pGRGCYK-RbZLLS1EcNylpFPNpzNlTngQ4Q8XM5x2hzE=.205d3f40-845a-45e6-9b8d-8a0220ecd2ad@github.com>

On Fri, 28 Feb 2025 10:14:41 GMT, Adam Sotona <asotona at openjdk.org> wrote:

>> Ah I see. I wouldn't say that's an issue with `ClassValue` -- but more of an issue with the fact that the class is not a valid key in your case (and `ClassValue` is better alternative to a  `Map<Class, V>`).
>> 
>> But here I wonder -- if the passed quotable instance does have an associated `Quoted` object -- doesn't it mean that it must be a class generated by the metafactory, and one that is specialized for that particular code model? E.g. I'm wondering if the class alone (after you verified there's a `Quoted` object in there) isn't already a good key?
>
> Which class? I use codeLambda class as a key.

BTW: ThreadLocal does not have this problem, it can be set at any time by any value.

-------------

PR Review Comment: https://git.openjdk.org/babylon/pull/331#discussion_r1975175450

From asotona at openjdk.org  Fri Feb 28 10:40:56 2025
From: asotona at openjdk.org (Adam Sotona)
Date: Fri, 28 Feb 2025 10:40:56 GMT
Subject: git: openjdk/babylon: code-reflection: Adjusted tests to use execution
 of a method and implementation of the execution model cache
Message-ID: <575325b8-ac57-44a4-bc7e-3bdbb4a58a36@openjdk.org>

Changeset: 74b14659
Branch: code-reflection
Author:    Adam Sotona <asotona at openjdk.org>
Date:      2025-02-28 10:38:51 +0000
URL:       https://git.openjdk.org/babylon/commit/74b146592cadeb25ce0b7ea151e405999b2205cf

Adjusted tests to use execution of a method and implementation of the execution model cache

Reviewed-by: psandoz

! cr-examples/onnx/src/main/java/oracle/code/onnx/OnnxProtoPrinter.java
! cr-examples/onnx/src/main/java/oracle/code/onnx/OnnxRuntime.java
! cr-examples/onnx/src/test/java/oracle/code/onnx/CNNTest.java
! cr-examples/onnx/src/test/java/oracle/code/onnx/SimpleTest.java


From asotona at openjdk.org  Fri Feb 28 10:42:07 2025
From: asotona at openjdk.org (Adam Sotona)
Date: Fri, 28 Feb 2025 10:42:07 GMT
Subject: [code-reflection] RFR: Adjusted tests to use execution of a method
 and implementation of the execution model cache [v2]
In-Reply-To: <GzQ6cmCzw0MZANYL03vzPCWBXsvN37-0ffdZn3ChfkI=.e7308c28-4460-43f7-ad29-c096569da242@github.com>
References: <fh6o1Sf1-Hu57rvlxCW4MmP5C9pMVPlAzO5ao6eOFFE=.2a9557c5-52b7-4070-8562-94dede52272b@github.com>
 <GzQ6cmCzw0MZANYL03vzPCWBXsvN37-0ffdZn3ChfkI=.e7308c28-4460-43f7-ad29-c096569da242@github.com>
Message-ID: <54Ss3qqDvY9TDqEag6Gf5njuxXIb-nZQx5R7Sa9qCsg=.695a6a91-3b5f-4205-9c8f-75141a72eff4@github.com>

On Fri, 28 Feb 2025 09:59:46 GMT, Adam Sotona <asotona at openjdk.org> wrote:

>> Adjusted tests to use execution of a method and implementation of the execution model cache.
>
> Adam Sotona has updated the pull request incrementally with one additional commit since the last revision:
> 
>   recommended changes applied

Thanks for all the reviews and comments! It would be interesting to follow up on some topics.

-------------

PR Comment: https://git.openjdk.org/babylon/pull/331#issuecomment-2690307730

From asotona at openjdk.org  Fri Feb 28 10:42:07 2025
From: asotona at openjdk.org (Adam Sotona)
Date: Fri, 28 Feb 2025 10:42:07 GMT
Subject: [code-reflection] Integrated: Adjusted tests to use execution of a
 method and implementation of the execution model cache
In-Reply-To: <fh6o1Sf1-Hu57rvlxCW4MmP5C9pMVPlAzO5ao6eOFFE=.2a9557c5-52b7-4070-8562-94dede52272b@github.com>
References: <fh6o1Sf1-Hu57rvlxCW4MmP5C9pMVPlAzO5ao6eOFFE=.2a9557c5-52b7-4070-8562-94dede52272b@github.com>
Message-ID: <LB6wIBvgofiHmJQ_ia8wM4rA89fghdYeCBX1LEMRvAs=.84fd592e-5114-4be3-8c5c-4df8dc1bd025@github.com>

On Thu, 27 Feb 2025 16:50:04 GMT, Adam Sotona <asotona at openjdk.org> wrote:

> Adjusted tests to use execution of a method and implementation of the execution model cache.

This pull request has now been integrated.

Changeset: 74b14659
Author:    Adam Sotona <asotona at openjdk.org>
URL:       https://git.openjdk.org/babylon/commit/74b146592cadeb25ce0b7ea151e405999b2205cf
Stats:     165 lines in 4 files changed: 76 ins; 31 del; 58 mod

Adjusted tests to use execution of a method and implementation of the execution model cache

Reviewed-by: psandoz

-------------

PR: https://git.openjdk.org/babylon/pull/331

From mcimadamore at openjdk.org  Fri Feb 28 10:55:17 2025
From: mcimadamore at openjdk.org (Maurizio Cimadamore)
Date: Fri, 28 Feb 2025 10:55:17 GMT
Subject: [code-reflection] RFR: Adjusted tests to use execution of a method
 and implementation of the execution model cache [v2]
In-Reply-To: <pGRGCYK-RbZLLS1EcNylpFPNpzNlTngQ4Q8XM5x2hzE=.205d3f40-845a-45e6-9b8d-8a0220ecd2ad@github.com>
References: <fh6o1Sf1-Hu57rvlxCW4MmP5C9pMVPlAzO5ao6eOFFE=.2a9557c5-52b7-4070-8562-94dede52272b@github.com>
 <kJollT6OPI-6HqMcz8yTHd37EjFduGxBOfBq7wD_OuY=.9a2fae87-1b2b-4e1a-982d-b15856ff43ad@github.com>
 <_iuflwkfM_Mijonhyrg3vCqMnvD0eYy6dkQY1hD7mR0=.2f8c0746-4507-497e-814e-be945edeb4e2@github.com>
 <lBxmJshq_VXbLGrxBEiSxt4vR050p3RJ5owpajloVao=.ff691c8c-0db5-49ec-b412-fa6ce9acb603@github.com>
 <V4vVzxFaz4BAteJliy6dKPJ1aVusPSmZoJec_04cX_k=.e260e90f-730b-4571-9905-2383ff695552@github.com>
 <lqC1pv2FFKRX6Z7pO8HulNRXJFM_njcs1dETtJJgQg8=.5c085682-d2b0-4dd4-a73e-6531439e7679@github.com>
 <h5bW5Y5UGZolBhAlwPZQXrnSPpT8qvfJMdEdKZtwGhk=.40c99e26-ba91-49d7-9b19-025a873c022b@github.com>
 <fJp0uk-barf-PMT4J7onjAv0ZUN6N_Aj9oD9xDd1HUs=.2187cadb-a0cd-451f-b8e6-30435d0ebf0d@github.com>
 <pGRGCYK-RbZLLS1EcNylpFPNpzNlTngQ4Q8XM5x2hzE=.205d3f40-845a-45e6-9b8d-8a0220ecd2ad@github.com>
Message-ID: <sIytNwSRjEFdZzS9MB2Hph8LmN8x4E1jF02DT0ifWHI=.280e29a2-2778-4ca7-b27c-d5aa1689849e@github.com>

On Fri, 28 Feb 2025 10:24:17 GMT, Adam Sotona <asotona at openjdk.org> wrote:

>> Which class? I use codeLambda class as a key.
>
> BTW: ThreadLocal does not have this problem, it can be set at any time by any value.

Ok - the issue here is that you need the lookup and the Quoted object to create the CachedModel instance

-------------

PR Review Comment: https://git.openjdk.org/babylon/pull/331#discussion_r1975212068

From mcimadamore at openjdk.org  Fri Feb 28 12:46:49 2025
From: mcimadamore at openjdk.org (Maurizio Cimadamore)
Date: Fri, 28 Feb 2025 12:46:49 GMT
Subject: [code-reflection] RFR: Model lifetimes of onnx session-related objects
 more explicitly
Message-ID: <s1RyI3MmwaIiMa3H__eLqjIAHbGb058ONRI3UZVtUkY=.4fd6ec15-bf2b-4f10-b168-c7d9d0dc23ed@github.com>

The class representing an onnx session is auto closeable. But, in the current code, a session is closed immediately after its `run` method is called. This is problematic because a session returns some ORTValues (tensors) which also need to be freed, but that cannot be freed immediately after calling `run` (as they need to be used by clients).

To address this problem, I tweaked the session code to accept an external arena. All the allocation of session-related data structures now happens using that external arena. This means that the client can now be in charge of managing the lifetime of a session (see changes to MNIST demo).

To test, I tweaked the MNIST code to do 10K iterations on each button pressed. Predictably, a single button pressed resulted in over 3g of memory being leaked. With these changes the memory arrives at ~400K (there is still some minor leak, but not sure worth pushing more).

If the changes to the demo are not deemed good, I can withdraw this PR -- I mostly wanted to capture the result of my exploration somewhere.

-------------

Commit messages:
 - Fix bug: replace confined arena with automatic arena in OnnxInterpreter
 - Model lifetimes of session-related objects more explicitly

Changes: https://git.openjdk.org/babylon/pull/332/files
  Webrev: https://webrevs.openjdk.org/?repo=babylon&pr=332&range=00
  Stats: 90 lines in 4 files changed: 24 ins; 9 del; 57 mod
  Patch: https://git.openjdk.org/babylon/pull/332.diff
  Fetch: git fetch https://git.openjdk.org/babylon.git pull/332/head:pull/332

PR: https://git.openjdk.org/babylon/pull/332

From mcimadamore at openjdk.org  Fri Feb 28 13:10:09 2025
From: mcimadamore at openjdk.org (Maurizio Cimadamore)
Date: Fri, 28 Feb 2025 13:10:09 GMT
Subject: [code-reflection] RFR: Model lifetimes of onnx session-related
 objects more explicitly
In-Reply-To: <s1RyI3MmwaIiMa3H__eLqjIAHbGb058ONRI3UZVtUkY=.4fd6ec15-bf2b-4f10-b168-c7d9d0dc23ed@github.com>
References: <s1RyI3MmwaIiMa3H__eLqjIAHbGb058ONRI3UZVtUkY=.4fd6ec15-bf2b-4f10-b168-c7d9d0dc23ed@github.com>
Message-ID: <ZCl9dkU0hPyYzymSPQczRobiGF-fGoYOiN8o--VIIbE=.29c14409-5459-4701-8077-b7bbaf096a93@github.com>

On Fri, 28 Feb 2025 12:42:24 GMT, Maurizio Cimadamore <mcimadamore at openjdk.org> wrote:

> If the changes to the demo are not deemed good, I can withdraw this PR -- I mostly wanted to capture the result of my exploration somewhere.

I have tried an experiment where I used an automatic arena to model the session lifetime -- but unfortunately the GC doesn't seem to get rid of off-heap memory fast enough -- probably there's not enough Java object being allocated to justify a reachability analysis.

-------------

PR Comment: https://git.openjdk.org/babylon/pull/332#issuecomment-2690607634

From mcimadamore at openjdk.org  Fri Feb 28 14:42:17 2025
From: mcimadamore at openjdk.org (Maurizio Cimadamore)
Date: Fri, 28 Feb 2025 14:42:17 GMT
Subject: [code-reflection] RFR: Model lifetimes of onnx session-related
 objects more explicitly
In-Reply-To: <s1RyI3MmwaIiMa3H__eLqjIAHbGb058ONRI3UZVtUkY=.4fd6ec15-bf2b-4f10-b168-c7d9d0dc23ed@github.com>
References: <s1RyI3MmwaIiMa3H__eLqjIAHbGb058ONRI3UZVtUkY=.4fd6ec15-bf2b-4f10-b168-c7d9d0dc23ed@github.com>
Message-ID: <QDzVMGVkAShwKawjx1AZNxg9qTNfA51VmRM0DilaPyo=.6357ce17-5386-4c6c-b8bd-5ddca73e0b29@github.com>

On Fri, 28 Feb 2025 12:42:24 GMT, Maurizio Cimadamore <mcimadamore at openjdk.org> wrote:

> The class representing an onnx session is auto closeable. But, in the current code, a session is closed immediately after its `run` method is called. This is problematic because a session returns some ORTValues (tensors) which also need to be freed, but that cannot be freed immediately after calling `run` (as they need to be used by clients).
> 
> To address this problem, I tweaked the session code to accept an external arena. All the allocation of session-related data structures now happens using that external arena. This means that the client can now be in charge of managing the lifetime of a session (see changes to MNIST demo).
> 
> To test, I tweaked the MNIST code to do 10K iterations on each button pressed. Predictably, a single button pressed resulted in over 3g of memory being leaked. With these changes the memory arrives at ~400K (there is still some minor leak, but not sure worth pushing more).
> 
> If the changes to the demo are not deemed good, I can withdraw this PR -- I mostly wanted to capture the result of my exploration somewhere.

cr-examples/onnx/src/main/java/oracle/code/onnx/OnnxInterpreter.java line 61:

> 59:                               .toList(),
> 60:                     schema.outputs().size(),
> 61:                     attributeMap, Arena.ofAuto());

Using an explicit arena here would cause just too much ripple effects in all the auto-generated op classes

-------------

PR Review Comment: https://git.openjdk.org/babylon/pull/332#discussion_r1975524474

From mcimadamore at openjdk.org  Fri Feb 28 14:51:08 2025
From: mcimadamore at openjdk.org (Maurizio Cimadamore)
Date: Fri, 28 Feb 2025 14:51:08 GMT
Subject: [code-reflection] RFR: Model lifetimes of onnx session-related
 objects more explicitly
In-Reply-To: <s1RyI3MmwaIiMa3H__eLqjIAHbGb058ONRI3UZVtUkY=.4fd6ec15-bf2b-4f10-b168-c7d9d0dc23ed@github.com>
References: <s1RyI3MmwaIiMa3H__eLqjIAHbGb058ONRI3UZVtUkY=.4fd6ec15-bf2b-4f10-b168-c7d9d0dc23ed@github.com>
Message-ID: <3rEXfJ-X4gsGawE83JC6l6kGM-O6j6TIr2AZ0zQ71Kg=.7f50f341-638c-4737-a37a-fecb3c9dc001@github.com>

On Fri, 28 Feb 2025 12:42:24 GMT, Maurizio Cimadamore <mcimadamore at openjdk.org> wrote:

> The class representing an onnx session is auto closeable. But, in the current code, a session is closed immediately after its `run` method is called. This is problematic because a session returns some ORTValues (tensors) which also need to be freed, but that cannot be freed immediately after calling `run` (as they need to be used by clients).
> 
> To address this problem, I tweaked the session code to accept an external arena. All the allocation of session-related data structures now happens using that external arena. This means that the client can now be in charge of managing the lifetime of a session (see changes to MNIST demo).
> 
> To test, I tweaked the MNIST code to do 10K iterations on each button pressed. Predictably, a single button pressed resulted in over 3g of memory being leaked. With these changes the memory arrives at ~400K (there is still some minor leak, but not sure worth pushing more).
> 
> If the changes to the demo are not deemed good, I can withdraw this PR -- I mostly wanted to capture the result of my exploration somewhere.

cr-examples/onnx/src/test/java/oracle/code/onnx/MNISTDemo.java line 149:

> 147:                     var imageData = new float[IMAGE_SIZE * IMAGE_SIZE];
> 148:                     scaledImage.getData().getSamples(0, 0, IMAGE_SIZE, IMAGE_SIZE, 0, imageData);
> 149:                     var imageTensor = Tensor.ofShape(new long[]{1, 1, IMAGE_SIZE, IMAGE_SIZE}, imageData);

Would be nice if we could move this inside the TWR, so that we can associate the tensor memory with the arena session (and dispose everything in one go)

-------------

PR Review Comment: https://git.openjdk.org/babylon/pull/332#discussion_r1975538107

From mcimadamore at openjdk.org  Fri Feb 28 14:57:13 2025
From: mcimadamore at openjdk.org (Maurizio Cimadamore)
Date: Fri, 28 Feb 2025 14:57:13 GMT
Subject: [code-reflection] RFR: Model lifetimes of onnx session-related
 objects more explicitly
In-Reply-To: <s1RyI3MmwaIiMa3H__eLqjIAHbGb058ONRI3UZVtUkY=.4fd6ec15-bf2b-4f10-b168-c7d9d0dc23ed@github.com>
References: <s1RyI3MmwaIiMa3H__eLqjIAHbGb058ONRI3UZVtUkY=.4fd6ec15-bf2b-4f10-b168-c7d9d0dc23ed@github.com>
Message-ID: <k_XSmyUzdwi9AdTGZWXyhPnzOoAXhnZ6yzxw7Oj5Gnw=.4cef8879-48e5-474a-9780-4923127b5b24@github.com>

On Fri, 28 Feb 2025 12:42:24 GMT, Maurizio Cimadamore <mcimadamore at openjdk.org> wrote:

> With these changes the memory arrives at ~400K (there is still some minor leak, but not sure worth pushing more).

Actually, I stand corrected -- I don't think there's any leak after the changes in this PR are applied. While memory keeps increasing up to 550Kb -- it then stays pegged there. So what I thought was a leak was likely the Java heap expanding before the GC kicked in.

-------------

PR Comment: https://git.openjdk.org/babylon/pull/332#issuecomment-2690843490

From psandoz at openjdk.org  Fri Feb 28 21:08:07 2025
From: psandoz at openjdk.org (Paul Sandoz)
Date: Fri, 28 Feb 2025 21:08:07 GMT
Subject: [code-reflection] RFR: Model lifetimes of onnx session-related
 objects more explicitly
In-Reply-To: <s1RyI3MmwaIiMa3H__eLqjIAHbGb058ONRI3UZVtUkY=.4fd6ec15-bf2b-4f10-b168-c7d9d0dc23ed@github.com>
References: <s1RyI3MmwaIiMa3H__eLqjIAHbGb058ONRI3UZVtUkY=.4fd6ec15-bf2b-4f10-b168-c7d9d0dc23ed@github.com>
Message-ID: <36W4fRHm2uq3icp4f1mc3Yi73d_a4_zrLI7ZXBOipsU=.5051d503-b012-4c11-9230-3cce24bc94a2@github.com>

On Fri, 28 Feb 2025 12:42:24 GMT, Maurizio Cimadamore <mcimadamore at openjdk.org> wrote:

> The class representing an onnx session is auto closeable. But, in the current code, a session is closed immediately after its `run` method is called. This is problematic because a session returns some ORTValues (tensors) which also need to be freed, but that cannot be freed immediately after calling `run` (as they need to be used by clients).
> 
> To address this problem, I tweaked the session code to accept an external arena. All the allocation of session-related data structures now happens using that external arena. This means that the client can now be in charge of managing the lifetime of a session (see changes to MNIST demo).
> 
> To test, I tweaked the MNIST code to do 10K iterations on each button pressed. Predictably, a single button pressed resulted in over 3g of memory being leaked. With these changes the memory arrives at ~400K (there is still some minor leak, but not sure worth pushing more).
> 
> If the changes to the demo are not deemed good, I can withdraw this PR -- I mostly wanted to capture the result of my exploration somewhere.

I think you are on to something here. I am wondering if we can refame it like this:


try (OnnxRuntime or = OnnxRuntime.of(MethodHandles.lookup(), Area.ofConfined()) {
  or.execute(() -> cnn(imageTensor))
}


?

I realize there can be only one runtime instance of a native ONNX runtime but i don't think we should be beholden to that constraint regarding the scoping of executing one or more models and with tensor memory under same lookup and arena (and even execution mode). Maybe there is a better name than `OnnxRuntime` to avoid any confusion.

-------------

PR Comment: https://git.openjdk.org/babylon/pull/332#issuecomment-2691545090

From mcimadamore at openjdk.org  Fri Feb 28 22:24:03 2025
From: mcimadamore at openjdk.org (Maurizio Cimadamore)
Date: Fri, 28 Feb 2025 22:24:03 GMT
Subject: [code-reflection] RFR: Model lifetimes of onnx session-related
 objects more explicitly
In-Reply-To: <36W4fRHm2uq3icp4f1mc3Yi73d_a4_zrLI7ZXBOipsU=.5051d503-b012-4c11-9230-3cce24bc94a2@github.com>
References: <s1RyI3MmwaIiMa3H__eLqjIAHbGb058ONRI3UZVtUkY=.4fd6ec15-bf2b-4f10-b168-c7d9d0dc23ed@github.com>
 <36W4fRHm2uq3icp4f1mc3Yi73d_a4_zrLI7ZXBOipsU=.5051d503-b012-4c11-9230-3cce24bc94a2@github.com>
Message-ID: <aZZu9PNMaJT-9_gw8beTFcUO1r5IkAVKV0w4SDudTk8=.4dec9faf-3708-47cb-8cdf-5ab45e00f35c@github.com>

On Fri, 28 Feb 2025 21:05:49 GMT, Paul Sandoz <psandoz at openjdk.org> wrote:

> I think you are on to something here. I am wondering if we can refame it like this:
> 
> ```
> try (OnnxRuntime or = OnnxRuntime.of(MethodHandles.lookup(), Area.ofConfined()) {
>   or.execute(() -> cnn(imageTensor))
> }
> ```
> 
> ?
> 
> I realize there can be only one runtime instance of a native ONNX runtime but i don't think we should be beholden to that constraint regarding the scoping of executing one or more models and with tensor memory under same lookup and arena (and even execution mode). Maybe there is a better name than `OnnxRuntime` to avoid any confusion.

I think you are correct in that the high-level API we want doesn't really need to expose all this ONNX "cruft" (e.g. it seems neither high-level nor low-level enough). From the perspective of the Java client, there should be some kind of ONNX "session" (which might, or might not be related to the ONNX session struct in the C API) -- which has a user-defined lifetime. The only interesting method in this session object is "run" and when you "close", the lifetime ends, and all the resources are garbage collected.

Maybe we can even do something like:


try (OnnxSession onnx = OnnxSession.of()) {
  or.execute(lookup, () -> cnn(imageTensor))
}


And hide the confined arena creation inside the API alltogether. (I'm not sure where `lookup` goes).

-------------

PR Comment: https://git.openjdk.org/babylon/pull/332#issuecomment-2691645157

From duke at openjdk.org  Fri Feb 28 22:34:04 2025
From: duke at openjdk.org (Adam Pocock)
Date: Fri, 28 Feb 2025 22:34:04 GMT
Subject: [code-reflection] RFR: Model lifetimes of onnx session-related
 objects more explicitly
In-Reply-To: <s1RyI3MmwaIiMa3H__eLqjIAHbGb058ONRI3UZVtUkY=.4fd6ec15-bf2b-4f10-b168-c7d9d0dc23ed@github.com>
References: <s1RyI3MmwaIiMa3H__eLqjIAHbGb058ONRI3UZVtUkY=.4fd6ec15-bf2b-4f10-b168-c7d9d0dc23ed@github.com>
Message-ID: <wQsHxLFBElSJW38EV7-z6kwfrpftBp_NUxo_HWnOB-c=.0bc17cee-3bc9-42f6-9892-6b35dd861125@github.com>

On Fri, 28 Feb 2025 12:42:24 GMT, Maurizio Cimadamore <mcimadamore at openjdk.org> wrote:

> The class representing an onnx session is auto closeable. But, in the current code, a session is closed immediately after its `run` method is called. This is problematic because a session returns some ORTValues (tensors) which also need to be freed, but that cannot be freed immediately after calling `run` (as they need to be used by clients).
> 
> To address this problem, I tweaked the session code to accept an external arena. All the allocation of session-related data structures now happens using that external arena. This means that the client can now be in charge of managing the lifetime of a session (see changes to MNIST demo).
> 
> To test, I tweaked the MNIST code to do 10K iterations on each button pressed. Predictably, a single button pressed resulted in over 3g of memory being leaked. With these changes the memory arrives at ~400K (there is still some minor leak, but not sure worth pushing more).
> 
> If the changes to the demo are not deemed good, I can withdraw this PR -- I mostly wanted to capture the result of my exploration somewhere.

The session in the C API does expose other useful things which allow you to introspect over the size & shape of the inputs & outputs, along with any metadata the user put into the ONNX graph. For the babylon use case only the former seems relevant as there won't be any metadata from the protobuf.

Splitting the session from the environment is relevant, many ML pipelines contain multiple models (e.g. diffusion image generation systems are a few text embedding models, a diffusion model and a variational autoencoder which maps from the diffusion space into pixel space) and so there might need to be multiple sessions in flight at the same time. For an example of how the sessions work for a diffusion system you can see [this](https://github.com/oracle/sd4j).

-------------

PR Comment: https://git.openjdk.org/babylon/pull/332#issuecomment-2691656462