[code-reflection] RFR: Model lifetimes of onnx session-related objects more explicitly [v2]

Maurizio Cimadamore mcimadamore at openjdk.org
Mon Mar 3 12:21:38 UTC 2025


> The class representing an onnx session is auto closeable. But, in the current code, a session is closed immediately after its `run` method is called. This is problematic because a session returns some ORTValues (tensors) which also need to be freed, but that cannot be freed immediately after calling `run` (as they need to be used by clients).
> 
> To address this problem, I tweaked the session code to accept an external arena. All the allocation of session-related data structures now happens using that external arena. This means that the client can now be in charge of managing the lifetime of a session (see changes to MNIST demo).
> 
> To test, I tweaked the MNIST code to do 10K iterations on each button pressed. Predictably, a single button pressed resulted in over 3g of memory being leaked. With these changes the memory arrives at ~400K (there is still some minor leak, but not sure worth pushing more).
> 
> If the changes to the demo are not deemed good, I can withdraw this PR -- I mostly wanted to capture the result of my exploration somewhere.

Maurizio Cimadamore has updated the pull request incrementally with one additional commit since the last revision:

  Rename local arena variables

-------------

Changes:
  - all: https://git.openjdk.org/babylon/pull/332/files
  - new: https://git.openjdk.org/babylon/pull/332/files/09c802a1..615c5503

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=babylon&pr=332&range=01
 - incr: https://webrevs.openjdk.org/?repo=babylon&pr=332&range=00-01

  Stats: 9 lines in 2 files changed: 0 ins; 0 del; 9 mod
  Patch: https://git.openjdk.org/babylon/pull/332.diff
  Fetch: git fetch https://git.openjdk.org/babylon.git pull/332/head:pull/332

PR: https://git.openjdk.org/babylon/pull/332


More information about the babylon-dev mailing list