Question: remove memory heap segments

[leerho]

Hi Rado,

We use on-heap (and off-heap) segments today in Java 8.  Starting in 2017
we created a similar concept of MemorySegments, but using unsafe and other
internal classes to do it.  We needed an API that would enable us to read
and write to dynamic structs that could be on-heap, or off-heap in
explicitly allocated native memory, or in a ByteBuffer, or in memory-mapped
files.  Our segments have some of the similar concepts of Panama's FMA,
except FMA is much more powerful (and safer!).

Our use case is essentially what Maurizio was alluding to.  We have these
very compact, dynamic, C-like structs (and C-unions) that we need to access
and modify, and depending on the system application, they can be either
off-heap or on-heap or both.  And the API that operates on these structs
doesn't really care where they happen to be in memory.  However, when
creating, expanding or copying a struct or segment, which is mostly
performed in different processes, we do need to detect whether a specific
struct is in native memory or not and the new isNative() method addresses
that.  Bear in mind that our environments are typically very large systems
where we can have millions of these dynamic structs in memory at the same
time. So compact size and efficient performance is critical.   And until
Panama FMA, the Java language had very limited means of expressing this
kind of flexibility and power.

Lee.

On Fri, May 21, 2021 at 1:44 PM Maurizio Cimadamore <
maurizio.cimadamore at oracle.com> wrote:

> Hi Radoslaw,
> this is a topic we have considered during the design process. Heap
> segments are useful to interoperate with Java arrays and also to
> interoperate with heap-based byte-buffers (and, in the future, vectors).
> While it is possible to think about an API that only supports off-heap
> segments, we think that there are valid uses for heap segments too,
> especially in the context of foreign functions.
>
> Think about a function that returns a struct by value - if you want to
> pass that struct back to some other code, probably it would be better to
> just allocate that struct off-heap. But if the returned struct is only
> accessed one by the caller of the native function, using an heap segment
> as a backing storage is a perfectly reasonable alternative, which also
> requires less allocation cost (as the VM is much more optimized at
> allocating on-heap memory than it is at allocating off-heap memory).
> This can be done today, by wrapping a SegmentAllocator around an on-heap
> segment and pass that allocator to a native function which returns a
> struct by value.
>
> Other more futuristic use cases might allow you to pass on-heap segments
> to native functions directly, using _pinning_. The VM has ability to pin
> a region of heap for a certain duration - we do not have an API to
> expose this, but there are certainly context where something like this
> could come in very handy (of course this would have to be a restricted
> operation, as pinning memory for too long might create other problems).
>
> In the end, I think having the choice of using on-heap segments opens up
> interesting use cases for the API which would not be possible to express
> otherwise. As you said, having the same API to model both off heap and
> on-heap can lead to confusion, but I hope that in the new iteration some
> of that confusion is cleared [1].
>
> Other possible alternatives we have considered were to name native
> memory segments/address differently from their on-heap counterparts -
> e.g. by using different types. This doesn't work very well, as memory
> segments and addresses are often passed to var handles and method
> handles, and having a complex hierarchy of types can easily lead to
> "inexact" method handle call - so it is kind of a strong requirement for
> these API points to be monomorphic.
>
> Cheers
> Maurizio
>
> [1] - https://git.openjdk.java.net/panama-foreign/pull/514
>
>
> On 21/05/2021 19:19, Radosław Smogura wrote:
> > Hi all,
> >
> > I hope you have a good day.
> >
> > I’m not sure if this topic was actually touched, however I wonder if
> heap memory segments are useful, as for me those creates more noise.
> >
> > Let me explain my concern, (as probably you know) I try to write I/O
> layer with Panama (mainly for feedback purposes).
> >
> > I thought about creating NativeInputStream with semantics similar to
> plain stream, but with segment instead of byte array.
> >
> > However issue is that if user passes on heap segment I can’t downstream
> it to POSIX function.
> >
> > I think this, slightly, can be headache for such purposes, and will lead
> to if trees and very minor performance degradation.
> >
> > I think the need for checking if segment is native in the heap was as
> well expressed in
> https://mail.openjdk.java.net/pipermail/panama-dev/2021-May/013707.html
> >
> > So that’s why I wonder about on-heap segments.
> >
> > Kind regards,
> > Rado
> >
>