Question: remove memory heap segments

Sat May 22 09:58:22 UTC 2021

Hi,

I would like to add that there are frameworks built on Netty who
exclusively use on-heap buffers because it allows them to ignore the
reference counting API that we otherwise have on ByteBuf.
The drawback of that approach is that copying will happen on IO call sites,
and this is sometimes a reasonable trade-off for the convenience.
Since MemorySegments largely unify the API to memory, I find that you only
need to know if memory is on- or off-heap at IO call sites, so the height
of the if-trees will be limited.
My experience of building a new buffer API on top of memory segments is
that it's not a big deal that segments can be either on- or off-heap.
There were other limitations of the API that were more frustrating, but
these have all been fixed in the latest panama-foreign code.
We haven't looked closely at the ABI or jextract parts of panama-foreign,
but the ability to briefly pin heap segments sounds quite useful in that
context.

Cheers,
Chris

On Sat, 22 May 2021 at 07:46, leerho <leerho at gmail.com> wrote:

> Hi Rado,
>
> We use on-heap (and off-heap) segments today in Java 8.  Starting in 2017
> we created a similar concept of MemorySegments, but using unsafe and other
> internal classes to do it.  We needed an API that would enable us to read
> and write to dynamic structs that could be on-heap, or off-heap in
> explicitly allocated native memory, or in a ByteBuffer, or in memory-mapped
> files.  Our segments have some of the similar concepts of Panama's FMA,
> except FMA is much more powerful (and safer!).
>
> Our use case is essentially what Maurizio was alluding to.  We have these
> very compact, dynamic, C-like structs (and C-unions) that we need to access
> and modify, and depending on the system application, they can be either
> off-heap or on-heap or both.  And the API that operates on these structs
> doesn't really care where they happen to be in memory.  However, when
> creating, expanding or copying a struct or segment, which is mostly
> performed in different processes, we do need to detect whether a specific
> struct is in native memory or not and the new isNative() method addresses
> that.  Bear in mind that our environments are typically very large systems
> where we can have millions of these dynamic structs in memory at the same
> time. So compact size and efficient performance is critical.   And until
> Panama FMA, the Java language had very limited means of expressing this
> kind of flexibility and power.
>
> Lee.
>
> On Fri, May 21, 2021 at 1:44 PM Maurizio Cimadamore <
> maurizio.cimadamore at oracle.com> wrote:
>
> > Hi Radoslaw,
> > this is a topic we have considered during the design process. Heap
> > segments are useful to interoperate with Java arrays and also to
> > interoperate with heap-based byte-buffers (and, in the future, vectors).
> > While it is possible to think about an API that only supports off-heap
> > segments, we think that there are valid uses for heap segments too,
> > especially in the context of foreign functions.
> >
> > Think about a function that returns a struct by value - if you want to
> > pass that struct back to some other code, probably it would be better to
> > just allocate that struct off-heap. But if the returned struct is only
> > accessed one by the caller of the native function, using an heap segment
> > as a backing storage is a perfectly reasonable alternative, which also
> > requires less allocation cost (as the VM is much more optimized at
> > allocating on-heap memory than it is at allocating off-heap memory).
> > This can be done today, by wrapping a SegmentAllocator around an on-heap
> > segment and pass that allocator to a native function which returns a
> > struct by value.
> >
> > Other more futuristic use cases might allow you to pass on-heap segments
> > to native functions directly, using _pinning_. The VM has ability to pin
> > a region of heap for a certain duration - we do not have an API to
> > expose this, but there are certainly context where something like this
> > could come in very handy (of course this would have to be a restricted
> > operation, as pinning memory for too long might create other problems).
> >
> > In the end, I think having the choice of using on-heap segments opens up
> > interesting use cases for the API which would not be possible to express
> > otherwise. As you said, having the same API to model both off heap and
> > on-heap can lead to confusion, but I hope that in the new iteration some
> > of that confusion is cleared [1].
> >
> > Other possible alternatives we have considered were to name native
> > memory segments/address differently from their on-heap counterparts -
> > e.g. by using different types. This doesn't work very well, as memory
> > segments and addresses are often passed to var handles and method
> > handles, and having a complex hierarchy of types can easily lead to
> > "inexact" method handle call - so it is kind of a strong requirement for
> > these API points to be monomorphic.
> >
> > Cheers
> > Maurizio
> >
> > [1] - https://git.openjdk.java.net/panama-foreign/pull/514
> >
> >
> > On 21/05/2021 19:19, Radosław Smogura wrote:
> > > Hi all,
> > >
> > > I hope you have a good day.
> > >
> > > I’m not sure if this topic was actually touched, however I wonder if
> > heap memory segments are useful, as for me those creates more noise.
> > >
> > > Let me explain my concern, (as probably you know) I try to write I/O
> > layer with Panama (mainly for feedback purposes).
> > >
> > > I thought about creating NativeInputStream with semantics similar to
> > plain stream, but with segment instead of byte array.
> > >
> > > However issue is that if user passes on heap segment I can’t downstream
> > it to POSIX function.
> > >
> > > I think this, slightly, can be headache for such purposes, and will
> lead
> > to if trees and very minor performance degradation.
> > >
> > > I think the need for checking if segment is native in the heap was as
> > well expressed in
> > https://mail.openjdk.java.net/pipermail/panama-dev/2021-May/013707.html
> > >
> > > So that’s why I wonder about on-heap segments.
> > >
> > > Kind regards,
> > > Rado
> > >
> >
>