Question: remove memory heap segments

Sat May 22 14:18:21 UTC 2021

Hi all,

Thank you for your answers. It was just very loose idea and I didn’t even expect so many responses. I think that it came with some other thought which sometimes comes to my mind if segments can be a good replacement for arrays and bytebuffers (I.e. that’s something what I consider - should I provide integration with existing JDK and how deep it should be)

However with ability of pinning I think that things would be more then interesting - would like to see it - this would be a great performance gain for I/O.

Once again thank you for you answers and please have a great weekend.

Rado

On May 22, 2021, at 11:58 AM, Chris Vest <mr.chrisvest at gmail.com> wrote:

Hi,

I would like to add that there are frameworks built on Netty who exclusively use on-heap buffers because it allows them to ignore the reference counting API that we otherwise have on ByteBuf.
The drawback of that approach is that copying will happen on IO call sites, and this is sometimes a reasonable trade-off for the convenience.
Since MemorySegments largely unify the API to memory, I find that you only need to know if memory is on- or off-heap at IO call sites, so the height of the if-trees will be limited.
My experience of building a new buffer API on top of memory segments is that it's not a big deal that segments can be either on- or off-heap.
There were other limitations of the API that were more frustrating, but these have all been fixed in the latest panama-foreign code.
We haven't looked closely at the ABI or jextract parts of panama-foreign, but the ability to briefly pin heap segments sounds quite useful in that context.

Cheers,
Chris

On Sat, 22 May 2021 at 07:46, leerho <leerho at gmail.com<mailto:leerho at gmail.com>> wrote:
Hi Rado,

We use on-heap (and off-heap) segments today in Java 8.  Starting in 2017
we created a similar concept of MemorySegments, but using unsafe and other
internal classes to do it.  We needed an API that would enable us to read
and write to dynamic structs that could be on-heap, or off-heap in
explicitly allocated native memory, or in a ByteBuffer, or in memory-mapped
files.  Our segments have some of the similar concepts of Panama's FMA,
except FMA is much more powerful (and safer!).

Our use case is essentially what Maurizio was alluding to.  We have these
very compact, dynamic, C-like structs (and C-unions) that we need to access
and modify, and depending on the system application, they can be either
off-heap or on-heap or both.  And the API that operates on these structs
doesn't really care where they happen to be in memory.  However, when
creating, expanding or copying a struct or segment, which is mostly
performed in different processes, we do need to detect whether a specific
struct is in native memory or not and the new isNative() method addresses
that.  Bear in mind that our environments are typically very large systems
where we can have millions of these dynamic structs in memory at the same
time. So compact size and efficient performance is critical.   And until
Panama FMA, the Java language had very limited means of expressing this
kind of flexibility and power.

Lee.

On Fri, May 21, 2021 at 1:44 PM Maurizio Cimadamore <
maurizio.cimadamore at oracle.com<mailto:maurizio.cimadamore at oracle.com>> wrote:

> Hi Radoslaw,
> this is a topic we have considered during the design process. Heap
> segments are useful to interoperate with Java arrays and also to
> interoperate with heap-based byte-buffers (and, in the future, vectors).
> While it is possible to think about an API that only supports off-heap
> segments, we think that there are valid uses for heap segments too,
> especially in the context of foreign functions.
>
> Think about a function that returns a struct by value - if you want to
> pass that struct back to some other code, probably it would be better to
> just allocate that struct off-heap. But if the returned struct is only
> accessed one by the caller of the native function, using an heap segment
> as a backing storage is a perfectly reasonable alternative, which also
> requires less allocation cost (as the VM is much more optimized at
> allocating on-heap memory than it is at allocating off-heap memory).
> This can be done today, by wrapping a SegmentAllocator around an on-heap
> segment and pass that allocator to a native function which returns a
> struct by value.
>
> Other more futuristic use cases might allow you to pass on-heap segments
> to native functions directly, using _pinning_. The VM has ability to pin
> a region of heap for a certain duration - we do not have an API to
> expose this, but there are certainly context where something like this
> could come in very handy (of course this would have to be a restricted
> operation, as pinning memory for too long might create other problems).
>
> In the end, I think having the choice of using on-heap segments opens up
> interesting use cases for the API which would not be possible to express
> otherwise. As you said, having the same API to model both off heap and
> on-heap can lead to confusion, but I hope that in the new iteration some
> of that confusion is cleared [1].
>
> Other possible alternatives we have considered were to name native
> memory segments/address differently from their on-heap counterparts -
> e.g. by using different types. This doesn't work very well, as memory
> segments and addresses are often passed to var handles and method
> handles, and having a complex hierarchy of types can easily lead to
> "inexact" method handle call - so it is kind of a strong requirement for
> these API points to be monomorphic.
>
> Cheers
> Maurizio
>
> [1] - https://git.openjdk.java.net/panama-foreign/pull/514
>
>
> On 21/05/2021 19:19, Radosław Smogura wrote:
> > Hi all,
> >
> > I hope you have a good day.
> >
> > I’m not sure if this topic was actually touched, however I wonder if
> heap memory segments are useful, as for me those creates more noise.
> >
> > Let me explain my concern, (as probably you know) I try to write I/O
> layer with Panama (mainly for feedback purposes).
> >
> > I thought about creating NativeInputStream with semantics similar to
> plain stream, but with segment instead of byte array.
> >
> > However issue is that if user passes on heap segment I can’t downstream
> it to POSIX function.
> >
> > I think this, slightly, can be headache for such purposes, and will lead
> to if trees and very minor performance degradation.
> >
> > I think the need for checking if segment is native in the heap was as
> well expressed in
> https://mail.openjdk.java.net/pipermail/panama-dev/2021-May/013707.html
> >
> > So that’s why I wonder about on-heap segments.
> >
> > Kind regards,
> > Rado
> >
>