Using Layouts on heap - JDK18

[Maurizio Cimadamore]

On 03/06/2022 18:59, leerho wrote:
> Maurizio,
>
>     ```
>     MemorySegment segment = MemorySegment.ofArray(new
>     byte[layout.byteSize()]);
>     ```
>
>
> I thought of that too. It is obvious that I can derive a byteSize from 
> the layout, which passes to the ofArray method, but it isn't obvious 
> to me that I can subsequently access the array using the layout as I 
> don't see any linkage between the layout and the segment, because the 
> ofArray method is just getting a size and is oblivious to the fact 
> that I'm using a layout to get it.  Similarly, I can't find any 
> reference in the Layout API of a way to "assign" a layout to a 
> (previously created) segment.
Layouts cannot be "assigned" to segments. Layouts and segments are 
orthogonal abstractions. You create a segment, which contain some bytes, 
then you use layouts to _dereference_ the segment. A layout has 
knowledge about size, alignment and endianness, so if you pass that 
(along with a segment) to a dereference API, you have all the 
ingredients you need.
>
> I am just learning about layouts so I may be missing something.
>
> ###
>
>     * throw if the slice is not compatible with the layout alignment
>     constraints, or
>     * align the slice 
>
> 1. If the slice is just bytes wouldn't that always work?
> 2. I'm not sure what "align the slice" means? What would happen 
> underneath?  If the segment was created from an int[], and the layout 
> alignment required byte alignment -- what would "align the slice" do?
>
> #####
> Your mentioning of the alignment issue brings up another "asymmetry" 
> that I am somewhat troubled by.
>
> The "segment.asByteBuffer()" method is very limited to the case where 
> the segment was created from pure bytes.  If a segment wraps a heap 
> int[], and I try to acquire a ByteBuffer from the segment it fails.  
>  (And I do see the caveat "Throws UnsupportedOperationException" ... " 
> in the Javadocs.)  What also surprised me is that internally inside 
> the segment you retain the int[] structure!  Why?  Why don't you just 
> convert all inputs to just bytes?
You mean copying int[] into a byte[] ? That would not be just a "view" 
then, it would allocate memory. All the "as" methods are meant to create 
views over memory, and not copy contents.
>
> This is very different from ByteBuffer, where I can extract any of the 
> multibyte primitive arrays from the raw bytes of the buffer, because 
> the raw resource under the ByteBuffer is just a blob of bytes.  And 
> Unsafe provides even more flexibility.  I can start with an array of 
> longs and extract an array of ints because the underlying resource 
> view of Unsafe is just bytes.   So I am surprised you didn't do that 
> here.  I would think that having the internal view being just a blob 
> of bytes would be the most flexible.    If you want me to provide you 
> with a use case for this I can provide you one.

I think you are mistaken w.r.t. ByteBuffer. A ByteBuffer is backed by a 
byte[] - you can't turn that into an int[] w/o copying.

I also don't think that there is an Unsafe facility that lets you view a 
portion of a long[] as an int[] (you need a new Object header for the 
int[], so... again... allocation and copy).

Maybe you are talking about different things? Please provide a snippet 
of code using either Unsafe or ByteBuffer that does something that you 
think MemorySegment does not support. Perhaps the discussion would be 
easier that way.

Thanks
Maurizio

> Lee.
>
>
>
> On Fri, Jun 3, 2022 at 10:06 AM Maurizio Cimadamore 
> <maurizio.cimadamore at oracle.com> wrote:
>
>     Hi Lee,
>     in terms of heap allocation, how is your "allocate" different from
>     this:
>
>     ```
>     MemorySegment segment = MemorySegment.ofArray(new
>     byte[layout.byteSize()]);
>     ```
>
>     The above doesn't do any copy, it just creates a segment view over
>     the
>     array. (note that the carrier type used is important here - e.g. a
>     segment that wraps a byte[] has different properties when it comes to
>     alignment, than a segment that wraps long[]).
>
>     But then you bring up the slice method, which makes me think that,
>     perhaps you aren't after allocation at all - you have an existing
>     segment and you want to slice it, using a layout.
>
>     I think the slicing API overload you describe is a nice one (and one
>     that I found myself wanting at times too). Is "layout-oriented"
>     slicing
>     the main use case you are referring to here?
>
>     If we do this the "right" way, the slicing method would have to take
>     into account the required alignment of the provided layout and either:
>
>     * throw if the slice is not compatible with the layout alignment
>     constraints, or
>     * align the slice
>
>     Of these I would probably prefer the former, because it's "less
>     magic".
>
>     Maurizio
>
>     On 03/06/2022 17:56, leerho wrote:
>     > Hi,
>     > It is straightforward to create a native MemorySegment governed by a
>     > layout, but I can't see a way to create a heap segment governed
>     by a layout
>     > (without a copy).  We need to be able to read and write to
>     structs on-heap
>     > as well as off-heap.  Being able to overlay a Layout on a
>     segment derived
>     > from a ByteBuffer or from a mapped file would also be useful.
>     Perhaps these
>     > could be implemented via a slice?
>     >
>     > What I would like to see is something like:
>     >
>     > *MemorySegment *allocate*(MemoryLayout layout);  //base resource
>     is byte[]*
>     >
>     > OR
>     >
>     >> *MemorySegment *asSlice*(long offset, MemoryLayout layout)*
>     >
>     > The second could be applied to any base resource, not just off-heap.
>     >
>     > Am I missing something?
>     >
>     > Cheers,
>     > Lee.
>