zero-length segments

[Maurizio Cimadamore]

On 21/01/2022 11:33, Quân Anh Mai wrote:
> Hi,
>
> It seems the delegated Unsafe.allocateMemory and Unsafe.freeMemory 
> allows working with zero-sized segments. May I ask why do we have such 
> a restriction on MemorySegment.allocateNative?
As I explained, there's no reason as to why zero-sized segments are not 
allowed. It's just the way the API has started out.
> Furthermore, malloc and free also allow working with zero-sized 
> segments, so it seems that even the checks in Unsafe is unnecessary.

That's true - although I guess malloc has still freedom to return either 
NULL or a constant pointer - perhaps implementors of Unsafe wanted more 
portable behavior?

Maurizio

>
> Regards,
> Quan Anh
>
> malloc - cppreference.com 
> <https://urldefense.com/v3/__https://en.cppreference.com/w/c/memory/malloc__;!!ACWV5N9M2RV99hQ!Z98zoj9sxZ1kO4YtwhCwwgsnkad_tgvFkHtLZi9tan9NQvcVcSerxkcoOBCf1CkXMNfpurc$>
> free - cppreference.com 
> <https://urldefense.com/v3/__https://en.cppreference.com/w/c/memory/free__;!!ACWV5N9M2RV99hQ!Z98zoj9sxZ1kO4YtwhCwwgsnkad_tgvFkHtLZi9tan9NQvcVcSerxkcoOBCf1CkXAMZTOIg$>
>
> On Fri, 21 Jan 2022 at 18:51, Maurizio Cimadamore 
> <maurizio.cimadamore at oracle.com 
> <mailto:maurizio.cimadamore at oracle.com>> wrote:
>
>     Hi Michael,
>     I think I'm sympathetic with your argument. Not only most of the
>     other
>     factories, as you noticed, do allow for zero-length segments, but
>     there's also the MemorySegment::mapFile method, which specifically
>     returns an instance of a special subclass if the mapped size if zero.
>
>     I also did some more tests with the ByteBuffer API, which allows
>     allocation (as you mention) with size = 0, but also allows slicing
>     with
>     slice size = 0 and limit of 0.
>
>     (Java arrays are another case where creating a zero-element array is
>     indeed possible).
>
>     All this evidence point to the fact that, yes, saying no to zero-byte
>     allocation on memory segment (of any kind) is at the very least
>     problematic in terms of interop with existing APIs, as it will cause
>     surprising behavior.
>
>     For this reason, I believe the best course of action is to enhance
>     the
>     API in the way you suggest, and accept zero-sized segments.
>
>     Thanks for the feedback!
>
>     Thinking of possible workarounds in the short term - but maybe you
>     are
>     already doing it - you could have something like this:
>
>     ```
>     private static final MemorySegment EMPTY =
>     MemorySegment.ofByteBuffer(ByteBuffer.allocateDirect(0));
>
>     MemorySegment wrapAddress(MemoryAddress address, long size,
>     ResourceScope scope) {
>          return size == 0 ? EMPTY : MemorySegment.ofAddress(address,
>     size,
>     scope);
>     }
>     ```
>
>     In terms of performance, there are a couple of points to note:
>
>     * adding a singleton anonymous class for zero-sized segment might
>     cause
>     profile pollution when using the same callsite with empty and
>     non-empty
>     segments
>     * having a branch (like in the above code) so that a singleton is
>     returned if size == 0 effectively disables escape analysis most of
>     the
>     times (when size is not known by C2)
>
>     And, other pseudo-random considerations:
>
>     * Looking at the impl of ByteBuffer.allocateDirect, it seems like it
>     always allocates at least one byte:
>
>     ```
>     long size = Math.max(1L, (long)cap + (pa ? ps : 0));
>     Bits.reserveMemory(size, cap);
>     ```
>
>     You can see how this is suboptimal (and probably not what a
>     programmer
>     would expect).
>
>     * In the case of MemorySegment, returning a singleton is not really
>     possible, because (as for mapped segments) the user is also
>     providing a
>     scope parameter, and it expects that the returned segment will
>     have same
>     scope as the provided parameter.
>
>     * All things considered, given it's late for 18, I'd prefer to
>     address
>     this in 19 - but I do want to address it.
>
>     Thanks
>     Maurizio
>
>     On 20/01/2022 22:50, Michael Zucchi wrote:
>     >
>     > Morning all,
>     >
>     > After a long break i've started experimenting with the foreign
>     abi to
>     > replace jni.  One tool i'm working on is a vulkan binding generator
>     > that works directly from the xml specification and generates a
>     'nice'
>     > api (particularly focusing on constructors for all the hundreds of
>     > configuration structures needed for vulkan, plus the dynamic
>     function
>     > tables), and another tool generates high-level and potentially
>     object
>     > oriented api's from c header files, for this one I use a gcc
>     plugin to
>     > extract the structures and functions and i'm trying opencl and
>     ffmpeg
>     > as test cases (and to update some projects i maintain, zcl and
>     > jjmpeg).  I'm not using jextract because I want to create the high
>     > level api directly and provide much more control on the created
>     > classes, and also because i've got the time and nothing better
>     to do
>     > with it.
>     >
>     > In general it looks pretty good after so much work, but i've come
>     > across one oddity which adds complexity to the java code for no
>     > obvious reason.  Is there any specific reason you cannot create
>     zero
>     > length memory segments *in some cases*?
>     >
>     > e.g. something like this comes up often in C:
>     >
>     > struct blob {
>     >    size_t data_size;
>     >    uint8_t *data;
>     > }
>     >
>     > From java at some point you want to get a MemorySegment to access
>     > blob.data.  It might be through a high level api such as:
>     >
>     >   class blob {
>     >     MemorySegment getData() {
>     >         return MemorySegment.ofAddress(...);
>     >     }
>     >  }
>     >
>     > If the memory was allocated in the native code (quite common) this
>     > obvious java-side implementation just isn't possible with the
>     current
>     > MemorySegment implementation as it will fail in an unexpected (and
>     > imho unreasonable) way if size is 0.  You'd either need to wrap
>     > MemorySegment in some other structure which hides this detail with
>     > it's own special case code (seems redundant), return a null
>     > (apparently evil these days, and kinda messy anyway), or expose the
>     > detail by ensuring the callee checks size>0 before calling
>     getData()
>     > (yikes).
>     >
>     > One notes that it is inconsistent with the rest of the api:
>     >
>     > works:
>     >
>     segmentAllocator.allocate(MemoryLayout.structLayout().withBitAlignment(8))
>     > works:  segmentAllocator.allocateArray(JAVA_BYTE, 0);
>     > works: segmentAllocator.allocate(0);
>     > works:  segment.asSlice(offset, 0);
>     > works: MemorySegment.ofArray(new byte[0]);
>     > works: MemorySegment.ofByteBuffer(ByteBuffer.allocateDirect(0));
>     > (you get the idea ...)
>     >
>     > doesn't:
>     >
>     MemorySegment.allocateNative(MemoryLayout.structLayout().withBitAlignment(8),
>
>     > scope);
>     > doesn't: MemorySegment.allocateNative(0, 1, scope);
>     > doesn't: MemorySegment.ofAddress(addr, 0, scope);
>     >
>     > With the last one being the only way to wrap sized-allocations from
>     > native memory(?)  there seems to be no workaround possible.
>     >
>     > Both java and c specifically define zero-length allocations as
>     valid
>     > everywhere else because it simplifies a lot of code, and even the
>     > foreign-abi does for every other case, so why not here too?
>     >
>     > Cheers,
>     >  Z
>