zero-length segments

Fri Jan 21 15:15:45 UTC 2022

I've filed this:

https://bugs.openjdk.java.net/browse/JDK-8280460

to keep track of this issue.

Thanks
Maurizio

On 21/01/2022 14:25, Maurizio Cimadamore wrote:
>
> On 21/01/2022 11:33, Quân Anh Mai wrote:
>> Hi,
>>
>> It seems the delegated Unsafe.allocateMemory and Unsafe.freeMemory 
>> allows working with zero-sized segments. May I ask why do we have 
>> such a restriction on MemorySegment.allocateNative?
> As I explained, there's no reason as to why zero-sized segments are 
> not allowed. It's just the way the API has started out.
>> Furthermore, malloc and free also allow working with zero-sized 
>> segments, so it seems that even the checks in Unsafe is unnecessary.
>
> That's true - although I guess malloc has still freedom to return 
> either NULL or a constant pointer - perhaps implementors of Unsafe 
> wanted more portable behavior?
>
> Maurizio
>
>>
>> Regards,
>> Quan Anh
>>
>> malloc - cppreference.com 
>> <https://urldefense.com/v3/__https://en.cppreference.com/w/c/memory/malloc__;!!ACWV5N9M2RV99hQ!Z98zoj9sxZ1kO4YtwhCwwgsnkad_tgvFkHtLZi9tan9NQvcVcSerxkcoOBCf1CkXMNfpurc$>
>> free - cppreference.com 
>> <https://urldefense.com/v3/__https://en.cppreference.com/w/c/memory/free__;!!ACWV5N9M2RV99hQ!Z98zoj9sxZ1kO4YtwhCwwgsnkad_tgvFkHtLZi9tan9NQvcVcSerxkcoOBCf1CkXAMZTOIg$>
>>
>> On Fri, 21 Jan 2022 at 18:51, Maurizio Cimadamore 
>> <maurizio.cimadamore at oracle.com 
>> <mailto:maurizio.cimadamore at oracle.com>> wrote:
>>
>>     Hi Michael,
>>     I think I'm sympathetic with your argument. Not only most of the
>>     other
>>     factories, as you noticed, do allow for zero-length segments, but
>>     there's also the MemorySegment::mapFile method, which specifically
>>     returns an instance of a special subclass if the mapped size if 
>> zero.
>>
>>     I also did some more tests with the ByteBuffer API, which allows
>>     allocation (as you mention) with size = 0, but also allows slicing
>>     with
>>     slice size = 0 and limit of 0.
>>
>>     (Java arrays are another case where creating a zero-element array is
>>     indeed possible).
>>
>>     All this evidence point to the fact that, yes, saying no to 
>> zero-byte
>>     allocation on memory segment (of any kind) is at the very least
>>     problematic in terms of interop with existing APIs, as it will cause
>>     surprising behavior.
>>
>>     For this reason, I believe the best course of action is to enhance
>>     the
>>     API in the way you suggest, and accept zero-sized segments.
>>
>>     Thanks for the feedback!
>>
>>     Thinking of possible workarounds in the short term - but maybe you
>>     are
>>     already doing it - you could have something like this:
>>
>>     ```
>>     private static final MemorySegment EMPTY =
>>     MemorySegment.ofByteBuffer(ByteBuffer.allocateDirect(0));
>>
>>     MemorySegment wrapAddress(MemoryAddress address, long size,
>>     ResourceScope scope) {
>>          return size == 0 ? EMPTY : MemorySegment.ofAddress(address,
>>     size,
>>     scope);
>>     }
>>     ```
>>
>>     In terms of performance, there are a couple of points to note:
>>
>>     * adding a singleton anonymous class for zero-sized segment might
>>     cause
>>     profile pollution when using the same callsite with empty and
>>     non-empty
>>     segments
>>     * having a branch (like in the above code) so that a singleton is
>>     returned if size == 0 effectively disables escape analysis most of
>>     the
>>     times (when size is not known by C2)
>>
>>     And, other pseudo-random considerations:
>>
>>     * Looking at the impl of ByteBuffer.allocateDirect, it seems like it
>>     always allocates at least one byte:
>>
>>     ```
>>     long size = Math.max(1L, (long)cap + (pa ? ps : 0));
>>     Bits.reserveMemory(size, cap);
>>     ```
>>
>>     You can see how this is suboptimal (and probably not what a
>>     programmer
>>     would expect).
>>
>>     * In the case of MemorySegment, returning a singleton is not really
>>     possible, because (as for mapped segments) the user is also
>>     providing a
>>     scope parameter, and it expects that the returned segment will
>>     have same
>>     scope as the provided parameter.
>>
>>     * All things considered, given it's late for 18, I'd prefer to
>>     address
>>     this in 19 - but I do want to address it.
>>
>>     Thanks
>>     Maurizio
>>
>>     On 20/01/2022 22:50, Michael Zucchi wrote:
>>     >
>>     > Morning all,
>>     >
>>     > After a long break i've started experimenting with the foreign
>>     abi to
>>     > replace jni.  One tool i'm working on is a vulkan binding 
>> generator
>>     > that works directly from the xml specification and generates a
>>     'nice'
>>     > api (particularly focusing on constructors for all the hundreds of
>>     > configuration structures needed for vulkan, plus the dynamic
>>     function
>>     > tables), and another tool generates high-level and potentially
>>     object
>>     > oriented api's from c header files, for this one I use a gcc
>>     plugin to
>>     > extract the structures and functions and i'm trying opencl and
>>     ffmpeg
>>     > as test cases (and to update some projects i maintain, zcl and
>>     > jjmpeg).  I'm not using jextract because I want to create the high
>>     > level api directly and provide much more control on the created
>>     > classes, and also because i've got the time and nothing better
>>     to do
>>     > with it.
>>     >
>>     > In general it looks pretty good after so much work, but i've come
>>     > across one oddity which adds complexity to the java code for no
>>     > obvious reason.  Is there any specific reason you cannot create
>>     zero
>>     > length memory segments *in some cases*?
>>     >
>>     > e.g. something like this comes up often in C:
>>     >
>>     > struct blob {
>>     >    size_t data_size;
>>     >    uint8_t *data;
>>     > }
>>     >
>>     > From java at some point you want to get a MemorySegment to access
>>     > blob.data.  It might be through a high level api such as:
>>     >
>>     >   class blob {
>>     >     MemorySegment getData() {
>>     >         return MemorySegment.ofAddress(...);
>>     >     }
>>     >  }
>>     >
>>     > If the memory was allocated in the native code (quite common) this
>>     > obvious java-side implementation just isn't possible with the
>>     current
>>     > MemorySegment implementation as it will fail in an unexpected (and
>>     > imho unreasonable) way if size is 0.  You'd either need to wrap
>>     > MemorySegment in some other structure which hides this detail with
>>     > it's own special case code (seems redundant), return a null
>>     > (apparently evil these days, and kinda messy anyway), or expose 
>> the
>>     > detail by ensuring the callee checks size>0 before calling
>>     getData()
>>     > (yikes).
>>     >
>>     > One notes that it is inconsistent with the rest of the api:
>>     >
>>     > works:
>>     >
>> segmentAllocator.allocate(MemoryLayout.structLayout().withBitAlignment(8))
>>     > works:  segmentAllocator.allocateArray(JAVA_BYTE, 0);
>>     > works: segmentAllocator.allocate(0);
>>     > works:  segment.asSlice(offset, 0);
>>     > works: MemorySegment.ofArray(new byte[0]);
>>     > works: MemorySegment.ofByteBuffer(ByteBuffer.allocateDirect(0));
>>     > (you get the idea ...)
>>     >
>>     > doesn't:
>>     >
>> MemorySegment.allocateNative(MemoryLayout.structLayout().withBitAlignment(8),
>>
>>     > scope);
>>     > doesn't: MemorySegment.allocateNative(0, 1, scope);
>>     > doesn't: MemorySegment.ofAddress(addr, 0, scope);
>>     >
>>     > With the last one being the only way to wrap sized-allocations 
>> from
>>     > native memory(?)  there seems to be no workaround possible.
>>     >
>>     > Both java and c specifically define zero-length allocations as
>>     valid
>>     > everywhere else because it simplifies a lot of code, and even the
>>     > foreign-abi does for every other case, so why not here too?
>>     >
>>     > Cheers,
>>     >  Z
>>