zero-length segments
Chris Vest
mr.chrisvest at gmail.com
Fri Jan 21 16:55:55 UTC 2022
>
> > Furthermore, malloc and free also allow working with zero-sized
> > segments, so it seems that even the checks in Unsafe is unnecessary.
>
That's true - although I guess malloc has still freedom to return either
> NULL or a constant pointer - perhaps implementors of Unsafe wanted more
> portable behavior?
For size zero, malloc returns either NULL or a unique pointer that can
later be passed to free.
It cannot be a constant value, unless the constant is NULL, AFAIK.
Since foreign memory and foreign linker APIs may be used to interact with
platform specific APIs, it could be argued that MemorySegments should do
whatever the underlying malloc is doing.
However, it would also be inconvenient for Java to specify such ambiguous
behaviour.
So I think, if malloc returns NULL, it should internally be retried with
size 1 to get a unique pointer for the zero-sized segment.
That should get consistent behaviour everywhere.
On Fri, Jan 21, 2022 at 6:26 AM Maurizio Cimadamore <
maurizio.cimadamore at oracle.com> wrote:
>
> On 21/01/2022 11:33, Quân Anh Mai wrote:
> > Hi,
> >
> > It seems the delegated Unsafe.allocateMemory and Unsafe.freeMemory
> > allows working with zero-sized segments. May I ask why do we have such
> > a restriction on MemorySegment.allocateNative?
> As I explained, there's no reason as to why zero-sized segments are not
> allowed. It's just the way the API has started out.
> > Furthermore, malloc and free also allow working with zero-sized
> > segments, so it seems that even the checks in Unsafe is unnecessary.
>
> That's true - although I guess malloc has still freedom to return either
> NULL or a constant pointer - perhaps implementors of Unsafe wanted more
> portable behavior?
>
> Maurizio
>
> >
> > Regards,
> > Quan Anh
> >
> > malloc - cppreference.com
> > <
> https://urldefense.com/v3/__https://en.cppreference.com/w/c/memory/malloc__;!!ACWV5N9M2RV99hQ!Z98zoj9sxZ1kO4YtwhCwwgsnkad_tgvFkHtLZi9tan9NQvcVcSerxkcoOBCf1CkXMNfpurc$
> >
> > free - cppreference.com
> > <
> https://urldefense.com/v3/__https://en.cppreference.com/w/c/memory/free__;!!ACWV5N9M2RV99hQ!Z98zoj9sxZ1kO4YtwhCwwgsnkad_tgvFkHtLZi9tan9NQvcVcSerxkcoOBCf1CkXAMZTOIg$
> >
> >
> > On Fri, 21 Jan 2022 at 18:51, Maurizio Cimadamore
> > <maurizio.cimadamore at oracle.com
> > <mailto:maurizio.cimadamore at oracle.com>> wrote:
> >
> > Hi Michael,
> > I think I'm sympathetic with your argument. Not only most of the
> > other
> > factories, as you noticed, do allow for zero-length segments, but
> > there's also the MemorySegment::mapFile method, which specifically
> > returns an instance of a special subclass if the mapped size if zero.
> >
> > I also did some more tests with the ByteBuffer API, which allows
> > allocation (as you mention) with size = 0, but also allows slicing
> > with
> > slice size = 0 and limit of 0.
> >
> > (Java arrays are another case where creating a zero-element array is
> > indeed possible).
> >
> > All this evidence point to the fact that, yes, saying no to zero-byte
> > allocation on memory segment (of any kind) is at the very least
> > problematic in terms of interop with existing APIs, as it will cause
> > surprising behavior.
> >
> > For this reason, I believe the best course of action is to enhance
> > the
> > API in the way you suggest, and accept zero-sized segments.
> >
> > Thanks for the feedback!
> >
> > Thinking of possible workarounds in the short term - but maybe you
> > are
> > already doing it - you could have something like this:
> >
> > ```
> > private static final MemorySegment EMPTY =
> > MemorySegment.ofByteBuffer(ByteBuffer.allocateDirect(0));
> >
> > MemorySegment wrapAddress(MemoryAddress address, long size,
> > ResourceScope scope) {
> > return size == 0 ? EMPTY : MemorySegment.ofAddress(address,
> > size,
> > scope);
> > }
> > ```
> >
> > In terms of performance, there are a couple of points to note:
> >
> > * adding a singleton anonymous class for zero-sized segment might
> > cause
> > profile pollution when using the same callsite with empty and
> > non-empty
> > segments
> > * having a branch (like in the above code) so that a singleton is
> > returned if size == 0 effectively disables escape analysis most of
> > the
> > times (when size is not known by C2)
> >
> > And, other pseudo-random considerations:
> >
> > * Looking at the impl of ByteBuffer.allocateDirect, it seems like it
> > always allocates at least one byte:
> >
> > ```
> > long size = Math.max(1L, (long)cap + (pa ? ps : 0));
> > Bits.reserveMemory(size, cap);
> > ```
> >
> > You can see how this is suboptimal (and probably not what a
> > programmer
> > would expect).
> >
> > * In the case of MemorySegment, returning a singleton is not really
> > possible, because (as for mapped segments) the user is also
> > providing a
> > scope parameter, and it expects that the returned segment will
> > have same
> > scope as the provided parameter.
> >
> > * All things considered, given it's late for 18, I'd prefer to
> > address
> > this in 19 - but I do want to address it.
> >
> > Thanks
> > Maurizio
> >
> > On 20/01/2022 22:50, Michael Zucchi wrote:
> > >
> > > Morning all,
> > >
> > > After a long break i've started experimenting with the foreign
> > abi to
> > > replace jni. One tool i'm working on is a vulkan binding generator
> > > that works directly from the xml specification and generates a
> > 'nice'
> > > api (particularly focusing on constructors for all the hundreds of
> > > configuration structures needed for vulkan, plus the dynamic
> > function
> > > tables), and another tool generates high-level and potentially
> > object
> > > oriented api's from c header files, for this one I use a gcc
> > plugin to
> > > extract the structures and functions and i'm trying opencl and
> > ffmpeg
> > > as test cases (and to update some projects i maintain, zcl and
> > > jjmpeg). I'm not using jextract because I want to create the high
> > > level api directly and provide much more control on the created
> > > classes, and also because i've got the time and nothing better
> > to do
> > > with it.
> > >
> > > In general it looks pretty good after so much work, but i've come
> > > across one oddity which adds complexity to the java code for no
> > > obvious reason. Is there any specific reason you cannot create
> > zero
> > > length memory segments *in some cases*?
> > >
> > > e.g. something like this comes up often in C:
> > >
> > > struct blob {
> > > size_t data_size;
> > > uint8_t *data;
> > > }
> > >
> > > From java at some point you want to get a MemorySegment to access
> > > blob.data. It might be through a high level api such as:
> > >
> > > class blob {
> > > MemorySegment getData() {
> > > return MemorySegment.ofAddress(...);
> > > }
> > > }
> > >
> > > If the memory was allocated in the native code (quite common) this
> > > obvious java-side implementation just isn't possible with the
> > current
> > > MemorySegment implementation as it will fail in an unexpected (and
> > > imho unreasonable) way if size is 0. You'd either need to wrap
> > > MemorySegment in some other structure which hides this detail with
> > > it's own special case code (seems redundant), return a null
> > > (apparently evil these days, and kinda messy anyway), or expose the
> > > detail by ensuring the callee checks size>0 before calling
> > getData()
> > > (yikes).
> > >
> > > One notes that it is inconsistent with the rest of the api:
> > >
> > > works:
> > >
> >
> segmentAllocator.allocate(MemoryLayout.structLayout().withBitAlignment(8))
> > > works: segmentAllocator.allocateArray(JAVA_BYTE, 0);
> > > works: segmentAllocator.allocate(0);
> > > works: segment.asSlice(offset, 0);
> > > works: MemorySegment.ofArray(new byte[0]);
> > > works: MemorySegment.ofByteBuffer(ByteBuffer.allocateDirect(0));
> > > (you get the idea ...)
> > >
> > > doesn't:
> > >
> >
> MemorySegment.allocateNative(MemoryLayout.structLayout().withBitAlignment(8),
> >
> > > scope);
> > > doesn't: MemorySegment.allocateNative(0, 1, scope);
> > > doesn't: MemorySegment.ofAddress(addr, 0, scope);
> > >
> > > With the last one being the only way to wrap sized-allocations from
> > > native memory(?) there seems to be no workaround possible.
> > >
> > > Both java and c specifically define zero-length allocations as
> > valid
> > > everywhere else because it simplifies a lot of code, and even the
> > > foreign-abi does for every other case, so why not here too?
> > >
> > > Cheers,
> > > Z
> >
>
More information about the panama-dev
mailing list