New candidate JEP: 419: Foreign Function & Memory API (Second Incubator)

[leerho]

Wow!  Simple for you, perhaps :)  The irony is that using Unsafe was even
simpler! Nonetheless, we go forward :)

Forgive me for a little editorial comment, but big data systems that use
hundreds of gigabytes of off-heap memory, know that they will be constantly
overwriting these big segments as new data is paged in.  So the zeroing
operation is really a waste of time and CPU resources.  If it turns out
that if a specific overwrite does not exactly fill the segment it is
trivial and faster to just fill the remainder with zeros, which is usually
a tiny fraction of the overall space.

It would be really useful to have a simple flag option to the
*MemorySegment::allocateNative(...
boolean noZeroFill).*

Cheers,

Lee.


On Tue, Oct 12, 2021 at 1:16 PM Maurizio Cimadamore <
maurizio.cimadamore at oracle.com> wrote:

>
> On 12/10/2021 19:20, leerho wrote:
>
> Except that allocateNative does zero the memory :-) There are some
>> obscure flags by which you can disable this, but in general that's the
>> lay of the land.
>
>
> Where are these "obscure flags" documented?
> In general, where can I find documentation of all possible flags, say for
> JDK 17?
>
> I don't think there's a general documentation for all possible flag. The
> one in question is "-Djdk.internal.foreign.skipZeroMemory=true"
>
>
> If a more optimized allocator is needed, a
>> SegmentAllocator is probably the way to go, in which there's more
>> freedom as to how memory is actually allocated/inited.
>
>
> Could you give an example of how to implement the
> *SegmentAllocator::allocate(long,long)* method to return a new segment
> that does not do memory zeroing? (presumably without having to use "obscure
> flags" option)
>
> you can:
>
> 1. setup the method handles to call "malloc" and "free" using the linker
> API
> 2. when you receive an allocation request, you call malloc
> 3. that gives you back a MemoryAddress
> 4. you attach the memory address to a given ResourceScope (the client will
> have given that to you), to form a MemorySegment, using
> MemorySegment::allocateNative
> 5. you register a close action on the scope, so that when the scope is
> closed, "free" is called with that address
> 6. you return the segment to the user
>
> It's really simpler than it sounds. All the pieces are there. (I might
> follow up with a Gist to show something along these lines).
>
> Maurizio
>
>
> Thanks & Regards,
>
> Lee.
>
>
>
> On Tue, Oct 12, 2021 at 5:06 AM Sebastian Stenzel <
> sebastian.stenzel at gmail.com> wrote:
>
>>
>> > On 12. Oct 2021, at 13:13, Maurizio Cimadamore <
>> maurizio.cimadamore at oracle.com> wrote:
>> >
>> >
>> > On 12/10/2021 11:54, Sebastian Stenzel wrote:
>> >> Oh well... good to know! :D
>> >>
>> >> While my assumption was wrong, the conclusion stays the same: It
>> should be mentioned whether we can rely on one or the other behaviour.
>> >>
>> > For the records, in the current javadoc for
>> MemorySegment::allocateNative I see this sentence:
>> >
>> >> The block of off-heap memory associated with the returned native
>> memory segment is initialized to zero.
>> >
>> > Which is repeated across all the allocateNative factories.
>> >
>> > Maurizio
>> >
>>
>> My bad, I must have missed this. ��
>>
>>