[foreign-memaccess+abi] RFR: 8274157: java.foreign: Add method MemorySegment::isOverlapping

[Lee Rhodes]

On Tue, 28 Sep 2021 12:59:19 GMT, Julia Boes <jboes at openjdk.org> wrote:

> This change adds a method to the MemorySegment API that detects if two segments are overlapping.

I think you understand. But I haven't lost hope, that there aren't some
simple primitives that would allow copying of multiple structs from one
segment to another where a struct could be an arbitrary size in bytes (not
bits).

Imagine these segments are quite large ~ gigabytes.  Forcing me to allocate
a buffer segment of that size and perform redundant copy operations just
because I can't figure out whether these segments overlap in memory and
which segment precedes the other is a huge penalty.

I didn't realize you had already tossed the MemoryAddress:: segmentOffset. This
operation is crucial for this task.

The minimal primitive operations required would be

   - boolean Segment.isOverlapping(Segment); //if segments are in different
   spaces (heap, off-heap) the result is obviously false.
   - long startingAddress(Segment);  //this has to work whether it is
   on-heap or off-heap.

You may consider the underlying copy mechanics to do this to be
inefficient, perhaps, but it would still be orders-of-magnitude more
efficient than having to create a huge segment buffer and do all the
redundant copies.

I will construct a simulated java example using these primitives, just to
be clear. :)

Cheers,
Lee.

On Tue, Sep 28, 2021 at 5:04 PM Maurizio Cimadamore <
***@***.***> wrote:

> ***@***.**** commented on this pull request.
> ------------------------------
>
> In
> src/jdk.incubator.foreign/share/classes/jdk/incubator/foreign/MemorySegment.java
> <https://github.com/openjdk/panama-foreign/pull/585#discussion_r718049243>
> :
>
> > @@ -320,6 +320,20 @@ default MemorySegment asSlice(long offset) {
>       */
>      boolean isMapped();
>
> +    /**
> +     * Returns true if this segment overlaps the ***@***.*** other} segment.
>
> Ok, I guess I now figured out what you mean; you basically want to
> implement something that has same correctness guarantees as
> System.arraycopy which works on *arbitrary* element sizes, not just
> 8/16/32/64 byte sizes. The copy method we have is good (in the sense that
> you can also copy by bytes, or by int/long words), but if you also need
> multi-bit swap (where multi > 64) then you are out of luck (but I'm curious
> about this use case of extreme swappiness :-) ).
>
> It is possible, in the future, that with the help of the vector API, we
> might be able to indeed lift the restriction, and provide a bulk copy
> (written in Java :-) ) which works on elements bigger than 64 bits. In the
> meantime, we have two options:
>
>    - you "take the hit" when dealing with element sizes > 64bit (meaning
>    in these case you will need extra buffer)
>    - we provide enough primitives which allow you (or others) to
>    implement other copy routines which support similar guarantees as
>    System.arraycopy
>
> Another issue: note that the MemoryAddress::segmentOffset has also gone
> in the API refresh, and I believe your plan was probably to rely on it to
> do the computations; we could add it back if that was really an issue. Or
> we could come up with similar methods which work on segments instead (e.g.
> return the start/end offset of a segment into another segment - which
> combined with an overlapping slice region would probably do what you need
> to do).
>
> I don't know how common this issue is in your API, so I can't judge. The
> general feeling I get is that an overlapping predicate without some other
> ways to compute offsets between slices is not very useful - and so, in that
> case it would be better to do nothing.
>
> —
> You are receiving this because you were mentioned.
> Reply to this email directly, view it on GitHub
> <https://github.com/openjdk/panama-foreign/pull/585#discussion_r718049243>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/ADCXRQSYPPW4KEAVTUOB7WDUEJJZ3ANCNFSM5E5LPKJA>
> .
> Triage notifications on the go with GitHub Mobile for iOS
> <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675>
> or Android
> <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.
>
>

-------------

PR: https://git.openjdk.java.net/panama-foreign/pull/585