[foreign-memaccess+abi] RFR: 8274157: java.foreign: Add method MemorySegment::isOverlapping

[Maurizio Cimadamore]

On Tue, 28 Sep 2021 12:59:19 GMT, Julia Boes <jboes at openjdk.org> wrote:

> This change adds a method to the MemorySegment API that detects if two segments are overlapping.

Hi,
some context: the main reason we dropped `segmentOffset` form `MemoryAddress` is that `MemoryAddress` now has a clearer role, which is more geared towards native interop. In other words, if you don't care about foreign functions, I believe `MemoryAddress` is not for you (unless you want to store the address of an off-heap segment in some other segment).

Now, I believe that the idea of returning the overlapping slice, combined with a way to tell the start offset of a segment into another might work?


MemorySegment overlap_a = a.overlap(b);


Now, if `overlap` is not null, can't you derive useful parameters from it, and from `a` and `b` ?

e.g.


long overlap_start_a = a.offsetOf(overlap_a);
long overlap_end_a = overlap_start_a + overlap_a.byteSize()

Then, you can obtain same info for `b`:


MemorySegment overlap_b = b.overlap(a);
long overlap_start_b = b.offsetOf(overlap_b);
long overlap_end_b = overlap_start_b + overlap_b.byteSize()

This is (more or less) the info you are looking for, right?

> I think you understand. But I haven't lost hope, that there aren't some simple primitives that would allow copying of multiple structs from one segment to another where a struct could be an arbitrary size in bytes (not bits). Imagine these segments are quite large ~ gigabytes. Forcing me to allocate a buffer segment of that size and perform redundant copy operations just because I can't figure out whether these segments overlap in memory and which segment precedes the other is a huge penalty. I didn't realize you had already tossed the MemoryAddress:: segmentOffset. This operation is crucial for this task. The minimal primitive operations required would be - boolean Segment.isOverlapping(Segment); //if segments are in different spaces (heap, off-heap) the result is obviously false. - long startingAddress(Segment); //this has to work whether it is on-heap or off-heap. You may consider the underlying copy mechanics to do this to be inefficient, perhaps, but it would still be orders-o
 f-magnitude more efficient than having to create a huge segment buffer and do all the redundant copies. I will construct a simulated java example using these primitives, just to be clear. :) Cheers, Lee.
> […](#)
> On Tue, Sep 28, 2021 at 5:04 PM Maurizio Cimadamore < ***@***.***> wrote: ***@***.**** commented on this pull request. ------------------------------ In src/jdk.incubator.foreign/share/classes/jdk/incubator/foreign/MemorySegment.java <[#585 (comment)](https://github.com/openjdk/panama-foreign/pull/585#discussion_r718049243)> : > @@ -320,6 +320,20 @@ default MemorySegment asSlice(long offset) { */ boolean isMapped(); + /** + * Returns true if this segment overlaps the ***@***.*** other} segment. Ok, I guess I now figured out what you mean; you basically want to implement something that has same correctness guarantees as System.arraycopy which works on *arbitrary* element sizes, not just 8/16/32/64 byte sizes. The copy method we have is good (in the sense that you can also copy by bytes, or by int/long words), but if you also need multi-bit swap (where multi > 64) then you are out of luck (but I'm curious about this use case of extreme swappiness :-) ). It is possible, in the future,
  that with the help of the vector API, we might be able to indeed lift the restriction, and provide a bulk copy (written in Java :-) ) which works on elements bigger than 64 bits. In the meantime, we have two options: - you "take the hit" when dealing with element sizes > 64bit (meaning in these case you will need extra buffer) - we provide enough primitives which allow you (or others) to implement other copy routines which support similar guarantees as System.arraycopy Another issue: note that the MemoryAddress::segmentOffset has also gone in the API refresh, and I believe your plan was probably to rely on it to do the computations; we could add it back if that was really an issue. Or we could come up with similar methods which work on segments instead (e.g. return the start/end offset of a segment into another segment - which combined with an overlapping slice region would probably do what you need to do). I don't know how common this issue is in your API, so I can't judge. The ge
 neral feeling I get is that an overlapping predicate without some other ways to compute offsets between slices is not very useful - and so, in that case it would be better to do nothing. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <[#585 (comment)](https://github.com/openjdk/panama-foreign/pull/585#discussion_r718049243)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ADCXRQSYPPW4KEAVTUOB7WDUEJJZ3ANCNFSM5E5LPKJA> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.

-------------

PR: https://git.openjdk.java.net/panama-foreign/pull/585