[foreign] thoughts on finalizing foreign APIs

[Uwe Schindler]

Hi,

> > very interesting article. I love all the thoughts that go into
> > designing such a beautiful API!
> Thanks for reading!
> >
> > Just one question regarding the following:
> >
> >> we plan to move all the classes in the |jdk.incubator.foreign|package
> >> under the |java.lang.foreign|package [3
> >> <https://urldefense.com/v3/__https://inside.java/2021/09/16/finalizing-
> the-foreign-
> apis/*fn:3__;Iw!!ACWV5N9M2RV99hQ!bKUfgIRJ5EkYGF5QRQKOAL_XTjMISOp
> WqvUHfZGEhPJVp1yfqbIzKmxBmxyHxkTZq3EuLQE$>]
> >> in the |java.base| module.
> >
> > Is there any reason to place it in java.base? Just asking to avoid the
> > need to jigsaw 2.0 java.base with JDK 32 ;-)
> 
> The API is rather small-ish - and I think it also has very deep
> connection to the core of the platform. One main example: the
> implementations for the memory access var handle *have* to be defined in
> java.base. But if the API lives somewhere else (as it does now), it
> becomes very difficult to deal with these, as we have to introduce an
> intermediate interface (e.g. `MemorySegmentProxy`) which is used by
> java.base and implemented by the other module. Doing this has
> interesting ramifications: we need first to adapt all the var handles
> back to take a `MemorySegment`, not a `MemorySegmentProxy` otherwise the
> type mismatch will cause the var handle to be called with the `invoke`
> semantics, not with the `invokeExact`; secondly, we have seen cases
> where the introduction of these intermediate types was severely limiting
> C2 abilities to speculate on whether access was on- or off- heap (this
> is why at some point `MemorySegmentProxy` was changed from an interface
> to an abstract class).
> 
> There's just too many things to fight against if you move outside
> java.base; on top of that, add that we might want, in the future, to
> perhaps use Panama more in the JDK itself (e.g. to implement NIO), and
> maybe use segments to implement byte buffers. If we want to have a shot
> at doing any of these things, then I think it's clear(er?) that this API
> belongs in java.base.

I was thinking about that, too. From my previous performance tests done by Maurizio and myself, we had often the problem that it was very hard to figure out where some problems are coming from. E.g., nowadays also in JDK 17, the ResourceScope is used also inside the ByteBuffer API, so there is a lot of proxying done. Those functionalities like the scopes are so important and also affect java.base APIs (because there's changes in memory model behind it) that its really hard to put this into a separate package without wrappers and proxies everywhere. Try to read the current code around ResourceScope and how it is used in Panama, ByteBuffers or even Vector API!

You could for sure implement all that stuff in java.base and only add higher level APIs in a module (like the foreign linker), but this is not much on top of the base.

So I agree with Maurizio: It's for sure java.base!

My final comment: Great progress in API, I am very happy... - it fits Lucene's use cases perfectly - although I am very unhappy that the migration to java.base will take until Java 19!

Uwe