[foreign] thoughts on finalizing foreign APIs

[Maurizio Cimadamore]

On 16/09/2021 18:19, Sebastian Stenzel wrote:
> Hi Maurizio,
>
> very interesting article. I love all the thoughts that go into 
> designing such a beautiful API!
Thanks for reading!
>
> Just one question regarding the following:
>
>> we plan to move all the classes in the |jdk.incubator.foreign|package 
>> under the |java.lang.foreign|package [3 
>> <https://urldefense.com/v3/__https://inside.java/2021/09/16/finalizing-the-foreign-apis/*fn:3__;Iw!!ACWV5N9M2RV99hQ!bKUfgIRJ5EkYGF5QRQKOAL_XTjMISOpWqvUHfZGEhPJVp1yfqbIzKmxBmxyHxkTZq3EuLQE$>] 
>> in the |java.base| module.
>
> Is there any reason to place it in java.base? Just asking to avoid the 
> need to jigsaw 2.0 java.base with JDK 32 ;-)

The API is rather small-ish - and I think it also has very deep 
connection to the core of the platform. One main example: the 
implementations for the memory access var handle *have* to be defined in 
java.base. But if the API lives somewhere else (as it does now), it 
becomes very difficult to deal with these, as we have to introduce an 
intermediate interface (e.g. `MemorySegmentProxy`) which is used by 
java.base and implemented by the other module. Doing this has 
interesting ramifications: we need first to adapt all the var handles 
back to take a `MemorySegment`, not a `MemorySegmentProxy` otherwise the 
type mismatch will cause the var handle to be called with the `invoke` 
semantics, not with the `invokeExact`; secondly, we have seen cases 
where the introduction of these intermediate types was severely limiting 
C2 abilities to speculate on whether access was on- or off- heap (this 
is why at some point `MemorySegmentProxy` was changed from an interface 
to an abstract class).

There's just too many things to fight against if you move outside 
java.base; on top of that, add that we might want, in the future, to 
perhaps use Panama more in the JDK itself (e.g. to implement NIO), and 
maybe use segments to implement byte buffers. If we want to have a shot 
at doing any of these things, then I think it's clear(er?) that this API 
belongs in java.base.

Cheers
Maurizio

>
> Best regards
> Sebastian
>
>> Am 16.09.2021 um 16:59 schrieb Maurizio Cimadamore 
>> <maurizio.cimadamore at oracle.com>:
>>
>> Hi,
>> we have spent the last few weeks looking at the foreign APIs more 
>> holistically, to see what we could do to improve them, in order to 
>> get them preview/finalization ready. The resulting changes are 
>> described in the following write up:
>>
>> https://inside.java/2021/09/16/finalizing-the-foreign-apis/ 
>> <https://urldefense.com/v3/__https://inside.java/2021/09/16/finalizing-the-foreign-apis/__;!!ACWV5N9M2RV99hQ!bKUfgIRJ5EkYGF5QRQKOAL_XTjMISOpWqvUHfZGEhPJVp1yfqbIzKmxBmxyHxkTZqjdqAAs$>
>>
>> We think the proposed API [2, 3] is much simpler, tighter and safer - 
>> while retaining the same expressiveness of the existing API.
>>
>> Moving forward, we would like to start integrating the proposed 
>> changes into the panama repo, and incubate those changes in 18. We 
>> will then (finger crossed!) move towards finalizing the API, by 
>> making it a preview API in 19 (the document describes some basic 
>> shuffling moves on how we might want to do that). While we could 
>> technically preview things in 18, we believe that the extra steps 
>> required to move the API into java.base would push us too close to 
>> the code freeze deadline; besides there are still outstanding 
>> performance issues (such as [1]) which we'd like to have addressed, 
>> so that user experience can be as smooth as possible. For these 
>> reasons, incubating in 18 and previewing in 19 seems like the most 
>> sensible option at this point in time.
>>
>> Cheers
>> Maurizio
>>
>> [1] - https://bugs.openjdk.java.net/browse/JDK-8259609
>> [2] - 
>> http://cr.openjdk.java.net/~mcimadamore/panama/foreign-finalize-javadoc/javadoc/jdk/incubator/foreign/package-summary.html
>> [3] - 
>> https://github.com/mcimadamore/panama-foreign/tree/foreign-finalize%2Bjextract 
>> <https://urldefense.com/v3/__https://github.com/mcimadamore/panama-foreign/tree/foreign-finalize*2Bjextract__;JQ!!ACWV5N9M2RV99hQ!bKUfgIRJ5EkYGF5QRQKOAL_XTjMISOpWqvUHfZGEhPJVp1yfqbIzKmxBmxyHxkTZ984EEy4$>
>>
>>