changes to the foreign preview API

[Maurizio Cimadamore]

Hi,
as we've just integrated our first changeset which moves the Foreign 
Function & Memory API into java.base (as a preview API), I'd like to 
expand a bit more on the kind of API changes you might expect in the 
upcoming weeks. Disclaimer: these changes will only apply to the 
"foreign-preview" branch, so they won't affect the branch we currently 
use to generate EA binaries (in case we need to apply quick fixes there).

First, you can expect us playing with the package name some more; 
java.lang.foreign is really nothing more than an opening position, and 
it is likely that we will move to a package hierarchy where the Memory 
Access API and the Foreign Linker API live in two different packages 
(e.g. java.X vs. java.X.invoke, for some value of X).

Secondly, on the renaming front, as anticipated in [1], we are also 
planning a biggie rename from ResourceScope to MemorySession. This 
renaming comes from the consideration that, currently there isn't always 
a 1:1 relationship between a resource scope and a lexical scope (for 
instance in case where a resource scope is stored in a field). We feel 
that the MemorySession name is both more neutral w.r.t. suggesting any 
correspondence with lexical scope and, at the same time more specific 
(what's a "resource" ?).

We will also bring back the SegmentAllocator interface on MemorySession 
- which means that you can just use methods on SegmentAllocator on a 
MemorySession instance, which on the balance, works quite well, and 
makes client code a bit more concise.

The other change we'd like to make is to remove the scope() accessors 
from all the resources (MemorySegment, NativeSymbol, VaList). This comes 
from a desire to make the API more principled: only the owner of a 
scope/session should have the "right" to close it. In fact, we have 
spent significant API estate (e.g. ResourceScope::whileAlive) just from 
preventing random clients to shut down scopes. We now believe a better 
approach would be to simply make the scope associated e.g. with a memory 
segment inaccessible. In other words, a resource scope/memory session 
becomes a _capability_: it is up to the owner of the scope/session to 
decide who to share that capability with. Once a client has a scope, it 
can close that scope, but also affect the scope by registering cleanup 
actions, or allocating on that scope. Conversely, if a client only has a 
segment, there's no way for that client to affect the owning 
scope/session in a meaninful (and possibly, harmful) way.

Because of this, our plan is to drop API points to keep scopes/sessions 
alive for longer than their creators intended. Internally, the API will 
still make concurrent memory access safe, as well as protect segment 
from being closed while in the middle of a native call (or an async 
ByteBuffer IO operation). In fact, we think that the cases that are 
secured by the memory/linker API internals already constitute the vast 
majority of cases in which it makes sense to reach for a mechanism to 
keep a scope/session open. If needs be, this leaves us space to add 
other API methods in the future, such adding a simple higher-order 
funtion to execute an action on a resource while keeping its scope open, 
or, more generally, to add a scope dependency mechanism such as that 
described in [2].

Finally, we would like to tweak the policy of the --enable-native-access 
flag, as follows (this change was first discussed in [3]):

* when the flag is specified, we keep existing behavior. This means 
getting an exception on every unsafe access, if the accessing module not 
listed in the flag;
* when flag is not specified, access to unsafe functioalities is 
allowed, with runtime warning.

The new, more-tolerant behavior in the absence of flag would facilitate 
migration away from JNI in cases where altering the set of command line 
options might be difficult. This mode would also help us (later) migrate 
JNI to require a similar opt-in too, effectively putting both JNI and 
Panama on the same grounds. The new mode is transitional and won't be 
supported forever: in a future release, this mode will be disabled and 
the new flag will be required _always_. This would give us more time to 
improve the support for the --enable-native-access flag in cases where 
e.g. it is currently problematic to alter the command line options 
and/or to introduce a better way to propagate native dependencies across 
modules.

This should pretty much cover it. With the exception of renaming of 
classes and packages, we expect most of the idiomatic Panama code (such 
as that in our jextract examples [4]) to remain indeed very similar to 
its Java 18 counterpart.

Cheers
Maurizio

[1] - https://inside.java/2021/09/16/finalizing-the-foreign-apis/
[2] - https://inside.java/2021/05/28/taming-resource-scopes/
[3] - 
https://mail.openjdk.java.net/pipermail/panama-dev/2022-January/016011.html
[4] - 
https://github.com/openjdk/panama-foreign/blob/foreign-jextract/doc/panama_jextract.md