[foreign-abi] minimizing the surface of restricted native access

[Maurizio Cimadamore]

Hi,
over the last few weeks we had many discussions on the fact that some of 
the uses of the memory access API, especially when they intersect 
foreign function access, are inherently unsafe. Examples of these include:

* obtaining a null pointer
* forge constant pointers (e.g. a MemoryAddress whose value is 42)
* have memory access VarHandles which deal with MemoryAddress (e.g. to 
read and write addresses)
* interact with pointers which come from native code

In this email I'd like to focus on which strategies we could adopt to 
try and make the above operations as _safe_ as possible.

Let's start with nulls; every bit of code working with some native 
libraries will, at some point, need to utter NULL, right? Unfortunately, 
the memory access API does not (even under the system-abi branch) offer 
safe access to the null MemoryAddress. The reasoning behind it is that, 
since the address is NULL, if you tried to dereference it, you could 
crash the VM; granted we could special case it so that we threw a NPE 
instead. But then, what happens if you offset the NPE? Would the 
offsetted address still be covered by the NPE guarantee?

I think a more stable way to get there is to resort to the notion of a 
Nothing memory segment (this is not a new idea - in fact, even before I 
joined the project, the Panana/foreign work had a similar concept of 
Nothing region). What is the Nothing segment? It is a 'root' native 
segment (read: it cannot be closed), whose base address is 0 and whose 
length is also 0. Why is this segment interesting? Two reasons:

* the baseAddress() of the Nothing segment is... the NULL pointer
* every address derived from the Nothing segment is, by definition, out 
of bounds

The latter observation is key to provide safety: any attempt to 
dereference _any_ address (including NULL itself) derived from the 
Nothing segment will result in a IOOB exception! But wait - this means 
that we can also support use cases where the user wants to 'forge' an 
address with a given constant value. For instance:

Nothing.baseAddress().offset(42) // creates a constant address with value 42

This is pretty straightforward. Now, as long as the user does not 
dereference these addresses, everything is fine - that is, a NULL or a 
'forged' address can be passed to a native method handle. So far so good 
- we have covered two important use cases w/o the need of any extra 
restricted native operations.

But we can pull this string some more; let's consider the case of having 
a VarHandle which:

- (read) turned long values into MemoryAddress
- (write) turned MemoryAddress into a long

Such an abstraction would be immensely helpful to define e.g. native 
bindings. But, again, so far we have decided _against_ providing such a 
capability because, if we provided that, it would then be possible to, 
indirectly, forge pointers - e.g. by writing a long value into a memory 
location and then reading that same value as a MemoryAddress.

But, thanks to the Nothing region, we now have a way out: the 
dereference operation can be made safe, if the MemoryAddress generated 
by the VarHandle (upon read) is backed by the special Nothing region. 
This means that the user won't be able to dereference such an address, 
but this might be ok in cases where the address just needs to be written 
somewhere else, or passed to some native library.

In fact, more generally, we can use the same trick to model _all_ 
MemoryAddress(es) that are generated in the ABI layer (by native method 
handles). Granted, this will add some restrictions (a client cannot 
dereference or close them), but it will also help making the client code 
_safer_. In most cases, these limitations will end up being sensible 
ones, as most C libraries tend to fall into one of the following two 
categories:

1) caller is responsible for allocation; API is typically called with 
some 'out' parameter (e.g. fill_array(&buf) )
2) callee is responsible for allocation and deallocation; that is API 
provide symmetric points for allocate/deallocate (e.g. 
create_foo/destroy_foo)

In (1), the Java code calling the native API will create the 
MemorySegment itself; as such it can fully manage its spatial/temporal 
bounds.
In (2), the Java code deals with _opaque_ addresses, but such APIs are 
typically providing all required entry points so that the client never 
has to dereference or free directly.

So, I'm confident that the tricks explained so far will, in practice, 
handle the vast majority of the use cases _without any need for 
restricted native operations_. There are however cases where the caller 
knows _exactly_ what the spatial/temporal bounds of a returned address 
are; consider the following classic example:

char *strcat(char *dest, const char *src);

Here, the client is required to allocate a 'dest' buffer that is _big_ 
enough to hold the concatenated string. A pointer to the same buffer is 
then returned to the caller, as the documentation states:

>
> The  strcat() function appends the src string to the dest string, 
> over‐writing the terminating null byte ('\0') at the end of dest,  
> and  then
> adds  a  terminating  null  byte.  The strings may not overlap, and 
> the dest string must have enough space for the  result. [...]
> The strcat() [...] function return a pointer to the  resulting string 
> dest.


So here we have a problem: the Java code calling strcat has a fully 
managed segment for the buffer; but then strcat is called, and a _new_ 
address is returned; since the new address is generated by native code, 
it has to be modeled as an address backed by the special Nothing segment 
(see above). This means that the result of strcat is *less* powerful 
than its input (even though they are effectively the same address!).

A similar problem would occur when storing pointers inside a struct; 
let's say a client creates a struct and saves a pointer to a known 
segment B inside that struct. If the pointer is later retrieved, the var 
handle used to dereference the pointer struct field will return an 
address backed by the Nothing segment; so, again, we are in an 
asymmetric situation where writing then reading gives us an address that 
is not the same as the one we started with.

These asymmetries can, to some extent, still be (safely!) cured. Let's 
assume we add the the following API point to MemoryAddress:

MemoryAddress rebase(MemorySegment)

The semantics of this method is simple: if the address is contained in 
the supplied segment, this method returns a _new_ address that is 
re-interpreted as an offset into the supplied segment.

It is easy to see how rebase() can be used to fix all the asymmetries 
discussed above; for instance, when calling strcat, the client already 
knows the segment associated with the 'dest' parameter (and hence with 
the return value of that function). So, rebasing the returned address to 
the original segment will make the address fully trusted again (a 
similar approach can be made to work when accessing struct pointer 
fields, we leave that as an exercise to the reader).

But what if a client really wants to dereference a _random_ pointer 
generated by a library? In this case, the client has no well-known 
segment which can be used as a rebase target, so, what can be done? This 
is where we need to veer into restricted native access territory, and 
provide some escape hatch by which an untrusted address can be made 
trusted again. There are many ways to do that; perhaps the simplest is 
to add a new method on MemoryAddress:

MemoryAddress asUnchecked()

which returns a new memory address which point to same location as the 
original address, but whose access is neither spatially, nor temporally 
checked. Another way to get there would be to expose the dual of the 
Nothing region - e.g. the Everything region, in which no address can 
ever be out of bounds (obtaining such a segment would, again, be a 
restricted native operation which requires higher privileges). Then, the 
untrusted address can be made trusted by simply rebasing the untrusted 
address against the Everything segment.

This model has a pleasing property: access to all memory addresses 
(whether managed or coming from native code) is always _safe_ by 
default, and we have _safe_ way to express most of the idioms which 
frequently occur when working with native libraries. This has few 
advantages: first, it minimizes uses of restricted native operations 
(thus minimizing the needs of e.g. extra runtime flags). Secondly, it 
makes transitions between safe and unsafe _explicit_ and thus deadly 
simple to spot (for instance, a client could easily search for all 
usages of the 'MemoryAddress::asUnchecked()' method to narrow down 
potential issues).

There is, of course a limitation: this model assumes that a client will 
never need/want to 'free' a random pointer obtained by calling a native 
library: since the Nothing segment is always alive (by definition), it 
cannot be closed. While this restriction will be ok in most cases, there 
will be some APIs requiring this - examples are strdup, vasprintf, which 
require the client to specifically 'free' the given address. That said, 
this problem doesn't seem too difficult to solve: for instance a client 
can request a native MethodHandle which points to the 'free' stdlib 
function, and then can call it directly on the required address! I think 
such an approach would actually be preferable to an approach where 
MemorySegment::close() secretly implies calling free() on some address - 
which might be the case now, but might not be later on (e.g. if we 
replace the allocator used internally by the memory access API).

Thoughts?

Maurizio