Short feedback on the linker (CLinker)

[John Rose]

On Dec 21, 2020, at 3:32 AM, Maurizio Cimadamore <maurizio.cimadamore at oracle.com> wrote:
> 
> We have in the past discussed ways to "pin" heap memory, so that heap addresses could stay stable across a native call. This would be useful to implement logic like the one currently implemented in JNI critical sections, but I think it would always help in your case. That said, pinning heap objects would need to be made an unsafe operation (e.g. foreign restricted), given that it has the potential to undermine the correct behavior of the garbage collector, if abused. But yes, obtaining pointers from Java objects is something that is on our radar - it's technically not too hard to do that - the safety implication are the difficult part (as in many other areas of this project).

Some quick thoughts on that:

1. It would be enough to thread a JNI global handle through such
void* pointers, creating the handle when the function/void* pair
is registered, and deleting it (explicitly) when the pair is unregistered.
When the pair is invoked, the wrapper that calls into Java could be
adapted to dereference the JNI global handle into the underlying
object.  This would not interfere with the GC, except in the usual
way of potentially creating a storage leak if the handles were
forgotten about and never unregistered.

1a. In some cases a JNI weak handle might be helpful.  The Java
side would have to keep the object live.  There would still be
a potential leak of JNI handle bookkeeping resources if the
callbacks were forgotten, but not a leak of Java heap resources.

2. The problem of forgotten handles (failure to unregister)
can be addressed (relatively) safely by putting the whole
process in a try/finally block.  The finally clause would
unregister the callback *and* delete the JNI handle.

2a. If there are many function/void* pairs to register,
they could be attached to a single try/finally block,
via a scope object which “takes responsibility” for all
of them.

2b. A programmer who wants to avoid bookkeeping
about lifetimes (like C programmers sometimes do,
when they neglect calling the unregister functions)
could delegate callback registration to a server thread
whose only job is to hold a scope object alive (see 2a)
in a try/finally.  The thread would reify the set of
handler registrations it had performed.  Killing the
thread (cooperatively of course) would unregister
everything.

2c. As with the option of using weak JNI handles
(1a) a variation of the try/finally pattern would
make the unregistering of the handler a no-op,
and just clear the JNI global handle.  (An extra
indirection may be required to add the clearing
function.)  The result would be that when the
try/finally exits, the C code would still have a
registered function/void* pair somewhere, and
the JVM would have a global handle that is
“leaking”, but the invocation of the callback
would cause a (relatively) safe and sane NPE,
instead of a crash.  And no Java resources would
survive the exit of the try/finally, except a few
words relating to the void* handle (which now
would point to a null).

3. Such patterns could be captured by a tool and
“baked into” the callback mechanisms we support
now.  The result might be fewer functions being
spun by the Panama runtime, perhaps just one
per callback mechanism, with all the dispatching
pushed through the void* companion value.
Since the underlying patterns are not represented
in the C type system, manual annotations would
have to drive the tool’s wrapper generation.

3a. The C++ type system *does* represent such
patterns in its type system.  Converting a Java
object to a C++ object with virtual functions
generalizes the process of converting a lambda
to a function pointer, just as C++ objects generalize
C++ lambdas which generalize C function pointers.
The above patterns are probably useful for allowing
Java objects to be wrapped in C++ objects.