RFC: Refactoring SystemABI

[Jorn Vernee]

> 1) pass it as argument to a native function (this is the use case you
> have in mind)

I think Henry has a point here; A function that takes a function pointer 
currently mostly takes a Callback as argument on the Java side (at least 
that's how jextract models it right?). So to satisfy this scenario, we 
either need a way to convert a Pointer to a Callback, or return a 
Callback from SystemABI::upcallStub outright.

> But Callback is a higher level construct
> in my view - it requires a Scope and a functional interface, none of
> which are necessarily available in this low level view. Is there a
> lower level view to speak about function pointers?

Java doesn't have function types. The closest to that is a functional 
interface type, add a pointer to that and you basically have a Callback. 
The association with a scope only comes from the Pointer being 
associated with a scope. In a sense I feel like Callback is just a way 
to put a (pseudo) function type on a Pointer<?>.

I think Callback is a low-level enough abstraction. Maybe 
SystemABI::upcallStub could return a `Callback<?>` ? Initially the 
functional interface class would be null, and we could add a `<R> 
Callback<R> cast(Class<R> cls);` method to Callback to convert it to 
another type. The Callback implementation could save the 
NativeMethodType to check if a cast is valid.

> I believe one way to solve the issue could be by tweaking the
> SystemABI to accept a Pointer<?> also as an entry point for a downcall

This seems like a good idea to me; split the Library.Symbol parameter 
into `String name` and `Pointer<?> address` parameters, and then make 
the name parameter optional (with an overload). If we end up going with 
SystemABI::upcallStub returning a Callback, it already has a way to 
retrieve the entry point Pointer as well.

> - we can check the validity of the Pointer in Linux by calling dladdr

Not all function pointers are associated with a symbol. Symbols of 
internal functions are entirely optional for native code to have, but 
you still want to be able to have function pointers for them. There's 
also generated code which might not have a symbol associated with it. I 
think trying to validate the pointers is going a step too far.

Jorn

Maurizio Cimadamore schreef op 2018-12-04 01:08:
> On 03/12/2018 22:02, Henry Jen wrote:
>> 
>>> On Dec 3, 2018, at 1:21 PM, Maurizio Cimadamore 
>>> <maurizio.cimadamore at oracle.com> wrote:
>>> 
>>> 
>>> On 03/12/2018 20:19, Henry Jen wrote:
>>>> This looks good, I tried to have bound MH before, but because of 
>>>> that test, I decided to keep it as is as I thought that’s what we 
>>>> wanted.
>>>> 
>>>> By adding the UpcallInvoker, we basically achieve the same thing for 
>>>> Java code invoke callback from Java land, but that add cost to every 
>>>> NativeInvoker allocation, perhaps it should remains in 
>>>> CallbackImplGenerator as I expect this will only be used by binder?
>>> Not really - if you only interact with SystemABI, you don't even see 
>>> CallbackImplGenerator. So, SystemABI hands you back a Pointer<?>, how 
>>> do you call it?
>>> 
>> This is what I meant, upcall is for native to call java. This 
>> Pointer<?> is only interact with native calls, that is, when pass a 
>> function pointer to native API.
>> 
>> For Java to call Java, there is absolutely no reason to go via this 
>> path.
>> 
>> In case you get back a function pointer from native call and want to 
>> make call into it, isn’ t that suppose to deref the pointer and get 
>> the functional interface?
> 
> There are two possible uses for a Pointer you get back from
> SystemABI::upcallStub:
> 
> 1) pass it as argument to a native function (this is the use case you
> have in mind)
> 
> 2) pass it as the entry point of a native function
> 
> I think that, for completeness, both should be supported; let's say I
> have a native function returning a pointer to function; how do I call
> that function pointer using SystemABI?
> 
> Perhaps, you are saying that a function that returns a function
> pointer will return a Callback object - which then can be used to get
> to the functional interface. But Callback is a higher level construct
> in my view - it requires a Scope and a functional interface, none of
> which are necessarily available in this low level view. Is there a
> lower level view to speak about function pointers? My bet was that you
> can do 99% of function pointer just by passing void pointers around,
> and teaching SystemABI how to deal with 'its own' pointers.
> 
> Which is almost correct, but, as noted in my previous email, there's a
> slight mismatch between what is the input of SystemABI::downcallHandle
> and what's the output of SystemABI::upcallStub (or the return value of
> a native function returning a function pointer modeled as a
> Pointer<?>).
> 
> I believe one way to solve the issue could be by tweaking the
> SystemABI to accept a Pointer<?> also as an entry point for a downcall
> - we can check the validity of the Pointer in Linux by calling dladdr:
> 
> http://man7.org/linux/man-pages/man3/dladdr.3.html
> 
> Dunno if there is something similar for windows... I found this:
> 
> https://docs.microsoft.com/en-us/windows/desktop/api/Dbghelp/nf-dbghelp-symfromaddr
> 
> but seems related to debugging (although that seems to be used by
> Hotspot as well).
> 
> Maurizio