jextract woes

[Maurizio Cimadamore]

Thanks for the headsup - this is indeed an interesting case. My thinking 
here is: in classfile lingo, a native call can be thought of as an 
invokedynamic instruction with a certain recipe which describes the 
ingredient of the native call (layouts, entry point etc.). These recipes 
are expressed in Java, and they are called "bootstrap methods". When the 
invokedynamic instruction is first linked, the bootstrap method is 
called, and a method handle representing the computation associated with 
that callsite is then created.

It seems like we need at least one alternate bootstrap method which, 
instead of a constant address, takes an extra MethodHandle, which acts 
as an address generator. This way, the potentially expensive 
clGetExtensionFunctionAddressForPlatform function would only really be 
executed once (when the method is first linked) after which you get, as 
in the other, simpler case, a constant method handle the VM can just 
call and optimize at will.

The picture I painted is very possible assuming you can do (again) 
classfile generation. In future, it might also becomes possible to 
express invokedynamic calls using the Java source code (see [1]). So, I 
think jextract should provide the knobs for you to do so (more 
specifically the RuntimeHelper class).

In the meantime, it would not be hard for you to create a class which 
extends the jextract generated one, which adds the required additional 
extension handles (where the address is computed by calling the 
clGetExtensionFunctionAddressForPlatform handle). Or to use the jextract 
API to autogenerate this extension class. But this has to be 
domain-specific, so some extra coding would be required on your part 
(regardless of whether you go the classfile route or the sourcefile route).

Maurizio

[1] - https://www.youtube.com/watch?v=iSEjlLFCS3E

On 28/01/2020 03:37, Michael Zucchi wrote:
> OpenCL extensions provide extra sets of calls and constants but there 
> are no entry points provided in libOpenCL.so.  They need to be 
> resolved at runtime as they depend on the driver, aka 'platform'. 
> OpenCL 1.0/1.1 has clGetExtensionFunctionAddress but opencl 1.2 
> deprecated it in favour of clGetExtensionFunctionAddressForPlatform as 
> they are really platform specific.  Vulkan follows the same idea but 
> it covers the whole api apart from the function resolution functions.
>
> (incase you try jextract on cl_ext.h or cl_gl.h it and it seems to 
> work, libOpenCL can include extension entry point symbols, but it 
> isn't portable and it can't work for cl-platform-specific routines. 
> https://www.khronos.org/registry/OpenCL/sdk/1.2/docs/man/xhtml/clGetExtensionFunctionAddressForPlatform.html)
>
> So there needs to be a way to instantiate a set of method handles 
> whose entry addresses are resolved based on runtime state.  This isn't 
> possible now with jextract and also obviously conflicts with the 
> static method/handle route currently employed.
>
> With the old jextract I could directly call Libraries.bind() with a 
> custom name resolver (more code than with the low level api would 
> require).  With 'generate-abi' (my generator) I would just create a 
> concrete class with the constants/handles/calls and a factory method 
> or constructor which takes a name resolver.  The jni code basically 
> did this but in c-land.
>
> An easy low-level solution for jextract which maintains the current 
> static class design would be  to add an option to the method 
> generation.  Rather than method handles and methods which use them it 
> has methodhandle factory methods that take an address or address 
> resolution function*.  The 'public' library interface would just use 
> this to get a set of method handles it cares about and then invoke 
> them directly in it's methods.  This would force it to call 
> invoke(exact) directly but that isn't *much* worse than the untyped 
> method calls as of now and they are already exposed for such use anyway.
>
> I will re-implement what i had with zcl/jni on the branch using this 
> type of approach (but with my generator) to see how it works.
>
>  Z
>
> * For vulkan this is going to have be per-method since it everything a 
> big generated header file, or multiple runs of jextract with specific 
> methods filtered in.
> I want to look at it vulkan at some point, it's got some additional 
> problems to tackle.
>