notes on binding C++

[Vladimir Ivanov]

Samuel,

>>> That sounds great and all, but it has been almost 4 years now, and 
>>> the only feedback I am getting is that it does not yet support 
>>> function-like macros, inline functions, C++ templates, etc. Ok, then, 
>>> when? Why is a "limited amount of wrappers" taking so long?
>> Hi Samuel,
>> I think I've already responded to similar questions many times now, 
>> and I feel we're just going in circles; I sense your frustration, but 
>> unfortunately I do not have anything more to add to this discussion. 
>> I've told you what the focus is short/medium term (C interop), and 
>> what is the _rough_ plan after that. If that is not sufficient, or if 
>> you feel that we just got all the priorities wrong, or if you think 
>> we're just too slow, I'm sorry, but I don't think those kinds of 
>> concerns are of the kind that can be positively addressed over an 
>> email exchange.
> 
> I am sorry, I think that came out the wrong way. What I was trying to 
> say is that supporting C++ and making it as super efficient as you want 
> it to be is a *very hard problem*, and you will require assistance from 
> a lot of people--from outside Oracle--to pull this off. I am trying to 
> make you realize that, and yes I am frustrated that I am unable to 
> communicate this, even after almost 4 years...

I don't think anybody involved in the project has any illusions about 
the complexity of C++ support. That's one of the main reasons we haven't 
made much progress on that front yet. In that respect, JavaCPP design 
decision to delegate heavy lifting to native C++ compilers looks very 
attractive.

As Maurizio & Henry already pointed out, we have been focused on laying 
out foundation for further explorations. Most of the effort went into 
experiments with low-level mechanisms on the edge between Java & JVM.
Project Panama is in unique position here, because the scope of the 
project spans both Java & JVM layers. That's why we pay so much 
attention to the foundation.

JavaCPP and JNR are impressive examples of what can be done purely on 
library leve on top of stock JNI. So, it's very interesting to hear 
about your experience working with JNI, as the author of JavaCPP.

Anything on platform/JVM level that could improve/simplify your life?

Here's the list of new instruments we introduced in Panama so far:

   (1) Universal native call adapter (NativeInvoker [1])

Fully programmable Java -> native invocation mechanism which allows to 
express wide range of calling conventions at runtime. It was used to 
implement support for: varargs function calls, struct & vector 
arguments, callbacks.

   (2) Upcall (native -> Java) support (UncallHandler) [2]

Conversion between functional interfaces & function pointers enables 
calling instance methods on particular instances through function 
pointers on native side. For example, it supports passing a lambda as a 
callback into native function.

   (3) Machine code snippets (vectorSnippets branch [3])

Ability to wrap arbitrary machine code into a method handle and execute 
the code at runtime. JIT-compilers know about snippets and can inline 
their contents into generated code at their discretion.

   (4) MemoryRegion, Scope, Pointer, ... (java.nicl.types.* [4])

The API aims at providing safe, simple, and performant ways of handling 
allocations, updates, checking and life-cycle management of data both on 
and off the Java heap.

Does any of them sound useful in the context of JavaCPP? Or maybe any 
other ideas?

(I have been thinking about implementing exception handling support in 
new native call machinery (e.g., Java <-> native exception conversions), 
but haven't had time to experiment with that yet.)

Even if you are completely satisfied with JNI, JavaCPP can still benefit 
from work happening in Panama. In addition to new low-level API, there 
are plans to rewrite JNI support in JIT-compilers to benefit from newly 
introduced optimizations.

Best regards,
Vladimir Ivanov

[1] 
http://hg.openjdk.java.net/panama/dev/file/fd28575f60d7/src/java.base/share/classes/jdk/internal/nicl/NativeInvoker.java
 
http://hg.openjdk.java.net/panama/dev/file/fd28575f60d7/test/jdk/java/nicl/abi/sysv/x64/

[2] 
http://hg.openjdk.java.net/panama/dev/file/fd28575f60d7/src/java.base/share/classes/jdk/internal/nicl/UpcallHandler.java
 
http://hg.openjdk.java.net/panama/dev/file/fd28575f60d7/test/jdk/java/nicl/Upcall

[3] 
http://hg.openjdk.java.net/panama/dev/file/2a16a77edb27/test/jdk/panama/snippets/VectorUtils.java

[4] 
http://hg.openjdk.java.net/panama/dev/file/fd28575f60d7/src/java.base/share/classes/jdk/internal/nicl/types
 
http://hg.openjdk.java.net/panama/dev/file/fd28575f60d7/test/jdk/java/nicl/types

> To make this conversation constructive, though, I am also trying to make 
> suggestions, such as, why not use Clang? What is wrong with Clang? You 
> already use it for jextract, and Graal already supports LLVM. Plus, 
> people from Oracle Labs have already started work in this direction:
> Bringing Low-Level Languages to the JVM: Efficient Execution of LLVM IR 
> on Truffle (see "4. Native Calls and Memory Management")
> http://ssw.jku.at/General/Staff/ManuelRigger/VMIL16.pdf
> Is something like this going to be integrated into Panama?
> 
> Another constructive comment I would like to make is, would it be 
> possible to release all the relevant documents about jextract and Panama 
> so that people outside of Oracle could participate and make this a 
> community effort? If you need more data to convince managers that this 
> is required in order for this project to succeed, I can help with that! 
> However, as I mentioned previously, I feel that my help (or the help 
> from any outsiders for that matter) is not welcome here. It would be 
> great if this state of affairs could be improved! (This might be for 
> you, Bernard?)
> 
> Samuel