reusing more of LLVM

[Maurizio Cimadamore]

On 01/02/2019 19:49, Vladimir Ivanov wrote:
>
>> Thanks for taking the time to reply. I'm talking about SystemABI, in 
>> this case. Maurizio posted a demo of how to call functions like 
>> getpid(), pow(), qsort() here previously:
>> https://mail.openjdk.java.net/pipermail/panama-dev/2018-November/003316.html 
>>
>>
>> LLVM has all the machinery to JIT compile all that in the same way, 
>> and already includes ABIs for many architectures, see this tutorial 
>> for example where they wrap the API in a toy language:
>> http://llvm.org/docs/tutorial/LangImpl04.html#adding-a-jit-compiler
>>
>> So, why not reuse that code? I would really like to understand the 
>> reasons why this wasn't considered. Is it because it's a C++ API? But 
>> isn't that the goal of Panama Foreign/Native anyway? To let Java 
>> developers use C++ APIs? If there are technical reasons why this 
>> wouldn't work, I honestly can't think of anything.
>
> Sorry, I still don't fully understand your question. You start with 
> System ABI API and then jump to "LLVM as a JIT", but I consider those 
> as completely orthogonal (API vs implementation considerations).

Right - SystemABI, at least the one discussed in the the thread you 
mention, is just a Java API to get a MethodHandle given a native 
function. MethodHandle is the API toolkit that allows low level 
frameworks to speak about things that can be invoked (not necessarily 
Java methods) - it is a valuable tools in many context and, since its 
inception with JSR 292 it has changed the landscape when it comes to 
allow foreign languages to be implemented on top of the VM (many 
existing interpreters  for scripting languages, such as JRuby and Jython 
have in fact been rewritten to take advantages of these features).

So, what Panama's SystemABI does here is to simply close an hole: you 
can currently ask a MethodHandle for a Java method, or a MethodHandle 
for a native, JNI method. But there's no way to ask for MethodHandle 
that simply acts as an entry point to directly call an underlying native 
function (e.g. a MethodHandle which describes a call to 'getpid') - or 
the opposite: to turn an existing method handle into a pointer which can 
be passed to native code as a callback.

I don't see how this can be described as re-inventing LLVM - to me it 
seems like making the JSR 292 machinery aware of the new capabilities 
offered by the layers of integration provided by the Panama API.

Maurizio

>
> I'd like to reiterate that "LLVM as a JIT" discussion is out of scope 
> for Project Panama. Moreover, it's completely unrelated to API 
> considerations which should be implementation-agnostic: it doesn't 
> matter what the JVM relies on under-the-hood - C2, Graal, Terracotta, 
> LLVM - when it comes to API.
>
> From compiler framework (whichever you prefer more) perspective, ABI 
> support is a simple task (even not worth calling it a problem).
> The real challenges come when you try to mix code of different origins 
> (e.g., managed vs unmanaged) and want to minimize the overhead of 
> interactions while preserving correctness.
>
> On API level it pushes for a flexible API to communicate important 
> semantics both ways and to compiler, so it has enough information to 
> keep the boundary as thin as possible.
>
> That's why I'm confused when you refer to both System ABI *API* and 
> LLVM *compiler framework*.
>
> But if you are mostly concerned with implementation side, then I don't 
> see LLVM as a viable option (as of now), because there's no 
> production-ready LLVM-based compiler available in OpenJDK to rely on.
>
> Best regards,
> Vladimir Ivanov
>
>> On 2/1/19 9:17 AM, Vladimir Ivanov wrote:
>>> Samuel,
>>>
>>> Can you elaborate your point a bit, please?
>>>
>>> Shark was an attempt to bring LLVM-based JIT compiler to HotSpot 
>>> JVM, but that goal is orthogonal to Panama where the focus is on 
>>> interconnecting managed code with native code / hardware.
>>>
>>> LLVM definitely looks attractive in some respects, but I don't 
>>> understand what you mean by "reimplementing portions of LLVM in Java".
>>>
>>> Best regards,
>>> Vladimir Ivanov
>>>
>>> On 31/01/2019 14:40, Samuel Audet wrote:
>>>> I am starting the get the impression that Panama is basically 
>>>> reimplementing portions of LLVM in Java as part of SystemABI. How 
>>>> much effort would it take to, for example, implement the required 
>>>> subset of the Java ABI in LLVM? Doing it that way, we would not 
>>>> need to test the C/C++ ABI, at least, making it possible instead to 
>>>> reuse tests from the JDK itself, or am I talking nonsense here? 
>>>> Looks like that might be challenging as well: 
>>>> http://openjdk.java.net/jeps/8189173. In any case, it would be 
>>>> great if we could have a discussion about these things instead of 
>>>> having OpenJDK dictate everything!
>>