[nicl] branch is now closed

[Henry Jen]

> On Aug 31, 2018, at 11:28 PM, Samuel Audet <samuel.audet at gmail.com> wrote:
> 
> Now this is starting to sound like a plan! The point isn't about pointers or arrays, I'm not aware of all the discussions that went into that decision, and this is the point. All decisions related to Panama are made behind closed doors with no input from the community. It's the first time I hear about most of what you've just made public below, so this is a good start. Please keep making this kind of information public. I'm sure it will help make people feel welcome.
> 
> From the new information below, it seems like the plan is basically to upgrade a lot of the JVM based on ideas pretty much from LLVM, which sounds great, but LLVM happened because people started cooperating with each other: Apple, Microsoft, Intel, ARM, Qualcomm, Google, Cray, and many other smaller players. To achieve the same level of success with the JVM, there is going to be a need to gather the same level of cooperation. Your post makes me believe that the OpenJDK community is at least interested in the idea...
> 
> However, the fact is, there is still no cooperation whatsoever between Panama and GraalVM, but the parallel are striking, see this for example:  https://cornerwings.github.io/2018/07/graal-native-methods/
> 	Method		 	Time per iteration
> 	graal/native-image 	76.11 ns
> 	jni/unsafe 		109.42 ns
> Trying to get these teams on the same page might be a good start here. Why isn't this happening? I think it would be a useful exercise to understand the internal politics of Oracle, but maybe we don't need to know the details, just that the internal disputes are taken care of…
> 

Correct me if I am wrong, but those are different approaches and not exclusive. We noted that Graal polyglot solution, but it needs source code or LLVM bitcode, while here at Panama is focused on platform native format. 

Cheers,
Henry

> Samuel
> 
> On 08/27/2018 07:31 PM, Maurizio Cimadamore wrote:
>> On 25/08/18 09:34, Samuel Audet wrote:
>>> On 08/21/2018 06:34 PM, Maurizio Cimadamore wrote:
>>>> 
>>>> 
>>>> On 18/08/18 00:16, Samuel Audet wrote:
>>>>> About the Array type, it's still not clear to me why we can't just extend the Pointer with a "length"? It's not like pointers in C don't typically point to memory regions with length > 0: It's just often unknown. I've modeled the Pointer type in JavaCPP after Buffer (with position, limit, and capacity) and I haven't had any issues at all in years. None whatsoever. What is the benefit you perceive in mapping artificially 1 C concept into 2 Java concepts? I was not able to find the rationale from your document... If the idea is to provide safe get()/set() methods, why not simply use a layout? What does the Array type do that a layout could not do? That's basically what I do with the indexer package and it works fine:
>>>>> http://bytedeco.org/news/2014/12/23/third-release/
>>>>> Again, no issues whatsoever in years, without complicating the API! 
>>>> I'm not sure I follow your argument fully. In the C language spec there's not one concept, but two - we have array types and pointer types. Their behavior is quite different, so, for instance sizeof returns quite different things if you compare sizeof(int[]) with sizeof(int*). More generally, an array refers to a contiguous region of memory filled by N objects of the same kind. Layout-wise, an array is much more similar to a struct than it is to a pointer. E.g. int[3] and struct { int, int, int } have the same layout.
>>> 
>>> I fully agree there are a few differences that we can't deal with just pointers, but my point is, why not leave that up to a layout instead of introducing one more class to the mix? What precisely are the layouts not capable of accommodating that you had to introduce the Array class for?
>> I think you are mixing up different factors here.
>> - There is an API decision: what should be the (Java) carrier type for API points featuring C array types?
>> - Then there is the kind of mapping we want to achieve: we used to call this 'raw' mapping - that is, as close as possible to C semantics. More 'civilized' mapping will come to the fore at a later point.
>> - Then there is the role of layouts.
>> All these thee axis are quite orthogonal to each other.
>> In the C world, as I stressed, arrays and pointers are different things, though the C compiler might insert an auto conversion from the former to the latter and blur things a bit. So, I believe the sensible API choices would be:
>> 1) use Java array as carriers
>> 2) introduce a new carrier Array, and make it Array <: Pointer, to mimic the auto conversion
>> 3) introduce a new carrier Array - and have a way to go from Array to Pointer
>> As described in quite detail in the design document, using Java arrays as carriers is the wrong move. Java arrays comes with too many assumptions on how things are laid out in memory - and in many cases to go from a native array to a Java array you would need to copy. If this copy happens magically as soon as you call a struct getter returning an array, that is really bad: suppose you want to read an array from struct A, take its base pointer and pass it to function f() - why copy memory?
>> So, unless something like Array 2.0 comes to the rescue (that will allow modeling arrays as interfaces), using Java arrays is a prohibitive choice for the 'raw' mapping such as the one we're working on now. Later on, it will be possible to define 'civilized' bindings that automap native arrays to Java arrays 'magically' - but that has to be an opt-in, because of the performance model that comes with it. (very similar argument applies, BTW, to function pointer auto conversion).
>>> 
>>>> Of course there is a relationship between arrays and pointers, and that is caused by the fact that an array always decays to a 'base pointer' - when the context requires it. But the layout of this base pointer is rather different - e.g. if you take the base pointer of an int[3], you and up with a pointer whose pointee is 'int'.
>>>> 
>>>> Now, if we had a calculus for layouts which allowed to specify sizes as well - and maybe ours can do that with layout annotations:
>>>> 
>>>> u64(length=3):i32
>>>> 
>>>> Then it kind of seems that it would be possible to lump arrays and pointers together. But there's one detail that is omitted from this layout description: this description does not mandate that the elements occur 'right here right now'. That is, if I have a struct like this:
>>>> 
>>>> [ u64(length=3):i32 ]
>>>> 
>>>> Does the struct have a field of size 32 * 3 inline, or does the struct have a field of size 64 which points to some contiguous region of memory storing 3 ints? I believe the layout expression above suggests the latter.
>>> 
>>> So, maybe we should fix this in the layouts instead of coming up with Array just to patch over this?
>> And here we arrive at layouts: layouts are for describing region of memory (the region that, in the C lingo would be associated with an 'object'). A region of memory doesn't depend on what you do with it - it just exists, and the layout API gives you a way to describe its shape. So, the layout API has ways to describe sequence-like regions, and things like that.
>> When you have a pointer, the layout API can be used to describe the shape of the memory region associated with the pointee.
>> But using an 'array' layout and attaching that to a pointer doesn't really give you an array - it gives you a "pointer to an array". So I really can't parse your objections here. If I have struct like this:
>> struct foo {
>> int x[5];
>> }
>> I'd like the Java API to return something 'more' than Pointer - given that the C type of 'x' is not int* but int[5]. The model should take this distinction into account. I could understand a discussion between choices (2) and (3) - e.g. should Array just be a pointer + lentgth (e.g. use subtyping) or a separate abstraction; for now, our intuition on API design has told us that splitting arrays from pointers is the right move from an API perspective. So here we're talking API design - layout are totally orthogonal: carrier types in Java APIs and layouts are two different things (**).
>> (**) - yes, in the extreme case you could lump _all_ native types together, and just distinguish them using a layout field. But that would be such an awkward API to use that we're not entertaining that position in the design space. When you see a Java API , you'd like to know what to pass to it - and if everything is lumped, you turn a lot of statically checkable errors into dynamically checkable errors, to the point that the API is very hard to use right. So, I don't think layouts should be abused in this particular way.
>>> 
>>>> The Array API point is there to recognize this distinction: an array is a lump of memory; if you want you can take a pointer to the first array element out of it, and work that way - but the fact that it's easy to go in that direction doesn't mean there must be an 'is-a' relationship between the two entities. Of course, when you design an API there are many such decisions - John calls them with the apt definition splitting vs. lumping. At the time we considered both lumping (e.g. just use pointers) and splitting (the model we have now) and found that the latter offered much more clarity to the programmer. While arrays and pointers do overlap, there are certain differences (e.g. copy semantics on array struct field access) that, at the time we looked at them, pushed us towards the 'split' choice. It's not, of course, a choice set in stone, but I think it's one we're pretty happy with at the moment (unlike that, e.g. for function pointer automatic conversion).
>>> 
>>> Hum, it's sounds like you're effectively saying that layouts are inappropriate to map, for example, Protocol buffers or FlatBuffers, which are fast becoming the industry standard. Have you given any thoughts to Protocol Buffers or FlatBuffers and if so where can we find preliminary notes about those?
>> Nothing of that sort - in fact our layout calculus has been derived both by existing FFI use case _and_ by protocols use cases, see this very detailed email analyzing requirements of layouts in protocols:
>> http://mail.openjdk.java.net/pipermail/panama-dev/2018-February/000940.html
>>> 
>>>> As for your point on the relationship between Panama and Java arrays - Java arrays are essentially unsuitable for the kind of raw mapping we wanted to achieve at this stage. This does not preclude us to add a 'civilized' binding layer which autoconverts native arrays into Java arrays - but since that's an expensive move, it will be an opt in.
>>> 
>>> Good to hear that!
>>> 
>>>> The fact that Java arrays are not interfaces is biting us here, as well - Array 2.0 explored the possibility of having an Array<any X> interface, specialized in primitive types; with such support available we could indeed treat Panama arrays as specific instances of such a generalized interface. Until that support is available, we have to paper over the differences in other ways, which is what this API is attempting to do.
>>> 
>>> Right, but by not supporting normal arrays, you're preventing, for example, the vector team from benefiting from your work.
>>> 
>>> Vladimir expressed interest in the SVML intrinsics offered by the Intel C++ Compiler. That's basically a library of inline functions. We could have a header file with something like:
>>> 
>>> __m128 fast_sin(__m128 x) {
>>>     return __m128 _mm_sin_ps(x)
>>> }
>>> 
>>> __m256 fast_sin(__m256 x) {
>>>     return __m256 _mm256_sin_ps(x)
>>> }
>>> 
>>> And if Panama could inline this in Java, that's it! You've got a fast parallel version of sin() that is inlined with zero overhead when called from Java.
>> This stuff was already working at some point in the Panama repository. To make it work, we need suitable carrier types for vector types such as the one appearing in your signatures, and, to make things efficient, such carriers must better be value types (see Valhalla).
>> At this point we decided to split foreign an vector, for pragmatic reasons (the vector API was undergoing a lot of experimentation one year ago) - but note that the VM backend handling native calls still has support for vectorized types - we just miss the Java carriers to go with them, which will be conveniently provided by the vector API. So, when the two APIs will meet again we will just need to teach the binder about vector carriers, and the rest will just work (and we know this from experience, as the support was there).
>>> 
>>> So, you see, this kind of feature is in strong demand, and by your own team! I say we should do some dog fooding. Create something that others will actually want to use!
>> I hope I have shown that the cases you are thinking about are covered by our plan - we're not designing in silos and the Panama and Vector teams know about each other (if that is the concern you were trying to express).
>> If there is one issue that can be inferred from all of this is: to make a good Panama and Vector API (and here I'm talking about API not VM optimization internals), you need quite a bit of VM firepower. Both Panama and Vector would hugely benefit from (i) value types (e.g. if Panama pointers were values, their creation would be super cheap - ditto for vectorized types), (ii) generic specialization (both Panama classes and Vector classes are naturally generic over Java _primitive_ types) and (iii) Arrays 2.0 - as I've said above, some interface to abstract over the behavior of a Java array would be really useful in Panama.
>> So, all this to say - the APIs we are designing right now, do have to cope with some of the expressiveness limitations that come from the lack of such language/VM features. For instance, to counter the lack of generic specialization, Panama and Vector went in different paths: Panama used a single Pointer interface with boxed type parameter (but with MH under the hood to provide specialized access) - while the Vector API went down a path of manual specialization of the code. These are of course both suboptimal choices that we hope to be able to revert when more expressiveness will be available in the underlying layers of the language/VM.
>> But this has also nothing to do with pointer vs. arrays - that is just a C language distinction that we deemed important enough to reify in our Panama API. I understand other choices are possible, and I hope to have provided some rationale as to why we went the way we went about it.
>> Maurizio
>>> 
>>> If Panama could provide that kind of feature, many other users, at Skymind and elsewhere, could also use it to implement their own libraries of "fast math", so we wouldn't have to bother you guys all the time to implement some fast version of math. We'd just do it all ourselves as libraries! That would be great, wouldn't?
>>> 
>>> As always, please let us know how we can help. As Johan keeps reminding me, we could write JEPs and what not, but before we get there I would need to understand what would be required to make you interested in such a feature. In other words, what would you be looking for in the JEP to accept it yourself?
>>> 
>>> Samuel
>