Draft JEP: Vector API

[Frank Yuan]

> 
> Hi Frank,
> 
> As we progress and new language/runtime features become available i think we will be able to make improvements to the API. However,
> those language/runtime features will take some time to materialize.
> 
> In the near term we are trying to produce something useful for advanced developers they can use today, which means the API is bound by
> the current language/runtime constraints, and to some extent by the current CPU architectures.
> 
> My expectation is value types being developed in Project Valhalla will be very helpful. As value type features are incrementally introduced i
> expect the Vector API can be adapted (perhaps the internal intrinsic strategy will not change much  and will be more impacted by any
> future ARM SVE support).
> 

Thank you, Paul! I am looking forward to the future improvements!

> Here are some further thoughts:
> 
> - Vectors are morally values, we currently specify them to be value-based like Optional, we plan to have some opt-in HotSpot optimizations
> that can play fast and loose with identity to reduce the chance of boxing.
> 
> - Generics over primitive types and method specialization will help shrink the number of vector types and methods.
> 
> - Value types will enable compound numeric types such as Int128 or Complex. I am sure that will increase the desire for operator
> overloading :-) and perhaps it's possible to do something constrained in that area for numeric types and vector types, thereby making
> mathematical computations look more natural in their expression.
> 

Yes, this is my major point!

> - We have also been looking into a technique we call lambda cracking: taking a lambda expression and reliably deconstructing the AST
> (bytecodes are not necessarily the best place to start from in this respect). So we could express a computation in scalar form and “lift” it
> into vector form. We currently don’t have a higher-order map function on Vector that accepts a function operating on scalar values
> because we cannot reliably optimize this by deconstructing the lambda expression into equivalent vector operations.
> 

I suppose you would provide an interface, which allow the user to define an operation against the scalar of a type, and can apply it on a vector automatically.
It's convenient option, but most of the users who want to use vector computation should care the performance, so I think they should try to composite the vectorized function/operations as possible to construct their own functions.
As you said, it's hard to map to efficient vector operations from arbitrary expressions. 


Btw, talk about the future, we may consider this JEP with http://openjdk.java.net/jeps/8047074 together :)

Thanks
Frank

> Paul.
> 
> 
> > On Apr 12, 2018, at 12:44 AM, Frank Yuan <frank.yuan at oracle.com> wrote:
> >
> >
> >> -----Original Message-----
> >> From: Lupusoru, Razvan A [mailto:razvan.a.lupusoru at intel.com]
> >> Subject: RE: Draft JEP: Vector API
> >>
> >> Hi Frank,
> >>
> >> Thanks a lot for your feedback.
> >>
> >> As you point out, the Vector API works by providing instance methods for vector operations instead of exposing static methods for
> > the
> >> computations. The current interface is based on John Rose's Vector API strawman proposal [1] so I am hoping he can reply with some
> >> additional justification on doing it this way.
> >>
> >> That said, I can explain why from an implementation standpoint, this works. Right now, we rely on the following properties to be
> > able to do
> >> our translation from the API to actual vector instructions:
> >> - Being an instance method means we can override the implementations in each of the subclasses due to polymorphism. This is
> > important
> >> because as part of the intrinsification process we obtain various information like length and type of vector, along with
> > information about
> >> classes for unboxing/boxing. You will see this behavior difference by taking a look at Float128Vector.java and Float256Vector.java
> > and look at
> >> a method like "add" which shows these differences based on arguments passed to method that ends up getting intrinsified.
> >> - Second, we rely on successful type sharpening from C2. Because type profiling is primarily enabled for receiver types (default
> > argument
> >> profiling setting is not adequate), we use existing framework to determine appropriate vector type. This allows for inlining of
> > the
> >> appropriate methods which in turn pulls in information needed for intrinsification as noted in first bullet point. This way we do
> > not need a
> >> separate implementation for speculating or sharpening types. And because C2 is quite good at type sharpening, we can get good
> > vector
> >> code without overheads of class checks.
> >>
> >> Regarding to having a higher level API so that users do not have to worry about the details of vector programming, we agree that
> > it makes
> >> sense. Please see Sandhya's email to Ningsheng [2].
> >>
> >> From my point of view, I see two key ingredients in current API that are useful as building blocks for a higher level API that
> > automatically
> >> generates and handles things like tail processing: 1) Currently, algorithms can be written in a shape agnostic manner 2)
> > Operations have
> >> masked variants which are useful for excluding lanes from vector computations.
> >
> > Yes, it's useful as building blocks for a higher level API. It's good progress as the JEP Motivation said "The Vector API aims to
> > address these issues by providing a mechanism to write complex vector algorithms in Java.".
> > However, personally, I hope to have the easy vector API like Matlab, Numpy or something else in Java.
> >
> >>
> >> Hopefully this provides some clarification. Thanks again for taking a look at our proposal.
> >
> > Really appreciate your reply and your effort on this JEP!
> >
> > Frank
> >
> >>
> >> --Razvan
> >>
> >> -----Original Message-----
> >> From: Frank Yuan [mailto:frank.yuan at oracle.com]
> >> Sent: Monday, April 09, 2018 9:05 PM
> >> To: Viswanathan, Sandhya <sandhya.viswanathan at intel.com>; jdk-dev at openjdk.java.net; 'John Rose' <john.r.rose at oracle.com>
> >> Cc: 'Vladimir Ivanov' <vladimir.x.ivanov at oracle.com>; Aundhe, Shirish <shirish.aundhe at intel.com>; Lupusoru, Razvan A
> >> <razvan.a.lupusoru at intel.com>
> >> Subject: RE: Draft JEP: Vector API
> >>
> >> Hi Sandhya
> >>
> >> I read the JEP bug, and have 2 comments.
> >>
> >>
> >> The first one is about the stream style example code. Suppose Vector API is used for math computation, the code should likely
> > express the
> >> math concept, and look like the math formula.
> >>
> >> For example:
> >> void scalarComputation(float[] a, float[] b, float[] c) {
> >>   for (int i = 0; i < a.length; i++) {
> >>        c[i] = (a[i] * a[i] + b[i] * b[i]) * -1.0f;
> >>   }
> >> }
> >>
> >> The proposed Vector API code is like:
> >> Vector vc = va.
> >>        mul(va).
> >>        add(vb.mul(vb)).
> >>        neg();
> >>
> >> It's far away from the math expression: - ( va * va + vb * vb )
> >>
> >> I would like the following code more:
> >> Vector vc = neg(
> >>        add(mul(va, va), mul(vb, vb)));
> >>
> >>
> >> The second comment is that the users should not like to handle the details on the vector size, they only desire the vector
> > computation
> >> power! We should wrap all these details, provide a high level abstract for the vector in terms of the math concept, do the loop in
> > our private
> >> code and pad the tail.
> >> User just say "I want vector a subtract vector b", then we give him a result.
> >>
> >>
> >> Thanks
> >> Frank
> >>
> >>
> >>> -----Original Message-----
> >>> From: jdk-dev [mailto:jdk-dev-bounces at openjdk.java.net] On Behalf Of
> >>> Viswanathan, Sandhya
> >>> Subject: Draft JEP: Vector API
> >>>
> >>>
> >>>
> >>> This draft JEP contains a proposal to provide an incubating module to
> >>> express vector computations in Java that reliably compiles
> >> at runtime
> >>> to optimal vector hardware instructions on supported CPU architectures
> >>> and thus achieve superior performance than equivalent
> >> scalar
> >>> computations.
> >>>
> >>>
> >>> For more details, please see:
> >>> https://bugs.openjdk.java.net/browse/JDK-8201271
> >>>
> >>> Best Regards,
> >>> Sandhya
> >>>
> >>
> >
> >