Vector API "Straw Man" Boxed/Unboxed Vector Instantiation

[Brian Goetz]

For sure, convergence with Valhalla should be part of the goal. For 
example, we made API choices with VarHandles to not try and make them 
more type-safe, because (a) it was a low-level API and (b) the ability 
to make them type-safe at lower cost was coming, and we could wrap VHs 
later with specialize type-safe wrappers. We may want to consider 
running this play again, as you allude to in (3).

On 3/25/2016 5:45 PM, Graves, Ian L wrote:
> Hello!
>
> We've been working with some in-house implementations of the straw man[1] Vector API provided by John and test driving it with some workloads to look for any apparent pain points one could have when using the API to accelerate a common task.  One place we started was with simple matrix multiplication of an (let's say two dimensional) array of unboxed floats.  We assume the role of a user looking to accelerate a computation given data already in this format.  The unaltered straw man API gives us a method to instantiate vectors from an array in the Species of a given Vector with the fromArrayFactory(Float[] f, Integer offset) method.  We've been trying to avoid making any significant opinionated changes to the API or specific deviations in an implementation, but some of the issues around this workload warrant some thought (or some help, if somebody knows something I don't [quite likely!]).
>
>
> 1.       Little Quibble: It seems as though in order to instantiate a Vector by using a method of the Species, one first needs an instance of a species of that Vector (or a species).  Is this the case?  This isn't a big deal as we could provide a singleton dummy instance for this or something similar.
>
> 2.       Not-so-Little Quibble: Assuming that a user wants to leverage this API by starting with primitive types in arrays, the current iteration of the API would seem to require arrays to be boxed because the vector-building methods are all parameterized by element type.  For folks coming from primitive arrays, this seems like a non-starter, so maybe I am missing something.  What does the path to Vector usage look like if I am coming from primitive arrays?
>
> We've war-gamed a couple of possibilities here.  I'd love to get the thoughts of the people on this listserv.  I'll list them out.
>
>
> 1.       Vanilla/Status Quo:  We stick with requiring boxed types for now for elegance and simplicity's sake.  Construction via parameterized methods will be slower, and perhaps some other non-API method would be better suited to this (static factory methods, etc).
>
> 2.       Shell Game: Vector<Element,Shape> -> ConcreteVector takes on a similar flavor and relationship to BaseStream<Element, Stream> -> IntStream (say).  That is, we have element-wise concrete instances of these vectors (FloatVector<Shape> or FloatVector256/FloatVector8) that have a standardized suite of methods that support primitives and are named consistently across the API like (Int|Double|Long)Stream.  This design could incorporate additional interfaces or inherit from abstract classes that contain repeated functionality across element types that we support.
>
> 3.       Deus ex Machina: JDK 10 or onward introduces a form of specialization[2] powerful enough to let us have our cake and eat it too.  While perhaps this isn't something we can bet on to happen concurrent with this API design (or could we?), this might be something we should be planning for.  That is, if we know that the API in its current state would be enhanced almost wholesale by the introduction of specialization and/or value types, perhaps that would be a reason enough to leave it how it is and avoid fragmenting it into overly concretized instances.
>
> 4.       Boxing Elision: Perhaps there's a way we can elide the boxing on these arrays with compiler optimizations, or is there a way already?
>
> We are particularly interested in being able to build vectors from arrays because arrays are amenable to gathering[3] and scattering[4] instructions.  We can construct lightweight Vector views over arrays of primitives using gathering and scattering (for writes).  If the arrays are arrays of boxed types, that makes scattering and gathering a bit more complex as we add a non-uniform layer of indirection to each array index.  This is quite useful for workloads like matrix multiplication where it might make sense to stride an array to build a vector.
>
> I would love to get thoughts on how we might negotiate this boxing issue, if indeed it is an issue.
>
> Thanks!
>
> Ian
>
> [1] http://cr.openjdk.java.net/~jrose/arrays/vector/Vector.java
> [2] http://cr.openjdk.java.net/~briangoetz/valhalla/specialization.html
> [3] https://software.intel.com/sites/landingpage/IntrinsicsGuide/#text=gather_ps&techs=AVX,AVX2&expand=2798
> [4] https://software.intel.com/sites/landingpage/IntrinsicsGuide/#text=scatter_ps&techs=AVX_512&expand=2798,4072,2823,2839
>
>