Vector API : Lack of support for vectorized lookup tables

[Chris Hegarty]

Hi,


On 21/06/2024 21:11, Paul Sandoz wrote:
> Hi Daniel,
> 
> Thanks for the email discussion. I think we can do better and get to, or closer to, your expectation on the generation of a single instruction.
> 
> At the moment rearrange performs bounds checks on the shuffle argument (checking for exceptional indexes), and selectFrom converts a vector to a shuffle and calls rearrange. 

...

We've run into this separately too. If memory serves me correctly, it 
was slice(int) using rearrange internally to only then do expensive 
bounds checks on the implementation specific shuffle. ( I can try to dig 
out the details if I've mixed up something here. )

Bounds checks on input at the API level is expected, but not when the 
rearrange is an implementation detail.

More generally, the system property has been around for a while now, and 
from my understanding it's a workaround to the compiler not yet being 
able to eliminate the checks. Firstly, is this understanding correct? 
And if so, is the intention to eventually remove it when it is no longer 
needed?

...

 > To be fair, it is fine if folks say "No, we don't want to do this 
with Java Vector, we don't want to optimize indexing in Lucene, we want 
to do X".

Lucene is already using the Panama Vector API to improve the performance 
of vector similarity comparisons - for vector search. We would love to 
use it for more, but are largely hampered by the above issue. A less 
ideal situation is that in Elasticsearch [1] we're now using Panama FFI 
to workaround the limitations of the Vector API by linked to our own 
optimized implementations. This not ideal, and effectively robs Lucene 
of these potential performance benefits (since it's non-trivial to ship 
Lucene with a native lib)

-Chris.

[1] https://github.com/elastic/elasticsearch/tree/main/libs/simdvec