Vector API : Lack of support for vectorized lookup tables

[Daniel Lemire]

Good day,

For people reading this: What John refers to is an algorithm that does the following... I give you an ASCII string, say...

"I love Lucy".

Now you have matched the spaces (say). So that would be the bitset 01000010000 as in...

"I love Lucy"
 01000010000

and you want to get

"IloveLucy".

You can do this with a table lookup where the table is "I love Lucy" and the indexes are precomputed (we use 01000010000 as the key).

That's useful for a problem like base64 decoding because the base64 specification says that we must ignore ASCII white spaces.

It is stupidly efficient to do with SIMD.

Note that .NET does not yet support AVX-512 fully because if it did, it would be just one instruction... no need for a table... 

>  There is a question about how to handle out-of-range indexes in the permutation (lookup/table) vector; what is the policy for VectorTableLookup? Undefined behavior? Masking to power of two?
> 

In this particular instance, the algorithm is designed so that there is no out-of-bound access.

> This is a fair amount of design work. As a shortcut, I think we should support appropriate tools for third parties (like your team Daniel) to build such things. I think an unchecked memory segment might be a good place to store such tables, and for prototyping one might consider digging into internal (post-guard) access subroutines like fromMemorySegment0Template.
> 

If we can write demos... like a Java port of the library you just checked, then I think it could provide insights.

I am obviously game to do it... but I bet that I am not alone.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/panama-dev/attachments/20240917/e6da1dd0/attachment.htm>