<!DOCTYPE html><html><head><title></title><style type="text/css">p.MsoNormal,p.MsoNoSpacing{margin:0}</style></head><body><div>Good day,</div><div><br></div><div>For people reading this: What John refers to is an algorithm that does the following... I give you an ASCII string, say...<br></div><div><br></div><div>"I love Lucy".<br></div><div><br></div><div>Now you have matched the spaces (say). So that would be the bitset 01000010000 as in...<br></div><div><br></div><div><span class="font" style="font-family:menlo, consolas, monospace, sans-serif;">"I love Lucy"</span><span class="font" style="font-family:menlo, consolas, monospace, sans-serif;"><br></span></div><div><span class="font" style="font-family:menlo, consolas, monospace, sans-serif;"> 01000010000</span><br></div><div><br></div><div>and you want to get<br></div><div><br></div><div>"IloveLucy".<br></div><div><br></div><div>You can do this with a table lookup where the table is "I love Lucy" and the indexes are precomputed (we use 01000010000 as the key).<br></div><div><br></div><div>That's useful for a problem like base64 decoding because the base64 specification says that we must ignore ASCII white spaces.<br></div><div><br></div><div>It is stupidly efficient to do with SIMD.<br></div><div><br></div><div>Note that .NET does not yet support AVX-512 fully because if it did, it would be just one instruction... no need for a table... <br></div><div><br></div><blockquote type="cite" id="qt" style=""><div style="font-family:sans-serif;"><div class="qt-plaintext" style="text-wrap-mode:wrap;"><p dir="auto"> There is a question about how to <span style="text-wrap-mode:wrap;color:rgb(214, 216, 218);">handle out-of-range indexes in the permutation (lookup/table) vector; what is the policy for VectorTableLookup? Undefined behavior? Masking to power of two?</span><br></p></div></div></blockquote><div><br></div><div>In this particular instance, the algorithm is designed so that there is no out-of-bound access.<br></div><div><br></div><blockquote type="cite" id="qt" style=""><div style="font-family:sans-serif;"><div class="qt-plaintext" style="text-wrap-mode:wrap;"><p dir="auto">This is a fair amount of design work. As a shortcut, I think we should <span style="text-wrap-mode:wrap;color:rgb(214, 216, 218);">support appropriate tools for third parties (like your team Daniel) to build such things. I think an unchecked memory segment might be a good place to store such tables, and for prototyping one might consider digging into internal (post-guard) access subroutines like fromMemorySegment0Template.</span><br></p></div></div></blockquote><div><br></div><div>If we can write demos... like a Java port of the library you just checked, then I think it could provide insights.<br></div><div><br></div><div>I am obviously game to do it... but I bet that I am not alone.</div><div><br></div><div><br></div></body></html>