RFR: 8349637: Integer.numberOfLeadingZeros outputs incorrectly in certain cases [v2]

[Quan Anh Mai]

On Wed, 12 Feb 2025 16:14:05 GMT, Jasmine Karthikeyan <jkarthikeyan at openjdk.org> wrote:

>> src/hotspot/cpu/x86/c2_MacroAssembler_x86.cpp line 6242:
>> 
>>> 6240:   vpxor(xtmp2, xtmp2, xtmp2, vec_enc);
>>> 6241:   vpsrld(xtmp2, src, 24, vec_enc);
>>> 6242:   vpandn(src, xtmp2, src, vec_enc);
>> 
>> Are you sure this will work? I don't see how `x &~ (x >> 24)` can zero out all lower bits. Also, we should not kill `src` here.
>> 
>> I think a better solution is to do: `x > 0xFF ? x &~ 0xFF : x`. I chose this value because we have `0xFF` in `xtmp1` here.
>
> I believe the solution should work since my idea was to get rid of low bits when the high bits past 24 are all `1`, as that is the case where rounding behavior can incorrectly bump up the value. In the other cases, not removing all the low bits shouldn't have an impact on rounding and the exponent. At least, when running an [exhaustive test](https://gist.github.com/jaskarth/6b05352c3007a2650bf084fcb4c50c13) I wrote this patch fixes the output discrepancy. I can change it to the compare and blend you suggested, but I think that will be slower than this approach. Killing `src` is my mistake though, I will fix it in the next commit.

Yes you are right, that was my mistake, please make it clearer in the comment. I also think that you don't need to zero `xtmp2` before the right shift, am I right?

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/23579#discussion_r1953021165