RFR: 8204479: Bitwise AND on byte value sometimes produces wrong result

[Aleksey Shipilev]

Bug:
  https://bugs.openjdk.java.net/browse/JDK-8204479

Fix:
  http://cr.openjdk.java.net/~shade/8204479/webrev.03/

The trouble with unsigned comparisons is that original code would have generate:

 movzbl (mem), %r
 cmp %r, $imm
 jl ...

...while the new code with compUB_mem_imm matcher generates:

 cmpb (mem), $imm
 jl ...

Original code loaded non-sign-extended byte into int register and then safely did signed comparison.
New code does signed byte comparison against memory, which is incorrect. It is still safe when we do
proper LoadB, but not LoadUB.

Vladimir thinks the second matcher (testUB_mem_imm) suffers from the same issue. I don't think it is
ever exposed in our code, because comparison with zero usually conditionally jumps on ZF. But testb
does set SF to the MSB of and-ed operands, so we can accidentally use (incorrect) SF for unsigned
byte in conditional jump and fail? I haven't been able to reproduce that in the test though.

This partially reverts JDK-8203628 change:
  http://hg.openjdk.java.net/jdk/jdk/rev/197ee9d8e228

Testing: regression test, failing Nashorn tests

Thanks,
-Aleksey

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: OpenPGP digital signature
URL: <http://mail.openjdk.java.net/pipermail/hotspot-compiler-dev/attachments/20180608/4f81fbfe/signature.asc>