Optimize signed integer comparison

[Tobias Hartmann]

Hi,

it would be nice if someone could have a look at this. While working at 
JDK-8043284 [1] I noticed the following problem (see attached example 
Unreachable.java):

The method test(..) takes a char c, casts it to int and adds INT_MAX to 
it. If the result is equal to CHAR_MAX the method shouldNotReachHere(..) 
is executed. But no matter what value c has, shouldNotReachHere(...) is 
never executed because after the addition the result is either MAX_INT 
or overflows and has a value between INT_MIN and INT_MIN + CHAR_MAX - 1.

The problem is that the C2 compiler does not detect this unreachable 
code. The C2 type system merges the two possible integer ranges 
[INT_MIN] and [INT_MIN + CHAR_MAX -1] into the general [INT_MIN, 
INT_MAX] range. The CmpINode is then unable to detect the inequality and 
returns the TypeInt::CC condition code which we cannot optimize.

To fix this we can check in CmpINode::Value(..) if the result of the 
AddINode overflows and if so compare the two possible ranges with the 
target value (very similar to what Vladimir did for [2]). If both ranges 
are unequal we would return an unequal condition code and translate it 
to a boolean value in BoolNode::Value(..). This would lead to removal of 
all unreachable nodes in the graph.

However, the problem with this solution is that we currently do not have 
an unequal type (we only have LT, GT, EQ, LE, GE) and adding one seems 
to be difficult because the CC integer range [-1, 1] is already used 
(see type.cpp line 297). I wonder why we don't have an own type for the 
condition codes but use TypeInt.

Because my example is quite artificial, I did some testing with Octane 
and it looks like as if this happens quite often. I therefore think we 
should fix it.

What do you think?

Thanks,
Tobias

[1] https://bugs.openjdk.java.net/browse/JDK-8043284
[2] https://bugs.openjdk.java.net/browse/JDK-8042786
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Unreachable.java
Type: text/x-java
Size: 467 bytes
Desc: not available
URL: <http://mail.openjdk.java.net/pipermail/hotspot-compiler-dev/attachments/20140624/76145498/Unreachable.java>