8238691: C2: turn subtype check into macro node

[Vladimir Ivanov]

Hi Roland,

> http://cr.openjdk.java.net/~roland/8238691/webrev.00/

Finally! :-)

I verified that it fixes JDK-8220708 [1].

Overall, the patch looks good.

src/hotspot/share/opto/loopopts.cpp:

+  if (cmp_op == Op_SubTypeCheck) {
+    return NULL;
+  }

Please, add a comment.

Also, during performance testing we spotted that it becomes more likely 
to hit hard compilation limits with late expansion of subtype checks 
(e.g., on Octane -XX:MaxNodeLimit=160000 -XX:NodeLimitFudgeFactor=3200 
helped to recover performance).

It doesn't look specific to subtype checks, but more about how macro 
nodes are handled. Existing heuristics which rely on node count don't 
take into account how many macro nodes are there and may behave too 
aggressively pushing the compilation to the limit which is hit during 
macro expansion.

What's your take on that? I'm fine with handling it separately, but 
though the problem is not new, it looks like the patch may worsen the 
situation in some cases.

Meanwhile, it looks feasible to have a diagnostic flag to force eager 
subtype check expansion during parsing. We can remove it later.

Best regards,
Vladimir Ivanov

[1] https://bugs.openjdk.java.net/browse/JDK-8220708

$ .../java -showversion TestRegex
openjdk version "13.0.1" 2019-10-15
OpenJDK Runtime Environment (build 13.0.1+9)
OpenJDK 64-Bit Server VM (build 13.0.1+9, mixed mode, sharing)
Starting benchmark...
........................................................................................................................................................................................................Finished 
all threads
Finished running benchmark - took 6960 millis

$ .../java -showversion TestRegex
openjdk version "15-internal" 2020-09-15
OpenJDK Runtime Environment (build 15-internal+0-adhoc.vlivanov.jdk)
OpenJDK 64-Bit Server VM (build 15-internal+0-adhoc.vlivanov.jdk, mixed 
mode, sharing)
Starting benchmark...
........................................................................................................................................................................................................Finished 
all threads
Finished running benchmark - took 735 millis

> I propose we turn subtype check into a macro node. This patch adds a new
> node: SubTypeCheckNode which is a subclass of CmpNode. That node itself
> can float around but because it feeds into an If, it's expanded at the
> location of the If.
> 
> This should allow subtype checks that can't be compiled as a simple
> pointer comparison at parse time to:
> 
> - better constant fold if type information improves as compilation
>    proceeds
> 
> - be candidate for predication and split if
> 
> - have a chance of being compiled as a simple pointer comparison if type
>    information improves
> 
> The implementation is relatively straightforward as there's already
> logic to build a subtype check during macro expansion. Also, predication
> and split if comes for free because the new node floats and is a subtype
> of CmpP.
> 
> I changed the subtype check implementation to use immutable memory for
> the super klass check offset. The existing implementation of subtype
> check is a bit inconsistent when it comes to memory state handling: the
> only remaining load that doesn't consume immutable memory is for the
> secondary super type cache (which makes sense given that slot is
> mutable) but nothing mutates that memory in the C2 IR when
> PartialSubtypeCheckNode should I think. The new SubTypeCheck node
> doesn't carry memory state but it emulates the current implementation by
> walking the graph and looking for bottom memory which given nothing
> mutates the secondary super cache is the same memory state that is
> somewhat erroneously used today.
> 
> Roland.
>