Request for reviews (XS): 6552561: MaxRecursiveInlineLevel flag doesn't operate correctly

[Christian Thalinger]

http://cr.openjdk.java.net/~twisti/6552561/

6552561: MaxRecursiveInlineLevel flag doesn't operate correctly
Reviewed-by:

The current logic around MaxRecursiveInlineLevel does not calculate
the inline depth correctly in some situations resulting in different
recursive inlining depending on the compiled method.

The fix is to use the logic we already use for recursive method handle
targets.  This also handles indirect recursion correctly.

src/share/vm/opto/bytecodeInfo.cpp

---

Some before-after examples:

1. Direct recursive inlining...

...with -XX:MaxRecursiveInlineLevel=1:

Before:

    282    1             fib::fib (21 bytes)
                            @ 10   fib::fib (21 bytes)   inline (hot)
                              @ 10   fib::fib (21 bytes)   inline (hot)
                                @ 10   fib::fib (21 bytes)   recursively inlining too deep
                                @ 16   fib::fib (21 bytes)   recursively inlining too deep
                              @ 16   fib::fib (21 bytes)   inline (hot)
                                @ 10   fib::fib (21 bytes)   recursively inlining too deep
                                @ 16   fib::fib (21 bytes)   recursively inlining too deep
                            @ 16   fib::fib (21 bytes)   inline (hot)
                              @ 10   fib::fib (21 bytes)   inline (hot)
                                @ 10   fib::fib (21 bytes)   recursively inlining too deep
                                @ 16   fib::fib (21 bytes)   recursively inlining too deep
                              @ 16   fib::fib (21 bytes)   inline (hot)
                                @ 10   fib::fib (21 bytes)   recursively inlining too deep
                                @ 16   fib::fib (21 bytes)   recursively inlining too deep
    301    1 %           fib::main @ 2 (21 bytes)
                            @ 10   fib::fib (21 bytes)   inline (hot)
                              @ 10   fib::fib (21 bytes)   inline (hot)
                                @ 10   fib::fib (21 bytes)   recursively inlining too deep
                                @ 16   fib::fib (21 bytes)   recursively inlining too deep
                              @ 16   fib::fib (21 bytes)   inline (hot)
                                @ 10   fib::fib (21 bytes)   recursively inlining too deep
                                @ 16   fib::fib (21 bytes)   recursively inlining too deep

Results in different recursive inlining in the two compiles (2 vs. 1).

After:

    673    1             fib::fib (21 bytes)
                            @ 10   fib::fib (21 bytes)   inline (hot)
                              @ 10   fib::fib (21 bytes)   recursively inlining too deep
                              @ 16   fib::fib (21 bytes)   recursively inlining too deep
                            @ 16   fib::fib (21 bytes)   inline (hot)
                              @ 10   fib::fib (21 bytes)   recursively inlining too deep
                              @ 16   fib::fib (21 bytes)   recursively inlining too deep
    702    1 %           fib::main @ 2 (21 bytes)
                            @ 10   fib::fib (21 bytes)   inline (hot)
                              @ 10   fib::fib (21 bytes)   inline (hot)
                                @ 10   fib::fib (21 bytes)   recursively inlining too deep
                                @ 16   fib::fib (21 bytes)   recursively inlining too deep
                              @ 16   fib::fib (21 bytes)   inline (hot)
                                @ 10   fib::fib (21 bytes)   recursively inlining too deep
                                @ 16   fib::fib (21 bytes)   recursively inlining too deep

...with -XX:MaxRecursiveInlineLevel=0:

Before:

    396    1             fib::fib (21 bytes)
                            @ 10   fib::fib (21 bytes)   inline (hot)
                              @ 10   fib::fib (21 bytes)   recursively inlining too deep
                              @ 16   fib::fib (21 bytes)   recursively inlining too deep
                            @ 16   fib::fib (21 bytes)   inline (hot)
                              @ 10   fib::fib (21 bytes)   recursively inlining too deep
                              @ 16   fib::fib (21 bytes)   recursively inlining too deep
    414    1 %           fib::main @ 2 (21 bytes)
                            @ 10   fib::fib (21 bytes)   inline (hot)
                              @ 10   fib::fib (21 bytes)   recursively inlining too deep
                              @ 16   fib::fib (21 bytes)   recursively inlining too deep

Results in different recursive inlining in the two compiles (1 vs. 0).

After:

    602    1             fib::fib (21 bytes)
                            @ 10   fib::fib (21 bytes)   recursively inlining too deep
                            @ 16   fib::fib (21 bytes)   recursively inlining too deep
    633    1 %           fib::main @ 2 (21 bytes)
                            @ 10   fib::fib (21 bytes)   inline (hot)
                              @ 10   fib::fib (21 bytes)   recursively inlining too deep
                              @ 16   fib::fib (21 bytes)   recursively inlining too deep

2. Indirect recursive inlining...

...with -XX:MaxRecursiveInlineLevel=1:

Before:

    292    1 %           indirect::main @ 2 (21 bytes)
                            @ 10   indirect::foo (14 bytes)   inline (hot)
                              @ 10   indirect::bar (14 bytes)   inline (hot)
                                @ 10   indirect::foo (14 bytes)   inline (hot)
                                    @ 10   indirect::foo (14 bytes)   inline (hot)
                                      @ 10   indirect::bar (14 bytes)   inline (hot)
                                        @ 10   indirect::foo (14 bytes)   inline (hot)
                                          @ 10   indirect::bar (14 bytes)   inline (hot)
                                            @ 10   indirect::foo (14 bytes)   inline (hot)
                                              @ 10   indirect::bar (14 bytes)   inline (hot)
                                                @ 10   indirect::foo (14 bytes)   inlining too deep

Indirect recursive inlining not handled at all, the logic inlines until MaxInlineLevel is hit (which is 9 by default).

After:

    624    1 %           indirect::main @ 2 (21 bytes)
                            @ 10   indirect::foo (14 bytes)   inline (hot)
                              @ 10   indirect::bar (14 bytes)   inline (hot)
                                @ 10   indirect::foo (14 bytes)   inline (hot)
                                  @ 10   indirect::bar (14 bytes)   inline (hot)
                                    @ 10   indirect::foo (14 bytes)   recursively inlining too deep

Every recursive call is inlined once.

...with -XX:MaxRecursiveInlineLevel=0:

After:

    602    1             indirect::foo (14 bytes)
                            @ 10   indirect::bar (14 bytes)   inline (hot)
                              @ 10   indirect::foo (14 bytes)   recursively inlining too deep
    621    2             indirect::bar (14 bytes)
                            @ 10   indirect::foo (14 bytes)   inline (hot)
                              @ 10   indirect::bar (14 bytes)   recursively inlining too deep
    623    1 %           indirect::main @ 2 (21 bytes)
                            @ 10   indirect::foo (14 bytes)   inline (hot)
                              @ 10   indirect::bar (14 bytes)   inline (hot)
                                @ 10   indirect::foo (14 bytes)   recursively inlining too deep

No recursive inlining.