RFR (XS): 8035887: VM crashes trying to force inlining the recursive call

[Vladimir Kozlov]

On 2/28/14 9:27 AM, Vladimir Ivanov wrote:
> Vladimir, thanks for review!
>
> With the addition of recursive depth check for force_inline case, the only difference should be compiled lambda form
> case. But I haven't been able to come up with a test case which demonstrates the issue.
>
> Strangely, stack overflow in compiler thread Chris fixed a while back (8011138) was observed only in C2.
>
> I'd like to keep this fix as is for now. I'll spend more time investigating compiled lambda form recursive inlining
> behavior in C1, and file a bug if necessary.

Okay. Webrev.02 seems fine.

Thanks,
Vladimir K

>
> Best regards,
> Vladimir Ivanov
>
> On 2/27/14 9:27 PM, Vladimir Kozlov wrote:
>> Vladimir,
>>
>> I think C1 still missing check for recursive depth in case of
>> force_inline(). In C2 recursion check is done for all types of inlining.
>> Yes, in case of lambda inlining it needs to check receiver. Only then C1
>> and C2 will match.
>>
>> Thanks,
>> Vladimir K
>>
>> On 2/27/14 9:09 AM, Vladimir Ivanov wrote:
>>> http://cr.openjdk.java.net/~vlivanov/8035887/webrev.00/
>>> https://bugs.openjdk.java.net/browse/JDK-8035887
>>>
>>> 4 lines changed: 3 ins; 0 del; 1 mod
>>>
>>> C1 overflows the stack when it tries to inline a recursive call of a
>>> method which is forced for inlining by CompilerOracle.
>>>
>>> The problem is that C1 doesn't check inlining depth for methods forced
>>> for inlining by CompilerOracle.
>>>
>>> The fix is to add missing checks. I added 2 checks (total depth and
>>> recursive depth). The former is to avoid a situation
>>> (very unlikely) when a long chain of methods, which are forced for
>>> inlining, overflows compiler stack. The latter is to
>>> unify behavior between C1 & C2.
>>>
>>> No regression test is added because it can take very long time to
>>> provoke the crash in some configurations.
>>>
>>> Testing: failing test
>>>
>>> Best regards,
>>> Vladimir Ivanov