RFR (XS): 8028159: C2: compiler stack overflow during inlining of @ForceInline methods

[Vladimir Ivanov]

Thank you, Vladimir.

Best regards,
Vladimir Ivanov

On 11/13/13 11:36 PM, Vladimir Kozlov wrote:
> Sorry, I forgot about that. Then your code is fine.
>
> thanks,
> Vladimir
>
> On 11/12/13 10:17 PM, Vladimir Ivanov wrote:
>> Are you sure? Last time we decided to make it develop (corresponding
>> letter attached).
>>
>> Best regards,
>> Vladimir Ivanov
>>
>> On 11/13/13 10:09 PM, Vladimir Kozlov wrote:
>>> Make the flag product as other inline limitation flags.
>>>
>>> Thanks,
>>> Vladimir K
>>>
>>> On 11/13/13 6:22 AM, Vladimir Ivanov wrote:
>>>> http://cr.openjdk.java.net/~vlivanov/8028159/webrev.00/
>>>> https://bugs.openjdk.java.net/browse/JDK-8028159
>>>> 12 lines changed: 7 ins; 3 del; 2 mod
>>>>
>>>> There's a corner case when C2 inliner fails to delay inlining of
>>>> @ForceInline method (e.g. nested
>>>> MethodHandle::asType(MethodType) calls produce such pattern):
>>>>     @ForceInline m1 -> MH::invokeBasic -> @FI m2 -> MH::invokeBasic ->
>>>> @FI m3 -> ...
>>>>
>>>> Having method handle chain deep enough, it's possible to overflow
>>>> compiler thread stack and crash VM.
>>>>
>>>> The fix is to limit maximum inlining depth for @ForceInline method. By
>>>> default, MaxForceInlineLimit (==100) is much
>>>> larger than MaxInlineLevel(=9) for ordinary methods.
>>>>
>>>> Having it set to 100:
>>>>    - no crash observed
>>>>    - no inline bailouts due to MaxForceInlineLevel reached on octane
>>>> benchmarks
>>>>
>>>> John wrote the following:
>>>> "In stress tests, method handles can be arbitrarily deep. In typical
>>>> use cases they are not; they are adapted a fixed
>>>> number of times before being installed in a call site. It may be that
>>>> some of these extreme cases can be solved by
>>>> simply bailing out of the compile and continuing to interpret.
>>>>
>>>> A quick solution would be to have ForceInline fail after a larger
>>>> stack depth than normal.
>>>>
>>>> Since part of the problem is the ForceInline directive, we need to ask
>>>> whether they are all necessary. If we are
>>>> adjusting heuristics by ForceInline, some of them can be removed, if
>>>> we can detect the pattern from inside C2."
>>>>
>>>> Testing: failing test case, octane.
>>>>
>>>> PS: I tried another approach and experimented with late inlining
>>>> (delay residual call chain inlining). Though I
>>>> succeeded to delay inlining of MH::invokeBasic, I hit the same problem
>>>> there. Unless re-delay is supported during late
>>>> inlining, it's as easy to overflow compiler stack as when inlining
>>>> during parsing phase. Changing late inlining logic is
>>>> too much, IMO, for JDK8 at this phase.
>>>>
>>>> Thanks!
>>>>
>>>> Best regards,
>>>> Vladimir Ivanov