RFR (XS): 8028159: C2: compiler stack overflow during inlining of @ForceInline methods

[Vladimir Ivanov]

Are you sure? Last time we decided to make it develop (corresponding 
letter attached).

Best regards,
Vladimir Ivanov

On 11/13/13 10:09 PM, Vladimir Kozlov wrote:
> Make the flag product as other inline limitation flags.
>
> Thanks,
> Vladimir K
>
> On 11/13/13 6:22 AM, Vladimir Ivanov wrote:
>> http://cr.openjdk.java.net/~vlivanov/8028159/webrev.00/
>> https://bugs.openjdk.java.net/browse/JDK-8028159
>> 12 lines changed: 7 ins; 3 del; 2 mod
>>
>> There's a corner case when C2 inliner fails to delay inlining of
>> @ForceInline method (e.g. nested
>> MethodHandle::asType(MethodType) calls produce such pattern):
>>     @ForceInline m1 -> MH::invokeBasic -> @FI m2 -> MH::invokeBasic ->
>> @FI m3 -> ...
>>
>> Having method handle chain deep enough, it's possible to overflow
>> compiler thread stack and crash VM.
>>
>> The fix is to limit maximum inlining depth for @ForceInline method. By
>> default, MaxForceInlineLimit (==100) is much
>> larger than MaxInlineLevel(=9) for ordinary methods.
>>
>> Having it set to 100:
>>    - no crash observed
>>    - no inline bailouts due to MaxForceInlineLevel reached on octane
>> benchmarks
>>
>> John wrote the following:
>> "In stress tests, method handles can be arbitrarily deep. In typical
>> use cases they are not; they are adapted a fixed
>> number of times before being installed in a call site. It may be that
>> some of these extreme cases can be solved by
>> simply bailing out of the compile and continuing to interpret.
>>
>> A quick solution would be to have ForceInline fail after a larger
>> stack depth than normal.
>>
>> Since part of the problem is the ForceInline directive, we need to ask
>> whether they are all necessary. If we are
>> adjusting heuristics by ForceInline, some of them can be removed, if
>> we can detect the pattern from inside C2."
>>
>> Testing: failing test case, octane.
>>
>> PS: I tried another approach and experimented with late inlining
>> (delay residual call chain inlining). Though I
>> succeeded to delay inlining of MH::invokeBasic, I hit the same problem
>> there. Unless re-delay is supported during late
>> inlining, it's as easy to overflow compiler stack as when inlining
>> during parsing phase. Changing late inlining logic is
>> too much, IMO, for JDK8 at this phase.
>>
>> Thanks!
>>
>> Best regards,
>> Vladimir Ivanov
-------------- next part --------------
An embedded message was scrubbed...
From: Christian Thalinger <christian.thalinger at oracle.com>
Subject: Re: RFR (XS): 8012941: JSR 292: too deep inlining might crash
	compiler because of stack overflow
Date: Mon, 14 Oct 2013 14:12:17 -0700
Size: 4866
Url: http://mail.openjdk.java.net/pipermail/hotspot-compiler-dev/attachments/20131113/bdcbd853/AttachedMessage.nws