RFR (XXS) 8066292: Test java/nio/file/Files/CheckPermissions.java triggers stack overflow with -server -Xcomp

[Vladimir Kozlov]

On 2/23/16 5:59 PM, Dean Long wrote:
> On 2/23/2016 4:28 PM, Dean Long wrote:
>> On 2/23/2016 2:03 PM, Vladimir Kozlov wrote:
>>> On 2/22/16 2:31 PM, Dean Long wrote:
>>>> https://bugs.openjdk.java.net/browse/JDK-8066292
>>>>
>>>> http://cr.openjdk.java.net/~dlong/8066292/webrev/
>>>>
>>>> With -Xcomp, a cycle between compile requests and class loading can cause infinite recursion resulting in
>>>> StackOverflowError's that are silently ignored.  To break the cycle, this changeset removes the ancient call to
>>>> Method::load_signature_classes() for C2.  This call is no longer needed.
>>>
>>> "This call is no longer needed" because ...? Why it is not needed? C2 can't compile methods if these classes are not
>>> loaded.
>>>
>>
>> That was my understanding from internal discussions.  Chris or John, can you chime in here?
>>
>> I know that C2 creates placeholder classes if a class isn't loaded. Can you point me to where we fail the compilation
>> is signature classes are not loaded?
>>
>> dl
>>
>
> Hi Vladimir, I took another look, and I think you must have been thinking about inlining.  Yes, we won't inline a callee
> if its signature classes aren't loaded.  But if we really wanted to maximum inlining for -Xcomp mode, (which some people
> think should just go away), wouldn't it be better to execute the method in the interpreter a few times, so that more of
> the callee tree would have its classes loaded?

No, I am thinking about what happens when we start compiling a method which signature has unloaded klass.
 From what I see CI will created "unloaded" ciKlass in such case. It does not have any information except name.
And it looks like ciTypeFlow will treat it as NULL in such case (see ciTypeFlow::StateVector::do_null_assert()).
As result C2 may generate uncommon trap on first access to such parameter.

May be it is okay since -Xcomp usually produces uncommon traps a lot.

I could miss other places in C2 which handle such case.

Thanks,
Vladimir

>
> dl
>
>>> Thanks,
>>> Vladimir
>>>
>>>>
>>>> Tested in RBT with --test
>>>> vm.compiler.testlist,hotspot/test/:hotspot_compiler_closed,hotspot/test/:hotspot_compiler_1,hotspot/test/:hotspot_compiler_2,hotspot/test/:hotspot_compiler_3
>>>>
>>>>
>>>>
>>>> dl
>>
>