JDK 9 RFR of 8150206: Demote LFMultiThreadCachingTest.java to tier 2 until resolution of JDK-8150014

[joe darcy]

Hi Martin,

On 2/18/2016 10:18 PM, Martin Buchholz wrote:
> It's easy enough to get the test to pass by delambdafication.
>
> But we may not want to fix the test because it's not clear whether
> it's a jdk lambda bug or not, and we don't sweep bugs under the rug.
>
> Maybe instead of a "tier2" bucket we really want a "test found a bug
> that's not yet fixed" bucket?

Well, we have a bug for the test failure in JDK-8150014 and now this bug 
to demote the test from tier 1 to tier 2. The planned snapshot for the 
next promoted build is next Monday. I don't think we should knowingly 
let an intermittent tier 1 test failure escape to a promoted build.

Alternatives to achieve that goal include:

1) Demoting the test to tier 2 temporarily (If the cause of the problem 
is known, this is less helpful than if more data needed to be gathered.)

2) Putting the test on the problem list so it doesn't get run at all.

3) Changing the test so that is passes (delambdafication as you've 
suggested).

4) Changing / reverting the other libs code which triggered this failure.

The closest approach to "test found a bug that's not yet fixed" is 
putting the test on the problem list, option 2). I'm happy to do either 
1) or 2) under JDK-8150206 as an investigation into 3) or 4) proceeds 
over a longer period of time. I just don't think we should leave the 
test in its current flaky state for the next promotion.

Thanks,

-Joe

>
> On Thu, Feb 18, 2016 at 4:48 PM, joe darcy <joe.darcy at oracle.com> wrote:
>> Hello,
>>
>> Right now, we are seeing some cross-platform intermittent failures of
>>
>>      java/lang/invoke/LFCaching/LFMultiThreadCachingTest.java
>>
>> as noted in bug JDK-8150014. Until that test instability is resolved, the
>> test should be demoted from tier 1 to tier 2.
>>
>> Please see the patch below.
>>
>> (Circumstantial evidence points to the changes in JDK-8145485 being
>> involved.)
>>
>> -Joe
>>
>> diff -r 3973fe856db2 test/TEST.groups
>> --- a/test/TEST.groups    Wed Feb 17 12:47:35 2016 -0800
>> +++ b/test/TEST.groups    Thu Feb 18 16:44:40 2016 -0800
>> @@ -29,6 +29,7 @@
>>       :jdk_lang \
>>       -java/lang/ProcessHandle/TreeTest.java \
>>       -java/util/zip/TestLocalTime.java \
>> +    -java/lang/invoke/LFCaching/LFMultiThreadCachingTest.java \
>>       :jdk_util \
>>       -java/util/WeakHashMap/GCDuringIteration.java \
>>       -java/util/concurrent/ThreadPoolExecutor/ConfigChanges.java \
>> @@ -39,6 +40,7 @@
>>       :jdk_math
>>
>>   tier2 = \
>> +    java/lang/invoke/LFCaching/LFMultiThreadCachingTest.java \
>>       java/lang/ProcessHandle/TreeTest.java \
>>       java/util/zip/TestLocalTime.java \
>>       java/util/WeakHashMap/GCDuringIteration.java \
>>