RFR(T): 8247495: ProblemList vmTestbase/nsk/jvmti/SetFieldAccessWatch/setfldw001/TestDescription.java

serguei.spitsyn at oracle.com serguei.spitsyn at oracle.com
Sat Jun 13 00:28:24 UTC 2020 

On 6/12/20 17:22, Daniel D. Daugherty wrote:
> Hi Serguei,
>
> Thanks for reviewing! I pushed the changeset just before I took a dinner
> break

Great!

> so I won't be able to list you as a reviewer.

Not a big deal. :)

Thanks,
Serguei

>
> Dan
>
>
> On 6/12/20 6:40 PM, serguei.spitsyn at oracle.com wrote:
>> Hi Dan and Chris,
>>
>> Problem-listing it for Xcomp only looks right to me.
>> Thank you for taking care about it!
>>
>> Thanks,
>> Serguei
>>
>>
>> On 6/12/20 14:20, Chris Plummer wrote:
>>> On 6/12/20 1:59 PM, Daniel D. Daugherty wrote:
>>>> On 6/12/20 4:48 PM, Chris Plummer wrote:
>>>>> On 6/12/20 12:13 PM, Daniel D. Daugherty wrote:
>>>>>> On 6/12/20 2:58 PM, Chris Plummer wrote:
>>>>>>> On 6/12/20 11:52 AM, Daniel D. Daugherty wrote:
>>>>>>>> On 6/12/20 2:49 PM, Chris Plummer wrote:
>>>>>>>>> Hi Dan,
>>>>>>>>>
>>>>>>>>> What's the criteria for "noise".
>>>>>>>>
>>>>>>>> There is no specific criteria that I'm aware of.
>>>>>>>>
>>>>>>>> It popped up in today's JDK15 testing so it got on my radar 
>>>>>>>> (again).
>>>>>>>>
>>>>>>>>
>>>>>>>>> I don't consider the failures for this test as noisy. I only 
>>>>>>>>> see 3 in mach5 CI testing for all of JDK 15. JDK 14 does 
>>>>>>>>> appear to have been somewhat noisy, possibly enough so that it 
>>>>>>>>> looks like maybe something changed to reduce the number of 
>>>>>>>>> failures in 15. In any case, do you plan on backporting to 14?
>>>>>>>>
>>>>>>>> This failure has been around in one form or another since JDK7. 
>>>>>>>> If someone
>>>>>>>> decides to fix it, then they can un-ProblemList it.
>>>>>>>>
>>>>>>>> I'm planning to push it to JDK15 and JDK16. Those two releases 
>>>>>>>> are the focus
>>>>>>>> of my CI noise reduction efforts. I don't monitor the JDK14u CI...
>>>>>>>>
>>>>>>>> May I proceed with the ProblemListing?
>>>>>>> I just don't feel if we problem list tests with this failure 
>>>>>>> rate that in the long run it is a productive or good thing to 
>>>>>>> do. 3 failures during an entire 6 month CI test cycle seems 
>>>>>>> rather low to me. I'd like to get opinions from others.
>>>>>>
>>>>>> It's not just the failure rate. It's the fact that this bug has 
>>>>>> sat for
>>>>>> years without being fixed. I have tracked this bug for a very 
>>>>>> long time
>>>>>> since I'm the guy that filed both bugs.
>>>>>>
>>>>>> Mach5 is showing 54 sightings of 8205957 and here's the linking
>>>>>> distribution:
>>>>>>
>>>>>> $ sort /tmp/fred | uniq -c | sort -rn
>>>>>>   20 daniel.daugherty at oracle.com
>>>>>>   10 rahul.v.raghavan at oracle.com
>>>>>>    7 martin.thompson at oracle.com
>>>>>>    4 leonid.mesnik at oracle.com
>>>>>>    3 jesper.wilhelmsson at oracle.com
>>>>>>    3 chris.plummer at oracle.com
>>>>>>    2 mikael.vidstedt at oracle.com
>>>>>>    1 tobias.hartmann at oracle.com
>>>>>>    1 sangheon.kim at oracle.com
>>>>>>    1 kim.barrett at oracle.com
>>>>>>    1 daniil.x.titov at oracle.com
>>>>>>    1 calvin.cheung at oracle.com
>>>>>>
>>>>>> As you can see, I've observed and linked this bug a lot.
>>>>>> I'm tired of it.
>>>>> I still think that what is most relevant is how often it 
>>>>> reproduces with CI with the current release, and for that the # is 
>>>>> 3 times in 6 months. In our current test history it's failed 6 out 
>>>>> of 21390 runs, so you are disabling a test that passes 99.97% of 
>>>>> the time. My concern is that if a bug is introduced that makes it 
>>>>> start failing every run, or at least very frequently, it will be 
>>>>> missed. We need to carefully weigh the annoyance of failure noise 
>>>>> with the importance of test coverage. I don't think the balance is 
>>>>> right for this test to justify problem listing it.
>>>>>
>>>>> What you might want to consider is disabling it in the mode where 
>>>>> it seems to be failing. The failures all seem to be with -Xcomp. 
>>>>> Maybe you should just problem list it in ProblemList-Xcomp.txt.
>>>>
>>>> I didn't notice that this is an -Xcomp only failure. I was able to 
>>>> verify
>>>> that fact for 46 of the 54 sightings. For the 8 oldest sightings, 
>>>> the task
>>>> name has been lost to the dustbin of time so I can't confirm those.
>>>>
>>>> I can move the entry from test/hotspot/jtreg/ProblemList.txt to
>>>> test/hotspot/jtreg/ProblemList-Xcomp.txt.
>>>>
>>>> Here's the context diff:
>>>>
>>>> $ hg diff
>>>> diff -r 015533451f4c test/hotspot/jtreg/ProblemList-Xcomp.txt
>>>> --- a/test/hotspot/jtreg/ProblemList-Xcomp.txt    Fri Jun 12 
>>>> 09:31:08 2020 -0700
>>>> +++ b/test/hotspot/jtreg/ProblemList-Xcomp.txt    Fri Jun 12 
>>>> 16:58:18 2020 -0400
>>>> @@ -27,3 +27,4 @@
>>>>  #
>>>>  ############################################################################# 
>>>>
>>>>
>>>> +vmTestbase/nsk/jvmti/SetFieldAccessWatch/setfldw001/TestDescription.java 
>>>> 8205957 generic-all
>>>>
>>>>
>>>> Is this acceptable to you?
>>> Yes, that works for me.
>>>
>>> Chris
>>>>
>>>> Dan
>>>>
>>>>
>>>>
>>>>>
>>>>> Chris
>>>>>>
>>>>>> Dan
>>>>>>
>>>>>>
>>>>>>>
>>>>>>> Chris
>>>>>>>>
>>>>>>>> Dan
>>>>>>>>
>>>>>>>>>
>>>>>>>>> thanks,
>>>>>>>>>
>>>>>>>>> Chris
>>>>>>>>>
>>>>>>>>> On 6/12/20 9:46 AM, Daniel D. Daugherty wrote:
>>>>>>>>>> Greetings,
>>>>>>>>>>
>>>>>>>>>> It's time to reduce the noise in the CI so I'm ProblemListing 
>>>>>>>>>> tests.
>>>>>>>>>>
>>>>>>>>>> Here's the bug for failure:
>>>>>>>>>>
>>>>>>>>>>     JDK-8205957 setfldw001/TestDescription.java fails with 
>>>>>>>>>> bad field value
>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8205957
>>>>>>>>>>
>>>>>>>>>> and here's the bug for the ProblemListing:
>>>>>>>>>>
>>>>>>>>>>     JDK-8247495 ProblemList 
>>>>>>>>>> vmTestbase/nsk/jvmti/SetFieldAccessWatch/setfldw001/TestDescription.java 
>>>>>>>>>>
>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8247495
>>>>>>>>>>
>>>>>>>>>> I'm considering this a trivial change so I need a single 
>>>>>>>>>> (R)eviewer.
>>>>>>>>>>
>>>>>>>>>> Here's the context diff for the change:
>>>>>>>>>>
>>>>>>>>>> $ hg diff
>>>>>>>>>> diff -r 015533451f4c test/hotspot/jtreg/ProblemList.txt
>>>>>>>>>> --- a/test/hotspot/jtreg/ProblemList.txt    Fri Jun 12 
>>>>>>>>>> 09:31:08 2020 -0700
>>>>>>>>>> +++ b/test/hotspot/jtreg/ProblemList.txt    Fri Jun 12 
>>>>>>>>>> 12:40:17 2020 -0400
>>>>>>>>>> @@ -141,6 +141,7 @@
>>>>>>>>>>  vmTestbase/nsk/jvmti/scenarios/jni_interception/JI05/ji05t001/TestDescription.java 
>>>>>>>>>> 8219652 aix-ppc64
>>>>>>>>>>  vmTestbase/nsk/jvmti/scenarios/jni_interception/JI06/ji06t001/TestDescription.java 
>>>>>>>>>> 8219652 aix-ppc64
>>>>>>>>>>  vmTestbase/nsk/jvmti/SetJNIFunctionTable/setjniftab001/TestDescription.java 
>>>>>>>>>> 8219652 aix-ppc64
>>>>>>>>>> +vmTestbase/nsk/jvmti/SetFieldAccessWatch/setfldw001/TestDescription.java 
>>>>>>>>>> 8205957 generic-all
>>>>>>>>>>
>>>>>>>>>>  vmTestbase/gc/lock/jni/jnilock002/TestDescription.java 
>>>>>>>>>> 8208243,8192647 generic-all
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> This issue is actually much older than JDK-8205957 would 
>>>>>>>>>> indicate
>>>>>>>>>> (first sighting in JDK11 for that bug ID). The older version of
>>>>>>>>>> the test is covered by 
>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-6528079
>>>>>>>>>> and that failures first sighting is in JDK7.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Thanks, in advance, for any comments, questions, or suggestions.
>>>>>>>>>>
>>>>>>>>>> Dan
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>
>>>
>>>
>>
>

More information about the serviceability-dev mailing list