RFR(XS): 8212220 add code to verify results to metaspace/stressDictionary/StressDictionary.java

[David Holmes]

Hi Dan,

This seems very reasonable.

Thanks,
David

On 24/10/2018 5:20 AM, Daniel D. Daugherty wrote:
> On 10/23/18 11:57 AM, Daniel D. Daugherty wrote:
>> Greetings,
>>
>> The following test has been timing out intermittently in the
>> jdk/jdk CI lately:
>>
>>     vmTestbase/metaspace/stressDictionary/StressDictionary.java
>>
>> Those failures are being tracked by the following bug:
>>
>>     JDK-8211211 
>> vmTestbase/metaspace/stressDictionary/StressDictionary.java timeout
>>     https://bugs.openjdk.java.net/browse/JDK-8211211
>>
>> While investigating the timeouts, I noticed that the test is not doing
>> explicit verification of the results from the ExecutorService so I have
>> added code to do that.
>>
>> With the fix for JDK-8212028 in place, the test is timing out much more
>> frequently in the jdk/jdk CI; the total timeout for the test went from
>> 20 minutes down to 8 minutes. I have changed the default timeout value
>> for the test to 5 minutes (from 2 minutes) and with the new default
>> TIMEOUT_FACTOR of 4 (was 10), the total timeout for the test will be
>> restored to 20 minutes.
>>
>> The total timeout changes will allow the new results verification code
>> to be used in the infrequent 20 minute timeout cases that we originally
>> saw when JDK-8211211 was filed.
>>
>> Here's the URL for the test changes:
>>
>> http://cr.openjdk.java.net/~dcubed/8212220-webrev/0-for-jdk-jdk/
>>
>> Here's the sub-task bug ID for the test changes:
>>
>>     JDK-8212220 add code to verify results to 
>> metaspace/stressDictionary/StressDictionary.java
>>     https://bugs.openjdk.java.net/browse/JDK-8212220
>>
>> I'm currently running the revised test thru the Mach5 system:
>>
>>     builds-tier1,jdk-tier1,jdk-tier2,hs-tier1,hs-tier2,hs-tier3
>>
>> It's a bit of overkill, but I don't know how much of the timeout
>> is being caused by test system load so I'm trying to replicate
>> what is being done by the jdk/jdk CI system.
>>
>> Thanks, in advance, for any comments, questions or suggestions.
>>
>> Dan
> 
> Made one minor tweak to the new code:
> 
> $ diff 8211211.diff.debug.txt.01 8211211.diff.debug.txt.03
> 20c20
> < +            System.out.println("INFO: There are " + act_results + " 
> results.");
> ---
>  > +            System.err.println("INFO: There are " + act_results + " 
> results.");
> 41c41
> < +            System.out.println("INFO: no tasks were cancelled.");
> ---
>  > +            System.err.println("INFO: no tasks were cancelled.");
> 44c44
> < +            System.out.println("INFO: all tasks are done.");
> ---
>  > +            System.err.println("INFO: all tasks are done.");
> 
> During testing, the new "INFO" lines were getting lost in JTREG
> output truncation for System.out so I've moved the new INFO lines
> to System.err.
> 
> Dan
>