RFR 9 test-only RFR 8177328 : java/lang/ClassLoader/securityManager/ClassLoaderTest.java times out with -Xcomp

[Mandy Chung]

Looks like this test execs a new VM for 66 times to exhaust the combination of with and without security manager, with a valid or malformed policy, client & custom loader in unnamed, named, automatic module.  I think we could take out the automatic module case as named module would verify it.  One refactor you can consider  by separating them in several @run actions.  The timeout will apply to each action but that does not help shorten the test execution time.

Mandy

> On May 1, 2017, at 11:59 AM, Brent Christian <brent.christian at oracle.com> wrote:
> 
> Hi, Joe
> 
> The typical run time of this test is ~18s (on my modestly-equipped laptop).  So the test does run within a reasonable amount of time, IMO - under normal circumstances, anyway.
> 
> The increased timeout is to cover the less seldom test run configurations involving Xcomp.  Judging by when the test was added and the dates of the failed runs, I think this test has timed out since it was introduced, and there has not been an Xcomp regression.
> 
> The test checks for a recursive initialization issue (8168075), and so needs to launch JVMs using various combinations of system classloader/security manager+policy/various module types.  For this reason I think Xcomp hits this test particularly hard, recompiling the startup code on every VM launch.  I think we want to maintain thorough test coverage here, though.
> 
> Looking more into the failures, the worst case looks to be getting not quite halfway through the test.  We should be able to get away with a more modest increase to the timeout (600, instead of 1200), and still have the test pass, if you would prefer.
> 
> Thanks,
> -Brent
> 
> On 4/27/17 2:25 PM, joe darcy wrote:
>> I understand the interest in having test pass reliably, but I don't
>> think giving the test very large timeouts is the preferred way of
>> accomplishing that.
>> 
>> For all configurations, the test can now run for up to 20 minutes, up
>> from 4 minutes. We want to run the entire test suite, thousands of
>> tests, in about 20 minutes. The the timeout factor used for Xcomp run,
>> the test would probably now be able to run for over an hour before
>> timing out.
>> 
>> I suggest making the test run faster, or seeing if there has been a
>> regressions in Xcomp to make test perform more poorly there.
>> 
>> Thanks,
>> 
>> -Joe
>> 
>> 
>> On 4/27/2017 12:08 PM, Brent Christian wrote:
>>> Hi,
>>> 
>>> This test times out under our automated testing configurations that
>>> include -Xcomp.
>>> 
>>> Please review my change to increase the timeout for this test.  It is
>>> sufficient for the test configurations in question to pass on two
>>> different local machines (Mac & Linux).
>>> 
>>> diff -r 7c04ab31b4d6
>>> test/java/lang/ClassLoader/securityManager/ClassLoaderTest.java
>>> --- a/test/java/lang/ClassLoader/securityManager/ClassLoaderTest.java
>>> Wed Apr 26 09:37:23 2017 -0700
>>> +++ b/test/java/lang/ClassLoader/securityManager/ClassLoaderTest.java
>>> Thu Apr 27 12:03:33 2017 -0700
>>> @@ -29,7 +29,7 @@
>>>  * @library /lib/testlibrary
>>>  * @modules java.base/jdk.internal.module
>>>  * @build JarUtils CompilerUtils
>>> - * @run main/timeout=240 ClassLoaderTest
>>> + * @run main/timeout=1200 ClassLoaderTest
>>>  */
>>> 
>>> Thanks,
>>> -Brent
>>