Need reviewers - jdk testing changes 6888927

[Jonathan Gibbons]

Kelly O'Hair wrote:
>
>
> Jonathan Gibbons wrote:
>> Kelly,
>>
>> Wow, you've been busy.
>>
>> The thing that stood out to me was the use of SLASH_JAVA which is 
>> something of a Sun legacy which doesn't apply to folk outside Sun.  
>> And, looking at the file as a whole, it's clear there are many 
>> different sections, not all of which apply to the OpenJDK 
>> community.   Would it not help to split this file up into sections 
>> for the various test suites, or at least into a couple of files like 
>> jtreg.gmk and other.gmk so that people outside Sun have a better 
>> chance of figuring out this file.
>
> I took a stab at creating *.gmk files and did not like the result.
> Separating this stuff into separate make include files just makes it
> harder to maintain in my opinion. Completely separate Makefile's just 
> seemed
> like more work than it was worth. The sections on jck and vmsqe are 
> isolated
> as best I can in the one file, I didn't think that was so bad.
>
> One big Makefile has some big advantages, for now, I'd like to leave 
> it that way.
>
> -kto

OK. Thanks for trying.

-- Jon

>
>>
>> -- Jon
>>
>>
>> Kelly O'Hair wrote:
>>>
>>> I need some formal reviewers on these jdk/test/Makefile changes.
>>>
>>> More polish and changes may need to be done later, but there
>>> is value in what I have now, and I need lots of help to improve
>>> things (and fix some of our testcases).
>>>
>>> Here is the bugid and webrev:
>>>
>>> 6888927: Fix jdk jtreg tests to indicate which ones need othervm, 
>>> allow for use of samevm option
>>> http://cr.openjdk.java.net/~ohair/openjdk7/jdk7-build-samevm-6888927/webrev/ 
>>>
>>>
>>> The goal was to see if we could easily run most of the regression
>>> tests in the jdk/test directory, in minimum time, and without so
>>> much noise (e.g. do not run unstable or problematic tests).
>>> This could then be used as a benchmark to validate some quality measure
>>> of the jdk7 that was built.
>>>
>>> The primary changes include:
>>>   * New jdk/test/Makefile targets: jdk_all jdk_lang jdk_util jdk_nio 
>>> ...
>>>   * Addition of a jdk/test/ProblemList file to be used by 
>>> jdk/test/Makefile
>>>
>>> I tried to group all the tests (by directory names) into jdk_* sets, 
>>> then
>>> tried to balance them as to how long each set ran and if they could 
>>> be run
>>> with the jtreg -samevm option. The balancing was tricky, and will 
>>> need more
>>> polish, along with the virtual frame buffer idea. I also had to give up
>>> on the awt, rmi, and swing tests until the Xvfb issues are figured out.
>>>
>>> I chose to not actually modify the tests themselves if they needed 
>>> fixing,
>>> it was just too overwhelming. So developers and teams may want to 
>>> browse the
>>> ProblemList for their favorite tests (which aren't my favorites ;^) and
>>> consider what they might want to do.
>>>
>>> With this new jdk/test/Makefile, anybody can:
>>>
>>>   cd jdk/test
>>>   gnumake [PRODUCT_HOME=${YOUR_JDK7_HOME}] 
>>> [JT_HOME=${YOUR_JTREG_HOME}] jdk_all
>>>
>>> And run about 3,000+ tests that SHOULD PASS in roughly 2 hours if your
>>> machine isn't too slow and old. I was able to run it in 90 minutes on a
>>> monster OpenSolaris AMD machine which had 16Gb RAM and 16 2.2Ghz cpus.
>>>
>>> Then I added "-j 4" to the gnumake command line and it only took 
>>> 30mins!
>>> So the various jdk_* make targets can be run in parallel.
>>>
>>> Please let me know what you think.
>>>
>>> -kto
>>>
>>>
>>