How to test OpenJDK?

[Kelly O'Hair]

Hear Hear!  You got my vote. ;^)

-kto

Jonathan Gibbons wrote:
> There is no reason why it should not be possible to have 100% pass rate
> for most of the jdk tests as well.   If there are problematic tests in 
> the jdk
> test suite, we should identify those tests, and then logically partition 
> the
> tests into two groups -- those that should always be expected to pass, and
> those that don't, for whatever reason. That at least would allow to to set
> a quality bar for any OpenJDK build of which tests should always pass.
> 
> Frankly, I've never understood the attitude that it is OK to have tests 
> that
> fail, and which don't deserve attention, but as long as we have such tests,
> we should identify them, using any of the available jtreg mechanisms,
> such as keywords, @ignore, or exclude lists.
> 
> And, all the other, unmarked tests, should always pass, all the time, and
> if a test starts failing after you've changed code, you should assume 
> you've
> broken something and it deserves attention.
> 
> If necessary, we could/should have separate Makefile rules for testing 
> these
> two different groups of tests, so that we have an obvious 100% pass rate on
> the tests in the main group of good tests.
> 
> -- Jon
> 
> 
> On May 31, 2009, at 2:25 PM, Kelly O'Hair wrote:
> 
>> Cudos to Jon for having a testbase that is 100% pass, that IS the way 
>> to go. ;^)
>>
>> So I think that means you can
>>
>>   cd langtools/test
>>
>> and either run jtreg, or use the Makefile
>>
>>   make PRODUCT_HOME=your_j2sdk-image-dir JT_HOME=jtreg_home TESTDIRS=.
>>
>> How you run the jtreg tests is up to you.
>>
>> See http://openjdk.java.net/jtreg/faq.html
>>
>> -kto
>>
>> Jonathan Gibbons wrote:
>>> There are ~1500 tests in langtools/test, and all should pass.
>>> -- Jon
>>> On May 31, 2009, at 11:39 AM, Kelly O'Hair wrote:
>>>> See  http://openjdk.java.net/jtreg/
>>>>
>>>>
>>>> I tend to use the jdk/test/Makefile myself:
>>>>
>>>>  cd jdk/test && make PRODUCT_HOME=your_j2sdk-image-dir 
>>>> JT_HOME=jtreg_home TESTDIRS=java/lang
>>>>
>>>> There are quite a few tests in the jdk/test directory and the above
>>>> only runs the java/lang tests, which should all pass.
>>>>
>>>> As to which tests reliably pass and which fail, I can't tell you.
>>>> Maybe someone else can refer to something that documents what tests
>>>> are expected to pass in their area.
>>>>
>>>> -kto
>>>>
>>>> lianshun Liu wrote:
>>>>> Hi,
>>>>> When I've finished building OpenJDK, how can I make sure that it 
>>>>> will work well? Is there any test suites that can be used to test 
>>>>> the JVM?
>>>>> Thanks :)
>>>>> Lianshun Liu
>