In(Sanity) Testing Mondays

[Stephen F Northover]

Hello committers,

The test matrix will be in place on Monday and we will continue with the 
sanity testing this Monday.  Not much code has changed over the holidays 
but any change can cause a regression.  See 
https://wiki.openjdk.java.net/display/OpenJFX/8u20 for the different 
tests and platforms.

I don't expect too much excitement this Monday, however, the following 
Monday January 13 is our first milestone and this should be a bit more 
interesting.

Steve

On 2013-12-19 6:56 PM, Lisa Selle wrote:
> Hi all,
>
> Steve and I have update the test matrix for the coming weeks.  Due to 
> US holidays, this matrix will stay in place until next year. For the 
> desktop team, we aimed to have two people cover each part of ensemble 
> on various platforms, for embedded (since we have fewer people), one 
> person testing each part of ensemble.  For both teams we had 
> toys-subset and toys-controls (the two main groups of tests) tested by 
> 2 people each.
>
> Vacations are not taken into account, so it's possible some test 
> coverage will be missed each week but the overall coverage should be 
> reasonable, especially given the slower rate of change that's likely 
> due to the holidays.
>
> Thanks,
>
> Lisa and Steve
>
> On 12/18/2013 2:32 PM, Stephen F Northover wrote:
>> Hi all,
>>
>> So the sanity testing was a success.  I would not say that the 
>> universe has been fundamentally altered because of it but the idea 
>> that there is a weekly formal time when we kick the tires of FX is a 
>> good one.  There were some problems getting the correct bits to test 
>> but that will be resolved for next week. We will be doing this next 
>> week and for the foreseeable future as it was a good forcing function 
>> for people to get their environments up to date and make sure that 
>> things were generally working.
>>
>> The big winner as a test case was HelloSanity.  Although Ensemble 
>> exercises much of FX, some committers found the UI to be somewhat 
>> cumbersome for testing.  There is so much functionality in Ensemble, 
>> that a comprehensive test just takes too long.  Ensemble is still 
>> very important, but perhaps not the best sanity vehicle. I'm thinking 
>> that in future, we will look towards testing the FX toys that have 
>> recently been open sourced instead of having so many committers 
>> poking Ensemble.  Further, we need to add some more 3D testing to the 
>> list of apps we run (3DViewer).
>>
>> In order to avoid entering duplicates, on the day of the testing, we 
>> won't process outstanding issues so that testers can quickly scan the 
>> "One True Dashboard".
>>
>> https://javafx-jira.kenai.com/secure/Dashboard.jspa?selectPageId=11893
>>
>> So we will be sanity testing next Monday the 23rd and after that 
>> refining the process as we go.  As we approach our first 8u20 
>> milestone, the testing effort will ramp up but we can discuss 
>> specifics closer to the date.
>>
>> https://wiki.openjdk.java.net/display/OpenJFX/8u20
>>
>> Steve
>