Repository? -- How many lines of development?

[Joseph D. Darcy]

On 11/28/2016 3:30 PM, Martin Buchholz wrote:
>
>
> On Mon, Nov 28, 2016 at 2:07 PM, Joseph D. Darcy <joe.darcy at oracle.com 
> <mailto:joe.darcy at oracle.com>> wrote:
>
>
>     For the combined dev/master forest, the most recent integration
>     tag will have the same stability guarantees we have today so "pull
>     the most recent jdk-10+XYZ tag" to get a stable snapshot.
>
>
> But ... I want better than the stability guarantees we have today!

Over the course of JDK 9 we've made marked improvements to the 
regression test stability, at least on metrics we track internally on 
internal systems, so unfortunately am I not at liberty to share them. 
Briefly, the tier 1 regression tests in promoted builds are very stable, 
the tier 2 tests somewhat less stable, and so on.

>
> I want to obsolete those messages to quality-discuss with "new 
> failures" because it should not even be possible for master to get 
> into a state where regression tests have regressed (except for flakes 
> or external dependencies).

I don't know if those particular failures are actually in the promoted 
build or just an artifact of how that particular batch of tests was run. 
IIRC, we did not see a corresponding set of failures before the integration.

>     As an aside, for JDK 10 I'd also like to see promoted builds on a
>     more frequent schedule than once a week.
>
>
> People do "continuous testing and integration" these days.  Set up 
> your integration pipeline so that it is always running.  The pipeline 
> automatically promotes changesets to master when all tests pass.  
> Easy!  Then every master changeset is equally stable to a "promoted 
> build".

One of the internal systems mentioned above is a CI system that run 
regression tests after a change is pushed. The approximate integration 
model is then to promote known-good states of sources as vetted by the 
CI system.

If problems are fixed soon after they are identified, then a post-push 
system gives good results with avoiding the need more complexity on the 
front end to manage a series of in-flight patches.

Cheers,

-Joe
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.openjdk.java.net/pipermail/quality-discuss/attachments/20161128/bc261b68/attachment.html>