<html>
<head>
<meta content="text/html; charset=utf-8" http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
On 11/28/2016 3:30 PM, Martin Buchholz wrote:<br>
<blockquote
cite="mid:CA+kOe0_6WaKaAkCYMpD6KwOp2XHGYF7_hYy2xeEa9OoGE5LEag@mail.gmail.com"
type="cite">
<div dir="ltr"><br>
<div class="gmail_extra"><br>
<div class="gmail_quote">On Mon, Nov 28, 2016 at 2:07 PM,
Joseph D. Darcy <span dir="ltr"><<a
moz-do-not-send="true"
href="mailto:joe.darcy@oracle.com" target="_blank">joe.darcy@oracle.com</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
<div>
<div class="h5"><br>
</div>
</div>
For the combined dev/master forest, the most recent
integration tag will have the same stability guarantees we
have today so "pull the most recent jdk-10+XYZ tag" to get
a stable snapshot.<br>
<br>
</blockquote>
<div><br>
</div>
<div>But ... I want better than the stability guarantees we
have today! <br>
</div>
</div>
</div>
</div>
</blockquote>
<br>
Over the course of JDK 9 we've made marked improvements to the
regression test stability, at least on metrics we track internally
on internal systems, so unfortunately am I not at liberty to share
them. Briefly, the tier 1 regression tests in promoted builds are
very stable, the tier 2 tests somewhat less stable, and so on.<br>
<br>
<blockquote
cite="mid:CA+kOe0_6WaKaAkCYMpD6KwOp2XHGYF7_hYy2xeEa9OoGE5LEag@mail.gmail.com"
type="cite">
<div dir="ltr">
<div class="gmail_extra">
<div class="gmail_quote">
<div><br>
</div>
<div>I want to obsolete those messages to quality-discuss
with "new failures" because it should not even be possible
for master to get into a state where regression tests have
regressed (except for flakes or external dependencies).</div>
</div>
</div>
</div>
</blockquote>
<br>
I don't know if those particular failures are actually in the
promoted build or just an artifact of how that particular batch of
tests was run. IIRC, we did not see a corresponding set of failures
before the integration.<br>
<br>
<blockquote
cite="mid:CA+kOe0_6WaKaAkCYMpD6KwOp2XHGYF7_hYy2xeEa9OoGE5LEag@mail.gmail.com"
type="cite">
<div dir="ltr">
<div class="gmail_extra">
<div class="gmail_quote">
<div> </div>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
As an aside, for JDK 10 I'd also like to see promoted
builds on a more frequent schedule than once a week.<span
class=""><br>
</span></blockquote>
<div><br>
</div>
<div>People do "continuous testing and integration" these
days. Set up your integration pipeline so that it is
always running. The pipeline automatically promotes
changesets to master when all tests pass. Easy! Then
every master changeset is equally stable to a "promoted
build".</div>
</div>
</div>
</div>
</blockquote>
<br>
One of the internal systems mentioned above is a CI system that run
regression tests after a change is pushed. The approximate
integration model is then to promote known-good states of sources as
vetted by the CI system.<br>
<br>
If problems are fixed soon after they are identified, then a
post-push system gives good results with avoiding the need more
complexity on the front end to manage a series of in-flight patches.<br>
<br>
Cheers,<br>
<br>
-Joe<br>
</body>
</html>