From mark.reinhold at oracle.com Tue May 24 13:53:42 2011 From: mark.reinhold at oracle.com (mark.reinhold at oracle.com) Date: Tue, 24 May 2011 13:53:42 -0700 Subject: Mercurial forests for JDK 8 Message-ID: <20110524205342.AF2C211E4@eggemoggin.niobe.net> I'm in the process of creating the master forest, which will be rooted at http://hg.openjdk.java.net/jdk8/jdk8. What other group/integration forests do we want to have? JDK 7 has 21 (!) separate forests. Only these have been updated in 2011: 2d awt build deploy hotspot hotspot-comp hotspot-gc hotspot-rt l10n swing tl The VM team would prefer to host their group and integration forests under the HSX Project since they deliver into multiple JDK release trains, so unless there are other suggestions I'll just create these: 2d awt build deploy l10n swing tl Comments? - Mark From brian.goetz at oracle.com Tue May 24 14:03:46 2011 From: brian.goetz at oracle.com (Brian Goetz) Date: Tue, 24 May 2011 17:03:46 -0400 Subject: Mercurial forests for JDK 8 In-Reply-To: <20110524205342.AF2C211E4@eggemoggin.niobe.net> References: <20110524205342.AF2C211E4@eggemoggin.niobe.net> Message-ID: <4DDC1D32.3090004@oracle.com> We've got some "disconnected" forests too, such as lambda/lambda. Our intention is not to pour code from there directly into tl (since our commits are not jcheck-compliant), but instead do a "curated merge" that is jcheck-compliant. However, it still would be useful to have these repositories rebased to 8, to smooth out the process. On 5/24/2011 4:53 PM, mark.reinhold at oracle.com wrote: > I'm in the process of creating the master forest, which will be rooted at > http://hg.openjdk.java.net/jdk8/jdk8. > > What other group/integration forests do we want to have? > > JDK 7 has 21 (!) separate forests. Only these have been updated in 2011: > > 2d > awt > build > deploy > hotspot > hotspot-comp > hotspot-gc > hotspot-rt > l10n > swing > tl > > The VM team would prefer to host their group and integration forests > under the HSX Project since they deliver into multiple JDK release > trains, so unless there are other suggestions I'll just create these: > > 2d > awt > build > deploy > l10n > swing > tl > > Comments? > > - Mark From mark.reinhold at oracle.com Tue May 24 14:19:40 2011 From: mark.reinhold at oracle.com (mark.reinhold at oracle.com) Date: Tue, 24 May 2011 14:19:40 -0700 Subject: Mercurial forests for JDK 8 In-Reply-To: brian.goetz@oracle.com; Tue, 24 May 2011 17:03:46 EDT; <4DDC1D32.3090004@oracle.com> Message-ID: <20110524211940.F381911E4@eggemoggin.niobe.net> 2011/5/24 14:03 -0700, brian.goetz at oracle.com: > We've got some "disconnected" forests too, such as lambda/lambda. Our > intention is not to pour code from there directly into tl (since our commits > are not jcheck-compliant), but instead do a "curated merge" that is > jcheck-compliant. Right. We'll be doing the same for Jigsaw. > However, it still would be useful to have these repositories > rebased to 8, to smooth out the process. That's up those who are maintaining these "disconnected" forests. - Mark From mike.duigou at oracle.com Tue May 24 14:28:38 2011 From: mike.duigou at oracle.com (Mike Duigou) Date: Tue, 24 May 2011 14:28:38 -0700 Subject: Mercurial forests for JDK 8 In-Reply-To: <20110524205342.AF2C211E4@eggemoggin.niobe.net> References: <20110524205342.AF2C211E4@eggemoggin.niobe.net> Message-ID: <76357345-18AE-4B24-AC82-8B2A556F977F@oracle.com> It would be nice to see the number of repos reduced even further. awt/swing/2d merged? perhaps this could even include deploy as "client". Also perhaps a build/l10n merge? The rate at which changes currently propagate to the master and then back to the sub-respos is too slow. There is admittedly a tension between the risks of premature integration and those of delayed bug surfacing. The current repo policies seem to entirely favour the later which isn't necessarily the best answer. The more time that passes between a bug being introduced and it being recognized and fixed, the greater the cost in fixing it. I believe that faster dissemination of changesets will help to reveal introduced bugs more quickly and bring down the time and cost for fixing them. Combining repos seems a part of achieving this. Mike On May 24 2011, at 13:53 , mark.reinhold at oracle.com wrote: > I'm in the process of creating the master forest, which will be rooted at > http://hg.openjdk.java.net/jdk8/jdk8. > > What other group/integration forests do we want to have? > > JDK 7 has 21 (!) separate forests. Only these have been updated in 2011: > > 2d > awt > build > deploy > hotspot > hotspot-comp > hotspot-gc > hotspot-rt > l10n > swing > tl > > The VM team would prefer to host their group and integration forests > under the HSX Project since they deliver into multiple JDK release > trains, so unless there are other suggestions I'll just create these: > > 2d > awt > build > deploy > l10n > swing > tl > > Comments? > > - Mark From philip.race at oracle.com Tue May 24 14:35:25 2011 From: philip.race at oracle.com (Phil Race) Date: Tue, 24 May 2011 14:35:25 -0700 Subject: Mercurial forests for JDK 8 In-Reply-To: <76357345-18AE-4B24-AC82-8B2A556F977F@oracle.com> References: <20110524205342.AF2C211E4@eggemoggin.niobe.net> <76357345-18AE-4B24-AC82-8B2A556F977F@oracle.com> Message-ID: <4DDC249D.6020501@oracle.com> On 5/24/2011 2:28 PM, Mike Duigou wrote: > It would be nice to see the number of repos reduced even further. awt/swing/2d merged? perhaps this could even include deploy as "client". Also perhaps a build/l10n merge? For specific forests, that is a question for those groups, not for members of other groups. We've considered this in the past and prefer them separate. Also build is unrelated to l10n. I also think there's a tenuous connection between merging these and getting fixes into master at a faster rate. -phil. > The rate at which changes currently propagate to the master and then back to the sub-respos is too slow. There is admittedly a tension between the risks of premature integration and those of delayed bug surfacing. The current repo policies seem to entirely favour the later which isn't necessarily the best answer. The more time that passes between a bug being introduced and it being recognized and fixed, the greater the cost in fixing it. I believe that faster dissemination of changesets will help to reveal introduced bugs more quickly and bring down the time and cost for fixing them. Combining repos seems a part of achieving this. > > Mike > > On May 24 2011, at 13:53 , mark.reinhold at oracle.com wrote: > >> I'm in the process of creating the master forest, which will be rooted at >> http://hg.openjdk.java.net/jdk8/jdk8. >> >> What other group/integration forests do we want to have? >> >> JDK 7 has 21 (!) separate forests. Only these have been updated in 2011: >> >> 2d >> awt >> build >> deploy >> hotspot >> hotspot-comp >> hotspot-gc >> hotspot-rt >> l10n >> swing >> tl >> >> The VM team would prefer to host their group and integration forests >> under the HSX Project since they deliver into multiple JDK release >> trains, so unless there are other suggestions I'll just create these: >> >> 2d >> awt >> build >> deploy >> l10n >> swing >> tl >> >> Comments? >> >> - Mark From ahughes at redhat.com Tue May 24 14:51:05 2011 From: ahughes at redhat.com (Andrew John Hughes) Date: Tue, 24 May 2011 22:51:05 +0100 Subject: Mercurial forests for JDK 8 In-Reply-To: <20110524205342.AF2C211E4@eggemoggin.niobe.net> References: <20110524205342.AF2C211E4@eggemoggin.niobe.net> Message-ID: <20110524215105.GB2642@shelob.middle-earth.co.uk> On Tue, May 24, 2011 at 01:53:42PM -0700, mark.reinhold at oracle.com wrote: > I'm in the process of creating the master forest, which will be rooted at > http://hg.openjdk.java.net/jdk8/jdk8. > > What other group/integration forests do we want to have? > > JDK 7 has 21 (!) separate forests. Only these have been updated in 2011: > > 2d > awt > build > deploy > hotspot > hotspot-comp > hotspot-gc > hotspot-rt > l10n > swing > tl > > The VM team would prefer to host their group and integration forests > under the HSX Project since they deliver into multiple JDK release > trains, so unless there are other suggestions I'll just create these: > > 2d > awt > build > deploy > l10n > swing > tl > > Comments? > > - Mark Can we have an icedtea/jdk8 please? Thanks, -- Andrew :) Free Java Software Engineer Red Hat, Inc. (http://www.redhat.com) Support Free Java! Contribute to GNU Classpath and IcedTea http://www.gnu.org/software/classpath http://icedtea.classpath.org PGP Key: F5862A37 (https://keys.indymedia.org/) Fingerprint = EA30 D855 D50F 90CD F54D 0698 0713 C3ED F586 2A37 From artem.ananiev at oracle.com Wed May 25 05:28:04 2011 From: artem.ananiev at oracle.com (Artem Ananiev) Date: Wed, 25 May 2011 16:28:04 +0400 Subject: Mercurial forests for JDK 8 In-Reply-To: <4DDC249D.6020501@oracle.com> References: <20110524205342.AF2C211E4@eggemoggin.niobe.net> <76357345-18AE-4B24-AC82-8B2A556F977F@oracle.com> <4DDC249D.6020501@oracle.com> Message-ID: <4DDCF5D4.1080707@oracle.com> On 5/25/2011 1:35 AM, Phil Race wrote: > On 5/24/2011 2:28 PM, Mike Duigou wrote: >> It would be nice to see the number of repos reduced even further. >> awt/swing/2d merged? perhaps this could even include deploy as >> "client". Also perhaps a build/l10n merge? > > For specific forests, that is a question for those groups, not for > members of other groups. > We've considered this in the past and prefer them separate. Also build > is unrelated to l10n. I see some sense to merge AWT and Swing workspaces. Many recent fixes span both toolkits, and it would also simplify the integration process. Thanks, Artem > I also think there's a tenuous connection between merging these and > getting fixes into > master at a faster rate. > > -phil. > >> The rate at which changes currently propagate to the master and then >> back to the sub-respos is too slow. There is admittedly a tension >> between the risks of premature integration and those of delayed bug >> surfacing. The current repo policies seem to entirely favour the later >> which isn't necessarily the best answer. The more time that passes >> between a bug being introduced and it being recognized and fixed, the >> greater the cost in fixing it. I believe that faster dissemination of >> changesets will help to reveal introduced bugs more quickly and bring >> down the time and cost for fixing them. Combining repos seems a part >> of achieving this. >> >> Mike >> >> On May 24 2011, at 13:53 , mark.reinhold at oracle.com wrote: >> >>> I'm in the process of creating the master forest, which will be >>> rooted at >>> http://hg.openjdk.java.net/jdk8/jdk8. >>> >>> What other group/integration forests do we want to have? >>> >>> JDK 7 has 21 (!) separate forests. Only these have been updated in 2011: >>> >>> 2d >>> awt >>> build >>> deploy >>> hotspot >>> hotspot-comp >>> hotspot-gc >>> hotspot-rt >>> l10n >>> swing >>> tl >>> >>> The VM team would prefer to host their group and integration forests >>> under the HSX Project since they deliver into multiple JDK release >>> trains, so unless there are other suggestions I'll just create these: >>> >>> 2d >>> awt >>> build >>> deploy >>> l10n >>> swing >>> tl >>> >>> Comments? >>> >>> - Mark > From mark.reinhold at oracle.com Wed May 25 12:29:52 2011 From: mark.reinhold at oracle.com (mark.reinhold at oracle.com) Date: Wed, 25 May 2011 12:29:52 -0700 Subject: JDK 8 Mercurial forests available Message-ID: <20110525192952.71B6A2DC4@eggemoggin.niobe.net> The following group/integration forests are now open for business: http://hg.openjdk.java.net/jdk8/2d http://hg.openjdk.java.net/jdk8/awt http://hg.openjdk.java.net/jdk8/build http://hg.openjdk.java.net/jdk8/deploy http://hg.openjdk.java.net/jdk8/l10n http://hg.openjdk.java.net/jdk8/swing http://hg.openjdk.java.net/jdk8/tl We'll post an initial build schedule shortly. Be careful out there, - Mark From mark.reinhold at oracle.com Wed May 25 20:53:06 2011 From: mark.reinhold at oracle.com (mark.reinhold at oracle.com) Date: Wed, 25 May 2011 20:53:06 -0700 Subject: Mercurial forests for JDK 8 In-Reply-To: ahughes@redhat.com; Tue, 24 May 2011 22:51:05 BST; <20110524215105.GB2642@shelob.middle-earth.co.uk> Message-ID: <20110526035306.CE4502DCB@eggemoggin.niobe.net> 2011/5/24 14:51 -0700, ahughes at redhat.com: > Can we have an icedtea/jdk8 please? Sure -- done. - Mark From kelly.ohair at oracle.com Mon May 30 10:09:16 2011 From: kelly.ohair at oracle.com (Kelly O'Hair) Date: Mon, 30 May 2011 10:09:16 -0700 Subject: Mercurial forests for JDK 8 In-Reply-To: <4DDC249D.6020501@oracle.com> References: <20110524205342.AF2C211E4@eggemoggin.niobe.net> <76357345-18AE-4B24-AC82-8B2A556F977F@oracle.com> <4DDC249D.6020501@oracle.com> Message-ID: On May 24, 2011, at 2:35 PM, Phil Race wrote: > I also think there's a tenuous connection between merging these and getting fixes into > master at a faster rate. > > -phil. I have to agree with Phil here. Although the number of team forests does need to be controlled, the primary issue with getting fixes into the master repos is the speed and timing of integrations. In my opinion, the current static and somewhat fixed schedule of integrations needs to be made more dynamic, allowing for an integration to happen in a more 'on demand' way. The time it takes to do an integration also needs to be addressed. Integration testing needs to be completely automated, and false negatives need to be minimized to speed up the analysis phase. The actual integration itself cannot be completely automated (merges or syncs with the master must always be watched carefully) but many of the large steps in the integration process can be automated. Ideally, we should be able to have all team areas ready for an integration on a 24hour basis, based on an inspection of acceptable results, a sync with master changes, a quick re-build and re-test, and a push. Just my opinion. -kto From georges.saab at oracle.com Mon May 30 11:28:26 2011 From: georges.saab at oracle.com (Georges Saab) Date: Mon, 30 May 2011 11:28:26 -0700 Subject: Mercurial forests for JDK 8 In-Reply-To: References: <20110524205342.AF2C211E4@eggemoggin.niobe.net> <76357345-18AE-4B24-AC82-8B2A556F977F@oracle.com> <4DDC249D.6020501@oracle.com> Message-ID: If I have understood correctly the present system of the 'static and somewhat fixed schedule of integrations' was introduced a number of years ago when there were lots of issues with destabilizing (ok, build breaking!) changes coming in, And that it has been successful in preventing these, albeit at quite a bit of cost. So what (other than momentum) is preventing us from having a more nimble integration capability? In my naive view, it looks to be having better testing (as in more effective -- less but more tailored to hit the issues in a particular integration repo). I like Kelly's suggestion of viewing it as a 'budget of time' -- i.e. we should figure out the best set of tests for a repo that can be run within 24 hours'. I think there is more that can be done here to make sure integration repos stay clean... On 30 maj 2011, at 10.09, Kelly O'Hair wrote: > > On May 24, 2011, at 2:35 PM, Phil Race wrote: > >> I also think there's a tenuous connection between merging these and getting fixes into >> master at a faster rate. >> >> -phil. > > I have to agree with Phil here. > > Although the number of team forests does need to be controlled, the primary issue with getting fixes > into the master repos is the speed and timing of integrations. > In my opinion, the current static and somewhat fixed schedule of integrations needs to be made more dynamic, > allowing for an integration to happen in a more 'on demand' way. > The time it takes to do an integration also needs to be addressed. > Integration testing needs to be completely automated, and false negatives need to be minimized to > speed up the analysis phase. > > The actual integration itself cannot be completely automated (merges or syncs with the master must > always be watched carefully) but many of the large steps in the integration process can be automated. > Ideally, we should be able to have all team areas ready for an integration on a 24hour basis, based on > an inspection of acceptable results, a sync with master changes, a quick re-build and re-test, and a push. > Just my opinion. > > -kto From roger.calnan at oracle.com Mon May 30 11:45:24 2011 From: roger.calnan at oracle.com (Roger Calnan) Date: Mon, 30 May 2011 11:45:24 -0700 Subject: Mercurial forests for JDK 8 In-Reply-To: References: <20110524205342.AF2C211E4@eggemoggin.niobe.net> <76357345-18AE-4B24-AC82-8B2A556F977F@oracle.com> <4DDC249D.6020501@oracle.com> Message-ID: <7625B210-DBE7-4E74-9BE9-04B335A7D1AF@oracle.com> > So what (other than momentum) is preventing us from having a more nimble integration > capability? In my naive view, it looks to be having better testing (as in more effective -- less but > more tailored to hit the issues in a particular integration repo). a lot of it has been fear of allowing the master to become unstable. However some of the areas that were historically unstable are now no longer as active. Maybe we should take the opportunity at the start of a new feature release to remove some of the constraints we have had in place previously to see if there are still needed. Isn't it the case of saying "sufficient testing should be done before an integration to ensure the master stays stable". A lot of the infrastructure work is to cut down the cost of running that testing but it should be up to each area to determine what is "sufficient" depending on the integration. How about having an "integration baton" rather than a schedule and see if issues come up. There would then only be one regular slot for RE. Roger From Alan.Bateman at oracle.com Mon May 30 13:19:44 2011 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 30 May 2011 21:19:44 +0100 Subject: Mercurial forests for JDK 8 In-Reply-To: References: <20110524205342.AF2C211E4@eggemoggin.niobe.net> <76357345-18AE-4B24-AC82-8B2A556F977F@oracle.com> <4DDC249D.6020501@oracle.com> Message-ID: <4DE3FBE0.5040308@oracle.com> Georges Saab wrote: > If I have understood correctly the present system of the 'static and somewhat fixed schedule > of integrations' was introduced a number of years ago when there were lots of issues with > destabilizing (ok, build breaking!) changes coming in, And that it has been successful in > preventing these, albeit at quite a bit of cost. > > So what (other than momentum) is preventing us from having a more nimble integration > capability? In my naive view, it looks to be having better testing (as in more effective -- less but > more tailored to hit the issues in a particular integration repo). > > I like Kelly's suggestion of viewing it as a 'budget of time' -- i.e. we should figure out the best set > of tests for a repo that can be run within 24 hours'. I think there is more that can be done here > to make sure integration repos stay clean... > > Our processes are indeed legacy, and assume the worst. Much of the lag between snapshot and integration today is due to pre-integration testing, the need and duration of which vary by area. In the core area at least, then we have to make efforts to reduce or eliminate this. For starters then it would be great if OpenJDK had build and test bots so that we have daily or continuous building and testing on all integration forest (and all platforms). Another one is that we need to put more effort into improving the robustness and through-put of our tests, meaning the tests in the jdk repository. The langtools area gets an A+ grade but the jdk repository gets a D- because we have many tests that having timing issues, are slow, or are just too sensitive to the environment. In my view, we have to fix these issues so that we get to the point where we can quickly see from the test results whether we are integration ready or not. Bug verification and other work can happy lazily. I can't speak for the client and other areas but I will guess that not everything can be automated and that some level of special pre-integration testing will be required to be sure that regressions don't get into the master. -Alan. From kelly.ohair at oracle.com Mon May 30 16:20:37 2011 From: kelly.ohair at oracle.com (Kelly O'Hair) Date: Mon, 30 May 2011 16:20:37 -0700 Subject: Mercurial forests for JDK 8 In-Reply-To: <7625B210-DBE7-4E74-9BE9-04B335A7D1AF@oracle.com> References: <20110524205342.AF2C211E4@eggemoggin.niobe.net> <76357345-18AE-4B24-AC82-8B2A556F977F@oracle.com> <4DDC249D.6020501@oracle.com> <7625B210-DBE7-4E74-9BE9-04B335A7D1AF@oracle.com> Message-ID: On May 30, 2011, at 11:45 AM, Roger Calnan wrote: > How about having an "integration baton" rather than a > schedule and see if issues come up. There would then only be one regular slot for RE. The JavaFX team had a signup page that controlled 'who held the baton' so to speak, where RE would block out designated promotion times. I thought this worked out quite well, I think it was a 4 hour time slot that blocked all pushes to the master except for the engineer assigned the slot, so up to 6 integrations in 24hrs. But JavaFX had lighter weight requirements on integration testing, and heavier requirements on the weekly build promotion, which could block all integrations until build promotion requirements were met. -kto From david.katleman at oracle.com Tue May 31 10:02:26 2011 From: david.katleman at oracle.com (David Katleman) Date: Tue, 31 May 2011 10:02:26 -0700 Subject: Mercurial forests for JDK 8 In-Reply-To: <7625B210-DBE7-4E74-9BE9-04B335A7D1AF@oracle.com> References: <20110524205342.AF2C211E4@eggemoggin.niobe.net> <76357345-18AE-4B24-AC82-8B2A556F977F@oracle.com> <4DDC249D.6020501@oracle.com> <7625B210-DBE7-4E74-9BE9-04B335A7D1AF@oracle.com> Message-ID: <4DE51F22.7080909@oracle.com> On 5/30/2011 11:45 AM, Roger Calnan wrote: > depending on the integration. How about having an "integration baton" rather than a > schedule and see if issues come up. There would then only be one regular slot for RE. Integration schedules were setup not just for build stability, but also to spread out integrations through the entire week. We also manipulated the schedule to separate integrations that proved to be problematic. Prior to the schedules, integrations tended to bunch up just before the promotion dates, with little change going in for the first half of a week. That bunching contributed to build failures at promotion time, rather than having those failures earlier in a weekly cycle, where there is more time to recover. I'm not opposed to a more free style approach to integration, especially with the considerable consolidation of groups, as long as we can manage the bunching. Dave From Vita.Santrucek at oracle.com Tue May 31 13:45:15 2011 From: Vita.Santrucek at oracle.com (Vita Santrucek) Date: Tue, 31 May 2011 13:45:15 -0700 (PDT) Subject: Mercurial forests for JDK 8 In-Reply-To: <4DE51F22.7080909@oracle.com> References: <20110524205342.AF2C211E4@eggemoggin.niobe.net> <76357345-18AE-4B24-AC82-8B2A556F977F@oracle.com> <4DDC249D.6020501@oracle.com> <7625B210-DBE7-4E74-9BE9-04B335A7D1AF@oracle.com> <4DE51F22.7080909@oracle.com> Message-ID: [Just joined the alias and missed the early stage of this discussion] I agree that the past model was not very efficient and we had instances where an interaction bug was found 4 weeks after a change was first integrated, even though engineers were following the correct process. We need to fix that. In the end game of JDK 7 we did a first step towards more expedient integrations - there are not pre-determined integration slots, all integrations are performed on first-come, first-served basis after reserving a slot. In my opinion, to make the process faster, next we should optimize the pre-integration testing and integration itself: 1) split the integration repos - in the current model, the fastest/least buggy components need to wait for the slowest/... to finish their pre-integration testing (PIT). While in an ideal case we have working nightly/CI testing for all the sub-components, it is not the case, and those who are ahead of the game should get the ability to integrate when they are ready (and pioneer the process for the rest). My understanding is that this had already been considered. Probably, my team can help with implementing it. 2) QE as integrators - QE is responsible for the verifying the fixes and overal component quality. As such, QE should perform the gate-keeping function for their components. This will further speed up the frequent integrations and also it will allow QE to provide proper coverage before integrating. This model has been used by JRockit. So the proposed process would be: 1) developer creates a fix/feature 2) developer tests the fix on his local repo code 3) developer(s) integrate(s) a fix into the component repo 4) QE runs CI/PIT (coverage as agreed by the component team) 5) QE reserves a slot and integrates to the master repo 6) other component teams pull the changes (each promo time or more frequently) For some teams, this effectively takes the integration process from weeks to a day. Most interaction bugs would be found within a week. Vita On Tue, 31 May 2011, David Katleman wrote: ->Date: Tue, 31 May 2011 10:02:26 -0700 ->From: David Katleman ->To: jdk8-dev at openjdk.java.net ->Subject: Re: Mercurial forests for JDK 8 -> ->On 5/30/2011 11:45 AM, Roger Calnan wrote: ->> depending on the integration. How about having an "integration baton" ->> rather than a ->> schedule and see if issues come up. There would then only be one regular ->> slot for RE. -> ->Integration schedules were setup not just for build stability, but also to ->spread out integrations through the entire week. We also manipulated the ->schedule to separate integrations that proved to be problematic. -> ->Prior to the schedules, integrations tended to bunch up just before the ->promotion dates, with little change going in for the first half of a week. ->That bunching contributed to build failures at promotion time, rather than ->having those failures earlier in a weekly cycle, where there is more time to ->recover. -> ->I'm not opposed to a more free style approach to integration, especially with ->the considerable consolidation of groups, as long as we can manage the ->bunching. -> -> Dave -> -> From kelly.ohair at oracle.com Tue May 31 13:49:10 2011 From: kelly.ohair at oracle.com (Kelly O'Hair) Date: Tue, 31 May 2011 13:49:10 -0700 Subject: Mercurial forests for JDK 8 In-Reply-To: <4DE51F22.7080909@oracle.com> References: <20110524205342.AF2C211E4@eggemoggin.niobe.net> <76357345-18AE-4B24-AC82-8B2A556F977F@oracle.com> <4DDC249D.6020501@oracle.com> <7625B210-DBE7-4E74-9BE9-04B335A7D1AF@oracle.com> <4DE51F22.7080909@oracle.com> Message-ID: <7B1C4B5F-7D39-49D9-A138-49FF69460F57@oracle.com> On May 31, 2011, at 10:02 AM, David Katleman wrote: > On 5/30/2011 11:45 AM, Roger Calnan wrote: >> depending on the integration. How about having an "integration baton" rather than a >> schedule and see if issues come up. There would then only be one regular slot for RE. > > Integration schedules were setup not just for build stability, but also to spread out integrations through the entire week. We also manipulated the schedule to separate integrations that proved to be problematic. But part of the reason for that spreading out was also related to SQE resources, or their ability to turn around a PIT (Pre Integration Testing) report/approval certificate, which is a formalism I'd like to completely automate so we don't have a people resource limitation. The hardware/system resource issues can't be ignored of course. > > Prior to the schedules, integrations tended to bunch up just before the promotion dates, with little change going in for the first half of a week. That bunching contributed to build failures at promotion time, rather than having those failures earlier in a weekly cycle, where there is more time to recover. Yes, the old traffic jam issue just before a milestone or promotion build. :^( We should be operating on a train schedule, and if developer changes are not ready when it's time for the promotion train to leave the station, they just need to wait for the next train. Delaying the boarded passengers is unacceptable. And we should have a fixed number of integrations per 24hr period, just the time needed for sync&build&test. > I'm not opposed to a more free style approach to integration, especially with the considerable consolidation of groups, as long as we can manage the bunching. If the integrator has a fixed amount of time for doing his sync&build&test verification, then I think we have managed the bunching. It is most critical that the master repos always build, it's the testing that becomes more problematic, at least until we define what "testing" means for an integration. -kto > > Dave > > From kelly.ohair at oracle.com Tue May 31 17:36:24 2011 From: kelly.ohair at oracle.com (Kelly O'Hair) Date: Tue, 31 May 2011 17:36:24 -0700 Subject: Mercurial forests for JDK 8 In-Reply-To: References: <20110524205342.AF2C211E4@eggemoggin.niobe.net> <76357345-18AE-4B24-AC82-8B2A556F977F@oracle.com> <4DDC249D.6020501@oracle.com> <7625B210-DBE7-4E74-9BE9-04B335A7D1AF@oracle.com> <4DE51F22.7080909@oracle.com> Message-ID: <48A69F96-F0CC-4FF6-BFF0-BBA52870C1D1@oracle.com> On May 31, 2011, at 1:45 PM, Vita Santrucek wrote: > 2) QE as integrators - QE is responsible for the verifying the fixes and > overal component quality. As such, QE should perform the gate-keeping > function for their components. This will further speed up the frequent > integrations and also it will allow QE to provide proper coverage before > integrating. This model has been used by JRockit. I don't understand how changing from the current integration process we have now, to having it done by a QE integrator speeds anything up or can increase the frequency in any significant way. If you are suggesting having more integrators could speed things up, that may work to a degree, but at some point it won't help to just throw more people at it. > > So the proposed process would be: > > 1) developer creates a fix/feature > 2) developer tests the fix on his local repo code > 3) developer(s) integrate(s) a fix into the component repo > 4) QE runs CI/PIT (coverage as agreed by the component team) > 5) QE reserves a slot and integrates to the master repo > 6) other component teams pull the changes (each promo time or more > frequently) > > For some teams, this effectively takes the integration process from > weeks to a day. Most interaction bugs would be found within a week. I don't see the "day" here, and I'm not sure I see much difference from what is happening now. Steps 1-3 have been documented here: http://openjdk.java.net/guide/changePlanning.html#bug granted it needs a little refreshing. Steps 4-5 have traditionally been assigned to the official "integrator" for a team, who also serves as a go-between QE, the development team, and release engineering for that integration. It used to be a rotating assignment in the development team because the complications of syncs and merges can require developer involvement, or 'walk down the hall' face-to-face interactions. I'm concerned that "QE as integrators" just moves the work around and perhaps just moves the bottleneck around. -kto