<html><head></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><br><div><div>On Jan 29, 2012, at 10:23 AM, Georges Saab wrote:</div><br class="Apple-interchange-newline"><blockquote type="cite"><span class="Apple-style-span" style="border-collapse: separate; font-family: Times; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-align: -webkit-auto; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; font-size: medium; "><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><div><blockquote type="cite"><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><div><br class="Apple-interchange-newline">I'm missing something. How can everybody using the exact same system scale to 100's of developers?</div></div></blockquote><div><br></div><div>System = distributed build and test of OpenJDK</div></div></div></span></blockquote><div><br></div>Ah ha... I'm down in the trenches dealing with dozens of different OS's arch's variation machines.</div><div>You are speaking to a higher level, I need to crawl out of the basement.</div><div><br><blockquote type="cite"><span class="Apple-style-span" style="border-collapse: separate; font-family: Times; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-align: -webkit-auto; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; font-size: medium; "><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><div><div><br></div><div>Developers send in jobs </div><div>Jobs are distribute across a pool of (HW/OS) resources</div><div>The resources may be divided into pools dedicated to different tasks (RE/checkin/perf/stress)</div><div>The pools are populated initially according to predictions of load and then increased/rebalanced according to data on actual usage</div><div>No assumptions made about what exists on the machine other than HW/OS</div><div>The build and test tasks are self sufficient, i.e. bootstrap themselves </div><div>The bootstrapping is done in the same way for different build and test tasks</div></div></div></span></blockquote><div><br></div>Understood. We have talked about this before. I have also been on the search for the Holy Grail. ;^)</div><div>This is why I keep working on JPRT.</div><div><br><blockquote type="cite"><span class="Apple-style-span" style="border-collapse: separate; font-family: Times; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-align: -webkit-auto; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; font-size: medium; "><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><div><div><br></div><div>The only scaling aspect that seems at all challenging is that the current checkin system is designed to serialize checkins in a way that apparently does not scale -- here there are some decisions to be made and tradeoffs but this is nothing new in the world of Open community development (or any large team development for that matter)</div></div></div></span></blockquote><div><br></div></div><div>The serialize checkins issue can be minimized some by using distributed SCMs (Mercurial, Git, etc)</div><div>and using separate forests (fewer developers per source repository means fewer merge/sync issues)</div><div>and having an integrator merge into a master. This has proven to work in many situations but it</div><div>also creates delivery to master delays, especially if the integration process is too heavyweight.</div><div><br></div><div>The JDK projects has been doing this for a long time, I'm sure many people have opinions as to how</div><div>successful it is or isn't.</div><div><br></div><div>It is my opinion that merges/syncs are some of the most dangerous things you can do to a source base,</div><div>and anything we can do to avoid them is usually goodness, I don't think you should scale this without some</div><div>very great care.</div><div><br><blockquote type="cite"><span class="Apple-style-span" style="border-collapse: separate; font-family: Times; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-align: -webkit-auto; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; font-size: medium; "><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><div><br><blockquote type="cite"><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><div><br></div><div>And that one system will naturally change over time too, so unless you are able to prevent all change</div><div>to a system (impossible with security updates etc) every use of that 'same system' will be different.</div></div></blockquote><div><br></div><div>Yes, but it is possible to control this update and have a staging environment so you know that a HW/OS update will not break the existing successful build when rolled out to the build/test farm.</div></div></div></span></blockquote><br></div><div>Possible but not always easy. The auto updating of everything has increased significantly over the years,</div><div>making it harder to control completely.</div><div><br></div><div>I've been doing this build&test stuff long enough to never expect anything to be 100% reliable.</div><div>Hardware fails, software updates regress functionality, networks become unreliable, humans trip over</div><div>power cords, virus scanners break things, etc. It just happens, and often, it's not very predictable or reproducible.</div><div>You can do lots of things to minimize issues, but at some point you just have to accept a few risks because</div><div>the alternative just isn't feasible or just can't happen with the resources we have.</div><div><br></div><div>-kto</div><div><br></div><br></body></html>