Proposal to revise forest graph and integration practices for JDK 9

[Neil Richards]

Sounds like a useful process optimization to me :)
+1
Regards,
Neil

On Sat, 2013-11-23 at 08:34 -0800, Joe Darcy wrote:
> Hello,
> 
> The current arrangements of sets of integration forests for a JDK 
> platform release, like JDK 8, impose high overheads on development. I'm 
> proposing we use an alternate forest arrangement for JDK 9 that will 
> dramatically reduce the propagation time of fixes across the set of 
> forests. More details below.
> 
> JDK release projects for new Java SE platforms, like JDK 8 for Java SE 
> 8, have long used a graph of forests structured roughly as follow:
> 
> * A master forest for the release
> 
> * A thicket of integration forests for particular teams or technology 
> areas. Today in JDK 8, the "TL" forest hosts changes in tools related to 
> javac as well as core libraries. Forests for various client libraries 
> (2D, awt, swing) host changes in those areas. A HotSpot forest, fed in 
> turn by several HotSpot team forests, hosts VM-related changes. 
> Generally, each integration forest only accepts changes to a subset of 
> repositories. For example, the HotSpot-related forests typically only 
> accept changes to the hotspot repository. The TL forest accepts changes 
> to library-related repos (jdk, jaxp, jaxws, etc.) and langtools, but not 
> to hotspot.
> 
> After some amount of testing and other verification steps, changes in 
> one integration forest are integrated into master, typically on a weekly 
> or bi-weekly basis. From master, the fix then propagates down to 
> integration forests according to the policies of that forest. A fix 
> could be in master for several days or more before being propagated to a 
> particular integration forest.
> 
> While this structure has provided a great deal of cross-team isolation, 
> it has come at the cost of high propagation delays of fixes to all 
> forests. This propagation delay combined with only pushing fixes to a 
> subset of repos also severely complicates making coordinated changes 
> which span across repositories, as often occurred in Project Lambda and 
> which chronically occurs for technologies like servicability. To give a 
> representative example, consider a hypothetical change which requires 
> updates to both the HotSpot runtime area as well as core libraries. If 
> the change is first pushed to HotSpot, the propagation proceeds like:
> 
>      fix pushed to HotSpot runtime forest -> HotSpot main -> JDK 8 
> master -> TL -> core libs engineer's forest
> 
> At this point, the libraries half of the change can be pushed:
> 
>      core libs fix pushed to TL -> JDK 8 master -> HotSpot main -> 
> HotSpot runtime -> HotSpot runtime engineer's forest
> 
> This cycle to separately push both halves of what is conceptually a 
> single fix and wait for them to propagate can take about four weeks. 
> (Worse, often there needs to be a third push to complete the fix since 
> the first push is often to "accept old way or new way" and the third 
> push updates this to "accept new way only." When such a clean-up push is 
> needed, it takes another two weeks to fully propagate.)
> 
> Since more projects requiring cross-repository coordination are expected 
> in the future, I'm proposing a number of changes to the forest structure 
> and management policies in JDK 9 to reduce the propagation delays.
> 
> * A master forest that is a time-delayed version of dev; dev described 
> below.
> 
> * The dev forest conceptually replaces TL and hosts all 
> libraries-related changes. When the sources in dev are in a known-good 
> state, that state can be integrated to master. This integration cycle 
> would happen at least weekly. In a change from current practice, HotSpot 
> changes would be integrated into dev and *not* into master. All other 
> team forests would also integrate into dev rather than master.
> 
> * Coordinated HotSpot + other component fixes would *both* get first 
> pushed through the HotSpot forest. From hotspot the full fix would be 
> integrated into dev. If additional testing was appropriate for the 
> non-HotSpot fix, that testing should occur before integration into dev.
> 
> * Regular promoted builds based on master would continue.
> 
> By having team forests integrate directly into dev as well as having 
> many libraries developers pushing directly to dev, the dev forest serves 
> as an active collaboration area with greatly reduced propagation times 
> across the whole system. With this model there is less cross-team 
> isolation; teams and individuals are responsible for promptly fixing any 
> breakage which is introduced. If problems are not quickly addressed, a 
> problematic changeset may be anti-delta'ed.
> 
> (Conceptually, in this model a separate master forest is not strictly 
> needed since the known-good states could be indicated using a mechanism 
> like Hg tags. However, while adjusting to the new model and to allow for 
> fixes directly to master in exceptional circumstances, I'm proposing a 
> physically separate master forest be retained. The distinct URL of 
> master will also clearly indicate known-good states of the source code.)
> 
> Please send comments on the above by November 29.
> 
> Thanks,
> 
> -Joe
> 

-- 
Unless stated above:
IBM email: neil_richards at uk.ibm.com
IBM United Kingdom Limited - Registered in England and Wales with number 741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU