Proposal to revise forest graph and integration practices for JDK 9

[Brian Goetz]

A very positive step.  +1.

On 11/23/2013 11:34 AM, Joe Darcy wrote:
> Hello,
>
> The current arrangements of sets of integration forests for a JDK
> platform release, like JDK 8, impose high overheads on development. I'm
> proposing we use an alternate forest arrangement for JDK 9 that will
> dramatically reduce the propagation time of fixes across the set of
> forests. More details below.
>
> JDK release projects for new Java SE platforms, like JDK 8 for Java SE
> 8, have long used a graph of forests structured roughly as follow:
>
> * A master forest for the release
>
> * A thicket of integration forests for particular teams or technology
> areas. Today in JDK 8, the "TL" forest hosts changes in tools related to
> javac as well as core libraries. Forests for various client libraries
> (2D, awt, swing) host changes in those areas. A HotSpot forest, fed in
> turn by several HotSpot team forests, hosts VM-related changes.
> Generally, each integration forest only accepts changes to a subset of
> repositories. For example, the HotSpot-related forests typically only
> accept changes to the hotspot repository. The TL forest accepts changes
> to library-related repos (jdk, jaxp, jaxws, etc.) and langtools, but not
> to hotspot.
>
> After some amount of testing and other verification steps, changes in
> one integration forest are integrated into master, typically on a weekly
> or bi-weekly basis. From master, the fix then propagates down to
> integration forests according to the policies of that forest. A fix
> could be in master for several days or more before being propagated to a
> particular integration forest.
>
> While this structure has provided a great deal of cross-team isolation,
> it has come at the cost of high propagation delays of fixes to all
> forests. This propagation delay combined with only pushing fixes to a
> subset of repos also severely complicates making coordinated changes
> which span across repositories, as often occurred in Project Lambda and
> which chronically occurs for technologies like servicability. To give a
> representative example, consider a hypothetical change which requires
> updates to both the HotSpot runtime area as well as core libraries. If
> the change is first pushed to HotSpot, the propagation proceeds like:
>
>      fix pushed to HotSpot runtime forest -> HotSpot main -> JDK 8
> master -> TL -> core libs engineer's forest
>
> At this point, the libraries half of the change can be pushed:
>
>      core libs fix pushed to TL -> JDK 8 master -> HotSpot main ->
> HotSpot runtime -> HotSpot runtime engineer's forest
>
> This cycle to separately push both halves of what is conceptually a
> single fix and wait for them to propagate can take about four weeks.
> (Worse, often there needs to be a third push to complete the fix since
> the first push is often to "accept old way or new way" and the third
> push updates this to "accept new way only." When such a clean-up push is
> needed, it takes another two weeks to fully propagate.)
>
> Since more projects requiring cross-repository coordination are expected
> in the future, I'm proposing a number of changes to the forest structure
> and management policies in JDK 9 to reduce the propagation delays.
>
> * A master forest that is a time-delayed version of dev; dev described
> below.
>
> * The dev forest conceptually replaces TL and hosts all
> libraries-related changes. When the sources in dev are in a known-good
> state, that state can be integrated to master. This integration cycle
> would happen at least weekly. In a change from current practice, HotSpot
> changes would be integrated into dev and *not* into master. All other
> team forests would also integrate into dev rather than master.
>
> * Coordinated HotSpot + other component fixes would *both* get first
> pushed through the HotSpot forest. From hotspot the full fix would be
> integrated into dev. If additional testing was appropriate for the
> non-HotSpot fix, that testing should occur before integration into dev.
>
> * Regular promoted builds based on master would continue.
>
> By having team forests integrate directly into dev as well as having
> many libraries developers pushing directly to dev, the dev forest serves
> as an active collaboration area with greatly reduced propagation times
> across the whole system. With this model there is less cross-team
> isolation; teams and individuals are responsible for promptly fixing any
> breakage which is introduced. If problems are not quickly addressed, a
> problematic changeset may be anti-delta'ed.
>
> (Conceptually, in this model a separate master forest is not strictly
> needed since the known-good states could be indicated using a mechanism
> like Hg tags. However, while adjusting to the new model and to allow for
> fixes directly to master in exceptional circumstances, I'm proposing a
> physically separate master forest be retained. The distinct URL of
> master will also clearly indicate known-good states of the source code.)
>
> Please send comments on the above by November 29.
>
> Thanks,
>
> -Joe
>