Proposal to revise forest graph and integration practices for JDK 9

[Joe Darcy]

Hi Dalibor,

On 11/25/2013 5:40 AM, Dalibor Topic wrote:
> On 11/23/13 5:34 PM, Joe Darcy wrote:
>> Since more projects requiring cross-repository coordination are expected in the future, I'm proposing a number of changes to the forest structure and management policies in JDK 9 to reduce the propagation delays.
> This seems very close to how 7u works already, so it's a huge improvement over 8, but it comes with some historical baggage:
>   
>> By having team forests integrate directly into dev as well as having many libraries developers pushing directly to dev, the dev forest serves as an active collaboration area with greatly reduced propagation times across the whole system. With this model there is less cross-team isolation; teams and individuals are responsible for promptly fixing any breakage which is introduced. If problems are not quickly addressed, a problematic changeset may be anti-delta'ed.
> One of the useful things we did for 7u was to try to drive down the number of team forests to just -dev. We didn't quite succeed, as you can see from the hotspot team having their own 7u integration forest, but overall it was a major step in driving the complexity of (un)necessary interactions down, and enabling fixes to propagate faster.
>
> I would suggest that you consider doing something similar for JDK 9, as the spider web [0] of JDK 8's forests currently contains about a dozen of forests, and from the way the proposal is written it seems that you may like to do that, but unfortunately the proposal is not very explicit about it.
>
> So, I'd like to suggest that jdk9/dev replaces the equivalent of jdk9/tl, jdk9/awt, jdk9/swing, jdk9/2d, jdk9/build, jdk9/deploy, jdk9/l10n, jdk9/swing, nashorn/jdk9 in one go.

For the open components, that is essentially the proposal. Deploy and 
install are closed and have different testing requirements from other 
code so would likely remain separate.

>   If long term features require a throw-away integration forest, create one as you need it for as long as you need it, integrate and close that forest.
>
> I don't have an opinion on pruning hotspot forests (yet;), but you may want to consider where a fix containing changes both for hotspot and libraries would need to go to if you retain the current structure - gc? rt? hotspot-dev? hsx25? main? comp?

At least in JDK 8, I believe most of libs + HotSpot interactions were in 
runtime, so a push to rt would be most appropriate in that case. In 
general, I would recommend a coordinated push go into the HotSpot 
integration forest with the closest interaction.

>
>> (Conceptually, in this model a separate master forest is not strictly needed since the known-good states could be indicated using a mechanism like Hg tags. However, while adjusting to the new model and to allow for fixes directly to master in exceptional circumstances, I'm proposing a physically separate master forest be retained. The distinct URL of master will also clearly indicate known-good states of the source code.)
> One of the not very useful things we did for 7u was to keep a separate master forest.
>
> Considering the real benefits of a master forest [1], and the real drawbacks when it's being used with a rapidly changing tree underneath [2], I'd suggest you drop the master forest for JDK 9.

Thanks for the feedback on that point of the proposal; I'm still 
uncomfortable with collapsing master and dev before we put the model 
into practice.

>
> One thing missing from the proposal is jdk9 repository layout within the forests. I think that this may be good opportunity to simplify the current historically grown structure, for example by folding the nashorn and langtools repositories together, as well as corba, jaxp, jaxws and jdk.

Yes, that would be another level of reorganization to discuss.

Cheers,

-Joe