Proposal to revise forest graph and integration practices for JDK 9

[Staffan Larsen]

Dmitry,

We could (and should) increase the set of JDK tests we run in hotspot-rt nightlies.

/Staffan

On 2 dec 2013, at 06:50, Dmitry Samersoff <dmitry.samersoff at oracle.com> wrote:

> Joe,
> 
> Could we at least merge tl and hotspot-rt workspaces to a single one -
> tl-hotspot-rt?
> 
> Pushing JDK fixes to hotspot-rt and than merge it into tl a week or two
> later could be a pain because hotspot-rt has limited set of JDK tests in
> nightly and lots of changes happens in tl within a week.
> 
> -Dmitry
> 
> On 2013-12-02 09:04, Joe Darcy wrote:
>> Hi Dalibor,
>> 
>> On 11/25/2013 5:40 AM, Dalibor Topic wrote:
>>> On 11/23/13 5:34 PM, Joe Darcy wrote:
>>>> Since more projects requiring cross-repository coordination are
>>>> expected in the future, I'm proposing a number of changes to the
>>>> forest structure and management policies in JDK 9 to reduce the
>>>> propagation delays.
>>> This seems very close to how 7u works already, so it's a huge
>>> improvement over 8, but it comes with some historical baggage:
>>> 
>>>> By having team forests integrate directly into dev as well as having
>>>> many libraries developers pushing directly to dev, the dev forest
>>>> serves as an active collaboration area with greatly reduced
>>>> propagation times across the whole system. With this model there is
>>>> less cross-team isolation; teams and individuals are responsible for
>>>> promptly fixing any breakage which is introduced. If problems are not
>>>> quickly addressed, a problematic changeset may be anti-delta'ed.
>>> One of the useful things we did for 7u was to try to drive down the
>>> number of team forests to just -dev. We didn't quite succeed, as you
>>> can see from the hotspot team having their own 7u integration forest,
>>> but overall it was a major step in driving the complexity of
>>> (un)necessary interactions down, and enabling fixes to propagate faster.
>>> 
>>> I would suggest that you consider doing something similar for JDK 9,
>>> as the spider web [0] of JDK 8's forests currently contains about a
>>> dozen of forests, and from the way the proposal is written it seems
>>> that you may like to do that, but unfortunately the proposal is not
>>> very explicit about it.
>>> 
>>> So, I'd like to suggest that jdk9/dev replaces the equivalent of
>>> jdk9/tl, jdk9/awt, jdk9/swing, jdk9/2d, jdk9/build, jdk9/deploy,
>>> jdk9/l10n, jdk9/swing, nashorn/jdk9 in one go.
>> 
>> For the open components, that is essentially the proposal. Deploy and
>> install are closed and have different testing requirements from other
>> code so would likely remain separate.
>> 
>>>  If long term features require a throw-away integration forest,
>>> create one as you need it for as long as you need it, integrate and
>>> close that forest.
>>> 
>>> I don't have an opinion on pruning hotspot forests (yet;), but you may
>>> want to consider where a fix containing changes both for hotspot and
>>> libraries would need to go to if you retain the current structure -
>>> gc? rt? hotspot-dev? hsx25? main? comp?
>> 
>> At least in JDK 8, I believe most of libs + HotSpot interactions were in
>> runtime, so a push to rt would be most appropriate in that case. In
>> general, I would recommend a coordinated push go into the HotSpot
>> integration forest with the closest interaction.
>> 
>>> 
>>>> (Conceptually, in this model a separate master forest is not strictly
>>>> needed since the known-good states could be indicated using a
>>>> mechanism like Hg tags. However, while adjusting to the new model and
>>>> to allow for fixes directly to master in exceptional circumstances,
>>>> I'm proposing a physically separate master forest be retained. The
>>>> distinct URL of master will also clearly indicate known-good states
>>>> of the source code.)
>>> One of the not very useful things we did for 7u was to keep a separate
>>> master forest.
>>> 
>>> Considering the real benefits of a master forest [1], and the real
>>> drawbacks when it's being used with a rapidly changing tree underneath
>>> [2], I'd suggest you drop the master forest for JDK 9.
>> 
>> Thanks for the feedback on that point of the proposal; I'm still
>> uncomfortable with collapsing master and dev before we put the model
>> into practice.
>> 
>>> 
>>> One thing missing from the proposal is jdk9 repository layout within
>>> the forests. I think that this may be good opportunity to simplify the
>>> current historically grown structure, for example by folding the
>>> nashorn and langtools repositories together, as well as corba, jaxp,
>>> jaxws and jdk.
>> 
>> Yes, that would be another level of reorganization to discuss.
>> 
>> Cheers,
>> 
>> -Joe
> 
> 
> -- 
> Dmitry Samersoff
> Oracle Java development team, Saint Petersburg, Russia
> * I would love to change the world, but they won't give me the source code.