Proposal to revise forest graph and integration practices for JDK 9

[Vladimir Kozlov]

On 12/2/13 9:19 AM, Joe Darcy wrote:
> Hello Dmitry,
>
> In a future world, with better testing and better test stability, it would be reasonable to consider combining tl/dev
> and the HotSpot forests. However, that would not be an appropriate combination today.
>
> It would be possible (and encouraged!) for HotSpot main to pull down changes from dev on a regular basis, closer to
> daily rather than weekly.

It will only help for synchronized pushes into Hotspot and libs. Which is not such frequent (not daily).
The main problem is Nightly testing of Hotspot. Currently we use jdk bundles prepared by JPRT when we push changes into 
Hotspot. JPRT uses promoted JDK and that is the problem. If we want to use latest dev changes for Hotspot testing we 
need to have daily builds from dev changes which JPRT can use.
Yes, ideally it would be nice to build jdk for each Hotspot push and bundle it but JPRT does not have enough throughput 
for that now.

Regards,
Vladimir

>
> Cheers,
>
> -Joe
>
> On 12/01/2013 09:50 PM, Dmitry Samersoff wrote:
>> Joe,
>>
>> Could we at least merge tl and hotspot-rt workspaces to a single one -
>> tl-hotspot-rt?
>>
>> Pushing JDK fixes to hotspot-rt and than merge it into tl a week or two
>> later could be a pain because hotspot-rt has limited set of JDK tests in
>> nightly and lots of changes happens in tl within a week.
>>
>> -Dmitry
>>
>> On 2013-12-02 09:04, Joe Darcy wrote:
>>> Hi Dalibor,
>>>
>>> On 11/25/2013 5:40 AM, Dalibor Topic wrote:
>>>> On 11/23/13 5:34 PM, Joe Darcy wrote:
>>>>> Since more projects requiring cross-repository coordination are
>>>>> expected in the future, I'm proposing a number of changes to the
>>>>> forest structure and management policies in JDK 9 to reduce the
>>>>> propagation delays.
>>>> This seems very close to how 7u works already, so it's a huge
>>>> improvement over 8, but it comes with some historical baggage:
>>>>> By having team forests integrate directly into dev as well as having
>>>>> many libraries developers pushing directly to dev, the dev forest
>>>>> serves as an active collaboration area with greatly reduced
>>>>> propagation times across the whole system. With this model there is
>>>>> less cross-team isolation; teams and individuals are responsible for
>>>>> promptly fixing any breakage which is introduced. If problems are not
>>>>> quickly addressed, a problematic changeset may be anti-delta'ed.
>>>> One of the useful things we did for 7u was to try to drive down the
>>>> number of team forests to just -dev. We didn't quite succeed, as you
>>>> can see from the hotspot team having their own 7u integration forest,
>>>> but overall it was a major step in driving the complexity of
>>>> (un)necessary interactions down, and enabling fixes to propagate faster.
>>>>
>>>> I would suggest that you consider doing something similar for JDK 9,
>>>> as the spider web [0] of JDK 8's forests currently contains about a
>>>> dozen of forests, and from the way the proposal is written it seems
>>>> that you may like to do that, but unfortunately the proposal is not
>>>> very explicit about it.
>>>>
>>>> So, I'd like to suggest that jdk9/dev replaces the equivalent of
>>>> jdk9/tl, jdk9/awt, jdk9/swing, jdk9/2d, jdk9/build, jdk9/deploy,
>>>> jdk9/l10n, jdk9/swing, nashorn/jdk9 in one go.
>>> For the open components, that is essentially the proposal. Deploy and
>>> install are closed and have different testing requirements from other
>>> code so would likely remain separate.
>>>
>>>>    If long term features require a throw-away integration forest,
>>>> create one as you need it for as long as you need it, integrate and
>>>> close that forest.
>>>>
>>>> I don't have an opinion on pruning hotspot forests (yet;), but you may
>>>> want to consider where a fix containing changes both for hotspot and
>>>> libraries would need to go to if you retain the current structure -
>>>> gc? rt? hotspot-dev? hsx25? main? comp?
>>> At least in JDK 8, I believe most of libs + HotSpot interactions were in
>>> runtime, so a push to rt would be most appropriate in that case. In
>>> general, I would recommend a coordinated push go into the HotSpot
>>> integration forest with the closest interaction.
>>>
>>>>> (Conceptually, in this model a separate master forest is not strictly
>>>>> needed since the known-good states could be indicated using a
>>>>> mechanism like Hg tags. However, while adjusting to the new model and
>>>>> to allow for fixes directly to master in exceptional circumstances,
>>>>> I'm proposing a physically separate master forest be retained. The
>>>>> distinct URL of master will also clearly indicate known-good states
>>>>> of the source code.)
>>>> One of the not very useful things we did for 7u was to keep a separate
>>>> master forest.
>>>>
>>>> Considering the real benefits of a master forest [1], and the real
>>>> drawbacks when it's being used with a rapidly changing tree underneath
>>>> [2], I'd suggest you drop the master forest for JDK 9.
>>> Thanks for the feedback on that point of the proposal; I'm still
>>> uncomfortable with collapsing master and dev before we put the model
>>> into practice.
>>>
>>>> One thing missing from the proposal is jdk9 repository layout within
>>>> the forests. I think that this may be good opportunity to simplify the
>>>> current historically grown structure, for example by folding the
>>>> nashorn and langtools repositories together, as well as corba, jaxp,
>>>> jaxws and jdk.
>>> Yes, that would be another level of reorganization to discuss.
>>>
>>> Cheers,
>>>
>>> -Joe
>>
>