Proposal to revise forest graph and integration practices for JDK 9

[Dmitry Samersoff]

Staffan,

tl is very busy forest.

So if we have fixes A1 ... A5 in hotspot-rt and B1 ... B30 in tl, we
have to repeat all JDK tests because we tested JDK fix with obsoleted
codebase. Of course it's not always true, but it might be a problem.

   Also, for hotspot we have a method (jprt build) to confirm that merge
is correct before actually push the changes to ws.
   For JDK we don't have such method yet so we should either keep tl
locket for at least one nightly after merge or it's possible for
incorrect merge to spread across developers.

I think pushing hotspot changes to tl is much safer than push jdk
changes to rt. But the best (IMHO) just have one workspace.

-Dmitry



On 2013-12-02 17:19, Staffan Larsen wrote:
> Dmitry,
> 
> We could (and should) increase the set of JDK tests we run in hotspot-rt nightlies.
> 
> /Staffan
> 
> On 2 dec 2013, at 06:50, Dmitry Samersoff <dmitry.samersoff at oracle.com> wrote:
> 
>> Joe,
>>
>> Could we at least merge tl and hotspot-rt workspaces to a single one -
>> tl-hotspot-rt?
>>
>> Pushing JDK fixes to hotspot-rt and than merge it into tl a week or two
>> later could be a pain because hotspot-rt has limited set of JDK tests in
>> nightly and lots of changes happens in tl within a week.
>>
>> -Dmitry
>>
>> On 2013-12-02 09:04, Joe Darcy wrote:
>>> Hi Dalibor,
>>>
>>> On 11/25/2013 5:40 AM, Dalibor Topic wrote:
>>>> On 11/23/13 5:34 PM, Joe Darcy wrote:
>>>>> Since more projects requiring cross-repository coordination are
>>>>> expected in the future, I'm proposing a number of changes to the
>>>>> forest structure and management policies in JDK 9 to reduce the
>>>>> propagation delays.
>>>> This seems very close to how 7u works already, so it's a huge
>>>> improvement over 8, but it comes with some historical baggage:
>>>>
>>>>> By having team forests integrate directly into dev as well as having
>>>>> many libraries developers pushing directly to dev, the dev forest
>>>>> serves as an active collaboration area with greatly reduced
>>>>> propagation times across the whole system. With this model there is
>>>>> less cross-team isolation; teams and individuals are responsible for
>>>>> promptly fixing any breakage which is introduced. If problems are not
>>>>> quickly addressed, a problematic changeset may be anti-delta'ed.
>>>> One of the useful things we did for 7u was to try to drive down the
>>>> number of team forests to just -dev. We didn't quite succeed, as you
>>>> can see from the hotspot team having their own 7u integration forest,
>>>> but overall it was a major step in driving the complexity of
>>>> (un)necessary interactions down, and enabling fixes to propagate faster.
>>>>
>>>> I would suggest that you consider doing something similar for JDK 9,
>>>> as the spider web [0] of JDK 8's forests currently contains about a
>>>> dozen of forests, and from the way the proposal is written it seems
>>>> that you may like to do that, but unfortunately the proposal is not
>>>> very explicit about it.
>>>>
>>>> So, I'd like to suggest that jdk9/dev replaces the equivalent of
>>>> jdk9/tl, jdk9/awt, jdk9/swing, jdk9/2d, jdk9/build, jdk9/deploy,
>>>> jdk9/l10n, jdk9/swing, nashorn/jdk9 in one go.
>>>
>>> For the open components, that is essentially the proposal. Deploy and
>>> install are closed and have different testing requirements from other
>>> code so would likely remain separate.
>>>
>>>>  If long term features require a throw-away integration forest,
>>>> create one as you need it for as long as you need it, integrate and
>>>> close that forest.
>>>>
>>>> I don't have an opinion on pruning hotspot forests (yet;), but you may
>>>> want to consider where a fix containing changes both for hotspot and
>>>> libraries would need to go to if you retain the current structure -
>>>> gc? rt? hotspot-dev? hsx25? main? comp?
>>>
>>> At least in JDK 8, I believe most of libs + HotSpot interactions were in
>>> runtime, so a push to rt would be most appropriate in that case. In
>>> general, I would recommend a coordinated push go into the HotSpot
>>> integration forest with the closest interaction.
>>>
>>>>
>>>>> (Conceptually, in this model a separate master forest is not strictly
>>>>> needed since the known-good states could be indicated using a
>>>>> mechanism like Hg tags. However, while adjusting to the new model and
>>>>> to allow for fixes directly to master in exceptional circumstances,
>>>>> I'm proposing a physically separate master forest be retained. The
>>>>> distinct URL of master will also clearly indicate known-good states
>>>>> of the source code.)
>>>> One of the not very useful things we did for 7u was to keep a separate
>>>> master forest.
>>>>
>>>> Considering the real benefits of a master forest [1], and the real
>>>> drawbacks when it's being used with a rapidly changing tree underneath
>>>> [2], I'd suggest you drop the master forest for JDK 9.
>>>
>>> Thanks for the feedback on that point of the proposal; I'm still
>>> uncomfortable with collapsing master and dev before we put the model
>>> into practice.
>>>
>>>>
>>>> One thing missing from the proposal is jdk9 repository layout within
>>>> the forests. I think that this may be good opportunity to simplify the
>>>> current historically grown structure, for example by folding the
>>>> nashorn and langtools repositories together, as well as corba, jaxp,
>>>> jaxws and jdk.
>>>
>>> Yes, that would be another level of reorganization to discuss.
>>>
>>> Cheers,
>>>
>>> -Joe
>>
>>
>> -- 
>> Dmitry Samersoff
>> Oracle Java development team, Saint Petersburg, Russia
>> * I would love to change the world, but they won't give me the source code.
> 


-- 
Dmitry Samersoff
Oracle Java development team, Saint Petersburg, Russia
* I would love to change the world, but they won't give me the sources.