Cross Component (hotspot+jdk) Development in the Hotspot Group Repos

[Volker Simonis]

Just add build changes and you get another class of changes which often
require changing more than one repository in the forest.

I was (and still am :) a fan of the HotSpot Express model and used to work
and build Hotspot only for a long time. But after the introduction of the
new build system which resulted in dramatically improved build times I've
already switched to building whole forest for already quite some time now.

There's still a small question I have: what about other cross development
topics (e.g. top-level+jdk)? Will it be possible to "misuse" the new
hotspot repositories to keep such changes in sync? Or will other team
repositories like jdk-dev or jdk-client switch to such a model as well?

Thank you and best regards,
Volker

On Tuesday, September 9, 2014, Staffan Larsen <staffan.larsen at oracle.com>
wrote:

>
> ## tl;dr
>
> We propose a move to a Hotspot development model where we can do both
> hotspot and jdk changes in the hotspot group repos. This will require a
> fully populated JDK forest to push changes (whether hotspot or jdk
> changes) through JPRT. We do not expect these changes to have much
> affect on the open community, but it is good to note that there can be
> changes both in hotspot and jdk code coming through the hotspot
> repositories, and the best practise is to always clone and build the
> complete forest.
>
> We propose to do this change in a few weeks time.
>
> ## Problem
>
> We see an increasing number of features (small and large) that require
> concerted changes to both the hotspot and the jdk repos. Our current
> development model does not support this very well since it requires jdk
> changes to be made in jdk9/dev and hotspot changes to be made in the
> hotspot group repositories. Alternatively, such changes results in "flag
> days" where jdk and hotspot changes are pushed through the group repos
> with a lot of manual work and impact on everyone working in the group
> repos. Either way, the result is very slow and cumbersome development.
>
> Some examples where concerted changes have been required are JSR-292,
> default methods, Java Flight Recorder, work on annotations, moving Class
> fields to Java, many serviceability area tests, and so on. A lot of this
> work will continue and we will also see new things such as jigsaw that
> add to the mix.
>
> Doing concerted changes today takes a lot of manual effort and calendar
> time to make sure nothing break. In many cases the addition of a new
> feature needs to made first to a hotspot group repo. That change needs
> to propagate to jdk9/dev where library code can be changed to depend on
> it. Once that change has propagated back to the hotspot group repo, the
> final change can be made to remove the old implementation. This dance
> can take anywhere from 2 to 4 weeks to complete - for a single feature.
>
> There has also been quite a few cases where we missed taking the
> dependency into account which results in test failures in one or more
> repos. In some cases these failures go on for several weeks causing lots
> of extra work and confusion simply because it takes time for the fix to
> propagate through the repos.
>
> Instead, we want to move to a model where we can make both jdk and
> hotspot changes directly in the hotspot group repos. In that way the
> changes will always "travel together" through the repos. This will make
> our development cycle faster as well as more reliable.
>
> More or less by definition these type of changes introduce a stronger
> dependency between hotspot and the jdk. For the product as a whole to
> work correctly the right combination of hotspot and the jdk need to be
> used. We have long since removed the requirement that hotspot would
> support several jdk versions (known as the Hotspot Express - or hsx -
> model) and we continue to see a strong dependency, where matching code
> in hotspot and the jdk needs to be used.
>
> ## No More Dependency on Latest Promoted Build
>
> The strong dependency between hotspot and jdk makes it impossible for
> hotspot to depend on the latest promoted jdk build for testing and
> development. To elaborate on this; if a change with hotspot+jdk
> dependencies have been pushed to a group repo, it will not longer be
> possible to use the latest promoted build for running or testing the
> version of hotspot built in that repo -- the latest promoted build will
> not have the latest change to the jdk that hotspot now depends on (or
> vice versa).
>
> ##  Require Fully Populated JDK Forest
>
> The simple solution that we can switch to today is to always require a
> fully populated JDK forest when building (both locally and in JPRT). By
> this we mean a clone of all the repos in the forest under, for example,
> jdk9/hs-rt. JPRT would no longer be using the latest promoted build when
> creating bundles, instead it will build the code from the submitted
> forest.
>
> If all operations (builds, integrations, pushes, JPRT jobs) always work
> on the full forest, then there will never be a mismatch between the jdk
> and the hotspot code.
>
> The main drawbacks of this is that developers now need to clone, store
> and build a lot more code. Cloning the full forest takes longer than
> just cloning the hotspot forest. This can be alleviated by maintaining
> local cached versions. Storing full forests require more disk space.
> This can be mitigated by buying more disks or using a different workflow
> (for example Mercurial Queues). Building a full jdk takes longer, but
> hotspot is already one of the larger components to build and incremental
> builds are usually quite fast.
>
> ## Next Steps
>
> Given that we would like to improve the model we use for cross component
> development as soon as possible, we would like to switch to require a
> fully populated JDK forest for hotspot development. All the
> prerequisites for doing this are in place (changes to JPRT, both on the
> servers and to the configuration files in the source repos). A group of
> volunteering hotspot developers have been using full jdk repos for a
> while for day-to-day work (except pushes) and have not reported any
> showstopper problems.
>
> If no strong objections are rasied we need decide on a date when we
> throw the switch. A good date is probably after the 8u40 Feature
> Complete date of Mid September [0] so as not to impact that release
> (although this change will only apply to JDK 9 development for now).
>
> Regards,
> Jon Masamitsu, Karen Kinnear, Mikael Vidstedt,
> Staffan Larsen, Stefan Särne, Vladimir Kozlov
>
> [0] http://openjdk.java.net/projects/jdk8u/releases/8u40.html