Cross Component (hotspot+jdk) Development in the Hotspot Group Repos

Coleen Phillimore coleen.phillimore at oracle.com
Tue Sep 9 14:36:28 UTC 2014


Hi,

Is there a definitive guide on how to build the entire JDK on all 
platforms that is available on the open mailing lists?
http://openjdk.java.net/guide/ is all I found.   I don't know how to do 
this on Windows, for example.

Thanks,
Coleen

On 9/9/14, 2:02 AM, Staffan Larsen wrote:
> ## tl;dr
>
> We propose a move to a Hotspot development model where we can do both
> hotspot and jdk changes in the hotspot group repos. This will require a
> fully populated JDK forest to push changes (whether hotspot or jdk
> changes) through JPRT. We do not expect these changes to have much
> affect on the open community, but it is good to note that there can be
> changes both in hotspot and jdk code coming through the hotspot
> repositories, and the best practise is to always clone and build the
> complete forest.
>
> We propose to do this change in a few weeks time.
>
> ## Problem
>
> We see an increasing number of features (small and large) that require
> concerted changes to both the hotspot and the jdk repos. Our current
> development model does not support this very well since it requires jdk
> changes to be made in jdk9/dev and hotspot changes to be made in the
> hotspot group repositories. Alternatively, such changes results in "flag
> days" where jdk and hotspot changes are pushed through the group repos
> with a lot of manual work and impact on everyone working in the group
> repos. Either way, the result is very slow and cumbersome development.
>
> Some examples where concerted changes have been required are JSR-292,
> default methods, Java Flight Recorder, work on annotations, moving Class
> fields to Java, many serviceability area tests, and so on. A lot of this
> work will continue and we will also see new things such as jigsaw that
> add to the mix.
>
> Doing concerted changes today takes a lot of manual effort and calendar
> time to make sure nothing break. In many cases the addition of a new
> feature needs to made first to a hotspot group repo. That change needs
> to propagate to jdk9/dev where library code can be changed to depend on
> it. Once that change has propagated back to the hotspot group repo, the
> final change can be made to remove the old implementation. This dance
> can take anywhere from 2 to 4 weeks to complete - for a single feature.
>
> There has also been quite a few cases where we missed taking the
> dependency into account which results in test failures in one or more
> repos. In some cases these failures go on for several weeks causing lots
> of extra work and confusion simply because it takes time for the fix to
> propagate through the repos.
>
> Instead, we want to move to a model where we can make both jdk and
> hotspot changes directly in the hotspot group repos. In that way the
> changes will always "travel together" through the repos. This will make
> our development cycle faster as well as more reliable.
>
> More or less by definition these type of changes introduce a stronger
> dependency between hotspot and the jdk. For the product as a whole to
> work correctly the right combination of hotspot and the jdk need to be
> used. We have long since removed the requirement that hotspot would
> support several jdk versions (known as the Hotspot Express - or hsx -
> model) and we continue to see a strong dependency, where matching code
> in hotspot and the jdk needs to be used.
>
> ## No More Dependency on Latest Promoted Build
>
> The strong dependency between hotspot and jdk makes it impossible for
> hotspot to depend on the latest promoted jdk build for testing and
> development. To elaborate on this; if a change with hotspot+jdk
> dependencies have been pushed to a group repo, it will not longer be
> possible to use the latest promoted build for running or testing the
> version of hotspot built in that repo -- the latest promoted build will
> not have the latest change to the jdk that hotspot now depends on (or
> vice versa).
>
> ##  Require Fully Populated JDK Forest
>
> The simple solution that we can switch to today is to always require a
> fully populated JDK forest when building (both locally and in JPRT). By
> this we mean a clone of all the repos in the forest under, for example,
> jdk9/hs-rt. JPRT would no longer be using the latest promoted build when
> creating bundles, instead it will build the code from the submitted
> forest.
>
> If all operations (builds, integrations, pushes, JPRT jobs) always work
> on the full forest, then there will never be a mismatch between the jdk
> and the hotspot code.
>
> The main drawbacks of this is that developers now need to clone, store
> and build a lot more code. Cloning the full forest takes longer than
> just cloning the hotspot forest. This can be alleviated by maintaining
> local cached versions. Storing full forests require more disk space.
> This can be mitigated by buying more disks or using a different workflow
> (for example Mercurial Queues). Building a full jdk takes longer, but
> hotspot is already one of the larger components to build and incremental
> builds are usually quite fast.
>
> ## Next Steps
>
> Given that we would like to improve the model we use for cross component
> development as soon as possible, we would like to switch to require a
> fully populated JDK forest for hotspot development. All the
> prerequisites for doing this are in place (changes to JPRT, both on the
> servers and to the configuration files in the source repos). A group of
> volunteering hotspot developers have been using full jdk repos for a
> while for day-to-day work (except pushes) and have not reported any
> showstopper problems.
>
> If no strong objections are rasied we need decide on a date when we
> throw the switch. A good date is probably after the 8u40 Feature
> Complete date of Mid September [0] so as not to impact that release
> (although this change will only apply to JDK 9 development for now).
>
> Regards,
> Jon Masamitsu, Karen Kinnear, Mikael Vidstedt,
> Staffan Larsen, Stefan Särne, Vladimir Kozlov
>
> [0] http://openjdk.java.net/projects/jdk8u/releases/8u40.html



More information about the hotspot-dev mailing list