Process proposal for Updating JDK 7u with Hotspot Express...

[Erik Trimble]

Hi folks.

This email is to descibe the the process of integrating Hotspot Express
versions into the JDK 7u series, and open a comment on the this process.
It's rather long, so please read it carefully.

Unlike work which is put into the other repositories in the jdk7u/
forest, Hotspot does not push individual (or small groups) of fixes in.
This is because the Hotspot Express model eschews fixing existing
Hotspot versions in favor of delivering fixes into a constantly-moving
development train, then pushing that development train back into
existing Update release (both 6 and 7, in this case).

This prevents the Hotspot folks from having a maintain a huge number of
disparate releases, and trying to backport fixes from the mainline
development train.  It also allows for the inclusion of advanced
features into older JDK versions.

---

The very latest Hotspot development version is always found here:

http://hg.openjdk.java.net/hsx/hotspot-main/hotspot

Current, latest Hotspot version is HSX22.0. A copy of the latest STABLE
version of HSX22 (i.e. one which has undergone a QA cycle) is found
here:

http://hg.openjdk.java.net/hsx/hsx22/hotspot


Now, normally, after a QA cycle has passed, and the contents of
hsx22/hotspot are refreshed with the newest stable bits, it is then
promoted into the OpenJDK 8 repository
( hg.openjdk.java.net/jdk8/jdk8/hotspot )

Using the Hotspot Express model in the 7u series involves these steps
[note that all references to '7u' or '7u-dev' refer to either
http://hg.openjdk.java.net/jdk7u/jdk7u or
http://hg.openjdk.java.net/jdk7u/jdk7u-dev, respectively]:

1.) Periodically, the contents of the latest hsx/hsxN/hotspot repository
(i.e. the version matching the one currently in
hsx/hotspot-main/hotspot ) will be put through a new QA cycle, as a
candidate for inclusion in the 7u series.

2.) To be explicit, the QA cycle using this Hotspot Express snapshot
will be using the latest 7u JDK, NOT the 8 JDK. Thus, we expect to
discover any 7-specific issues with Hotspot BEFORE it is pushed into the
Integration repository (7u-dev).  

2a.) In addition, there will never be a Hotspot->7u integration until
AFTER the same Hotspot version has been promoted into the JDK 8 forest,
and undergone a full Release promotion cycle. This will be to make sure
that the Hotspot version in hsx/hsxN/hotspot is indeed stabilized and we
have worked out any immediately apparent serious issues. This may very
well mean that Hotspot will not immediately promote all Hotspot builds
unto 7u.  E.g. HS20 b01 may go to JDK 8 Build 01, but if there are
problems detected, then HS22 will not be pushed into 7u until those
issues are addressed in a new HS build. So, it is entirely possible that
an integration into 7u will actually encompass several Hotspot build
numbers.

3.) At the beginning of this QA cycle, a webrev will be created,
detailing ALL the changes vs the existing 7u/hotspot repository. That
is, this will be a very large webrev, as it includes all the fixes
between the latest development Hotspot and the existing 7u Hotspot.

4.) This webrev will be posted to cr.openjdk.java.net, and a "request
for integration" notice will be sent to jdk7u-dev at openjdk.java.net, as
normal for other integration requests.

5.) The 7u Technical Lead will approve the Hotspot update, taking into
consideration the timing of the push - that is, approvals should be
concerned with whether a new Hotspot version is appropriate given build
schedules, NOT on the technical merits. 

6.) After the approval has been received, the Hotspot snapshot
undergoing QA will be pushed into the 7u-dev/hotspot Integration area.

7.) Upon receipt of the QA certification (PIT cert) that Hotspot has
successfully passed all relevant testing (or, there are only
inconsequential errors which aren't large enough to warrant a respin),
the contents of 7u-dev/hotspot will be pushed up to the Master area
(7u/hotspot ).

A couple of notes:

a) I expect that #3 and #4 will happen almost simultaneously. That is,
our internal snapshot scripts will submit the job to QA and generate the
webrev together.

b) #5 should likely happen very shortly after #4, probably the same day.

c) Frankly, technical discussion of the merits of the Hotspot push is
not expected to be possible here (in the context of the integration,
that is). The webrev is an informational post, not a "please review and
approve" post. This is due to the very large number of CRs being fixed
(several dozen or more), and the highly complex nature of all of them.
Such review is carried out when the code is first pushed into
hsx/hotspot-main and JDK8, so a duplicate review here is not necessary
and is unlikely to be reasonably possible within the limited timeframe
that an integration must have. (Translation: we can't post the webrev
and wait 2 weeks for people to fully review and comment on things before
integrating it).  

d) People interested in hotspot development (which will concern use of
Hotspot in ALL JDK versions, 6, 7, and 8), should subscribe to
hotspot-dev at openjdk.java.net. 

e) the 7uN repositories will NOT follow the above procedure - as they
are "stabilization-only" repositories, Hotspot developers will be
pushing changes directly to the /hotspot repository under the relevant
7uN forest in the same manner as all other stabilization fixes from
other JDK developers. Thus, all fixes for Hotspot for 7uN releases will
follow the standard JDK processes.


One specific point where I'm not sure how we want to proceed is this:

Should #6 (the push of the Hotspot snapshot into 7u-dev/hotspot) happen
right after approval from the Technical Lead happen? If so, then it
likely will be BEFORE QA has finished on the snapshot. This would be in
line with the other JDK fixes, since they do not undergo QA before being
pushed to 7u-dev/*.  However, Hotspot is "special", so do we care to be
extra sure that 7u-dev/hotspot is stable?  

Note that code being pushed to 7u-dev/hotspot will ALWAYS have passed a
JPRT build (the basic all-platform build and sanity-check system
internal to Oracle), so even if we do chose to push the Hotspot snapshot
into 7u-dev before the QA cycle completes, we have reasonable assurances
that it works (at the very least, will build).

The push from 7u-dev/hotspot to 7u/hotspot will NEVER occur until a
passed QA cycle (PIT) has occurred. 

Do people think it's OK to pushed to 7u-dev before PIT completes or not?
If not, then the push to 7u-dev/hotspot will occur just before the push
to 7u/hotspot (i.e. in essentially the same operation).

---

If you have any comments or questions above the above, please speak up
NOW. I've tried to be very clear about this process, and I think
everything above is both reasonable and clear, but it may not be to some
folks. Please ask or comment Right Now. I'd like to get this comment
period over by 12pm (noon) Pacific Time, Friday, 12 August, so we can do
the first Hotspot->7u integration later that day (so the RE folks can do
a 7u build over the weekend).




-- 
Erik Trimble
Java Platform Group - Infrastructure
Mailstop:  usca22-317
Phone:  x67195
Santa Clara, CA
Timezone: US/Pacific (UTC-0800)