Process proposal for Updating JDK 7u with Hotspot Express...

[Poonam Bajaj]

On 8/11/2011 11:40 PM, Poonam Bajaj wrote:
> Hi Erik,
>
>> e) the 7uN repositories will NOT follow the above procedure - as they
>> are "stabilization-only" repositories, Hotspot developers will be
>> pushing changes directly to the /hotspot repository under the relevant
>> 7uN forest in the same manner as all other stabilization fixes from
>> other JDK developers. Thus, all fixes for Hotspot for 7uN releases will
>> follow the standard JDK processes.
> How will the versioning of hotspot under 7uN be handled ? Will we add 
> minor version number
> to it or will just increment its build number ?
>
> e.g. if 7u2 gets hs22-b01 and integrations(stabilization fixes) are 
> later made to 7u2/hotspot,
> what will be the 'hotspot' version after 7u2 promotion - something 
> like hs22.1(or an appropriate
> higher minor version number) or hs22-b02 (or a higher build number 
> depending upon the builds
> we'd have) ?
Answering part of my own question, we can not have HSxx-bxx to represent
the hotspot verisons in 7uN as that represents the builds from the mainline
Hotspot Express development train.

>> Should #6 (the push of the Hotspot snapshot into 7u-dev/hotspot) happen
>> right after approval from the Technical Lead happen? If so, then it
>> likely will be BEFORE QA has finished on the snapshot. This would be in
>> line with the other JDK fixes, since they do not undergo QA before being
>> pushed to 7u-dev/*.  However, Hotspot is "special", so do we care to be
>> extra sure that 7u-dev/hotspot is stable?  
> The changes going in with Hotspot push would be significantly large 
> and so can not be
> compared with the other individual JDK fixes going into 7u-dev 
> (without undergoing QA).
> I think we should be extra careful with hotspot and should be sure 
> that things are stable before
> pushing hotspot-snapshot to 7u-dev/hotspot.
>

Thanks,
Poonam

> Thanks,
> Poonam
>
> On 8/11/2011 7:46 AM, Erik Trimble wrote:
>> Hi folks.
>>
>> This email is to descibe the the process of integrating Hotspot Express
>> versions into the JDK 7u series, and open a comment on the this process.
>> It's rather long, so please read it carefully.
>>
>> Unlike work which is put into the other repositories in the jdk7u/
>> forest, Hotspot does not push individual (or small groups) of fixes in.
>> This is because the Hotspot Express model eschews fixing existing
>> Hotspot versions in favor of delivering fixes into a constantly-moving
>> development train, then pushing that development train back into
>> existing Update release (both 6 and 7, in this case).
>>
>> This prevents the Hotspot folks from having a maintain a huge number of
>> disparate releases, and trying to backport fixes from the mainline
>> development train.  It also allows for the inclusion of advanced
>> features into older JDK versions.
>>
>> ---
>>
>> The very latest Hotspot development version is always found here:
>>
>> http://hg.openjdk.java.net/hsx/hotspot-main/hotspot
>>
>> Current, latest Hotspot version is HSX22.0. A copy of the latest STABLE
>> version of HSX22 (i.e. one which has undergone a QA cycle) is found
>> here:
>>
>> http://hg.openjdk.java.net/hsx/hsx22/hotspot
>>
>>
>> Now, normally, after a QA cycle has passed, and the contents of
>> hsx22/hotspot are refreshed with the newest stable bits, it is then
>> promoted into the OpenJDK 8 repository
>> ( hg.openjdk.java.net/jdk8/jdk8/hotspot )
>>
>> Using the Hotspot Express model in the 7u series involves these steps
>> [note that all references to '7u' or '7u-dev' refer to either
>> http://hg.openjdk.java.net/jdk7u/jdk7u or
>> http://hg.openjdk.java.net/jdk7u/jdk7u-dev, respectively]:
>>
>> 1.) Periodically, the contents of the latest hsx/hsxN/hotspot repository
>> (i.e. the version matching the one currently in
>> hsx/hotspot-main/hotspot ) will be put through a new QA cycle, as a
>> candidate for inclusion in the 7u series.
>>
>> 2.) To be explicit, the QA cycle using this Hotspot Express snapshot
>> will be using the latest 7u JDK, NOT the 8 JDK. Thus, we expect to
>> discover any 7-specific issues with Hotspot BEFORE it is pushed into the
>> Integration repository (7u-dev). 
>> 2a.) In addition, there will never be a Hotspot->7u integration until
>> AFTER the same Hotspot version has been promoted into the JDK 8 forest,
>> and undergone a full Release promotion cycle. This will be to make sure
>> that the Hotspot version in hsx/hsxN/hotspot is indeed stabilized and we
>> have worked out any immediately apparent serious issues. This may very
>> well mean that Hotspot will not immediately promote all Hotspot builds
>> unto 7u.  E.g. HS20 b01 may go to JDK 8 Build 01, but if there are
>> problems detected, then HS22 will not be pushed into 7u until those
>> issues are addressed in a new HS build. So, it is entirely possible that
>> an integration into 7u will actually encompass several Hotspot build
>> numbers.
>>
>> 3.) At the beginning of this QA cycle, a webrev will be created,
>> detailing ALL the changes vs the existing 7u/hotspot repository. That
>> is, this will be a very large webrev, as it includes all the fixes
>> between the latest development Hotspot and the existing 7u Hotspot.
>>
>> 4.) This webrev will be posted to cr.openjdk.java.net, and a "request
>> for integration" notice will be sent to jdk7u-dev at openjdk.java.net, as
>> normal for other integration requests.
>>
>> 5.) The 7u Technical Lead will approve the Hotspot update, taking into
>> consideration the timing of the push - that is, approvals should be
>> concerned with whether a new Hotspot version is appropriate given build
>> schedules, NOT on the technical merits.
>> 6.) After the approval has been received, the Hotspot snapshot
>> undergoing QA will be pushed into the 7u-dev/hotspot Integration area.
>>
>> 7.) Upon receipt of the QA certification (PIT cert) that Hotspot has
>> successfully passed all relevant testing (or, there are only
>> inconsequential errors which aren't large enough to warrant a respin),
>> the contents of 7u-dev/hotspot will be pushed up to the Master area
>> (7u/hotspot ).
>>
>> A couple of notes:
>>
>> a) I expect that #3 and #4 will happen almost simultaneously. That is,
>> our internal snapshot scripts will submit the job to QA and generate the
>> webrev together.
>>
>> b) #5 should likely happen very shortly after #4, probably the same day.
>>
>> c) Frankly, technical discussion of the merits of the Hotspot push is
>> not expected to be possible here (in the context of the integration,
>> that is). The webrev is an informational post, not a "please review and
>> approve" post. This is due to the very large number of CRs being fixed
>> (several dozen or more), and the highly complex nature of all of them.
>> Such review is carried out when the code is first pushed into
>> hsx/hotspot-main and JDK8, so a duplicate review here is not necessary
>> and is unlikely to be reasonably possible within the limited timeframe
>> that an integration must have. (Translation: we can't post the webrev
>> and wait 2 weeks for people to fully review and comment on things before
>> integrating it). 
>> d) People interested in hotspot development (which will concern use of
>> Hotspot in ALL JDK versions, 6, 7, and 8), should subscribe to
>> hotspot-dev at openjdk.java.net.
>> e) the 7uN repositories will NOT follow the above procedure - as they
>> are "stabilization-only" repositories, Hotspot developers will be
>> pushing changes directly to the /hotspot repository under the relevant
>> 7uN forest in the same manner as all other stabilization fixes from
>> other JDK developers. Thus, all fixes for Hotspot for 7uN releases will
>> follow the standard JDK processes.
>>
>>
>> One specific point where I'm not sure how we want to proceed is this:
>>
>> Should #6 (the push of the Hotspot snapshot into 7u-dev/hotspot) happen
>> right after approval from the Technical Lead happen? If so, then it
>> likely will be BEFORE QA has finished on the snapshot. This would be in
>> line with the other JDK fixes, since they do not undergo QA before being
>> pushed to 7u-dev/*.  However, Hotspot is "special", so do we care to be
>> extra sure that 7u-dev/hotspot is stable? 
>> Note that code being pushed to 7u-dev/hotspot will ALWAYS have passed a
>> JPRT build (the basic all-platform build and sanity-check system
>> internal to Oracle), so even if we do chose to push the Hotspot snapshot
>> into 7u-dev before the QA cycle completes, we have reasonable assurances
>> that it works (at the very least, will build).
>>
>> The push from 7u-dev/hotspot to 7u/hotspot will NEVER occur until a
>> passed QA cycle (PIT) has occurred.
>> Do people think it's OK to pushed to 7u-dev before PIT completes or not?
>> If not, then the push to 7u-dev/hotspot will occur just before the push
>> to 7u/hotspot (i.e. in essentially the same operation).
>>
>> ---
>>
>> If you have any comments or questions above the above, please speak up
>> NOW. I've tried to be very clear about this process, and I think
>> everything above is both reasonable and clear, but it may not be to some
>> folks. Please ask or comment Right Now. I'd like to get this comment
>> period over by 12pm (noon) Pacific Time, Friday, 12 August, so we can do
>> the first Hotspot->7u integration later that day (so the RE folks can do
>> a 7u build over the weekend).
>>
>>
>>
>>
>>