Proposal: Allowing selective pushes to hotspot without jprt

Wed Nov 19 08:23:44 UTC 2014

On Wed, Nov 19, 2014 at 2:53 AM, David Holmes <david.holmes at oracle.com> wrote:
> On 18/11/2014 8:03 PM, Volker Simonis wrote:
>>
>> On Mon, Nov 10, 2014 at 3:38 PM, Mikael Vidstedt
>> <mikael.vidstedt at oracle.com> wrote:
>>>
>>>
>>> On 2014-11-07 14:12, Volker Simonis wrote:
>>>>
>>>>
>>>> On Fri, Nov 7, 2014 at 12:56 PM, Mikael Vidstedt
>>>> <mikael.vidstedt at oracle.com> wrote:
>>>>>
>>>>>
>>>>> Volker,
>>>>>
>>>>> Thanks for reminding me, this totally slipped my mind.
>>>>>
>>>>> I think it's fair to say say we've given this enough time for feedback,
>>>>> and
>>>>> that the feedback has been all supportive. With that in mind I consider
>>>>> the
>>>>> proposal approved and effective immediately!
>>>>>
>>>> OK great. So does this mean we can now push reviewed changes to the
>>>> ppc/aix subdirs right away?
>>>
>>>
>>>
>>> That is indeed the idea - modulo the "if at review code review time a
>>> change
>>> is for some reason deemed to be risky and/or otherwise have impact on
>>> shared
>>> files" part which, again, hopefully is rare.
>>>
>>
>> You're right - it works!
>> I've just pushed my first AIX-only change to hotspot-rt!
>
>
> Congratulations!
>
> Unfortunately it caused us a problem as now the repos can change whilst a
> job is going through JPRT - this requires a new merge due to multiple heads
> and so triggered a failure. But Mikael is working on it :)

It's astonishing how even the smallest changes can introduce
non-foreseeable problems.

Hopefully Mikael will be able to fix the problem somehow (if he didn't
had too much of the champagne already :)

Regards,
Volker

>
> David
>
>
>> Thanks,
>> Volker
>>
>>> Cheers,
>>> Mikael
>>>
>>>
>>>>
>>>>> Cheers,
>>>>> Mikael
>>>>>
>>>>>
>>>>> On 2014-11-06 15:35, Volker Simonis wrote:
>>>>>>
>>>>>>
>>>>>> Hi Mikael,
>>>>>>
>>>>>> just wanted to ask what's the status of this project?
>>>>>> I hope it was not just a JavaOne hoax :)
>>>>>>
>>>>>> Regards,
>>>>>> Volker
>>>>>>
>>>>>>
>>>>>> On Fri, Sep 19, 2014 at 8:47 PM, Volker Simonis
>>>>>> <volker.simonis at gmail.com> wrote:
>>>>>>>
>>>>>>>
>>>>>>> Thanks Mikael, that sounds good!
>>>>>>>
>>>>>>> Regards,
>>>>>>> Volker
>>>>>>>
>>>>>>>
>>>>>>> On Fri, Sep 19, 2014 at 8:03 PM, Mikael Vidstedt
>>>>>>> <mikael.vidstedt at oracle.com> wrote:
>>>>>>>>
>>>>>>>>
>>>>>>>> Volker,
>>>>>>>>
>>>>>>>> The proposal is only to change how the changes are pushed, not which
>>>>>>>> forests
>>>>>>>> changes can be pushed to. That is, we would still require hotspot
>>>>>>>> changes to
>>>>>>>> be pushed to one of the group repositories (jdk9/hs-{comp,gc,rt}) or
>>>>>>>> to
>>>>>>>> the
>>>>>>>> jdk8u/hs-dev forest (jdk8u), but I propose that the relaxation be
>>>>>>>> applied on
>>>>>>>> all those (four) forests. Reasonable?
>>>>>>>>
>>>>>>>> Cheers,
>>>>>>>> Mikael
>>>>>>>>
>>>>>>>>
>>>>>>>> On 2014-09-12 11:38, Volker Simonis wrote:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Hi Mikael,
>>>>>>>>>
>>>>>>>>> there's one more question that came to my mind: will the new rule
>>>>>>>>> apply to all hotspot respitories (i.e. jdk9/hs-rt/hotspot,
>>>>>>>>> jdk9/hs-comp/hotspot, jdk9/hs-gc/hotspot, jdk9/hs-hs/hotspot AND
>>>>>>>>> jdk8u/jdk8u-dev/hotspot, jdk8u/hs-dev/hotspot) ?
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> Volker
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Thu, Sep 11, 2014 at 12:16 AM, Mikael Vidstedt
>>>>>>>>> <mikael.vidstedt at oracle.com> wrote:
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Andrew/Volker,
>>>>>>>>>>
>>>>>>>>>> Thanks for the positive feedback. The goal of the proposal is to
>>>>>>>>>> simplify
>>>>>>>>>> pushing changes which are effectively not tested by the jprt
>>>>>>>>>> system
>>>>>>>>>> anyway.
>>>>>>>>>> The proposed relaxation would not affect work on other
>>>>>>>>>> infrastructure
>>>>>>>>>> projects in any relevant way, but would hopefully improve all our
>>>>>>>>>> lives
>>>>>>>>>> significantly immediately.
>>>>>>>>>>
>>>>>>>>>> Cheers,
>>>>>>>>>> Mikael
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On 2014-09-10 01:45, Volker Simonis wrote:
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Hi Mikael,
>>>>>>>>>>>
>>>>>>>>>>> thanks a lot for this proposal. I think this will dramatically
>>>>>>>>>>> simplify our work to keep our ports up to date! So I fully
>>>>>>>>>>> support
>>>>>>>>>>> it.
>>>>>>>>>>>
>>>>>>>>>>> Nevertheless, I think this can only be a first step towards fully
>>>>>>>>>>> open
>>>>>>>>>>> the JPRT system to developers outside Oracle. With "opening" I
>>>>>>>>>>> mean
>>>>>>>>>>> to
>>>>>>>>>>> allow OpenJDK commiters from outside Oracle to submit and run
>>>>>>>>>>> JPRT
>>>>>>>>>>> jobs as well as allowing porting projects to add hardware which
>>>>>>>>>>> builds
>>>>>>>>>>> and tests the HotSpot on alternative platforms.
>>>>>>>>>>>
>>>>>>>>>>> So while I'm all in favor of your proposal I hope you can allay
>>>>>>>>>>> my
>>>>>>>>>>> doubts that this simplification will hopefully not push the
>>>>>>>>>>> realization of a truly OPEN JPRT system even further away.
>>>>>>>>>>>
>>>>>>>>>>> Regards,
>>>>>>>>>>> Volker
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Tue, Sep 9, 2014 at 11:24 PM, Mikael Vidstedt
>>>>>>>>>>> <mikael.vidstedt at oracle.com> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> All,
>>>>>>>>>>>>
>>>>>>>>>>>> Made up primarily of low level C++ code, the Hotspot codebase is
>>>>>>>>>>>> highly
>>>>>>>>>>>> platform dependent and also tightly coupled with the tool chains
>>>>>>>>>>>> on
>>>>>>>>>>>> the
>>>>>>>>>>>> various platforms. Each platform/tool chain combination has its
>>>>>>>>>>>> set
>>>>>>>>>>>> of
>>>>>>>>>>>> special quirks, and code must be implemented in a way such that
>>>>>>>>>>>> it
>>>>>>>>>>>> only
>>>>>>>>>>>> relies on the common subset of syntax and functionality across
>>>>>>>>>>>> all
>>>>>>>>>>>> these
>>>>>>>>>>>> combinations. History has taught us that even simple changes can
>>>>>>>>>>>> have
>>>>>>>>>>>> surprising results when compiled with different compilers.
>>>>>>>>>>>>
>>>>>>>>>>>> For more than a decade the Hotspot team has ensured a minimum
>>>>>>>>>>>> quality
>>>>>>>>>>>> level
>>>>>>>>>>>> by requiring all pushes to be done through a build and test
>>>>>>>>>>>> system
>>>>>>>>>>>> (jprt)
>>>>>>>>>>>> which guarantees that the code resulting from applying a set of
>>>>>>>>>>>> changes
>>>>>>>>>>>> builds on a set of core platforms and that a set of core tests
>>>>>>>>>>>> pass.
>>>>>>>>>>>> Only
>>>>>>>>>>>> if
>>>>>>>>>>>> all the builds and tests pass will the changes actually be
>>>>>>>>>>>> pushed
>>>>>>>>>>>> to
>>>>>>>>>>>> the
>>>>>>>>>>>> target repository.
>>>>>>>>>>>>
>>>>>>>>>>>> We believe that testing like the above, in combination with
>>>>>>>>>>>> later
>>>>>>>>>>>> stages
>>>>>>>>>>>> of
>>>>>>>>>>>> testing, is vital to ensuring that the quality level of the
>>>>>>>>>>>> Hotspot
>>>>>>>>>>>> code
>>>>>>>>>>>> remains high and that developers do not run into situations
>>>>>>>>>>>> where
>>>>>>>>>>>> the
>>>>>>>>>>>> latest
>>>>>>>>>>>> version has build errors on some platforms.
>>>>>>>>>>>>
>>>>>>>>>>>> Recently the AIX/PPC port was added to the set of OpenJDK
>>>>>>>>>>>> platforms.
>>>>>>>>>>>> From
>>>>>>>>>>>> a
>>>>>>>>>>>> Hotspot perspective this new platform added a set of AIX/PPC
>>>>>>>>>>>> specific
>>>>>>>>>>>> files
>>>>>>>>>>>> including some platform specific changes to shared code. The
>>>>>>>>>>>> AIX/PPC
>>>>>>>>>>>> platform is not tested by Oracle as part of Hotspot push jobs.
>>>>>>>>>>>> The
>>>>>>>>>>>> same
>>>>>>>>>>>> thing applies for the shark and zero versions of Hotspot.
>>>>>>>>>>>>
>>>>>>>>>>>> While Hotspot developers remain committed to making sure changes
>>>>>>>>>>>> are
>>>>>>>>>>>> developed in a way such that the quality level remains high
>>>>>>>>>>>> across
>>>>>>>>>>>> all
>>>>>>>>>>>> platforms and variants, because of the above mentioned
>>>>>>>>>>>> complexities
>>>>>>>>>>>> it
>>>>>>>>>>>> is
>>>>>>>>>>>> inevitable that from time to time changes will be made which
>>>>>>>>>>>> introduce
>>>>>>>>>>>> issues on specific platforms or tool chains not part of the core
>>>>>>>>>>>> testing.
>>>>>>>>>>>>
>>>>>>>>>>>> To allow these issues to be resolved more quickly I would like
>>>>>>>>>>>> to
>>>>>>>>>>>> propose
>>>>>>>>>>>> a
>>>>>>>>>>>> relaxation in the requirements on how changes to Hotspot are
>>>>>>>>>>>> pushed.
>>>>>>>>>>>> Specifically I would like to allow for direct pushes to the
>>>>>>>>>>>> hotspot/
>>>>>>>>>>>> repository of files specific to the following
>>>>>>>>>>>> ports/variants/tools:
>>>>>>>>>>>>
>>>>>>>>>>>> * AIX
>>>>>>>>>>>> * PPC
>>>>>>>>>>>> * Shark
>>>>>>>>>>>> * Zero
>>>>>>>>>>>>
>>>>>>>>>>>> Today this translates into the following files:
>>>>>>>>>>>>
>>>>>>>>>>>> - src/cpu/ppc/**
>>>>>>>>>>>> - src/cpu/zero/**
>>>>>>>>>>>> - src/os/aix/**
>>>>>>>>>>>> - src/os_cpu/aix_ppc/**
>>>>>>>>>>>> - src/os_cpu/bsd_zero/**
>>>>>>>>>>>> - src/os_cpu/linux_ppc/**
>>>>>>>>>>>> - src/os_cpu/linux_zero/**
>>>>>>>>>>>>
>>>>>>>>>>>> Note that all changes are still required to go through the
>>>>>>>>>>>> normal
>>>>>>>>>>>> development and review cycle; the proposed relaxation only
>>>>>>>>>>>> applies
>>>>>>>>>>>> to
>>>>>>>>>>>> how
>>>>>>>>>>>> the changes are pushed.
>>>>>>>>>>>>
>>>>>>>>>>>> If at code review time a change is for some reason deemed to be
>>>>>>>>>>>> risky
>>>>>>>>>>>> and/or
>>>>>>>>>>>> otherwise have impact on shared files the reviewer may request
>>>>>>>>>>>> that
>>>>>>>>>>>> the
>>>>>>>>>>>> change to go through the regular push testing. For changes only
>>>>>>>>>>>> touching
>>>>>>>>>>>> the
>>>>>>>>>>>> above set of files this expected to be rare.
>>>>>>>>>>>>
>>>>>>>>>>>> Please let me know what you think.
>>>>>>>>>>>>
>>>>>>>>>>>> Cheers,
>>>>>>>>>>>> Mikael
>>>>>>>>>>>>
>>>
>