Bug System Pilot Dev Workflow: DRAFT Accepted/Understood

Tue Jan 17 15:49:18 UTC 2012

On 01/17/12 01:46, David Holmes wrote:
> On 17/01/2012 6:52 PM, David Holmes wrote:
>> Just one comment on this:
>>
>> > We rename "Accepted" to "Open" with the following definition: Initial
>> > triage has determined that the bug/feature request contains
>> > sufficient information to estimate the basic feasibility of
>> > addressing the problem and to prioritize the request. If this is a
>> > bug, there is no guarantee that the problem has been reproduced.
>>
>> I find those two statements somewhat contradictory. If the problem may
>> not have even been reproduced how can you have triaged it enough to have
>> "sufficient information to estimate the basic feasibility of addressing
>> the problem and to prioritize the request" ?
>
> Okay maybe not so contradictory in some cases eg if external triage 
> has already been done.
>
> I think what I'm missing from this definition is understanding the 
> criteria for moving from New to Open.
I don't know that you can define a one-size-fits-all criterion.

I think you're conceptually talking about a transfer of responsibility. 
A New issue primarily belongs to somebody in the "initial evaluator" 
role. The initial evaluator is responsible for working with the filer to 
ensure the report is reasonably complete, and evaluate the issues to the 
best of his/her ability. When moving the bug to Open, the primary 
responsibility is transferred to the Development role. The reason for 
transfer can be anything from "I need your help deciding if this is an 
issue and if so, how important it is" to "I need this fixed, yesterday".

How, much the initial evaluator can/should do in terms of reproducing 
and evaluating priority depends on skills, resources, complexity, 
experience, authority, etc.

At least, you can't define a much more specific criteria, than the 
proposed definition, without first defining "who" does it. And it need 
not always be the same. For issues filed against a release under 
development, it's typically a lead/architect. For a release that has 
already shipped, it might be a support function.

>
> David
>
>> David
>> -----
>>
>> On 17/01/2012 5:43 PM, Iris Clark wrote:
>>> Hi!
>>>
>>> First, thanks for all for the feedback. It's been very interesting
>>> reading it over the past few weeks. I've had a lot to think about.
>>> Let's start to wrap this topic up. We've got lots more to cover.
>>>
>>> For each area or topic of discussion, I'll be providing a summary of
>>> the various arguments, a decision for what the pilot should look like,
>>> and reasons for that decision. This message is about the "Accepted"
>>> and "Understood" statuses.
>>>
>>> Expect an update to the DRAFT to reflect this decision.
>>>
>>> Thanks,
>>> iris
>>>
>>> DECISION
>>>
>>> Based on the extensive discussion around the proposed JIRA "Accepted"
>>> and "Understood" status of the Pilot Developer Workflow, here's what I
>>> think should be done for the pilot.
>>>
>>> We rename "Accepted" to "Open" with the following definition: Initial
>>> triage has determined that the bug/feature request contains sufficient
>>> information to estimate the basic feasibility of addressing the
>>> problem and to prioritize the request. If this is a bug, there is no
>>> guarantee that the problem has been reproduced.
>>>
>>> We rename "Understood" to "Evaluated" with the following substatuses:
>>> "Cause Known" and "Fix Understood". Definitions are as follows:
>>>
>>> Evaluated - A developer has performed some investigation to understand
>>> the nature of the problem.
>>>
>>> Cause Known - A developer has performed sufficient investigation to
>>> gain a basic understanding of how this bug/feature request should be
>>> addressed. There is no requirement that an actual solution be known at
>>> this time.
>>>
>>> Fix Understood - A developer believes that they have a feasible
>>> approach to address the problem. For straight-forward problems, an
>>> actual solution may be known. More complex problems may still need
>>> additional study to work out the details.
>>>
>>> REASONS
>>>
>>> For Accepted/Open, one of the main problems is its name which could be
>>> misinterpreted to mean that the reported problem has been reproduced
>>> and a commitment to fix is implied, neither of which may be true. By
>>> changing the name to "Open", I hope the interpretation will be more in
>>> line with our current use and definition.
>>>
>>> Understood/Evaluated is more complicated. In general, I tend to favor
>>> simplicity whenever possible. I think simplicity encourages more
>>> consistent use which helps everybody in interpreting a bug's status.
>>> However, some of the historical statuses have been well-justified and
>>> are still in current use by bug owners and release management. For
>>> these reasons, I had to make a compromise. For the pilot, I'd like to
>>> see how the solution described above works. Users who feel that their
>>> process moves so quickly that this status will be unnecessary will
>>> have the option of skipping it while those that find it useful for
>>> tracking and release management still have access to that queryable 
>>> data.
>>>
>>> I fully anticipate that canned queries will be included in basic
>>> dashboards for various types of users (dev, release management,
>>> quality, etc.) so that we're all looking at the same thing. I'll start
>>> a thread to explicitly determine a set of useful queries soon.
>>>
>>> BACKGROUND
>>>
>>> Here's a summary of the relevant portions of the discussion.
>>>
>>> David Holmes:
>>> http://mail.openjdk.java.net/pipermail/discuss/2011-December/002251.html 
>>>
>>>
>>> David points out a discrepancy in the definition of "Understood." He
>>> would prefer to not collapse the existing BT "Cause Known" and BT "Fix
>>> Understood" into a single JIRA "Understood" status. The argument he
>>> makes that keeping these statuses allows new developers to easily
>>> identify potential "starter" bugs is very compelling.
>>>
>>> http://mail.openjdk.java.net/pipermail/discuss/2011-December/002267.html 
>>>
>>>
>>> David provides additional support to his argument by pointing out that
>>> the information provided by these old statuses has been used
>>> extensively by the products he's worked on and is extremely useful for
>>> anybody monitoring a bug's progress.
>>>
>>> Joe Darcy:
>>> http://mail.openjdk.java.net/pipermail/discuss/2011-December/002259.html 
>>>
>>>
>>> Joe agrees with David that the collapse to JIRA "Understood" is
>>> problematic.
>>>
>>> http://mail.openjdk.java.net/pipermail/discuss/2011-December/002262.html 
>>>
>>>
>>> He recommends that we have one status "Investigating" with three
>>> substatuses "Accepted", "Cause Known", and "Fix Understood".
>>>
>>> Martijn Verburg:
>>>
>>> http://mail.openjdk.java.net/pipermail/discuss/2011-December/002242.html 
>>>
>>>
>>> Martijn indicates that his interpretation of "Accepted" implies
>>> "Verified. He shares David's concerns around the definition of
>>> "Understood".
>>>
>>> Alan Bateman:
>>>
>>> http://mail.openjdk.java.net/pipermail/discuss/2011-December/002243.html 
>>>
>>>
>>> Alan points out the same discrepancy that David has noted.
>>>
>>> Richard Bair:
>>> http://mail.openjdk.java.net/pipermail/discuss/2011-December/002261.html 
>>>
>>>
>>> Generally, Richard would prefer fewer statuses and says that his bugs
>>> would probably go from "Accept" to "Fixed". He believes that
>>> additional statuses add unnecessary overhead.
>>>
>>> http://mail.openjdk.java.net/pipermail/discuss/2011-December/002290.html 
>>>
>>>
>>> He explicitly agrees with BrianG's reasons (see below) for
>>> recommending collapse of JIRA "Accepted" and JIRA "Understood".
>>>
>>> Brian Beck:
>>> http://mail.openjdk.java.net/pipermail/discuss/2011-December/002264.html 
>>>
>>>
>>> BrianB believes that not all developers find frequent updates
>>> suitable. From his perspective as an FX Manager, he believes that the
>>> only statuses which need to be distinguished are
>>> Open/Accepted/Triaged, In Progress, and Resolved and that the bug's
>>> description and comments are the best places to record details
>>> necessary to understand the bug's progress.
>>>
>>> http://mail.openjdk.java.net/pipermail/discuss/2012-January/002303.html
>>>
>>> BrianB suggests that BT "Cause Known" and "Fix Understood" may be
>>> helpful for individual developers, but not for those managing the
>>> entire release.
>>>
>>> Brian Goetz:
>>> http://mail.openjdk.java.net/pipermail/discuss/2011-December/002289.html 
>>>
>>>
>>> BrianG would prefer that JIRA "Accepted/Understood" be merged into
>>> something like "Open" as the proposed JIRA statuses are difficult to
>>> clearly delineate and that transition is not useful for
>>> non-development stakeholders. BrianG does not like the names
>>> "Accepted" and "Understood" as they may construed as commitments to 
>>> fix.
>>>
>>> http://mail.openjdk.java.net/pipermail/discuss/2011-December/002292.html 
>>>
>>>
>>> BrianG suggests that substatuses could reasonably be used as
>>> annotations; but use of them as a regular part of the workflow tends
>>> to lead to problems which might require boundary systems to solve.
>>>
>>> http://mail.openjdk.java.net/pipermail/discuss/2011-December/002294.html 
>>>
>>>
>>> BrianG boils down the need for boundary systems to desire have
>>> everybody use the same queries and recommends canned queries
>>> reproduced to dashboards.
>>>
>>> Georges Saab
>>> http://mail.openjdk.java.net/pipermail/discuss/2011-December/002291.html 
>>>
>>>
>>> Georges suggests that if we allow for substatuses, we can have some
>>> simplicity at the top-level and optionally use the substatus for
>>> additional information. He does not like some of the JIRA status
>>> names, and considers "Accepted" the worst offender.
>>>
>>> http://mail.openjdk.java.net/pipermail/discuss/2011-December/002293.html 
>>>
>>>
>>> Georges believes that substatuses provide useful queryable information
>>> which could be used to define SLAs (service level agreements) that
>>> might be used as metrics to determine where investment is needed. He
>>> acknowledges that the use of SLAs should be carefully considered. One
>>> example of potentially useful substatuses is "New:Unqualified" and
>>> "New:Qualified" to identify bugs which have passed initial triage.
>>>
>>> http://mail.openjdk.java.net/pipermail/discuss/2011-December/002295.html 
>>>
>>>
>>> Georges agrees that boundary system can be avoided with well-selected
>>> standard queries for common operations.