Bug System Pilot Dev Workflow: DRAFT Accepted/Understood

Tue Jan 17 20:16:32 UTC 2012

I think "Open" is "Triaged", just by a different name.

Richard

On Jan 17, 2012, at 12:05 PM, John Pampuch wrote:

> So I'll bite.  Can we consider a 'triaged' state, and an 'evaluated' state?  Or perhaps a sub-state to distinguish the two?
> 
> (My apologies to Rich!)
> 
> -John
> 
> On 1/17/12 11:56 AM, Georges Saab wrote:
>> inline...
>> 
>> On 17 jan 2012, at 01:46, David Holmes wrote:
>> 
>>> On 17/01/2012 6:52 PM, David Holmes wrote:
>>>> Just one comment on this:
>>>> 
>>>>> We rename "Accepted" to "Open" with the following definition: Initial
>>>>> triage has determined that the bug/feature request contains
>>>>> sufficient information to estimate the basic feasibility of
>>>>> addressing the problem and to prioritize the request. If this is a
>>>>> bug, there is no guarantee that the problem has been reproduced.
>>>> I find those two statements somewhat contradictory. If the problem may
>>>> not have even been reproduced how can you have triaged it enough to have
>>>> "sufficient information to estimate the basic feasibility of addressing
>>>> the problem and to prioritize the request" ?
>>> Okay maybe not so contradictory in some cases eg if external triage has already been done.
>>> 
>>> I think what I'm missing from this definition is understanding the criteria for moving from New to Open.
>> Initial triage should ensure that the bug is assigned to the right group and has the information
>> needed to assess where looking at it should come relative to other work/bugs.  Until we know otherwise
>> we have to assume that it is a real issue and treat it as such (which is an evaluation task not
>> an initial triage task).
>> 
>> For initial triage I would suggest the things that should be reviewed are:
>>    1. To which group the bug is assigned
>>    2. In the context of which release it should be investigated (aka 'targeted' in the current system though
>> I know people have issues with that)
>>    3. Correctness of the priority given the knowledge we have now.
>>    4. Any other obvious missing information which would mean it should immediately be assigned back
>> to the submitter as NMI (Needs More Information)
>> 
>>    /GES
>> 
>>> David
>>> 
>>>> David
>>>> -----
>>>> 
>>>> On 17/01/2012 5:43 PM, Iris Clark wrote:
>>>>> Hi!
>>>>> 
>>>>> First, thanks for all for the feedback. It's been very interesting
>>>>> reading it over the past few weeks. I've had a lot to think about.
>>>>> Let's start to wrap this topic up. We've got lots more to cover.
>>>>> 
>>>>> For each area or topic of discussion, I'll be providing a summary of
>>>>> the various arguments, a decision for what the pilot should look like,
>>>>> and reasons for that decision. This message is about the "Accepted"
>>>>> and "Understood" statuses.
>>>>> 
>>>>> Expect an update to the DRAFT to reflect this decision.
>>>>> 
>>>>> Thanks,
>>>>> iris
>>>>> 
>>>>> DECISION
>>>>> 
>>>>> Based on the extensive discussion around the proposed JIRA "Accepted"
>>>>> and "Understood" status of the Pilot Developer Workflow, here's what I
>>>>> think should be done for the pilot.
>>>>> 
>>>>> We rename "Accepted" to "Open" with the following definition: Initial
>>>>> triage has determined that the bug/feature request contains sufficient
>>>>> information to estimate the basic feasibility of addressing the
>>>>> problem and to prioritize the request. If this is a bug, there is no
>>>>> guarantee that the problem has been reproduced.
>>>>> 
>>>>> We rename "Understood" to "Evaluated" with the following substatuses:
>>>>> "Cause Known" and "Fix Understood". Definitions are as follows:
>>>>> 
>>>>> Evaluated - A developer has performed some investigation to understand
>>>>> the nature of the problem.
>>>>> 
>>>>> Cause Known - A developer has performed sufficient investigation to
>>>>> gain a basic understanding of how this bug/feature request should be
>>>>> addressed. There is no requirement that an actual solution be known at
>>>>> this time.
>>>>> 
>>>>> Fix Understood - A developer believes that they have a feasible
>>>>> approach to address the problem. For straight-forward problems, an
>>>>> actual solution may be known. More complex problems may still need
>>>>> additional study to work out the details.
>>>>> 
>>>>> REASONS
>>>>> 
>>>>> For Accepted/Open, one of the main problems is its name which could be
>>>>> misinterpreted to mean that the reported problem has been reproduced
>>>>> and a commitment to fix is implied, neither of which may be true. By
>>>>> changing the name to "Open", I hope the interpretation will be more in
>>>>> line with our current use and definition.
>>>>> 
>>>>> Understood/Evaluated is more complicated. In general, I tend to favor
>>>>> simplicity whenever possible. I think simplicity encourages more
>>>>> consistent use which helps everybody in interpreting a bug's status.
>>>>> However, some of the historical statuses have been well-justified and
>>>>> are still in current use by bug owners and release management. For
>>>>> these reasons, I had to make a compromise. For the pilot, I'd like to
>>>>> see how the solution described above works. Users who feel that their
>>>>> process moves so quickly that this status will be unnecessary will
>>>>> have the option of skipping it while those that find it useful for
>>>>> tracking and release management still have access to that queryable data.
>>>>> 
>>>>> I fully anticipate that canned queries will be included in basic
>>>>> dashboards for various types of users (dev, release management,
>>>>> quality, etc.) so that we're all looking at the same thing. I'll start
>>>>> a thread to explicitly determine a set of useful queries soon.
>>>>> 
>>>>> BACKGROUND
>>>>> 
>>>>> Here's a summary of the relevant portions of the discussion.
>>>>> 
>>>>> David Holmes:
>>>>> http://mail.openjdk.java.net/pipermail/discuss/2011-December/002251.html
>>>>> 
>>>>> David points out a discrepancy in the definition of "Understood." He
>>>>> would prefer to not collapse the existing BT "Cause Known" and BT "Fix
>>>>> Understood" into a single JIRA "Understood" status. The argument he
>>>>> makes that keeping these statuses allows new developers to easily
>>>>> identify potential "starter" bugs is very compelling.
>>>>> 
>>>>> http://mail.openjdk.java.net/pipermail/discuss/2011-December/002267.html
>>>>> 
>>>>> David provides additional support to his argument by pointing out that
>>>>> the information provided by these old statuses has been used
>>>>> extensively by the products he's worked on and is extremely useful for
>>>>> anybody monitoring a bug's progress.
>>>>> 
>>>>> Joe Darcy:
>>>>> http://mail.openjdk.java.net/pipermail/discuss/2011-December/002259.html
>>>>> 
>>>>> Joe agrees with David that the collapse to JIRA "Understood" is
>>>>> problematic.
>>>>> 
>>>>> http://mail.openjdk.java.net/pipermail/discuss/2011-December/002262.html
>>>>> 
>>>>> He recommends that we have one status "Investigating" with three
>>>>> substatuses "Accepted", "Cause Known", and "Fix Understood".
>>>>> 
>>>>> Martijn Verburg:
>>>>> 
>>>>> http://mail.openjdk.java.net/pipermail/discuss/2011-December/002242.html
>>>>> 
>>>>> Martijn indicates that his interpretation of "Accepted" implies
>>>>> "Verified. He shares David's concerns around the definition of
>>>>> "Understood".
>>>>> 
>>>>> Alan Bateman:
>>>>> 
>>>>> http://mail.openjdk.java.net/pipermail/discuss/2011-December/002243.html
>>>>> 
>>>>> Alan points out the same discrepancy that David has noted.
>>>>> 
>>>>> Richard Bair:
>>>>> http://mail.openjdk.java.net/pipermail/discuss/2011-December/002261.html
>>>>> 
>>>>> Generally, Richard would prefer fewer statuses and says that his bugs
>>>>> would probably go from "Accept" to "Fixed". He believes that
>>>>> additional statuses add unnecessary overhead.
>>>>> 
>>>>> http://mail.openjdk.java.net/pipermail/discuss/2011-December/002290.html
>>>>> 
>>>>> He explicitly agrees with BrianG's reasons (see below) for
>>>>> recommending collapse of JIRA "Accepted" and JIRA "Understood".
>>>>> 
>>>>> Brian Beck:
>>>>> http://mail.openjdk.java.net/pipermail/discuss/2011-December/002264.html
>>>>> 
>>>>> BrianB believes that not all developers find frequent updates
>>>>> suitable. From his perspective as an FX Manager, he believes that the
>>>>> only statuses which need to be distinguished are
>>>>> Open/Accepted/Triaged, In Progress, and Resolved and that the bug's
>>>>> description and comments are the best places to record details
>>>>> necessary to understand the bug's progress.
>>>>> 
>>>>> http://mail.openjdk.java.net/pipermail/discuss/2012-January/002303.html
>>>>> 
>>>>> BrianB suggests that BT "Cause Known" and "Fix Understood" may be
>>>>> helpful for individual developers, but not for those managing the
>>>>> entire release.
>>>>> 
>>>>> Brian Goetz:
>>>>> http://mail.openjdk.java.net/pipermail/discuss/2011-December/002289.html
>>>>> 
>>>>> BrianG would prefer that JIRA "Accepted/Understood" be merged into
>>>>> something like "Open" as the proposed JIRA statuses are difficult to
>>>>> clearly delineate and that transition is not useful for
>>>>> non-development stakeholders. BrianG does not like the names
>>>>> "Accepted" and "Understood" as they may construed as commitments to fix.
>>>>> 
>>>>> http://mail.openjdk.java.net/pipermail/discuss/2011-December/002292.html
>>>>> 
>>>>> BrianG suggests that substatuses could reasonably be used as
>>>>> annotations; but use of them as a regular part of the workflow tends
>>>>> to lead to problems which might require boundary systems to solve.
>>>>> 
>>>>> http://mail.openjdk.java.net/pipermail/discuss/2011-December/002294.html
>>>>> 
>>>>> BrianG boils down the need for boundary systems to desire have
>>>>> everybody use the same queries and recommends canned queries
>>>>> reproduced to dashboards.
>>>>> 
>>>>> Georges Saab
>>>>> http://mail.openjdk.java.net/pipermail/discuss/2011-December/002291.html
>>>>> 
>>>>> Georges suggests that if we allow for substatuses, we can have some
>>>>> simplicity at the top-level and optionally use the substatus for
>>>>> additional information. He does not like some of the JIRA status
>>>>> names, and considers "Accepted" the worst offender.
>>>>> 
>>>>> http://mail.openjdk.java.net/pipermail/discuss/2011-December/002293.html
>>>>> 
>>>>> Georges believes that substatuses provide useful queryable information
>>>>> which could be used to define SLAs (service level agreements) that
>>>>> might be used as metrics to determine where investment is needed. He
>>>>> acknowledges that the use of SLAs should be carefully considered. One
>>>>> example of potentially useful substatuses is "New:Unqualified" and
>>>>> "New:Qualified" to identify bugs which have passed initial triage.
>>>>> 
>>>>> http://mail.openjdk.java.net/pipermail/discuss/2011-December/002295.html
>>>>> 
>>>>> Georges agrees that boundary system can be avoided with well-selected
>>>>> standard queries for common operations.