Bug System Pilot Dev Workflow: DRAFT Accepted/Understood

Tue Jan 17 07:43:23 UTC 2012

Hi!

First, thanks for all for the feedback.  It's been very interesting reading it over the past few weeks.  I've had a lot to think about.  Let's start to wrap this topic up.  We've got lots more to cover.

For each area or topic of discussion, I'll be providing a summary of the various arguments, a decision for what the pilot should look like, and reasons for that decision.  This message is about the "Accepted" and "Understood" statuses. 

Expect an update to the DRAFT to reflect this decision.

Thanks,
iris

DECISION

Based on the extensive discussion around the proposed JIRA "Accepted" and "Understood" status of the Pilot Developer Workflow, here's what I think should be done for the pilot.

We rename "Accepted" to "Open" with the following definition: Initial triage has determined that the bug/feature request contains sufficient information to estimate the basic feasibility of addressing the problem and to prioritize the request.  If this is a bug, there is no guarantee that the problem has been reproduced.

We rename "Understood" to "Evaluated" with the following substatuses: "Cause Known" and "Fix Understood".  Definitions are as follows:

Evaluated - A developer has performed some investigation to understand the nature of the problem.

Cause Known - A developer has performed sufficient investigation to gain a basic understanding of how this bug/feature request should be addressed.  There is no requirement that an actual solution be known at this time.

Fix Understood - A developer believes that they have a feasible approach to address the problem.  For straight-forward problems, an actual solution may be known.  More complex problems may still need additional study to work out the details.

REASONS

For Accepted/Open, one of the main problems is its name which could be misinterpreted to mean that the reported problem has been reproduced and a commitment to fix is implied, neither of which may be true.  By changing the name to "Open", I hope the interpretation will be more in line with our current use and definition.

Understood/Evaluated is more complicated.  In general, I tend to favor simplicity whenever possible.  I think simplicity encourages more consistent use which helps everybody in interpreting a bug's status.  However, some of the historical statuses have been well-justified and are still in current use by bug owners and release management.  For these reasons, I had to make a compromise.  For the pilot, I'd like to see how the solution described above works.  Users who feel that their process moves so quickly that this status will be unnecessary will have the option of skipping it while those that find it useful for tracking and release management still have access to that queryable data.

I fully anticipate that canned queries will be included in basic dashboards for various types of users (dev, release management, quality, etc.) so that we're all looking at the same thing.  I'll start a thread to explicitly determine a set of useful queries soon.

BACKGROUND

Here's a summary of the relevant portions of the discussion.

David Holmes:
  http://mail.openjdk.java.net/pipermail/discuss/2011-December/002251.html

David points out a discrepancy in the definition of "Understood."  He would prefer to not collapse the existing BT "Cause Known" and BT "Fix Understood" into a single JIRA "Understood" status.  The argument he makes that keeping these statuses allows new developers to easily identify potential "starter" bugs is very compelling.

  http://mail.openjdk.java.net/pipermail/discuss/2011-December/002267.html

David provides additional support to his argument by pointing out that the information provided by these old statuses has been used extensively by the products he's worked on and is extremely useful for anybody monitoring a bug's progress.

Joe Darcy:
  http://mail.openjdk.java.net/pipermail/discuss/2011-December/002259.html

Joe agrees with David that the collapse to JIRA "Understood" is problematic.

  http://mail.openjdk.java.net/pipermail/discuss/2011-December/002262.html

He recommends that we have one status "Investigating" with three substatuses "Accepted", "Cause Known", and "Fix Understood".

Martijn Verburg: 

  http://mail.openjdk.java.net/pipermail/discuss/2011-December/002242.html

Martijn indicates that his interpretation of "Accepted" implies "Verified.  He shares David's concerns around the definition of "Understood".

Alan Bateman:

  http://mail.openjdk.java.net/pipermail/discuss/2011-December/002243.html

Alan points out the same discrepancy that David has noted.

Richard Bair:
  http://mail.openjdk.java.net/pipermail/discuss/2011-December/002261.html

Generally, Richard would prefer fewer statuses and says that his bugs would probably go from "Accept" to "Fixed".  He believes that additional statuses add unnecessary overhead.  

  http://mail.openjdk.java.net/pipermail/discuss/2011-December/002290.html

He explicitly agrees with BrianG's reasons (see below) for recommending collapse of JIRA "Accepted" and JIRA "Understood".

Brian Beck:
  http://mail.openjdk.java.net/pipermail/discuss/2011-December/002264.html

BrianB believes that not all developers find frequent updates suitable.  From his perspective as an FX Manager, he believes that the only statuses which need to be distinguished are Open/Accepted/Triaged, In Progress, and Resolved and that the bug's description and comments are the best places to record details necessary to understand the bug's progress.

  http://mail.openjdk.java.net/pipermail/discuss/2012-January/002303.html

BrianB suggests that BT "Cause Known" and "Fix Understood" may be helpful for individual developers, but not for those managing the entire release.

Brian Goetz:
  http://mail.openjdk.java.net/pipermail/discuss/2011-December/002289.html

BrianG would prefer that JIRA "Accepted/Understood" be merged into something like "Open" as the proposed JIRA statuses are difficult to clearly delineate and that transition is not useful for non-development stakeholders.  BrianG does not like the names "Accepted" and "Understood" as they may construed as commitments to fix.

  http://mail.openjdk.java.net/pipermail/discuss/2011-December/002292.html

BrianG suggests that substatuses could reasonably be used as annotations; but use of them as a regular part of the workflow tends to lead to problems which might require boundary systems to solve.

  http://mail.openjdk.java.net/pipermail/discuss/2011-December/002294.html

BrianG boils down the need for boundary systems to desire have everybody use the same queries and recommends canned queries reproduced to dashboards.

Georges Saab
  http://mail.openjdk.java.net/pipermail/discuss/2011-December/002291.html

Georges suggests that if we allow for substatuses, we can have some simplicity at the top-level and optionally use the substatus for additional information.  He does not like some of the JIRA status names, and considers "Accepted" the worst offender.

  http://mail.openjdk.java.net/pipermail/discuss/2011-December/002293.html

Georges believes that substatuses provide useful queryable information which could be used to define SLAs (service level agreements) that might be used as metrics to determine where investment is needed.  He acknowledges that the use of SLAs should be carefully considered.  One example of potentially useful substatuses is "New:Unqualified" and "New:Qualified" to identify bugs which have passed initial triage.

  http://mail.openjdk.java.net/pipermail/discuss/2011-December/002295.html

Georges agrees that boundary system can be avoided with well-selected standard queries for common operations.