RFR: Updates to the JBS section of the OpenJDK Dev Guide [v2]

Tue Apr 4 02:34:38 UTC 2023

On Thu, 23 Mar 2023 10:06:01 GMT, calnan <duke at openjdk.org> wrote:

>> Added overview on triaging issues, the bug states and process.
>
> calnan has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Rewrite of the Issue Type table, minor updates, and changes for formatting consistency across the file

src/guide/jbs-jdk-bug-system.md line 25:

> 23: The most common Issue types are:
> 24: <br />
> 25: <table class="dictionary" summary="JBS Label Dictionary">

"label" ?? Copy'n'paste error?

src/guide/jbs-jdk-bug-system.md line 29:

> 27: <tr>
> 28:     <td class="dictionary">[Bug]{.jbs-field}</td>
> 29:     <td class="dictionary">Used when reporting a problem: crashes; hangs; failures of functionality etc.</td>

The "etc" could be a bit too open to interpretation. A "bug" should relate to functional correctness - a deviation from behaviour that can be tied back to a specification. Anything else, including performance concerns, is generally not a bug but an enhancement. Though it is not clear cut as a significant performance regression may be classified as a "bug", for example.

src/guide/jbs-jdk-bug-system.md line 34:

> 32:     <td class="dictionary"><i>Improvement</i></td>
> 33:     <td class="dictionary">When requesting an improvement in the JDK, the issue type used depends on the size/scope of the improvement:<br />
> 34:     ([Enhancement]{.jbs-field}) - for a small improvement to existing functionality<br />

"Enhancement" is used for improvements small and large - basically anything that is not a "bug". Perhaps we should say "small to medium" with the understanding that "large" implies big enough to be a JEP?

src/guide/jbs-jdk-bug-system.md line 44:

> 42: <tr>
> 43:     <td class="dictionary">[Task]{.jbs-field}</td>
> 44:     <td class="dictionary">Where something needs to happen other than a code change - a request for a new JBS version value for example</td>

This one is very grey. "Task" does not get used much in the hotspot area, for example. I think a task may typically be something for which no changeset for the repo would be generated; though if you interpret "Task" as anything pertaining to a change other than to code, then the use of task could increase dramatically. But in the build area changes don't relate to "code" per-se, so would they all be tasks? I think not. In conclusion it is very hard to know when "Task" is suitable for use.

src/guide/jbs-jdk-bug-system.md line 47:

> 45: </tr>
> 46: <tr>
> 47:     <td class="dictionary"><i>Vulnerability</i></td>

"Vulnerability" is not a JBS issue type, so I don't think this belongs in this table. 

Also general users may not understand what is meant by "vulnerability" here - perhaps "security vulnerability" would be better?

src/guide/jbs-jdk-bug-system.md line 53:

> 51: 
> 52: <br>
> 53: A few things to keep in mind when filing an issue to report a problem:

"problem" here could mean "bug" or "enhancement" depending on the nature of the problem. But some of the below relates to filing an issue for non-problems too.

src/guide/jbs-jdk-bug-system.md line 55:

> 53: A few things to keep in mind when filing an issue to report a problem:
> 54: 
> 55: * Before filing, verify that there isn't an issue already filed.

Suggestion: "open issue". You may find dozens of resolved/closed issues if you search for something like the name of a failing tests. You may also find a closed issue that was "not reproducible" that it may be appropriate to re-open.

Also note that for those without direct JBS access this advice is somewhat moot because even if they find an existing issue, they can't tell anyone "hey I hit this problem too". Not sure what we can do there.

src/guide/jbs-jdk-bug-system.md line 63:

> 61:     * If the failure is found in an update train of the JDK (e.g. 11.0.x), please make an effort to see if it is also present in [mainline](https://github.com/openjdk/jdk).
> 62: * Set the priority
> 63:     * It's not the reporter's responsibility to set a correct priority, but a qualified guess is always better than the default.

I don't think this is useful unless we actually specify what P1-P5 should mean. Otherwise may as well take the default and let triage teams re-evaluate it.

src/guide/jbs-jdk-bug-system.md line 68:

> 66:     * error messages
> 67:     * assert messages
> 68:     * stack trace

I'd qualify this by saying attach hs_err logs, or other log files, rather than putting the content of the log in the issue descriptions.

src/guide/jbs-jdk-bug-system.md line 76:

> 74:     * passwords, logins, machine names
> 75:     * logs which may include sensitive data
> 76: * If the failure isn't reproducible with an existing OpenJDK test, attach a reproducer if possible, while in a number of cases it isn't possible, having a test case will decrease the time required to resolve the issue.

I suggest splitting this into two sentences at "while".

src/guide/jbs-jdk-bug-system.md line 77:

> 75:     * logs which may include sensitive data
> 76: * If the failure isn't reproducible with an existing OpenJDK test, attach a reproducer if possible, while in a number of cases it isn't possible, having a test case will decrease the time required to resolve the issue.
> 77: * Only set [CPU]{.jbs-field} and/or [OS]{.jbs-field} fields if the bug **ONLY** happens on that particular platform or set of platforms.

This would be ideal but in practice very hard. Most people would set this to the platform they saw the problem on. There is very little chance they can know, or can find out, whether it only happens on that platform. The person later working on the bug can update these fields if needed, but in my experience very few people use these fields in a meaningful way.

src/guide/jbs-jdk-bug-system.md line 78:

> 76: * If the failure isn't reproducible with an existing OpenJDK test, attach a reproducer if possible, while in a number of cases it isn't possible, having a test case will decrease the time required to resolve the issue.
> 77: * Only set [CPU]{.jbs-field} and/or [OS]{.jbs-field} fields if the bug **ONLY** happens on that particular platform or set of platforms.
> 78: * Including the java -fullversion is encouraged for bugs in the JVM, hangs, network issues where the exact version can be critical to determine what fixes may be missing from an older version.

This sentence doesn't read well and I'm not sure what point it is trying to make. `java -version` is what is needed and that is a superset of `java -fullversion`.

src/guide/jbs-jdk-bug-system.md line 90:

> 88: 
> 89: ### Implementing a Large Change
> 90: When managing the work for a large change, especially when the work involves multiple engineers, it is recommended that the work is distributed across one or more "implementation" issues which should be linked to the main [Enhancement]{.jbs-field} with a "blocks" link along with any relevant CSRs. The [Enhancement]{.jbs-field} should not be considered done until all the blocking element are completed.  The use of subTasks for [Enhancement]{.jbs}s is not recommended unless all the [Sub-task]{.jbs-field}s are relevant to the fix, if it were to be backported.

To me this is exactly what sub-tasks should be used for. ???

src/guide/jbs-jdk-bug-system.md line 107:

> 105: <tr>
> 106:     <td class="dictionary">Open</td>
> 107:     <td class="dictionary">Most straightforward issues stay in this state until they are closed.  If the issue has some attention then use "In Progress" to show more clearly that the issue is being worked</td>

If you start working on an issue then you should mark it in-progress by selecting "Work started", not just leave it "open". This gives a clear indication that the assignee is actually working on something and it isn't just one of a dozen bugs assigned to someone that they may, or may not, eventually get around to looking at.

src/guide/jbs-jdk-bug-system.md line 110:

> 108: </tr>
> 109: <tr>
> 110:     <td class="dictionary">In Progress</td>

See above.

src/guide/jbs-jdk-bug-system.md line 171:

> 169: ## Triaging an issue
> 170: 
> 171: First give the issue a general [Review]{.jbs-field}

What does the formatting `[Review]{.jbs-field}` mean here? It is very hard to understand how this document is structured when it isn't actually using markdown as expected.

src/guide/jbs-jdk-bug-system.md line 189:

> 187:     1. if the bug also exists in older versions it may require backporting
> 188:         * the decision to backport should be made inline with the guidelines of the lead for the release
> 189:         * There are two options for creating backport issue(s) to track the backport: one is to create it manually once it is agreed that the bug should be backported; the second, is to let the bots create the backport issue once you push the fix to the repo.

"bots" needs some context here.

src/guide/jbs-jdk-bug-system.md line 192:

> 190:     1. Only one fixversion should ever be set, if the issue is to be fixed in additional releases then a separate backport must be created (See Working with backports in JBS).  There are exceptions to this rule for: CSRs; and, Release Notes.
> 191: 1. make sure the bug has all the required labels – JBS Label Dictionary
> 192:     1. bugs that are new in the current release: 'regression'

This is tricky. We don't generally use "regression" during active development of the current release, it tends to be used mainly for issues that are found after a release has been released e.g. if we release JDK 19 and then a bug report comes in because something that worked in 18 now doesn't, then it is a regression in 19. But if we are in the process of developing a release and we have added some new thing and then we find a bug in that then we don't mark that as a regression. And a lot of the time if we introduce a bug "by mistake" we don't tend to mark it as a regression.

src/guide/jbs-jdk-bug-system.md line 198:

> 196:     1. project specific issues usually have their own labels as well
> 197: 
> 198: At this point move the issue into the [Open]{.jbs-field} state, bring high priority (P1, P2) bugs to the attention of the team.

What "team"?

-------------

PR Review Comment: https://git.openjdk.org/guide/pull/100#discussion_r1142848252
PR Review Comment: https://git.openjdk.org/guide/pull/100#discussion_r1142850113
PR Review Comment: https://git.openjdk.org/guide/pull/100#discussion_r1142848979
PR Review Comment: https://git.openjdk.org/guide/pull/100#discussion_r1142855644
PR Review Comment: https://git.openjdk.org/guide/pull/100#discussion_r1142851072
PR Review Comment: https://git.openjdk.org/guide/pull/100#discussion_r1142856511
PR Review Comment: https://git.openjdk.org/guide/pull/100#discussion_r1142857844
PR Review Comment: https://git.openjdk.org/guide/pull/100#discussion_r1142887246
PR Review Comment: https://git.openjdk.org/guide/pull/100#discussion_r1142887934
PR Review Comment: https://git.openjdk.org/guide/pull/100#discussion_r1142888337
PR Review Comment: https://git.openjdk.org/guide/pull/100#discussion_r1142889300
PR Review Comment: https://git.openjdk.org/guide/pull/100#discussion_r1142890629
PR Review Comment: https://git.openjdk.org/guide/pull/100#discussion_r1142891511
PR Review Comment: https://git.openjdk.org/guide/pull/100#discussion_r1142894429
PR Review Comment: https://git.openjdk.org/guide/pull/100#discussion_r1142894668
PR Review Comment: https://git.openjdk.org/guide/pull/100#discussion_r1142898212
PR Review Comment: https://git.openjdk.org/guide/pull/100#discussion_r1142899459
PR Review Comment: https://git.openjdk.org/guide/pull/100#discussion_r1142911842
PR Review Comment: https://git.openjdk.org/guide/pull/100#discussion_r1142912247