RFR: JDK-8298405: Support Markdown in Documentation Comments [v32]

[Pavel Rappo]

On Thu, 15 Feb 2024 00:30:25 GMT, Jonathan Gibbons <jjg at openjdk.org> wrote:

>> Please review a patch to add support for Markdown syntax in documentation comments, as described in the associated JEP.
>> 
>> Notable features:
>> 
>> * support for `///` documentation comments in `JavaTokenizer`
>> * new module `jdk.internal.md` -- a private copy of the `commonmark-java` library
>> * updates to `DocCommentParser` to treat `///` comments as Markdown
>> * updates to the standard doclet to render Markdown comments in HTML
>
> Jonathan Gibbons has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 44 commits:
> 
>  - fill in `visitRawText` in `CommentHelper.getTags` visitor
>  - fixes for the "New API" page
>  - change "standard" to "traditional" when referring to a comment
>  - Merge remote-tracking branch 'upstream/master' into 8298405.doclet-markdown-v3
>  - Merge remote-tracking branch 'upstream/master' into 8298405.doclet-markdown-v3
>  - improve support for DocCommentParser.LineKind
>  - Merge remote-tracking branch 'upstream/master' into 8298405.doclet-markdown-v3 # Please enter a commit message to explain why this merge is necessary, # especially if it merges an updated upstream into a topic branch. # # Lines starting with '#' will be ignored, and an empty message aborts # the
>    commit.
>  - update copyright year on test
>  - refactor recent new test case in TestMarkdown.java
>  - address review feedback
>  - ... and 34 more: https://git.openjdk.org/jdk/compare/8765b176...2801c2e1

I'm again looking `LineKind`. This time because of 9eaf84e5dd6.

src/jdk.compiler/share/classes/com/sun/tools/javac/parser/DocCommentParser.java line 422:

> 420:                             defaultContentCharacter();
> 421:                         }
> 422:                     }

Is it to pass `` through to Markdown, to allow it to deal with Markdown escapes?

src/jdk.compiler/share/classes/com/sun/tools/javac/parser/DocCommentParser.java line 1401:

> 1399:      */
> 1400:     enum LineKind {
> 1401:         BLANK(Pattern.compile("[ \t]*")),

`BLANK` is a pseudo kind, because it is set manually, but never returned from `peekLine()`. I wonder if we can change it somehow.

src/jdk.compiler/share/classes/com/sun/tools/javac/parser/DocCommentParser.java line 1433:

> 1431:          * @see <a href="https://spec.commonmark.org/0.30/#list-items">List items</a>
> 1432:          */
> 1433:         BULLETED_LIST_ITEM(Pattern.compile("[-+*] .*")),

This comment is about `BULLETED_LIST_ITEM` and `ORDERED_LIST_ITEM` constants. I know that we don't need to be very precise, but perhaps in this case we should. While the CommonMark spec is a vague on that, from my experiments with [dingus](https://spec.commonmark.org/dingus/), it appears that a list marker can be preceded and followed by some number of whitespace characters.

-------------

PR Review: https://git.openjdk.org/jdk/pull/16388#pullrequestreview-1883374712
PR Review Comment: https://git.openjdk.org/jdk/pull/16388#discussion_r1491362821
PR Review Comment: https://git.openjdk.org/jdk/pull/16388#discussion_r1491344667
PR Review Comment: https://git.openjdk.org/jdk/pull/16388#discussion_r1491354450