On Markdown in (Java) documentation comments...

[Jonathan Gibbons]

The draft JEP is now available here:

https://openjdk.org/jeps/8299906

-- Jon

On 1/23/23 10:21 AM, Jonathan Gibbons wrote:
> As some folk have already noticed, we are investigating the possibility of
> supporting Markdown in Java documentation comments.
>
> Why are we doing this?  It has been an informal request for a while now, since
> the emergence of Markdown as a popular authoring format, and so  we're doing it
> for all the same reasons that authors prefer to use Markdown  instead of raw
> HTML in standalone documentation. In particular, using raw HTML is somewhat
> clunky and visually intrusive, for simple formatting tasks, like font changes,
> lists, and tables, that are often desired in API documentation.
>
> What challenges need to be addressed?  There are two primary challenges to be
> addressed: which version of Markdown, and, how to incorporate the ability to use
> Markdown in documentation comments?
>
>  From its beginning, Markdown suffered from having difficult implementations,
> without a common specification, and often with somewhat different behaviors. The
> problem was exacerbated by the introduction of multiple disparate extensions,
> all seeking to provide additional functionality not supported in the original
> version. This has been somewhat ameliorated by the rise of CommonMark, which
> aims to provide a more formal specification for Markdown, and the adoption of
> CommonMark (albeit often with some extensions) on some popular social platforms.
>
> In terms of using Markdown within documentation comments, we do not want to
> replace the support for the existing functionality of inline tags (like`{@link
> ...}`) or block tags (like`@param`,`@return`, and`@throws`), most of which do
> not have an equivalent in basic Markdown. (The one obvious tag that does have a
> replacement is the use of back-ticks for a code span instead of`{@code ...}`.)
> It would be impractical to replace the use of all existing tags, and the ability
> to provide user-defined tags, with Markdown extensions, and so we are left with
> wanting to accommodate the use of standard and user-defined tags alongside
> Markdown in documentation comments. A slightly different way of saying that is
> that we want to  support the use of Markdown constructs in documentation comments
> anywhere that raw HTML can be used in a documentation comment today.
>
> Markdown is not HTML. While it does accommodate the use of appropriate HTML
> elements, Markdown interprets some otherwise-plain text differently.  Most
> notable is its interpretation of whitespace characters. In general, whitespace
> is not significant in HTML (except in`<pre>` elements), but in Markdown, a
> blank line is interpreted as a paragraph break. This by itself is enough to mean
> that we cannot enable Markdown by default everywhere, for both new and existing
> documentation comments.  That implies we need to support both traditional
> comments (that were not written with Markdown in mind), and comments that use
> the ability to use Markdown.  And in that heterogeneous world, we need to be
> able to generate API documentation from code that may use a mixture of comment
> formats. Of note is the ability for the`{@inheritDoc}` tag to reference text
> in different classes, which may even be in different projects.  For example,
> a project providing customized collections (for example, like Guava), may want
> to inherit documentation from the core collections classes in Java SE.
>
> Markdown is not without its problems. It adopts an error-free policy of "if it's
> not a valid syntactic construct, it must be literal text". In no cases does it
> report errors. If input looks like it might be HTML, it is passed through to the
> output, where it may be caught downstream by an HTML validator if it wasn't
> actually valid HTML, and if the author chooses to run such a tool. If the input
> contains HTML with a typo (like a missing quote for an attribute value),
> Markdown will treat that fragment of the input as literal text, and generate the
> appropriate HTML for that literal text. In that case, the erroneous input will
> _not_ be caught by a downstream HTML validator, and if not caught by
> proofreading the output, the erroneous input will make it all the way "in clear"
> to the generated output. Given the empirical evidence that authors do_not_ proofread their generated documentation, and you have an increased risk of
> problems in the generated output.
>
> Why this is issue of error handling not more of an issue in other contexts? In
> other contexts, Markdown is typically write-once and/or somewhat short-lived.
> And, there is typically very low latency to "preview" the rendered output,
> either with an explicit "Preview" button, or maybe with a "live preview" that
> updates as you type.  If you have access to a Markdown editor with live preview,
> try typing in a string like`<a href="example.com">example</a>` and see how it
> is previewed in its incomplete state and then its final state. So, whether you
> are typing in some sort of design document, or a commit message on Github, it is
> typically easy to verify whether the rendering is as expected. In contrast,
> documentation comments are likely to be long-lived, for as long as the API
> itself is maintained, and maybe edited or updated by different authors over
> time, using different tools. And running`javadoc` to generate the API
> documentation may be seen as an afterthought, after the work of developing and
> debugging the implementation of the underlying API.  It would help if we could
> reduce the latency to view the rendered form of a documentation comment, either
> with a "Preview" button, or "live preview", and while that may be out of scope
> in terms of a project to define the ability to use Markdown, we can at least
> define the feature in a way that may increase the adoption in authoring tools.
>
> Looking at tools,`javadoc` is not the only tool to process documentation
> comments. As well IDEs and other authoring tools,`jshell` also provides the
> ability to display documentation comments. Perhaps even more than`javadoc` itself,`jshell` may be called upon to display both "traditional" comments, for
> files or libraries that do not use Markdown comments, or to handle Markdown
> comments, for files or libraries that do use them. Again, we need to consider
> the needs of and implications for other tools in the ecosystem  when designing a
> new feature like support for Markdown in documentation comments.
>
> -- Jon
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/javadoc-dev/attachments/20230208/037d5ae4/attachment-0001.htm>