On Markdown in (Java) documentation comments...

[Jonathan Gibbons]

As some folk have already noticed, we are investigating the possibility of
supporting Markdown in Java documentation comments.

Why are we doing this?  It has been an informal request for a while now, since
the emergence of Markdown as a popular authoring format, and so  we're doing it
for all the same reasons that authors prefer to use Markdown  instead of raw
HTML in standalone documentation. In particular, using raw HTML is somewhat
clunky and visually intrusive, for simple formatting tasks, like font changes,
lists, and tables, that are often desired in API documentation.

What challenges need to be addressed?  There are two primary challenges to be
addressed: which version of Markdown, and, how to incorporate the ability to use
Markdown in documentation comments?

 From its beginning, Markdown suffered from having difficult implementations,
without a common specification, and often with somewhat different behaviors. The
problem was exacerbated by the introduction of multiple disparate extensions,
all seeking to provide additional functionality not supported in the original
version. This has been somewhat ameliorated by the rise of CommonMark, which
aims to provide a more formal specification for Markdown, and the adoption of
CommonMark (albeit often with some extensions) on some popular social platforms.

In terms of using Markdown within documentation comments, we do not want to
replace the support for the existing functionality of inline tags (like`{@link
...}`) or block tags (like`@param`,`@return`, and`@throws`), most of which do
not have an equivalent in basic Markdown. (The one obvious tag that does have a
replacement is the use of back-ticks for a code span instead of`{@code ...}`.)
It would be impractical to replace the use of all existing tags, and the ability
to provide user-defined tags, with Markdown extensions, and so we are left with
wanting to accommodate the use of standard and user-defined tags alongside
Markdown in documentation comments. A slightly different way of saying that is
that we want to  support the use of Markdown constructs in documentation comments
anywhere that raw HTML can be used in a documentation comment today.

Markdown is not HTML. While it does accommodate the use of appropriate HTML
elements, Markdown interprets some otherwise-plain text differently.  Most
notable is its interpretation of whitespace characters. In general, whitespace
is not significant in HTML (except in`<pre>` elements), but in Markdown, a
blank line is interpreted as a paragraph break. This by itself is enough to mean
that we cannot enable Markdown by default everywhere, for both new and existing
documentation comments.  That implies we need to support both traditional
comments (that were not written with Markdown in mind), and comments that use
the ability to use Markdown.  And in that heterogeneous world, we need to be
able to generate API documentation from code that may use a mixture of comment
formats. Of note is the ability for the`{@inheritDoc}` tag to reference text
in different classes, which may even be in different projects.  For example,
a project providing customized collections (for example, like Guava), may want
to inherit documentation from the core collections classes in Java SE.

Markdown is not without its problems. It adopts an error-free policy of "if it's
not a valid syntactic construct, it must be literal text". In no cases does it
report errors. If input looks like it might be HTML, it is passed through to the
output, where it may be caught downstream by an HTML validator if it wasn't
actually valid HTML, and if the author chooses to run such a tool. If the input
contains HTML with a typo (like a missing quote for an attribute value),
Markdown will treat that fragment of the input as literal text, and generate the
appropriate HTML for that literal text. In that case, the erroneous input will
_not_ be caught by a downstream HTML validator, and if not caught by
proofreading the output, the erroneous input will make it all the way "in clear"
to the generated output. Given the empirical evidence that authors do_not_ proofread their generated documentation, and you have an increased risk of
problems in the generated output.

Why this is issue of error handling not more of an issue in other contexts? In
other contexts, Markdown is typically write-once and/or somewhat short-lived.
And, there is typically very low latency to "preview" the rendered output,
either with an explicit "Preview" button, or maybe with a "live preview" that
updates as you type.  If you have access to a Markdown editor with live preview,
try typing in a string like`<a href="example.com">example</a>` and see how it
is previewed in its incomplete state and then its final state. So, whether you
are typing in some sort of design document, or a commit message on Github, it is
typically easy to verify whether the rendering is as expected. In contrast,
documentation comments are likely to be long-lived, for as long as the API
itself is maintained, and maybe edited or updated by different authors over
time, using different tools. And running`javadoc` to generate the API
documentation may be seen as an afterthought, after the work of developing and
debugging the implementation of the underlying API.  It would help if we could
reduce the latency to view the rendered form of a documentation comment, either
with a "Preview" button, or "live preview", and while that may be out of scope
in terms of a project to define the ability to use Markdown, we can at least
define the feature in a way that may increase the adoption in authoring tools.

Looking at tools,`javadoc` is not the only tool to process documentation
comments. As well IDEs and other authoring tools,`jshell` also provides the
ability to display documentation comments. Perhaps even more than`javadoc` itself,`jshell` may be called upon to display both "traditional" comments, for
files or libraries that do not use Markdown comments, or to handle Markdown
comments, for files or libraries that do use them. Again, we need to consider
the needs of and implications for other tools in the ecosystem  when designing a
new feature like support for Markdown in documentation comments.

-- Jon
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/javadoc-dev/attachments/20230123/691b655d/attachment.htm>