RFR: JDK-8267204: Expose access to underlying streams in DocletEnvironment

[Jonathan Gibbons]

On Wed, 19 May 2021 16:22:24 GMT, Pavel Rappo <prappo at openjdk.org> wrote:

>> What would be the best code locations to look at to get familiar with how javac works with standard streams, diagnostic output, and the like?

Look at `Log.java`, and the introduction of `WriterKind.STDOUT, STDERR`, and the use thereof.

Before there was `WriterKind.STDOUT, STDERR`, javac had the same trouble that remains in javadoc ... 

1. how to map diagnostics to streams?
2. how to handle command-line help and other "expected" output like from `-Xprint` ?

In javac then, and still in javadoc today, it was hampered by the use of public single-stream constructors in the `Main` class.  javac actually had it "worse", in that at one point it completely flipped from using `System.out` (only) to using `System.err` (only). (Sort of like the Earth's magnetic poles flipping!) There is still code in `jtreg` to handle that flip, although the flip occurred in versions of JDK that are long unsupported.

Part of the realization of the solution came in the JDK 9 timeframe, when someone (I won't name names) came up with the simple rule that "expected" output (like command-line help) should go to `stdout`/`System.out` so that it can be piped into other tools like `grep` etc.  All "other"/"unexpected" output should go to `stderr`/`System.err`. That's when we expanded the set of `WriterKind` in `Log` from 3 to 5, and providing access to both kinds of output stream.  Note, this does *not* mean 5 streams; it just means a mapping from 5 reasons to the one of the available streams.

*Before:* `ERROR, WARN, NOTE`    *After:* `ERROR, WARN, NOTE, STDOUT, STDERR`

So, depending on why you wanted a stream, you could get the right one.   At one point, IIRC, javadoc allowed you to specify 3 streams (!!) for error, warning, notes, but generally, we settled on 1 or 2 streams, representing `System.out` and `System.err` and aligned per the tool's choice for the diagnostic kinds.  `javac` went "all-in" for "all diagnostics to `STDERR` and command-line help, etc to `STDOUT`.   javadoc was going through its own turmoil at the time (new doclet + HTML5) and so we left javadoc alone, supported by some deprecated constructors in `Log`, to give javadoc what it needs.  So, that's generally the history of javac and javadoc streams.

I'm looking to eventually align `javadoc` with `javac`: all diagnostics go to `STDERR` and direct access to `STDOUT`(in particular) when that is appropriate.  That will be a separate commit/PR, with a tiny incompatibility that most folk will not notice.  It will mean that diagnostics NOTEs written from a doclet when javadoc is run from the command line will then appear on `STDERR` instead of `STDOUT` as now.  For the standard doclet, we will update it so that all those tracing messages (like `Generating .... file.html`) will be written directly to `STDOUT` and not written with `Reporter.printNote`, so the mainstream use of the standard doclet will not be affected.   But, all that change will be in a follow-up PR. For now I'm not changing anything in any incompatible way. This PR is just about exposing the necessary underlying streams.
And, I still believe the `DocletEnvironment` is the right place to make the streams available; not the `Reporter` interface.

Note, I use terms like `System.out` and `System.err` but think of them as "conceptual terms, and not always the exact same-named streams. Sometimes, alternative streams for the same conceptual purpose can be passed in to constructors etc.

-------------

PR: https://git.openjdk.java.net/jdk/pull/4077