StandardDoclet requires private DocletEnvironment (due to a cast in WorkArounds)

[Dawid Weiss]

I thought I'd revive this old thread -- didn't have time to check your
suggestion until now, Jon. I tried the DocumentationTool approach.
You're right - there's a lot of magic going on behind the scenes in
javadoc... :) Please read if you have a spare moment (nothing urgent).

So I set up a basic snippet that generates javadoc from code and it
works... until you wrap it with an (empty) ForwardingJavaFileManager.
So something like this works:

JavaFileManager fileManager =
    jdoc.getStandardFileManager(diagnosticListener, Locale.ROOT,
StandardCharsets.UTF_8);

// fileManager = new ForwardingJavaFileManager<>(fileManager) {};

List<String> options =
    List.of(
        "-d", "output", "-subpackages", "com.carrotsearch",
        "-sourcepath", "src/main/java");

DocumentationTool.DocumentationTask task =
    jdoc.getTask(extraOutput, fileManager, diagnosticListener, null,
options, compilationUnits);

But as soon as you uncomment forwarding file manager line the output becomes:

> NOTE: javadoc: Loading source files for package com.carrotsearch...
> ERROR: compiler message file broken: key=javadoc.err.msg arguments=javadoc, No source files for package com.carrotsearch, {2}, {3}, {4}, {5}, {6}, {7}
> NOTE: javadoc: 1 error

The difference has to do with SOURCE_PATH location lookup (when
javadoc is calling fm.list(...)). The returned file manager is
JavacFileManager but this check in Start.java only conditionally
passes certain options:

if (fileManager instanceof BaseFileManager) {
    ((BaseFileManager) fileManager).handleOptions(fileManagerOpts);
}

I worked this around by passing the required option explicitly
(created a full delegate
to StandardJavaFileManager to get access to setLocation):

fileManager = new ForwardingStandardJavaFileManager(fileManager);
fileManager.setLocation(StandardLocation.SOURCE_PATH, List.of(new
File("src/main/java")));

then overrode getFileForOutput, wrapped everything in a zip file
system and - behold - it worked! :)

It's all rather hacky but I ran it successfully on Lucene (core). I
didn't see much in terms of speed improvements -- one average one
second faster than full disk-writing version:

no output: ~10.5s   -
Zip:  ~11s   size on disk: 5.3 MB
Disk (SSD): ~11.5s, size on disk: 27.6 MB
Disk (HDD): ~12s,  size on disk: 27.6 MB

I still think such a "zip" destination would be handy. It'd make
cleanup or dependency checks in build systems like gradle faster
(fewer files to check and/or remove).

Anyway, enjoyed the experiment - thank you for the hint!

Dawid



On Thu, May 7, 2020 at 4:59 PM Jonathan Gibbons
<jonathan.gibbons at oracle.com> wrote:
>
> Dawid,
>
> I agree that the situation is less than ideal.  The possibility/desire
> to subtype the standard doclet and to provide a modified environment
> was not anticipated when we wrote the current version of the doclet ...
> and even the need for that internal 'WorkArounds' class is less than
> ideal, as you might guess from the name.
>
> But, keep asking these questions; it helps us to know how folk are
> trying to use tools like javadoc.
>
> -- Jon
>
> On 5/7/20 7:40 AM, Dawid Weiss wrote:
> > Thanks Jon. It is a pity there is so little flexibility around the
> > standard doclet (or its HTML producer) because rewriting everything
> > from scratch is out of question. I'll take a look at getting it to
> > work via tool provider - thank you for the hint!
> >
> > D.
> >
> > On Thu, May 7, 2020 at 4:33 PM Jonathan Gibbons
> > <jonathan.gibbons at oracle.com> wrote:
> >> Dawid,
> >>
> >> The issue about being difficult to subtype the StandardDoclet is a
> >> known issue.
> >>
> >> You should be able to provide your own file manager if you invoke
> >> javadoc via the API available in javax.tools.
> >>
> >> https://docs.oracle.com/en/java/javase/11/docs/api/java.compiler/javax/tools/ToolProvider.html#getSystemDocumentationTool()
> >>
> >> -- Jon
> >>
> >>
> >> On 5/7/20 1:29 AM, Dawid Weiss wrote:
> >>> Hello everyone,
> >>>
> >>> I had a somewhat wild idea of trying to modify the behavior of the
> >>> StandardDoclet to write directly to a ZIP file (instead of the file
> >>> system). All the pieces fall into place relatively easily:
> >>>
> >>> 1) Create a custom Doclet that extends (public) StandardDoclet and
> >>> overrides the run method with:
> >>>
> >>> public boolean run(DocletEnvironment docEnv) {
> >>>       return super.run(new CustomEnvironment(docEnv));
> >>> }
> >>>
> >>> 2) CustomEnvironment delegates everything to docEnv except it returns
> >>> a ForwardingJavaFileManager over the docEnv.getJavaFileManager with a
> >>> bunch of "get*Input" methods substituted.
> >>>
> >>>
> >>> The only problem is - it doesn't work. :) Internally WorkArounds has a
> >>> cast to an internal implementation type:
> >>>
> >>> public WorkArounds(BaseConfiguration configuration) {
> >>>       [...]
> >>>       this.toolEnv = ((DocEnvImpl)this.configuration.docEnv).toolEnv;
> >>> }
> >>>
> >>> And it's a showstopper (maybe there are others but I didn't have a
> >>> chance to try).
> >>>
> >>> Is there any way forward out of this (or an alternative take I can try)?
> >>>
> >>> Dawid