Potential bugs when using the JavaCompiler API.
Jonathan Gibbons
jonathan.gibbons at oracle.com
Fri Dec 18 22:17:09 UTC 2020
Hi,
Thanks for your interest in this area.
The bugs, such as they are, are more in the specification than the
implementation, and are somewhat tricky to fix as well as might be
expected. There is one task that is part bug, part enhancement,
depending on your point of view, and equally tricky to figure out, which
is one of the reasons it has not been done before now.
Generally, you should not think of the `JavaFileManager` API and
`JavaCompiler.getTask` options as equivalent. The file manager is a
lower level abstraction than the compiler, and applicable in more
contexts than `javac`. By itself, it is not wrong for a file manager to
have combinations of locations set, because there may be situations
(i.e. some tools) where that is desirable and appropriate. And,
incorrect settings for a file manager are generally reported by throwing
`IllegalArgumentException` in preference to using a diagnostic
listener. And, while it does not make sense to allow conflicting
options like `--source-path` and `--module-source-path` on the javac
command line, it seemed better to permit file managers with both set and
to figure a reasonable policy to handle that situation.
Also, note that when using the Compiler API and `JavaCompiler.getTask`,
the compiler has to be able to accept any suitable implementation of a
file manager, and not just the standard one returned by the JDK
implementation of
`ToolProvider.getSystemJavaCompiler().getStandardJavaFileManager`. And,
for better or worse, there is no API on a file manager to determine if
errors have occurred, in a way that would be reasonable for the
internals of javac to detect and react to.
Your comments related to your example with `task1` and `task2` are
somewhat reasonable. Although it is on one hand not surprising that
options to getTask affect the file manager, on the other, it would be
nice if the changes were transient. The general idea we have had in the
past is to create a temporary file manager inside the compiler that
wraps and delegates to the given file manager, except for the arguments
given to the options. However we have not thought through any detailed
design to make that happen.
As for the specification, yes, the API in this area is somewhat
under-specified. Part of it is a general problem with JDK API
specifications: both JavaCompiler and JavaFileManager are
service-provider interfaces, and while it is easy enough to specify the
common properties of all implementations of these interfaces, there is
not a good way/place to specify the additional characteristics and
properties of specific implementations. For the JavaCompiler API, we
want to allow other vendors to provide alternate implementations of a
Java Compiler (subject to all the appropriate conformance rules), and so
it is inappropriate in the Java SE class `javax.tools.JavaCompiler` to
give JDK-specific details of the JDK implementation of that interface.
And, for any implementations of `JavaFileManager`, we don't want to
over-specify it and so restrict the kinds of implementations anyone can
provide. The one thing of note is that whereas there was no good
solution to this specification problem in JDK 6, since JDK 9, and the
Java Platform Module System, we do have a reasonable place to put some
of this specification, and that is the module documentation for the
`jdk.compiler` module. In that module, we could write paragraphs that
give additional specification details for the implementation of the
`JavaCompiler` interface provided by the module, and for the standard
file manager returned by the `getStandardJavaFileManager` method of that
implementation. It's "not great" that the documentation with be
narrative paragraphs not directly linked into the shared documentation
in the `javax.tools` interface, but at least it would be somewhere,
which is better than now.
-- Jon
On 12/18/20 6:38 AM, Guoxiong Li wrote:
> Hi all,
>
> When I was working on the JDK-8232939[1], I found some strange points
> and suspicious bugs. The origin of the issues is the `Context` of
> `StandardJavaFileManager`(eg: JavacFileManager) and the `Context` of
> `CompilationTask`(eg: JavacTaskImpl) is not same. As we can see in
> class `JavacTool`, the methods `getStandardFileManager` and `getTask`
> both create its new `Context`.
>
> When creating its new `Context` to isolate. We can call it as the
> *decoupled* part of `StandardJavaFileManager` and `CompilationTask`.
> When using `getTask` with a file manager to get a `CompilationTask`,
> the `getTask` will set some properties of the file manager. We can
> call it as the *coupled* part of `StandardJavaFileManager` and
> `CompilationTask`.
>
> The decoupled and coupled part of `StandardJavaFileManager` and
> `CompilationTask` are strange and both have suspicious bugs which are
> shown below.
>
> 1. Bugs about the decoupled part.
> The properties of `JavacFileManager`, especially `Log` and
> `Options`, are not same as the corresponding items of `JavacTaskImpl`.
> Some issues will occur.
>
> The first example is that the OptionGroup.FILEMANAGER options
> won't be validated. This is a bug about the different `Options`.
> As you can see the test code[2]. `--module-source-path` and
> `--source-path` are both set. It is wrong because the man document
> states that users can use only one of `--module-source-path` and
> `--source-path`, and can't use both. These wrong code are accepted by
> the compiler now. Because when using the compiler API, the options
> property of file manager only contain the OptionGroup.FILEMANAGER
> options and the options property of compiler task only contain
> the OptionGroup.BASIC options. When validating options[3], only the
> options of compiler task will be validated. That is said,
> the OptionGroup.FILEMANAGER options won't be validated. It is
> unacceptable because the manner of compiler API is different from the
> compiler command line.
>
> The second example is JDK-8232939[1]. It is a bug about the
> different `Log.`
> The file manager generates the errors during the parse stage and
> reports the error by using its own `Log`. After the parse stage, the
> compiler uses `stopIfError` to check if the errors occur and will stop
> the later stages if any error occurs. Unfortunately, the method
> `stopIfError` checks the `Log` of the compiler instead of the `Log` of
> the file manager so that it can't find any error generated by the file
> manager. So the compiler continues the later stages unexpectedly.
>
> These two bugs(one about `Options`, another about `Log`) show that
> current decoupled way is not so good. If we deep into all the
> properties of these two different context, some more bugs may be found.
>
> 2. Bugs about the coupled way.
> This aspect are more important and more unacceptable. As you can
> see the code snippet below.
> ```
> var compiler = ToolProvider.getSystemJavaCompiler();
> var fm = compiler.getStandardFileManager(null, null, null);
> var files = fm.getJavaFileObjects(Path.of("Test.java"));
> JavaCompiler.CompilationTask task1 = compiler.getTask(null,
> fm, null, List.of("-XDrawDiagnostics", "-encoding", "ASCII"), null,
> files);
> JavaCompiler.CompilationTask task2 = compiler.getTask(null,
> fm, null, List.of("-XDrawDiagnostics"/*some more options*/), null, files);
> System.out.println(task1.call());
> System.out.println(task2.call());
> ```
> The OptionGroup.FILEMANAGER options of `task1`("-encoding",
> "ASCII") will be taken to `task2` and vice versa. Because the options
> of file manager won't be cleared. It required the users clear the
> options manually. Unfortunately, the users, included me, use it
> incorrectly. The probably correct usage is shown below: run the first
> task and clear the information of previous task before getting another
> task. As you can see, the clear work is hard because the users don't
> know the internal of the compiler. Currently I can't find a good way
> to clear, too.
> ```
> var compiler = ToolProvider.getSystemJavaCompiler();
> var fm = compiler.getStandardFileManager(null, null, null);
> var files = fm.getJavaFileObjects(Path.of("Test.java"));
> JavaCompiler.CompilationTask task1 = compiler.getTask(null,
> fm, null, List.of("-XDrawDiagnostics", "-encoding", "ASCII"), null,
> files);
> System.out.println(task1.call());
> // Do some clear work. Omit.
> JavaCompiler.CompilationTask task2 = compiler.getTask(null,
> fm, null, List.of("-XDrawDiagnostics"), null, files);
> System.out.println(task2.call());
> ```
> As the document stated:
> > The standard file manager serves two purposes:
> > basic building block for customizing how a compiler reads
> and writes files
> > sharing between multiple compilation tasks
> But the options and task's own individual other information
> perhaps shouldn't be shared.
>
>
> To solve the problem completely, we should find a better way to
> decouple file manager and compilation task instead of using the new
> `Context` directly and simply.
> I have been thinking of a better solution these days but I can't find
> a satisfying solution. I want to share the problem with you to see
> your opinions. This is the reason of this email.
>
> Any opinion or idea are appreciated!
>
> [1] https://bugs.openjdk.java.net/browse/JDK-8232939
> <https://bugs.openjdk.java.net/browse/JDK-8232939>
> [2]
> https://github.com/openjdk/jdk/blob/853c04712d24788b6d30d54443ac9277a898311b/test/langtools/tools/javac/modules/AnnotationProcessing.java#L1607
> <https://github.com/openjdk/jdk/blob/853c04712d24788b6d30d54443ac9277a898311b/test/langtools/tools/javac/modules/AnnotationProcessing.java#L1607>
> [3]
> https://github.com/openjdk/jdk/blob/853c04712d24788b6d30d54443ac9277a898311b/src/jdk.compiler/share/classes/com/sun/tools/javac/main/Arguments.java#L409
> <https://github.com/openjdk/jdk/blob/853c04712d24788b6d30d54443ac9277a898311b/src/jdk.compiler/share/classes/com/sun/tools/javac/main/Arguments.java#L409>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.java.net/pipermail/compiler-dev/attachments/20201218/ab928e39/attachment-0001.htm>
More information about the compiler-dev
mailing list