Explicitly empty sourcepath regression

Wed Jun 21 01:23:11 UTC 2017

I'm still very interested in hearing your thoughts on the issues I brought
up in the previous post to this thread, but I wanted to share an update
with this group about how the Gradle team has decided to handle the
sourcepath situation for our 4.1 release.

We have decided to just drop the `-sourcepath ""` arguments from our
`javac` invocations when we know that we are compiling for Java 9 AND that
there is a `mould-info.java` on the command-line.

We considered writing our own `JavaFileManager` implementation to wrap the
standard `JavaFileManager` and filter results from calls to `list()` with
`SOURCE_PATH` to eliminate any source files which were not already known to
Gradle. However, this would have meant that we would need to synthesize a
sourcepath which contained all of the source files we were passing on the
command-line so that the standard `JavaFileManager` could find the files in
the first place. Rather than tackle this complexity at this point, we
decided to settle for the implicitly empty `sourcepath` which seems to work
when compiling a single module.

If you change the behavior of the implicitly empty `sourcepath` to match
that of the explicitly empty `sourcepath` Gradle will not work with the
release of the JDK that makes that change. So, it would be nice to get a
warning if the decision is made to change those semantics to match.

Thanks,
Pepper

On Mon, Jun 19, 2017 at 11:07 AM Pepper Lebeck-Jobe <pepper at gradle.com>
wrote:

> A few (fairly inconsequential comments in-line and then an unresolved
> question at the bottom. Sorry to put the most important part last. Feel
> free to skip ahead to that question as I think it is the most important
> thing in the mail.
>
> On Mon, Jun 19, 2017 at 12:30 AM Jonathan Gibbons <
> jonathan.gibbons at oracle.com> wrote:
>
>>
>>
>> On 06/15/2017 09:07 PM, Pepper Lebeck-Jobe wrote:
>> > *Question*
>> > Why must the source files which make up a module be on the source path
>> for
>> > the module to be compiled?
>>
>> There are a number of aspects to the answer for this.
>>
>> First, some background.
>>
>> It has been a decision of long standing that the module membership of a
>> compilation
>> unit is determined by the "host system", which in the case of
>> command-line javac,
>> means by the position of the file in the file system. The alternative,
>> which was rejected
>> early on, was to have to edit every single compilation unit in code that
>> was being
>> modularized to have a module declaration at the top, preceding the
>> package declaration.
>>
>
> I'm wondering if another alternative was considered. Namely, require
> explicit declaration in the module declaration for all packages which make
> up the module (even if they aren't exported by the module.) This would
> provide the same mapping as having each compilation unit declare module
> membership, but it would be consolidated in the module description and not
> require specification for every class declaration as membership in a
> package would imply membership in the module which has declared ownership
> of that package.
>
>
>> While it may seem to follow that for any compilation unit, you could
>> just look in some
>> appropriate enclosing directory to find the module declaration, there
>> are some
>> important use cases where that is not enough. Generally, these use cases
>> are when
>> the source for a module is spread across different directory hierarchies
>> with different
>> directories for the package root. The two most common cases of this are
>> when
>> some of the source for a module is generated, or when the source is a
>> combination
>> of some platform-independent code and some platform specific code. The
>> ability to
>> merge directory hierarchies like this is managed by using paths, as in
>> source paths
>> or class paths.
>>
>> Now, to some more specific reasons for the design decision.
>>
>> First ... consistency. It has always been the case for Jigsaw javac that
>> when compiling
>> code from multiple modules together, all the source files given on the
>> command line
>> had to be present on the module source path .. meaning, on the source
>> path for
>> a module on the module source path. That was always required for the
>> reasons described
>> earlier, to be able to determine the module membership of each source
>> file specified on
>> the comment line.  That was initially different from the case of
>> compiling a single module,
>> which initially was more like compiling "traditional" non-modular code.
>> In time, it became
>> clear that was a bad choice and it was better to have the compilation
>> requirements
>> for all modular code be more consistent, whether compiling one module or
>> many.
>>
>
> But, right now, I don't think these requirements actually are consistent
> between compiling one module or many. For example, imagine this directory
> layout.
>
> src/
>   module-info.java
>   com/
>     foo/
>       ModuleClass.java
> otherSrc/
>   com/
>     bar/
>       NotAModuleClass.java
>
> If I compile with this command line:
> javac -d build/classes/foo.module -sourcepath src:otherSrc $(find . -name
> "*.java")
>
> Then, javac will assume that com.bar.NotAModuleClass is a member of
> `foo.module` (the module declared in src/module-info.java) even though it
> is not rooted in the same filesystem directory with the module declaration
> file or in a directory with the same name as the declared module. Is the
> reasoning behind the design of this behavior summed up by this rule:
>
> if (there is exactly one module declaration among the sources on the
> command line) {
>   all source files visible on the sourcepath must be part of that module
> }
>
> I'm not saying that this is a totally unreasonable rule. But, it does seem
> less strict than what would happen if multiple modules were being compiled
> with the --module-source-path argument and there was some rouge class file
> which wasn't on the --module-source-path but was specified on the
> command-line.
>
> Second ... to avoid obscure errors. In the initial design, when
>> compiling a single module,
>> javac tried to infer the module being compiled from the presence of a
>> module declaration
>> in a compilation unit specified on the command line, or on the source
>> path or a module
>> declaration on the class path.  That led to the possibility of errors in
>> which the module
>> membership  of a compilation unit specified on the command  line (as
>> determined by its
>> position in the file system) could be different from the inferred module
>> being compiled.
>> For example, consider the case where the source path is empty, and the
>> command line
>> contains the source files for the module declaration for one module, and
>> the class
>> declarations for different module. There is no way, in that case, for
>> javac to detect the
>> misconfiguration and give a meaningful message. The solution was to
>> require that when
>> compiling modular code for a single module, all source files must appear
>> on the source
>> path so that javac can ensure that all sources files are part of the
>> same module.
>>
>
> I guess this answers my previous question. So, yes, the rule is: if a
> source file is both specified on the command line with a module declaration
> and in the sourcepath, then it must be meant to be part of the module being
> declared.
>
> I do believe this makes the semantics of the sourcepath difficult to
> understand because it has a different/additional meaning depending on
> whether or not a module-info.java file is among the sources being compiled.
>
>
>> That all being said, I understand the concerns that this sets up the
>> possibility of
>> files being implicitly compiled when that is not desired, by virtue of
>> being found
>> on the source path. I also agree that -implicit:none is not an ideal
>> solution to the
>> issue.  In time, we could consider a new option to better address the
>> problem.
>> In the meantime, the short term options are to either synthesize a
>> suitable source
>> directory with links, or to use the Compiler API to provide a custom
>> file manager that
>> uses an equivalent synthetic source path.
>>
>
> In the short-term, we can probably synthesize a temporary directory with
> all the source files we know about and use that directory as the only
> element on the sourcepath when we know we are compiling a single module.
>
> -- Jon
>>
>
> I really appreciate your detailed explanation of why the design decisions
> have been made the way they have, but I feel like I'm still missing one
> important piece of the puzzle.
>
> According to this documentation:
> http://docs.oracle.com/javase/9/tools/javac.htm#JSWOR627
>
> ---
> If -classpath, -classpath,, or -cp aren’t specified, then the user class
> path is the current directory.
>
> If the -sourcepath option isn’t specified, then the user class path is
> also searched for source files.
> ---
>
> So, if the default behavior (when -sourcepath is not specified) is
> essentially to treat the classpath as the sourcepath.
>
> Why in my example project here: https://github.com/eljobe/modules
> does this work
> javac -cp '' -d build/classes $(find src -name "*.java")
> but this fail
> javac -sourcepath '' -cp '' -d build/classes $(find src -name "*.java")
>
> It seems to me, that according to the documentation, this is only the
> difference between an implicitly (because the -cp option is empty) empty
> sourcepath and an explicitly empty sourcepath.
>
> One possibility is that the documentation is wrong and the default
> sourcepath includes some directories in addition to the classpath. In which
> case, I'd like to know what those directories are.
>
> I'm not sure what other explanation would cause this difference in
> behavior.
>
> Thanks again,
> Pepper
>