Compiling Java 9

[Alex Buckley]

Hi Stephan,

You have a lot of questions here, mainly not about the OpenJDK 
implementation of JSR 379 but rather about JSR 379 itself. I'm working 
on JLS text that will be presented alongside lang-vm.html in the next 
few weeks, but here are some quick points:

- lang-vm.html does not say that restricted keywords are keywords only 
inside a module declaration; it says they are keywords _solely where 
they appear as terminals in ModuleDeclaration_.

- Bear in mind that the JLS has always been very abstract about where 
source code comes from. For example, the storage of tokens matching 
CompilationUnit has never been mandated to be a file called XYZ on a 
hierarchical filesystem. (Note that JLS 7.6 merely allows the host 
system to _choose_ to enforce a particular rule.) Nor has a compiler 
been mandated to understand a "classpath", or a "sourcepath". With 
modules, the main JLS issue is that observability of compilation units 
is no longer the sole domain of the host system, or the same for all 
compilation units. For a compilation unit in a given module, the host 
system will be mandated to limit the compilation units which are 
observable based on the result of resolving the given module's 
dependencies with the Java Platform Module System.

- Accessibility is a secondary issue because while it's changing in 6.6 
to integrate modular exports, everything that relies on accessibility 
such as overload resolution in 15.12.2.1 then "just works". Yes, there 
are some references to 'public' which need to be qualified as 'public in 
an exported package', such as in 8.4.8[.*], but this is mostly 
mechanical. There are some technical changes to the scope of a top level 
package, but qualified names in general do not change their structure or 
meaning.

Alex

On 11/8/2016 10:28 AM, Stephan Herrmann wrote:
> I'm wondering what will be expected from a compiler for Java 9.
> I hope this is a suitable place to ask this kind of questions.
>
>
> The first thing we know: a compiler has to be able to translate module-info.java (or whatever that file's name may be - btw: in absence of a definite rule, is a compiler expected to find module declarations in files of any possible name?).
>
> My first question concerns the grammar of module declarations:
> I don't think the definition of "restricted keywords" and javac speak the same language. The only thing I can definitely infer from lang-vm.html is: restricted keywords are keywords only inside a module declaration (btw: how does the parser know, it's inside a module declaration?). By contrast javac implements a much more permissive strategy, almost s.t. like: "they are only keywords when they are keywords, otherwise they are identifiers". To wit, javac's parser happily accepts this declaration:
>     module module {            // second word is an identifier
>         requires requires;     // second word is an identifier
>         exports to to exports; // words #2 and #4 are identifiers
>         uses module;           // second word is an identifier
>         provides uses with to; // words #2 and #4 are identifiers
>     }
> This is not only very weird, there are lots of terminals in a module declaration where a restricted keyword is accepted as an identifier. It also breaks with standard compiler technology: it doesn't seem to be possible to use a traditional pipeline of scanner & parser, if the scanner cannot decide the kind of a token without consulting the parser (or keeping part of the parser state in the scanner itself).
> This is to say: the strategy implemented by javac will affect all tool providers, not only compilers, but also editors etc. In absence of a stateless scanner, even syntax highlighting may not be possible without a full parser. Parsing in the presence of syntax errors (to offer IDE functionality on work-in-progress files) becomes close to impossible.
> Have these consequences been carefully weighed?
>
> The next area where compilers will have to change is the new semantics of "accessible". Clearly, rules like JLS 6.6.1. (Determining Accessibility) and JLS 8.4.8.1. (Overriding (by Instance Methods)) need to be changed.
> Has any of this work started yet? Can we have a look at any drafts?
>
> Next, I wonder, how much additional static validation compilers should perform.
> For one, SOTMS speaks of "reliable configurations". Are compilers expected to check these rules? If so, have they been spelled out in checkable form somewhere (e.g., what is the meaning of "interfere" in the sentence "modules defining identically-named packages do not interfere with each other")?
> Secondly, the concept of modules makes it possible to create new forms of "API leaks": expose a type, whose public signatures mention inaccessible types - maybe inaccessible only to some consumers. If you add overloading to the mix, this can create ugly effects, which may defeat benefits of modularity.
> Is it planned that tools will help users to avoid such bogus situations? Or at least gracefully handle the case when it occurs?
> Obviously, some validity constraints can be checked by tools other than the compiler, like linker or VM, but defining a module system as part of the language bears the unique opportunity to provide the most immediate feedback to users. Will this opportunity be used?
>
>
> Other concerns are more of an organizational kind:
>
> What modules must be present to compile a given module? Is the set of (implied) readable dependencies sufficient, or is the compiler required to look at the full transitive closure?
> Users coming from a plain-classpath world, may not expect much sophistication, but introducing a module system bears the unique opportunity to avoid the situation where building just one module triggers the download of huge amounts of transitive dependencies ("downloading the internet").
> Which is to say: Java 9 has the chance to significantly reduce the resource requirements (bandwidth, storage, main memory, compile time) for compiling any given module.
> Any attempts in this direction seem to require a fresh look specifically at the interaction of accessibility and overloading. Has anything been planned/discussed in this direction?
>
> In Java 8, any package-qualified type name is sufficient for identifying a type during compilation. Will the same assumption also hold for compiling Java 9, or is a compiler required to distinguish identically named types from different modules? While this seems to be a question of compiler implementation only, it also seems the answer depends on answering most of the above questions, like additional static validation, the need to look at the transitive closure of module dependencies etc.
> Most importantly, for implementing a compiler for Java 9 this question implies a very fundamental design decision, that should be made before any other changes towards Java 9. Changing a compiler implementation from package.type names to module.package.type names is a huge change that cannot be implemented as a quick after thought.
> Have any amendments to JLS chapter 6 been drafted to define the new requirements for a compiler regarding qualified names?
>
>
> regards,
> Stephan
>