Compiling Java 9 (take 2)

[Alex Buckley]

On 1/4/2017 12:44 PM, Stephan Herrmann wrote:
> I have started to dig through the JLS changes (2016-12-16) to figure out
> requirements for a Java 9 compiler without consulting javac, but I'm
> having some difficulties understanding the interplay of scope,
> visibility, importing, accessibility and observability at this level.
>
> For instance, this already starts when re-reading unchanged parts from
> 6.3 and 6.4.1:
> 6.3:
>      The scope of a declaration is the region of the program within
> which the entity declared by the declaration can be referred to using a
> simple name, provided it is visible (§6.4.1).
> 6.4.1:
>      A declaration d is said to be visible at point p in a program if
> the scope of d includes p, and d is not shadowed by any other
> declaration at p.
> => For a declaration to be in scope it must be visible, and for being
> visible it must be in scope. This is beyond me.

Yes, the historic wording around scope et al is often troublesome. (As 
an example, see JDK-4420532.) The precise "layering" of one concept on 
top of another is hard to see.

> Is it correct (and intended) that "visibility of a declaration" (6.3) is
> a different concept than "visibility of a compilation unit" (7.3)? (Are
> we so entangled with overloading that we need to overload the terms in
> our spec, too? :) ).

Yes, and I owe you some apologies for this. The "visibility of a 
declaration" is an obscure corner of the scope rules, and I intend to 
rename it. Visibility will unambiguously be "of a compilation unit", 
since that's key to how the Java Language supports the JPMS.

> I'm also surprised that some requirements are indeed specified with
> regard to compilation units. Modules export packages, and types are
> declared in packages. Isn't it possible to define requirements due to
> JPMS solely in those terms (modules, packages, types), and avoid to
> mention compilation units in these rules? What do compilation units add
> conceptually to JPMS (besides adding complexity)?

Per 7.3: "The ordinary compilation units that are visible to M drive the 
packages that are visible to M (§7.4.3), which in turn drives the top 
level packages in scope for code in compilation units associated with M 
(§6.3)."

That is, the effect of 'requires' statements is to move from a flat 
space of observable compilation units to a module-keyed space of visible 
compilation units. If a compiler is compiling module M that requires N, 
and module O that requires P, then the compiler has decided that 
compilation units in M and N and O and P are observable, but when 
processing a compilation unit in (say) M, only the compilation units in 
N (and not O or P) are visible.

> As I already mentioned overloading, here's my example of the day:
>
> //--- Base/module-info.java
> module Base {
>      exports base;
> }
> //--- Base/other/Other.java
> package other;
>
> /** I'm not exported */
> public class Other {
>      public void test() {
>          System.out.println("Other.test");
>      }
> }
> //--- Base/base/Base.java
> package base;
>
> import other.Other;
>
> /** I'm exported */
> public class Base {
>      public void test(Object other) {
>          System.out.println("Object");
>      }
>      public void test(Other other) {
>          System.out.println("Other");
>          other.test();
>      }
> }
> //---
>
> //--- Test/module-info.java
> module Test {
>      requires Base;
> }
> //--- Test/other/Other.java
> package other;
>
> public class Other {
> }
> //--- Test/test/Test.java
> package test;
>
> import base.Base;
> import other.Other;
>
> public class Test {
>      void test(Base b, Other other, Object o) {
>          b.test(o); // OK
>          b.test(other); // ??
>      }
>      public static void main(String[] args) {
>          new Test().test(new Base(), new Other(), new Object());
>      }
> }
> //---
>
>
>
> Firstly, I'm surprised that javac (ea-149) compiles this fine but trying
> to run the program under JPMS crashes with
> Error occurred during initialization of VM
> java.lang.reflect.LayerInstantiationException: Package other in both
> module Base and module Test
>          at java.lang.reflect.Layer.fail(java.base at 9-ea/Layer.java:716)
>          at
> java.lang.reflect.Layer.checkBootModulesForDuplicatePkgs(java.base at 9-ea/Layer.java:674)
>
>          at
> java.lang.reflect.Layer.defineModules(java.base at 9-ea/Layer.java:610)
>          at
> java.lang.reflect.Layer.defineModules(java.base at 9-ea/Layer.java:383)
>          at
> jdk.internal.module.ModuleBootstrap.boot(java.base at 9-ea/ModuleBootstrap.java:307)
>
>          at java.lang.System.initPhase2(java.base at 9-ea/System.java:1927)
>
> Is that because the compiler knows nothing about layers?
> Is there any tool that statically checks whether a given modular program
> can possibly run in the default Layer implementation?

How did you invoke javac? Which mode of javac (see "Compile time" in JEP 
261) are you in?

> Secondly, I'm confused how overload resolution for b.test(..) should
> happen.
> We have two candidates, where one parameter type is accessible (Object)
> and the other is not (other.Other).
> In all of JLS chapter 15 (2016-12-16) I cannot find anything that would
> allow the compiler to consider accessibility of parameter types during
> overload resolution.

Right, nothing has changed in this respect.

> Hence, I see two possible answers by a compiler:
> (1) resolve b.test(other) to test(Other) because both types other.Other
> appear to be the same type, and hence test(Other) is applicable and more
> specific than test(Object).
> (2) reject the program because of a name clash on other.Other.
>
> Javac (ea-149) does neither, but resolves b.test(other) to test(Object).
> Apparently, javac "knows" that Base/other.Other and Test/other.Other are
> distinct types.

It sounds like you're compiling in multi-module mode, where javac does 
indeed know this.

> Given that types are identified by qualified names that
> do not contain the module name, this distinction doesn't seem to be
> possible per JLS.

Per 7.3, javac is associating a first other.Other type with module Base, 
and a second other.Other type with module Test. The first and second 
types have no relationship to each other, so there are no conversions 
between them, so Base's test(other.Other) method is not applicable to 
the invocation b.test(other).

It's true that 7.3 also says that all the compilation units of Base are 
visible to Other, since Other reads Base. javac is effectively hiding 
Base's other.Other type from code in Other, since Other already has an 
other.Other type. It's possible we need to specify this explicitly.

Alex