Refactorability proposal...

[Bryan Atsatt]

Stanley M. Ho wrote:
> Hi Bryan,
>
> I agreed that refactorability is indeed an important issue we need to 
> address. That said, there are couple concerns I have related to step 
> 1b. I'm not arguing against import-by-package at runtime; only against 
> the transformation of import-by-module to import-by-package at 
> compile-time.
>
> A compile/build-time transformation from import-by-module to 
> import-by-package could fundamentally be done in one of two ways.
>
> First, it could involve analysis of the module's method signatures. 
> The issue is that the analysis may not be accurate all the time (e.g. 
> code that uses reflection, Class.forName(), etc.). The programmer will 
> then be required do extra trial-and-error steps to fix up the 
> dependencies in the metadata, and this is not acceptable.
>
> Second, it could simply expand import-by-module as the set of exported 
> packages in the imported module, then there is a problem when 
> migrating from one version of an imported module to a newer one and 
> the set of packages have changed. Suppose a core module in JDK 7 
> exports package P and the one in JDK 8 exports packages P and Q. If a 
> module imports JDK 7+ and is compiled against JDK 7, step 1b will 
> transform the module to import only the packages that are known in the 
> core module in JDK 7. When this module runs in JDK 8 or later, it will 
> not see the new packages. This is problematic because many modules 
> will compile against an older version of the JDK and want to call new 
> API in newer JDK through reflection; in general, this situation 
> applies to using other libraries as well. This will also cause problem 
> in applications that involve weaving at runtime and the exact required 
> packages are not known at compile-time.
This is why OSGi has dynamic-import. And we need it as well. The fact 
that a forName/loadClass() happens to work *without* an explicit import 
is nothing but a ticking time-bomb.
>
> There are some general problems with expanding modules to packages at 
> compile/build-time, regardless of how the expansion is achieved.
>
> First, there are modules (e.g. classpath module) that cannot logically 
> have exported packages which are known at compile-time and at runtime, 
> thus it would be impossible to perform the transformation in this 
> case. (Technically, we can go through every JAR in the classpath in 
> every launch to find out the exports of the classpath module at 
> runtime, but this has huge startup performance impact.)
This seems like a very special case to me. First, a "classpath" module 
is, by definition, a very strange beast in that it is fully dynamic: any 
importer can be broken at any time by a simple command-line change. I 
can certainly understand a classpath module existing to support legacy 
code, but I have a hard time imagining creating a new module that would 
express a dependency on such a thing.

Even so, I concede that there are likely special cases where import by 
"something other than package name" at runtime is useful.

Really, I just want to make it possible for a module developer to be in 
a position to "know" that splitting one module into multiple modules 
will not break existing importers. I'm quite open to other solutions, as 
long as they are reasonable.
>
> Second, it implies that people export packages but import modules. 
> This will be hard to explain to beginners.
So explain that you both export and import packages, but we provide 
syntactic sugar for both for convenience. (I'd even by happy to 
eliminate that convenience on the import, to simplify things further, as 
you know.)

This is a silly argument IMO (sorry). In modern IDEs, when you type in 
an unqualified class name, the system pops up and asks you which one you 
mean. Without it, you have to (gasp) type a bunch of import statements 
at the top of your file. Why didn't James make this easier by allowing 
us to import a jar?
>
> Last, but perhaps the most important, it means that unresolveable 
> packages at runtime will generate error messages which the programmer 
> must manually map back to imported modules and the transformation 
> process implemented by a tool. The fundamental aim of a programming 
> system should be to reduce the semantic gap between what a programmer 
> sees in code and what a programmer sees at execution; the mapping from 
> packages back to modules increases the semantic gap.
I agree there is a semantic gap, but not the one you cite.

Either refactoring is supported, or it isn't. If it isn't, this 
particular error should not occur. If it is supported, but an 
import-by-module exists, it will be even worse than you suggest: there 
won't be *any* failure at resolution time.

Say I import "module X", which contains package X and also happens to 
contain packages Y and Z which I "silently" depend on. I compile and run 
fine, but in some future X is split into X, Y and Z modules, and, for 
some reason module Z is not installed. With my proposal, resolution for 
my module will fail ("package Z not found") iff there is no installed 
module containing Z.

Without my proposal (or some other similar solution), resolution will 
succeed when it shouldn't, and a NoClassDefFoundError will occur at some 
random point in execution naming a class in a package that was never 
explicitly imported. How is that better?
>
> I accept that pushing module imports through to runtime makes it hard 
> to "rewrite" a program (i.e. to refactor module members) but it makes 
> it easy to "read" the program (in the sense of a small semantic gap 
> between what imports are read/written and what imports are executed). 
> A core Java principle has always been that reading is more important 
> than writing, so we do not believe increased ease of refactoring on 
> relatively rare occasions is worth complicating every single reading 
> of a dependency.
Well there's a distinct difference between our perceptions: "relatively 
rare occasions" strikes me as extremely misleading. Though I have 
significant, direct, and exactly analogous experience that tells me the 
frequency will be higher than you seem to expect, I'm perfectly happy to 
concede that it may be lower. But unless you believe that it will be 
zero, we need a tenable solution.
> I still think the refactorability problem you brought up is very 
> important to address. 
Well at least we can agree on that :^). Any ideas that are less ugly 
than mine?

// Bryan