Comments on the proof

[Jesse Glick]

On 12/25/2011 09:30 AM, Jaroslav Tulach wrote:
> If there is a "master" classloader in each application that
> sees all classes of all available modules (which is sort of true in NetBeans
> Runtime Container

It is worth explaining that this is useful for two reasons:

1. As a default value for Thread.getContextClassLoader(), which many legacy libraries assume can "see" every class in the app (unlike Class.forName(String) which only 
works to find classes in dependencies of the caller). A safer but awkward workaround is temporarily setting the CCL before making any call into such a library, then 
resetting it in a finally block.

2. For loading an implementation class referred to by name from a static resource, where locating the (module) ClassLoader defining that resource is impractical. Bug 
#6865375 [1] gives a typical case. Jigsaw proposes to bypass the problem in the limited case of ServiceLoader by using some kind of index to directly find modules 
providing a given service, but other service locator frameworks are left to find their own solution.

> the requirement that each FQN
> in this "master" classloader represents at most one class naturally leads to
> requirement that "only one version of any particular module of a given name is
> allowed"

Rather, that only one module is allowed to define a given class - i.e. org.w3c.dom.v2 @ 2.0 plus org.w3c.dom.v3 @ 3.0 is just as bad, but having multiple versions of a 
module with the same name is fine so long as all the packages in it are renamed.

The NetBeans master loader simply punts in this case, refusing to load any of the alternatives.

> unless you control and verify state of the whole repository (as Glassfish,
> Eclipse, Fedora, Ubuntu and NetBeans do), you'll face incompatibilities far
> [more] often

Of course a repository of Java modules might be as tightly controlled as the ones mentioned above - but not if it, say, mirrors Maven Central.

> I am glad avoiding NP-completeness is preferred. It increases my enthusiasm to
> seek for a dependency system that would be powerful enough and avoid it.

An apparent precondition of your proof about complete repositories which was not adequately emphasized is that a module must declare itself incompatible if anything it 
reexports is incompatibly changed. As I have written before, this is not a restriction to be taken lightly! Consider that java.beans.PropertyEditor.getCustomEditor() 
returns a java.awt.Component, i.e. module java.beans @ 1.8 requires public module java.awt @ [1.8,2), so anyone with a dep on java.beans [1.8,2) gets an implicit dep on 
java.awt @ [1.8,2) as well. Now suppose that later it is decided to finally just delete java.awt.List.delItems(int,int) which probably no one calls anyway, and to be on 
the safe side java.awt is bumped up to 2.0 accordingly. But now the next release of java.beans must reexport java.awt @ [2.0,3), meaning by the postulated rule it must 
also be marked 2.0. And that means that a module with a property editor based on Swing, never using java.awt.List at all and otherwise happy with java.awt @ [1.8,3), is 
forced to drop support for java.beans 1.x. Indeed a module using java.beans.Introspector for server configuration from properties files and never using PropertyEditor or 
loading any AWT classes whatsoever will also be forced to drop java.beans 1.x. Making the java.beans -> java.awt dependency "optional" does not seem to solve anything, 
since you would still have to mark the new java.beans version incompatible for the benefit of clients which do use PropertyEditor.getCustomEditor.

The situation would be more robust if reexports were dropped, modules were required to explicitly specify all their direct dependencies but given the option of permitting 
more lax ranges according to their actual usage, and the module system permitted multiple versions of a module to be loaded so long as class linkage remained consistent - 
which I think comes down to merely forbidding a module to explicitly import two versions of another module at once (ignoring transitive dependencies), which is 
uncontroversial. Whether such a system has a polynomial resolver, I am not sure.

My proposal for per-package stability in the NetBeans module system optimistically assumes that most incompatible changes are really compatible for most clients and 
checks each affected dependency for actual linkage problems [2]. (Complete rewrites of an API would merit a new module name.) Transitive propagation of potential 
incompatibility is less onerous in such a case, since clients can just keep declaring dependencies on old versions even after unrelated incompatibilities appear, but 
would still feel unintuitive to developers (the author of java.beans in the above example). The NetBeans system is unaffected by the NP-completeness issue (since there is 
no multiversion module repository and hence no version resolution), so propagating incompatibilities is not required to make resolution tractable; a repository-based 
system with a linkage check and no propagation requirement could still be hit with intractable problems in case actual linkage errors appear, but since _inferred_ 
dependency ranges would often be more forgiving than _declared_ ranges, resolution would on average be much simpler.

> in the "modulepath" mode [...the] compiler should detect if two classes representing the
> same FQN are available and emit a compilation error in such case.

Does not seem to be explicitly mentioned as a requirement anywhere, but I agree this is a good idea.

> the module system should check that types are not re-exported without
> using "requires public".

My impression is that the currently proposed "required public" does _not_ work this way - that it is entirely optional, a convenience for the importer. The module-aware 
compiler could try to enforce the restriction you propose... but see above for the problems this can cause for importers using only unrelated subsets of an API.

> if there is an exported class that extends non-
> public type or returns non-public type from a visible method, the compiler
> should emit an error.

This is a good idea, and indeed the language could enforce this sensible restriction even without modules; currently you need to use third-party tools to check for such 
mistakes. Anyway, this is different than what you suggested above - that an exported class should be prohibited from referring in its public signature to an exported type 
from another module unless that dependency were marked "public".


[1] http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6865375
[2] http://wiki.netbeans.org/NbmPackageStability#Handling_possibly_broken_dependencies