The rationale for runtime modularization

[Rafael Winterhalter]

Hi Mark,

thank you for your answer. I am glad to hear that the feedback is taken
into serious consideration.

Here is however one point I disagree with: Maven does not offer
compile-time moduarization. Maven only assures that all dependencies are in
scope but it does not offer any means to cut down this dependency tree to
the set of classes that a user intends to include. Just as an example: my
current work assignment declares a dependency on a framework that defines
107 classes. This framework declares (transitive) dependencies that result
in a total of 7681 classes becoming visible by including this library as a
dependency. Relevant for the project (the "non-internal API") are however
only six classes. Without Jigsaw, this results in a 1200% overhead of
visible classes that can now accidentally be used in my project (via this
single dependency out of many).

Of course, an IDE provides help with locating classes in the framework's
namespace. But it is easy enough to accedentally break out of scope,
especially with unexperienced developers on the team. With the eyes of a
developer for end-user software, this is where I see my biggest benefit of
using Jigsaw. If the Java compiler stops me from using non-exported and
non-imported classes, this is a feature that improves my work and that
Maven does not and cannot support.

On the other side, what I do not want as an end-user is to apply some
startup configuration that I do not understand in order to run a framework
that was previously working out-of-the-box. If it becomes a requirement to
set specific switches before starting my application, this is additional
work that bothers me. Of course, as a library developer I would then rather
discard the modularized bundeling to improve user experience. The missing
library moduarization would however defeat the purpose of project Jigsaw
because an end-user developer is no longer able to only import the relevant
parts of a library.

Despite, if I wanted to offer modularization for my library which is
currently compiled to Java 6, I would still be in trouble. I personally
prefer a migration to Java 8 but many users of my library are still bound
to old versions of Java. Also, I have a competitor (both libraries are
FOSS) that is still supporting Java 5. Because of this, I do not currently
plan to migrate to Java 7 within the next three years. Being bound to Java
6, I would however still like to add a module descriptor to my deployed jar
file. I am sure there will eventually be Maven/Gradle plugins for creating
such module descriptors. With the current model this is however impossible
as the runtime behavior is suddenly dependant on the existance of this
file. This renders a catch 22: end-users have no incentive to migrate
because of libraries not offering modularization. And libraries have no
inceptive to migrate because of the majorities of their users still running
Java 8 and earlier. In my eyes, compile-time moduarlization would resolve
this problem by offering library developers a first-mover advantage.

I am sure that I can say more once all APIs for manipulating the module
graph are in place. I will continue to test the EA builds and share my
experiences once I gained additional insights.

Best regards, Rafael

2015-12-04 1:19 GMT+01:00 <mark.reinhold at oracle.com>:

> 2015/12/3 1:36 -0800, Rafael Winterhalter <rafael.wth at gmail.com>:
> > I am writing this after looking into Jigsaw for about two months and
> after
> > running my open-source libraries and several enterprise applications of
> my
> > current employer using the EA builds.
>
> Glad to hear it!
>
> > In the context of these experiences, I wanted to ask for the rationale of
> > project Jigsaw to enforce modularization at runtime.
>
> Short answer: Strong encapsulation at run time greatly improves both
> security and maintainability.  (Strong encapsulation at compile time
> makes it much easier to prepare for strong encapsulation at run time.)
>
> > In the software I have searched, reflection is predominantly used for
> > interacting with code that is unknown during compilation. Of course,
> > sometimes reflection is used for abstracting code from types but this is
> > extremely rare as generic types pretty much cover this need.
> >
> > In practice, this implies that any reflective invocation requires an
> > explicit module check. This check can be easily forgotten.
>
> The failure in this case should be easy to diagnose, and the remedy
> would be to invoke the Module::addReads method.  (If the target module
> doesn't export the relevant package then you'll need to arrange for that
> too, either via a command-line flag or possibly by some more convenient
> means yet to be devised.)
>
> >                                                            Furthermore,
> the
> > compiler does not remind of missing edges when migrating software either.
> > It seems to me that the assumption for the runtime checks is that
> > invocations might accidentally cross module boundaries.
>
> The checks performed by the reflection APIs themselves are intended to
> match those performed by the compiler and the VM for non-reflective
> operations, so whether accidental or not a reflective attempt to break
> encapsulation will be caught.
>
> >                                                         From the code
> that
> > I looked at, I argue that this is already the primary intention when
> using
> > reflection.
>
> Yes, it's probably more common to use reflection to access types in
> other modules rather than in the same module.
>
> > One might argue that the same holds for reflection on non-public types.
> > However, in this case I believe that security concerns are the main
> reason
> > for enforcing accessability where access can be denied by a security
> > manager.
>
> That concern also applies to public types which are not meant to be used
> in unintended ways -- and there are lots of those, both in the JDK and
> in general.
>
> >          Modules on the other hand cannot provide additional security as
> > there is always an opt-out for non-modularized code to avoid such
> barriers.
>
> Non-modularized code can only break through module barriers if explicit
> permission is given for that, on the command line.  So yes, there's an
> opt-out mechanism, but it forces the end user to be aware that the
> system as a whole will run with compromised integrity.
>
> > I am sure this possibility was considered and I wonder why it was not
> > implemented. Looking at the very little practical relevance of heap
> > pollution caused by type-erasure, I believe that compile-time
> > modularization would work well in this case, too. The Java compiler could
> > enforce module boundaries while users of reflection would not be bothered
> > with the boundaries they intended to cross in the first place. As the
> most
> > important benefit, migration would be much easier. Libraries could add
> > module descriptors to pre-Java-9 bundles without needing to alter code
> > since runtime behavior does not longer depend on the bundling format.
>
> If strong encapsulation is to mean anything then it must be enforced at
> run time, so a purely compile-time approach is just not viable.  (One
> could argue that we already have compile-time modularization today, with
> build tools such as Maven, in which case if compile-time modularization
> were sufficient then we never would've started this project.)
>
> It's become clear from all the recent feedback that the current design
> might be making life a bit too hard for framework libraries that make
> heavy use of reflection, so as I indicated in a nearby thread we're
> definitely going to look into ways to address that.
>
> - Mark
>