The critical missing pieces and a path forward

[David M. Lloyd]

On 05/08/2017 09:57 AM, mark.reinhold at oracle.com wrote:
> 2017/5/5 11:15:47 -0700, david.lloyd at redhat.com:
>> Fellow experts,
>>
>> We and other EC members and community members have, on several
>> occasions, communicated the various deficiencies we see in the
>> specification.  This is an update to make sure that it is very clear
>> which few technical, objective criteria are being missed by the current
>> specification that actually pushes it over the line to unacceptability
>> to Red Hat in its current form.  I can't speak for other EG or EC
>> members, but I believe this accurately summarizes our position.
> 
> Thank you for the summary.  We've discussed all of these issues before.
> Since then I've seen no new information that persuades me to change my
> views on them, but I'll summarize those views here for the sake of the
> public record.
> 
> First, however, a broader point.
> 
> Part of our collective role here is to act as responsible stewards of
> the Java Platform.  If we are to take that at all seriously then we must
> think as carefully as we can about every change or addition that we make.
> Every change to existing functionality carries compatibility risk.  Every
> addition places hard constraints upon how the platform can evolve in the
> future.  That a change or addition appears useful to some is just the
> start of what must be a thorough analysis of its wider and long-term
> consequences.  That a change or addition is "quite easy" to implement
> is completely irrelevant.
> 
> History bears witness to this.  In the early years of Java we sometimes
> chose to expose low-level functionality, or violate fundamental language
> invariants, or leave APIs open for extension without having designed them
> for that purpose.  In each case it was convenient and expedient and, yes,
> "quite easy", but twenty years later we're still paying for many of those
> decisions.
> 
> To cite just one example, consider the fact that `final` does not really
> mean "final", which I've mentioned before [1].  The core reflection API
> introduced in Java 1.1 brought the ability to change the value of `final`
> fields.  This seemed useful, and implementing it was "quite easy" at the
> time, and it did not have any obvious impact on the rest of the system.
> In the long run, however, it has greatly complicated if not prevented JIT
> compiler optimizations based upon or enabled by constant folding -- which
> is an awful lot of them.
> 
> In the context of a library or an application the consequences of any
> particular design choice are often apparent, and it's often possible to
> correct a bad choice over time.  We're working here, however, on the Java
> Platform itself.  Millions of developers and users and customers depend
> upon it, every day.  In that context the consequences of any particular
> design choice are often difficult to predict -- and bad choices are
> sometimes impossible to correct.
> 
> If responsible stewardship means anything then it means that we must
> reject any proposal that merely appears useful to some and is "quite
> easy" to implement.
> 
> To act otherwise would be the height of irresponsibility.
> 
> Now, on to your criteria.
> 
>> The first criterion is to allow cycles to exist among modules at run
>> time.  Our experience with deploying graphs which include third-party
>> modules leads me to believe that this is going to be real problem that
>> people encounter, which will be surprising and (in some cases) very
>> difficult to fix or work around.
> 
>  From previous conversations my understanding is that your experience is
> based upon deploying graphs of existing unchanged non-modular JAR files,
> with all of their messy flaws, in a Java EE application server.  From a
> purely technical perspective, it's impressive that you can do that.  If
> we were aiming in this JSR only to describe and standardize that practice
> then I would agree that cycles should be allowed.
> 
> A primary goal of this JSR is, however, to create a module system that's
> approachable by all developers -- not just developers of application
> servers -- for use in the construction, maintenance, and deployment of
> libraries and large applications.  This demands that we think more
> broadly about how to help all developers write programs "in the large".
> One result of that thinking is that this module system imposes some
> constraints that are intended to make working systems easier to reason
> about, and to guide developers toward better practices.  Adapting to
> these constraints will sometimes require changes to existing code, but
> we've assumed from the start that modularization is an active process
> undertaken by the maintainer of a component rather than something done
> after-the-fact by someone else who cannot modify the component.
> 
> Forbidding cyclic module relationships is one of these new constraints.
> That such relationships are generally a bad idea is not just my personal
> view; see, e.g., Kirk Knoernschild's compelling arguments against them
> in his fine book, _Java Application Architecture_ [2], specifically in
> §4.4, "Cyclic Dependencies -- The Death Knell".

I think this argument is deliberately missing the point.  Today, it is 
relatively difficult to successfully build a piece of Java code which 
has cyclic build dependencies.  But when you're talking about a 
potentially large graph, a cyclic dependency is not always a design 
choice: sometimes, it's an accident of assembly.  A harmless accident, 
which should not be punished.

> As I wrote previously [3], if real experience with this module system as
> used directly by developers reveals that cycles should be allowed then in
> a future release we can revise the specification to allow them.  If we do
> so now, however, then we can never go back on that choice.

Conceptually speaking, why would we ever need back out on the choice? 
The only possible presentation of this problem is that all module builds 
had succeeded, but a run-time cycle is unexpectedly introduced when the 
final application is assembled.  There is no design principle being 
violated in this case; it's a far more obscure happening.  To enforce at 
this level doesn't really help users in any way.  If they hit this error 
at run time - maybe even during final or near-final testing - now our 
users must potentially go back to the design stage to completely 
refactor one or more modules.

Think of, say, an enterprise environment where different components are 
developed by different teams, and they later decide over time that each 
must use APIs from one another, for various reasons.  They should not 
have to fight over which group must refactor their module to split API 
and implementation.  Sometimes, particularly in an enterprise 
environment, such a split just doesn't make any sense.

If a module has no cycles at build, the run time should always accept it 
without error.  I think this makes far more sense as an ideology than 
the idea that modules must have no cycles at build time (where design 
has an impact) and also at run time (where design takes a back seat to 
assembly and deployment).

I think we have concrete reasons to allow cycles, and abstract reasons 
to disallow them.  And maybe I'm wrong and this kind of thing just won't 
happen with Java 9 modules; but if it does, it could potentially cause 
an unreasonably disproportionate amount of pain in comparison to the 
(AFAICT zero) benefit of enforcing what can only be described as an 
arbitrary restriction.

-- 
- DML