#LayerPrimitives aka allowing to add private package at runtime to a module ?

[David M. Lloyd]

On 03/16/2017 06:33 AM, Remi Forax wrote:
> This is currently a point where several of us disagree, here is my opinion on it.

Thanks for the reply.

> First, we have to all recognize that making something mutable, the content of a module or anything else, makes maintenance, debugging, etc harder than having the same thing non mutable.
> So the default choice should to choosing the immutable solution apart if there are good argument.

This is a good argument.  In my initial JBoss Modules implementation, 
modules were immutable.  Our first service container was also immutable. 
  But these models provided too many restrictions to us and to end users 
and in the end we found these ideals to be unrealistic, even misguided, 
when applied to this kind of thing.

Yes maintenance and debug cost for us is important - more important than 
you know, even.  But, a framework has one (or a constant number of) 
developer(s), whereas it has many, many users - a number which grows 
without bound, if a framework is successful.  In this light, saving cost 
to the developer(s) of the framework (even in this hypothetical 
situation) is meaningless if that cost savings means your users have to 
make up the difference.

In any event though, it's just an academic debate because modules in the 
Jigsaw implementation are not, and never have been, immutable.  The 
methods on Module and Layer.Controller that do exist could never 
function otherwise, and there's the matter of dynamic proxies of course, 
which also generate packages at run time.  If the JDK must do it for its 
framework support classes, it's a safe bet that others need to as well. 
To produce an obvious and trivial example, any EJB-compliant 
implementation must also have the ability to generate dynamic proxy 
classes in the same way that the JDK does (but with a different base 
class).  JPA uses all kinds of proxies.  Many frameworks and specs 
require generation of classes at run time.  And so on.  Every one of 
these things have the same security requirements as JDK proxies.

So in order to support the immutability argument, now containers have to 
guess every package they might need to generate or load into up front, 
and frankly most containers don't work that way.  Why?  Because until 
now, class loading was 100% lazy, and this is has proven to be a *good 
model*.

> Also, in term of implementation, making the content of a module mutable can be seen as a cache invalidation issue, and we all know that cache invalidation is a hard problem especially when it's on global states.

This I find to be a weaker argument.  What cache are we talking about, 
and what are the practical problems that make this intractable?

> With JPMS, a module is now part of the Java language, so it seems weird to make it mutable the same way, a java.lang.Class is not mutable in Java.
> But that's not true, in fact a java.lang.Class is mutable in Java, but you can only mutate it using an agent.

It's clear that modules are not to be treated like classes, because (for 
example) circularity among modules is considered "bad" whereas 
circularity among classes has been shown to be indispensable, and 
classes within a class loader or module are loaded and resolved lazily 
whereas modules are loaded and resolved aggressively, etc.  But I see 
your point that only "special" citizens ought to mutate a module.  I 
agree, but I do not think this should be limited to agents: containers 
also have a similar need for similar reasons.

> I'm pretty sure that if we do not make the content of a module mutable, people will hack the JPMS implementation to make it mutable, because during the development you want to be able to add private package like you want to add new method in an existing class without having to restart the program (think things like JRebel or Vert.x here).
>
> So in my opinion, we should have a mechanism to add private package to a module, there real question is should this mechanism only visible from an agent or should it be also available from the LayerController ?

Even considering Java EE alone requires the framework to have the 
ability to do these things.  It's not so hard to imagine that there are 
many related use cases.

> Aside, there is something that make me uncomfortable in the mails from David or Thomas, I'm maybe wrong, but it seems to me that you are thinking that you will be able to propose to your users to use the JPMS runtime with the current existing modules. I think your are chasing an unicorn/trying to solve a problem nobody care, i maintain several OSGI bundles and i do not want an implementation of OSGI to see them as JPMS modules, not by default because they will not work, the encapsulation model of JPMS is far stricter than the current model (i think it's the same issue with JBoss Module but i've no experience with them). As a user, if i create an OSGI module with a module-info or a bundle manifest that let me specified that this is a JPMS module, then i want it to be seen as a JPMS module but not by default. We have introduced a modulepath which is not a replacement of the classpath exactly for the same reason. So the interropt between OSGI and JPMS or JBoss Module and JPMS should be seen in my opinion with that idea in mind. In that case, supporting fragment that have also a JPMS module can be seen as a corner case, having to iterate over private packages is also not an issue because the existing manifest and the manifest of a JPMS module can be different, having more info, so at runtime the private packages will be available.

There are multiple answers.  First, JPMS justified the new reflection 
capabilities on a security basis.  And believe me, I argued hard against 
that.  :-)  If this basis is to hold up, then containers need to be able 
to extend these guarantees and options to their deployments.  To quote 
Mark, "The lead of Oracle's Java Vulnerability Team estimates that at 
least a third of all the vulnerabilities reported since JDK 7 GA would 
have been prevented if we'd had the ability to strongly encapsulate 
JDK-internal packages."  We (JBoss) have had our own share of CVEs as 
well, and the same arguments could be applied here just as easily.  Why 
should containers be second-class when it comes to security?  How does 
it impact users to say "we've solved a major problem with security but 
the solution is only available to code and containers that have not been 
written yet, sorry"?

Second, JPMS introduces a new deployment and packaging model.  If users 
use them, containers will need to support them.  And AFAICT Java EE 9 is 
still operating under the assumption that this packaging model will be 
used by deployments.  So in order to interoperate with them, existing 
systems (the same existing systems that have the dynamicity requirements 
above) have to be able to synthesize a way for such modules to be 
deployed and interoperate with the existing systems.  The best way is to 
have the actual JPMS implementation do this in the normal way.

These problems (and most others that I've raised) are derived from the 
idea we have chosen the implementation, and then are deciding what use 
cases are valid based on the design parameters of the chosen 
implementation.  Instead we *must* have a design and implementation 
which meets the use cases.

Any pretense of being guided by the requirements has long since been 
exposed as being applied inconsistently (at best) as it suites the 
implementation.  But we can't decide that the use case of 
interoperability is invalid just because the implementation does not 
presently make it convenient.

-- 
- DML