Sharing experiences with the latest EA build // #ReflectiveAccessToNonExportedTypes #ResourceEncapsulation

[Alan Bateman]

Thanks for the write-up, a few questions/comments inlined.


On 30/09/2016 13:35, Rafael Winterhalter wrote:

> With the newest EA build, out of 7000 unit tests, only
> 42 tests still fail and those are all because of using non-exported API of
> the JCL.
Just curious, what is "JCL" here?


> This is of course not great as plain jars cannot be packaged
> into jdeps but I hope to include a generated module-info.class at some
> point even in a Java 6 jar in order to allow doing this.
I'm also curious what "plain jars cannot be packaged into jdeps" means. 
Do you mean the jdeps tool, maybe you mean jlink?


> Creating the
> first module already helped to uncover a minor breach of the intended
> module boundaries that was now discovered by the Java 9 compiler.
I assume these were warnings previously (as javac has emitted warnings 
since JDK 6 when it sees references to JDK-internal APIs).


> One
> minor problem was that many times, several Maven modules declared the same
> packages to communicate "cross-module shared classes" via package private
> types. This required some refactoring of classes and splitting up of some
> modules.
The split-package issue is going to challenge. In this case it sounds 
like it's all application and can be refactored without breaking others.

Also there is also no shame is leaving some JARs on the class path.


>
> After deploying the application with some first modules on the module path,
> I encountered similar problems. Adding correct exports to cover the use of
> reflection is unfortunately non-trivial.
Can you expand a bit on this? Is this modules in this initial set 
reflecting on each other? If they are reflecting on other parts of the 
system then there should be no issue when they are weak or automatic 
modules, or just left on the class path.


> Step-by-step migration is
> unfortunately impossible as I always need to consider all of our code and
> the library code.
In the previous paragraph then it sounds like you were doing it 
incrementally but here it sounds like you ended moving all components, 
is that right? Did you trying using automatic modules or did you migrate 
everything to explicit modules ("explicit module" meaning it has a 
compiled module declaration).


> There is (obviously) no way to check at compile time if a
> module is accessed refletively from the class path what only leaves
> runtime. This is also true for non-reflective access if I do not recompile
> all code on the class path when transforming a module. This turned out to
> be quite tedious, especially, since some problems only occur when
> triggering a particular subroutine of the application. Unit tests do not
> offer a good alternative either as I cannot really separate what tests are
> supposed to have access to the module internals and which should not. The
> application has thousands of tests, some of them not making much sense to
> me, going through them would be economically impractical. Integration tests
> sometimes uncover those access problems but unfortunately often disguise
> them in the error reports.
If I read this correctly then everything was migrated to explicit and 
strong modules. I assume it would have been a lot easier to get going if 
using starting out with automatic (or even weak) modules.


> :
>
> I currently experiment with a Java agent to force Java 9 to adopt this
> desired behavior by manipulating the boot layer's module graph or by
> stubbing AccessibleObject::checkCanSetAccessible.
That sounds like a gross hack, something that could break at any time.


> :
>
> Finally, I already identified some Maven modules which would be difficult
> to Java-9-modularize as they basically only work with reflection and are
> used throughout many parts of the application. It feels a bit strange to
> export packages from the users of this module as those "reflection modules"
> are only used via several indirections. I would prefer it if I could
> declare the module to be a form of "transitive module" which is simply
> allowed to reflect as it was allowed previously. I understand that security
> might be an issue here but why not simply ask a potentially present
> security manager if such a transitive module is allowed to be used?
With strong encapsulation then it's the module author that decides which 
parts of the module can be accessed by others. The weak modules proposal 
is a good starting point for application modules can take advantage of 
reliable configuration without opting in to strong encapsulation.


> :
>
> In my trial runs, I did not identify any problems with
> #ResourceEncapsulation as all of our custom "cross-module files" were put
> into the base folder, i.e. the unnamed module where they remain available.
> I do however not know how this would affect other applications that put
> resources into packages.
Thanks for confirming. It remains to be seen how much the ClassLoader 
APIs are used to get at non-class resources that are co-located with 
.class files.

> :
>
> PS: I noticed that when running java (without commands), it still lists the
> non-GNU-style options that are no longer supported.
Can you paste in `java -version` and give an example?

-Alan