Refresh of module system coming to jdk9/dev soon

[Alan Bateman]

Just a heads-up that the code review and cleanup for an update of the 
module system is currently underway on jigsaw-dev [1] with a view to 
pushing the changes to jdk9/dev soon, maybe for jdk-9+148 that will 
promote next week.

For those that have been trying out modules with regular JDK 9 builds 
then be aware that `requires public` changes to `requires transitive`. 
In addition, the binary representation of the module declaration 
(module-info.class) has changed so that you need to recompile any 
modules that were compiled with previous JDK 9 builds.

This refresh includes a disruptive change that is important to 
understand. As things stand today in JDK 9 then you use setAccessible to 
break into non-public elements of any type in exported packages (you can 
hack into private fields of any type in java.util example). However, it 
cannot be used to break into any type in non-exported package (you can't 
break into types in jdk.internal.misc or sun.security.x509 for example). 
The current specified behavior was a compromise for the initial 
integration of the module system. It is of course not very satisfactory, 
hence the #AwkwardStrongEncapsulation issue [2] on the JSR 376 issues 
list. With the updated proposal in the JSR, this refresh changes 
setAccessible further so that it cannot be used to break into non-public 
types, or non-public elements of public types, in exported packages. 
Code that uses setAccessible to hack into the private constructor of 
java.lang.invoke.MethodHandles.Lookup will be disappointed for example.

This change will expose hacks in many existing libraries and tools (as 
happened when setAccessible was changed to prevent it being used to 
break into sun.* packages). It will not be popular. As a workaround then 
a new command line option `--add-opens` can be used to open specific 
packages for "deep reflection". For example, a really popular build tool 
fails with this refresh because it uses setAccessible + core reflection 
to hack into a private field of an unmodifiable collection so that it 
can mutate it, facepalm! This code will continue to work as before when 
run with `--add-opens java.base/java.util=ALL-UNNAMED` to open the 
package java.util in module java.base to "all unnamed modules" (think 
class path).

As I said, this change will not be popular but please bear with it until 
the extent of the issues uncovered is more widely understood. We need 
all the help we can get to identify issues and get them reported (and 
hopefully fixed) by the libraries and tools with the technical debt (and 
we expect a lot of it will be technical debt). For those working on 
OpenJDK and the JDK 9 project then it's very possible that some of these 
issues will redirect to the JDK as requests for new APIs or hooks (and I 
assume will need to be looked at on a case-by-case basis).

Any help reporting issues to popular tools and libraries would be 
appreciated. A debugging aid that is useful to identify issues is to run 
with -Dsun.reflect.debugModuleAccessChecks=true to get a stack trace 
when setAccessible fails, this is particularly useful when code swallows 
exceptions without any logging.

One final point, and only interesting to those working in OpenJDK, is 
that this refresh will require upgrading your build of jtreg. The 
changes to work with this refresh have been in the code-tools/jtreg 
repository for some time. The TEST.ROOT in each repository has been 
updated to require the new version.

-Alan

[1] 
http://mail.openjdk.java.net/pipermail/jigsaw-dev/2016-November/010249.html
[2] 
http://openjdk.java.net/projects/jigsaw/spec/issues/#AwkwardStrongEncapsulation