Refresh of module system coming to jdk9/dev soon

[Martin Buchholz]

Whitebox testing is the most obvious example of code that has a good excuse
for poking inside implementation details.
For running jsr166 tests, I'm happy to report that the following seems to
work:

  <jvmarg value="-Xbootclasspath/p:@{classes}" unless:set="modules"/>
  <jvmarg line="--patch-module java.base=@{classes}" if:set="modules"/>
+ <jvmarg line="--add-opens java.base/java.util=ALL-UNNAMED"
if:set="modules"/>
+ <jvmarg line="--add-opens java.base/java.util.concurrent=ALL-UNNAMED"
if:set="modules"/>
+ <jvmarg line="--add-opens
java.base/java.util.concurrent.atomic=ALL-UNNAMED" if:set="modules"/>
+ <jvmarg line="--add-opens
java.base/java.util.concurrent.locks=ALL-UNNAMED" if:set="modules"/>

It would be nice if there was explicit guidance for authors of tooling
software like test harnesses.  jtreg has some magic in JTRegModuleHelper
that calls Module.implAddOpens reflectively, but that's not going to work
for ordinary software, and there's no cookbook recipe for how to do that.
 jtreg has the advantage that it is already spawning other java processes,
and so can inject arbitrary command line flags.

It would be nice if the production software could define test-only methods
(think "checkInvariants()") that could be invoked without having to resort
to reflection.  Google has some use-this-method-in-tests-only tooling that
uses annotations, but the enforcement is external to the jdk.  I keep
hoping that module systems would somehow work at the method level instead
of the package level, because we already have visibility keywords like
private at the method level.

Giving access to internals to legitimate tools while preventing other uses
seems like a Hard Problem.  For tools like debuggers or heap analyzers that
really want to see everything, there will be a strong desire to "disable
all module checks".  Or maybe the right way is for users to patch them into
java.base?  Guidance needed.


On Wed, Nov 30, 2016 at 4:39 AM, Alan Bateman <Alan.Bateman at oracle.com>
wrote:

> Just a heads-up that the code review and cleanup for an update of the
> module system is currently underway on jigsaw-dev [1] with a view to
> pushing the changes to jdk9/dev soon, maybe for jdk-9+148 that will promote
> next week.
>
> For those that have been trying out modules with regular JDK 9 builds then
> be aware that `requires public` changes to `requires transitive`. In
> addition, the binary representation of the module declaration
> (module-info.class) has changed so that you need to recompile any modules
> that were compiled with previous JDK 9 builds.
>
> This refresh includes a disruptive change that is important to understand.
> As things stand today in JDK 9 then you use setAccessible to break into
> non-public elements of any type in exported packages (you can hack into
> private fields of any type in java.util example). However, it cannot be
> used to break into any type in non-exported package (you can't break into
> types in jdk.internal.misc or sun.security.x509 for example). The current
> specified behavior was a compromise for the initial integration of the
> module system. It is of course not very satisfactory, hence the
> #AwkwardStrongEncapsulation issue [2] on the JSR 376 issues list. With the
> updated proposal in the JSR, this refresh changes setAccessible further so
> that it cannot be used to break into non-public types, or non-public
> elements of public types, in exported packages. Code that uses
> setAccessible to hack into the private constructor of
> java.lang.invoke.MethodHandles.Lookup will be disappointed for example.
>
> This change will expose hacks in many existing libraries and tools (as
> happened when setAccessible was changed to prevent it being used to break
> into sun.* packages). It will not be popular. As a workaround then a new
> command line option `--add-opens` can be used to open specific packages for
> "deep reflection". For example, a really popular build tool fails with this
> refresh because it uses setAccessible + core reflection to hack into a
> private field of an unmodifiable collection so that it can mutate it,
> facepalm! This code will continue to work as before when run with
> `--add-opens java.base/java.util=ALL-UNNAMED` to open the package
> java.util in module java.base to "all unnamed modules" (think class path).
>
> As I said, this change will not be popular but please bear with it until
> the extent of the issues uncovered is more widely understood. We need all
> the help we can get to identify issues and get them reported (and hopefully
> fixed) by the libraries and tools with the technical debt (and we expect a
> lot of it will be technical debt). For those working on OpenJDK and the JDK
> 9 project then it's very possible that some of these issues will redirect
> to the JDK as requests for new APIs or hooks (and I assume will need to be
> looked at on a case-by-case basis).
>
> Any help reporting issues to popular tools and libraries would be
> appreciated. A debugging aid that is useful to identify issues is to run
> with -Dsun.reflect.debugModuleAccessChecks=true to get a stack trace when
> setAccessible fails, this is particularly useful when code swallows
> exceptions without any logging.
>
> One final point, and only interesting to those working in OpenJDK, is that
> this refresh will require upgrading your build of jtreg. The changes to
> work with this refresh have been in the code-tools/jtreg repository for
> some time. The TEST.ROOT in each repository has been updated to require the
> new version.
>
> -Alan
>
> [1] http://mail.openjdk.java.net/pipermail/jigsaw-dev/2016-Novem
> ber/010249.html
> [2] http://openjdk.java.net/projects/jigsaw/spec/issues/#Awkward
> StrongEncapsulation
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.openjdk.java.net/pipermail/jtreg-use/attachments/20161205/8755f8ef/attachment.html>