Proposal (revised): Allow illegal access to internal APIs by default in JDK 9

Mon Jun 5 23:41:49 UTC 2017

I think this is the most pragmatic and reasoned middle ground one could possibly imagine.

I agree with the finely-tuned choices, specifically:

  - Not going completely silent.  Some asked for the ability to completely shut off the warnings.  This goes a little too far to one extreme.  There has to be some cost or discomfort or there won’t be the right movement.  IMHO, it would have been too far back the other way to go silent.  People will see these messages, they will post them to StackOverflow and get an education on how to go forward.  We need that.

  - Not choosing `--illegal-access=deny` by default for this version.  Some cited this as a setback.  There is an argument to be made that it would have been the bigger setback to go too strict too soon.  The danger would be when developers who don’t know this conversation and the details behind it learn “use this flag to make problems go away”.  At which point it becomes cargo cult and they’ll be afraid to stop using it because they never understood it completely in the first place.  Worse, we’re now buried under 3 years of lazy StackOverflow answers effectively telling people how to go backwards.

So although it feels less strict, one set of choices creates a conversation on how to move forward and one creates a conversation on how to go backward.

Of course time will tell, but I believe this proposal gets us what we want the soonest.

-David

> On Jun 5, 2017, at 11:45 AM, mark.reinhold at oracle.com wrote:
> 
> (Thanks for all the feedback on the initial proposal [1].  Here's a
> revised version, which incorporates some of the suggestions received and
> includes a bit more advice.  An implementation is already available for
> testing in the Jigsaw EA builds [2].  Further comments welcome!)
> 
> Over time, as we've gotten closer and closer to the JDK 9 GA date, more
> and more developers have begun paying attention to the actual changes in
> this release.  The strong encapsulation of JDK-internal APIs has, in
> particular, triggered many worried expressions of concern that code that
> works on JDK 8 today will not work on JDK 9 tomorrow, yet no advance
> warning of this change was given at run time in JDK 8.
> 
> To help the entire ecosystem migrate to the modular Java platform at a
> more relaxed pace I hereby propose to allow illegal-access operations to
> internal APIs from code on the class path by default in JDK 9, and to
> disallow them in a future release.  This will enable smoother application
> migration in the near term, yet still enable and motivate the maintainers
> of libraries and frameworks that use JDK-internal APIs to fix their code
> to use proper exported APIs.
> 
> New command-line option: `--illegal-access`
> -------------------------------------------
> 
> The recently-introduced `--permit-illegal-access` option [3] will be
> replaced by a more-general option, `--illegal-access`.  This option takes
> a single keyword parameter to specify a mode of operation, as follows:
> 
>  `--illegal-access=permit`
> 
>    This mode opens each package in each module in the run-time image to
>    code in all unnamed modules, i.e., code on the class path, if that
>    package existed in JDK 8.  This enables both static access, i.e., by
>    compiled bytecode, and deep reflective access, via the platform's
>    various reflection APIs.
> 
>    The first reflective-access operation to any such package causes a
>    warning to be issued, but no warnings are issued after that point.
>    This single warning describes how to enable further warnings.
> 
>    This mode will be the default for JDK 9.  It will be removed in a
>    future release.
> 
>  `--illegal-access=warn`
> 
>    This mode is identical to `permit` except that a warning message is
>    issued for each illegal reflective-access operation.  This is roughly
>    equivalent to the current `--permit-illegal-access` option.
> 
>  `--illegal-access=debug`
> 
>    This mode is identical to `warn` except both a warning message and a
>    stack trace are issued for each illegal reflective-access operation.
>    This is roughly equivalent to combining `--permit-illegal-access`
>    with `-Dsun.reflect.debugModuleAccessChecks`.
> 
>  `--illegal-access=deny`
> 
>    This mode disables all illegal-access operations except for those
>    enabled by other command-line options, e.g., `--add-opens`.
> 
>    This mode will become the default in a future release.
> 
> When `deny` becomes the default mode then `permit` will likely remain
> supported for at least one release, so that developers can continue to
> migrate their code.  The `permit`, `warn`, and `debug` modes will, over
> time, be removed, as will the `--illegal-access` option itself.  (For
> launch-script compatibility the unsupported modes will most likely just
> be ignored, after issuing a warning to that effect.)
> 
> How to prepare for the future
> -----------------------------
> 
> The default mode, `--illegal-access=permit`, is intended to make you
> aware when you have code on the class path that reflectively accesses
> some JDK-internal API at least once.  To learn about all such accesses
> you can use the `warn` or `debug` modes.  For each library or framework
> on the class path that requires illegal access you have two options:
> 
>  - If the component's maintainers have already released a new,
>    fixed version that no longer uses JDK-internal APIs then you
>    can consider upgrading to that version.
> 
>  - If the component still needs to be fixed then we encourage you
>    to contact its maintainers and ask them to replace their use
>    of JDK-internal APIs with proper exported APIs [4].
> 
> If you must continue to use a component that requires illegal access then
> you can eliminate the warning messages by using one or more `--add-opens`
> options to open just those internal packages to which access is required.
> 
> To verify that your application is ready for the future, run it with
> `--illegal-access=deny` along with any necessary `--add-opens` options.
> Any remaining illegal-access errors will most likely be due to static
> references from compiled code to JDK-internal APIs.  You can identify
> those by running the `jdeps` tool with the `--jdk-internals` option.
> (JDK 9 does not issue warnings for illegal static-access operations
> because that would require deep JVM changes and degrade performance.)
> 
> Warning messages
> ----------------
> 
> The warning message issued when an illegal reflective-access operation is
> detected has the following form:
> 
>    WARNING: Illegal reflective access by $PERPETRATOR to $VICTIM
> 
> where:
> 
>  - $PERPETRATOR is the fully-qualified name of the type containing
>    the code that invoked the reflective operation in question plus
>    the code source (i.e., JAR-file path), if available, and
> 
>  - $VICTIM is a string that describes the member being accessed,
>    including the fully-qualified name of the enclosing type
> 
> In JDK 9's default mode, `--illegal-access=permit`, at most one of these
> warning messages will be issued, accompanied by additional instructive
> text.  Here is an example, from running Jython on the current Jigsaw EA
> build [2]:
> 
>    $ java -jar jython-standalone-2.7.0.jar
>    WARNING: An illegal reflective access operation has occurred
>    WARNING: Illegal reflective access by jnr.posix.JavaLibCHelper (file:/tmp/jython-standalone-2.7.0.jar) to method sun.nio.ch.SelChImpl.getFD()
>    WARNING: Please consider reporting this to the maintainers of jnr.posix.JavaLibCHelper
>    WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
>    WARNING: All illegal access operations will be denied in a future release
>    Jython 2.7.0 (default:9987c746f838, Apr 29 2015, 02:25:11) 
>    [OpenJDK 64-Bit Server VM (Oracle Corporation)] on java9-internal
>    Type "help", "copyright", "credits" or "license" for more information.
>>>> ^D
>    $ 
> 
> If `--illegal-access=warn` is used then only warnings are displayed, with
> no instructive text.  The run-time system makes a best-effort attempt to
> suppress duplicate warnings for the same $PERPETRATOR and $VICTIM.  Here
> is an example, again running Jython:
> 
>    $ java --illegal-access=warn -jar jython-standalone-2.7.0.jar
>    WARNING: Illegal reflective access by jnr.posix.JavaLibCHelper (file:/tmp/jython-standalone-2.7.0.jar) to method sun.nio.ch.SelChImpl.getFD()
>    WARNING: Illegal reflective access by jnr.posix.JavaLibCHelper (file:/tmp/jython-standalone-2.7.0.jar) to field sun.nio.ch.FileChannelImpl.fd
>    WARNING: Illegal reflective access by jnr.posix.JavaLibCHelper (file:/tmp/jython-standalone-2.7.0.jar) to field java.io.FileDescriptor.fd
>    WARNING: Illegal reflective access by org.python.core.PySystemState (file:/tmp/jython-standalone-2.7.0.jar) to method java.io.Console.encoding()
>    Jython 2.7.0 (default:9987c746f838, Apr 29 2015, 02:25:11) 
>    [OpenJDK 64-Bit Server VM (Oracle Corporation)] on java9-internal
>    Type "help", "copyright", "credits" or "license" for more information.
>>>> ^D
>    $ 
> 
> Notes
> -----
> 
>  - There is no `--illegal-access` mode that suppresses all warnings.
>    This is intentional: It ensures that developers know that all
>    illegal-access operations will be denied by default in a future
>    release, at which time code that generates warnings today will fail.
>    Warnings can be suppressed completely via one or more `--add-opens`
>    options.
> 
>  - The first proposal [1] opened every package in every explicit module,
>    rather than just the packages in modules in the run-time image, to
>    every unnamed module.  Peter Levart pointed out [5] that this could
>    tempt developers to use internal APIs that are new in JDK 9 (e.g.,
>    `jdk.internal.misc.Unsafe`) and thus make the eventual transition
>    from JDK 9 no less painful than that from JDK 8.  This proposal thus
>    only opens internal packages that existed in JDK 8.
> 
>  - This proposal will require adjustments to JEP 260, "Encapsulate Most
>    Internal APIs" [6].  APIs that are internal to the JDK will still be
>    strongly encapsulated from the standpoint of code in modules, whether
>    those modules are automatic or explicit, but they will not appear to
>    be encapsulated at run time from the standpoint of code on the class
>    path.
> 
>  - This change will not magically solve every JDK 9 adoption problem.
>    The concrete types of the built-in class loaders are still different,
>    `rt.jar` is still gone, the layout of a system image is still not the
>    same, and the version string still has a new format.
> 
> 
> [1] http://mail.openjdk.java.net/pipermail/jigsaw-dev/2017-May/012673.html
> [2] http://jdk.java.net/jigsaw/
> [3] http://mail.openjdk.java.net/pipermail/jigsaw-dev/2017-March/011763.html
> [4] This will usually but not always be possible, since there are still a
>    few critical internal APIs without exported replacements [6].
> [5] http://mail.openjdk.java.net/pipermail/jigsaw-dev/2017-May/012708.html
> [6] http://openjdk.java.net/jeps/260