Disallowing the dynamic loading of agents by default (revised)

[mark.reinhold at oracle.com]

Thanks to everyone for the quick feedback on this topic, and especially
to Andrew for the constructive dialogue.

Here's a revised proposal:

  - Define a new VM option, `-XX:+EnableDynamicAgentLoading`, that's
    on by default in JDK 9 but off by default in JDK 10.

    This will allow launch scripts that use this option on JDK 10 to
    work on JDK 9 without change, and will allow early testing of the
    JDK 10 behavior on JDK 9.

  - Revise the `com.sun.tools.attach` API to forbid attachment to the
    current process or to an ancestor of the current process, and
    define a read-only system property that allows such attachment to
    be enabled via the command line.

    This will discourage the inadvertent use of libraries that, for
    better or for worse, intentionally violate strong encapsulation.

  - Enhance the `-jar` launcher option so that if the JAR file being
    launched contains a `Premain-Class` attribute then it's launched
    as both an application and as an agent for that application.

    This will allow `java -jar foo.jar` to be used in place of the
    more verbose `java -javaagent:foo.jar -jar foo.jar` [1].

Taken together, these changes are intended to enable the continued use
of legitimate dynamically-loaded agents without change on JDK 9 and with
a small change on JDK 10.  That later change will align the treatment of
such agents with the other means of breaking encapsulation (`--add-opens`,
etc.) in order to ensure integrity by default for all code.

This proposal does not attempt to lock down platform classes as distinct
from user classes.  Many agents have legitimate reasons to transform
platform classes, so an additional mechanism to protect those classes
does not appear to be worthwhile.

Comments?

- Mark


[1] http://mail.openjdk.java.net/pipermail/jigsaw-dev/2017-April/012000.html