JEP draft: Disallow the Dynamic Loading of Agents by Default

[Dan Heidinga]

Hi Ron,

Thanks for writing up the JEP draft outlining the proposal to disallow
dynamic loading of agents by default.  The Red Hat Java team has continued
to discuss this proposal internally and with our stakeholders.

While there is a general agreement (or at least acceptance) with the
overall direction of this proposal, we still see the concerns raised by
Andrew [0] as needing to be addressed.

So let’s start with the high-order bit: timing.

The JEP draft states it “intends to adopt the proposal [to disable agents
by default] six years later, in 2023, for JDK 21.”  We would like to see
this targeted to JDK 22 instead as the change has not been widely
evangelized yet and comes as a surprise to many, both to those actively
developing OpenJDK and to those monitoring its development.

We owe it to the community to give this proposal enough bake time,
especially given that JDK 21 is an LTS release.  Though the official
position is that LTS releases are no different than any other release, the
actions of JDK consumers make them special.  Users will be surprised by
this change when they migrate from JDK 17 to 21.  If we delay till JDK 22,
we give the ecosystem time to adapt to the change before the next LTS.

Additionally, users who have tested with the -XX:-EnableDynamicAgentLoading
option will have false expectations if they validated their use of jcmd to
load the agent as the behaviour was not correct prior to JDK 21 [1].

The next concern is one you call out in the draft that “Java's excellent
serviceability has long been a source of pride for the platform.”  We
agree!

Java currently has an excellent, prime position in Observability
capabilities. For better or for worse, there are many Observability tools
that have relied on dynamic attach to provide the necessary monitoring for
workloads

It’s important we give Java’s monitoring tools sufficient time to migrate
comfortably without shocking the ecosystem by changing the default in an
LTS.  By delaying the change till JDK 22, we give the ecosystem 2 years to
migrate and to prepare users for the change.

Additionally, this provides the time for Java’s profiling tools to adapt as
well.  And for the “ad-hoc troubleshooting” tools - like Byteman - to
retrain their users.

Finally, while it’s easy to agree with the principle that “the application
owner is given the final say on whether to allow dynamic loading of
agents”, the argument can (and should!) be made that those application
owners have made that final decision by deploying the libraries and tools
that use dynamic attach.  A JVM command line argument is not the only way
to express their consent for code to run on their system.

For many production uses, the reality is more complicated than a single
“application owner”. Take a Kubernetes cluster for example.

Who is the application owner when deploying to a Kubernetes cluster?  The
dev team that develops the application?  The operations team that manages
the cluster?  The monitoring team that monitors the applications? The
Support team that troubleshoots issues with the deployment?

We should be careful not to understate or misrepresent the complex web of
“owners” that are currently able (and, for business reasons, need) to apply
agents dynamically.  Downplaying the complexity many organizations
experience when dealing with changes to command line options (as an
example) weakens the argument for changing today’s status quo.

We also know that in many cases customers and users may not be in a
position to modify startup scripts (e.g. even to add in an extra parameter)
as to do so may invalidate support contracts, etc.

Dynamically attached agents have been a “superpower” of Java and their use
has greatly benefited the ecosystem as they’ve helped service and support
use cases that otherwise aren’t possible, as they helped propel Java to the
forefront of the Observability tooling, and allowed many other useful
libraries to be developed.

Let’s delay flipping the default until JDK 22 to give the breadth of the
ecosystem time to absorb this change.

–Dan


[0]
https://mail.openjdk.org/pipermail/serviceability-dev/2023-March/047084.html
[1] https://bugs.openjdk.org/browse/JDK-8304438
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/serviceability-dev/attachments/20230428/8ef46b5f/attachment.htm>