<html><body><div style="font-family: arial, helvetica, sans-serif; font-size: 12pt; color: #000000"><div><br></div><div><br></div><hr id="zwchr" data-marker="__DIVIDER__"><div data-marker="__HEADERS__"><blockquote style="border-left:2px solid #1010FF;margin-left:5px;padding-left:5px;color:#000;font-weight:normal;font-style:normal;text-decoration:none;font-family:Helvetica,Arial,sans-serif;font-size:12pt;"><b>From: </b>"Brian Goetz" <brian.goetz@oracle.com><br><b>To: </b>"Remi Forax" <forax@univ-mlv.fr>, "Dan Heidinga" <heidinga@redhat.com><br><b>Cc: </b>"leyden-dev" <leyden-dev@openjdk.java.net><br><b>Sent: </b>Friday, August 5, 2022 10:39:40 PM<br><b>Subject: </b>Re: Bytecode transformation investigation<br></blockquote></div><div data-marker="__QUOTED_TEXT__"><blockquote style="border-left:2px solid #1010FF;margin-left:5px;padding-left:5px;color:#000;font-weight:normal;font-style:normal;text-decoration:none;font-family:Helvetica,Arial,sans-serif;font-size:12pt;"><font size="4"><font face="monospace">Remi;<br><br>
I think this misses a bigger picture here. A key goal of Leyden
is that we be able to _selectively and flexibly constrain and
shift dynamism_. We don't want to force users to decide at
compile time whether they want to AOT it, partially evaluate the
program, gather profiling data, constrain away indy and other
classloading, etc; we want them to be able to write their
program and run it on a dynamic VM, as well as choosing to
condense it (perhaps in a series of phases) to shift some
behavior from runtime to an earlier phase, all completely
optionally. <br><br>
This is why Dan has focused on jlink; because jlink is
positioned as the thing you run when you're ready to accept some
tighter coupling in exchange for a smaller or faster deployment
unit. So the choice of `jlink` here is entirely appropriate,
and has the advantage that developers can do their
develop-test-run cycle with the lightest possible build chain,
and spend more cycles to condense the program to a
smaller/faster one only when spending those cycles has a
positive return. </font></font></blockquote><div><br></div><div>Thanks for the re-explaining the problem to me.<br data-mce-bogus="1"></div><div><br data-mce-bogus="1"></div><div>The downside I see is that most of the tooling we have is not ready for that, nobody test after jlink, that's why people expect 100% compatibility with the bytecode generated by javac.</div><div><br data-mce-bogus="1"></div><div>I don't have the magic solution, i just know that doing the transformation at the same time as the generation of the classfiles is easier.<br data-mce-bogus="1"></div><div><br data-mce-bogus="1"></div><div>Rémi<br data-mce-bogus="1"></div><div><br data-mce-bogus="1"></div><blockquote style="border-left:2px solid #1010FF;margin-left:5px;padding-left:5px;color:#000;font-weight:normal;font-style:normal;text-decoration:none;font-family:Helvetica,Arial,sans-serif;font-size:12pt;"><font size="4"><font face="monospace"><br><br></font></font><br>
<div class="moz-cite-prefix">On 8/5/2022 1:49 PM, Remi Forax wrote:<br>
</div>
<blockquote cite="mid:1395024911.19046050.1659721771245.JavaMail.zimbra@u-pem.fr">
<pre class="moz-quote-pre">----- Original Message -----
</pre>
<blockquote>
<pre class="moz-quote-pre">From: "Dan Heidinga" <a class="moz-txt-link-rfc2396E" href="mailto:heidinga@redhat.com" target="_blank"><heidinga@redhat.com></a>
To: "Brian Goetz" <a class="moz-txt-link-rfc2396E" href="mailto:brian.goetz@oracle.com" target="_blank"><brian.goetz@oracle.com></a>
Cc: "leyden-dev" <a class="moz-txt-link-rfc2396E" href="mailto:leyden-dev@openjdk.java.net" target="_blank"><leyden-dev@openjdk.java.net></a>
Sent: Friday, August 5, 2022 4:49:31 PM
Subject: Re: Bytecode transformation investigation
</pre>
</blockquote>
<blockquote>
<pre class="moz-quote-pre">Responding to one piece of this now as it's important to get everyone
on the same page with the requirements. And I know I've tripped over
the "move fast, break things" philosophy multiple times in this space
before coming to this conclusion.
</pre>
<blockquote>
<pre class="moz-quote-pre">From a specification perspective, there are multiple separate specifications
viewpoints to consider: JLS, JDK and JVMS. From a JLS perspective, I would say
that if the Java *compiler* were to do what your jlink plugin does, this would
be a reasonable way to implement a compiler for the Java language -- the
classfiles emitted would respect the semantics of the language. There's
nothing that says a Java compiler has to translate lambdas with indy, or with
hidden classes, so if the indy never got generated, that's not a problem.
>From the JDK+JVMS perspective, it starts to get a little murky, and one of the
goals of Leyden is to bring more clarity to this area. The compiler emits
certain classfiles with `invokedynamic`, and then some build-time tool rewrites
these classes to be different. Is this OK? If the build-time tool is just
"Dan's Magic Unofficial (Not) Java Bytecode Mangler", then this is the sort of
build time mangling people do every day. But we want this to be an official
part of the platform, so I think there's a little more specification work to be
done to allow (and specify) such transformations. This is not a deal breaker,
but we need to apply more thought here. I think there are two categories of
new work here: some specification work to characterize what build-time
transformations like this are allowed to do or not do, and your transformer
will likely want a specification for what it does as well.
</pre>
</blockquote>
<pre class="moz-quote-pre">What if we doubled down on treating all pre-runtime bytecode
transformations as optional behaviours akin to "Dan's Magic Unofficial
(Not) Java Bytecode Mangler" despite shiping with the platform? Each
transformation - jlink plugin? - could be self describing so users
know what they are opting (the key point!) into when they enable the
transformation. This allows treating these transformations as a
pre-step that has significant leeway on what it does provided the
modified classfiles run correctly.
</pre>
</blockquote>
<pre class="moz-quote-pre">I don't think it should be run by jlink but more as a post process step of javac, more like annotation processors.
It will work with anything that using invokedynamic.
If the transformation are done by jlink, you can do more transformation, resolve Class.forName() / ServiceLoader by example, but you are in closed world assumption.
</pre>
<blockquote>
<pre class="moz-quote-pre">The JVM's role is then to load / verify / execute the classes as
required by the application and defined by the JVM specification.
Anything done to the classfiles prior to that is outside the JVM
spec's remit.
This "user opt-in to transformations" model shrinks the two categories
to one: specifying what a transformer does. As the first
"specification work to characterize what build-time transformations
like this are allowed to do or not do" category is answered with
"whatever they want, provided they generate valid classfiles". And if
the user is opting-in for an application-specific runtime (jlinked),
then why not?
Although it's kind of satisfying to say we can do what we want here,
it doesn't actually work. Why? Because this model destroys any
invariants built into the JDK platform.
Don't like how a method operates? Transform it to do something else!
Introduce bugs! Open security holes! It's trivially easy to break
the platform invariants, get surprising results, or open subtle
security holes here. Basically, all the concerns raised with Native
Image's Substitution mechanism come into play here. Though it's
possible to do many of these things today with JVMTI agents or even
user written jlink plugins (or historically by hand hacking rt.jar),
it's less common because it's hard! and because users have been
rightfully wary of what this can do to their applications.
</pre>
</blockquote>
<pre class="moz-quote-pre">Why do you want the user to be able to opt-in to an unbounded set of transformation ?
You can be far more restrictive by saying that you only have one javac flag to opt-in to a more "static" view of the world, using a bytecode transformer or not becomes an implementation details in that case.
</pre>
<blockquote>
<pre class="moz-quote-pre">Not to mention that Support Engineers will hate us if we take this
approach as it's hard to argue something isn't a supported config if
jdk ships the transformation that breaks the invariant.
All that to say, I think the "specification work to characterize what
build-time transformations like this are allowed to do or not do" is
important to this work actually being successful.
</pre>
</blockquote>
<pre class="moz-quote-pre">yes
</pre>
<blockquote>
<pre class="moz-quote-pre">--Dan
</pre>
</blockquote>
<pre class="moz-quote-pre">Rémi
</pre>
</blockquote>
<br><br></blockquote></div></div></body></html>