[11u] RFC: Update Graal in OpenJDK 11 Updates?

[Andrew Dinn]

Hi Christoph,

On 24/03/2020 12:14, Langer, Christoph wrote:
> thanks for bringing this discussion to the current point.

> Unfortunately, I believe the latest replies from Doug didn't go to
> the list - but Andrew has cited them ��

Doug did copy his reply to the list but, of course, he is not
subscribed. His replies came direct to me of course. They may well be
waiting for a list moderator to allow them to proceed to the rest of the
list.

> So, I understand that the changes comprised by the big bulk change of
> labs-openjdk-11 -
> https://github.com/graalvm/labs-openjdk-11/commit/fbbdf5f15300716110d2ac9b9bd7f70b03b35a7d
> are first and foremost necessary to enable the usage of libgraal from
> the oracle/graal repo in JDK11. If OpenJDK 11 Updates doesn't have
> the changes, libgraal can't be used with it. Correct?

That appears to be the sum of what has changed. Of course, there is
always the possibility that some details of how that is done turn out be
important.

Indeed, Red Hat has just discovered one interesting wrinkle which might
be important:

The upstream patch that implements the bulk of the support for libgraal
(JDK-8220623) included changes to JVMCI code that resolves queries about
the structure of the underlying Hotspot code base things like listing
methods or fields of a resolved java type etc. In the jdk11u code that
is done by accessing the Java Class<?> and using java.lang.reflect APIs.
In the patch this is modified to use native calls that access the C++
types Klass, Method etc. One of the consequences of that is a difference
in how class bytecode gets loaded and resolved. The use of
java.lang.reflect require resolution of classes and so may perform
bytecode load for dependent classes earlier than happens whne querying
the C++ Klass model.

As a consequence, when running on OpenJDK jdk11u (i.e. without the
libgraal patch) the pointsto analysis traversal of the class base may
try to load bytecode for edge classes that not actually needed in the
closed world model while on the labs jdk11u the classbase query may not
need to load that bytecode. In some cases this may lead to a
ClassNotFoundException on stock jdk11u which does not happen on labs
jdk11u. Of course, this can usually be resolved by adding more jars to
the classpath.

Now I don't think that the above problem necessarily automatically
justifies backporting the libgraal patch -- note that just as regards
this specific part of the change that involves exporting to the JVCMI
module JNI access to the underlying JVM Klass model. However, it does
mean that users will encounter different behaviour on stock jdk11u even
when working with the current GraalVM code base.

> I also see a second potential motivation why OpenJDK 11 updates could
> be interested to incorporate those kind of changes. This is to be on
> par with the main Oracle JDK 11 Updates release. I think I mentioned
> that in my initial mail but it hasn't been part of the recent
> discussion between the both of you, Andrew and Doug. So, while we
> don't exactly know what Oracle did to their JDK11 Updates codebase,
> the backports of subsequent changes indicate that they must have
> incorporated something like the big bulk change of labs-openjdk-11.
> That would mean their in-tree Graal code of 11u is at the newer JDK13
> level and hence the AOT and C2 replacement scenarios would have
> received those substantial updates.

I'm not sure about this but I don't think the specific path I mention
above has been backported to Oracle's jdk11u.

> On the other hand, I think for 11u we should be cautious and err on
> the side of stability. That would mean we deliberately don't pull
> those JVMCI and Graal updates into 11u. I think that's a bit the
> direction this discussion was moving to.

Well, I think that's wise but I just wanted to note that it's about
balancing disparity in behaviors like the one described above against
the risk to non-Graal users.

> In any case, putting the general decision aside, I also believe, that
> the monolith commit is not the right way to get the changes into the
> OpenJDK 11 Updates project. When I was trying it on JDK11u, it was
> more of an exploration for me. But if we're going to do the exercise
> of updating Graal and JVMCI in OpenJDK11u to the JDK13 level, we
> should probably go change by change.

Oh we should definitely try to split out whatever we do backport from
that mega-patch so that it relates to the original upstream patches.
However, I'm not sure what it is we would actually need -- if any - from
that original set. I think we can only realistically answer that as we
progress in our tests of building Graal using the current jdk11u. If a
showstopper issue emerges we may want to consider backporting more
upstream jdk13u code but I think we need to judge every further patch on
its merits and the danger it presents to normal non-Graal uses.

> To enable this, I've created a JBS filter that contains all items Tom
> has mentioned in his list plus several Oracle 11.0.7 and 11.0.8
> backports that we had to defer and which are not yet backported into
> JDK11u: https://bugs.openjdk.java.net/browse/JDK-8238758?filter=38696

Thanks for doing that. I'll look at this and no doubt Severin also will.
We will also liaise with our Quarkus team to help pin down whether any
of these patches are really needed to fix issues with using stock jdk11u.

regards,


Andrew Dinn
-----------