[11u] RFC: Update Graal in OpenJDK 11 Updates?

[Doug Simon]

> On 23 Mar 2020, at 13:36, Andrew Dinn <adinn at redhat.com> wrote:
> 
> Hi Doug,
> 
> On 23/03/2020 10:25, Doug Simon wrote:
>> The Graal sources in graalvm/labs-openjdk-11
>> <https://urldefense.com/v3/__https://github.com/graalvm/labs-openjdk-11__;!!GqivPVa7Brio!M-pcUkpK03riaDVCDWa7ewkFb-YF4FfWv5YR6E7mIAMPlQWNxJYlW2uQgdXeHWJ5$ > are not updated with those
>> from oracle/graal <https://urldefense.com/v3/__https://github.com/oracle/graal__;!!GqivPVa7Brio!M-pcUkpK03riaDVCDWa7ewkFb-YF4FfWv5YR6E7mIAMPlQWNxJYlW2uQgR5KwNjv$ >. They are mostly as
>> they were when JDK 11 was released modulo changes need to adapt to the
>> JVMCI updates. As you state, the primary reason for this is that the
>> sources from oracle/graal are used when building GraalVM on top
>> of graalvm/labs-openjdk-11. We mostly ignore the Graal sources in
>> GraalVM/labs-openjdk-11.
> I understand that and appreciate that any need to update the in-tree
> jdk11u graal code in tandem with jdk11u hotspot changes is not for the
> sake of the wider GraalVM project. It is needed to ensure that this
> in-tree graal code is consistent with the hotspot code and hence
> continues to allow use of the (in-tree) GraalVM compiler as a JVMCI JIT
> replacement for C2 or to underpin the implementation of jaotc i.e. to
> keep these two in-tree components in sync.
> 
> To address Christoph's question:
> 
> At Red Hat we are in the situation where we know that the labs  jdk11u
> changes currently backported to OpenJDK jdk11u are enough to build the
> (out-of-tree) Graal compiler and substratevm deliverables. In order to
> allow this to work the extra build step of generating and installing
> static libs must be performed. HOwever, that extra step uses build
> system patches added to head and backported to jdk11u i.e. is catered
> for /within/ the current release. So, in sum: as far as the mx build
> step is concerned a current jdk11u build /can/ be used in place of the
> Oracle labs download.
> 
> We also know that the resulting build of the compiler and substratevm
> tree can be used to successfully generate native images with that same
> jdk11u. That's not to say everything works but we do know that some
> programs can be built and run as native images. We do not know (because
> we have not tried it) how well we current jdk11u supports Truffle (or
> the LLVM backend).
> 
> So, modulo further testing throwing up problems, we may perhaps be able
> to leave jdk11u as is and still be able to use it to support building of
> graal and native images. So, if there are pending JVMCI-related changes
> to hotspot and, in consequence, to the in-tree graal code then we really
> need a good reason to backport them.
> 
> Which consideration applies to the changes Christoph mentioned (they
> appear to be to do with routing exceptions out of hotspot via JVMCI and
> look relatively innocuous to me but don't count that as a full review
> just yet). So, can you provide a reason why these are important or even
> critical patches?

Sorry, this thread is long and has mentioned a number of patches. Which ones are you referring to here?

> Are there already known issues we have not yet come across in our
> testing that justify back-porting them? Alternatively, are there
> upstream changes either in place or in progress in the out-of-tree graal
> code base that have to depend on the level of JVMCI support these
> patches provide (which seems essentially to be par with jdk13u) and
> therefore imply that a backport is going to be absolutely necessary?

I’m having trouble parsing that question. Are you asking “is there anything in GitHub Graal that depends on the JVMCI changes in labs-openjdk-11”?

> 
> The mandatory follow-up question is whether these proposed changes
> necessitate consequent updates to the in-tree graal code and if so which
> specific ones. I'll take the whole bundle as an answer but it would be
> much better to know if some of the contained changes are optional. If we
> do end up backporting upstream code we would much prefer to do it patch
> by patch rather than trying to push the bundled update found in the labs
> jdk11u. Reducing the backport to only essential stuff would be much
> preferred. We need to be sure to guarantee continuity and reliability of
> function for a JVMCI-enabled  Graal JIT and Graal-based jaotc and of
> course we also need to make an assessment as regards security.

I believe Tom has previously provided a breakdown of the patches in labs-openjdk-11. Everything in labs-openjdk-11 is essential for supporting complete GraalVM functionality as far as I understand otherwise it would not be there.

> 
> Finally, I am asking that now re Chrisoph's current proposed changes.
> However, as you can imagine, the same questions are rightly going to be
> asked as regards future requests for changes.

I think the high order point is that porting the changes in labs-openjdk-11 to jdk11u is going to be complex due the way we back-ported changes from JDK 13. With more time, we would have tried to be more principled about it but on the other hand we were keen to release GraalVM on JDK 11 given how much demand there was for it. 

It might help to have a Zoom call to discuss this further. I want Tom to be on the call so some time between 18:00 - 21:00 CET would probably work. I can do any day except Tuesday.

-Doug