[11u] RFC: Update Graal in OpenJDK 11 Updates?

[Gil Tene]

> On Mar 13, 2020, at 8:10 AM, Langer, Christoph <christoph.langer at sap.com> wrote:
> 
> Hi Gil,
> 
>> -----Original Message-----
>> From: Gil Tene <gil at azul.com>
>> Sent: Freitag, 13. März 2020 15:31
>> To: Langer, Christoph <christoph.langer at sap.com>
>> Cc: Andrew Hughes <gnu.andrew at redhat.com>; jdk-updates-
>> dev at openjdk.java.net; Andrew Dinn <adinn at redhat.com>; Severin
>> Gehwolf <sgehwolf at redhat.com>; Andrew Haley <aph at redhat.com>;
>> Thomas Wuerthinger <thomas.wuerthinger at oracle.com>; Doug Simon
>> <doug.simon at oracle.com>; Tom Rodriguez <tom.rodriguez at oracle.com>;
>> Volker Simonis <volker.simonis at gmail.com>
>> Subject: Re: [11u] RFC: Update Graal in OpenJDK 11 Updates?
>> 
>> 
>> 
>> Sent from my iPad
>> 
>>> On Mar 13, 2020, at 1:04 AM, Langer, Christoph
>> <christoph.langer at sap.com> wrote:
>>> 
>>> Hi Andrew, all,
>>> 
>>>> -----Original Message-----
>>>> From: Andrew Hughes <gnu.andrew at redhat.com>
>>>> Sent: Donnerstag, 12. März 2020 19:27
>>>> To: Langer, Christoph <christoph.langer at sap.com>; jdk-updates-
>>>> dev at openjdk.java.net; Andrew Dinn <adinn at redhat.com>; Severin
>>>> Gehwolf <sgehwolf at redhat.com>; Andrew Haley <aph at redhat.com>;
>>>> Thomas Wuerthinger <thomas.wuerthinger at oracle.com>; Doug Simon
>>>> <doug.simon at oracle.com>; Tom Rodriguez
>> <tom.rodriguez at oracle.com>;
>>>> Volker Simonis <volker.simonis at gmail.com>
>>>> Subject: Re: [11u] RFC: Update Graal in OpenJDK 11 Updates?
>>>> 
>>>> 
>>>> 
>>>>> On 12/03/2020 12:03, Langer, Christoph wrote:
>>>>> Hi all,
>>>>> 
>>>>> as you may or may not know, the GraalVM Community Edition (CE)
>>>> supports OpenJDK 11 update releases. In order to do that, there exists a
>>>> clone of the OpenJDK repository in the GraalVM GitHub space [0]. This
>> fork,
>>>> however, contains several additional patches, obviously mainly in the area
>> of
>>>> JVMCI and the Graal compiler. There seems to be a certain interest of the
>>>> involved parties, that is, the GraalVM community, Oracle Labs and
>> probably
>>>> also the OpenJDK community, to keep these repositories as close as
>> possible.
>>>>> 
>>>>> One motivation could be to allow building of GraalVM native images
>> from
>>>> OpenJDK distributions, such as mentioned in a GitHub issue raised against
>>>> the graal project [1].
>>>>> 
>>>>> Another motivation probably comes from the Graal community to
>> reduce
>>>> the maintenance effort from merging the OpenJDK changes.
>>>>> 
>>>>> Furthermore, it has been policy in the OpenJDK 11 Updates project to try
>> to
>>>> follow what Oracle is doing in their JDK 11 LTS update releases. Lately,
>> there
>>>> obviously have been backports for JVMCI and Graal which completely
>> don't
>>>> match with the current state of OpenJDK 11u. When researching this I
>>>> learned that the prerequisite of these kind of changes is a larger update
>> of
>>>> JVMCI and Graal. The basis for this change could be a commit from the
>> labs-
>>>> openjdk-11 Graal repository [2]. This would update JVMCI and Graal to
>> the
>>>> state of OpenJDK 13. More details can be taken out of the conversation of
>>>> the GitHub issue [1].
>>>>> 
>>>>> I took this change, applied it to OpenJDK 11u-dev, resolved some
>> conflicts
>>>> and made sure it builds on all platforms (that we support in the SAP
>> team). A
>>>> prerequisite would also be a backport of JDK-8208686, at least the hotspot
>>>> part of it, as provided in [4]. The Graal/Java parts of JDK-8208686 are
>>>> contained in [3]. With this work applied, regression testing looks quite
>>>> promising. There are a few test failures in the NMT area, some crashes
>> with
>>>> aot and a few minor Graal related test errors, but nothing that couldn't be
>>>> solved with too much of effort, I guess. But I can't make any judgement
>> on
>>>> the state of the Graal compiler itself, since we don't have much test
>> coverage
>>>> for that in our infrastructure.
>>>>> 
>>>>> However, the main point for me is to get a discussion started. First of all
>> it is
>>>> about whether we want to have this kind of update at all in OpenJDK 11
>>>> Updates? If so, I'd need some help by people with expertise and test
>>>> infrastructure for the Graal related part. That's nothing we can cover in
>> the
>>>> SAP team right now. If we decide against this update, we'll have give up
>> our
>>>> policy about following the Oracle 11 Updates for JVMCI/Graal.
>>>>> 
>>>>> Looking forward to get input about this.
>>>>> 
>>>>> Thanks and best regards
>>>>> Christoph
>>>>> 
>>>>> [0] https://github.com/graalvm/labs-openjdk-11/
>>>>> [1] https://github.com/oracle/graal/issues/2196
>>>>> [2] https://github.com/graalvm/labs-openjdk-
>>>> 11/commit/fbbdf5f15300716110d2ac9b9bd7f70b03b35a7d
>>>>> [3] http://cr.openjdk.java.net/~clanger/webrevs/graal13.11u/
>>>>> [4] http://cr.openjdk.java.net/~clanger/webrevs/8208686.11u/
>>>>> 
>>>> 
>>>> I'm aware of these changes for Graal and have been working on getting
>>>> some of them into OpenJDK 11u.
>>>> 
>>>> I don't think we should just apply these monolithic commits, but instead
>>>> work out the relevant issues in OpenJDK 13 being backported and do it
>>>> correctly. We want 11u to be as maintainable as possible.
>>> 
>>> I've got some sympathy with that approach. We can certainly have a look to
>> identify the single changes and try to apply them one by one... It'll probably
>> also help to identify the sources of regressions when we run into some.
>>> 
>>> But, the general question that needs to be answered before is: Do we
>> want this to happen at all? Any opinions/thoughts?
>> 
>> Can you elaborate on what we believe the Oracle 11u has done so far on this
>> matter?
> 
> In short, I believe Graal and JVMCI in Oracle 11u was updated to the JDK13 level. And from there on further patches have been backported.
> 
> The Graal/JVMCI update as a monolith would probably look like this: https://github.com/graalvm/labs-openjdk-11/commit/fbbdf5f15300716110d2ac9b9bd7f70b03b35a7d.
> And additional backports come on top, like most of the 23 items left in this query: https://bugs.openjdk.java.net/issues/?filter=37811

Can someone from Oracle comment on where Graal support is moving to in
Oracle JDK 11? And/or confirm that it is being updated to be roughly in line
 with ~JDK13?

On the assumption that Oracle is updating JVMCI and Graal in Oracle JDK 11
to JDK13 levels, I think that it would be appropriate to do the same in
OpenJDK 11u. IMO it would rise the "something is missing" level that should
be addressed.

However, as there are several changes in the monolith link above that touch
central pieces of code in the JVM, and do so outside of #if INCLUDE_JVMCI
 blocks, this looks like a large piece of work that needs to be done carefully
and with lots of though when it comes to code that is not localized to jvmci and
graal modules and jvmci-specific jvm files. I assume the above links are
just a pointer to the possible scope, and not necessarily a suggestion for
what the backport will look like exactly.

The backport there may very well end up being close to what we'd want for
11u, but I certainly wouldn't want to just take this bulk stuff from
graalvm/labs-openjdk-11 and assume that is was well though through for
handling non-Graal use modes (which are the main concern for
OpenJDK 11u, and probably of no significant concern to the Graal project).

> 
> 
> Best regards
> Christoph
>