RFR (11u, XXL): Upstream/backport Shenandoah to JDK11u
Gil Tene
gil at azul.com
Fri Jun 26 23:39:03 UTC 2020
As I noted before, we have serious reservations about adding major new features to a
stable LTS that is upstream of all of 11u. Based on daily interactions with tons of OpenJDK
end users, I can say that the vast majority of end users that are ramping up on 11u are
demanding stability and maturity above all else. The addition of such major changes in
an 11u update will cause many of them to stall updates, and potentially stall the uptake
of 11u altogether, waiting for 11u to actually stabilize first (evidenced by no longer doing
such things), so they can keep up with fixes and security issues without having to take on
potential regressions driven by late added or experimental features.
If you want a modern GA'ed OpenJDK with all the latest features in it, use one. Please
please please don't force people who choose to stick with a given OpenJDK version for
stability, and knowingly did not move to a later OpenJDK version, to deploy the newly
implemented features that you like by pushing for back-porting them to the upstream
project that they get their security updates from...
I can see a potentially good case for a downstream-from-11u project that incorporates
ongoing development (as opposed to bug fixes) from later upstream versions, and would
be happy to collaborate on one. But the 11u that is upstream of all other 11u's should not
be seen as a developer's playground, or a place to add cool new features to a stable
release. The upstream versions are there for that purpose. The valid interests of
deep-bench, self-supporting groups making use of OpenJDK, who can take on the stability
risks involved due to the depth of their technical bench, should not overcome the interests
of the millions of consumers of the 11u builds that do not have that luxury, and who
need and expect stable (in the changing only where necessary sense) 11u releases.
I'd like to point out that 13u already includes Shenandoah. 15u (expected in 3 months)
already includes Shenandoah as a non-experimental feature. And we can all collaborate
on back-porting bug-fixes to those to updates to keep a lively version of OpenJDK that
includes Shenandoah up to date from a bug fix and security update point of view. For
those insisting on waiting for an LTS (on the assumption that LTSs are long term stable
and don't get those sort of messing-with-after-the-fact done to them, mind you), 17u will
be out in 15 months.
So we are not talking about some far-away future for end users that want Shenandoah.
It is here, now, in multiple consumable forms, and available for people not looking for
the stability of an LTS, and even for people who do (as they can use the Red Hat 11u
builds). It it will be available in more forms very soon with no effort, and we don't have
to destabilize 11u for others to make that happen.
— Gil.
> On Jun 26, 2020, at 1:51 PM, Mathiske, Bernd <mathiske at amazon.com> wrote:
>
> Hi Roman,
>
> We are also in favor of upstreaming Shenandoah to jdk11u, for the same reasons Aditya listed, and we agree with everything else he has written. Similar to Aditya’s message below, we believe that Shenandoah has strong demand and potential for those running jdk11u. Targeting jdk11u is the most efficient way to reach a large user base and to enable production use of Shenandoah.
>
> At Amazon, we plan to continue to contribute to Shenandoah, and we fully intend to help maintain Shenandoah in jdk11u.
>
> Best,
>
> Bernd, Kelvin, Azeem
>
> On 6/26/20, 10:36 AM, "jdk-updates-dev on behalf of Aditya Mandaleeka" <jdk-updates-dev-retn at openjdk.java.net on behalf of adityam at microsoft.com> wrote:
>
> Hi Roman,
>
> I am in favor of this proposed upstreaming of Shenandoah into jdk11u. Microsoft
> has some long-term commitments to Java 11 customers and having Shenandoah
> included in jdk11u upstream would definitely be well-received by my team and by
> our customers. There is demand for this kind of GC option on OpenJDK 11, and it
> would be beneficial to the community for it to be available on the upstream
> OpenJDK rather than only in specific downstream releases. Coupled with the fact
> that Shenandoah has been tested on JDK 11 for quite a while now, I think this
> upstreaming makes sense.
>
> You mentioned in your original mail that your team at RedHat intends to maintain
> this proposed jdk11u version the same way it has been maintained in
> shenandoah/jdk11. That is great--from what I’ve seen in these past few months
> that I’ve been involved, I’ve been impressed by the way backports to
> shenandoah/jdk11 are being handled, the effort that is made to keep the
> Shenandoah code closely aligned between sh/11 and jdk/jdk, and the care taken to
> ensure that the impact on shared code is kept to a minimum.
>
> At Microsoft, we’ve already been contributing patches and investigating and
> reporting issues based on our testing of Shenandoah, both on jdk/jdk and
> shenandoah/jdk11. We intend to continue this kind of involvement, so if this
> upstreaming to jdk11u goes through, your team will also have our support in
> maintaining Shenandoah there.
>
> Thanks,
> Aditya
>
> -----Original Message-----
> From: jdk-updates-dev <jdk-updates-dev-retn at openjdk.java.net> On Behalf Of Roman Kennke
> Sent: Thursday, June 25, 2020 4:16 AM
> To: jdk-updates-dev at openjdk.java.net
> Subject: Re: RFR (11u, XXL): Upstream/backport Shenandoah to JDK11u
>
> I would like to revive this RFR, I believe we haven't come to a
> conclusion yet. I still think it would be very useful to have the
> Shenandoah GC included in jdk11u upstream. I have heard from several
> vendors who would like to ship Shenandoah, but maintaining the rather
> huge Shenandoah backport patch is an absolute no-go.
>
> This is an updated patch that includes numerous bugfixes and
> improvements in Shenandoah. It is what Red Hat is going to ship in
> their 11.0.8 release.
>
> Full webrev:
> https://nam06.safelinks.protection.outlook.com/?url=https:%2F%2Fcr.openjdk.java.net%2F~rkennke%2Fshenandoah-jdk11u-upstream%2Fwebrev.04-all%2F&data=02%7C01%7Cadityam%40microsoft.com%7C7aec62734d22458243d908d818f974c5%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637286806967016879&sdata=G1HjEJ68qFJ2uwRyxfFKNE8gcbsFXEdYi%2B9RyrUeG60%3D&reserved=0
>
> Webrev only for shared-code changes:
> https://nam06.safelinks.protection.outlook.com/?url=https:%2F%2Fcr.openjdk.java.net%2F~rkennke%2Fshenandoah-jdk11u-upstream%2Fwebrev.04-shared%2F&data=02%7C01%7Cadityam%40microsoft.com%7C7aec62734d22458243d908d818f974c5%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637286806967026872&sdata=RrKNOvxc3%2FheThyi2cMiazE%2BxC9UfARxfTbmuoc640Y%3D&reserved=0
>
> Please let me know what you think.
>
> Roman
>
>> On Fri, 2020-01-17 at 18:05 +0100, Roman Kennke wrote:
>> Hello,
>>
>> The Shenandoah GC has been integrated into jdk12 a little over a year
>> ago:
>>
>> https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2Fhg.openjdk.java.net%2Fjdk%2Fjdk%2Frev%2F9c18c9d839d3&data=02%7C01%7Cadityam%40microsoft.com%7C7aec62734d22458243d908d818f974c5%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637286806967026872&sdata=QJo2xwdHiEuaqsz26P2j%2BZdE6j3AxUvpnHVTjD9XGXI%3D&reserved=0
>>
>> The Shenandoah team has been maintaining the Shenandoah-enabled
>> jdk11u
>> downstream repository since the inception of jdk11 under:
>>
>> https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2Fhg.openjdk.java.net%2Fshenandoah%2Fjdk11&data=02%7C01%7Cadityam%40microsoft.com%7C7aec62734d22458243d908d818f974c5%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637286806967026872&sdata=P%2FLkJ5VCrq4oWzik3WZ3GTy1a3mA52mxkITasawIMk0%3D&reserved=0
>>
>> In order to make it more useful and accessible for users, we would
>> like
>> to make it available in upstream jdk11u. The Shenandoah team at Red
>> Hat
>> intends to maintain it the same way it had been maintained in
>> shenandoah/jdk11, in the course of regular jdk11u maintenance Red Hat
>> already does.
>>
>> Thanks to recent changes in Shenandoah over the last year, we are now
>> able to fit the GC interface in jdk11 almost exactly. There are a few
>> exceptions though. We tried to follow the following pattern for all
>> shared-code changes where it made sense:
>>
>> #if INCLUDE_SHENANDOAH_GC
>> if (UseShenandoahGC) {
>> ...
>> }
>> #endif
>>
>> This ensures that the Shenandoah-specific changes are cut out of the
>> build when building without Shenandoah, and avoid those code paths at
>> runtime when running without Shenandoah.
>>
>> Also, there are a few places where this was not possible or where it
>> didn't seem feasible, but we tried to keep them few and obvious.
>> Whenever this is the case, we are mirroring as close as possible what
>> we
>> have in jdk/jdk.
>>
>> Architecture-wise, Shenandoah runs on x86_64, aarch64, x86_32. Other
>> architectures still build fine, and Shenandoah gets automatically
>> disabled during configure if not supported. OS-wise, Shenandoah runs
>> on
>> Linux, Windows, and is known to run on MacOS X and Solaris too.
>>
>> I wasn't sure which form this should take. I decided to start with
>> the
>> easiest possible form this could take: a simple patch/webrev and an
>> RFR.
>> Let me know if you need a JIRA issue or any other formalities to
>> track that.
>>
>> Full webrev:
>> https://nam06.safelinks.protection.outlook.com/?url=https:%2F%2Fcr.openjdk.java.net%2F~rkennke%2Fshenandoah-jdk11u-upstream%2Fwebrev.02-all%2F&data=02%7C01%7Cadityam%40microsoft.com%7C7aec62734d22458243d908d818f974c5%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637286806967026872&sdata=eHspbGt5Eox4EMv8hstTBwsqFYefGNO2acGymP7ZfN0%3D&reserved=0
>>
>> Webrev only for shared-code changes:
>> https://nam06.safelinks.protection.outlook.com/?url=https:%2F%2Fcr.openjdk.java.net%2F~rkennke%2Fshenandoah-jdk11u-upstream%2Fwebrev.02-shared%2F&data=02%7C01%7Cadityam%40microsoft.com%7C7aec62734d22458243d908d818f974c5%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637286806967026872&sdata=LGghbNRyHPSJLK8l2Oyhxh1lhD46LmvZtN1G5htYBpk%3D&reserved=0
>>
>> The webrevs are based on and depend on the arraycopy patch proposed
>> for
>> review here:
>>
>> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmail.openjdk.java.net%2Fpipermail%2Fjdk-updates-dev%2F2020-January%2F002396.html&data=02%7C01%7Cadityam%40microsoft.com%7C7aec62734d22458243d908d818f974c5%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637286806967026872&sdata=Kw0qkfvtVNVrCiMeypwr4BHX9e3bZafJnVCMVY%2BvOOw%3D&reserved=0
>>
>> Testing:
>> The code has been tested many many times, continuously while
>> maintaining
>> it. It has also served as the basis for RPMs and shipped to
>> customers.
>> We are running all sorts of tests of the hotspot/jtreg testsuite, in
>> particular hotspot_gc_shenandoah, CTW tests, the regular tier* tests,
>> several popular benchmarks (specjvm, specjbb on a regular basis), on
>> a
>> nightly basis.
>>
>> Please let me know what you think.
>>
>> Thanks,
>> Roman
>>
>>
>>
>>
>>
>>
>>
>
>
More information about the jdk-updates-dev
mailing list