RFR (11u, XXL): Upstream/backport Shenandoah to JDK11u

Fri Jun 26 17:28:51 UTC 2020

Hi Roman,

I am in favor of this proposed upstreaming of Shenandoah into jdk11u. Microsoft
has some long-term commitments to Java 11 customers and having Shenandoah
included in jdk11u upstream would definitely be well-received by my team and by
our customers. There is demand for this kind of GC option on OpenJDK 11, and it
would be beneficial to the community for it to be available on the upstream
OpenJDK rather than only in specific downstream releases. Coupled with the fact
that Shenandoah has been tested on JDK 11 for quite a while now, I think this
upstreaming makes sense.

You mentioned in your original mail that your team at RedHat intends to maintain
this proposed jdk11u version the same way it has been maintained in
shenandoah/jdk11. That is great--from what I’ve seen in these past few months
that I’ve been involved, I’ve been impressed by the way backports to
shenandoah/jdk11 are being handled, the effort that is made to keep the
Shenandoah code closely aligned between sh/11 and jdk/jdk, and the care taken to
ensure that the impact on shared code is kept to a minimum.

At Microsoft, we’ve already been contributing patches and investigating and
reporting issues based on our testing of Shenandoah, both on jdk/jdk and
shenandoah/jdk11. We intend to continue this kind of involvement, so if this
upstreaming to jdk11u goes through, your team will also have our support in
maintaining Shenandoah there.

Thanks,
Aditya

-----Original Message-----
From: jdk-updates-dev <jdk-updates-dev-retn at openjdk.java.net> On Behalf Of Roman Kennke
Sent: Thursday, June 25, 2020 4:16 AM
To: jdk-updates-dev at openjdk.java.net
Subject: Re: RFR (11u, XXL): Upstream/backport Shenandoah to JDK11u

I would like to revive this RFR, I believe we haven't come to a
conclusion yet. I still think it would be very useful to have the
Shenandoah GC included in jdk11u upstream. I have heard from several
vendors who would like to ship Shenandoah, but maintaining the rather
huge Shenandoah backport patch is an absolute no-go.

This is an updated patch that includes numerous bugfixes and
improvements in Shenandoah. It is what Red Hat is going to ship in
their 11.0.8 release.

Full webrev:
https://nam06.safelinks.protection.outlook.com/?url=https:%2F%2Fcr.openjdk.java.net%2F~rkennke%2Fshenandoah-jdk11u-upstream%2Fwebrev.04-all%2F&data=02%7C01%7Cadityam%40microsoft.com%7C7aec62734d22458243d908d818f974c5%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637286806967016879&sdata=G1HjEJ68qFJ2uwRyxfFKNE8gcbsFXEdYi%2B9RyrUeG60%3D&reserved=0

Webrev only for shared-code changes:
https://nam06.safelinks.protection.outlook.com/?url=https:%2F%2Fcr.openjdk.java.net%2F~rkennke%2Fshenandoah-jdk11u-upstream%2Fwebrev.04-shared%2F&data=02%7C01%7Cadityam%40microsoft.com%7C7aec62734d22458243d908d818f974c5%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637286806967026872&sdata=RrKNOvxc3%2FheThyi2cMiazE%2BxC9UfARxfTbmuoc640Y%3D&reserved=0

Please let me know what you think.

Roman

> On Fri, 2020-01-17 at 18:05 +0100, Roman Kennke wrote:
> Hello,
> 
> The Shenandoah GC has been integrated into jdk12 a little over a year
> ago:
> 
> https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2Fhg.openjdk.java.net%2Fjdk%2Fjdk%2Frev%2F9c18c9d839d3&data=02%7C01%7Cadityam%40microsoft.com%7C7aec62734d22458243d908d818f974c5%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637286806967026872&sdata=QJo2xwdHiEuaqsz26P2j%2BZdE6j3AxUvpnHVTjD9XGXI%3D&reserved=0
> 
> The Shenandoah team has been maintaining the Shenandoah-enabled
> jdk11u
> downstream repository since the inception of jdk11 under:
> 
> https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2Fhg.openjdk.java.net%2Fshenandoah%2Fjdk11&data=02%7C01%7Cadityam%40microsoft.com%7C7aec62734d22458243d908d818f974c5%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637286806967026872&sdata=P%2FLkJ5VCrq4oWzik3WZ3GTy1a3mA52mxkITasawIMk0%3D&reserved=0
> 
> In order to make it more useful and accessible for users, we would
> like
> to make it available in upstream jdk11u. The Shenandoah team at Red
> Hat
> intends to maintain it the same way it had been maintained in
> shenandoah/jdk11, in the course of regular jdk11u maintenance Red Hat
> already does.
> 
> Thanks to recent changes in Shenandoah over the last year, we are now
> able to fit the GC interface in jdk11 almost exactly. There are a few
> exceptions though. We tried to follow the following pattern for all
> shared-code changes where it made sense:
> 
> #if INCLUDE_SHENANDOAH_GC
>   if (UseShenandoahGC) {
>     ...
>   }
> #endif
> 
> This ensures that the Shenandoah-specific changes are cut out of the
> build when building without Shenandoah, and avoid those code paths at
> runtime when running without Shenandoah.
> 
> Also, there are a few places where this was not possible or where it
> didn't seem feasible, but we tried to keep them few and obvious.
> Whenever this is the case, we are mirroring as close as possible what
> we
> have in jdk/jdk.
> 
> Architecture-wise, Shenandoah runs on x86_64, aarch64, x86_32. Other
> architectures still build fine, and Shenandoah gets automatically
> disabled during configure if not supported. OS-wise, Shenandoah runs
> on
> Linux, Windows, and is known to run on MacOS X and Solaris too.
> 
> I wasn't sure which form this should take. I decided to start with
> the
> easiest possible form this could take: a simple patch/webrev and an
> RFR.
> Let me know if you need a JIRA issue or any other formalities to
> track that.
> 
> Full webrev:
> https://nam06.safelinks.protection.outlook.com/?url=https:%2F%2Fcr.openjdk.java.net%2F~rkennke%2Fshenandoah-jdk11u-upstream%2Fwebrev.02-all%2F&data=02%7C01%7Cadityam%40microsoft.com%7C7aec62734d22458243d908d818f974c5%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637286806967026872&sdata=eHspbGt5Eox4EMv8hstTBwsqFYefGNO2acGymP7ZfN0%3D&reserved=0
> 
> Webrev only for shared-code changes:
> https://nam06.safelinks.protection.outlook.com/?url=https:%2F%2Fcr.openjdk.java.net%2F~rkennke%2Fshenandoah-jdk11u-upstream%2Fwebrev.02-shared%2F&data=02%7C01%7Cadityam%40microsoft.com%7C7aec62734d22458243d908d818f974c5%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637286806967026872&sdata=LGghbNRyHPSJLK8l2Oyhxh1lhD46LmvZtN1G5htYBpk%3D&reserved=0
> 
> The webrevs are based on and depend on the arraycopy patch proposed
> for
> review here:
> 
> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmail.openjdk.java.net%2Fpipermail%2Fjdk-updates-dev%2F2020-January%2F002396.html&data=02%7C01%7Cadityam%40microsoft.com%7C7aec62734d22458243d908d818f974c5%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637286806967026872&sdata=Kw0qkfvtVNVrCiMeypwr4BHX9e3bZafJnVCMVY%2BvOOw%3D&reserved=0
> 
> Testing:
> The code has been tested many many times, continuously while
> maintaining
> it. It has also served as the basis for RPMs and shipped to
> customers.
> We are running all sorts of tests of the hotspot/jtreg testsuite, in
> particular hotspot_gc_shenandoah, CTW tests, the regular tier* tests,
> several popular benchmarks (specjvm, specjbb on a regular basis), on
> a
> nightly basis.
> 
> Please let me know what you think.
> 
> Thanks,
> Roman
> 
> 
> 
> 
> 
> 
> 