From claes.redestad at oracle.com Wed Jul 1 00:07:22 2020 From: claes.redestad at oracle.com (Claes Redestad) Date: Wed, 1 Jul 2020 02:07:22 +0200 Subject: ... In-Reply-To: References: <3595e963-fa5b-bfce-c463-841b84d4103f@redhat.com> <6468CB08-3DB5-488E-9122-E3699A95218B@amazon.com> <7FDB6E37-E850-49DF-8255-059E7F9771C5@azul.com> <1551271D-7BEB-4CE2-A432-A2C62A813143@azul.com> <8bccbed2-0d9a-b198-1366-4b66a2ca5446@redhat.com> <5DBDA26C-E160-4175-97F7-21CC7FEC0DD8@azul.com> <32219A90-E722-43DF-863B-3EF3084DE8F7@azul.com> Message-ID: <55b122c0-a92c-2adf-36e9-3a5c6faaf48a@oracle.com> On 2020-06-30 08:03, Roman Kennke wrote: > How is the JFR backport any different, though? One major vendor > included it in their commercial offering. Other vendors wanted to > include it too, in an 8u LTS that was arguably much longer down the > path of stabilization. There was nothing broken in OpenJDK 8u as it > was, that needed fixing IMO. It would have been just as reasonable to > tell users who want JFR in their JVM to use a later LTS instead. And it > came with very considerable risks, I'd argue with much higher risk than > adding Shenandoah, because it required hooks and changes all over the > VM. Why is it treated different, then? What qualifies JFR as necessity > that justifies taking such high risk in such a mature and stabilized > release? Why didn't we have this discussion back then? I made the case back then that backporting JFR was the wrong call, mainly on the basis that what was proposed to be backported was essentially a very much reworked feature that had had a lot of brain surgery compared to the JFR in Oracle's 8u. What's now offered in OpenJDK 8u is *not* a drop-in replacement to the JFR Oracle JDK has in 8u. Something on the surface very similar to it, sure. But not the same thing. This huge risk, and the fragmentation it brings with it, IMHO means that the JFR backport should have been considered unacceptable for 8u LTS. Still should be. That's the only thing I think Gil is slightly wrong about in this discussion. $.02 /Claes From takiguc at linux.vnet.ibm.com Wed Jul 1 05:48:30 2020 From: takiguc at linux.vnet.ibm.com (Ichiroh Takiguchi) Date: Wed, 01 Jul 2020 14:48:30 +0900 Subject: [11u] CSR JDK-8245689 Align some one-way conversion in MS950 charset with Windows In-Reply-To: References: Message-ID: <95414bbbed1aa20c0376bb90fe690e69@linux.vnet.ibm.com> Hello Christoph. I appreciate your suggestion and sorry for bad response. I created new CSR for 11u is JDK-8248305 [1] (Base CSR was JDK-8233385 [2], I just copied the contents from JDK-8233385 [2]) Could you review JDK-8248305 [1] ? [1] https://bugs.openjdk.java.net/browse/JDK-8248305 [2] https://bugs.openjdk.java.net/browse/JDK-8233385 Thanks, Ichiroh Takiguchi On 2020-06-23 13:38, Langer, Christoph wrote: > Hi Ichiroh, > > I've just checked your request. > > The bug-item you created > https://bugs.openjdk.java.net/browse/JDK-8245689 is of type > "backport". I guess you've intuitively used "More" -> "Create > Backport" of the original CSR. Unfortunately, that doesn't work > correctly. > So, I've made JDK-8245689 a backport of the original bug > (https://bugs.openjdk.java.net/browse/JDK-8232161). It should get > resolved once you actually push the backport (under its original bug > id, of course). However, from that backport bug (JDK-8245689) you'll > have to create (and edit) the backport CSR via "More" -> "Create CSR". > Please let me know when you've done that and I can review. > > Cheers > Christoph > >> -----Original Message----- >> From: jdk-updates-dev On >> Behalf Of Ichiroh Takiguchi >> Sent: Montag, 25. Mai 2020 03:55 >> To: jdk-updates-dev at openjdk.java.net >> Subject: [11u] CSR JDK-8245689 Align some one-way conversion in MS950 >> charset with Windows >> >> Hello. >> >> Could you review JDK-8245689 [1] ? >> >> Parent CSR was JDK-8233385 Align some one-way conversion in MS950 >> charset with Windows [2] >> >> Actual fix was JDK-8232161 Align some one-way conversion in MS950 >> charset with Windows [3] >> The fix can be applied against 11u-dev without any change. >> >> [1] https://bugs.openjdk.java.net/browse/JDK-8245689 >> [2] https://bugs.openjdk.java.net/browse/JDK-8233385 >> [3] https://bugs.openjdk.java.net/browse/JDK-8232161 >> >> Thanks, >> Ichiroh Takiguchi >> IBM Japan, Ltd. From fzakkak at redhat.com Wed Jul 1 12:12:09 2020 From: fzakkak at redhat.com (Foivos Zakkak) Date: Wed, 1 Jul 2020 15:12:09 +0300 Subject: [11u] RFR: backport JDK-8247246: Add explicit ResolvedJavaType.link and expose, presence of default methods Message-ID: <9fdcf8e5-1888-014e-2aaf-05688fee91fc@redhat.com> Hello all, I am attaching a backport https://bugs.openjdk.java.net/browse/JDK-8247246 to jdk11u-dev. This backport fixes https://github.com/oracle/graal/issues/2628 The backport applied cleanly except for some minor issues which I resolved by changing: JVMCI_THROW(NullPointerException); to THROW(vmSymbols::java_lang_NullPointerException()); and Klass* klass = JVMCIENV->asKlass(jvmci_type); to Klass* klass = CompilerToVM::asKlass(jvmci_type); The backport has been successfully tested using: make run-test TEST="hotspot/jtreg/compiler/jvmci" make run-test-tier1 I have also successfully built graal master against the patched JDK Thanks, Foivos Zakkak PS: Pretty happy to submit my first contribution to OpenJDK -------------- next part -------------- A non-text attachment was scrubbed... Name: 8247246-jdk11u-dev-backport.patch Type: text/x-patch Size: 17601 bytes Desc: not available URL: From aph at redhat.com Wed Jul 1 15:10:49 2020 From: aph at redhat.com (Andrew Haley) Date: Wed, 1 Jul 2020 16:10:49 +0100 Subject: RFR (11u, XXL): Upstream/backport Shenandoah to JDK11u In-Reply-To: <5DBDA26C-E160-4175-97F7-21CC7FEC0DD8@azul.com> References: <3595e963-fa5b-bfce-c463-841b84d4103f@redhat.com> <6468CB08-3DB5-488E-9122-E3699A95218B@amazon.com> <7FDB6E37-E850-49DF-8255-059E7F9771C5@azul.com> <1551271D-7BEB-4CE2-A432-A2C62A813143@azul.com> <8bccbed2-0d9a-b198-1366-4b66a2ca5446@redhat.com> <5DBDA26C-E160-4175-97F7-21CC7FEC0DD8@azul.com> Message-ID: Before I go into this, I will make one thing clear: I am not advocating for Shenandoah to go in to JDK 11. In the end the decision is up to me, and I have not decided yet. I can't simultaneously be the arbiter and an advocate for one of the sides. However, am I going to examine your argument in orderto find out for myself how much of it stands up. On 29/06/2020 20:04, Gil Tene wrote: > >> On Jun 29, 2020, at 10:10 AM, Andrew Haley wrote: >> >> On 29/06/2020 17:38, Gil Tene wrote: >>>> Or is it that all changes to JDK11u are in-principle bad, regardless >>>> of their actual risk, which we should not consider? >>> >>> All feature additions in updates for an already released version (as >>> opposed to bug fixes and security updates, which those updates >>> are the vehicle for) are in-principle bad IMO. Sometimes we have to >>> do the bad thing because it is less bad than the alternative (the LTS >>> will break for many people if we don't do it). But it is still bad. >> >> OK, so that is your position, and the in-principle argument is AFAICS >> independent of the practical risk. > > Right. For non-neccesary *feature additions* to an existing LTS. So, your position is based on the solid rock of a principle that is entirely non-negotiable except for one thing: necessity, which inevitably is a judgment call. But this is not a principle that the project has ever followed (AFAIK), so we'd have to decide to accept it in order for your argument certainly to succeed. And right now I can't see any convincing reason why we would want to bind ourselves in that way. >> That fits with my understanding of your position: no matter how >> helpful a feature is, or how low the risk of adding it is, it should >> not happen. So, your conclusion is *entirely independent* of the >> actual ratio of risk to reward. No evaluation of either is necessary >> because the conclusion will always be the same: no. > > Exactly. For non-neccesary *feature additions* to an existing LTS. >> >> I hope that you will agree that your position on this matter is an >> extreme one; in fact is the most extreme position it is possible for >> anyone to take. > > I disagree. I don't think this is an extreme position at all. In fact, I > think it is quite mainstream, and represents the most common > understanding of what stable releases are, and what most consumers > of updates to such releases think is going on. It is extreme in the sense that it is hard to imagine a software community accepting any more extreme position, even if that position is a popular one. (It is surely possible for a majority to take an extreme position!) > But I think that where you think the vast majority of the community > sits on this position being "extreme" or the opposite being the > extreme one depends on who you think the community is. > > This list tends to be dominated by developers of OpenJDK. > > I claim that our community (for the updates projects) is the *users* > of OpenJDK. OK, but IMO the last thing this conversation needs is anyone claiming to represent the "silent majority". We the developers are what I mean when I talk about the OpenJDK community, and the OpenJDK community of developers is the consensus I seek. Consensus does not mean unanimity: that would give any developer an effective veto. >> Please allow me to suggest a little thought experiment. >> >> Let's say that we could import Feature X (which is Shenandoah, but I'd >> like to do make this discussion more general) without any risk. Sure, >> risk is never zero, but let's say: >> >> 1. All changes to the source code are surrounded by compile-time >> #if USE_SHENANDOAH_GC. >> >> 2. USE_SHENANDOAH_GC is never set unless it is explicitly requested by >> a configure argument. >> >> 3. All changes to the source code within the USE_SHENANDOAH_GC markers >> are also guarded by runtime tests of the form >> if (UseShenandoahGC) { ... >> >> I maintain that no-one will ever be forced to use X. Firstly, no >> downstream builder of our release JDK 11u will even build X unless >> they explicitly choose to. Also, unless UseShenandoahGC is enabled on >> the Java command line, no downstream user will be forced to use it >> either. >> >> What, in your opinion, are the practical risks to production users of >> the above? I'm assuming that we can confirm that the above plan works >> and does not touch anything it shouldn't by a combination of tooling >> and several sets of eyeballs. That's not perfect, but nothing is. > > Let me answer both in general, and then in the specific: > > General: > > First, my point i that the practical risk does not matter, as no necessity > has been demonstrated or even argued for. And I claim that this is not > an extreme position (and that if we want to label things as extreme, > the opposite would be the extreme). > > Next, I'll certainly accept that with careful review of some things it is > quite possible to keep risks very low. E.g. in cases where ALL > code is protected by #ifdef the risk is somewhat mitigated. > And that in cases where ALL non-ifdef-covered runtime flags > protect code paths the risk mitigation is not quite as good, but > stlll could be good. > > I'll even go farther and say that risk can (and should) be seriously > reduced even when the above stated protections are not practical. > Deep review (with an eye to minimizing risk around change, as > opposed to the typical review that focuses on correctness, code > structure, cleanliness, elegance, reuse, maintainability, etc.) is key to > minimizing risk when risk HAS to be taken. And as you well know, we > are going through that excercize right now with TLS1.3 in 8u. Indeed we are. > None of these are good enough to overcome the "no necessity" bar > IMO, because how good and low risk a change is is irrelevant > (in an LTS update) when it is unnecessary. OK, I understand that point. > Specific: > The above two are generic policy approaches. When it comes to the > specific case here (the addition of a new collector to 11u), which touches > a huge number of source files, some of the above description only apply > "whereever reasonably possible", and actual code changes to common > paths that are not protected by #ifdefs avoided statically via launch time > configuration flags exist (i.e. every dynamic execution of some common > code is now goigg through different logic that is considering the > configuration flags, or worse, common code changes exist where no > conditionals are applied at all). > That is not a subtle difference. Indeed not. That, if true, would be a powerful argument against accepting Shenandoah. It would also be an argument based on engineering issues rather than those of high-minded principle. > With that said, this last argument is not a logic path I'm looking to go > down, because I don't think the additional risk involved in something as > big and as intrusive as adding Shenandoah actually matters here. I'd > be making the same neccesity-based arguments for other features that > don't share that level change and risk, so I don't want to rathole in > discussing the specific code risks in this case. I understand that too, and appealing to a hard-and-fast rule means that we don't even have to examine the proposed patch to see how risky it is. -- Andrew Haley (he/him) Java Platform Lead Engineer Red Hat UK Ltd. https://keybase.io/andrewhaley EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From ron.pressler at oracle.com Wed Jul 1 16:09:35 2020 From: ron.pressler at oracle.com (Ron Pressler) Date: Wed, 1 Jul 2020 17:09:35 +0100 Subject: RFR (11u, XXL): Upstream/backport Shenandoah to JDK11u In-Reply-To: References: <3595e963-fa5b-bfce-c463-841b84d4103f@redhat.com> Message-ID: Without expressing an opinion on this particular issue, I?d just like to respond to two points you made: On 29 June 2020 at 19:32:31, Lindenmaier, Goetz (goetz.lindenmaier at sap.com(mailto:goetz.lindenmaier at sap.com)) wrote: > Hi, > > We as SAP would not object against this change. > Several reasons: > > 1. I had a look at the shared changes. > If this is all, it is very unlikely that this change breaks > the current configuration of jdk11u. Nice job of bringing > this to 11. > > 2. In the past, the VMs got a lot of new features > in hotspot every half year. This worked, too. > We even delivered Java 4-7 with hotspot of 8, > without breaking our huge software stack. Yes, > there were errors, but much more were solved > by this approach. That was when the number of people working on the current JDK with the new VM features was >10x the number of people working on Updates today. It?s hard to compare the old model wit the new. > > 3. My experience is that it is much harder to > move software to new Java versions than to > bring a feature to an older VM.? Again, this should not be the case with the *new* release model unless 1. the application suffers from severe technical debt and relies on deprectated/unmaintained/internal code or 2. something? is wrong or unusual about the new release. In other words, this? shouldn?t be the case unless there is something wrong with your? app or with the JDK.? Under the new release model, reasonably well-maintained applications? should normally take very little effort to update to a new feature release. > We have lots of software > running where the development teams are almost > gone. But the maintainers of the VM on SAP side, > and the operators of the software at customer side, > are still there. These are the ones that have to deal > with the bugs introduced by the downport. These are also > the ones whose problems are solved by a better GC. > The application developers would have to deal with > bringing the software to new Java versions, but they > are no more available. > > Naturally, this does not hold for features that > break compatibility of the libraries or the language. > > But please configure the build so that Shenandoah is > not built per default. See make/autoconf/hotspot.m4. > > Shenandoah could be enabled later, but at first I would > prefer if it is not compiled into everybodys binary. > > Best regards, > Goetz. > > PS: I hope this mail gets delivered to the list, a row > of my mails today didn't make it there. > > > > > > > > -----Original Message----- > > From: jdk-updates-dev On Behalf > > Of Roman Kennke > > Sent: Thursday, June 25, 2020 1:16 PM > > To: jdk-updates-dev at openjdk.java.net > > Subject: Re: RFR (11u, XXL): Upstream/backport Shenandoah to JDK11u > > > > I would like to revive this RFR, I believe we haven't come to a > > conclusion yet. I still think it would be very useful to have the > > Shenandoah GC included in jdk11u upstream. I have heard from several > > vendors who would like to ship Shenandoah, but maintaining the rather > > huge Shenandoah backport patch is an absolute no-go. > > > > This is an updated patch that includes numerous bugfixes and > > improvements in Shenandoah. It is what Red Hat is going to ship in > > their 11.0.8 release. > > > > Full webrev: > > https://cr.openjdk.java.net/~rkennke/shenandoah-jdk11u- > > upstream/webrev.04-all/ > > > > Webrev only for shared-code changes: > > https://cr.openjdk.java.net/~rkennke/shenandoah-jdk11u- > > upstream/webrev.04-shared/ > > > > Please let me know what you think. > > > > Roman > > > > > On Fri, 2020-01-17 at 18:05 +0100, Roman Kennke wrote: > > > Hello, > > > > > > The Shenandoah GC has been integrated into jdk12 a little over a year > > > ago: > > > > > > http://hg.openjdk.java.net/jdk/jdk/rev/9c18c9d839d3 > > > > > > The Shenandoah team has been maintaining the Shenandoah-enabled > > > jdk11u > > > downstream repository since the inception of jdk11 under: > > > > > > http://hg.openjdk.java.net/shenandoah/jdk11 > > > > > > In order to make it more useful and accessible for users, we would > > > like > > > to make it available in upstream jdk11u. The Shenandoah team at Red > > > Hat > > > intends to maintain it the same way it had been maintained in > > > shenandoah/jdk11, in the course of regular jdk11u maintenance Red Hat > > > already does. > > > > > > Thanks to recent changes in Shenandoah over the last year, we are now > > > able to fit the GC interface in jdk11 almost exactly. There are a few > > > exceptions though. We tried to follow the following pattern for all > > > shared-code changes where it made sense: > > > > > > #if INCLUDE_SHENANDOAH_GC > > > if (UseShenandoahGC) { > > > ... > > > } > > > #endif > > > > > > This ensures that the Shenandoah-specific changes are cut out of the > > > build when building without Shenandoah, and avoid those code paths at > > > runtime when running without Shenandoah. > > > > > > Also, there are a few places where this was not possible or where it > > > didn't seem feasible, but we tried to keep them few and obvious. > > > Whenever this is the case, we are mirroring as close as possible what > > > we > > > have in jdk/jdk. > > > > > > Architecture-wise, Shenandoah runs on x86_64, aarch64, x86_32. Other > > > architectures still build fine, and Shenandoah gets automatically > > > disabled during configure if not supported. OS-wise, Shenandoah runs > > > on > > > Linux, Windows, and is known to run on MacOS X and Solaris too. > > > > > > I wasn't sure which form this should take. I decided to start with > > > the > > > easiest possible form this could take: a simple patch/webrev and an > > > RFR. > > > Let me know if you need a JIRA issue or any other formalities to > > > track that. > > > > > > Full webrev: > > > https://cr.openjdk.java.net/~rkennke/shenandoah-jdk11u- > > upstream/webrev.02-all/ > > > > > > Webrev only for shared-code changes: > > > https://cr.openjdk.java.net/~rkennke/shenandoah-jdk11u- > > upstream/webrev.02-shared/ > > > > > > The webrevs are based on and depend on the arraycopy patch proposed > > > for > > > review here: > > > > > > https://mail.openjdk.java.net/pipermail/jdk-updates-dev/2020- > > January/002396.html > > > > > > Testing: > > > The code has been tested many many times, continuously while > > > maintaining > > > it. It has also served as the basis for RPMs and shipped to > > > customers. > > > We are running all sorts of tests of the hotspot/jtreg testsuite, in > > > particular hotspot_gc_shenandoah, CTW tests, the regular tier* tests, > > > several popular benchmarks (specjvm, specjbb on a regular basis), on > > > a > > > nightly basis. > > > > > > Please let me know what you think. > > > > > > Thanks, > > > Roman > > > > > > > > > > > > > > > > > > > > > > From sgehwolf at redhat.com Wed Jul 1 16:27:58 2020 From: sgehwolf at redhat.com (Severin Gehwolf) Date: Wed, 01 Jul 2020 18:27:58 +0200 Subject: [11u] RFR: backport JDK-8247246: Add explicit ResolvedJavaType.link and expose, presence of default methods In-Reply-To: <9fdcf8e5-1888-014e-2aaf-05688fee91fc@redhat.com> References: <9fdcf8e5-1888-014e-2aaf-05688fee91fc@redhat.com> Message-ID: <0d26983c2aba501c995dc73cbd84de58023b34b6.camel@redhat.com> Hi Foivos, On Wed, 2020-07-01 at 15:12 +0300, Foivos Zakkak wrote: > Hello all, > > I am attaching a backport > https://bugs.openjdk.java.net/browse/JDK-8247246 to jdk11u-dev. For reference, this is the original change in JDK 15: https://hg.openjdk.java.net/jdk/jdk15/rev/b58fc6058055 > This backport fixes https://github.com/oracle/graal/issues/2628 > > The backport applied cleanly except for some minor issues which I > resolved by changing: > > JVMCI_THROW(NullPointerException); > to > THROW(vmSymbols::java_lang_NullPointerException()); > > and > > Klass* klass = JVMCIENV->asKlass(jvmci_type); > to > Klass* klass = CompilerToVM::asKlass(jvmci_type); > > > The backport has been successfully tested using: > > make run-test TEST="hotspot/jtreg/compiler/jvmci" > make run-test-tier1 > > I have also successfully built graal master against the patched JDK src/hotspot/share/jvmci/jvmciCompilerToVM.cpp The original change didn't include a copyright update. So should the backport. This actually applies to all files touched. Thanks, Severin > Thanks, > Foivos Zakkak > > PS: Pretty happy to submit my first contribution to OpenJDK > From fzakkak at redhat.com Wed Jul 1 18:09:22 2020 From: fzakkak at redhat.com (Foivos Zakkak) Date: Wed, 1 Jul 2020 21:09:22 +0300 Subject: [11u] RFR: backport JDK-8247246: Add explicit ResolvedJavaType.link and expose, presence of default methods In-Reply-To: <0d26983c2aba501c995dc73cbd84de58023b34b6.camel@redhat.com> References: <9fdcf8e5-1888-014e-2aaf-05688fee91fc@redhat.com> <0d26983c2aba501c995dc73cbd84de58023b34b6.camel@redhat.com> Message-ID: Hi Severin, Thanks for your review. I am attaching a new patch without the copyright updates this time. Regards, Foivos On 01/07/2020 19:27, Severin Gehwolf wrote: > Hi Foivos, > > On Wed, 2020-07-01 at 15:12 +0300, Foivos Zakkak wrote: >> Hello all, >> >> I am attaching a backport >> https://bugs.openjdk.java.net/browse/JDK-8247246 to jdk11u-dev. > For reference, this is the original change in JDK 15: > https://hg.openjdk.java.net/jdk/jdk15/rev/b58fc6058055 > >> This backport fixes https://github.com/oracle/graal/issues/2628 >> >> The backport applied cleanly except for some minor issues which I >> resolved by changing: >> >> JVMCI_THROW(NullPointerException); >> to >> THROW(vmSymbols::java_lang_NullPointerException()); >> >> and >> >> Klass* klass = JVMCIENV->asKlass(jvmci_type); >> to >> Klass* klass = CompilerToVM::asKlass(jvmci_type); >> >> >> The backport has been successfully tested using: >> >> make run-test TEST="hotspot/jtreg/compiler/jvmci" >> make run-test-tier1 >> >> I have also successfully built graal master against the patched JDK > src/hotspot/share/jvmci/jvmciCompilerToVM.cpp > > The original change didn't include a copyright update. So should the > backport. > > This actually applies to all files touched. > > Thanks, > Severin > >> Thanks, >> Foivos Zakkak >> >> PS: Pretty happy to submit my first contribution to OpenJDK >> -------------- next part -------------- A non-text attachment was scrubbed... Name: 8247246-jdk11u-dev-backport.patch Type: text/x-patch Size: 15751 bytes Desc: not available URL: From gil at azul.com Wed Jul 1 18:47:34 2020 From: gil at azul.com (Gil Tene) Date: Wed, 1 Jul 2020 18:47:34 +0000 Subject: RFR (11u, XXL): Upstream/backport Shenandoah to JDK11u In-Reply-To: References: <3595e963-fa5b-bfce-c463-841b84d4103f@redhat.com> <6468CB08-3DB5-488E-9122-E3699A95218B@amazon.com> <7FDB6E37-E850-49DF-8255-059E7F9771C5@azul.com> <1551271D-7BEB-4CE2-A432-A2C62A813143@azul.com> <8bccbed2-0d9a-b198-1366-4b66a2ca5446@redhat.com> <5DBDA26C-E160-4175-97F7-21CC7FEC0DD8@azul.com> Message-ID: > On Jul 1, 2020, at 8:10 AM, Andrew Haley > wrote: > > Before I go into this, I will make one thing clear: I am not > advocating for Shenandoah to go in to JDK 11. > > In the end the decision is up to me, and I have not decided yet. I > can't simultaneously be the arbiter and an advocate for one of the > sides. Thank you for the reasoned consideration of the multiple sides of this issue and of the more general "policy questions" that it raises. > > However, am I going to examine your argument in orderto find out for > myself how much of it stands up. > > On 29/06/2020 20:04, Gil Tene wrote: >> >>> On Jun 29, 2020, at 10:10 AM, Andrew Haley > wrote: >>> >>> On 29/06/2020 17:38, Gil Tene wrote: >>>>> Or is it that all changes to JDK11u are in-principle bad, regardless >>>>> of their actual risk, which we should not consider? >>>> >>>> All feature additions in updates for an already released version (as >>>> opposed to bug fixes and security updates, which those updates >>>> are the vehicle for) are in-principle bad IMO. Sometimes we have to >>>> do the bad thing because it is less bad than the alternative (the LTS >>>> will break for many people if we don't do it). But it is still bad. >>> >>> OK, so that is your position, and the in-principle argument is AFAICS >>> independent of the practical risk. >> >> Right. For non-neccesary *feature additions* to an existing LTS. > > So, your position is based on the solid rock of a principle that is > entirely non-negotiable except for one thing: necessity, which > inevitably is a judgment call. Yes. gauging necessity is a judgement call. A hard one. One that is likely debatable on a case by case basis. And one that ultimately you, as the project lead, will have to make. Focusing on that "neccesary or not" judgment call and avoiding dealing with things that don't rise to the necessity bar level that *you* determine will make things simple. It will allows us to avoid the various "but this is low risk, and my company really wants it, and I know many people that would be happy to see it" discussions that seek to increase value as an argument for adding a feature in, and focus on a simple gate first. In the thinking process I am suggesting, value should only matter when a thing rises to a certain level of necessity. Above that level, Both risk and value matter. But below it, value should be ignored, and necessity should be the only thing being debated. You can certainly use some of the value-building arguments to argue for necessity as well, and that is a fine thing to do. It's just that "I really like it and would appreciate it being added" arguments would obviously not count. Let me show examples of things that could make sense to argue for new feature inclusion: - E.g. if we could show that without a new collector JDK 11 will show significant degradation from it's current behavior on some new, here, or obviously-coming hardware configurations, that could move this subject into the "gut wrenching" mode I talk about earlier in the thread. [Sort of where TLS 1.3 is for 8u. Sort of where my thinking about aarch64 for 8u is.] - E.g. if we identified some dramatic and degenerate performance issue that is popping up in some newly common field situations (e.g. logging, containers, cloud functions or KNative) and can be addressed technically, it may cross into necessity. - E.g. if, during the "still struggling with adoption" phase of the LTS we identified a performance issue that is a significant regression from the performance of existing code an earlier LTS, and is potentially hindering adoption as a result, and if that issue can be addressed technically, it may cross into necessity. [I think of 11u as still in that "still struggling with adoption" phase, and of e.g. the log4j2 performance regressions due to stack walking as a potential example, but I am hoping to see 11u move out of the mode where fixing such things would be accepted as necessity soon. We should hopefully not be fighting regresison based adoption-preventing bugs e.g. 3+ years past an LTS GA date, either because they are gone, or because adoption is good enough that we don't care enough to gauge them necessary to fix]. There are obviously lots of other examples possible for necessary feature inclusion. My point is that the argument for and against them should be framed around what makes them necessary, not what makes them valuable or low risk. Would anyone like to put forward arguments for what makes this change necessary? > > But this is not a principle that the project has ever followed > (AFAIK), so we'd have to decide to accept it in order for your > argument certainly to succeed. And right now I can't see any > convincing reason why we would want to bind ourselves in that way. > >>> That fits with my understanding of your position: no matter how >>> helpful a feature is, or how low the risk of adding it is, it should >>> not happen. So, your conclusion is *entirely independent* of the >>> actual ratio of risk to reward. No evaluation of either is necessary >>> because the conclusion will always be the same: no. >> >> Exactly. For non-neccesary *feature additions* to an existing LTS. >>> >>> I hope that you will agree that your position on this matter is an >>> extreme one; in fact is the most extreme position it is possible for >>> anyone to take. >> >> I disagree. I don't think this is an extreme position at all. In fact, I >> think it is quite mainstream, and represents the most common >> understanding of what stable releases are, and what most consumers >> of updates to such releases think is going on. > > It is extreme in the sense that it is hard to imagine a software > community accepting any more extreme position, even if that position > is a popular one. (It is surely possible for a majority to take an > extreme position!) There are multiple levels that would be "more extreme", and that some might still find reasonable. For example: - No feature additions of any kind. Regardless of necessity (e.g. not even a TLS 1.3 8u thing, no fixing to work on new hardware or OS versions, etc.) This is the "If you want that feature, no matter how necessary, move to a new version"). - Only include bug fixes that fix crashes with default flag settings. (e.g. don't fix bugs that crash with G1 on 8u, or with ParallelGC on 11u). Or more likely the same with a small set of "mainstream flags". (e.g. Obviously accept -Xloggc as a mainstream flag, accept several collectors as "will fix crashing bugs in", but don't do fixes for crashes under "uncommon combinations"). - Only include security fixes, and no other bug fixes, no matter how bad. (this is actually a common mode for the last legs in maintaining a project) - Only include security fixes for issues with a CVSS score above 7.0, and no other bug fixes, no matter how bad. Each of these level exists in various modes of maintenance for projects, and I am not advocating for any of them for 11u (and obviously not even for 8u, as I'm one of the advocates of TLS1.3 work there). Hence I claim that my position sits squarely in the moderate middle, with plenty of more extreme positions possible on either side of this. Those include the "don't fix non-security stuff" on one side and the "add features that would make people happier but are not otherwise necessary" on the other, and probably many other examples. > >> But I think that where you think the vast majority of the community >> sits on this position being "extreme" or the opposite being the >> extreme one depends on who you think the community is. >> >> This list tends to be dominated by developers of OpenJDK. >> >> I claim that our community (for the updates projects) is the *users* >> of OpenJDK. > > OK, but IMO the last thing this conversation needs is anyone claiming > to represent the "silent majority". I am not claiming to be their representative. I am pointing out that they exist, and are an important constituency that is affected by the decisions we make here. And I am making arguments that IMO would be made by users had they been active on the list. People can pick apart those arguments on their merits. The fact that my voice, at the point where I started making these "user centric" arguments, seemed to be alone on that side (with a near consensus building towards accepting such changes) does give me serious concern, but I am not looking to claim some position as a proxy for the masses here. I'm hoping to convince multiple others on this list to apply more of the "what would I think if I were running 11u LTS in production and this change came along" thinking in considering both this specific change, and the way we argue for or against such changes in general. > We the developers are what I mean > when I talk about the OpenJDK community, and the OpenJDK community of > developers is the consensus I seek. Consensus does not mean unanimity: > that would give any developer an effective veto. I don't have a veto, and I know it. I just have a sometimes loud voice. ? Gil. From yan at azul.com Thu Jul 2 09:38:05 2020 From: yan at azul.com (Yuri Nesterenko) Date: Thu, 2 Jul 2020 12:38:05 +0300 Subject: [13u] RFR 8238284: [macos] Zero VM build fails due to an obvious typo In-Reply-To: References: Message-ID: <26f67249-d192-6483-fb06-d0f0dd4d82c0@azul.com> OK, looks good to me. --yan On 30.06.2020 16:00, Vladimir Kempik wrote: > Hello > Please review this backport of JDK-8238284 to jdk13u > > The patch didn?t apply cleanly. Original 8238284 fixed two issues in jdk15: > JDK-8237437 - since jdk14 > JDK-8165929 - since jdk11 > > Jdk13 only has 8165929 so only part of original fix needs to be backported to jdk13, that part applies cleanly. > > This is part2 out of 3 fixes needed to fix zero vm on macos. > > Fix has also part for linux but it wasn?t affected in my testing. > > Risks are low because the patch just adds const keyword to some vars for zerovm > > The bug: https://bugs.openjdk.java.net/browse/JDK-8238284 > The webrev: http://cr.openjdk.java.net/~vkempik/8238284/webrev.13u.00/ > Original review https://mail.openjdk.java.net/pipermail/hotspot-dev/2020-January/040681.html > Other two mentioned bugs: https://bugs.openjdk.java.net/browse/JDK-8234737 , https://bugs.openjdk.java.net/browse/JDK-8165929 > > Thanks, Vladimir > From neugens at redhat.com Thu Jul 2 09:59:31 2020 From: neugens at redhat.com (Mario Torre) Date: Thu, 2 Jul 2020 11:59:31 +0200 Subject: ... In-Reply-To: <55b122c0-a92c-2adf-36e9-3a5c6faaf48a@oracle.com> References: <3595e963-fa5b-bfce-c463-841b84d4103f@redhat.com> <6468CB08-3DB5-488E-9122-E3699A95218B@amazon.com> <7FDB6E37-E850-49DF-8255-059E7F9771C5@azul.com> <1551271D-7BEB-4CE2-A432-A2C62A813143@azul.com> <8bccbed2-0d9a-b198-1366-4b66a2ca5446@redhat.com> <5DBDA26C-E160-4175-97F7-21CC7FEC0DD8@azul.com> <32219A90-E722-43DF-863B-3EF3084DE8F7@azul.com> <55b122c0-a92c-2adf-36e9-3a5c6faaf48a@oracle.com> Message-ID: On Wed, Jul 1, 2020 at 2:08 AM Claes Redestad wrote: > This huge risk, and the fragmentation it brings with it, IMHO means that > the JFR backport should have been considered unacceptable for 8u LTS. > Still should be. That's the only thing I think Gil is slightly wrong > about in this discussion. JFR in 8u was a [calculated] risk because of the level of changes and this is why it took almost one year, and it's still not really finished. But there's no fragmentation or incompatibility with Oracle JDK because the recordings can be read by the same tools and it doesn't matter what JDK produces them. There's no specification for the recording format on purpose but you have tools and they are available to all and they work with all the different flavours. The additional API that was introduced isn't used by legacy code that runs on 8u, so all existing code works as it did before. If anything, it offers a new migration path now. Everyone wins big time now. In general, I think we need to get over the mentality that it's ok to have downstream distributions of jdk8u differentiated by some key (or perceived key) features, perhaps closed and only available from one vendor. Historically, that is what has happened with jdk8u though and we now have to deal with the consequences. JFR support was developed in Oracle?s proprietary downstream of jdk8 and also added by Ali Baba and Azul to their proprietary releases. Shenandoah was developed in Red Hat?s downstream of jdk8 and only made its way upstream in a later release. Several other features were added in other vendor?s releases. Of course, features should be developed upstream first and there ought to be no desire to backport but because of history jdk8u /is/ now still feature-fragmented, with Shenandoah the main outstanding feature that still exists downstream. There is a gain as well as a risk from integrating downstream features, like JFR or Shenandoah, that make sense for a significant number of users, into the upstream jdk8u tree because this feature-fragmentation runs the risk of introducing more bugs and more variation in behaviour. This is what fragmentation means, and I'm glad Oracle was aware of this when you released JFR and other proprietary features into upstream OpenJDK, that was a huge achievement. OpenJDK is too important and cannot be an "open core" development model, and if some vendors still want to think this can be the case, they can't really complain about people stepping in and fixing the void. This is true for all projects that have a somewhat critical mass, btw, but OpenJDK is clearly the centerpiece of the industry today, we must protect it at all costs. I can understand that something isn't accepted on a technical ground, and not everything makes sense to have upstream, but we must be able to discuss and accept changes on something other than a purely theoretical perspective, otherwise we will always have the risk of a conflict of interest laying in the "niet"; for example, a downstream proprietary distribution with a special GC that covers the same areas as Shenandoah or a downstream implementation that offers a proprietary port of a Serviceability API etc... So far in this discussion I've not really heard any actual technical merit, which is what I expected to be discussed instead. Cheers, Mario -- Mario Torre Associate Manager, Software Engineering Red Hat GmbH 9704 A60C B4BE A8B8 0F30 9205 5D7E 4952 3F65 7898 From claes.redestad at oracle.com Thu Jul 2 11:01:42 2020 From: claes.redestad at oracle.com (Claes Redestad) Date: Thu, 2 Jul 2020 13:01:42 +0200 Subject: ... In-Reply-To: References: <3595e963-fa5b-bfce-c463-841b84d4103f@redhat.com> <6468CB08-3DB5-488E-9122-E3699A95218B@amazon.com> <7FDB6E37-E850-49DF-8255-059E7F9771C5@azul.com> <1551271D-7BEB-4CE2-A432-A2C62A813143@azul.com> <8bccbed2-0d9a-b198-1366-4b66a2ca5446@redhat.com> <5DBDA26C-E160-4175-97F7-21CC7FEC0DD8@azul.com> <32219A90-E722-43DF-863B-3EF3084DE8F7@azul.com> <55b122c0-a92c-2adf-36e9-3a5c6faaf48a@oracle.com> Message-ID: On 2020-07-02 11:59, Mario Torre wrote: > On Wed, Jul 1, 2020 at 2:08 AM Claes Redestad wrote: > >> This huge risk, and the fragmentation it brings with it, IMHO means that >> the JFR backport should have been considered unacceptable for 8u LTS. >> Still should be. That's the only thing I think Gil is slightly wrong >> about in this discussion. > > JFR in 8u was a [calculated] risk because of the level of changes and > this is why it took almost one year, and it's still not really > finished. But there's no fragmentation or incompatibility with Oracle > JDK because the recordings can be read by the same tools and it > doesn't matter what JDK produces them. There's no specification for > the recording format on purpose but you have tools and they are > available to all and they work with all the different flavours. The > additional API that was introduced isn't used by legacy code that runs > on 8u, so all existing code works as it did before. If anything, it > offers a new migration path now. Everyone wins big time now. First off: everything here is my personal opinions. There already *was* a migration path: upgrade to OpenJDK 11u. And no, just because the formats of the recordings are compatible (modulo the namespace of events), that doesn't mean that the features are interchangeable. That events have the same performance characteristics. That events are triggered for the same things. Etc. JFR cuts much deeper, and every analysis I've seen of the compatibility only goes skin deep, since you need access to the closed code to fully see the difference. I will always view the JFR backport as an unhealthy catering to the wants of users, not to what was ultimately best for them (and the ecosystem). No, it never was a necessity for 8u, unless necessity is defined as the "necessity to migrate off vendor x", which is a pretty shitty take to make by anyone in a community of users, developers *and* vendors. Either way Oracle is fully committed to supporting and maintaining JFR in the open in 11u and onwards (to the extent our backports are visible for the jdk-updates project - but that's another battle..) > > In general, I think we need to get over the mentality that it's ok to > have downstream distributions of jdk8u differentiated by some key (or > perceived key) features, perhaps closed and only available from one > vendor. Historically, that is what has happened with jdk8u though and > we now have to deal with the consequences. JFR support was developed > in Oracle?s proprietary downstream of jdk8 and also added by Ali Baba > and Azul to their proprietary releases. Shenandoah was developed in > Red Hat?s downstream of jdk8 and only made its way upstream in a later > release. Several other features were added in other vendor?s releases. Sure! And while Oracle's 8u is very "differentiated", it's also an artifact of history. We've since 11u committed fully to develop all features in the open and maintain/develop them in the open. Both since that saves time and resources, but also because it makes good business sense. What vendors do with open sourced features in their own private forks/repos is however (fortunately?) entirely up to them. We can only appeal to the moral argument that backporting features to older releases creates fragmentation and a race to the bottom that ultimately does our users a disservice for the benefit of some brand recognition or whatever. I speak personally, not for Oracle, when I say that I think all these feature backports are bad. > > Of course, features should be developed upstream first and there ought > to be no desire to backport but because of history jdk8u /is/ now > still feature-fragmented, with Shenandoah the main outstanding feature > that still exists downstream. There is a gain as well as a risk from > integrating downstream features, like JFR or Shenandoah, that make > sense for a significant number of users, into the upstream jdk8u tree > because this feature-fragmentation runs the risk of introducing more > bugs and more variation in behaviour. This is what fragmentation > means, and I'm glad Oracle was aware of this when you released JFR and > other proprietary features into upstream OpenJDK, that was a huge > achievement. > > OpenJDK is too important and cannot be an "open core" development > model, and if some vendors still want to think this can be the case, > they can't really complain about people stepping in and fixing the > void. This is true for all projects that have a somewhat critical > mass, btw, but OpenJDK is clearly the centerpiece of the industry > today, we must protect it at all costs. I don't think "open core" is morally bad, but I agree that there's ample evidence that roping off the "best" features stifles innovation and adoption. It also creates a nasty dynamic in the community where rather than cooperating on the shared feature set you'll see competitors spending most of their time reverse-engineer or trying to provide alternatives to those differentiating features, which gets messy fast. At least our team is not doing closed features any more, but I can't speak for Oracle strategy in neither the long nor short term. I hope we'll continue our OpenJDK strategy of full openness and cooperation on the mainline, and I personally hope we'll take steps to be more open and active in the updates projects, too. > > I can understand that something isn't accepted on a technical ground, > and not everything makes sense to have upstream, but we must be able > to discuss and accept changes on something other than a purely > theoretical perspective, otherwise we will always have the risk of a > conflict of interest laying in the "niet"; for example, a downstream > proprietary distribution with a special GC that covers the same areas > as Shenandoah or a downstream implementation that offers a proprietary > port of a Serviceability API etc... > > So far in this discussion I've not really heard any actual technical > merit, which is what I expected to be discussed instead. That it's bad to force the risk imposed by adding well-meaning features on people who expect only stability and security fixes is a technical argument. The argument is that the acceptable risk to impose on risk-averse people depending on a LTS for operations of stable systems should be "none", unless that risk is absolutely necessary to take for the continued operation of the service. The TLS 1.3 feature might be the only one I've seen in these discussions that I think meets that bar. And it's a good bar. /Claes > > Cheers, > Mario > > -- > Mario Torre > Associate Manager, Software Engineering > Red Hat GmbH > 9704 A60C B4BE A8B8 0F30 9205 5D7E 4952 3F65 7898 > From Martijn.Verburg at microsoft.com Thu Jul 2 12:32:22 2020 From: Martijn.Verburg at microsoft.com (Martijn Verburg) Date: Thu, 2 Jul 2020 12:32:22 +0000 Subject: RFR (11u, XXL): Upstream/backport Shenandoah to JDK11u In-Reply-To: References: <3595e963-fa5b-bfce-c463-841b84d4103f@redhat.com> <6468CB08-3DB5-488E-9122-E3699A95218B@amazon.com> <7FDB6E37-E850-49DF-8255-059E7F9771C5@azul.com> <1551271D-7BEB-4CE2-A432-A2C62A813143@azul.com> <8bccbed2-0d9a-b198-1366-4b66a2ca5446@redhat.com> <5DBDA26C-E160-4175-97F7-21CC7FEC0DD8@azul.com> , Message-ID: Hi all, Appreciate all the excellent points being made here on both sides. I'll snip some of the long thread and reply to some of Gil's points below which I think speaks to the necessity angle. ________________________________ > In the thinking process I am suggesting, the value should only matter > when a thing rises to a certain level of necessity. Above that level, > Both risk and value matter. But below it, the value should be ignored, > and necessity should be the only thing being debated. > You can certainly use some of the value-building arguments to > argue for necessity as well, and that is a fine thing to do. It's just that > "I really like it and would appreciate it being added" arguments > would obviously not count. Let me show examples of things that > could make sense to argue for new feature inclusion: > - E.g. if we could show that without a new collector JDK 11 > will show significant degradation from its current behavior > on some new, here, or obviously coming hardware configurations, > that could move this subject into the "gut-wrenching" mode I > talk about earlier in the thread. > [Sort of where TLS 1.3 is for 8u. Sort of where my thinking about > aarch64 for 8u is.] Putting on my Microsoft Hat - we have several 1st and 3rd party users that fall into this category. They're currently being forced to use an expensive architectural workaround or moving off the JVM altogether because there isn't a Garbage Collector in for Hotspot in Java 11 that can manage their mission-critical workloads. They also are unable to migrate to newer versions of Java because the libraries, frameworks, and tooling that they utilize have in a lot of cases only just managed to get to Java 11 compatibility and it may be several years before they see support for the tip (leading to 17). Now, this is clearly a single data point from a single cloud vendor, but this does represent many end users. As another data point, before being acquired by MSFT we also saw several customers at jClarity with similar challenges from about 2015 onwards. Indeed, we built a whole business around using tooling/ML to try and configure the existing 11 collectors for these users ??. There was a wide range of types of applications that this impacted, across all industry sectors. There was a not-insignificant cost to the customer for using a solution like ours, plus all the time and effort investigating and making changes. Sometimes it just simply wasn't possible and so we'd work with that customer to either re-architect their application (ouch!) or recommended a non-Hotspot JVM that could cope (Zing being the prominent one of those). > - E.g. if we identified some dramatic and degenerate > performance issue that is popping up in some newly common > field situations (e.g. logging, containers, cloud functions or KNative) > and can be addressed technically, it may cross into a necessity. Without going into customer specifics, we see two major use cases that we see as common which are causing serious pain for users today and have been for several years now. - One is the 'chain of microservices' pattern, where each individual JVM service needs to be tuned to have the smallest, less frequent pauses possible for the end to end transaction to meet its SLA. - The other is Big Data - we're seeing some stupendously sized heaps that need supporting, G1 is pretty darn good, but starts to struggle at that scale. > There are obviously lots of other examples possible for necessary > feature inclusion. My point is that the argument for and against them > should be framed around what makes them necessary, not what > makes them valuable or low risk. > Would anyone like to put forward arguments for what makes this > change necessary? Hopefully, a couple of data points I mentioned above helps. Again, it's just from my jClarity/MSFT experience so, YMMV, but I do feel they are industry trends that are not going away and so I believe this passes the necessity test. I'll add that I'm glad we can have these reasoned discussions in this community, it's a strength of OpenJDK that we do so. Cheers, Martijn From gil at azul.com Thu Jul 2 15:22:36 2020 From: gil at azul.com (Gil Tene) Date: Thu, 2 Jul 2020 15:22:36 +0000 Subject: ... In-Reply-To: References: <3595e963-fa5b-bfce-c463-841b84d4103f@redhat.com> <6468CB08-3DB5-488E-9122-E3699A95218B@amazon.com> <7FDB6E37-E850-49DF-8255-059E7F9771C5@azul.com> <1551271D-7BEB-4CE2-A432-A2C62A813143@azul.com> <8bccbed2-0d9a-b198-1366-4b66a2ca5446@redhat.com> <5DBDA26C-E160-4175-97F7-21CC7FEC0DD8@azul.com> <32219A90-E722-43DF-863B-3EF3084DE8F7@azul.com> <55b122c0-a92c-2adf-36e9-3a5c6faaf48a@oracle.com>, Message-ID: <5AA70DC6-F066-487B-AA1F-36EE441A71B7@azul.com> >> On Jul 2, 2020, at 3:00 AM, Mario Torre wrote: > ... > In general, I think we need to get over the mentality that it's ok to > have downstream distributions of jdk8u differentiated by some key (or > perceived key) features, perhaps closed and only available from one > vendor. Historically, that is what has happened with jdk8u though and > we now have to deal with the consequences. JFR support was developed > in Oracle?s proprietary downstream of jdk8 and also added by Ali Baba > and Azul to their proprietary releases. Shenandoah was developed in > Red Hat?s downstream of jdk8 and only made its way upstream in a later > release. Several other features were added in other vendor?s releases. > > Of course, features should be developed upstream first and there ought > to be no desire to backport but because of history jdk8u /is/ now > still feature-fragmented, with Shenandoah the main outstanding feature > that still exists downstream. There is a gain as well as a risk from > integrating downstream features, like JFR or Shenandoah, that make > sense for a significant number of users, into the upstream jdk8u tree > because this feature-fragmentation runs the risk of introducing more > bugs and more variation in behaviour. This is what fragmentation > means, and I'm glad Oracle was aware of this when you released JFR and > other proprietary features into upstream OpenJDK, that was a huge > achievement. > > OpenJDK is too important and cannot be an "open core" development > model, and if some vendors still want to think this can be the case, > they can't really complain about people stepping in and fixing the > void. This is true for all projects that have a somewhat critical > mass, btw, but OpenJDK is clearly the centerpiece of the industry > today, we must protect it at all costs. > > I can understand that something isn't accepted on a technical ground, > and not everything makes sense to have upstream, but we must be able > to discuss and accept changes on something other than a purely > theoretical perspective, otherwise we will always have the risk of a > conflict of interest laying in the "niet"; for example, a downstream > proprietary distribution with a special GC that covers the same areas > as Shenandoah or a downstream implementation that offers a proprietary > port of a Serviceability API etc... Mario, it is pretty rich for an IBM / Red Hat employee to be (repeatedly) using this nefarious ?conflict of interest? argument to question the motivation of people rather than arguing on the technical merits. Please stop. Zulu and DragonWell are no more proprietary than Red Hat?s builds of OpenJDK. Neither are Corretto, SapMachine, Liberica, and any other distro that uses the upstream OpenJDK projects (tip, feature releases, and update projects) as a place to come together, contribute, coordinate, align on common denominators, and share our work for good. The good people working on those distributions use the updates projects, specifically, to coordinate a common denominator for the long term maintenance of stable versions, including shared bug fixes and security updates in an upstream, with 11u being one such updates project location. People working on every one of the above named distros contribute actual code to the common updates projects. All in all, I?d say this coordination has been going pretty pretty well, and that it has managed to resist the commercial Interests of individual companies that project members happen to work for, or of the distros they work on in their day jobs, from seeking to shut out or invalidate the participation others or their distros. Last I looked, Red Hat has a common, long standing (and in no way nefarious) practice of adding custom and special features to their downstream, trademarked distributions of OSS software. These include special versions of Linux kernels 2.6.32 (RHEL 6.x), 3.10 (RHEL 7.x), and 4.18 (RHEL 8.x). It obviously includes the Red Hat builds of OpenJDK, which differ from the upstream in pretty significant ways (more so than most other distros listed above). I?m sure you don?t mean to be arguing that this is ?fragmentation? only when it is done by companies that are not your employer, or by distros other than the one you yourself work on. And I know you won?t find me claiming that repeatedly on this list. There are millions of production, non-RHEL users of Linux distros that did not include Red Hat special modifications to e.g. the 2.6.32 kernels (e.g. THP, included in RHEL6.x) or 3.10 kernels (e.g. memfd, included in RHEL 7.x). The features did appear on later-version upstream kernel.org kernels, but I assume nobody was arguing for forcing all non-RHEL production users of 2.6.32 or 3.10 kernels (or other stable kernels) to accept the inclusion of those new features with their security updates. Similarly, there are millions of production, non-Red-Hat-builds-of-OpenJDK users of OpenJDK 11u and 8u distros where Shenandoah does not exist, and neither do various other Red Hat distro-special changes. Similarly, Shenandoah as a feature is already in the later upstream OpenJDK versions (12, 13u, 14u), and is therefore part of downstream distros of those later versions. What we are discussing here is whether the existing users of all other 11u distros should be forced to accept it as a change with their coming security updates. Some users of some distros (including e.g. massive users of Zulu, like Microsoft, and of Corretto, like Amazon) are saying that they are fine with it, will welcome it (even if it came in an 11.0.9 update for example) and would make good use if it. Some users (including other massive users of Zulu, and of other distos) are saying the exact opposite, noting that they are already in production with e.g. 11.0.7 on stable, non-continuously-changing applications and services, will need 11.0.8, 11.0.9, ... for security updates, but would absolutely not want to see any features added in any latter 11.0.x LTS updates. For an updates project, this is what technical (and not at all theoretical) perspectives look like. A choice will need to be made for 11u. And it?s not a hard choice. It may be an annoying choice, and an example where the right choice is not to do that thing we really want to do. But such is life. To me, the choice seems obvious in an LTS updates project. We don?t have to like it. > So far in this discussion I've not really heard any actual technical > merit, which is what I expected to be discussed instead. > > Cheers, > Mario > > -- > Mario Torre > Associate Manager, Software Engineering > Red Hat GmbH > 9704 A60C B4BE A8B8 0F30 9205 5D7E 4952 3F65 7898 From sgehwolf at redhat.com Thu Jul 2 15:28:57 2020 From: sgehwolf at redhat.com (Severin Gehwolf) Date: Thu, 02 Jul 2020 17:28:57 +0200 Subject: [11u] RFR: backport JDK-8247246: Add explicit ResolvedJavaType.link and expose, presence of default methods In-Reply-To: References: <9fdcf8e5-1888-014e-2aaf-05688fee91fc@redhat.com> <0d26983c2aba501c995dc73cbd84de58023b34b6.camel@redhat.com> Message-ID: <16c83fe7bed8945f2a2526765b422ea8abad91ac.camel@redhat.com> Hi, On Wed, 2020-07-01 at 21:09 +0300, Foivos Zakkak wrote: > Hi Severin, > > Thanks for your review. > I am attaching a new patch without the copyright updates this time. Looks good to me. FWIW, I've uploaded a webrev of this patch here: http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8247246/01/webrev/ Thanks, Severin > Regards, > Foivos > > On 01/07/2020 19:27, Severin Gehwolf wrote: > > Hi Foivos, > > > > On Wed, 2020-07-01 at 15:12 +0300, Foivos Zakkak wrote: > > > Hello all, > > > > > > I am attaching a backport > > > https://bugs.openjdk.java.net/browse/JDK-8247246 to jdk11u-dev. > > For reference, this is the original change in JDK 15: > > https://hg.openjdk.java.net/jdk/jdk15/rev/b58fc6058055 > > > > > This backport fixes https://github.com/oracle/graal/issues/2628 > > > > > > The backport applied cleanly except for some minor issues which I > > > resolved by changing: > > > > > > JVMCI_THROW(NullPointerException); > > > to > > > THROW(vmSymbols::java_lang_NullPointerException()); > > > > > > and > > > > > > Klass* klass = JVMCIENV->asKlass(jvmci_type); > > > to > > > Klass* klass = CompilerToVM::asKlass(jvmci_type); > > > > > > > > > The backport has been successfully tested using: > > > > > > make run-test TEST="hotspot/jtreg/compiler/jvmci" > > > make run-test-tier1 > > > > > > I have also successfully built graal master against the patched JDK > > src/hotspot/share/jvmci/jvmciCompilerToVM.cpp > > > > The original change didn't include a copyright update. So should the > > backport. > > > > This actually applies to all files touched. > > > > Thanks, > > Severin > > > > > Thanks, > > > Foivos Zakkak > > > > > > PS: Pretty happy to submit my first contribution to OpenJDK > > > From neugens at redhat.com Thu Jul 2 16:30:18 2020 From: neugens at redhat.com (Mario Torre) Date: Thu, 2 Jul 2020 18:30:18 +0200 Subject: ... In-Reply-To: <5AA70DC6-F066-487B-AA1F-36EE441A71B7@azul.com> References: <3595e963-fa5b-bfce-c463-841b84d4103f@redhat.com> <6468CB08-3DB5-488E-9122-E3699A95218B@amazon.com> <7FDB6E37-E850-49DF-8255-059E7F9771C5@azul.com> <1551271D-7BEB-4CE2-A432-A2C62A813143@azul.com> <8bccbed2-0d9a-b198-1366-4b66a2ca5446@redhat.com> <5DBDA26C-E160-4175-97F7-21CC7FEC0DD8@azul.com> <32219A90-E722-43DF-863B-3EF3084DE8F7@azul.com> <55b122c0-a92c-2adf-36e9-3a5c6faaf48a@oracle.com> <5AA70DC6-F066-487B-AA1F-36EE441A71B7@azul.com> Message-ID: On Thu, Jul 2, 2020 at 5:22 PM Gil Tene wrote: > Mario, it is pretty rich for an IBM / Red Hat employee to be (repeatedly) > using this nefarious ?conflict of interest? argument to question the > motivation of people rather than arguing on the technical merits. > > Please stop. Gil, I'm sorry from all the things I wrote you picked this one line, my intention is to explain why the discussion needs to steer on the technical matter and not imply any one is promoting a self interest, I hope you realise this. Btw, I'm not sure who else said this "repeatedly"? Anyway. > And I know you won?t find me claiming that repeatedly on this list. It wouldn't be appropriate, not just because all of our code is open sourced and anyone can use it in their downstreams (and people do, it's the case, for example, in CentOS or Fedora and other commercial derivatives), but also because we are talking about OpenJDK here. > For an updates project, this is what technical (and not at all theoretical) > perspectives look like. You bring the experience of one group and claim it's universally valid across all groups of users. We will never make everyone happy, but we can bring the reasoning on the specific merit of the patch instead of universally rule things out just because one player of this community doesn't accept it. You can still disable Shenandoah in your build setup and will be like it was never there in the first place. After all, OpenJDK doesn't really do binary distributions, so how a user consumes the JDK is really via downstream vendors. Cheers, Mario -- Mario Torre Associate Manager, Software Engineering Red Hat GmbH 9704 A60C B4BE A8B8 0F30 9205 5D7E 4952 3F65 7898 From rkennke at redhat.com Thu Jul 2 19:36:50 2020 From: rkennke at redhat.com (Roman Kennke) Date: Thu, 02 Jul 2020 21:36:50 +0200 Subject: RFR (11u, XXL): Upstream/backport Shenandoah to JDK11u In-Reply-To: References: <3595e963-fa5b-bfce-c463-841b84d4103f@redhat.com> <6468CB08-3DB5-488E-9122-E3699A95218B@amazon.com> <7FDB6E37-E850-49DF-8255-059E7F9771C5@azul.com> <1551271D-7BEB-4CE2-A432-A2C62A813143@azul.com> <8bccbed2-0d9a-b198-1366-4b66a2ca5446@redhat.com> <5DBDA26C-E160-4175-97F7-21CC7FEC0DD8@azul.com> Message-ID: <3100b3953af89ffbcd103e13372207fffc6be219.camel@redhat.com> > Would anyone like to put forward arguments for what makes this > change necessary? Let me try. While Shenandoah GC is certainly not your usual mainstream garbage collector, it has proven to be very useful for certain use-cases: - It maintains consistently low pause times, even with large heaps - It works very well in resource-restricted environments, e.g. containers or slow hardware (e.g. embedded systems), again with good latency/low pause times (part of that story is compressed-oops support) - It is maintained in 11u, which means it receives bug-fixes, and occasional improvements where they make sense (i.e. don't touch non- Shenandoah code) To my knowledge, no other GC in 11u fills that niche. It doesn't seem to be a small and esoteric niche that almost nobody would use either: the whole reason why we're offering this for inclusion in JDK11u is that other major contributors/vendors have asked for it. Originally that was Amazon and Azul, and recently Microsoft and Alibaba have voiced their interest too. Martin has sent some more details about Microsoft's use-case for it, I guess the other vendors would have similar stories to tell. My point of view is that it would be in the best interest of JDK11u if all downstreams would build on the same source base. It seems more useful if all parties work together on the same 11u mainline tree, instead of half of the contributors work together on some shared downstream (e.g. jdk11u-plus or even shenandoah/jdk11), and the other half on the 'real thing'. I am not sure if such fragmentation would serve us better overall. I wouldn't think so. In summary, my points of necessity are: - Shenandoah fills a void in GC space: large heaps and container/resource-restricted environments, while maintaining great latencies - Avoidance of fragmentation/unnecessary downstream forks Thanks, Roman From goetz.lindenmaier at sap.com Thu Jul 2 20:35:49 2020 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Thu, 2 Jul 2020 20:35:49 +0000 Subject: RFR (11u, XXL): Upstream/backport Shenandoah to JDK11u In-Reply-To: References: <3595e963-fa5b-bfce-c463-841b84d4103f@redhat.com> Message-ID: Hi Ron > > We even delivered Java 4-7 with hotspot of 8, > > without breaking our huge software stack > > ... > That was when the number of people working on the current JDK with > the new VM features was >10x the number of people working on > Updates today. It?s hard to compare the old model wit the new. We were roughly 15 people working on this. > > 3. My experience is that it is much harder to > > move software to new Java versions than to > > bring a feature to an older VM. > > Again, this should not be the case with the *new* release model > unless 1. the application suffers from severe technical debt and > relies on deprectated/unmaintained/internal code or 2. something > is wrong or unusual about the new release. In other words, this > shouldn?t be the case unless there is something wrong with your > app or with the JDK. > > Under the new release model, reasonably well-maintained applications > should normally take very little effort to update to a new feature > release. In the new release model, the feature releases are outdated before anybody started to do anything. Lots of software still runs on 5, 6 etc! 8 was extended until 2027 because people don't want to move. But yes, for an app that is released every two weeks to the cloud, and built on 11 or more recent, according to modern software architecture principles, this works, and actually happens. I just wanted to say that for us it was a success story to bring compressed oops, metaspace etc. to SAP JVM 4-8, and it worked, and it didn't cause tremendous support, it rather reduced the support. Best regards, Goetz. > > We have lots of software > > running where the development teams are almost > > gone. But the maintainers of the VM on SAP side, > > and the operators of the software at customer side, > > are still there. These are the ones that have to deal > > with the bugs introduced by the downport. These are also > > the ones whose problems are solved by a better GC. > > The application developers would have to deal with > > bringing the software to new Java versions, but they > > are no more available. > > > > Naturally, this does not hold for features that > > break compatibility of the libraries or the language. > > > > But please configure the build so that Shenandoah is > > not built per default. See make/autoconf/hotspot.m4. > > > > Shenandoah could be enabled later, but at first I would > > prefer if it is not compiled into everybodys binary. > > > > Best regards, > > Goetz. > > > > PS: I hope this mail gets delivered to the list, a row > > of my mails today didn't make it there. > > > > > > > > > > > > > > > -----Original Message----- > > > From: jdk-updates-dev On Behalf > > > Of Roman Kennke > > > Sent: Thursday, June 25, 2020 1:16 PM > > > To: jdk-updates-dev at openjdk.java.net > > > Subject: Re: RFR (11u, XXL): Upstream/backport Shenandoah to JDK11u > > > > > > I would like to revive this RFR, I believe we haven't come to a > > > conclusion yet. I still think it would be very useful to have the > > > Shenandoah GC included in jdk11u upstream. I have heard from several > > > vendors who would like to ship Shenandoah, but maintaining the rather > > > huge Shenandoah backport patch is an absolute no-go. > > > > > > This is an updated patch that includes numerous bugfixes and > > > improvements in Shenandoah. It is what Red Hat is going to ship in > > > their 11.0.8 release. > > > > > > Full webrev: > > > https://cr.openjdk.java.net/~rkennke/shenandoah-jdk11u- > > > upstream/webrev.04-all/ > > > > > > Webrev only for shared-code changes: > > > https://cr.openjdk.java.net/~rkennke/shenandoah-jdk11u- > > > upstream/webrev.04-shared/ > > > > > > Please let me know what you think. > > > > > > Roman > > > > > > > On Fri, 2020-01-17 at 18:05 +0100, Roman Kennke wrote: > > > > Hello, > > > > > > > > The Shenandoah GC has been integrated into jdk12 a little over a year > > > > ago: > > > > > > > > http://hg.openjdk.java.net/jdk/jdk/rev/9c18c9d839d3 > > > > > > > > The Shenandoah team has been maintaining the Shenandoah-enabled > > > > jdk11u > > > > downstream repository since the inception of jdk11 under: > > > > > > > > http://hg.openjdk.java.net/shenandoah/jdk11 > > > > > > > > In order to make it more useful and accessible for users, we would > > > > like > > > > to make it available in upstream jdk11u. The Shenandoah team at Red > > > > Hat > > > > intends to maintain it the same way it had been maintained in > > > > shenandoah/jdk11, in the course of regular jdk11u maintenance Red > Hat > > > > already does. > > > > > > > > Thanks to recent changes in Shenandoah over the last year, we are now > > > > able to fit the GC interface in jdk11 almost exactly. There are a few > > > > exceptions though. We tried to follow the following pattern for all > > > > shared-code changes where it made sense: > > > > > > > > #if INCLUDE_SHENANDOAH_GC > > > > if (UseShenandoahGC) { > > > > ... > > > > } > > > > #endif > > > > > > > > This ensures that the Shenandoah-specific changes are cut out of the > > > > build when building without Shenandoah, and avoid those code paths > at > > > > runtime when running without Shenandoah. > > > > > > > > Also, there are a few places where this was not possible or where it > > > > didn't seem feasible, but we tried to keep them few and obvious. > > > > Whenever this is the case, we are mirroring as close as possible what > > > > we > > > > have in jdk/jdk. > > > > > > > > Architecture-wise, Shenandoah runs on x86_64, aarch64, x86_32. Other > > > > architectures still build fine, and Shenandoah gets automatically > > > > disabled during configure if not supported. OS-wise, Shenandoah runs > > > > on > > > > Linux, Windows, and is known to run on MacOS X and Solaris too. > > > > > > > > I wasn't sure which form this should take. I decided to start with > > > > the > > > > easiest possible form this could take: a simple patch/webrev and an > > > > RFR. > > > > Let me know if you need a JIRA issue or any other formalities to > > > > track that. > > > > > > > > Full webrev: > > > > https://cr.openjdk.java.net/~rkennke/shenandoah-jdk11u- > > > upstream/webrev.02-all/ > > > > > > > > Webrev only for shared-code changes: > > > > https://cr.openjdk.java.net/~rkennke/shenandoah-jdk11u- > > > upstream/webrev.02-shared/ > > > > > > > > The webrevs are based on and depend on the arraycopy patch > proposed > > > > for > > > > review here: > > > > > > > > https://mail.openjdk.java.net/pipermail/jdk-updates-dev/2020- > > > January/002396.html > > > > > > > > Testing: > > > > The code has been tested many many times, continuously while > > > > maintaining > > > > it. It has also served as the basis for RPMs and shipped to > > > > customers. > > > > We are running all sorts of tests of the hotspot/jtreg testsuite, in > > > > particular hotspot_gc_shenandoah, CTW tests, the regular tier* tests, > > > > several popular benchmarks (specjvm, specjbb on a regular basis), on > > > > a > > > > nightly basis. > > > > > > > > Please let me know what you think. > > > > > > > > Thanks, > > > > Roman > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > From Martijn.Verburg at microsoft.com Thu Jul 2 21:13:27 2020 From: Martijn.Verburg at microsoft.com (Martijn Verburg) Date: Thu, 2 Jul 2020 21:13:27 +0000 Subject: RFR (11u, XXL): Upstream/backport Shenandoah to JDK11u In-Reply-To: References: <3595e963-fa5b-bfce-c463-841b84d4103f@redhat.com> <6468CB08-3DB5-488E-9122-E3699A95218B@amazon.com> <7FDB6E37-E850-49DF-8255-059E7F9771C5@azul.com> <1551271D-7BEB-4CE2-A432-A2C62A813143@azul.com> <8bccbed2-0d9a-b198-1366-4b66a2ca5446@redhat.com> <5DBDA26C-E160-4175-97F7-21CC7FEC0DD8@azul.com> , , Message-ID: Cheers, Martijn ________________________________ From: jdk-updates-dev on behalf of Martijn Verburg Sent: Thursday, July 2, 2020 13:32 To: Gil Tene ; Andrew Haley Cc: Roman Kennke ; Bernd Mathiske ; jdk-updates-dev at openjdk.java.net ; Nilsen, Kelvin ; Jiva, Azeem Subject: Re: RFR (11u, XXL): Upstream/backport Shenandoah to JDK11u Hi all, Appreciate all the excellent points being made here on both sides. I'll snip some of the long thread and reply to some of Gil's points below which I think speaks to the necessity angle. ________________________________ > - E.g. if we could show that without a new collector JDK 11 > will show significant degradation from its current behavior > on some new, here, or obviously coming hardware configurations, > that could move this subject into the "gut-wrenching" mode I > talk about earlier in the thread. > [Sort of where TLS 1.3 is for 8u. Sort of where my thinking about > aarch64 for 8u is.] >> Putting on my Microsoft Hat - we have several 1st and 3rd party users that fall into this category. >>They're currently being forced to use an expensive architectural workaround or moving off the JVM altogether because >>there isn't a Garbage Collector in for Hotspot in Java 11 that can manage their mission-critical workloads. I'd like to offer a clarification on the above. There are of course GC's available for many Mission Critical workloads. However, there are a subset of those workloads where a low pause collector such as Shenandoah or ZGC are required to meet SLAs. I'd also like to acknowledge that ZGC does of course exist on 11 and we continue to evaluate that as well (and on tip). However, my understanding is that the lead maintainer (Oracle) is focused on tip (which is totally understandable!) and so maintenance on 11 does become more challenging. Cheers, Martijn From ron.pressler at oracle.com Thu Jul 2 23:11:39 2020 From: ron.pressler at oracle.com (Ron Pressler) Date: Fri, 3 Jul 2020 00:11:39 +0100 Subject: RFR (11u, XXL): Upstream/backport Shenandoah to JDK11u In-Reply-To: References: <3595e963-fa5b-bfce-c463-841b84d4103f@redhat.com> <6468CB08-3DB5-488E-9122-E3699A95218B@amazon.com> <7FDB6E37-E850-49DF-8255-059E7F9771C5@azul.com> <1551271D-7BEB-4CE2-A432-A2C62A813143@azul.com> <8bccbed2-0d9a-b198-1366-4b66a2ca5446@redhat.com> <5DBDA26C-E160-4175-97F7-21CC7FEC0DD8@azul.com> Message-ID: There is no way moving off of Java isn?t at least 10x more costly than updating? to JDK 15. Alternatively, there?s no way it?s cheaper than paying someone to? offset the risk of new features with a support contract. Again, without commenting on this particular decision (I don?t know the details? of the Shenandoah patch), there is an easy, cheap solution for those who want? new features. I think that much of the problem is a lack of education about how? easy it is in most cases to run on the current JDK and that it is very often, the? easiest, cheapest and safest approach. ? Ron On 2 July 2020 at 22:14:04, Martijn Verburg (martijn.verburg at microsoft.com) wrote: Cheers, Martijn ________________________________ From: jdk-updates-dev on behalf of Martijn Verburg Sent: Thursday, July 2, 2020 13:32 To: Gil Tene ; Andrew Haley Cc: Roman Kennke ; Bernd Mathiske ; jdk-updates-dev at openjdk.java.net ; Nilsen, Kelvin ; Jiva, Azeem Subject: Re: RFR (11u, XXL): Upstream/backport Shenandoah to JDK11u Hi all, Appreciate all the excellent points being made here on both sides. I'll snip some of the long thread and reply to some of Gil's points below which I think speaks to the necessity angle. ________________________________ > - E.g. if we could show that without a new collector JDK 11 > will show significant degradation from its current behavior > on some new, here, or obviously coming hardware configurations, > that could move this subject into the "gut-wrenching" mode I > talk about earlier in the thread. > [Sort of where TLS 1.3 is for 8u. Sort of where my thinking about > aarch64 for 8u is.] >> Putting on my Microsoft Hat - we have several 1st and 3rd party users that fall into this category. >>They're currently being forced to use an expensive architectural workaround or moving off the JVM altogether because >>there isn't a Garbage Collector in for Hotspot in Java 11 that can manage their mission-critical workloads. I'd like to offer a clarification on the above. There are of course GC's available for many Mission Critical workloads. However, there are a subset of those workloads where a low pause collector such as Shenandoah or ZGC are required to meet SLAs. I'd also like to acknowledge that ZGC does of course exist on 11 and we continue to evaluate that as well (and on tip). However, my understanding is that the lead maintainer (Oracle) is focused on tip (which is totally understandable!) and so maintenance on 11 does become more challenging. Cheers, Martijn From ron.pressler at oracle.com Thu Jul 2 23:18:54 2020 From: ron.pressler at oracle.com (Ron Pressler) Date: Fri, 3 Jul 2020 00:18:54 +0100 Subject: RFR (11u, XXL): Upstream/backport Shenandoah to JDK11u In-Reply-To: References: <3595e963-fa5b-bfce-c463-841b84d4103f@redhat.com> Message-ID: Feature releases were always outdated quickly even back when they were called ?limited update? releases. We can?t compare them to the old major? releases because they are not the same at all. When 8u20 came out there were >100 developers working on 8u40 and zero people working on 8. Now, when 11 comes out, hundreds of people work on 12, and a few stay with 11. Comparing either 11u or 15 to a major release like 8 doesn?t make sense, each for a different reason. The new model is just incomparable to the old one. ? Ron On 2 July 2020 at 21:35:56, Lindenmaier, Goetz (goetz.lindenmaier at sap.com) wrote: Hi Ron > > We even delivered Java 4-7 with hotspot of 8, > > without breaking our huge software stack > > ... > That was when the number of people working on the current JDK with > the new VM features was >10x the number of people working on > Updates today. It?s hard to compare the old model wit the new. We were roughly 15 people working on this. > > 3. My experience is that it is much harder to > > move software to new Java versions than to > > bring a feature to an older VM. > > Again, this should not be the case with the *new* release model > unless 1. the application suffers from severe technical debt and > relies on deprectated/unmaintained/internal code or 2. something > is wrong or unusual about the new release. In other words, this > shouldn?t be the case unless there is something wrong with your > app or with the JDK. > > Under the new release model, reasonably well-maintained applications > should normally take very little effort to update to a new feature > release. In the new release model, the feature releases are outdated before anybody started to do anything. Lots of software still runs on 5, 6 etc! 8 was extended until 2027 because people don't want to move. But yes, for an app that is released every two weeks to the cloud, and built on 11 or more recent, according to modern software architecture principles, this works, and actually happens. I just wanted to say that for us it was a success story to bring compressed oops, metaspace etc. to SAP JVM 4-8, and it worked, and it didn't cause tremendous support, it rather reduced the support. Best regards, Goetz. > > We have lots of software > > running where the development teams are almost > > gone. But the maintainers of the VM on SAP side, > > and the operators of the software at customer side, > > are still there. These are the ones that have to deal > > with the bugs introduced by the downport. These are also > > the ones whose problems are solved by a better GC. > > The application developers would have to deal with > > bringing the software to new Java versions, but they > > are no more available. > > > > Naturally, this does not hold for features that > > break compatibility of the libraries or the language. > > > > But please configure the build so that Shenandoah is > > not built per default. See make/autoconf/hotspot.m4. > > > > Shenandoah could be enabled later, but at first I would > > prefer if it is not compiled into everybodys binary. > > > > Best regards, > > Goetz. > > > > PS: I hope this mail gets delivered to the list, a row > > of my mails today didn't make it there. > > > > > > > > > > > > > > > -----Original Message----- > > > From: jdk-updates-dev On Behalf > > > Of Roman Kennke > > > Sent: Thursday, June 25, 2020 1:16 PM > > > To: jdk-updates-dev at openjdk.java.net > > > Subject: Re: RFR (11u, XXL): Upstream/backport Shenandoah to JDK11u > > > > > > I would like to revive this RFR, I believe we haven't come to a > > > conclusion yet. I still think it would be very useful to have the > > > Shenandoah GC included in jdk11u upstream. I have heard from several > > > vendors who would like to ship Shenandoah, but maintaining the rather > > > huge Shenandoah backport patch is an absolute no-go. > > > > > > This is an updated patch that includes numerous bugfixes and > > > improvements in Shenandoah. It is what Red Hat is going to ship in > > > their 11.0.8 release. > > > > > > Full webrev: > > > https://cr.openjdk.java.net/~rkennke/shenandoah-jdk11u- > > > upstream/webrev.04-all/ > > > > > > Webrev only for shared-code changes: > > > https://cr.openjdk.java.net/~rkennke/shenandoah-jdk11u- > > > upstream/webrev.04-shared/ > > > > > > Please let me know what you think. > > > > > > Roman > > > > > > > On Fri, 2020-01-17 at 18:05 +0100, Roman Kennke wrote: > > > > Hello, > > > > > > > > The Shenandoah GC has been integrated into jdk12 a little over a year > > > > ago: > > > > > > > > http://hg.openjdk.java.net/jdk/jdk/rev/9c18c9d839d3 > > > > > > > > The Shenandoah team has been maintaining the Shenandoah-enabled > > > > jdk11u > > > > downstream repository since the inception of jdk11 under: > > > > > > > > http://hg.openjdk.java.net/shenandoah/jdk11 > > > > > > > > In order to make it more useful and accessible for users, we would > > > > like > > > > to make it available in upstream jdk11u. The Shenandoah team at Red > > > > Hat > > > > intends to maintain it the same way it had been maintained in > > > > shenandoah/jdk11, in the course of regular jdk11u maintenance Red > Hat > > > > already does. > > > > > > > > Thanks to recent changes in Shenandoah over the last year, we are now > > > > able to fit the GC interface in jdk11 almost exactly. There are a few > > > > exceptions though. We tried to follow the following pattern for all > > > > shared-code changes where it made sense: > > > > > > > > #if INCLUDE_SHENANDOAH_GC > > > > if (UseShenandoahGC) { > > > > ... > > > > } > > > > #endif > > > > > > > > This ensures that the Shenandoah-specific changes are cut out of the > > > > build when building without Shenandoah, and avoid those code paths > at > > > > runtime when running without Shenandoah. > > > > > > > > Also, there are a few places where this was not possible or where it > > > > didn't seem feasible, but we tried to keep them few and obvious. > > > > Whenever this is the case, we are mirroring as close as possible what > > > > we > > > > have in jdk/jdk. > > > > > > > > Architecture-wise, Shenandoah runs on x86_64, aarch64, x86_32. Other > > > > architectures still build fine, and Shenandoah gets automatically > > > > disabled during configure if not supported. OS-wise, Shenandoah runs > > > > on > > > > Linux, Windows, and is known to run on MacOS X and Solaris too. > > > > > > > > I wasn't sure which form this should take. I decided to start with > > > > the > > > > easiest possible form this could take: a simple patch/webrev and an > > > > RFR. > > > > Let me know if you need a JIRA issue or any other formalities to > > > > track that. > > > > > > > > Full webrev: > > > > https://cr.openjdk.java.net/~rkennke/shenandoah-jdk11u- > > > upstream/webrev.02-all/ > > > > > > > > Webrev only for shared-code changes: > > > > https://cr.openjdk.java.net/~rkennke/shenandoah-jdk11u- > > > upstream/webrev.02-shared/ > > > > > > > > The webrevs are based on and depend on the arraycopy patch > proposed > > > > for > > > > review here: > > > > > > > > https://mail.openjdk.java.net/pipermail/jdk-updates-dev/2020- > > > January/002396.html > > > > > > > > Testing: > > > > The code has been tested many many times, continuously while > > > > maintaining > > > > it. It has also served as the basis for RPMs and shipped to > > > > customers. > > > > We are running all sorts of tests of the hotspot/jtreg testsuite, in > > > > particular hotspot_gc_shenandoah, CTW tests, the regular tier* tests, > > > > several popular benchmarks (specjvm, specjbb on a regular basis), on > > > > a > > > > nightly basis. > > > > > > > > Please let me know what you think. > > > > > > > > Thanks, > > > > Roman > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > From adinn at redhat.com Fri Jul 3 08:29:31 2020 From: adinn at redhat.com (Andrew Dinn) Date: Fri, 3 Jul 2020 09:29:31 +0100 Subject: RFR (11u, XXL): Upstream/backport Shenandoah to JDK11u In-Reply-To: <3100b3953af89ffbcd103e13372207fffc6be219.camel@redhat.com> References: <3595e963-fa5b-bfce-c463-841b84d4103f@redhat.com> <6468CB08-3DB5-488E-9122-E3699A95218B@amazon.com> <7FDB6E37-E850-49DF-8255-059E7F9771C5@azul.com> <1551271D-7BEB-4CE2-A432-A2C62A813143@azul.com> <8bccbed2-0d9a-b198-1366-4b66a2ca5446@redhat.com> <5DBDA26C-E160-4175-97F7-21CC7FEC0DD8@azul.com> <3100b3953af89ffbcd103e13372207fffc6be219.camel@redhat.com> Message-ID: On 02/07/2020 20:36, Roman Kennke wrote: > In summary, my points of necessity are: > - Shenandoah fills a void in GC space: large heaps and > container/resource-restricted environments, while maintaining great > latencies > - Avoidance of fragmentation/unnecessary downstream forks And that brings me to a critique of Gil's 'in principle' argument. Gil says we don't ever need to backport features, as opposed to critical missing functionality (like the TLS code) or security/bug fixes (as happens at every CPU). His reasoning is that there can be no risk from sticking with the status quo to weigh against the risk that might be introduced by pushing jdk11u into upstream i.e. his equation always favours a conservative approach. Well, I am afraid I see a risk here which does need to be balanced. It's a well-known and widespread experience that maintaining a code base offers a very real risk of injecting new defects when you try to fix old ones. The academic literature and associated industrial experience of defect injection derives from at least as early as the mid 80s (the work I am aware of from that period was by IBM, I believe ;-). So, what this means is that maintaining two jdk11u trees offers more opportunities for bug injection when backporting critical missing functionality or security/bug fixes than maintaining one consolidated tree. We have two bites of the cherry where we might crack a tooth on the pip. We at Red Hat (well, our Shenandoah team in particular) know this because we have been maintaining the two trees in question. Andrew Haley and I also encountered the same concerns maintaining the AArch64 tree before it was upstreamed -- and in that case we have direct experience of the before and after situation which tells us that the risk on the status quo side is very real. That experience means we can identify and quantify risk on both sides of this Shenandoah equation and we certainly know already that it is not a one way street. Now, I am not saying anything about which way the balance lies in this specific case. I am happy to trust the those who review the proposed patch to follow the arguments of the Shenandoah team and their own understanding of the code, assess the evidence and make a balanced judgement (I am confident we have a sound review practice in this project). What I do suggest is that the in principle argument is a red herring and that we always need to look at and consider carefully what risk is involved in the status quo vs the proposed new circumstance when deciding on a backport. regards, Andrew Dinn ----------- Red Hat Distinguished Engineer Red Hat UK Ltd Registered in England and Wales under Company Registration No. 03798903 Directors: Michael Cunningham, Michael ("Mike") O'Neill From adinn at redhat.com Fri Jul 3 08:37:01 2020 From: adinn at redhat.com (Andrew Dinn) Date: Fri, 3 Jul 2020 09:37:01 +0100 Subject: [11u] RFR: backport JDK-8247246: Add explicit ResolvedJavaType.link and expose, presence of default methods In-Reply-To: <16c83fe7bed8945f2a2526765b422ea8abad91ac.camel@redhat.com> References: <9fdcf8e5-1888-014e-2aaf-05688fee91fc@redhat.com> <0d26983c2aba501c995dc73cbd84de58023b34b6.camel@redhat.com> <16c83fe7bed8945f2a2526765b422ea8abad91ac.camel@redhat.com> Message-ID: Hi Foivos/Severin, On 02/07/2020 16:28, Severin Gehwolf wrote: > FWIW, I've uploaded a webrev of this patch here: > http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8247246/01/webrev/ Yes, that patch looks fine to me as well, Nice work, Foivos! regards, Andrew Dinn ----------- Red Hat Distinguished Engineer Red Hat UK Ltd Registered in England and Wales under Company Registration No. 03798903 Directors: Michael Cunningham, Michael ("Mike") O'Neill From gouessej at orange.fr Fri Jul 3 10:46:17 2020 From: gouessej at orange.fr (gouessej at orange.fr) Date: Fri, 3 Jul 2020 12:46:17 +0200 (CEST) Subject: "JDK-8203281 : [Windows] JComboBox change in ui when editor.setBorder() is called" not backported into OpenJDK 11 Message-ID: <49646522.2975.1593773177911.JavaMail.www@wwinf1g33> Hello The fix of the following bug affecting OpenJDK 11 which is a "long term support" version hasn't been backported into OpenJDK 11: https://bugs.java.com/bugdatabase/view_bug.do?bug_id=8203281 I use OpenJDK 11.0.7 (provided by AdoptOpenJDK). What is the plan? Of course, I don't reproduce this bug with OpenJDK 14.0.1 but it's not a long term support version. Keep up the good work. Best regards. From sgehwolf at redhat.com Fri Jul 3 12:23:32 2020 From: sgehwolf at redhat.com (Severin Gehwolf) Date: Fri, 03 Jul 2020 14:23:32 +0200 Subject: [11u] RFR: backport JDK-8247246: Add explicit ResolvedJavaType.link and expose, presence of default methods In-Reply-To: References: <9fdcf8e5-1888-014e-2aaf-05688fee91fc@redhat.com> <0d26983c2aba501c995dc73cbd84de58023b34b6.camel@redhat.com> <16c83fe7bed8945f2a2526765b422ea8abad91ac.camel@redhat.com> Message-ID: <822c5e96d729befeadd51da5ac1435578bbc87b5.camel@redhat.com> On Fri, 2020-07-03 at 09:37 +0100, Andrew Dinn wrote: > Hi Foivos/Severin, > > On 02/07/2020 16:28, Severin Gehwolf wrote: > > FWIW, I've uploaded a webrev of this patch here: > > http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8247246/01/webrev/ > Yes, that patch looks fine to me as well, Nice work, Foivos! Thanks! Since Foivos isn't OpenJDK author (yet), I've added a "Fix Request" comment and applied the jdk11u-fix-request label on behalf of him to ask for backport approval. Cheers, Severin From goetz.lindenmaier at sap.com Mon Jul 6 06:30:01 2020 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Mon, 6 Jul 2020 06:30:01 +0000 Subject: [ping] [11u] RFR: 8243029: Rewrite javax/net/ssl/compatibility/Compatibility.java with a flexible interop test framework Message-ID: Hi, Could someone please review? It's only a test change! Best regards, Goetz. > -----Original Message----- > From: jdk-updates-dev On > Behalf Of Lindenmaier, Goetz > Sent: Friday, June 19, 2020 3:13 PM > To: jdk-updates-dev at openjdk.java.net > Subject: [CAUTION] [11u] RFR: 8243029: Rewrite > javax/net/ssl/compatibility/Compatibility.java with a flexible interop test > framework > > I would like to downport this for parity with 11.0.9-oracle. > This change caused me some headache: > > First, there were some files to resolve: > > Cert.java could not be deleted because it differs in certificate > EC_RSA_PRIME256V1_SHA256 > > Compatibility.java could not be deleted because it differs in > test compile instructions. The test in 11 forces Java 1.6 > compatibility, while the test in the patch forces Java 1.7 > compatibility. > > UseCase.java could not be deleted because it differs in > one line in Object[][] PARAMS: > jdk11u-dev: > FULL_CASES ? CIPHER_SUITES : MANDATORY_CIPHER_SUITES, > original change: > FULL_CASES || FULL_CIPHER_SUITES ? CIPHER_SUITES : > MANDATORY_CIPHER_SUITES, > > test/lib/jdk/test/lib/security/CertUtils.java > patched fine after downporting "8228967: Trust/Key store and SSL context > utilities for tests" > as prerequisite. > > The test descriptions before required -source/target 1.6 and 1.7. > I thought about keeping it that way, but that is not possible > as the new code uses Java 8 features as Lambdas and Predicates. > So the tests now require 1.8. > > HrrTest.java still would not work. > I removed the test case for TLS_CHACHA20_POLY1305_SHA256 because it > only came in 12 with "8140466: ChaCha20 and Poly1305 TLS Cipher Suites" > > > README > The readme of jdk11 mentions jdk 6 to 10. The source patched > originally by the change mentioned jdk 7 to 10. > Because the new test files contain Java 8 coding, I adapted > the readme to mention 8 as oldest version. > > I ran the test as normal jtreg tests (testing with the VM built from > the current repo). You need to pass -m to jtreg as the tests are > marked as manual tests. > This works fine. > > I also tried to run the test as described in the readme. > This did not work. Among others, I get errors that certain > enums naming Suites are not known. I tried to run the > tests in jdk/jdk, there they do not compile. So I gave up on > this. > The readme anyways is inprecise and/or wrong, it mentions the > wrong test file to call, and omits that '-m' must be set. > > So please review: > http://cr.openjdk.java.net/~goetz/wr20/8243029-ssl_test_framework- > jdk11/01/ > > Bug: > https://bugs.openjdk.java.net/browse/JDK-8243029 > https://hg.openjdk.java.net/jdk/jdk/rev/b6b4506a7827 > > Best regards, > Goetz. > > > From goetz.lindenmaier at sap.com Mon Jul 6 08:38:05 2020 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Mon, 6 Jul 2020 08:38:05 +0000 Subject: [11u] RFR: 8234347: "Turkey" meta time zone does not generate composed localized names Message-ID: Hi, I am downporting this for parity with 11.0.9-oracle. I had to do a row of trivial resolves: http://cr.openjdk.java.net/~goetz/wr20/8234347-Turkey_meta_time-jdk11/01 make/CompileToolsJdk.gmk Copyright. make/CopyInterimCLDRConverter.gmk This file gets deleted. It differs in the Copyright and the call to MakeTargetDir in the original patch. file make/Main.gmk Differs in indentation make/jdk/src/classes/build/tools/cldrconverter/Bundle.java Only context differs of the third chunk src/java.base/share/classes/java/time/format/DateTimeFormatterBuilder.java Copyright. The modified tests pass, it all runs in our CI tonight. Please review. https://bugs.openjdk.java.net/browse/JDK-8234347 https://hg.openjdk.java.net/jdk/jdk/rev/10e939d362fc Best regards, Goetz. From goetz.lindenmaier at sap.com Mon Jul 6 09:01:05 2020 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Mon, 6 Jul 2020 09:01:05 +0000 Subject: "JDK-8203281 : [Windows] JComboBox change in ui when editor.setBorder() is called" not backported into OpenJDK 11 In-Reply-To: <49646522.2975.1593773177911.JavaMail.www@wwinf1g33> References: <49646522.2975.1593773177911.JavaMail.www@wwinf1g33> Message-ID: Hi gouessej, I'll bring this to jdk11u 11.0.9, which will be released in October. The real bug link is https://bugs.openjdk.java.net/browse/JDK-8203281. Here you can see where this got fixed, and also whether I push it to 11.0.9. Best regards, Goetz. > -----Original Message----- > From: jdk-updates-dev On Behalf > Of gouessej at orange.fr > Sent: Friday, July 3, 2020 12:46 PM > To: jdk-updates-dev at openjdk.java.net > Subject: "JDK-8203281 : [Windows] JComboBox change in ui when > editor.setBorder() is called" not backported into OpenJDK 11 > Importance: High > > Hello > > The fix of the following bug affecting OpenJDK 11 which is a "long term > support" version hasn't been backported into OpenJDK 11: > https://bugs.java.com/bugdatabase/view_bug.do?bug_id=8203281 > > I use OpenJDK 11.0.7 (provided by AdoptOpenJDK). What is the plan? Of > course, I don't reproduce this bug with OpenJDK 14.0.1 but it's not a long > term support version. > > Keep up the good work. Best regards. From goetz.lindenmaier at sap.com Mon Jul 6 10:17:06 2020 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Mon, 6 Jul 2020 10:17:06 +0000 Subject: RFR [jdk11]: 8212200: assert(on_stack()) failed when shared java.lang.object is redefined by JVMTI agent In-Reply-To: References: Message-ID: Hi Matthias, I had a look at your change. classFile.cpp: You skipped one chunk, as the fixed code is not in 11. OK. systemDictionary.hpp You removed comment " and a flag word // that makes some minor distinctions, like whether the klass // is preloaded, optional, release-specific, etc." I think this should be kept, as this is still relevant in 11. Your change does not touch this flag. Other changes are ok. heapShared.cpp Just indentation, ok. For the fixes to the test you included in your change. Please downport them as changes of its own, and push the three changes together. Best regards, Goetz. > -----Original Message----- > From: jdk-updates-dev On Behalf > Of Baesken, Matthias > Sent: Tuesday, June 30, 2020 9:36 AM > To: 'jdk-updates-dev at openjdk.java.net' > Subject: [CAUTION] RFR [jdk11]: 8212200: assert(on_stack()) failed when > shared java.lang.object is redefined by JVMTI agent > > Hello, please review the jdk11 backport of 8212200 . > > I had to do a few slight adjustments to adjust the src changes of 8212200 to > jdk11 . > > The test > ( test/hotspot/jtreg/runtime/SharedArchiveFile/serviceability/ReplaceCritical > Classes.java.) from the original jdk/jdk webrev of 8212200 did not work. > So I had to include adjustments from 8221918 and 8213275 to fix the test . > > > > Bug/webrev : > > https://bugs.openjdk.java.net/browse/JDK-8212200 > > http://cr.openjdk.java.net/~mbaesken/webrevs/8212200.0/ > > > Original jdk/jdk webrev : > > http://hg.openjdk.java.net/jdk/jdk/rev/625f6c742392 > > > Thanks, Matthias From hohensee at amazon.com Mon Jul 6 19:03:18 2020 From: hohensee at amazon.com (Hohensee, Paul) Date: Mon, 6 Jul 2020 19:03:18 +0000 Subject: [11u] RFR/A: 8231209: [REDO] JDK-8207266 ThreadMXBean::getThreadAllocatedBytes() can be quicker for self thread In-Reply-To: <09229907-C812-4974-BF5D-8619FEA083BC@amazon.com> References: <09229907-C812-4974-BF5D-8619FEA083BC@amazon.com> Message-ID: The CSRs have been approved, and the JBS issues tagged (8231209, 8231968 and the corresponding backport issues 8247806 and 8247809). Also, Oracle has now committed to backport these to their 11.0.9. Please approve. Thanks, Paul From: serviceability-dev on behalf of "Hohensee, Paul" Date: Thursday, June 18, 2020 at 12:29 PM To: "jdk-updates-dev at openjdk.java.net" Cc: serviceability-dev Subject: [11u] RFR/A: 8231209: [REDO] JDK-8207266 ThreadMXBean::getThreadAllocatedBytes() can be quicker for self thread This request is for a pair of backports. 8231209 is the first (and primary), 8231968 is a minor cleanup. There are CSR?s for both. The effect is to add a convenience method getCurrentThreadAllocatedBytes() to com.sun.management.ThreadMXBean that can be implemented more efficiently than the equivalent getThreadAllocatedBytes(long id), and to make the implementation of getThreadAllocatedBytes(long id) and getThreadAllocatedBytes(long[] id) more efficient. These methods are heavily used by heap profiling tools, including Amazon?s, and their efficiency is important to us. There is no effect on the TCK because com.sun.management is a platform-specific package. See the CSRs for more detail. The patches apply cleanly (in sequence) to 11u, but I?m posting a review/approval request because the backport CSRs need approval. Once the backport CSRs are reviewed, finalized, and approved, l can tag 8231209 and 8231968. Tested with hotspot/jtreg/vmTestbase/nsk/monitoring jdk/com/sun/management jdk/jdk/jfr/event/runtime The same tests pass/fail as with unpatched jdk11u-dev. JDK-8231209: [REDO] JDK-8207266 ThreadMXBean::getThreadAllocatedBytes() can be quicker for self thread Original RFE: https://bugs.openjdk.java.net/browse/JDK-8231209 Original Patch: https://hg.openjdk.java.net/jdk/jdk/rev/c29e49148be7 Original CSR: https://bugs.openjdk.java.net/browse/JDK-8231374 Original review thread: https://mail.openjdk.java.net/pipermail/serviceability-dev/2019-September/029208.html Backport RFE: https://bugs.openjdk.java.net/browse/JDK-8247806 Backport CSR: https://bugs.openjdk.java.net/browse/JDK-8247807 JDK-8231968: getCurrentThreadAllocatedBytes default implementation s/b getThreadAllocatedBytes Original RFE: https://bugs.openjdk.java.net/browse/JDK-8231968 Original Patch: https://hg.openjdk.java.net/jdk/jdk/rev/5bb426e9acc4 Original CSR: https://bugs.openjdk.java.net/browse/JDK-8232072 Original review thread: https://mail.openjdk.java.net/pipermail/serviceability-dev/2019-October/029659.html Backport RFE: https://bugs.openjdk.java.net/browse/JDK-8247809 Backport CSR: https://bugs.openjdk.java.net/browse/JDK-8247810 Thanks, Paul From matthias.baesken at sap.com Tue Jul 7 07:50:02 2020 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Tue, 7 Jul 2020 07:50:02 +0000 Subject: [11u] RFR: 8246203: Segmentation fault in verification due to stack overflow with -XX:+VerifyIterativeGVN Message-ID: >Hi, > >I would downport this for parity with 11.0.9-oracle. > >I had to edit another verify() call to make it compile, see phaseX.cpp. >http://cr.openjdk.java.net/~goetz/wr20/8246209-VerifyIterativeGVN-jdk11/01/ > >Please review. > >https://bugs.openjdk.java.net/browse/JDK-8246203 >https://hg.openjdk.java.net/jdk/jdk/rev/49fee4ea8073 > >Best regards, > Goetz. Hi Goetz, the backport looks good to me . Best regards, Matthias From matthias.baesken at sap.com Tue Jul 7 08:10:28 2020 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Tue, 7 Jul 2020 08:10:28 +0000 Subject: [11u] RFR: 8234347: "Turkey" meta time zone does not generate composed localized names Message-ID: Hi Goetz, the backport looks good to me . Thanks, Matthias ------------------------ Hi, I am downporting this for parity with 11.0.9-oracle. I had to do a row of trivial resolves: http://cr.openjdk.java.net/~goetz/wr20/8234347-Turkey_meta_time-jdk11/01 make/CompileToolsJdk.gmk Copyright. make/CopyInterimCLDRConverter.gmk This file gets deleted. It differs in the Copyright and the call to MakeTargetDir in the original patch. file make/Main.gmk Differs in indentation make/jdk/src/classes/build/tools/cldrconverter/Bundle.java Only context differs of the third chunk src/java.base/share/classes/java/time/format/DateTimeFormatterBuilder.java Copyright. The modified tests pass, it all runs in our CI tonight. Please review. https://bugs.openjdk.java.net/browse/JDK-8234347 https://hg.openjdk.java.net/jdk/jdk/rev/10e939d362fc Best regards, Goetz. From goetz.lindenmaier at sap.com Tue Jul 7 11:52:01 2020 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Tue, 7 Jul 2020 11:52:01 +0000 Subject: [11u] RFR: 8240169: javadoc fails to link to non-modular api docs Message-ID: Hi I am downporting this for parity with 11.0.9-oracle. http://cr.openjdk.java.net/~goetz/wr20/8240169-javadoc-jdk11/01/ I had to do a row of adaptions to this change. src/jdk.javadoc/share/classes/jdk/javadoc/internal/doclets/toolkit/util/Extern.java The code differs slightly because in 11, configuration.utils is dereferenced everywhere, but in 15 utils is saved in a field of the object. Further down, put() is used in 11 where putIfAbsent() is used in 15. Further, code is formatted differently. Another chunk uses resources.getText, where in 11 it is configuration.getText. In checkLinkCompatibility() I added {} so it looks similar to 15. Besides resolving, I had to add "import javax.tools.Diagnostic.Kind;" JavadocTester.java 15 prints the faulty html file if the test fails. This was introduced in "8210031: implementation for JVM Constants API" which is huge. I would like to include this printing in this change, as it helps a lot to understand the issue if the test fails. It helped me to find out where I had resolved wrong. test/langtools/jdk/javadoc/doclet/testClassCrossReferences/TestClassCrossReferences.java The method is marked public in 15, therefore the patch does not apply. Further, I had to adapt the templates matched. In 11, "?is-external=true" is printed to the urls and "externalLink" is "external-link" in 15. Also, the "Overrides" heading is formatted differently. TestLinkOptionWithModule.java This needed similar adaptions of patterns as TestClassCrossReferences.java. Please review. https://bugs.openjdk.java.net/browse/JDK-8240169 https://hg.openjdk.java.net/jdk/jdk/rev/3b557aef43c4 Best regards, Goetz. From matthias.baesken at sap.com Tue Jul 7 12:03:18 2020 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Tue, 7 Jul 2020 12:03:18 +0000 Subject: RFR [jdk11]: 8212200: assert(on_stack()) failed when shared java.lang.object is redefined by JVMTI agent In-Reply-To: References: Message-ID: Hi G?tz , >For the fixes to the test you included in your change. >Please downport them as changes of its own, and >push the three changes together. Okay I requested https://bugs.openjdk.java.net/browse/JDK-8221918 and added this review to the comments , hope that's fine with you. Best regards, Matthias ( and btw downporting all 3 as single patches generates additional effort + room for error . ) -----Original Message----- From: Lindenmaier, Goetz Sent: Montag, 6. Juli 2020 12:17 To: Baesken, Matthias ; 'jdk-updates-dev at openjdk.java.net' Subject: RE: RFR [jdk11]: 8212200: assert(on_stack()) failed when shared java.lang.object is redefined by JVMTI agent Hi Matthias, I had a look at your change. classFile.cpp: You skipped one chunk, as the fixed code is not in 11. OK. systemDictionary.hpp You removed comment " and a flag word // that makes some minor distinctions, like whether the klass // is preloaded, optional, release-specific, etc." I think this should be kept, as this is still relevant in 11. Your change does not touch this flag. Other changes are ok. heapShared.cpp Just indentation, ok. For the fixes to the test you included in your change. Please downport them as changes of its own, and push the three changes together. Best regards, Goetz. > -----Original Message----- > From: jdk-updates-dev On Behalf > Of Baesken, Matthias > Sent: Tuesday, June 30, 2020 9:36 AM > To: 'jdk-updates-dev at openjdk.java.net' > Subject: [CAUTION] RFR [jdk11]: 8212200: assert(on_stack()) failed when > shared java.lang.object is redefined by JVMTI agent > > Hello, please review the jdk11 backport of 8212200 . > > I had to do a few slight adjustments to adjust the src changes of 8212200 to > jdk11 . > > The test > ( test/hotspot/jtreg/runtime/SharedArchiveFile/serviceability/ReplaceCritical > Classes.java.) from the original jdk/jdk webrev of 8212200 did not work. > So I had to include adjustments from 8221918 and 8213275 to fix the test . > > > > Bug/webrev : > > https://bugs.openjdk.java.net/browse/JDK-8212200 > > http://cr.openjdk.java.net/~mbaesken/webrevs/8212200.0/ > > > Original jdk/jdk webrev : > > http://hg.openjdk.java.net/jdk/jdk/rev/625f6c742392 > > > Thanks, Matthias From goetz.lindenmaier at sap.com Tue Jul 7 12:42:12 2020 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Tue, 7 Jul 2020 12:42:12 +0000 Subject: RFR [jdk11]: 8212200: assert(on_stack()) failed when shared java.lang.object is redefined by JVMTI agent In-Reply-To: References: Message-ID: Yes, that is fine. Thank you. Best regards, Goetz. > -----Original Message----- > From: Baesken, Matthias > Sent: Tuesday, July 7, 2020 2:03 PM > To: Lindenmaier, Goetz ; 'jdk-updates- > dev at openjdk.java.net' > Subject: RE: RFR [jdk11]: 8212200: assert(on_stack()) failed when shared > java.lang.object is redefined by JVMTI agent > > Hi G?tz , > > >For the fixes to the test you included in your change. > >Please downport them as changes of its own, and > >push the three changes together. > > Okay I requested > > https://bugs.openjdk.java.net/browse/JDK-8221918 > > and added this review to the comments , hope that's fine with you. > > Best regards, Matthias > > > ( and btw downporting all 3 as single patches generates additional effort + > room for error . ) > > > -----Original Message----- > From: Lindenmaier, Goetz > Sent: Montag, 6. Juli 2020 12:17 > To: Baesken, Matthias ; 'jdk-updates- > dev at openjdk.java.net' > Subject: RE: RFR [jdk11]: 8212200: assert(on_stack()) failed when shared > java.lang.object is redefined by JVMTI agent > > Hi Matthias, > > I had a look at your change. > > classFile.cpp: > You skipped one chunk, as the fixed code is not in 11. OK. > > systemDictionary.hpp > You removed comment > " and a flag word > // that makes some minor distinctions, like whether the klass > // is preloaded, optional, release-specific, etc." > I think this should be kept, as this is still relevant in 11. > Your change does not touch this flag. > > Other changes are ok. > > heapShared.cpp > Just indentation, ok. > > For the fixes to the test you included in your change. > Please downport them as changes of its own, and > push the three changes together. > > Best regards, > Goetz. > > > -----Original Message----- > > From: jdk-updates-dev On > Behalf > > Of Baesken, Matthias > > Sent: Tuesday, June 30, 2020 9:36 AM > > To: 'jdk-updates-dev at openjdk.java.net' dev at openjdk.java.net> > > Subject: [CAUTION] RFR [jdk11]: 8212200: assert(on_stack()) failed when > > shared java.lang.object is redefined by JVMTI agent > > > > Hello, please review the jdk11 backport of 8212200 . > > > > I had to do a few slight adjustments to adjust the src changes of 8212200 > to > > jdk11 . > > > > The test > > > ( test/hotspot/jtreg/runtime/SharedArchiveFile/serviceability/ReplaceCritical > > Classes.java.) from the original jdk/jdk webrev of 8212200 did not work. > > So I had to include adjustments from 8221918 and 8213275 to fix the test . > > > > > > > > Bug/webrev : > > > > https://bugs.openjdk.java.net/browse/JDK-8212200 > > > > http://cr.openjdk.java.net/~mbaesken/webrevs/8212200.0/ > > > > > > Original jdk/jdk webrev : > > > > http://hg.openjdk.java.net/jdk/jdk/rev/625f6c742392 > > > > > > Thanks, Matthias From vicente.romero at oracle.com Tue Jul 7 18:56:33 2020 From: vicente.romero at oracle.com (Vicente Romero) Date: Tue, 7 Jul 2020 14:56:33 -0400 Subject: RFR: [Backport] JDK11-8193367 annotated type variables bounds crash javac In-Reply-To: References: <7BB770AF-AB6A-447F-A8F2-E850851F365D@oracle.com> <21A12194-2F55-4E6D-8D30-B8143781DBFA@oracle.com> Message-ID: Hi Bernard, Adam located the other issue [1] that needs to be backported before the backport for JDK-8193367. Once [1] has been backported JDK-8193367 can also be backporte as-is, Thanks, Vicente [1] https://bugs.openjdk.java.net/browse/JDK-8213703 On 7/6/20 3:39 PM, B. Blaser wrote: > Hi Adam & Vicente, > > I wrote the original patch and as far as I can remember, Adam is right > that it's only replacing 'bound' with 'getUpperBound()' and the > current difference looks therefore pertinent in the sense that the > call to 'isIntersectionOrUnionType' seems to come from another patch > that might be backported too? But note that this isn't an approval as > I'm not a Reviewer and I would need to look at your patch more > attentively. > > Cheers, > Bernard > > On Mon, 29 Jun 2020 at 11:40, Adam Sotona wrote: >> I've double-checked it, the patch is only replacing .bound with .getUpperBound() and implementing the .getUpperBound(). >> The semantical change by implementing .isIntersectionOrUnionType() comes from some other patch. >> >> Adam >> >> On 26 Jun 2020, at 18:27, Vicente Romero wrote: >> >> but isn't the case the the original patch is returning true if the upper bound of the type variable is a union or an intersection while the patch adapted to 11 is only returning true if that upper bound is an intersection? what would happen if is is an union? >> >> Vicente >> >> On 6/26/20 2:39 AM, Adam Sotona wrote: >> >> Hi Vicente, >> I found the code isIntersectionOrUnionType(Type t) is already there - that is why the patch conflict appears. >> >> Thanks, >> Adam >> >> On 25 Jun 2020, at 21:57, Vicente Romero wrote: >> >> Hi Adam, >> >> On 6/23/20 6:55 AM, Adam Sotona wrote: >> >> Hi, >> Please review backport of 8193367 into JDK 11. >> >> Original patch at http://hg.openjdk.java.net/jdk/jdk/rev/a772e65727c5 has just minor conflicts in copyright headers and in one code fragment with JDK 11 repository. >> >> New patch differs in functionality with the original just in one block in src/jdk.compiler/share/classes/com/sun/tools/javac/comp/LambdaToMethod.java: >> < - if (tv.bound.getKind() == TypeKind.INTERSECTION) { >> < + if (tv.getUpperBound().getKind() == TypeKind.INTERSECTION) { >> >> >> this difference seems like an important semantic change compared to what the original patch is doing. I guess you will need to port method: `boolean isIntersectionOrUnionType(Type t)` too >> >> versus: >>> - return isIntersectionOrUnionType(tv.bound); >>> + return isIntersectionOrUnionType(tv.getUpperBound()); >> Patched JDK 11 passed all Tier 1, 2 and 3 tests. >> >> Original JBS: https://bugs.openjdk.java.net/browse/JDK-8193367 >> Webrev: http://cr.openjdk.java.net/~asotona/8193367/webrev.00/ >> Backport JBS: https://bugs.openjdk.java.net/browse/JDK-8248014 >> >> Thanks, >> Adam >> >> >> Thanks, >> Vicente >> >> >> >> From goetz.lindenmaier at sap.com Wed Jul 8 06:48:11 2020 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Wed, 8 Jul 2020 06:48:11 +0000 Subject: [11u] RFR: 8244719: CTW: C2 compilation fails with "assert(!VerifyHashTableKeys || _hash_lock == 0) failed: remove node from hash table before modifying it" In-Reply-To: <6DCA03CE-884E-494E-9704-06FC2E2D6666@amazon.com> References: <6DCA03CE-884E-494E-9704-06FC2E2D6666@amazon.com> Message-ID: Hi Paul, Thanks for reviewing! Best regards, Goetz. > -----Original Message----- > From: Hohensee, Paul > Sent: Monday, June 29, 2020 10:47 PM > To: Lindenmaier, Goetz ; jdk-updates- > dev at openjdk.java.net > Subject: RE: [11u] RFR: 8244719: CTW: C2 compilation fails with > "assert(!VerifyHashTableKeys || _hash_lock == 0) failed: remove node from > hash table before modifying it" > > Lgtm. > > Paul > > ?On 6/25/20, 12:33 AM, "jdk-updates-dev on behalf of Lindenmaier, Goetz" > goetz.lindenmaier at sap.com> wrote: > > Hi, > > I am downporting this for parity with 11.0.9-oracle. > I had to adapt the bytecode version in the jasm file at line 26 > to make it compile with 11. Test passes. > > http://cr.openjdk.java.net/~goetz/wr20/8244719-remove_from_hash- > jdk11/01/ > > Please review. > > https://bugs.openjdk.java.net/browse/JDK-8244719 > https://hg.openjdk.java.net/jdk/jdk/rev/e8d34f3f6833 > > Best regards, > Goetz From goetz.lindenmaier at sap.com Wed Jul 8 06:51:43 2020 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Wed, 8 Jul 2020 06:51:43 +0000 Subject: [11u] RFR: 8246203: Segmentation fault in verification due to stack overflow with -XX:+VerifyIterativeGVN In-Reply-To: References: Message-ID: Hi Matthias, Thanks for reviewing! Best regards, Goetz. From: Baesken, Matthias Sent: Tuesday, July 7, 2020 9:50 AM To: 'jdk-updates-dev at openjdk.java.net' Cc: Lindenmaier, Goetz Subject: RE: [11u] RFR: 8246203: Segmentation fault in verification due to stack overflow with -XX:+VerifyIterativeGVN >Hi, > >I would downport this for parity with 11.0.9-oracle. > >I had to edit another verify() call to make it compile, see phaseX.cpp. >http://cr.openjdk.java.net/~goetz/wr20/8246209-VerifyIterativeGVN-jdk11/01/ > >Please review. > >https://bugs.openjdk.java.net/browse/JDK-8246203 >https://hg.openjdk.java.net/jdk/jdk/rev/49fee4ea8073 > >Best regards, > Goetz. Hi Goetz, the backport looks good to me . Best regards, Matthias From christoph.langer at sap.com Wed Jul 8 07:28:46 2020 From: christoph.langer at sap.com (Langer, Christoph) Date: Wed, 8 Jul 2020 07:28:46 +0000 Subject: RFR [jdk11]: 8212200: assert(on_stack()) failed when shared java.lang.object is redefined by JVMTI agent In-Reply-To: References: Message-ID: Hi Matthias, can you post an updated webrev of the fix you're going to push for JDK-8212200? I assume 8221918 and 8213275 would apply cleanly on top of it, right? Cheers Christoph > -----Original Message----- > From: jdk-updates-dev On > Behalf Of Lindenmaier, Goetz > Sent: Dienstag, 7. Juli 2020 14:42 > To: Baesken, Matthias ; 'jdk-updates- > dev at openjdk.java.net' > Subject: [CAUTION] RE: RFR [jdk11]: 8212200: assert(on_stack()) failed when > shared java.lang.object is redefined by JVMTI agent > > Yes, that is fine. Thank you. > > Best regards, > Goetz. > > > -----Original Message----- > > From: Baesken, Matthias > > Sent: Tuesday, July 7, 2020 2:03 PM > > To: Lindenmaier, Goetz ; 'jdk-updates- > > dev at openjdk.java.net' > > Subject: RE: RFR [jdk11]: 8212200: assert(on_stack()) failed when shared > > java.lang.object is redefined by JVMTI agent > > > > Hi G?tz , > > > > >For the fixes to the test you included in your change. > > >Please downport them as changes of its own, and > > >push the three changes together. > > > > Okay I requested > > > > https://bugs.openjdk.java.net/browse/JDK-8221918 > > > > and added this review to the comments , hope that's fine with you. > > > > Best regards, Matthias > > > > > > ( and btw downporting all 3 as single patches generates additional effort > + > > room for error . ) > > > > > > -----Original Message----- > > From: Lindenmaier, Goetz > > Sent: Montag, 6. Juli 2020 12:17 > > To: Baesken, Matthias ; 'jdk-updates- > > dev at openjdk.java.net' > > Subject: RE: RFR [jdk11]: 8212200: assert(on_stack()) failed when shared > > java.lang.object is redefined by JVMTI agent > > > > Hi Matthias, > > > > I had a look at your change. > > > > classFile.cpp: > > You skipped one chunk, as the fixed code is not in 11. OK. > > > > systemDictionary.hpp > > You removed comment > > " and a flag word > > // that makes some minor distinctions, like whether the klass > > // is preloaded, optional, release-specific, etc." > > I think this should be kept, as this is still relevant in 11. > > Your change does not touch this flag. > > > > Other changes are ok. > > > > heapShared.cpp > > Just indentation, ok. > > > > For the fixes to the test you included in your change. > > Please downport them as changes of its own, and > > push the three changes together. > > > > Best regards, > > Goetz. > > > > > -----Original Message----- > > > From: jdk-updates-dev On > > Behalf > > > Of Baesken, Matthias > > > Sent: Tuesday, June 30, 2020 9:36 AM > > > To: 'jdk-updates-dev at openjdk.java.net' > dev at openjdk.java.net> > > > Subject: [CAUTION] RFR [jdk11]: 8212200: assert(on_stack()) failed when > > > shared java.lang.object is redefined by JVMTI agent > > > > > > Hello, please review the jdk11 backport of 8212200 . > > > > > > I had to do a few slight adjustments to adjust the src changes of 8212200 > > to > > > jdk11 . > > > > > > The test > > > > > ( > test/hotspot/jtreg/runtime/SharedArchiveFile/serviceability/ReplaceCritical > > > Classes.java.) from the original jdk/jdk webrev of 8212200 did not work. > > > So I had to include adjustments from 8221918 and 8213275 to fix the test > . > > > > > > > > > > > > Bug/webrev : > > > > > > https://bugs.openjdk.java.net/browse/JDK-8212200 > > > > > > http://cr.openjdk.java.net/~mbaesken/webrevs/8212200.0/ > > > > > > > > > Original jdk/jdk webrev : > > > > > > http://hg.openjdk.java.net/jdk/jdk/rev/625f6c742392 > > > > > > > > > Thanks, Matthias From matthias.baesken at sap.com Wed Jul 8 09:10:56 2020 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Wed, 8 Jul 2020 09:10:56 +0000 Subject: RFR [jdk11]: 8212200: assert(on_stack()) failed when shared java.lang.object is redefined by JVMTI agent In-Reply-To: References: Message-ID: Hello here is the updated webrev : http://cr.openjdk.java.net/~mbaesken/webrevs/8212200.2/ - test is rebroken ( == doesn't include the 2 fixes) -comment changes in systemDictionary.hpp adjusted Best regards, Matthias -----Original Message----- From: Langer, Christoph Sent: Mittwoch, 8. Juli 2020 09:29 To: Lindenmaier, Goetz ; Baesken, Matthias ; 'jdk-updates-dev at openjdk.java.net' Subject: RE: RFR [jdk11]: 8212200: assert(on_stack()) failed when shared java.lang.object is redefined by JVMTI agent Hi Matthias, can you post an updated webrev of the fix you're going to push for JDK-8212200? I assume 8221918 and 8213275 would apply cleanly on top of it, right? Cheers Christoph > -----Original Message----- > From: jdk-updates-dev On > Behalf Of Lindenmaier, Goetz > Sent: Dienstag, 7. Juli 2020 14:42 > To: Baesken, Matthias ; 'jdk-updates- > dev at openjdk.java.net' > Subject: [CAUTION] RE: RFR [jdk11]: 8212200: assert(on_stack()) failed when > shared java.lang.object is redefined by JVMTI agent > > Yes, that is fine. Thank you. > > Best regards, > Goetz. > > > -----Original Message----- > > From: Baesken, Matthias > > Sent: Tuesday, July 7, 2020 2:03 PM > > To: Lindenmaier, Goetz ; 'jdk-updates- > > dev at openjdk.java.net' > > Subject: RE: RFR [jdk11]: 8212200: assert(on_stack()) failed when shared > > java.lang.object is redefined by JVMTI agent > > > > Hi G?tz , > > > > >For the fixes to the test you included in your change. > > >Please downport them as changes of its own, and > > >push the three changes together. > > > > Okay I requested > > > > https://bugs.openjdk.java.net/browse/JDK-8221918 > > > > and added this review to the comments , hope that's fine with you. > > > > Best regards, Matthias > > > > > > ( and btw downporting all 3 as single patches generates additional effort > + > > room for error . ) > > > > > > -----Original Message----- > > From: Lindenmaier, Goetz > > Sent: Montag, 6. Juli 2020 12:17 > > To: Baesken, Matthias ; 'jdk-updates- > > dev at openjdk.java.net' > > Subject: RE: RFR [jdk11]: 8212200: assert(on_stack()) failed when shared > > java.lang.object is redefined by JVMTI agent > > > > Hi Matthias, > > > > I had a look at your change. > > > > classFile.cpp: > > You skipped one chunk, as the fixed code is not in 11. OK. > > > > systemDictionary.hpp > > You removed comment > > " and a flag word > > // that makes some minor distinctions, like whether the klass > > // is preloaded, optional, release-specific, etc." > > I think this should be kept, as this is still relevant in 11. > > Your change does not touch this flag. > > > > Other changes are ok. > > > > heapShared.cpp > > Just indentation, ok. > > > > For the fixes to the test you included in your change. > > Please downport them as changes of its own, and > > push the three changes together. > > > > Best regards, > > Goetz. > > > > > -----Original Message----- > > > From: jdk-updates-dev On > > Behalf > > > Of Baesken, Matthias > > > Sent: Tuesday, June 30, 2020 9:36 AM > > > To: 'jdk-updates-dev at openjdk.java.net' > dev at openjdk.java.net> > > > Subject: [CAUTION] RFR [jdk11]: 8212200: assert(on_stack()) failed when > > > shared java.lang.object is redefined by JVMTI agent > > > > > > Hello, please review the jdk11 backport of 8212200 . > > > > > > I had to do a few slight adjustments to adjust the src changes of 8212200 > > to > > > jdk11 . > > > > > > The test > > > > > ( > test/hotspot/jtreg/runtime/SharedArchiveFile/serviceability/ReplaceCritical > > > Classes.java.) from the original jdk/jdk webrev of 8212200 did not work. > > > So I had to include adjustments from 8221918 and 8213275 to fix the test > . > > > > > > > > > > > > Bug/webrev : > > > > > > https://bugs.openjdk.java.net/browse/JDK-8212200 > > > > > > http://cr.openjdk.java.net/~mbaesken/webrevs/8212200.0/ > > > > > > > > > Original jdk/jdk webrev : > > > > > > http://hg.openjdk.java.net/jdk/jdk/rev/625f6c742392 > > > > > > > > > Thanks, Matthias From christoph.langer at sap.com Wed Jul 8 09:23:57 2020 From: christoph.langer at sap.com (Langer, Christoph) Date: Wed, 8 Jul 2020 09:23:57 +0000 Subject: RFR [jdk11]: 8212200: assert(on_stack()) failed when shared java.lang.object is redefined by JVMTI agent In-Reply-To: References: Message-ID: Hi Matthias, thanks - this looks good to me. Best regards Christoph > -----Original Message----- > From: Baesken, Matthias > Sent: Mittwoch, 8. Juli 2020 11:11 > To: Langer, Christoph ; Lindenmaier, Goetz > ; 'jdk-updates-dev at openjdk.java.net' updates-dev at openjdk.java.net> > Subject: RE: RFR [jdk11]: 8212200: assert(on_stack()) failed when shared > java.lang.object is redefined by JVMTI agent > > Hello here is the updated webrev : > > > http://cr.openjdk.java.net/~mbaesken/webrevs/8212200.2/ > > > - test is rebroken ( == doesn't include the 2 fixes) > -comment changes in systemDictionary.hpp adjusted > > Best regards, Matthias > > > > -----Original Message----- > From: Langer, Christoph > Sent: Mittwoch, 8. Juli 2020 09:29 > To: Lindenmaier, Goetz ; Baesken, Matthias > ; 'jdk-updates-dev at openjdk.java.net' updates-dev at openjdk.java.net> > Subject: RE: RFR [jdk11]: 8212200: assert(on_stack()) failed when shared > java.lang.object is redefined by JVMTI agent > > Hi Matthias, > > can you post an updated webrev of the fix you're going to push for JDK- > 8212200? > > I assume 8221918 and 8213275 would apply cleanly on top of it, right? > > Cheers > Christoph > > > -----Original Message----- > > From: jdk-updates-dev On > > Behalf Of Lindenmaier, Goetz > > Sent: Dienstag, 7. Juli 2020 14:42 > > To: Baesken, Matthias ; 'jdk-updates- > > dev at openjdk.java.net' > > Subject: [CAUTION] RE: RFR [jdk11]: 8212200: assert(on_stack()) failed > when > > shared java.lang.object is redefined by JVMTI agent > > > > Yes, that is fine. Thank you. > > > > Best regards, > > Goetz. > > > > > -----Original Message----- > > > From: Baesken, Matthias > > > Sent: Tuesday, July 7, 2020 2:03 PM > > > To: Lindenmaier, Goetz ; 'jdk-updates- > > > dev at openjdk.java.net' > > > Subject: RE: RFR [jdk11]: 8212200: assert(on_stack()) failed when shared > > > java.lang.object is redefined by JVMTI agent > > > > > > Hi G?tz , > > > > > > >For the fixes to the test you included in your change. > > > >Please downport them as changes of its own, and > > > >push the three changes together. > > > > > > Okay I requested > > > > > > https://bugs.openjdk.java.net/browse/JDK-8221918 > > > > > > and added this review to the comments , hope that's fine with you. > > > > > > Best regards, Matthias > > > > > > > > > ( and btw downporting all 3 as single patches generates additional effort > > + > > > room for error . ) > > > > > > > > > -----Original Message----- > > > From: Lindenmaier, Goetz > > > Sent: Montag, 6. Juli 2020 12:17 > > > To: Baesken, Matthias ; 'jdk-updates- > > > dev at openjdk.java.net' > > > Subject: RE: RFR [jdk11]: 8212200: assert(on_stack()) failed when shared > > > java.lang.object is redefined by JVMTI agent > > > > > > Hi Matthias, > > > > > > I had a look at your change. > > > > > > classFile.cpp: > > > You skipped one chunk, as the fixed code is not in 11. OK. > > > > > > systemDictionary.hpp > > > You removed comment > > > " and a flag word > > > // that makes some minor distinctions, like whether the klass > > > // is preloaded, optional, release-specific, etc." > > > I think this should be kept, as this is still relevant in 11. > > > Your change does not touch this flag. > > > > > > Other changes are ok. > > > > > > heapShared.cpp > > > Just indentation, ok. > > > > > > For the fixes to the test you included in your change. > > > Please downport them as changes of its own, and > > > push the three changes together. > > > > > > Best regards, > > > Goetz. > > > > > > > -----Original Message----- > > > > From: jdk-updates-dev On > > > Behalf > > > > Of Baesken, Matthias > > > > Sent: Tuesday, June 30, 2020 9:36 AM > > > > To: 'jdk-updates-dev at openjdk.java.net' > > dev at openjdk.java.net> > > > > Subject: [CAUTION] RFR [jdk11]: 8212200: assert(on_stack()) failed > when > > > > shared java.lang.object is redefined by JVMTI agent > > > > > > > > Hello, please review the jdk11 backport of 8212200 . > > > > > > > > I had to do a few slight adjustments to adjust the src changes of > 8212200 > > > to > > > > jdk11 . > > > > > > > > The test > > > > > > > ( > > > test/hotspot/jtreg/runtime/SharedArchiveFile/serviceability/ReplaceCritical > > > > Classes.java.) from the original jdk/jdk webrev of 8212200 did not > work. > > > > So I had to include adjustments from 8221918 and 8213275 to fix the > test > > . > > > > > > > > > > > > > > > > Bug/webrev : > > > > > > > > https://bugs.openjdk.java.net/browse/JDK-8212200 > > > > > > > > http://cr.openjdk.java.net/~mbaesken/webrevs/8212200.0/ > > > > > > > > > > > > Original jdk/jdk webrev : > > > > > > > > http://hg.openjdk.java.net/jdk/jdk/rev/625f6c742392 > > > > > > > > > > > > Thanks, Matthias From goetz.lindenmaier at sap.com Wed Jul 8 10:46:54 2020 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Wed, 8 Jul 2020 10:46:54 +0000 Subject: [11u] RFR: 8236464: SO_LINGER option is ignored by SSLSocket in JDK 11 Message-ID: Hi, I am downporting this for parity with 11.0.9-oracle. I had to do quite some adaptions: http://cr.openjdk.java.net/~goetz/wr20/8236464-SSLSocket_LINGER-jdk11/01/ src/java.base/share/classes/sun/security/ssl/SSLSocketImpl.java The change renames a method that is not there in 11. I skipped the renaming and applied the functional change that is in the same chunk. src/java.base/share/classes/sun/security/ssl/TransportContext.java The code deleted differs, because in 15 explicit locking is used. I added a synchronized block in the method introduced. Please review. https://bugs.openjdk.java.net/browse/JDK-8236464 https://hg.openjdk.java.net/jdk/jdk/rev/c783b60f48db Best regards, Goetz From rkennke at redhat.com Wed Jul 8 18:31:17 2020 From: rkennke at redhat.com (Roman Kennke) Date: Wed, 08 Jul 2020 20:31:17 +0200 Subject: RFR (11u, XXL): Upstream/backport Shenandoah to JDK11u In-Reply-To: References: <3595e963-fa5b-bfce-c463-841b84d4103f@redhat.com> <6468CB08-3DB5-488E-9122-E3699A95218B@amazon.com> Message-ID: Hi Gil & all, I put in some extra work and had a deep look at the remaining unguarded changes in shared-code, with an eye to either eliminating them altogether, or - where this is not possible - guard them with build- time #if INCLUDE_SHENANDOAHGC or similar. The guiding principle in this effort has been that building without Shenandoah (--with-jvm-features=- shenandoahgc) would compile *the exact same* code that it does now. I believe I achieved this with the following proposed changes: Full webrev (incl. Shenandoah-only files): https://cr.openjdk.java.net/~rkennke/shenandoah-jdk11u-upstream/webrev.06-all/ Shared-code-only changes: https://cr.openjdk.java.net/~rkennke/shenandoah-jdk11u-upstream/webrev.06-shared/ The latter webrev exludes all files that are only compiled with Shenandoah enabled. Also, compared to previous webrevs, this excludes Shenandoah by default. As far as I can tell, this literally compiles the same code into libjvm.so as it does now, when building without Shenandoah. The only exception is the inclusion of the global flag UseShenandoahGC, which is by-design. When this is enabled in a build without Shenandoah, it prints the appropriate error message and exist (exactly like with any other GC that is not built-in). I've been thinking about if we can come up with a mechanical proof of correctness. The first best thing would have been reproducible builds, but we don't have that (and the UseShenandoahGC flag would mess it up too). The other option would be to dump preprocessed sources and diff them, but that is disturbed by the necessary inclusion of "utilities/macros.hpp" which gives us the INCLUDE_SHENANDOAHGC macro. Which leaves us with carefully inspecting the webrev by multiple sets of eyes and brains. Gil, would the change like this ease your concerns? Thank you, Roman On Fri, 2020-06-26 at 23:39 +0000, Gil Tene wrote: > As I noted before, we have serious reservations about adding major > new features to a > stable LTS that is upstream of all of 11u. Based on daily > interactions with tons of OpenJDK > end users, I can say that the vast majority of end users that are > ramping up on 11u are > demanding stability and maturity above all else. The addition of such > major changes in > an 11u update will cause many of them to stall updates, and > potentially stall the uptake > of 11u altogether, waiting for 11u to actually stabilize first > (evidenced by no longer doing > such things), so they can keep up with fixes and security issues > without having to take on > potential regressions driven by late added or experimental features. > > If you want a modern GA'ed OpenJDK with all the latest features in > it, use one. Please > please please don't force people who choose to stick with a given > OpenJDK version for > stability, and knowingly did not move to a later OpenJDK version, to > deploy the newly > implemented features that you like by pushing for back-porting them > to the upstream > project that they get their security updates from... > > I can see a potentially good case for a downstream-from-11u project > that incorporates > ongoing development (as opposed to bug fixes) from later upstream > versions, and would > be happy to collaborate on one. But the 11u that is upstream of all > other 11u's should not > be seen as a developer's playground, or a place to add cool new > features to a stable > release. The upstream versions are there for that purpose. The valid > interests of > deep-bench, self-supporting groups making use of OpenJDK, who can > take on the stability > risks involved due to the depth of their technical bench, should not > overcome the interests > of the millions of consumers of the 11u builds that do not have that > luxury, and who > need and expect stable (in the changing only where necessary sense) > 11u releases. > > I'd like to point out that 13u already includes Shenandoah. 15u > (expected in 3 months) > already includes Shenandoah as a non-experimental feature. And we can > all collaborate > on back-porting bug-fixes to those to updates to keep a lively > version of OpenJDK that > includes Shenandoah up to date from a bug fix and security update > point of view. For > those insisting on waiting for an LTS (on the assumption that LTSs > are long term stable > and don't get those sort of messing-with-after-the-fact done to them, > mind you), 17u will > be out in 15 months. > > So we are not talking about some far-away future for end users that > want Shenandoah. > It is here, now, in multiple consumable forms, and available for > people not looking for > the stability of an LTS, and even for people who do (as they can use > the Red Hat 11u > builds). It it will be available in more forms very soon with no > effort, and we don't have > to destabilize 11u for others to make that happen. > > ? Gil. > > > On Jun 26, 2020, at 1:51 PM, Mathiske, Bernd > > wrote: > > > > Hi Roman, > > > > We are also in favor of upstreaming Shenandoah to jdk11u, for the > > same reasons Aditya listed, and we agree with everything else he > > has written. Similar to Aditya?s message below, we believe that > > Shenandoah has strong demand and potential for those running > > jdk11u. Targeting jdk11u is the most efficient way to reach a > > large user base and to enable production use of Shenandoah. > > > > At Amazon, we plan to continue to contribute to Shenandoah, and we > > fully intend to help maintain Shenandoah in jdk11u. > > > > Best, > > > > Bernd, Kelvin, Azeem > > > > ?On 6/26/20, 10:36 AM, "jdk-updates-dev on behalf of Aditya > > Mandaleeka" > adityam at microsoft.com> wrote: > > > > Hi Roman, > > > > I am in favor of this proposed upstreaming of Shenandoah into > > jdk11u. Microsoft > > has some long-term commitments to Java 11 customers and having > > Shenandoah > > included in jdk11u upstream would definitely be well-received by > > my team and by > > our customers. There is demand for this kind of GC option on > > OpenJDK 11, and it > > would be beneficial to the community for it to be available on > > the upstream > > OpenJDK rather than only in specific downstream releases. > > Coupled with the fact > > that Shenandoah has been tested on JDK 11 for quite a while now, > > I think this > > upstreaming makes sense. > > > > You mentioned in your original mail that your team at RedHat > > intends to maintain > > this proposed jdk11u version the same way it has been maintained > > in > > shenandoah/jdk11. That is great--from what I?ve seen in these > > past few months > > that I?ve been involved, I?ve been impressed by the way > > backports to > > shenandoah/jdk11 are being handled, the effort that is made to > > keep the > > Shenandoah code closely aligned between sh/11 and jdk/jdk, and > > the care taken to > > ensure that the impact on shared code is kept to a minimum. > > > > At Microsoft, we?ve already been contributing patches and > > investigating and > > reporting issues based on our testing of Shenandoah, both on > > jdk/jdk and > > shenandoah/jdk11. We intend to continue this kind of > > involvement, so if this > > upstreaming to jdk11u goes through, your team will also have our > > support in > > maintaining Shenandoah there. > > > > Thanks, > > Aditya > > > > -----Original Message----- > > From: jdk-updates-dev On > > Behalf Of Roman Kennke > > Sent: Thursday, June 25, 2020 4:16 AM > > To: jdk-updates-dev at openjdk.java.net > > Subject: Re: RFR (11u, XXL): Upstream/backport Shenandoah to > > JDK11u > > > > I would like to revive this RFR, I believe we haven't come to a > > conclusion yet. I still think it would be very useful to have > > the > > Shenandoah GC included in jdk11u upstream. I have heard from > > several > > vendors who would like to ship Shenandoah, but maintaining the > > rather > > huge Shenandoah backport patch is an absolute no-go. > > > > This is an updated patch that includes numerous bugfixes and > > improvements in Shenandoah. It is what Red Hat is going to ship > > in > > their 11.0.8 release. > > > > Full webrev: > > > > https://nam06.safelinks.protection.outlook.com/?url=https:%2F%2Fcr.openjdk.java.net%2F~rkennke%2Fshenandoah-jdk11u-upstream%2Fwebrev.04-all%2F&data=02%7C01%7Cadityam%40microsoft.com%7C7aec62734d22458243d908d818f974c5%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637286806967016879&sdata=G1HjEJ68qFJ2uwRyxfFKNE8gcbsFXEdYi%2B9RyrUeG60%3D&reserved=0 > > > > Webrev only for shared-code changes: > > > > https://nam06.safelinks.protection.outlook.com/?url=https:%2F%2Fcr.openjdk.java.net%2F~rkennke%2Fshenandoah-jdk11u-upstream%2Fwebrev.04-shared%2F&data=02%7C01%7Cadityam%40microsoft.com%7C7aec62734d22458243d908d818f974c5%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637286806967026872&sdata=RrKNOvxc3%2FheThyi2cMiazE%2BxC9UfARxfTbmuoc640Y%3D&reserved=0 > > > > Please let me know what you think. > > > > Roman > > > > > On Fri, 2020-01-17 at 18:05 +0100, Roman Kennke wrote: > > > Hello, > > > > > > The Shenandoah GC has been integrated into jdk12 a little over a > > > year > > > ago: > > > > > > https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2Fhg.openjdk.java.net%2Fjdk%2Fjdk%2Frev%2F9c18c9d839d3&data=02%7C01%7Cadityam%40microsoft.com%7C7aec62734d22458243d908d818f974c5%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637286806967026872&sdata=QJo2xwdHiEuaqsz26P2j%2BZdE6j3AxUvpnHVTjD9XGXI%3D&reserved=0 > > > > > > The Shenandoah team has been maintaining the Shenandoah-enabled > > > jdk11u > > > downstream repository since the inception of jdk11 under: > > > > > > https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2Fhg.openjdk.java.net%2Fshenandoah%2Fjdk11&data=02%7C01%7Cadityam%40microsoft.com%7C7aec62734d22458243d908d818f974c5%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637286806967026872&sdata=P%2FLkJ5VCrq4oWzik3WZ3GTy1a3mA52mxkITasawIMk0%3D&reserved=0 > > > > > > In order to make it more useful and accessible for users, we > > > would > > > like > > > to make it available in upstream jdk11u. The Shenandoah team at > > > Red > > > Hat > > > intends to maintain it the same way it had been maintained in > > > shenandoah/jdk11, in the course of regular jdk11u maintenance Red > > > Hat > > > already does. > > > > > > Thanks to recent changes in Shenandoah over the last year, we are > > > now > > > able to fit the GC interface in jdk11 almost exactly. There are a > > > few > > > exceptions though. We tried to follow the following pattern for > > > all > > > shared-code changes where it made sense: > > > > > > #if INCLUDE_SHENANDOAH_GC > > > if (UseShenandoahGC) { > > > ... > > > } > > > #endif > > > > > > This ensures that the Shenandoah-specific changes are cut out of > > > the > > > build when building without Shenandoah, and avoid those code > > > paths at > > > runtime when running without Shenandoah. > > > > > > Also, there are a few places where this was not possible or where > > > it > > > didn't seem feasible, but we tried to keep them few and obvious. > > > Whenever this is the case, we are mirroring as close as possible > > > what > > > we > > > have in jdk/jdk. > > > > > > Architecture-wise, Shenandoah runs on x86_64, aarch64, x86_32. > > > Other > > > architectures still build fine, and Shenandoah gets automatically > > > disabled during configure if not supported. OS-wise, Shenandoah > > > runs > > > on > > > Linux, Windows, and is known to run on MacOS X and Solaris too. > > > > > > I wasn't sure which form this should take. I decided to start > > > with > > > the > > > easiest possible form this could take: a simple patch/webrev and > > > an > > > RFR. > > > Let me know if you need a JIRA issue or any other formalities to > > > track that. > > > > > > Full webrev: > > > https://nam06.safelinks.protection.outlook.com/?url=https:%2F%2Fcr.openjdk.java.net%2F~rkennke%2Fshenandoah-jdk11u-upstream%2Fwebrev.02-all%2F&data=02%7C01%7Cadityam%40microsoft.com%7C7aec62734d22458243d908d818f974c5%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637286806967026872&sdata=eHspbGt5Eox4EMv8hstTBwsqFYefGNO2acGymP7ZfN0%3D&reserved=0 > > > > > > Webrev only for shared-code changes: > > > https://nam06.safelinks.protection.outlook.com/?url=https:%2F%2Fcr.openjdk.java.net%2F~rkennke%2Fshenandoah-jdk11u-upstream%2Fwebrev.02-shared%2F&data=02%7C01%7Cadityam%40microsoft.com%7C7aec62734d22458243d908d818f974c5%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637286806967026872&sdata=LGghbNRyHPSJLK8l2Oyhxh1lhD46LmvZtN1G5htYBpk%3D&reserved=0 > > > > > > The webrevs are based on and depend on the arraycopy patch > > > proposed > > > for > > > review here: > > > > > > https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmail.openjdk.java.net%2Fpipermail%2Fjdk-updates-dev%2F2020-January%2F002396.html&data=02%7C01%7Cadityam%40microsoft.com%7C7aec62734d22458243d908d818f974c5%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637286806967026872&sdata=Kw0qkfvtVNVrCiMeypwr4BHX9e3bZafJnVCMVY%2BvOOw%3D&reserved=0 > > > > > > Testing: > > > The code has been tested many many times, continuously while > > > maintaining > > > it. It has also served as the basis for RPMs and shipped to > > > customers. > > > We are running all sorts of tests of the hotspot/jtreg testsuite, > > > in > > > particular hotspot_gc_shenandoah, CTW tests, the regular tier* > > > tests, > > > several popular benchmarks (specjvm, specjbb on a regular basis), > > > on > > > a > > > nightly basis. > > > > > > Please let me know what you think. > > > > > > Thanks, > > > Roman > > > > > > > > > > > > > > > > > > > > > From gil at azul.com Wed Jul 8 21:39:52 2020 From: gil at azul.com (Gil Tene) Date: Wed, 8 Jul 2020 21:39:52 +0000 Subject: RFR (11u, XXL): Upstream/backport Shenandoah to JDK11u In-Reply-To: References: <3595e963-fa5b-bfce-c463-841b84d4103f@redhat.com> <6468CB08-3DB5-488E-9122-E3699A95218B@amazon.com> Message-ID: > On Jul 8, 2020, at 11:31 AM, Roman Kennke wrote: > > Hi Gil & all, > > I put in some extra work and had a deep look at the remaining unguarded > changes in shared-code, with an eye to either eliminating them > altogether, or - where this is not possible - guard them with build- > time #if INCLUDE_SHENANDOAHGC or similar. The guiding principle in this > effort has been that building without Shenandoah (--with-jvm-features=- > shenandoahgc) would compile *the exact same* code that it does now. I > believe I achieved this with the following proposed changes: > > Full webrev (incl. Shenandoah-only files): > https://cr.openjdk.java.net/~rkennke/shenandoah-jdk11u-upstream/webrev.06-all/ > > Shared-code-only changes: > https://cr.openjdk.java.net/~rkennke/shenandoah-jdk11u-upstream/webrev.06-shared/ > > The latter webrev exludes all files that are only compiled with > Shenandoah enabled. > > Also, compared to previous webrevs, this excludes Shenandoah by > default. > > As far as I can tell, this literally compiles the same code into > libjvm.so as it does now, when building without Shenandoah. The only > exception is the inclusion of the global flag UseShenandoahGC, which is > by-design. When this is enabled in a build without Shenandoah, it > prints the appropriate error message and exist (exactly like with any > other GC that is not built-in). For the purpose of this thought exercise, I think we could consider that flag addition as a separate change (which would happen before all the other stuff). So that we can consider the "would compile *the exact same* code" statements in the context of a builds that already include the flag. > > I've been thinking about if we can come up with a mechanical proof of > correctness. The first best thing would have been reproducible builds, > but we don't have that (and the UseShenandoahGC flag would mess it up > too). The other option would be to dump preprocessed sources and diff > them, but that is disturbed by the necessary inclusion of > "utilities/macros.hpp" which gives us the INCLUDE_SHENANDOAHGC macro. I think that a mechanized way to verify the assumption will be key, as you will want it verified not just initially, but with every future change. figuring out how to do that is a key question... Yes, reproducible builds would be great, but we probably don't want to hold our breath for that. Maybe something around comparing individual object file contents that go into the final link can work? > > Which leaves us with carefully inspecting the webrev by multiple sets > of eyes and brains. Eye and brain inspection is unlikely to catch innocent looking "leaks" in this model 6 months after the initial merge effort is completed, when someone touches a line on the wrong side of an ifdef, or makes a change to the scope of an ifdef or to what turns it on it. I think that an ongoing means of verification is needed as part of a solution like this for it to be viable maintainable. I'm not sure what that would look like. But it may be worth thinking up some techniques. I'm also not sure how one would police future code changes and backports of fixes from usptream versions, unless each and every review of code carefully takes into consideration the "is this Shenandoah related or not? Should it go in ifdefs or not?" questions. Can we come up with systemic ways to know that a code change being backported is Shenandoah-related? E.g. if the upstream code shape and ifdef'ing rules remained similar to the downstream, we may be able to keep things workable, but I suspect trhat the upstream (especially after 15u) will be tempted to remove the strict ifdef'ing pretty quickly, and start placing code changes in common code, and as those changes will likely be needed in backporting gixes to the "Shenandoah enabled 11u" mode, carefully examination on a case by case basis will be needed, at least as long as "no shenandoah exists" downstreams of 11u still depend on it for updates. > > Gil, would the change like this ease your concerns? If we could provably show that no code differences exist in builds that do no choose to include the change, it would be hard for me to argue against upstream inclusion based on it having a potential for breaking a downstream distro's stability, no matte rhow neccesary or not it is? So yes, the proposed approach could work if we can figure out the technical ways to achieve it and "keep it achieved" for some time into the future. We will likely need to be able to mechanically verify that things remain like this even as code is introduced in the future. Manual review cannot achieve that alone probably, so we will likely need some build tooling to verify this every time. How long will we need to be able to keep proving this for? I'm not sure. One could argue that it is several years, but I can see an argument that says that after enough time (2 years?) and enough production exposure of the 11u Shenandoah upstream thing, the world will gain enough confidence in it that we no longer need to keep proving the quality suggested. The most telling sign will be whether or not downstream 11u distros that do not include Shenandoah keep producing updates. Once none of those non-Shenandoah build updates happen, there will obviously be no point in keeping the thing proveable in every update? I for one can imagine making Zulu 11 versions of both kinds, letting people choose, and even encouraging new users to use the Shanadoah-enabled one. Once the number of people who seek 11u updates that don't have Shenandoah in them any more drops to some truly tiny number (tiny enough that we feel we can force them to the other updates], I'd happily try to force the issue with them and scratch Zulu and it's mass number of downstream users from the "people who may care" list? > Thank you, > Roman > > > On Fri, 2020-06-26 at 23:39 +0000, Gil Tene wrote: >> As I noted before, we have serious reservations about adding major >> new features to a >> stable LTS that is upstream of all of 11u. Based on daily >> interactions with tons of OpenJDK >> end users, I can say that the vast majority of end users that are >> ramping up on 11u are >> demanding stability and maturity above all else. The addition of such >> major changes in >> an 11u update will cause many of them to stall updates, and >> potentially stall the uptake >> of 11u altogether, waiting for 11u to actually stabilize first >> (evidenced by no longer doing >> such things), so they can keep up with fixes and security issues >> without having to take on >> potential regressions driven by late added or experimental features. >> >> If you want a modern GA'ed OpenJDK with all the latest features in >> it, use one. Please >> please please don't force people who choose to stick with a given >> OpenJDK version for >> stability, and knowingly did not move to a later OpenJDK version, to >> deploy the newly >> implemented features that you like by pushing for back-porting them >> to the upstream >> project that they get their security updates from... >> >> I can see a potentially good case for a downstream-from-11u project >> that incorporates >> ongoing development (as opposed to bug fixes) from later upstream >> versions, and would >> be happy to collaborate on one. But the 11u that is upstream of all >> other 11u's should not >> be seen as a developer's playground, or a place to add cool new >> features to a stable >> release. The upstream versions are there for that purpose. The valid >> interests of >> deep-bench, self-supporting groups making use of OpenJDK, who can >> take on the stability >> risks involved due to the depth of their technical bench, should not >> overcome the interests >> of the millions of consumers of the 11u builds that do not have that >> luxury, and who >> need and expect stable (in the changing only where necessary sense) >> 11u releases. >> >> I'd like to point out that 13u already includes Shenandoah. 15u >> (expected in 3 months) >> already includes Shenandoah as a non-experimental feature. And we can >> all collaborate >> on back-porting bug-fixes to those to updates to keep a lively >> version of OpenJDK that >> includes Shenandoah up to date from a bug fix and security update >> point of view. For >> those insisting on waiting for an LTS (on the assumption that LTSs >> are long term stable >> and don't get those sort of messing-with-after-the-fact done to them, >> mind you), 17u will >> be out in 15 months. >> >> So we are not talking about some far-away future for end users that >> want Shenandoah. >> It is here, now, in multiple consumable forms, and available for >> people not looking for >> the stability of an LTS, and even for people who do (as they can use >> the Red Hat 11u >> builds). It it will be available in more forms very soon with no >> effort, and we don't have >> to destabilize 11u for others to make that happen. >> >> ? Gil. >> >>> On Jun 26, 2020, at 1:51 PM, Mathiske, Bernd >>> wrote: >>> >>> Hi Roman, >>> >>> We are also in favor of upstreaming Shenandoah to jdk11u, for the >>> same reasons Aditya listed, and we agree with everything else he >>> has written. Similar to Aditya?s message below, we believe that >>> Shenandoah has strong demand and potential for those running >>> jdk11u. Targeting jdk11u is the most efficient way to reach a >>> large user base and to enable production use of Shenandoah. >>> >>> At Amazon, we plan to continue to contribute to Shenandoah, and we >>> fully intend to help maintain Shenandoah in jdk11u. >>> >>> Best, >>> >>> Bernd, Kelvin, Azeem >>> >>> ?On 6/26/20, 10:36 AM, "jdk-updates-dev on behalf of Aditya >>> Mandaleeka" >> adityam at microsoft.com> wrote: >>> >>> Hi Roman, >>> >>> I am in favor of this proposed upstreaming of Shenandoah into >>> jdk11u. Microsoft >>> has some long-term commitments to Java 11 customers and having >>> Shenandoah >>> included in jdk11u upstream would definitely be well-received by >>> my team and by >>> our customers. There is demand for this kind of GC option on >>> OpenJDK 11, and it >>> would be beneficial to the community for it to be available on >>> the upstream >>> OpenJDK rather than only in specific downstream releases. >>> Coupled with the fact >>> that Shenandoah has been tested on JDK 11 for quite a while now, >>> I think this >>> upstreaming makes sense. >>> >>> You mentioned in your original mail that your team at RedHat >>> intends to maintain >>> this proposed jdk11u version the same way it has been maintained >>> in >>> shenandoah/jdk11. That is great--from what I?ve seen in these >>> past few months >>> that I?ve been involved, I?ve been impressed by the way >>> backports to >>> shenandoah/jdk11 are being handled, the effort that is made to >>> keep the >>> Shenandoah code closely aligned between sh/11 and jdk/jdk, and >>> the care taken to >>> ensure that the impact on shared code is kept to a minimum. >>> >>> At Microsoft, we?ve already been contributing patches and >>> investigating and >>> reporting issues based on our testing of Shenandoah, both on >>> jdk/jdk and >>> shenandoah/jdk11. We intend to continue this kind of >>> involvement, so if this >>> upstreaming to jdk11u goes through, your team will also have our >>> support in >>> maintaining Shenandoah there. >>> >>> Thanks, >>> Aditya >>> >>> -----Original Message----- >>> From: jdk-updates-dev On >>> Behalf Of Roman Kennke >>> Sent: Thursday, June 25, 2020 4:16 AM >>> To: jdk-updates-dev at openjdk.java.net >>> Subject: Re: RFR (11u, XXL): Upstream/backport Shenandoah to >>> JDK11u >>> >>> I would like to revive this RFR, I believe we haven't come to a >>> conclusion yet. I still think it would be very useful to have >>> the >>> Shenandoah GC included in jdk11u upstream. I have heard from >>> several >>> vendors who would like to ship Shenandoah, but maintaining the >>> rather >>> huge Shenandoah backport patch is an absolute no-go. >>> >>> This is an updated patch that includes numerous bugfixes and >>> improvements in Shenandoah. It is what Red Hat is going to ship >>> in >>> their 11.0.8 release. >>> >>> Full webrev: >>> >>> https://nam06.safelinks.protection.outlook.com/?url=https:%2F%2Fcr.openjdk.java.net%2F~rkennke%2Fshenandoah-jdk11u-upstream%2Fwebrev.04-all%2F&data=02%7C01%7Cadityam%40microsoft.com%7C7aec62734d22458243d908d818f974c5%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637286806967016879&sdata=G1HjEJ68qFJ2uwRyxfFKNE8gcbsFXEdYi%2B9RyrUeG60%3D&reserved=0 >>> >>> Webrev only for shared-code changes: >>> >>> https://nam06.safelinks.protection.outlook.com/?url=https:%2F%2Fcr.openjdk.java.net%2F~rkennke%2Fshenandoah-jdk11u-upstream%2Fwebrev.04-shared%2F&data=02%7C01%7Cadityam%40microsoft.com%7C7aec62734d22458243d908d818f974c5%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637286806967026872&sdata=RrKNOvxc3%2FheThyi2cMiazE%2BxC9UfARxfTbmuoc640Y%3D&reserved=0 >>> >>> Please let me know what you think. >>> >>> Roman >>> >>>> On Fri, 2020-01-17 at 18:05 +0100, Roman Kennke wrote: >>>> Hello, >>>> >>>> The Shenandoah GC has been integrated into jdk12 a little over a >>>> year >>>> ago: >>>> >>>> https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2Fhg.openjdk.java.net%2Fjdk%2Fjdk%2Frev%2F9c18c9d839d3&data=02%7C01%7Cadityam%40microsoft.com%7C7aec62734d22458243d908d818f974c5%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637286806967026872&sdata=QJo2xwdHiEuaqsz26P2j%2BZdE6j3AxUvpnHVTjD9XGXI%3D&reserved=0 >>>> >>>> The Shenandoah team has been maintaining the Shenandoah-enabled >>>> jdk11u >>>> downstream repository since the inception of jdk11 under: >>>> >>>> https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2Fhg.openjdk.java.net%2Fshenandoah%2Fjdk11&data=02%7C01%7Cadityam%40microsoft.com%7C7aec62734d22458243d908d818f974c5%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637286806967026872&sdata=P%2FLkJ5VCrq4oWzik3WZ3GTy1a3mA52mxkITasawIMk0%3D&reserved=0 >>>> >>>> In order to make it more useful and accessible for users, we >>>> would >>>> like >>>> to make it available in upstream jdk11u. The Shenandoah team at >>>> Red >>>> Hat >>>> intends to maintain it the same way it had been maintained in >>>> shenandoah/jdk11, in the course of regular jdk11u maintenance Red >>>> Hat >>>> already does. >>>> >>>> Thanks to recent changes in Shenandoah over the last year, we are >>>> now >>>> able to fit the GC interface in jdk11 almost exactly. There are a >>>> few >>>> exceptions though. We tried to follow the following pattern for >>>> all >>>> shared-code changes where it made sense: >>>> >>>> #if INCLUDE_SHENANDOAH_GC >>>> if (UseShenandoahGC) { >>>> ... >>>> } >>>> #endif >>>> >>>> This ensures that the Shenandoah-specific changes are cut out of >>>> the >>>> build when building without Shenandoah, and avoid those code >>>> paths at >>>> runtime when running without Shenandoah. >>>> >>>> Also, there are a few places where this was not possible or where >>>> it >>>> didn't seem feasible, but we tried to keep them few and obvious. >>>> Whenever this is the case, we are mirroring as close as possible >>>> what >>>> we >>>> have in jdk/jdk. >>>> >>>> Architecture-wise, Shenandoah runs on x86_64, aarch64, x86_32. >>>> Other >>>> architectures still build fine, and Shenandoah gets automatically >>>> disabled during configure if not supported. OS-wise, Shenandoah >>>> runs >>>> on >>>> Linux, Windows, and is known to run on MacOS X and Solaris too. >>>> >>>> I wasn't sure which form this should take. I decided to start >>>> with >>>> the >>>> easiest possible form this could take: a simple patch/webrev and >>>> an >>>> RFR. >>>> Let me know if you need a JIRA issue or any other formalities to >>>> track that. >>>> >>>> Full webrev: >>>> https://nam06.safelinks.protection.outlook.com/?url=https:%2F%2Fcr.openjdk.java.net%2F~rkennke%2Fshenandoah-jdk11u-upstream%2Fwebrev.02-all%2F&data=02%7C01%7Cadityam%40microsoft.com%7C7aec62734d22458243d908d818f974c5%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637286806967026872&sdata=eHspbGt5Eox4EMv8hstTBwsqFYefGNO2acGymP7ZfN0%3D&reserved=0 >>>> >>>> Webrev only for shared-code changes: >>>> https://nam06.safelinks.protection.outlook.com/?url=https:%2F%2Fcr.openjdk.java.net%2F~rkennke%2Fshenandoah-jdk11u-upstream%2Fwebrev.02-shared%2F&data=02%7C01%7Cadityam%40microsoft.com%7C7aec62734d22458243d908d818f974c5%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637286806967026872&sdata=LGghbNRyHPSJLK8l2Oyhxh1lhD46LmvZtN1G5htYBpk%3D&reserved=0 >>>> >>>> The webrevs are based on and depend on the arraycopy patch >>>> proposed >>>> for >>>> review here: >>>> >>>> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmail.openjdk.java.net%2Fpipermail%2Fjdk-updates-dev%2F2020-January%2F002396.html&data=02%7C01%7Cadityam%40microsoft.com%7C7aec62734d22458243d908d818f974c5%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637286806967026872&sdata=Kw0qkfvtVNVrCiMeypwr4BHX9e3bZafJnVCMVY%2BvOOw%3D&reserved=0 >>>> >>>> Testing: >>>> The code has been tested many many times, continuously while >>>> maintaining >>>> it. It has also served as the basis for RPMs and shipped to >>>> customers. >>>> We are running all sorts of tests of the hotspot/jtreg testsuite, >>>> in >>>> particular hotspot_gc_shenandoah, CTW tests, the regular tier* >>>> tests, >>>> several popular benchmarks (specjvm, specjbb on a regular basis), >>>> on >>>> a >>>> nightly basis. >>>> >>>> Please let me know what you think. >>>> >>>> Thanks, >>>> Roman >>>> >>>> >>>> >>>> >>>> >>>> >>>> From rkennke at redhat.com Wed Jul 8 21:58:16 2020 From: rkennke at redhat.com (Roman Kennke) Date: Wed, 08 Jul 2020 23:58:16 +0200 Subject: RFR (11u, XXL): Upstream/backport Shenandoah to JDK11u In-Reply-To: References: <3595e963-fa5b-bfce-c463-841b84d4103f@redhat.com> <6468CB08-3DB5-488E-9122-E3699A95218B@amazon.com> Message-ID: <33d0bd6339d4c9455c3309baa48e38e621fb9a79.camel@redhat.com> On Wed, 2020-07-08 at 21:39 +0000, Gil Tene wrote: > > On Jul 8, 2020, at 11:31 AM, Roman Kennke > > wrote: > > > > Hi Gil & all, > > > > I put in some extra work and had a deep look at the remaining > > unguarded > > changes in shared-code, with an eye to either eliminating them > > altogether, or - where this is not possible - guard them with > > build- > > time #if INCLUDE_SHENANDOAHGC or similar. The guiding principle in > > this > > effort has been that building without Shenandoah (--with-jvm- > > features=- > > shenandoahgc) would compile *the exact same* code that it does now. > > I > > believe I achieved this with the following proposed changes: > > > > Full webrev (incl. Shenandoah-only files): > > https://cr.openjdk.java.net/~rkennke/shenandoah-jdk11u-upstream/webrev.06-all/ > > > > Shared-code-only changes: > > https://cr.openjdk.java.net/~rkennke/shenandoah-jdk11u-upstream/webrev.06-shared/ > > > > The latter webrev exludes all files that are only compiled with > > Shenandoah enabled. > > > > Also, compared to previous webrevs, this excludes Shenandoah by > > default. > > > > As far as I can tell, this literally compiles the same code into > > libjvm.so as it does now, when building without Shenandoah. The > > only > > exception is the inclusion of the global flag UseShenandoahGC, > > which is > > by-design. When this is enabled in a build without Shenandoah, it > > prints the appropriate error message and exist (exactly like with > > any > > other GC that is not built-in). > > For the purpose of this thought exercise, I think we could consider > that flag > addition as a separate change (which would happen before all the > other > stuff). So that we can consider the "would compile *the exact same* > code" > statements in the context of a builds that already include the flag. > Ok. > > I've been thinking about if we can come up with a mechanical proof > > of > > correctness. The first best thing would have been reproducible > > builds, > > but we don't have that (and the UseShenandoahGC flag would mess it > > up > > too). The other option would be to dump preprocessed sources and > > diff > > them, but that is disturbed by the necessary inclusion of > > "utilities/macros.hpp" which gives us the INCLUDE_SHENANDOAHGC > > macro. > > I think that a mechanized way to verify the assumption will be key, > as you > will want it verified not just initially, but with every future > change. figuring > out how to do that is a key question... > > Yes, reproducible builds would be great, but we probably don't want > to > hold our breath for that. Maybe something around comparing individual > object file contents that go into the final link can work? > Maybe. I'm not familiar enough with object file formats to see quickly how to do that, or if it's even feasibible > > Which leaves us with carefully inspecting the webrev by multiple > > sets > > of eyes and brains. > > Eye and brain inspection is unlikely to catch innocent looking > "leaks" in > this model 6 months after the initial merge effort is completed, when > someone touches a line on the wrong side of an ifdef, or makes a > change to the scope of an ifdef or to what turns it on it. > > I think that an ongoing means of verification is needed as part of a > solution like this for it to be viable maintainable. I'm not sure > what > that would look like. But it may be worth thinking up some > techniques. Sorry I don't understand. Why is the usual practices of backporting, reviewing, testing for jdk11u etc not good enough to police ongoing work? >From our long experience with backporting both non-Shenandoah and Shenandoah stuff to 11u, we know that they rarely, if ever, come into conflict with each other, and it's usually quite clear what is what. > I'm also not sure how one would police future code changes and > backports of fixes from usptream versions, unless each and every > review of code carefully takes into consideration the "is this > Shenandoah > related or not? Should it go in ifdefs or not?" questions. Easy: if the change has 'gc-shenandoah' label in the bug report, it's Shenandoah, otherwise it's not. :-) We are very careful to get that part right for our own tracking. Shared-code is very rarely touched by Shenandoah changes anyway. > Can we come up with systemic ways to know that a code change being > backported is Shenandoah-related? E.g. if the upstream code shape and > ifdef'ing rules remained similar to the downstream, The ifdef'ing rules are already different: there are (almost) no such #ifdefs in later JDKs, because it's all isolated by the proper GC interfaces. Most of those exist in 11u, but not all, that's why we had to put those #ifs in the patch. Some stuff has been accepted as general change in later JDKs (e.g. the stuff in ciInstanceKlass.cpp), it appears as Shenandoah change here, because we are the only users of it. > we may be able to > keep things workable, but I suspect trhat the upstream (especially > after > 15u) will be tempted to remove the strict ifdef'ing pretty quickly, > and start > placing code changes in common code, and as those changes will likely > be needed in backporting gixes to the "Shenandoah enabled 11u" mode, > carefully examination on a case by case basis will be needed, at > least as > long as "no shenandoah exists" downstreams of 11u still depend on it > for updates. Sorry, I still don't see this as a problem. Careful review of backports, looking at the gc-shenandoah tag in bug-reports, testing of builds both with and without Shenandoah, that should cover it? > > Gil, would the change like this ease your concerns? > > If we could provably show that no code differences exist in builds > that > do no choose to include the change, it would be hard for me to argue > against upstream inclusion based on it having a potential for > breaking > a downstream distro's stability, no matte rhow neccesary or not it > is? > > So yes, the proposed approach could work if we can figure out > the technical ways to achieve it and "keep it achieved" for some time > into the future. > > We will likely need to be able to mechanically verify that things > remain > like this even as code is introduced in the future. Manual review > cannot > achieve that alone probably, so we will likely need some build > tooling > to verify this every time. Another problem with that is this: we can probably mechanically prove it now, because we have the vanilla 11u build to compare against. Once Shenandoah is integrated, what would you compare the -shenandoahgc build against? In other words: what exactly is it that you want to prove then? I don't understand it. Cheers, Roman From rkennke at redhat.com Wed Jul 8 23:07:15 2020 From: rkennke at redhat.com (Roman Kennke) Date: Thu, 09 Jul 2020 01:07:15 +0200 Subject: RFR (11u, XXL): Upstream/backport Shenandoah to JDK11u In-Reply-To: <4F72AD88-0165-497B-B32A-212A5AB076F0@azul.com> References: <3595e963-fa5b-bfce-c463-841b84d4103f@redhat.com> <6468CB08-3DB5-488E-9122-E3699A95218B@amazon.com> <33d0bd6339d4c9455c3309baa48e38e621fb9a79.camel@redhat.com> <4F72AD88-0165-497B-B32A-212A5AB076F0@azul.com> Message-ID: On Wed, 2020-07-08 at 22:35 +0000, Gil Tene wrote: > > > > On Jul 8, 2020, at 2:58 PM, Roman Kennke > > wrote: > > > > On Wed, 2020-07-08 at 21:39 +0000, Gil Tene wrote: > > > > On Jul 8, 2020, at 11:31 AM, Roman Kennke > > > > wrote: > > > > > > > > Hi Gil & all, > > > > > > > > I put in some extra work and had a deep look at the remaining > > > > unguarded > > > > changes in shared-code, with an eye to either eliminating them > > > > altogether, or - where this is not possible - guard them with > > > > build- > > > > time #if INCLUDE_SHENANDOAHGC or similar. The guiding principle > > > > in > > > > this > > > > effort has been that building without Shenandoah (--with-jvm- > > > > features=- > > > > shenandoahgc) would compile *the exact same* code that it does > > > > now. > > > > I > > > > believe I achieved this with the following proposed changes: > > > > > > > > Full webrev (incl. Shenandoah-only files): > > > > https://cr.openjdk.java.net/~rkennke/shenandoah-jdk11u-upstream/webrev.06-all/ > > > > > > > > Shared-code-only changes: > > > > https://cr.openjdk.java.net/~rkennke/shenandoah-jdk11u-upstream/webrev.06-shared/ > > > > > > > > The latter webrev exludes all files that are only compiled with > > > > Shenandoah enabled. > > > > > > > > Also, compared to previous webrevs, this excludes Shenandoah by > > > > default. > > > > > > > > As far as I can tell, this literally compiles the same code > > > > into > > > > libjvm.so as it does now, when building without Shenandoah. The > > > > only > > > > exception is the inclusion of the global flag UseShenandoahGC, > > > > which is > > > > by-design. When this is enabled in a build without Shenandoah, > > > > it > > > > prints the appropriate error message and exist (exactly like > > > > with > > > > any > > > > other GC that is not built-in). > > > > > > For the purpose of this thought exercise, I think we could > > > consider > > > that flag > > > addition as a separate change (which would happen before all the > > > other > > > stuff). So that we can consider the "would compile *the exact > > > same* > > > code" > > > statements in the context of a builds that already include the > > > flag. > > > > > > > Ok. > > > > > > I've been thinking about if we can come up with a mechanical > > > > proof > > > > of > > > > correctness. The first best thing would have been reproducible > > > > builds, > > > > but we don't have that (and the UseShenandoahGC flag would mess > > > > it > > > > up > > > > too). The other option would be to dump preprocessed sources > > > > and > > > > diff > > > > them, but that is disturbed by the necessary inclusion of > > > > "utilities/macros.hpp" which gives us the INCLUDE_SHENANDOAHGC > > > > macro. > > > > > > I think that a mechanized way to verify the assumption will be > > > key, > > > as you > > > will want it verified not just initially, but with every future > > > change. figuring > > > out how to do that is a key question... > > > > > > Yes, reproducible builds would be great, but we probably don't > > > want > > > to > > > hold our breath for that. Maybe something around comparing > > > individual > > > object file contents that go into the final link can work? > > > > > > > Maybe. I'm not familiar enough with object file formats to see > > quickly > > how to do that, or if it's even feasibible > > > > > > Which leaves us with carefully inspecting the webrev by > > > > multiple > > > > sets > > > > of eyes and brains. > > > > > > Eye and brain inspection is unlikely to catch innocent looking > > > "leaks" in > > > this model 6 months after the initial merge effort is completed, > > > when > > > someone touches a line on the wrong side of an ifdef, or makes a > > > change to the scope of an ifdef or to what turns it on it. > > > > > > I think that an ongoing means of verification is needed as part > > > of a > > > solution like this for it to be viable maintainable. I'm not sure > > > what > > > that would look like. But it may be worth thinking up some > > > techniques. > > > > Sorry I don't understand. Why is the usual practices of > > backporting, > > reviewing, testing for jdk11u etc not good enough to police ongoing > > work? > > > > From our long experience with backporting both non-Shenandoah and > > Shenandoah stuff to 11u, we know that they rarely, if ever, come > > into > > conflict with each other, and it's usually quite clear what is > > what. > > > > > I'm also not sure how one would police future code changes and > > > backports of fixes from usptream versions, unless each and every > > > review of code carefully takes into consideration the "is this > > > Shenandoah > > > related or not? Should it go in ifdefs or not?" questions. > > > > Easy: if the change has 'gc-shenandoah' label in the bug report, > > it's > > Shenandoah, otherwise it's not. :-) We are very careful to get that > > part right for our own tracking. Shared-code is very rarely touched > > by > > Shenandoah changes anyway. > > > > > Can we come up with systemic ways to know that a code change > > > being > > > backported is Shenandoah-related? E.g. if the upstream code shape > > > and > > > ifdef'ing rules remained similar to the downstream, > > > > The ifdef'ing rules are already different: there are (almost) no > > such > > #ifdefs in later JDKs, because it's all isolated by the proper GC > > interfaces. Most of those exist in 11u, but not all, that's why we > > had > > to put those #ifs in the patch. Some stuff has been accepted as > > general > > change in later JDKs (e.g. the stuff in ciInstanceKlass.cpp), it > > appears as Shenandoah change here, because we are the only users of > > it. > > > > > we may be able to > > > keep things workable, but I suspect trhat the upstream > > > (especially > > > after > > > 15u) will be tempted to remove the strict ifdef'ing pretty > > > quickly, > > > and start > > > placing code changes in common code, and as those changes will > > > likely > > > be needed in backporting gixes to the "Shenandoah enabled 11u" > > > mode, > > > carefully examination on a case by case basis will be needed, at > > > least as > > > long as "no shenandoah exists" downstreams of 11u still depend on > > > it > > > for updates. > > > > Sorry, I still don't see this as a problem. Careful review of > > backports, looking at the gc-shenandoah tag in bug-reports, testing > > of > > builds both with and without Shenandoah, that should cover it? > > > > > > Gil, would the change like this ease your concerns? > > > > > > If we could provably show that no code differences exist in > > > builds > > > that > > > do no choose to include the change, it would be hard for me to > > > argue > > > against upstream inclusion based on it having a potential for > > > breaking > > > a downstream distro's stability, no matte rhow neccesary or not > > > it > > > is? > > > > > > So yes, the proposed approach could work if we can figure out > > > the technical ways to achieve it and "keep it achieved" for some > > > time > > > into the future. > > > > > > We will likely need to be able to mechanically verify that things > > > remain > > > like this even as code is introduced in the future. Manual review > > > cannot > > > achieve that alone probably, so we will likely need some build > > > tooling > > > to verify this every time. > > > > Another problem with that is this: we can probably mechanically > > prove > > it now, because we have the vanilla 11u build to compare against. > > Once > > Shenandoah is integrated, what would you compare the -shenandoahgc > > build against? > > > > In other words: what exactly is it that you want to prove then? I > > don't > > understand it. > > I'm not sure I understand it either. ;-) Haha, ok right :-) > I'm basically asking the "if we prove it now", how do we know that > the > same holds later?" question. > > Delaying some big impact by one update is clearly not the goal you > are proposing. I don't understand this sentence (but it's quite late here...) > I believe we agree that the quality you suggest would need to be > retained into future > builds. Having nothing to prevent it from breaking (e.g. by accident) > is probably > dangerous. So the question becomes "how do we ask the question of > whether or not > we still have not introduced any differences?" in future updates. Ok. > Let's start from the assumption that we can prove something now (and > I > do think we should be able to find a mechanical thing to do that). > That will show > that the quality we seek to keep has been kept at a point in time... Yes. Let's assume that. (As an aside, if you'd actually look at the patch, I hope you'd find it quite obvious that it does the right thing. And you might be surprised how relatively few such changes we actually have.) > For as long as the statement of "without Shenandoah enabled, the > build > does not have any actual code bits related to the Shenandoah back- > port" > needs to hold, we need a way to show that the same still holds, > Either by > proving that the original proof still holds in the presence of > changes, or > by re-proving it somehow. > Let me elaborate a little bit on my experiences with backports, and mixing both Shenandoah and non-Shenandoah backports. In the many months since we maintain Shenandoah in our 11u, the overhelming majority of Shenandoah-related backports did not touch any shared code at all. That's because the existing GC interfaces isolate GCs really well in 11u. The few occasions where we did change shared- code was 1. When we switched to LRB. We would not usually backport such drastic changes, but it seemed prudent here, because it did actually decrease the shared-code exposure drastically. 2. in preparation for this upstreaming patch - more pruning and isolation of shared-code changes. In the very rare situations where a backport would require shared-code changes (I can't actually remember any, apart from the two that I just mentioned), we carefully consider if it's actually necessary to backport. For example, we have not backported concurrent class- unloading support to 11u precisely because it would require (significant) changes outside of Shenandoah. *If* a critical backport (say, a bugfix) requires changes outside of Shenandoah, it would have to be properly guarded by the same means as we do in the proposed upstreaming patch. We - the Shenandoah team - would be aware of that and mention it in relevant reviews. It would also prominently show up in a patch because it has files without 'shenandoah' in their path names. And from there, it's a matter of carefully considering, reviewing and testing it. I don't think this would silently sneak in somehow. I can't think of a situation where we ever had the reverse problem: a shared-code change touching on something Shenandoah-related. Also, while we did have a flurry of Shenandoah-related backports in the past, because of stabilization and new features (e.g. LRB and friends), this is most likely not going to continue into the future. I expect less Shenandoah-backporting traffic, basically limited to bugfixes and improvements that don't touch shared-code. We have a couple of features on our todo list for later JDKs, but none of them sound like candidates for backporting. > The notion that our normal review processes will catch everything > that can break > that initial prooved state seems a bit optimistic to me. The review > process will be > well intentioned, and we'll try to tag things right, but one mistaken > application or > move of code across ifdef lines, or integration of some mis-tagged or > unnoticed tag > thing into shared code will break the statement? > How is that any different from the situation that we already have? Hotspot code already has similar #ifdefs sprinkled all around, e.g. for other GCs, for JFR, platform #ifdefs, and so on. How is the situation different for Shenandoah, why do we need special rules or process or even proofs for that? As far as I can see, our current high-standard development and review practices already cover it very well. > I believe that we can avoid this once, at the start, with a > mechanical proof. Can > we keep it up? What sort fo rules or invariants can we come up with > that we can > use to verify and show that the quality we seek to keep has been kept > through later > updates? Well yes, the usual proper care that everybody is taking in making the backports, reviewing the backports, testing it, etc. > Let's try to think of some?. As long as we don't know what the exact question/problem even might be, we can hardly come up with an answer/solution. Roman From gil at azul.com Thu Jul 9 04:53:13 2020 From: gil at azul.com (Gil Tene) Date: Thu, 9 Jul 2020 04:53:13 +0000 Subject: RFR (11u, XXL): Upstream/backport Shenandoah to JDK11u In-Reply-To: References: <3595e963-fa5b-bfce-c463-841b84d4103f@redhat.com> <6468CB08-3DB5-488E-9122-E3699A95218B@amazon.com> <33d0bd6339d4c9455c3309baa48e38e621fb9a79.camel@redhat.com> <4F72AD88-0165-497B-B32A-212A5AB076F0@azul.com> Message-ID: <7959350D-EABE-461E-90AB-167214C55D32@azul.com> > On Jul 8, 2020, at 4:07 PM, Roman Kennke wrote: > > On Wed, 2020-07-08 at 22:35 +0000, Gil Tene wrote: >> >> >>> On Jul 8, 2020, at 2:58 PM, Roman Kennke >>> wrote: >>> >>> ... >>> In other words: what exactly is it that you want to prove then? I >>> don't >>> understand it. >> >> I'm not sure I understand it either. ;-) > > Haha, ok right :-) > >> I'm basically asking the "if we prove it now", how do we know that >> the >> same holds later?" question. >> >> Delaying some big impact by one update is clearly not the goal you >> are proposing. > > I don't understand this sentence (but it's quite late here...) > >> I believe we agree that the quality you suggest would need to be >> retained into future >> builds. Having nothing to prevent it from breaking (e.g. by accident) >> is probably >> dangerous. So the question becomes "how do we ask the question of >> whether or not >> we still have not introduced any differences?" in future updates. > > Ok. > >> Let's start from the assumption that we can prove something now (and >> I >> do think we should be able to find a mechanical thing to do that). >> That will show >> that the quality we seek to keep has been kept at a point in time... > > Yes. Let's assume that. (As an aside, if you'd actually look at the > patch, I hope you'd find it quite obvious that it does the right thing. > And you might be surprised how relatively few such changes we actually > have.) > >> For as long as the statement of "without Shenandoah enabled, the >> build >> does not have any actual code bits related to the Shenandoah back- >> port" >> needs to hold, we need a way to show that the same still holds, >> Either by >> proving that the original proof still holds in the presence of >> changes, or >> by re-proving it somehow. >> > > Let me elaborate a little bit on my experiences with backports, and > mixing both Shenandoah and non-Shenandoah backports. > > In the many months since we maintain Shenandoah in our 11u, the > overhelming majority of Shenandoah-related backports did not touch any > shared code at all. That's because the existing GC interfaces isolate > GCs really well in 11u. The few occasions where we did change shared- > code was 1. When we switched to LRB. We would not usually backport such > drastic changes, but it seemed prudent here, because it did actually > decrease the shared-code exposure drastically. 2. in preparation for > this upstreaming patch - more pruning and isolation of shared-code > changes. > > In the very rare situations where a backport would require shared-code > changes (I can't actually remember any, apart from the two that I just > mentioned), we carefully consider if it's actually necessary to > backport. For example, we have not backported concurrent class- > unloading support to 11u precisely because it would require > (significant) changes outside of Shenandoah. *If* a critical backport > (say, a bugfix) requires changes outside of Shenandoah, it would have > to be properly guarded by the same means as we do in the proposed > upstreaming patch. We - the Shenandoah team - would be aware of that > and mention it in relevant reviews. It would also prominently show up > in a patch because it has files without 'shenandoah' in their path > names. And from there, it's a matter of carefully considering, > reviewing and testing it. I don't think this would silently sneak in > somehow. > > I can't think of a situation where we ever had the reverse problem: a > shared-code change touching on something Shenandoah-related. > > Also, while we did have a flurry of Shenandoah-related backports in the > past, because of stabilization and new features (e.g. LRB and friends), > this is most likely not going to continue into the future. I expect > less Shenandoah-backporting traffic, basically limited to bugfixes and > improvements that don't touch shared-code. We have a couple of features > on our todo list for later JDKs, but none of them sound like candidates > for backporting. > >> The notion that our normal review processes will catch everything >> that can break >> that initial prooved state seems a bit optimistic to me. The review >> process will be >> well intentioned, and we'll try to tag things right, but one mistaken >> application or >> move of code across ifdef lines, or integration of some mis-tagged or >> unnoticed tag >> thing into shared code will break the statement? >> > > How is that any different from the situation that we already have? > Hotspot code already has similar #ifdefs sprinkled all around, e.g. for > other GCs, for JFR, platform #ifdefs, and so on. How is the situation > different for Shenandoah, why do we need special rules or process or > even proofs for that? As far as I can see, our current high-standard > development and review practices already cover it very well. > >> I believe that we can avoid this once, at the start, with a >> mechanical proof. Can >> we keep it up? What sort fo rules or invariants can we come up with >> that we can >> use to verify and show that the quality we seek to keep has been kept >> through later >> updates? > > Well yes, the usual proper care that everybody is taking in making the > backports, reviewing the backports, testing it, etc. > >> Let's try to think of some?. > > As long as we don't know what the exact question/problem even might be, > we can hardly come up with an answer/solution. I'm not trying to be difficult here. I'm just going with the basic line of logic, and looking for a way to implement it. IF we can PROVE that the actual code built is not changed from the prior-to-this-change code when built without (--with-jvm-features=- shenandoahgc), THEN there is no basis to argue that the code change creates any form of change-related impact for existing users and distros as they update to inherit new upstream changes for bug fixes and security updates. I agree with that statement. And IF we can make it true, I must obviously remove my reservations to adding a new collector to trhe 11u upstream for use in distros that want to build it in, as necessity no longer comes into play in the argument. We would not have consider the change from the point of view of other distros build without (--with-jvm-features=-shenandoahgc), and of their users: we would have PROVEN that they are not affected in any way. That is the key point of agreement, and a powerful argument. It is a way to remove my original concerns. To build to something useful from there, I'm looking to see how we can both establish the "no change" situation (prove it), and to maintain that situation as true over time (prove it for the next update). Unless we can prove it for the next update, all we would have done is delay the feared impact to other downstream distros and users by one quarter. And that would not remove any of the necessity-based concerns. If there is a viable proof for the stating point, it seems like there should be a way to continue proving the same over time. And I think we should look for some technique that would provide that. Various "we will be very careful after the first change" arguments, without a means to prove things, don't provide more comfort for the same reason the "we have been very careful" ones for the original change don't: without a proof mechanism that shows that there is no change, we go back to the original "change in an LTS requires a level of necessity" argument. As for the difference from other things we review when doing backports into LTSs: I don't know of other examples where we have tried to prove that something has zero changes in code in order to allow it in over concerns, let alone continually review to maintain that zero changes statement. It's new territory. I did not propose the proof logic as basis for inclusion regardless of necessity, I just accept it as a possible path, and I'm going with the "what if we could prove it?" logic to see if we can make it work. If we switch back to "but we can't prove it" or "we shouldn't need to prove it" in the middle of looking for that, then that just brings me back to my original concerns. ? Gil. From rkennke at redhat.com Thu Jul 9 05:57:27 2020 From: rkennke at redhat.com (Roman Kennke) Date: Thu, 09 Jul 2020 07:57:27 +0200 Subject: RFR (11u, XXL): Upstream/backport Shenandoah to JDK11u In-Reply-To: <7959350D-EABE-461E-90AB-167214C55D32@azul.com> References: <3595e963-fa5b-bfce-c463-841b84d4103f@redhat.com> <6468CB08-3DB5-488E-9122-E3699A95218B@amazon.com> <33d0bd6339d4c9455c3309baa48e38e621fb9a79.camel@redhat.com> <4F72AD88-0165-497B-B32A-212A5AB076F0@azul.com> <7959350D-EABE-461E-90AB-167214C55D32@azul.com> Message-ID: <39b7b9292d64cfc77aa3b6ad48e5d922a4ceb5b4.camel@redhat.com> On Thu, 2020-07-09 at 04:53 +0000, Gil Tene wrote: > > On Jul 8, 2020, at 4:07 PM, Roman Kennke > > wrote: > > > > On Wed, 2020-07-08 at 22:35 +0000, Gil Tene wrote: > > > > > > > On Jul 8, 2020, at 2:58 PM, Roman Kennke > > > > wrote: > > > > > > > > ... > > > > In other words: what exactly is it that you want to prove then? > > > > I > > > > don't > > > > understand it. > > > > > > I'm not sure I understand it either. ;-) > > > > Haha, ok right :-) > > > > > I'm basically asking the "if we prove it now", how do we know > > > that > > > the > > > same holds later?" question. > > > > > > Delaying some big impact by one update is clearly not the goal > > > you > > > are proposing. > > > > I don't understand this sentence (but it's quite late here...) > > > > > I believe we agree that the quality you suggest would need to be > > > retained into future > > > builds. Having nothing to prevent it from breaking (e.g. by > > > accident) > > > is probably > > > dangerous. So the question becomes "how do we ask the question of > > > whether or not > > > we still have not introduced any differences?" in future updates. > > > > Ok. > > > > > Let's start from the assumption that we can prove something now > > > (and > > > I > > > do think we should be able to find a mechanical thing to do > > > that). > > > That will show > > > that the quality we seek to keep has been kept at a point in > > > time... > > > > Yes. Let's assume that. (As an aside, if you'd actually look at the > > patch, I hope you'd find it quite obvious that it does the right > > thing. > > And you might be surprised how relatively few such changes we > > actually > > have.) > > > > > For as long as the statement of "without Shenandoah enabled, the > > > build > > > does not have any actual code bits related to the Shenandoah > > > back- > > > port" > > > needs to hold, we need a way to show that the same still holds, > > > Either by > > > proving that the original proof still holds in the presence of > > > changes, or > > > by re-proving it somehow. > > > > > > > Let me elaborate a little bit on my experiences with backports, and > > mixing both Shenandoah and non-Shenandoah backports. > > > > In the many months since we maintain Shenandoah in our 11u, the > > overhelming majority of Shenandoah-related backports did not touch > > any > > shared code at all. That's because the existing GC interfaces > > isolate > > GCs really well in 11u. The few occasions where we did change > > shared- > > code was 1. When we switched to LRB. We would not usually backport > > such > > drastic changes, but it seemed prudent here, because it did > > actually > > decrease the shared-code exposure drastically. 2. in preparation > > for > > this upstreaming patch - more pruning and isolation of shared-code > > changes. > > > > In the very rare situations where a backport would require shared- > > code > > changes (I can't actually remember any, apart from the two that I > > just > > mentioned), we carefully consider if it's actually necessary to > > backport. For example, we have not backported concurrent class- > > unloading support to 11u precisely because it would require > > (significant) changes outside of Shenandoah. *If* a critical > > backport > > (say, a bugfix) requires changes outside of Shenandoah, it would > > have > > to be properly guarded by the same means as we do in the proposed > > upstreaming patch. We - the Shenandoah team - would be aware of > > that > > and mention it in relevant reviews. It would also prominently show > > up > > in a patch because it has files without 'shenandoah' in their path > > names. And from there, it's a matter of carefully considering, > > reviewing and testing it. I don't think this would silently sneak > > in > > somehow. > > > > I can't think of a situation where we ever had the reverse problem: > > a > > shared-code change touching on something Shenandoah-related. > > > > Also, while we did have a flurry of Shenandoah-related backports in > > the > > past, because of stabilization and new features (e.g. LRB and > > friends), > > this is most likely not going to continue into the future. I expect > > less Shenandoah-backporting traffic, basically limited to bugfixes > > and > > improvements that don't touch shared-code. We have a couple of > > features > > on our todo list for later JDKs, but none of them sound like > > candidates > > for backporting. > > > > > The notion that our normal review processes will catch everything > > > that can break > > > that initial prooved state seems a bit optimistic to me. The > > > review > > > process will be > > > well intentioned, and we'll try to tag things right, but one > > > mistaken > > > application or > > > move of code across ifdef lines, or integration of some mis- > > > tagged or > > > unnoticed tag > > > thing into shared code will break the statement? > > > > > > > How is that any different from the situation that we already have? > > Hotspot code already has similar #ifdefs sprinkled all around, e.g. > > for > > other GCs, for JFR, platform #ifdefs, and so on. How is the > > situation > > different for Shenandoah, why do we need special rules or process > > or > > even proofs for that? As far as I can see, our current high- > > standard > > development and review practices already cover it very well. > > > > > I believe that we can avoid this once, at the start, with a > > > mechanical proof. Can > > > we keep it up? What sort fo rules or invariants can we come up > > > with > > > that we can > > > use to verify and show that the quality we seek to keep has been > > > kept > > > through later > > > updates? > > > > Well yes, the usual proper care that everybody is taking in making > > the > > backports, reviewing the backports, testing it, etc. > > > > > Let's try to think of some?. > > > > As long as we don't know what the exact question/problem even might > > be, > > we can hardly come up with an answer/solution. > > I'm not trying to be difficult here. I'm just going with the basic > line > of logic, and looking for a way to implement it. > Yeah. Let me just state for the record, that I find this is getting slightly ridiculous ;-) Let me try anyway. It looks to me that this is a prime example for proof-by-induction. Let's try to formulate it. State 0 is our current jdk11u state. Shenandoah does obviously not leak into the build, because it's not included yet. State 1 is the state after the initial inclusion of Shenandoah. Let's assume we can prove that between 0 and 1, nothing leaks into the build when building with --with-jvm-features=-shenandoahgc. That proof will have to be determined, maybe comparing object files would work. Then, for any further changeset to be backported, this would be the N- >N+1 case: - if the changeset is not Shenandoah-related, it'll obviously change the outcome of the build, but also obviously doesn't leak any Shenandoah-related changes. - otherwise, if the changeset is Shenandoah-related, we can run the same proof that we did from 0->1 for N->N+1, and proof that no additional Shenandoah-related changes leaks into non-Shenandoah build. Right? It depends on the correct classification what constitutes a Shenandoah- related change and what doesn't. But it must, I see no way around that. >From my perspective and in my experience, this is really easy though, and can be achieved by applying some common sense. (Hey, when reviewing, you really ought to look at the bug - and spot gc-shenandoah label - and also look at the patch and understand what it all does, and come to some conclusion.) Now, about that proof: I will spend the day looking if we can do it by comparing object files of builds. E.g. do builds (-shenandoahgc) before/after a change is applied, run checksums over each object file, and compare those checksums. Let's see if there's any pitfalls there. What do you think? Roman From goetz.lindenmaier at sap.com Thu Jul 9 09:40:06 2020 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Thu, 9 Jul 2020 09:40:06 +0000 Subject: RFR (11u, XXL): Upstream/backport Shenandoah to JDK11u In-Reply-To: <39b7b9292d64cfc77aa3b6ad48e5d922a4ceb5b4.camel@redhat.com> References: <3595e963-fa5b-bfce-c463-841b84d4103f@redhat.com> <6468CB08-3DB5-488E-9122-E3699A95218B@amazon.com> <33d0bd6339d4c9455c3309baa48e38e621fb9a79.camel@redhat.com> <4F72AD88-0165-497B-B32A-212A5AB076F0@azul.com> <7959350D-EABE-461E-90AB-167214C55D32@azul.com> <39b7b9292d64cfc77aa3b6ad48e5d922a4ceb5b4.camel@redhat.com> Message-ID: Hi Roman, What about creating a webrev with those changes that will be compiled if you configure without shenandoahgc? I know there is a webrev with only the shared changes, but they contain a lot of #define coding, or such under protection by the flag that enables Shenandoah, which should be constant 'false' if Shenandoah is disabled, right? As I read the code, there should remain only a few. Then we can easily assess the actual risk. If something appears risky for the existing code, we can guard it by #ifdefs. Best regards, Goetz. > -----Original Message----- > From: jdk-updates-dev On Behalf > Of Roman Kennke > Sent: Thursday, July 9, 2020 7:57 AM > To: Gil Tene > Cc: Bernd Mathiske ; jdk-updates- > dev at openjdk.java.net; Nilsen, Kelvin ; Jiva, Azeem > > Subject: Re: RFR (11u, XXL): Upstream/backport Shenandoah to JDK11u > > On Thu, 2020-07-09 at 04:53 +0000, Gil Tene wrote: > > > On Jul 8, 2020, at 4:07 PM, Roman Kennke > > > wrote: > > > > > > On Wed, 2020-07-08 at 22:35 +0000, Gil Tene wrote: > > > > > > > > > On Jul 8, 2020, at 2:58 PM, Roman Kennke > > > > > wrote: > > > > > > > > > > ... > > > > > In other words: what exactly is it that you want to prove then? > > > > > I > > > > > don't > > > > > understand it. > > > > > > > > I'm not sure I understand it either. ;-) > > > > > > Haha, ok right :-) > > > > > > > I'm basically asking the "if we prove it now", how do we know > > > > that > > > > the > > > > same holds later?" question. > > > > > > > > Delaying some big impact by one update is clearly not the goal > > > > you > > > > are proposing. > > > > > > I don't understand this sentence (but it's quite late here...) > > > > > > > I believe we agree that the quality you suggest would need to be > > > > retained into future > > > > builds. Having nothing to prevent it from breaking (e.g. by > > > > accident) > > > > is probably > > > > dangerous. So the question becomes "how do we ask the question of > > > > whether or not > > > > we still have not introduced any differences?" in future updates. > > > > > > Ok. > > > > > > > Let's start from the assumption that we can prove something now > > > > (and > > > > I > > > > do think we should be able to find a mechanical thing to do > > > > that). > > > > That will show > > > > that the quality we seek to keep has been kept at a point in > > > > time... > > > > > > Yes. Let's assume that. (As an aside, if you'd actually look at the > > > patch, I hope you'd find it quite obvious that it does the right > > > thing. > > > And you might be surprised how relatively few such changes we > > > actually > > > have.) > > > > > > > For as long as the statement of "without Shenandoah enabled, the > > > > build > > > > does not have any actual code bits related to the Shenandoah > > > > back- > > > > port" > > > > needs to hold, we need a way to show that the same still holds, > > > > Either by > > > > proving that the original proof still holds in the presence of > > > > changes, or > > > > by re-proving it somehow. > > > > > > > > > > Let me elaborate a little bit on my experiences with backports, and > > > mixing both Shenandoah and non-Shenandoah backports. > > > > > > In the many months since we maintain Shenandoah in our 11u, the > > > overhelming majority of Shenandoah-related backports did not touch > > > any > > > shared code at all. That's because the existing GC interfaces > > > isolate > > > GCs really well in 11u. The few occasions where we did change > > > shared- > > > code was 1. When we switched to LRB. We would not usually backport > > > such > > > drastic changes, but it seemed prudent here, because it did > > > actually > > > decrease the shared-code exposure drastically. 2. in preparation > > > for > > > this upstreaming patch - more pruning and isolation of shared-code > > > changes. > > > > > > In the very rare situations where a backport would require shared- > > > code > > > changes (I can't actually remember any, apart from the two that I > > > just > > > mentioned), we carefully consider if it's actually necessary to > > > backport. For example, we have not backported concurrent class- > > > unloading support to 11u precisely because it would require > > > (significant) changes outside of Shenandoah. *If* a critical > > > backport > > > (say, a bugfix) requires changes outside of Shenandoah, it would > > > have > > > to be properly guarded by the same means as we do in the proposed > > > upstreaming patch. We - the Shenandoah team - would be aware of > > > that > > > and mention it in relevant reviews. It would also prominently show > > > up > > > in a patch because it has files without 'shenandoah' in their path > > > names. And from there, it's a matter of carefully considering, > > > reviewing and testing it. I don't think this would silently sneak > > > in > > > somehow. > > > > > > I can't think of a situation where we ever had the reverse problem: > > > a > > > shared-code change touching on something Shenandoah-related. > > > > > > Also, while we did have a flurry of Shenandoah-related backports in > > > the > > > past, because of stabilization and new features (e.g. LRB and > > > friends), > > > this is most likely not going to continue into the future. I expect > > > less Shenandoah-backporting traffic, basically limited to bugfixes > > > and > > > improvements that don't touch shared-code. We have a couple of > > > features > > > on our todo list for later JDKs, but none of them sound like > > > candidates > > > for backporting. > > > > > > > The notion that our normal review processes will catch everything > > > > that can break > > > > that initial prooved state seems a bit optimistic to me. The > > > > review > > > > process will be > > > > well intentioned, and we'll try to tag things right, but one > > > > mistaken > > > > application or > > > > move of code across ifdef lines, or integration of some mis- > > > > tagged or > > > > unnoticed tag > > > > thing into shared code will break the statement? > > > > > > > > > > How is that any different from the situation that we already have? > > > Hotspot code already has similar #ifdefs sprinkled all around, e.g. > > > for > > > other GCs, for JFR, platform #ifdefs, and so on. How is the > > > situation > > > different for Shenandoah, why do we need special rules or process > > > or > > > even proofs for that? As far as I can see, our current high- > > > standard > > > development and review practices already cover it very well. > > > > > > > I believe that we can avoid this once, at the start, with a > > > > mechanical proof. Can > > > > we keep it up? What sort fo rules or invariants can we come up > > > > with > > > > that we can > > > > use to verify and show that the quality we seek to keep has been > > > > kept > > > > through later > > > > updates? > > > > > > Well yes, the usual proper care that everybody is taking in making > > > the > > > backports, reviewing the backports, testing it, etc. > > > > > > > Let's try to think of some?. > > > > > > As long as we don't know what the exact question/problem even might > > > be, > > > we can hardly come up with an answer/solution. > > > > I'm not trying to be difficult here. I'm just going with the basic > > line > > of logic, and looking for a way to implement it. > > > > Yeah. Let me just state for the record, that I find this is getting > slightly ridiculous ;-) > > Let me try anyway. > > It looks to me that this is a prime example for proof-by-induction. > Let's try to formulate it. > > State 0 is our current jdk11u state. Shenandoah does obviously not leak > into the build, because it's not included yet. > > State 1 is the state after the initial inclusion of Shenandoah. Let's > assume we can prove that between 0 and 1, nothing leaks into the build > when building with --with-jvm-features=-shenandoahgc. That proof will > have to be determined, maybe comparing object files would work. > > Then, for any further changeset to be backported, this would be the N- > >N+1 case: > > - if the changeset is not Shenandoah-related, it'll obviously change > the outcome of the build, but also obviously doesn't leak any > Shenandoah-related changes. > - otherwise, if the changeset is Shenandoah-related, we can run the > same proof that we did from 0->1 for N->N+1, and proof that no > additional Shenandoah-related changes leaks into non-Shenandoah build. > > Right? > > It depends on the correct classification what constitutes a Shenandoah- > related change and what doesn't. But it must, I see no way around that. > From my perspective and in my experience, this is really easy though, > and can be achieved by applying some common sense. (Hey, when > reviewing, you really ought to look at the bug - and spot gc-shenandoah > label - and also look at the patch and understand what it all does, and > come to some conclusion.) > > > Now, about that proof: I will spend the day looking if we can do it by > comparing object files of builds. E.g. do builds (-shenandoahgc) > before/after a change is applied, run checksums over each object file, > and compare those checksums. Let's see if there's any pitfalls there. > > What do you think? > > Roman > From neugens at redhat.com Thu Jul 9 09:44:00 2020 From: neugens at redhat.com (Mario Torre) Date: Thu, 9 Jul 2020 11:44:00 +0200 Subject: RFR (11u, XXL): Upstream/backport Shenandoah to JDK11u In-Reply-To: References: <3595e963-fa5b-bfce-c463-841b84d4103f@redhat.com> <6468CB08-3DB5-488E-9122-E3699A95218B@amazon.com> Message-ID: Hi Roman, Can you please start a new thread to allow for a proper review? (Whether approved by maintainer or not I think the patch will want a review, right?) We can continue this thread dedicated to the philosophical questions about life, universe and backports :) Cheers, Mario On Wed, Jul 8, 2020 at 8:32 PM Roman Kennke wrote: > > Hi Gil & all, > > I put in some extra work and had a deep look at the remaining unguarded > changes in shared-code, with an eye to either eliminating them > altogether, or - where this is not possible - guard them with build- > time #if INCLUDE_SHENANDOAHGC or similar. The guiding principle in this > effort has been that building without Shenandoah (--with-jvm-features=- > shenandoahgc) would compile *the exact same* code that it does now. I > believe I achieved this with the following proposed changes: > > Full webrev (incl. Shenandoah-only files): > https://cr.openjdk.java.net/~rkennke/shenandoah-jdk11u-upstream/webrev.06-all/ > > Shared-code-only changes: > https://cr.openjdk.java.net/~rkennke/shenandoah-jdk11u-upstream/webrev.06-shared/ > > The latter webrev exludes all files that are only compiled with > Shenandoah enabled. > > Also, compared to previous webrevs, this excludes Shenandoah by > default. > > As far as I can tell, this literally compiles the same code into > libjvm.so as it does now, when building without Shenandoah. The only > exception is the inclusion of the global flag UseShenandoahGC, which is > by-design. When this is enabled in a build without Shenandoah, it > prints the appropriate error message and exist (exactly like with any > other GC that is not built-in). > > I've been thinking about if we can come up with a mechanical proof of > correctness. The first best thing would have been reproducible builds, > but we don't have that (and the UseShenandoahGC flag would mess it up > too). The other option would be to dump preprocessed sources and diff > them, but that is disturbed by the necessary inclusion of > "utilities/macros.hpp" which gives us the INCLUDE_SHENANDOAHGC macro. > > Which leaves us with carefully inspecting the webrev by multiple sets > of eyes and brains. > > Gil, would the change like this ease your concerns? > > Thank you, > Roman > > > On Fri, 2020-06-26 at 23:39 +0000, Gil Tene wrote: > > As I noted before, we have serious reservations about adding major > > new features to a > > stable LTS that is upstream of all of 11u. Based on daily > > interactions with tons of OpenJDK > > end users, I can say that the vast majority of end users that are > > ramping up on 11u are > > demanding stability and maturity above all else. The addition of such > > major changes in > > an 11u update will cause many of them to stall updates, and > > potentially stall the uptake > > of 11u altogether, waiting for 11u to actually stabilize first > > (evidenced by no longer doing > > such things), so they can keep up with fixes and security issues > > without having to take on > > potential regressions driven by late added or experimental features. > > > > If you want a modern GA'ed OpenJDK with all the latest features in > > it, use one. Please > > please please don't force people who choose to stick with a given > > OpenJDK version for > > stability, and knowingly did not move to a later OpenJDK version, to > > deploy the newly > > implemented features that you like by pushing for back-porting them > > to the upstream > > project that they get their security updates from... > > > > I can see a potentially good case for a downstream-from-11u project > > that incorporates > > ongoing development (as opposed to bug fixes) from later upstream > > versions, and would > > be happy to collaborate on one. But the 11u that is upstream of all > > other 11u's should not > > be seen as a developer's playground, or a place to add cool new > > features to a stable > > release. The upstream versions are there for that purpose. The valid > > interests of > > deep-bench, self-supporting groups making use of OpenJDK, who can > > take on the stability > > risks involved due to the depth of their technical bench, should not > > overcome the interests > > of the millions of consumers of the 11u builds that do not have that > > luxury, and who > > need and expect stable (in the changing only where necessary sense) > > 11u releases. > > > > I'd like to point out that 13u already includes Shenandoah. 15u > > (expected in 3 months) > > already includes Shenandoah as a non-experimental feature. And we can > > all collaborate > > on back-porting bug-fixes to those to updates to keep a lively > > version of OpenJDK that > > includes Shenandoah up to date from a bug fix and security update > > point of view. For > > those insisting on waiting for an LTS (on the assumption that LTSs > > are long term stable > > and don't get those sort of messing-with-after-the-fact done to them, > > mind you), 17u will > > be out in 15 months. > > > > So we are not talking about some far-away future for end users that > > want Shenandoah. > > It is here, now, in multiple consumable forms, and available for > > people not looking for > > the stability of an LTS, and even for people who do (as they can use > > the Red Hat 11u > > builds). It it will be available in more forms very soon with no > > effort, and we don't have > > to destabilize 11u for others to make that happen. > > > > ? Gil. > > > > > On Jun 26, 2020, at 1:51 PM, Mathiske, Bernd > > > wrote: > > > > > > Hi Roman, > > > > > > We are also in favor of upstreaming Shenandoah to jdk11u, for the > > > same reasons Aditya listed, and we agree with everything else he > > > has written. Similar to Aditya?s message below, we believe that > > > Shenandoah has strong demand and potential for those running > > > jdk11u. Targeting jdk11u is the most efficient way to reach a > > > large user base and to enable production use of Shenandoah. > > > > > > At Amazon, we plan to continue to contribute to Shenandoah, and we > > > fully intend to help maintain Shenandoah in jdk11u. > > > > > > Best, > > > > > > Bernd, Kelvin, Azeem > > > > > > ?On 6/26/20, 10:36 AM, "jdk-updates-dev on behalf of Aditya > > > Mandaleeka" > > adityam at microsoft.com> wrote: > > > > > > Hi Roman, > > > > > > I am in favor of this proposed upstreaming of Shenandoah into > > > jdk11u. Microsoft > > > has some long-term commitments to Java 11 customers and having > > > Shenandoah > > > included in jdk11u upstream would definitely be well-received by > > > my team and by > > > our customers. There is demand for this kind of GC option on > > > OpenJDK 11, and it > > > would be beneficial to the community for it to be available on > > > the upstream > > > OpenJDK rather than only in specific downstream releases. > > > Coupled with the fact > > > that Shenandoah has been tested on JDK 11 for quite a while now, > > > I think this > > > upstreaming makes sense. > > > > > > You mentioned in your original mail that your team at RedHat > > > intends to maintain > > > this proposed jdk11u version the same way it has been maintained > > > in > > > shenandoah/jdk11. That is great--from what I?ve seen in these > > > past few months > > > that I?ve been involved, I?ve been impressed by the way > > > backports to > > > shenandoah/jdk11 are being handled, the effort that is made to > > > keep the > > > Shenandoah code closely aligned between sh/11 and jdk/jdk, and > > > the care taken to > > > ensure that the impact on shared code is kept to a minimum. > > > > > > At Microsoft, we?ve already been contributing patches and > > > investigating and > > > reporting issues based on our testing of Shenandoah, both on > > > jdk/jdk and > > > shenandoah/jdk11. We intend to continue this kind of > > > involvement, so if this > > > upstreaming to jdk11u goes through, your team will also have our > > > support in > > > maintaining Shenandoah there. > > > > > > Thanks, > > > Aditya > > > > > > -----Original Message----- > > > From: jdk-updates-dev On > > > Behalf Of Roman Kennke > > > Sent: Thursday, June 25, 2020 4:16 AM > > > To: jdk-updates-dev at openjdk.java.net > > > Subject: Re: RFR (11u, XXL): Upstream/backport Shenandoah to > > > JDK11u > > > > > > I would like to revive this RFR, I believe we haven't come to a > > > conclusion yet. I still think it would be very useful to have > > > the > > > Shenandoah GC included in jdk11u upstream. I have heard from > > > several > > > vendors who would like to ship Shenandoah, but maintaining the > > > rather > > > huge Shenandoah backport patch is an absolute no-go. > > > > > > This is an updated patch that includes numerous bugfixes and > > > improvements in Shenandoah. It is what Red Hat is going to ship > > > in > > > their 11.0.8 release. > > > > > > Full webrev: > > > > > > https://nam06.safelinks.protection.outlook.com/?url=https:%2F%2Fcr.openjdk.java.net%2F~rkennke%2Fshenandoah-jdk11u-upstream%2Fwebrev.04-all%2F&data=02%7C01%7Cadityam%40microsoft.com%7C7aec62734d22458243d908d818f974c5%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637286806967016879&sdata=G1HjEJ68qFJ2uwRyxfFKNE8gcbsFXEdYi%2B9RyrUeG60%3D&reserved=0 > > > > > > Webrev only for shared-code changes: > > > > > > https://nam06.safelinks.protection.outlook.com/?url=https:%2F%2Fcr.openjdk.java.net%2F~rkennke%2Fshenandoah-jdk11u-upstream%2Fwebrev.04-shared%2F&data=02%7C01%7Cadityam%40microsoft.com%7C7aec62734d22458243d908d818f974c5%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637286806967026872&sdata=RrKNOvxc3%2FheThyi2cMiazE%2BxC9UfARxfTbmuoc640Y%3D&reserved=0 > > > > > > Please let me know what you think. > > > > > > Roman > > > > > > > On Fri, 2020-01-17 at 18:05 +0100, Roman Kennke wrote: > > > > Hello, > > > > > > > > The Shenandoah GC has been integrated into jdk12 a little over a > > > > year > > > > ago: > > > > > > > > https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2Fhg.openjdk.java.net%2Fjdk%2Fjdk%2Frev%2F9c18c9d839d3&data=02%7C01%7Cadityam%40microsoft.com%7C7aec62734d22458243d908d818f974c5%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637286806967026872&sdata=QJo2xwdHiEuaqsz26P2j%2BZdE6j3AxUvpnHVTjD9XGXI%3D&reserved=0 > > > > > > > > The Shenandoah team has been maintaining the Shenandoah-enabled > > > > jdk11u > > > > downstream repository since the inception of jdk11 under: > > > > > > > > https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2Fhg.openjdk.java.net%2Fshenandoah%2Fjdk11&data=02%7C01%7Cadityam%40microsoft.com%7C7aec62734d22458243d908d818f974c5%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637286806967026872&sdata=P%2FLkJ5VCrq4oWzik3WZ3GTy1a3mA52mxkITasawIMk0%3D&reserved=0 > > > > > > > > In order to make it more useful and accessible for users, we > > > > would > > > > like > > > > to make it available in upstream jdk11u. The Shenandoah team at > > > > Red > > > > Hat > > > > intends to maintain it the same way it had been maintained in > > > > shenandoah/jdk11, in the course of regular jdk11u maintenance Red > > > > Hat > > > > already does. > > > > > > > > Thanks to recent changes in Shenandoah over the last year, we are > > > > now > > > > able to fit the GC interface in jdk11 almost exactly. There are a > > > > few > > > > exceptions though. We tried to follow the following pattern for > > > > all > > > > shared-code changes where it made sense: > > > > > > > > #if INCLUDE_SHENANDOAH_GC > > > > if (UseShenandoahGC) { > > > > ... > > > > } > > > > #endif > > > > > > > > This ensures that the Shenandoah-specific changes are cut out of > > > > the > > > > build when building without Shenandoah, and avoid those code > > > > paths at > > > > runtime when running without Shenandoah. > > > > > > > > Also, there are a few places where this was not possible or where > > > > it > > > > didn't seem feasible, but we tried to keep them few and obvious. > > > > Whenever this is the case, we are mirroring as close as possible > > > > what > > > > we > > > > have in jdk/jdk. > > > > > > > > Architecture-wise, Shenandoah runs on x86_64, aarch64, x86_32. > > > > Other > > > > architectures still build fine, and Shenandoah gets automatically > > > > disabled during configure if not supported. OS-wise, Shenandoah > > > > runs > > > > on > > > > Linux, Windows, and is known to run on MacOS X and Solaris too. > > > > > > > > I wasn't sure which form this should take. I decided to start > > > > with > > > > the > > > > easiest possible form this could take: a simple patch/webrev and > > > > an > > > > RFR. > > > > Let me know if you need a JIRA issue or any other formalities to > > > > track that. > > > > > > > > Full webrev: > > > > https://nam06.safelinks.protection.outlook.com/?url=https:%2F%2Fcr.openjdk.java.net%2F~rkennke%2Fshenandoah-jdk11u-upstream%2Fwebrev.02-all%2F&data=02%7C01%7Cadityam%40microsoft.com%7C7aec62734d22458243d908d818f974c5%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637286806967026872&sdata=eHspbGt5Eox4EMv8hstTBwsqFYefGNO2acGymP7ZfN0%3D&reserved=0 > > > > > > > > Webrev only for shared-code changes: > > > > https://nam06.safelinks.protection.outlook.com/?url=https:%2F%2Fcr.openjdk.java.net%2F~rkennke%2Fshenandoah-jdk11u-upstream%2Fwebrev.02-shared%2F&data=02%7C01%7Cadityam%40microsoft.com%7C7aec62734d22458243d908d818f974c5%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637286806967026872&sdata=LGghbNRyHPSJLK8l2Oyhxh1lhD46LmvZtN1G5htYBpk%3D&reserved=0 > > > > > > > > The webrevs are based on and depend on the arraycopy patch > > > > proposed > > > > for > > > > review here: > > > > > > > > https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmail.openjdk.java.net%2Fpipermail%2Fjdk-updates-dev%2F2020-January%2F002396.html&data=02%7C01%7Cadityam%40microsoft.com%7C7aec62734d22458243d908d818f974c5%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637286806967026872&sdata=Kw0qkfvtVNVrCiMeypwr4BHX9e3bZafJnVCMVY%2BvOOw%3D&reserved=0 > > > > > > > > Testing: > > > > The code has been tested many many times, continuously while > > > > maintaining > > > > it. It has also served as the basis for RPMs and shipped to > > > > customers. > > > > We are running all sorts of tests of the hotspot/jtreg testsuite, > > > > in > > > > particular hotspot_gc_shenandoah, CTW tests, the regular tier* > > > > tests, > > > > several popular benchmarks (specjvm, specjbb on a regular basis), > > > > on > > > > a > > > > nightly basis. > > > > > > > > Please let me know what you think. > > > > > > > > Thanks, > > > > Roman > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- Mario Torre Associate Manager, Software Engineering Red Hat GmbH 9704 A60C B4BE A8B8 0F30 9205 5D7E 4952 3F65 7898 From rkennke at redhat.com Thu Jul 9 09:47:57 2020 From: rkennke at redhat.com (Roman Kennke) Date: Thu, 09 Jul 2020 11:47:57 +0200 Subject: RFR (11u, XXL): Upstream/backport Shenandoah to JDK11u In-Reply-To: References: <3595e963-fa5b-bfce-c463-841b84d4103f@redhat.com> <6468CB08-3DB5-488E-9122-E3699A95218B@amazon.com> <33d0bd6339d4c9455c3309baa48e38e621fb9a79.camel@redhat.com> <4F72AD88-0165-497B-B32A-212A5AB076F0@azul.com> <7959350D-EABE-461E-90AB-167214C55D32@azul.com> <39b7b9292d64cfc77aa3b6ad48e5d922a4ceb5b4.camel@redhat.com> Message-ID: <0f3d787d327f9057e20737e9dc3a3337fe98ec4a.camel@redhat.com> I think the -shared webrev is the one you want to review. Yes, it contains all the #if stuff, but you want to review this too, to ensure I did not make a mistake. Roman On Thu, 2020-07-09 at 09:40 +0000, Lindenmaier, Goetz wrote: > Hi Roman, > > What about creating a webrev with those changes that > will be compiled if you configure without shenandoahgc? > I know there is a webrev with only the shared changes, > but they contain a lot of #define coding, or such under > protection by the flag that enables Shenandoah, which > should be constant 'false' if Shenandoah is disabled, right? > As I read the code, there should remain only a few. > > Then we can easily assess the actual risk. If something > appears risky for the existing code, we can guard it > by #ifdefs. > > Best regards, > Goetz. > > > -----Original Message----- > > From: jdk-updates-dev On > > Behalf > > Of Roman Kennke > > Sent: Thursday, July 9, 2020 7:57 AM > > To: Gil Tene > > Cc: Bernd Mathiske ; jdk-updates- > > dev at openjdk.java.net; Nilsen, Kelvin ; Jiva, > > Azeem > > > > Subject: Re: RFR (11u, XXL): Upstream/backport Shenandoah to JDK11u > > > > On Thu, 2020-07-09 at 04:53 +0000, Gil Tene wrote: > > > > On Jul 8, 2020, at 4:07 PM, Roman Kennke > > > > wrote: > > > > > > > > On Wed, 2020-07-08 at 22:35 +0000, Gil Tene wrote: > > > > > > On Jul 8, 2020, at 2:58 PM, Roman Kennke < > > > > > > rkennke at redhat.com> > > > > > > wrote: > > > > > > > > > > > > ... > > > > > > In other words: what exactly is it that you want to prove > > > > > > then? > > > > > > I > > > > > > don't > > > > > > understand it. > > > > > > > > > > I'm not sure I understand it either. ;-) > > > > > > > > Haha, ok right :-) > > > > > > > > > I'm basically asking the "if we prove it now", how do we know > > > > > that > > > > > the > > > > > same holds later?" question. > > > > > > > > > > Delaying some big impact by one update is clearly not the > > > > > goal > > > > > you > > > > > are proposing. > > > > > > > > I don't understand this sentence (but it's quite late here...) > > > > > > > > > I believe we agree that the quality you suggest would need to > > > > > be > > > > > retained into future > > > > > builds. Having nothing to prevent it from breaking (e.g. by > > > > > accident) > > > > > is probably > > > > > dangerous. So the question becomes "how do we ask the > > > > > question of > > > > > whether or not > > > > > we still have not introduced any differences?" in future > > > > > updates. > > > > > > > > Ok. > > > > > > > > > Let's start from the assumption that we can prove something > > > > > now > > > > > (and > > > > > I > > > > > do think we should be able to find a mechanical thing to do > > > > > that). > > > > > That will show > > > > > that the quality we seek to keep has been kept at a point in > > > > > time... > > > > > > > > Yes. Let's assume that. (As an aside, if you'd actually look at > > > > the > > > > patch, I hope you'd find it quite obvious that it does the > > > > right > > > > thing. > > > > And you might be surprised how relatively few such changes we > > > > actually > > > > have.) > > > > > > > > > For as long as the statement of "without Shenandoah enabled, > > > > > the > > > > > build > > > > > does not have any actual code bits related to the Shenandoah > > > > > back- > > > > > port" > > > > > needs to hold, we need a way to show that the same still > > > > > holds, > > > > > Either by > > > > > proving that the original proof still holds in the presence > > > > > of > > > > > changes, or > > > > > by re-proving it somehow. > > > > > > > > > > > > > Let me elaborate a little bit on my experiences with backports, > > > > and > > > > mixing both Shenandoah and non-Shenandoah backports. > > > > > > > > In the many months since we maintain Shenandoah in our 11u, the > > > > overhelming majority of Shenandoah-related backports did not > > > > touch > > > > any > > > > shared code at all. That's because the existing GC interfaces > > > > isolate > > > > GCs really well in 11u. The few occasions where we did change > > > > shared- > > > > code was 1. When we switched to LRB. We would not usually > > > > backport > > > > such > > > > drastic changes, but it seemed prudent here, because it did > > > > actually > > > > decrease the shared-code exposure drastically. 2. in > > > > preparation > > > > for > > > > this upstreaming patch - more pruning and isolation of shared- > > > > code > > > > changes. > > > > > > > > In the very rare situations where a backport would require > > > > shared- > > > > code > > > > changes (I can't actually remember any, apart from the two that > > > > I > > > > just > > > > mentioned), we carefully consider if it's actually necessary to > > > > backport. For example, we have not backported concurrent class- > > > > unloading support to 11u precisely because it would require > > > > (significant) changes outside of Shenandoah. *If* a critical > > > > backport > > > > (say, a bugfix) requires changes outside of Shenandoah, it > > > > would > > > > have > > > > to be properly guarded by the same means as we do in the > > > > proposed > > > > upstreaming patch. We - the Shenandoah team - would be aware of > > > > that > > > > and mention it in relevant reviews. It would also prominently > > > > show > > > > up > > > > in a patch because it has files without 'shenandoah' in their > > > > path > > > > names. And from there, it's a matter of carefully considering, > > > > reviewing and testing it. I don't think this would silently > > > > sneak > > > > in > > > > somehow. > > > > > > > > I can't think of a situation where we ever had the reverse > > > > problem: > > > > a > > > > shared-code change touching on something Shenandoah-related. > > > > > > > > Also, while we did have a flurry of Shenandoah-related > > > > backports in > > > > the > > > > past, because of stabilization and new features (e.g. LRB and > > > > friends), > > > > this is most likely not going to continue into the future. I > > > > expect > > > > less Shenandoah-backporting traffic, basically limited to > > > > bugfixes > > > > and > > > > improvements that don't touch shared-code. We have a couple of > > > > features > > > > on our todo list for later JDKs, but none of them sound like > > > > candidates > > > > for backporting. > > > > > > > > > The notion that our normal review processes will catch > > > > > everything > > > > > that can break > > > > > that initial prooved state seems a bit optimistic to me. The > > > > > review > > > > > process will be > > > > > well intentioned, and we'll try to tag things right, but one > > > > > mistaken > > > > > application or > > > > > move of code across ifdef lines, or integration of some mis- > > > > > tagged or > > > > > unnoticed tag > > > > > thing into shared code will break the statement? > > > > > > > > > > > > > How is that any different from the situation that we already > > > > have? > > > > Hotspot code already has similar #ifdefs sprinkled all around, > > > > e.g. > > > > for > > > > other GCs, for JFR, platform #ifdefs, and so on. How is the > > > > situation > > > > different for Shenandoah, why do we need special rules or > > > > process > > > > or > > > > even proofs for that? As far as I can see, our current high- > > > > standard > > > > development and review practices already cover it very well. > > > > > > > > > I believe that we can avoid this once, at the start, with a > > > > > mechanical proof. Can > > > > > we keep it up? What sort fo rules or invariants can we come > > > > > up > > > > > with > > > > > that we can > > > > > use to verify and show that the quality we seek to keep has > > > > > been > > > > > kept > > > > > through later > > > > > updates? > > > > > > > > Well yes, the usual proper care that everybody is taking in > > > > making > > > > the > > > > backports, reviewing the backports, testing it, etc. > > > > > > > > > Let's try to think of some?. > > > > > > > > As long as we don't know what the exact question/problem even > > > > might > > > > be, > > > > we can hardly come up with an answer/solution. > > > > > > I'm not trying to be difficult here. I'm just going with the > > > basic > > > line > > > of logic, and looking for a way to implement it. > > > > > > > Yeah. Let me just state for the record, that I find this is getting > > slightly ridiculous ;-) > > > > Let me try anyway. > > > > It looks to me that this is a prime example for proof-by-induction. > > Let's try to formulate it. > > > > State 0 is our current jdk11u state. Shenandoah does obviously not > > leak > > into the build, because it's not included yet. > > > > State 1 is the state after the initial inclusion of Shenandoah. > > Let's > > assume we can prove that between 0 and 1, nothing leaks into the > > build > > when building with --with-jvm-features=-shenandoahgc. That proof > > will > > have to be determined, maybe comparing object files would work. > > > > Then, for any further changeset to be backported, this would be the > > N- > > > N+1 case: > > > > - if the changeset is not Shenandoah-related, it'll obviously > > change > > the outcome of the build, but also obviously doesn't leak any > > Shenandoah-related changes. > > - otherwise, if the changeset is Shenandoah-related, we can run the > > same proof that we did from 0->1 for N->N+1, and proof that no > > additional Shenandoah-related changes leaks into non-Shenandoah > > build. > > > > Right? > > > > It depends on the correct classification what constitutes a > > Shenandoah- > > related change and what doesn't. But it must, I see no way around > > that. > > From my perspective and in my experience, this is really easy > > though, > > and can be achieved by applying some common sense. (Hey, when > > reviewing, you really ought to look at the bug - and spot gc- > > shenandoah > > label - and also look at the patch and understand what it all does, > > and > > come to some conclusion.) > > > > > > Now, about that proof: I will spend the day looking if we can do it > > by > > comparing object files of builds. E.g. do builds (-shenandoahgc) > > before/after a change is applied, run checksums over each object > > file, > > and compare those checksums. Let's see if there's any pitfalls > > there. > > > > What do you think? > > > > Roman > > From goetz.lindenmaier at sap.com Thu Jul 9 09:51:44 2020 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Thu, 9 Jul 2020 09:51:44 +0000 Subject: RFR (11u, XXL): Upstream/backport Shenandoah to JDK11u In-Reply-To: References: <3595e963-fa5b-bfce-c463-841b84d4103f@redhat.com> <6468CB08-3DB5-488E-9122-E3699A95218B@amazon.com> Message-ID: Yes, a good point. ... I had missed the mail with the new webrev ... Best regards, Goetz. > -----Original Message----- > From: jdk-updates-dev On Behalf > Of Mario Torre > Sent: Thursday, July 9, 2020 11:44 AM > To: Roman Kennke > Cc: Gil Tene ; Bernd Mathiske ; jdk- > updates-dev at openjdk.java.net; Nilsen, Kelvin ; Jiva, > Azeem > Subject: Re: RFR (11u, XXL): Upstream/backport Shenandoah to JDK11u > > Hi Roman, > > Can you please start a new thread to allow for a proper review? > (Whether approved by maintainer or not I think the patch will want a > review, right?) > > We can continue this thread dedicated to the philosophical questions > about life, universe and backports :) > > Cheers, > Mario > > On Wed, Jul 8, 2020 at 8:32 PM Roman Kennke > wrote: > > > > Hi Gil & all, > > > > I put in some extra work and had a deep look at the remaining unguarded > > changes in shared-code, with an eye to either eliminating them > > altogether, or - where this is not possible - guard them with build- > > time #if INCLUDE_SHENANDOAHGC or similar. The guiding principle in this > > effort has been that building without Shenandoah (--with-jvm-features=- > > shenandoahgc) would compile *the exact same* code that it does now. I > > believe I achieved this with the following proposed changes: > > > > Full webrev (incl. Shenandoah-only files): > > https://cr.openjdk.java.net/~rkennke/shenandoah-jdk11u- > upstream/webrev.06-all/ > > > > Shared-code-only changes: > > https://cr.openjdk.java.net/~rkennke/shenandoah-jdk11u- > upstream/webrev.06-shared/ > > > > The latter webrev exludes all files that are only compiled with > > Shenandoah enabled. > > > > Also, compared to previous webrevs, this excludes Shenandoah by > > default. > > > > As far as I can tell, this literally compiles the same code into > > libjvm.so as it does now, when building without Shenandoah. The only > > exception is the inclusion of the global flag UseShenandoahGC, which is > > by-design. When this is enabled in a build without Shenandoah, it > > prints the appropriate error message and exist (exactly like with any > > other GC that is not built-in). > > > > I've been thinking about if we can come up with a mechanical proof of > > correctness. The first best thing would have been reproducible builds, > > but we don't have that (and the UseShenandoahGC flag would mess it up > > too). The other option would be to dump preprocessed sources and diff > > them, but that is disturbed by the necessary inclusion of > > "utilities/macros.hpp" which gives us the INCLUDE_SHENANDOAHGC macro. > > > > Which leaves us with carefully inspecting the webrev by multiple sets > > of eyes and brains. > > > > Gil, would the change like this ease your concerns? > > > > Thank you, > > Roman > > > > > > On Fri, 2020-06-26 at 23:39 +0000, Gil Tene wrote: > > > As I noted before, we have serious reservations about adding major > > > new features to a > > > stable LTS that is upstream of all of 11u. Based on daily > > > interactions with tons of OpenJDK > > > end users, I can say that the vast majority of end users that are > > > ramping up on 11u are > > > demanding stability and maturity above all else. The addition of such > > > major changes in > > > an 11u update will cause many of them to stall updates, and > > > potentially stall the uptake > > > of 11u altogether, waiting for 11u to actually stabilize first > > > (evidenced by no longer doing > > > such things), so they can keep up with fixes and security issues > > > without having to take on > > > potential regressions driven by late added or experimental features. > > > > > > If you want a modern GA'ed OpenJDK with all the latest features in > > > it, use one. Please > > > please please don't force people who choose to stick with a given > > > OpenJDK version for > > > stability, and knowingly did not move to a later OpenJDK version, to > > > deploy the newly > > > implemented features that you like by pushing for back-porting them > > > to the upstream > > > project that they get their security updates from... > > > > > > I can see a potentially good case for a downstream-from-11u project > > > that incorporates > > > ongoing development (as opposed to bug fixes) from later upstream > > > versions, and would > > > be happy to collaborate on one. But the 11u that is upstream of all > > > other 11u's should not > > > be seen as a developer's playground, or a place to add cool new > > > features to a stable > > > release. The upstream versions are there for that purpose. The valid > > > interests of > > > deep-bench, self-supporting groups making use of OpenJDK, who can > > > take on the stability > > > risks involved due to the depth of their technical bench, should not > > > overcome the interests > > > of the millions of consumers of the 11u builds that do not have that > > > luxury, and who > > > need and expect stable (in the changing only where necessary sense) > > > 11u releases. > > > > > > I'd like to point out that 13u already includes Shenandoah. 15u > > > (expected in 3 months) > > > already includes Shenandoah as a non-experimental feature. And we can > > > all collaborate > > > on back-porting bug-fixes to those to updates to keep a lively > > > version of OpenJDK that > > > includes Shenandoah up to date from a bug fix and security update > > > point of view. For > > > those insisting on waiting for an LTS (on the assumption that LTSs > > > are long term stable > > > and don't get those sort of messing-with-after-the-fact done to them, > > > mind you), 17u will > > > be out in 15 months. > > > > > > So we are not talking about some far-away future for end users that > > > want Shenandoah. > > > It is here, now, in multiple consumable forms, and available for > > > people not looking for > > > the stability of an LTS, and even for people who do (as they can use > > > the Red Hat 11u > > > builds). It it will be available in more forms very soon with no > > > effort, and we don't have > > > to destabilize 11u for others to make that happen. > > > > > > ? Gil. > > > > > > > On Jun 26, 2020, at 1:51 PM, Mathiske, Bernd > > > > wrote: > > > > > > > > Hi Roman, > > > > > > > > We are also in favor of upstreaming Shenandoah to jdk11u, for the > > > > same reasons Aditya listed, and we agree with everything else he > > > > has written. Similar to Aditya?s message below, we believe that > > > > Shenandoah has strong demand and potential for those running > > > > jdk11u. Targeting jdk11u is the most efficient way to reach a > > > > large user base and to enable production use of Shenandoah. > > > > > > > > At Amazon, we plan to continue to contribute to Shenandoah, and we > > > > fully intend to help maintain Shenandoah in jdk11u. > > > > > > > > Best, > > > > > > > > Bernd, Kelvin, Azeem > > > > > > > > ?On 6/26/20, 10:36 AM, "jdk-updates-dev on behalf of Aditya > > > > Mandaleeka" > > > adityam at microsoft.com> wrote: > > > > > > > > Hi Roman, > > > > > > > > I am in favor of this proposed upstreaming of Shenandoah into > > > > jdk11u. Microsoft > > > > has some long-term commitments to Java 11 customers and having > > > > Shenandoah > > > > included in jdk11u upstream would definitely be well-received by > > > > my team and by > > > > our customers. There is demand for this kind of GC option on > > > > OpenJDK 11, and it > > > > would be beneficial to the community for it to be available on > > > > the upstream > > > > OpenJDK rather than only in specific downstream releases. > > > > Coupled with the fact > > > > that Shenandoah has been tested on JDK 11 for quite a while now, > > > > I think this > > > > upstreaming makes sense. > > > > > > > > You mentioned in your original mail that your team at RedHat > > > > intends to maintain > > > > this proposed jdk11u version the same way it has been maintained > > > > in > > > > shenandoah/jdk11. That is great--from what I?ve seen in these > > > > past few months > > > > that I?ve been involved, I?ve been impressed by the way > > > > backports to > > > > shenandoah/jdk11 are being handled, the effort that is made to > > > > keep the > > > > Shenandoah code closely aligned between sh/11 and jdk/jdk, and > > > > the care taken to > > > > ensure that the impact on shared code is kept to a minimum. > > > > > > > > At Microsoft, we?ve already been contributing patches and > > > > investigating and > > > > reporting issues based on our testing of Shenandoah, both on > > > > jdk/jdk and > > > > shenandoah/jdk11. We intend to continue this kind of > > > > involvement, so if this > > > > upstreaming to jdk11u goes through, your team will also have our > > > > support in > > > > maintaining Shenandoah there. > > > > > > > > Thanks, > > > > Aditya > > > > > > > > -----Original Message----- > > > > From: jdk-updates-dev On > > > > Behalf Of Roman Kennke > > > > Sent: Thursday, June 25, 2020 4:16 AM > > > > To: jdk-updates-dev at openjdk.java.net > > > > Subject: Re: RFR (11u, XXL): Upstream/backport Shenandoah to > > > > JDK11u > > > > > > > > I would like to revive this RFR, I believe we haven't come to a > > > > conclusion yet. I still think it would be very useful to have > > > > the > > > > Shenandoah GC included in jdk11u upstream. I have heard from > > > > several > > > > vendors who would like to ship Shenandoah, but maintaining the > > > > rather > > > > huge Shenandoah backport patch is an absolute no-go. > > > > > > > > This is an updated patch that includes numerous bugfixes and > > > > improvements in Shenandoah. It is what Red Hat is going to ship > > > > in > > > > their 11.0.8 release. > > > > > > > > Full webrev: > > > > > > > > > https://nam06.safelinks.protection.outlook.com/?url=https:%2F%2Fcr.openj > dk.java.net%2F~rkennke%2Fshenandoah-jdk11u-upstream%2Fwebrev.04- > all%2F&data=02%7C01%7Cadityam%40microsoft.com%7C7aec62734d2 > 2458243d908d818f974c5%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C > 0%7C637286806967016879&sdata=G1HjEJ68qFJ2uwRyxfFKNE8gcbsFXEd > Yi%2B9RyrUeG60%3D&reserved=0 > > > > > > > > Webrev only for shared-code changes: > > > > > > > > > https://nam06.safelinks.protection.outlook.com/?url=https:%2F%2Fcr.openj > dk.java.net%2F~rkennke%2Fshenandoah-jdk11u-upstream%2Fwebrev.04- > shared%2F&data=02%7C01%7Cadityam%40microsoft.com%7C7aec6273 > 4d22458243d908d818f974c5%7C72f988bf86f141af91ab2d7cd011db47%7C1 > %7C0%7C637286806967026872&sdata=RrKNOvxc3%2FheThyi2cMiazE% > 2BxC9UfARxfTbmuoc640Y%3D&reserved=0 > > > > > > > > Please let me know what you think. > > > > > > > > Roman > > > > > > > > > On Fri, 2020-01-17 at 18:05 +0100, Roman Kennke wrote: > > > > > Hello, > > > > > > > > > > The Shenandoah GC has been integrated into jdk12 a little over a > > > > > year > > > > > ago: > > > > > > > > > > > https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2Fhg.op > enjdk.java.net%2Fjdk%2Fjdk%2Frev%2F9c18c9d839d3&data=02%7C01% > 7Cadityam%40microsoft.com%7C7aec62734d22458243d908d818f974c5%7C7 > 2f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637286806967026872&a > mp;sdata=QJo2xwdHiEuaqsz26P2j%2BZdE6j3AxUvpnHVTjD9XGXI%3D&r > eserved=0 > > > > > > > > > > The Shenandoah team has been maintaining the Shenandoah- > enabled > > > > > jdk11u > > > > > downstream repository since the inception of jdk11 under: > > > > > > > > > > > https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2Fhg.op > enjdk.java.net%2Fshenandoah%2Fjdk11&data=02%7C01%7Cadityam%4 > 0microsoft.com%7C7aec62734d22458243d908d818f974c5%7C72f988bf86f14 > 1af91ab2d7cd011db47%7C1%7C0%7C637286806967026872&sdata=P% > 2FLkJ5VCrq4oWzik3WZ3GTy1a3mA52mxkITasawIMk0%3D&reserved=0 > > > > > > > > > > In order to make it more useful and accessible for users, we > > > > > would > > > > > like > > > > > to make it available in upstream jdk11u. The Shenandoah team at > > > > > Red > > > > > Hat > > > > > intends to maintain it the same way it had been maintained in > > > > > shenandoah/jdk11, in the course of regular jdk11u maintenance Red > > > > > Hat > > > > > already does. > > > > > > > > > > Thanks to recent changes in Shenandoah over the last year, we are > > > > > now > > > > > able to fit the GC interface in jdk11 almost exactly. There are a > > > > > few > > > > > exceptions though. We tried to follow the following pattern for > > > > > all > > > > > shared-code changes where it made sense: > > > > > > > > > > #if INCLUDE_SHENANDOAH_GC > > > > > if (UseShenandoahGC) { > > > > > ... > > > > > } > > > > > #endif > > > > > > > > > > This ensures that the Shenandoah-specific changes are cut out of > > > > > the > > > > > build when building without Shenandoah, and avoid those code > > > > > paths at > > > > > runtime when running without Shenandoah. > > > > > > > > > > Also, there are a few places where this was not possible or where > > > > > it > > > > > didn't seem feasible, but we tried to keep them few and obvious. > > > > > Whenever this is the case, we are mirroring as close as possible > > > > > what > > > > > we > > > > > have in jdk/jdk. > > > > > > > > > > Architecture-wise, Shenandoah runs on x86_64, aarch64, x86_32. > > > > > Other > > > > > architectures still build fine, and Shenandoah gets automatically > > > > > disabled during configure if not supported. OS-wise, Shenandoah > > > > > runs > > > > > on > > > > > Linux, Windows, and is known to run on MacOS X and Solaris too. > > > > > > > > > > I wasn't sure which form this should take. I decided to start > > > > > with > > > > > the > > > > > easiest possible form this could take: a simple patch/webrev and > > > > > an > > > > > RFR. > > > > > Let me know if you need a JIRA issue or any other formalities to > > > > > track that. > > > > > > > > > > Full webrev: > > > > > > https://nam06.safelinks.protection.outlook.com/?url=https:%2F%2Fcr.openj > dk.java.net%2F~rkennke%2Fshenandoah-jdk11u-upstream%2Fwebrev.02- > all%2F&data=02%7C01%7Cadityam%40microsoft.com%7C7aec62734d2 > 2458243d908d818f974c5%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C > 0%7C637286806967026872&sdata=eHspbGt5Eox4EMv8hstTBwsqFYefG > NO2acGymP7ZfN0%3D&reserved=0 > > > > > > > > > > Webrev only for shared-code changes: > > > > > > https://nam06.safelinks.protection.outlook.com/?url=https:%2F%2Fcr.openj > dk.java.net%2F~rkennke%2Fshenandoah-jdk11u-upstream%2Fwebrev.02- > shared%2F&data=02%7C01%7Cadityam%40microsoft.com%7C7aec6273 > 4d22458243d908d818f974c5%7C72f988bf86f141af91ab2d7cd011db47%7C1 > %7C0%7C637286806967026872&sdata=LGghbNRyHPSJLK8l2Oyhxh1lhD4 > 6LmvZtN1G5htYBpk%3D&reserved=0 > > > > > > > > > > The webrevs are based on and depend on the arraycopy patch > > > > > proposed > > > > > for > > > > > review here: > > > > > > > > > > > https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmail. > openjdk.java.net%2Fpipermail%2Fjdk-updates-dev%2F2020- > January%2F002396.html&data=02%7C01%7Cadityam%40microsoft.com > %7C7aec62734d22458243d908d818f974c5%7C72f988bf86f141af91ab2d7cd0 > 11db47%7C1%7C0%7C637286806967026872&sdata=Kw0qkfvtVNVrCiM > eypwr4BHX9e3bZafJnVCMVY%2BvOOw%3D&reserved=0 > > > > > > > > > > Testing: > > > > > The code has been tested many many times, continuously while > > > > > maintaining > > > > > it. It has also served as the basis for RPMs and shipped to > > > > > customers. > > > > > We are running all sorts of tests of the hotspot/jtreg testsuite, > > > > > in > > > > > particular hotspot_gc_shenandoah, CTW tests, the regular tier* > > > > > tests, > > > > > several popular benchmarks (specjvm, specjbb on a regular basis), > > > > > on > > > > > a > > > > > nightly basis. > > > > > > > > > > Please let me know what you think. > > > > > > > > > > Thanks, > > > > > Roman > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > Mario Torre > Associate Manager, Software Engineering > Red Hat GmbH > 9704 A60C B4BE A8B8 0F30 9205 5D7E 4952 3F65 7898 From rkennke at redhat.com Thu Jul 9 09:56:45 2020 From: rkennke at redhat.com (Roman Kennke) Date: Thu, 09 Jul 2020 11:56:45 +0200 Subject: RFR (11u, XXL): Upstream/backport Shenandoah to JDK11u In-Reply-To: References: <3595e963-fa5b-bfce-c463-841b84d4103f@redhat.com> <6468CB08-3DB5-488E-9122-E3699A95218B@amazon.com> <33d0bd6339d4c9455c3309baa48e38e621fb9a79.camel@redhat.com> <4F72AD88-0165-497B-B32A-212A5AB076F0@azul.com> <7959350D-EABE-461E-90AB-167214C55D32@azul.com> <39b7b9292d64cfc77aa3b6ad48e5d922a4ceb5b4.camel@redhat.com> Message-ID: The only change that should affect libjvm.so is this one: https://cr.openjdk.java.net/~rkennke/shenandoah-jdk11u-upstream/webrev.06-shared/src/hotspot/share/gc/shared/gc_globals.hpp.udiff.html (the exposure of the UseShenandoahGC flag) and https://cr.openjdk.java.net/~rkennke/shenandoah-jdk11u-upstream/webrev.06-shared/src/hotspot/share/gc/shared/gcConfig.cpp.udiff.html ... the NON_SHENANDOAHGC part that prints the failure and exists when that flag is selected. ... all assuming that I made no mistakes in the rest of the -shared changes. Roman On Thu, 2020-07-09 at 09:40 +0000, Lindenmaier, Goetz wrote: > Hi Roman, > > What about creating a webrev with those changes that > will be compiled if you configure without shenandoahgc? > I know there is a webrev with only the shared changes, > but they contain a lot of #define coding, or such under > protection by the flag that enables Shenandoah, which > should be constant 'false' if Shenandoah is disabled, right? > As I read the code, there should remain only a few. > > Then we can easily assess the actual risk. If something > appears risky for the existing code, we can guard it > by #ifdefs. > > Best regards, > Goetz. > > > -----Original Message----- > > From: jdk-updates-dev On > > Behalf > > Of Roman Kennke > > Sent: Thursday, July 9, 2020 7:57 AM > > To: Gil Tene > > Cc: Bernd Mathiske ; jdk-updates- > > dev at openjdk.java.net; Nilsen, Kelvin ; Jiva, > > Azeem > > > > Subject: Re: RFR (11u, XXL): Upstream/backport Shenandoah to JDK11u > > > > On Thu, 2020-07-09 at 04:53 +0000, Gil Tene wrote: > > > > On Jul 8, 2020, at 4:07 PM, Roman Kennke > > > > wrote: > > > > > > > > On Wed, 2020-07-08 at 22:35 +0000, Gil Tene wrote: > > > > > > On Jul 8, 2020, at 2:58 PM, Roman Kennke < > > > > > > rkennke at redhat.com> > > > > > > wrote: > > > > > > > > > > > > ... > > > > > > In other words: what exactly is it that you want to prove > > > > > > then? > > > > > > I > > > > > > don't > > > > > > understand it. > > > > > > > > > > I'm not sure I understand it either. ;-) > > > > > > > > Haha, ok right :-) > > > > > > > > > I'm basically asking the "if we prove it now", how do we know > > > > > that > > > > > the > > > > > same holds later?" question. > > > > > > > > > > Delaying some big impact by one update is clearly not the > > > > > goal > > > > > you > > > > > are proposing. > > > > > > > > I don't understand this sentence (but it's quite late here...) > > > > > > > > > I believe we agree that the quality you suggest would need to > > > > > be > > > > > retained into future > > > > > builds. Having nothing to prevent it from breaking (e.g. by > > > > > accident) > > > > > is probably > > > > > dangerous. So the question becomes "how do we ask the > > > > > question of > > > > > whether or not > > > > > we still have not introduced any differences?" in future > > > > > updates. > > > > > > > > Ok. > > > > > > > > > Let's start from the assumption that we can prove something > > > > > now > > > > > (and > > > > > I > > > > > do think we should be able to find a mechanical thing to do > > > > > that). > > > > > That will show > > > > > that the quality we seek to keep has been kept at a point in > > > > > time... > > > > > > > > Yes. Let's assume that. (As an aside, if you'd actually look at > > > > the > > > > patch, I hope you'd find it quite obvious that it does the > > > > right > > > > thing. > > > > And you might be surprised how relatively few such changes we > > > > actually > > > > have.) > > > > > > > > > For as long as the statement of "without Shenandoah enabled, > > > > > the > > > > > build > > > > > does not have any actual code bits related to the Shenandoah > > > > > back- > > > > > port" > > > > > needs to hold, we need a way to show that the same still > > > > > holds, > > > > > Either by > > > > > proving that the original proof still holds in the presence > > > > > of > > > > > changes, or > > > > > by re-proving it somehow. > > > > > > > > > > > > > Let me elaborate a little bit on my experiences with backports, > > > > and > > > > mixing both Shenandoah and non-Shenandoah backports. > > > > > > > > In the many months since we maintain Shenandoah in our 11u, the > > > > overhelming majority of Shenandoah-related backports did not > > > > touch > > > > any > > > > shared code at all. That's because the existing GC interfaces > > > > isolate > > > > GCs really well in 11u. The few occasions where we did change > > > > shared- > > > > code was 1. When we switched to LRB. We would not usually > > > > backport > > > > such > > > > drastic changes, but it seemed prudent here, because it did > > > > actually > > > > decrease the shared-code exposure drastically. 2. in > > > > preparation > > > > for > > > > this upstreaming patch - more pruning and isolation of shared- > > > > code > > > > changes. > > > > > > > > In the very rare situations where a backport would require > > > > shared- > > > > code > > > > changes (I can't actually remember any, apart from the two that > > > > I > > > > just > > > > mentioned), we carefully consider if it's actually necessary to > > > > backport. For example, we have not backported concurrent class- > > > > unloading support to 11u precisely because it would require > > > > (significant) changes outside of Shenandoah. *If* a critical > > > > backport > > > > (say, a bugfix) requires changes outside of Shenandoah, it > > > > would > > > > have > > > > to be properly guarded by the same means as we do in the > > > > proposed > > > > upstreaming patch. We - the Shenandoah team - would be aware of > > > > that > > > > and mention it in relevant reviews. It would also prominently > > > > show > > > > up > > > > in a patch because it has files without 'shenandoah' in their > > > > path > > > > names. And from there, it's a matter of carefully considering, > > > > reviewing and testing it. I don't think this would silently > > > > sneak > > > > in > > > > somehow. > > > > > > > > I can't think of a situation where we ever had the reverse > > > > problem: > > > > a > > > > shared-code change touching on something Shenandoah-related. > > > > > > > > Also, while we did have a flurry of Shenandoah-related > > > > backports in > > > > the > > > > past, because of stabilization and new features (e.g. LRB and > > > > friends), > > > > this is most likely not going to continue into the future. I > > > > expect > > > > less Shenandoah-backporting traffic, basically limited to > > > > bugfixes > > > > and > > > > improvements that don't touch shared-code. We have a couple of > > > > features > > > > on our todo list for later JDKs, but none of them sound like > > > > candidates > > > > for backporting. > > > > > > > > > The notion that our normal review processes will catch > > > > > everything > > > > > that can break > > > > > that initial prooved state seems a bit optimistic to me. The > > > > > review > > > > > process will be > > > > > well intentioned, and we'll try to tag things right, but one > > > > > mistaken > > > > > application or > > > > > move of code across ifdef lines, or integration of some mis- > > > > > tagged or > > > > > unnoticed tag > > > > > thing into shared code will break the statement? > > > > > > > > > > > > > How is that any different from the situation that we already > > > > have? > > > > Hotspot code already has similar #ifdefs sprinkled all around, > > > > e.g. > > > > for > > > > other GCs, for JFR, platform #ifdefs, and so on. How is the > > > > situation > > > > different for Shenandoah, why do we need special rules or > > > > process > > > > or > > > > even proofs for that? As far as I can see, our current high- > > > > standard > > > > development and review practices already cover it very well. > > > > > > > > > I believe that we can avoid this once, at the start, with a > > > > > mechanical proof. Can > > > > > we keep it up? What sort fo rules or invariants can we come > > > > > up > > > > > with > > > > > that we can > > > > > use to verify and show that the quality we seek to keep has > > > > > been > > > > > kept > > > > > through later > > > > > updates? > > > > > > > > Well yes, the usual proper care that everybody is taking in > > > > making > > > > the > > > > backports, reviewing the backports, testing it, etc. > > > > > > > > > Let's try to think of some?. > > > > > > > > As long as we don't know what the exact question/problem even > > > > might > > > > be, > > > > we can hardly come up with an answer/solution. > > > > > > I'm not trying to be difficult here. I'm just going with the > > > basic > > > line > > > of logic, and looking for a way to implement it. > > > > > > > Yeah. Let me just state for the record, that I find this is getting > > slightly ridiculous ;-) > > > > Let me try anyway. > > > > It looks to me that this is a prime example for proof-by-induction. > > Let's try to formulate it. > > > > State 0 is our current jdk11u state. Shenandoah does obviously not > > leak > > into the build, because it's not included yet. > > > > State 1 is the state after the initial inclusion of Shenandoah. > > Let's > > assume we can prove that between 0 and 1, nothing leaks into the > > build > > when building with --with-jvm-features=-shenandoahgc. That proof > > will > > have to be determined, maybe comparing object files would work. > > > > Then, for any further changeset to be backported, this would be the > > N- > > > N+1 case: > > > > - if the changeset is not Shenandoah-related, it'll obviously > > change > > the outcome of the build, but also obviously doesn't leak any > > Shenandoah-related changes. > > - otherwise, if the changeset is Shenandoah-related, we can run the > > same proof that we did from 0->1 for N->N+1, and proof that no > > additional Shenandoah-related changes leaks into non-Shenandoah > > build. > > > > Right? > > > > It depends on the correct classification what constitutes a > > Shenandoah- > > related change and what doesn't. But it must, I see no way around > > that. > > From my perspective and in my experience, this is really easy > > though, > > and can be achieved by applying some common sense. (Hey, when > > reviewing, you really ought to look at the bug - and spot gc- > > shenandoah > > label - and also look at the patch and understand what it all does, > > and > > come to some conclusion.) > > > > > > Now, about that proof: I will spend the day looking if we can do it > > by > > comparing object files of builds. E.g. do builds (-shenandoahgc) > > before/after a change is applied, run checksums over each object > > file, > > and compare those checksums. Let's see if there's any pitfalls > > there. > > > > What do you think? > > > > Roman > > From aleksei.voitylov at bell-sw.com Thu Jul 9 11:47:16 2020 From: aleksei.voitylov at bell-sw.com (Aleksei Voitylov) Date: Thu, 9 Jul 2020 14:47:16 +0300 Subject: RFR (11u, XXL): Upstream/backport Shenandoah to JDK11u In-Reply-To: References: <3595e963-fa5b-bfce-c463-841b84d4103f@redhat.com> <6468CB08-3DB5-488E-9122-E3699A95218B@amazon.com> <33d0bd6339d4c9455c3309baa48e38e621fb9a79.camel@redhat.com> <4F72AD88-0165-497B-B32A-212A5AB076F0@azul.com> <7959350D-EABE-461E-90AB-167214C55D32@azul.com> <39b7b9292d64cfc77aa3b6ad48e5d922a4ceb5b4.camel@redhat.com> Message-ID: <92b8b029-c9ea-606f-4509-f6374dba5aa5@bell-sw.com> Hi Roman, I briefly looked at http://cr.openjdk.java.net/~rkennke/shenandoah-jdk11u-upstream/webrev.06-all/, assuming this is the most recent diff. While the code itself is mostly guarded by #ifdefs, I see a couple of extra includes here and there which are not guarded by #ifdefs. Did you verify this does not do some redifinition? Assuming this moves forward, we plan to test the most recent patch on some architectures that will not support Shenandoah in a couple of weeks and it would be good to know what to look for in terms of binary differences. Last, what is your expectation for the Shenandoah supported platforms for 11u? I'd assume x86_64, x86_32 and AArch64 will be supported by Red Hat, if integrated, right? -Aleksei On 09/07/2020 12:56, Roman Kennke wrote: > The only change that should affect libjvm.so is this one: > > https://cr.openjdk.java.net/~rkennke/shenandoah-jdk11u-upstream/webrev.06-shared/src/hotspot/share/gc/shared/gc_globals.hpp.udiff.html > > (the exposure of the UseShenandoahGC flag) and > > https://cr.openjdk.java.net/~rkennke/shenandoah-jdk11u-upstream/webrev.06-shared/src/hotspot/share/gc/shared/gcConfig.cpp.udiff.html > > ... the NON_SHENANDOAHGC part that prints the failure and exists when > that flag is selected. > > ... all assuming that I made no mistakes in the rest of the -shared > changes. > > Roman > > > On Thu, 2020-07-09 at 09:40 +0000, Lindenmaier, Goetz wrote: >> Hi Roman, >> >> What about creating a webrev with those changes that >> will be compiled if you configure without shenandoahgc? >> I know there is a webrev with only the shared changes, >> but they contain a lot of #define coding, or such under >> protection by the flag that enables Shenandoah, which >> should be constant 'false' if Shenandoah is disabled, right? >> As I read the code, there should remain only a few. >> >> Then we can easily assess the actual risk. If something >> appears risky for the existing code, we can guard it >> by #ifdefs. >> >> Best regards, >> Goetz. >> >>> -----Original Message----- >>> From: jdk-updates-dev On >>> Behalf >>> Of Roman Kennke >>> Sent: Thursday, July 9, 2020 7:57 AM >>> To: Gil Tene >>> Cc: Bernd Mathiske ; jdk-updates- >>> dev at openjdk.java.net; Nilsen, Kelvin ; Jiva, >>> Azeem >>> >>> Subject: Re: RFR (11u, XXL): Upstream/backport Shenandoah to JDK11u >>> >>> On Thu, 2020-07-09 at 04:53 +0000, Gil Tene wrote: >>>>> On Jul 8, 2020, at 4:07 PM, Roman Kennke >>>>> wrote: >>>>> >>>>> On Wed, 2020-07-08 at 22:35 +0000, Gil Tene wrote: >>>>>>> On Jul 8, 2020, at 2:58 PM, Roman Kennke < >>>>>>> rkennke at redhat.com> >>>>>>> wrote: >>>>>>> >>>>>>> ... >>>>>>> In other words: what exactly is it that you want to prove >>>>>>> then? >>>>>>> I >>>>>>> don't >>>>>>> understand it. >>>>>> I'm not sure I understand it either. ;-) >>>>> Haha, ok right :-) >>>>> >>>>>> I'm basically asking the "if we prove it now", how do we know >>>>>> that >>>>>> the >>>>>> same holds later?" question. >>>>>> >>>>>> Delaying some big impact by one update is clearly not the >>>>>> goal >>>>>> you >>>>>> are proposing. >>>>> I don't understand this sentence (but it's quite late here...) >>>>> >>>>>> I believe we agree that the quality you suggest would need to >>>>>> be >>>>>> retained into future >>>>>> builds. Having nothing to prevent it from breaking (e.g. by >>>>>> accident) >>>>>> is probably >>>>>> dangerous. So the question becomes "how do we ask the >>>>>> question of >>>>>> whether or not >>>>>> we still have not introduced any differences?" in future >>>>>> updates. >>>>> Ok. >>>>> >>>>>> Let's start from the assumption that we can prove something >>>>>> now >>>>>> (and >>>>>> I >>>>>> do think we should be able to find a mechanical thing to do >>>>>> that). >>>>>> That will show >>>>>> that the quality we seek to keep has been kept at a point in >>>>>> time... >>>>> Yes. Let's assume that. (As an aside, if you'd actually look at >>>>> the >>>>> patch, I hope you'd find it quite obvious that it does the >>>>> right >>>>> thing. >>>>> And you might be surprised how relatively few such changes we >>>>> actually >>>>> have.) >>>>> >>>>>> For as long as the statement of "without Shenandoah enabled, >>>>>> the >>>>>> build >>>>>> does not have any actual code bits related to the Shenandoah >>>>>> back- >>>>>> port" >>>>>> needs to hold, we need a way to show that the same still >>>>>> holds, >>>>>> Either by >>>>>> proving that the original proof still holds in the presence >>>>>> of >>>>>> changes, or >>>>>> by re-proving it somehow. >>>>>> >>>>> Let me elaborate a little bit on my experiences with backports, >>>>> and >>>>> mixing both Shenandoah and non-Shenandoah backports. >>>>> >>>>> In the many months since we maintain Shenandoah in our 11u, the >>>>> overhelming majority of Shenandoah-related backports did not >>>>> touch >>>>> any >>>>> shared code at all. That's because the existing GC interfaces >>>>> isolate >>>>> GCs really well in 11u. The few occasions where we did change >>>>> shared- >>>>> code was 1. When we switched to LRB. We would not usually >>>>> backport >>>>> such >>>>> drastic changes, but it seemed prudent here, because it did >>>>> actually >>>>> decrease the shared-code exposure drastically. 2. in >>>>> preparation >>>>> for >>>>> this upstreaming patch - more pruning and isolation of shared- >>>>> code >>>>> changes. >>>>> >>>>> In the very rare situations where a backport would require >>>>> shared- >>>>> code >>>>> changes (I can't actually remember any, apart from the two that >>>>> I >>>>> just >>>>> mentioned), we carefully consider if it's actually necessary to >>>>> backport. For example, we have not backported concurrent class- >>>>> unloading support to 11u precisely because it would require >>>>> (significant) changes outside of Shenandoah. *If* a critical >>>>> backport >>>>> (say, a bugfix) requires changes outside of Shenandoah, it >>>>> would >>>>> have >>>>> to be properly guarded by the same means as we do in the >>>>> proposed >>>>> upstreaming patch. We - the Shenandoah team - would be aware of >>>>> that >>>>> and mention it in relevant reviews. It would also prominently >>>>> show >>>>> up >>>>> in a patch because it has files without 'shenandoah' in their >>>>> path >>>>> names. And from there, it's a matter of carefully considering, >>>>> reviewing and testing it. I don't think this would silently >>>>> sneak >>>>> in >>>>> somehow. >>>>> >>>>> I can't think of a situation where we ever had the reverse >>>>> problem: >>>>> a >>>>> shared-code change touching on something Shenandoah-related. >>>>> >>>>> Also, while we did have a flurry of Shenandoah-related >>>>> backports in >>>>> the >>>>> past, because of stabilization and new features (e.g. LRB and >>>>> friends), >>>>> this is most likely not going to continue into the future. I >>>>> expect >>>>> less Shenandoah-backporting traffic, basically limited to >>>>> bugfixes >>>>> and >>>>> improvements that don't touch shared-code. We have a couple of >>>>> features >>>>> on our todo list for later JDKs, but none of them sound like >>>>> candidates >>>>> for backporting. >>>>> >>>>>> The notion that our normal review processes will catch >>>>>> everything >>>>>> that can break >>>>>> that initial prooved state seems a bit optimistic to me. The >>>>>> review >>>>>> process will be >>>>>> well intentioned, and we'll try to tag things right, but one >>>>>> mistaken >>>>>> application or >>>>>> move of code across ifdef lines, or integration of some mis- >>>>>> tagged or >>>>>> unnoticed tag >>>>>> thing into shared code will break the statement? >>>>>> >>>>> How is that any different from the situation that we already >>>>> have? >>>>> Hotspot code already has similar #ifdefs sprinkled all around, >>>>> e.g. >>>>> for >>>>> other GCs, for JFR, platform #ifdefs, and so on. How is the >>>>> situation >>>>> different for Shenandoah, why do we need special rules or >>>>> process >>>>> or >>>>> even proofs for that? As far as I can see, our current high- >>>>> standard >>>>> development and review practices already cover it very well. >>>>> >>>>>> I believe that we can avoid this once, at the start, with a >>>>>> mechanical proof. Can >>>>>> we keep it up? What sort fo rules or invariants can we come >>>>>> up >>>>>> with >>>>>> that we can >>>>>> use to verify and show that the quality we seek to keep has >>>>>> been >>>>>> kept >>>>>> through later >>>>>> updates? >>>>> Well yes, the usual proper care that everybody is taking in >>>>> making >>>>> the >>>>> backports, reviewing the backports, testing it, etc. >>>>> >>>>>> Let's try to think of some?. >>>>> As long as we don't know what the exact question/problem even >>>>> might >>>>> be, >>>>> we can hardly come up with an answer/solution. >>>> I'm not trying to be difficult here. I'm just going with the >>>> basic >>>> line >>>> of logic, and looking for a way to implement it. >>>> >>> Yeah. Let me just state for the record, that I find this is getting >>> slightly ridiculous ;-) >>> >>> Let me try anyway. >>> >>> It looks to me that this is a prime example for proof-by-induction. >>> Let's try to formulate it. >>> >>> State 0 is our current jdk11u state. Shenandoah does obviously not >>> leak >>> into the build, because it's not included yet. >>> >>> State 1 is the state after the initial inclusion of Shenandoah. >>> Let's >>> assume we can prove that between 0 and 1, nothing leaks into the >>> build >>> when building with --with-jvm-features=-shenandoahgc. That proof >>> will >>> have to be determined, maybe comparing object files would work. >>> >>> Then, for any further changeset to be backported, this would be the >>> N- >>>> N+1 case: >>> - if the changeset is not Shenandoah-related, it'll obviously >>> change >>> the outcome of the build, but also obviously doesn't leak any >>> Shenandoah-related changes. >>> - otherwise, if the changeset is Shenandoah-related, we can run the >>> same proof that we did from 0->1 for N->N+1, and proof that no >>> additional Shenandoah-related changes leaks into non-Shenandoah >>> build. >>> >>> Right? >>> >>> It depends on the correct classification what constitutes a >>> Shenandoah- >>> related change and what doesn't. But it must, I see no way around >>> that. >>> From my perspective and in my experience, this is really easy >>> though, >>> and can be achieved by applying some common sense. (Hey, when >>> reviewing, you really ought to look at the bug - and spot gc- >>> shenandoah >>> label - and also look at the patch and understand what it all does, >>> and >>> come to some conclusion.) >>> >>> >>> Now, about that proof: I will spend the day looking if we can do it >>> by >>> comparing object files of builds. E.g. do builds (-shenandoahgc) >>> before/after a change is applied, run checksums over each object >>> file, >>> and compare those checksums. Let's see if there's any pitfalls >>> there. >>> >>> What do you think? >>> >>> Roman >>> From rkennke at redhat.com Thu Jul 9 12:17:09 2020 From: rkennke at redhat.com (Roman Kennke) Date: Thu, 09 Jul 2020 14:17:09 +0200 Subject: RFR (11u, XXL): Upstream/backport Shenandoah to JDK11u In-Reply-To: <92b8b029-c9ea-606f-4509-f6374dba5aa5@bell-sw.com> References: <3595e963-fa5b-bfce-c463-841b84d4103f@redhat.com> <6468CB08-3DB5-488E-9122-E3699A95218B@amazon.com> <33d0bd6339d4c9455c3309baa48e38e621fb9a79.camel@redhat.com> <4F72AD88-0165-497B-B32A-212A5AB076F0@azul.com> <7959350D-EABE-461E-90AB-167214C55D32@azul.com> <39b7b9292d64cfc77aa3b6ad48e5d922a4ceb5b4.camel@redhat.com> <92b8b029-c9ea-606f-4509-f6374dba5aa5@bell-sw.com> Message-ID: <5851a942b801889fa251148d452a750b05984b36.camel@redhat.com> On Thu, 2020-07-09 at 14:47 +0300, Aleksei Voitylov wrote: > Hi Roman, > > I briefly looked at > http://cr.openjdk.java.net/~rkennke/shenandoah-jdk11u-upstream/webrev.06-all/, > assuming this is the most recent diff. While the code itself is > mostly > guarded by #ifdefs, I see a couple of extra includes here and there > which are not guarded by #ifdefs. Did you verify this does not do > some > redifinition? We need to include "utilities/macros.hpp" wherever we use INCLUDE_SHENANDOAHGC, that header should not define any code. Including other headers would be a mistake. I'm currently trying to verify that the resulting object files are identical. > Assuming this moves forward, we plan to test the most recent patch on > some architectures that will not support Shenandoah in a couple of > weeks > and it would be good to know what to look for in terms of binary > differences. There should be none. It would compile with the flag UseShenandoahGC, and when invoked +UseShenandoahGC it should print an error and exit. > Last, what is your expectation for the Shenandoah supported platforms > for 11u? I'd assume x86_64, x86_32 and AArch64 will be supported by > Red > Hat, if integrated, right? Yes, that is correct. Thanks, Roman > -Aleksei > > On 09/07/2020 12:56, Roman Kennke wrote: > > The only change that should affect libjvm.so is this one: > > > > https://cr.openjdk.java.net/~rkennke/shenandoah-jdk11u-upstream/webrev.06-shared/src/hotspot/share/gc/shared/gc_globals.hpp.udiff.html > > > > (the exposure of the UseShenandoahGC flag) and > > > > https://cr.openjdk.java.net/~rkennke/shenandoah-jdk11u-upstream/webrev.06-shared/src/hotspot/share/gc/shared/gcConfig.cpp.udiff.html > > > > ... the NON_SHENANDOAHGC part that prints the failure and exists > > when > > that flag is selected. > > > > ... all assuming that I made no mistakes in the rest of the -shared > > changes. > > > > Roman > > > > > > On Thu, 2020-07-09 at 09:40 +0000, Lindenmaier, Goetz wrote: > > > Hi Roman, > > > > > > What about creating a webrev with those changes that > > > will be compiled if you configure without shenandoahgc? > > > I know there is a webrev with only the shared changes, > > > but they contain a lot of #define coding, or such under > > > protection by the flag that enables Shenandoah, which > > > should be constant 'false' if Shenandoah is disabled, right? > > > As I read the code, there should remain only a few. > > > > > > Then we can easily assess the actual risk. If something > > > appears risky for the existing code, we can guard it > > > by #ifdefs. > > > > > > Best regards, > > > Goetz. > > > > > > > -----Original Message----- > > > > From: jdk-updates-dev > > > > On > > > > Behalf > > > > Of Roman Kennke > > > > Sent: Thursday, July 9, 2020 7:57 AM > > > > To: Gil Tene > > > > Cc: Bernd Mathiske ; jdk-updates- > > > > dev at openjdk.java.net; Nilsen, Kelvin ; > > > > Jiva, > > > > Azeem > > > > > > > > Subject: Re: RFR (11u, XXL): Upstream/backport Shenandoah to > > > > JDK11u > > > > > > > > On Thu, 2020-07-09 at 04:53 +0000, Gil Tene wrote: > > > > > > On Jul 8, 2020, at 4:07 PM, Roman Kennke < > > > > > > rkennke at redhat.com> > > > > > > wrote: > > > > > > > > > > > > On Wed, 2020-07-08 at 22:35 +0000, Gil Tene wrote: > > > > > > > > On Jul 8, 2020, at 2:58 PM, Roman Kennke < > > > > > > > > rkennke at redhat.com> > > > > > > > > wrote: > > > > > > > > > > > > > > > > ... > > > > > > > > In other words: what exactly is it that you want to > > > > > > > > prove > > > > > > > > then? > > > > > > > > I > > > > > > > > don't > > > > > > > > understand it. > > > > > > > I'm not sure I understand it either. ;-) > > > > > > Haha, ok right :-) > > > > > > > > > > > > > I'm basically asking the "if we prove it now", how do we > > > > > > > know > > > > > > > that > > > > > > > the > > > > > > > same holds later?" question. > > > > > > > > > > > > > > Delaying some big impact by one update is clearly not the > > > > > > > goal > > > > > > > you > > > > > > > are proposing. > > > > > > I don't understand this sentence (but it's quite late > > > > > > here...) > > > > > > > > > > > > > I believe we agree that the quality you suggest would > > > > > > > need to > > > > > > > be > > > > > > > retained into future > > > > > > > builds. Having nothing to prevent it from breaking (e.g. > > > > > > > by > > > > > > > accident) > > > > > > > is probably > > > > > > > dangerous. So the question becomes "how do we ask the > > > > > > > question of > > > > > > > whether or not > > > > > > > we still have not introduced any differences?" in future > > > > > > > updates. > > > > > > Ok. > > > > > > > > > > > > > Let's start from the assumption that we can prove > > > > > > > something > > > > > > > now > > > > > > > (and > > > > > > > I > > > > > > > do think we should be able to find a mechanical thing to > > > > > > > do > > > > > > > that). > > > > > > > That will show > > > > > > > that the quality we seek to keep has been kept at a point > > > > > > > in > > > > > > > time... > > > > > > Yes. Let's assume that. (As an aside, if you'd actually > > > > > > look at > > > > > > the > > > > > > patch, I hope you'd find it quite obvious that it does the > > > > > > right > > > > > > thing. > > > > > > And you might be surprised how relatively few such changes > > > > > > we > > > > > > actually > > > > > > have.) > > > > > > > > > > > > > For as long as the statement of "without Shenandoah > > > > > > > enabled, > > > > > > > the > > > > > > > build > > > > > > > does not have any actual code bits related to the > > > > > > > Shenandoah > > > > > > > back- > > > > > > > port" > > > > > > > needs to hold, we need a way to show that the same still > > > > > > > holds, > > > > > > > Either by > > > > > > > proving that the original proof still holds in the > > > > > > > presence > > > > > > > of > > > > > > > changes, or > > > > > > > by re-proving it somehow. > > > > > > > > > > > > > Let me elaborate a little bit on my experiences with > > > > > > backports, > > > > > > and > > > > > > mixing both Shenandoah and non-Shenandoah backports. > > > > > > > > > > > > In the many months since we maintain Shenandoah in our 11u, > > > > > > the > > > > > > overhelming majority of Shenandoah-related backports did > > > > > > not > > > > > > touch > > > > > > any > > > > > > shared code at all. That's because the existing GC > > > > > > interfaces > > > > > > isolate > > > > > > GCs really well in 11u. The few occasions where we did > > > > > > change > > > > > > shared- > > > > > > code was 1. When we switched to LRB. We would not usually > > > > > > backport > > > > > > such > > > > > > drastic changes, but it seemed prudent here, because it did > > > > > > actually > > > > > > decrease the shared-code exposure drastically. 2. in > > > > > > preparation > > > > > > for > > > > > > this upstreaming patch - more pruning and isolation of > > > > > > shared- > > > > > > code > > > > > > changes. > > > > > > > > > > > > In the very rare situations where a backport would require > > > > > > shared- > > > > > > code > > > > > > changes (I can't actually remember any, apart from the two > > > > > > that > > > > > > I > > > > > > just > > > > > > mentioned), we carefully consider if it's actually > > > > > > necessary to > > > > > > backport. For example, we have not backported concurrent > > > > > > class- > > > > > > unloading support to 11u precisely because it would require > > > > > > (significant) changes outside of Shenandoah. *If* a > > > > > > critical > > > > > > backport > > > > > > (say, a bugfix) requires changes outside of Shenandoah, it > > > > > > would > > > > > > have > > > > > > to be properly guarded by the same means as we do in the > > > > > > proposed > > > > > > upstreaming patch. We - the Shenandoah team - would be > > > > > > aware of > > > > > > that > > > > > > and mention it in relevant reviews. It would also > > > > > > prominently > > > > > > show > > > > > > up > > > > > > in a patch because it has files without 'shenandoah' in > > > > > > their > > > > > > path > > > > > > names. And from there, it's a matter of carefully > > > > > > considering, > > > > > > reviewing and testing it. I don't think this would silently > > > > > > sneak > > > > > > in > > > > > > somehow. > > > > > > > > > > > > I can't think of a situation where we ever had the reverse > > > > > > problem: > > > > > > a > > > > > > shared-code change touching on something Shenandoah- > > > > > > related. > > > > > > > > > > > > Also, while we did have a flurry of Shenandoah-related > > > > > > backports in > > > > > > the > > > > > > past, because of stabilization and new features (e.g. LRB > > > > > > and > > > > > > friends), > > > > > > this is most likely not going to continue into the future. > > > > > > I > > > > > > expect > > > > > > less Shenandoah-backporting traffic, basically limited to > > > > > > bugfixes > > > > > > and > > > > > > improvements that don't touch shared-code. We have a couple > > > > > > of > > > > > > features > > > > > > on our todo list for later JDKs, but none of them sound > > > > > > like > > > > > > candidates > > > > > > for backporting. > > > > > > > > > > > > > The notion that our normal review processes will catch > > > > > > > everything > > > > > > > that can break > > > > > > > that initial prooved state seems a bit optimistic to me. > > > > > > > The > > > > > > > review > > > > > > > process will be > > > > > > > well intentioned, and we'll try to tag things right, but > > > > > > > one > > > > > > > mistaken > > > > > > > application or > > > > > > > move of code across ifdef lines, or integration of some > > > > > > > mis- > > > > > > > tagged or > > > > > > > unnoticed tag > > > > > > > thing into shared code will break the statement? > > > > > > > > > > > > > How is that any different from the situation that we > > > > > > already > > > > > > have? > > > > > > Hotspot code already has similar #ifdefs sprinkled all > > > > > > around, > > > > > > e.g. > > > > > > for > > > > > > other GCs, for JFR, platform #ifdefs, and so on. How is the > > > > > > situation > > > > > > different for Shenandoah, why do we need special rules or > > > > > > process > > > > > > or > > > > > > even proofs for that? As far as I can see, our current > > > > > > high- > > > > > > standard > > > > > > development and review practices already cover it very > > > > > > well. > > > > > > > > > > > > > I believe that we can avoid this once, at the start, with > > > > > > > a > > > > > > > mechanical proof. Can > > > > > > > we keep it up? What sort fo rules or invariants can we > > > > > > > come > > > > > > > up > > > > > > > with > > > > > > > that we can > > > > > > > use to verify and show that the quality we seek to keep > > > > > > > has > > > > > > > been > > > > > > > kept > > > > > > > through later > > > > > > > updates? > > > > > > Well yes, the usual proper care that everybody is taking in > > > > > > making > > > > > > the > > > > > > backports, reviewing the backports, testing it, etc. > > > > > > > > > > > > > Let's try to think of some?. > > > > > > As long as we don't know what the exact question/problem > > > > > > even > > > > > > might > > > > > > be, > > > > > > we can hardly come up with an answer/solution. > > > > > I'm not trying to be difficult here. I'm just going with the > > > > > basic > > > > > line > > > > > of logic, and looking for a way to implement it. > > > > > > > > > Yeah. Let me just state for the record, that I find this is > > > > getting > > > > slightly ridiculous ;-) > > > > > > > > Let me try anyway. > > > > > > > > It looks to me that this is a prime example for proof-by- > > > > induction. > > > > Let's try to formulate it. > > > > > > > > State 0 is our current jdk11u state. Shenandoah does obviously > > > > not > > > > leak > > > > into the build, because it's not included yet. > > > > > > > > State 1 is the state after the initial inclusion of Shenandoah. > > > > Let's > > > > assume we can prove that between 0 and 1, nothing leaks into > > > > the > > > > build > > > > when building with --with-jvm-features=-shenandoahgc. That > > > > proof > > > > will > > > > have to be determined, maybe comparing object files would work. > > > > > > > > Then, for any further changeset to be backported, this would be > > > > the > > > > N- > > > > > N+1 case: > > > > - if the changeset is not Shenandoah-related, it'll obviously > > > > change > > > > the outcome of the build, but also obviously doesn't leak any > > > > Shenandoah-related changes. > > > > - otherwise, if the changeset is Shenandoah-related, we can run > > > > the > > > > same proof that we did from 0->1 for N->N+1, and proof that no > > > > additional Shenandoah-related changes leaks into non-Shenandoah > > > > build. > > > > > > > > Right? > > > > > > > > It depends on the correct classification what constitutes a > > > > Shenandoah- > > > > related change and what doesn't. But it must, I see no way > > > > around > > > > that. > > > > From my perspective and in my experience, this is really easy > > > > though, > > > > and can be achieved by applying some common sense. (Hey, when > > > > reviewing, you really ought to look at the bug - and spot gc- > > > > shenandoah > > > > label - and also look at the patch and understand what it all > > > > does, > > > > and > > > > come to some conclusion.) > > > > > > > > > > > > Now, about that proof: I will spend the day looking if we can > > > > do it > > > > by > > > > comparing object files of builds. E.g. do builds (- > > > > shenandoahgc) > > > > before/after a change is applied, run checksums over each > > > > object > > > > file, > > > > and compare those checksums. Let's see if there's any pitfalls > > > > there. > > > > > > > > What do you think? > > > > > > > > Roman > > > > From aleksei.voitylov at bell-sw.com Thu Jul 9 12:33:53 2020 From: aleksei.voitylov at bell-sw.com (Aleksei Voitylov) Date: Thu, 9 Jul 2020 15:33:53 +0300 Subject: RFR (11u, XXL): Upstream/backport Shenandoah to JDK11u In-Reply-To: <5851a942b801889fa251148d452a750b05984b36.camel@redhat.com> References: <3595e963-fa5b-bfce-c463-841b84d4103f@redhat.com> <6468CB08-3DB5-488E-9122-E3699A95218B@amazon.com> <33d0bd6339d4c9455c3309baa48e38e621fb9a79.camel@redhat.com> <4F72AD88-0165-497B-B32A-212A5AB076F0@azul.com> <7959350D-EABE-461E-90AB-167214C55D32@azul.com> <39b7b9292d64cfc77aa3b6ad48e5d922a4ceb5b4.camel@redhat.com> <92b8b029-c9ea-606f-4509-f6374dba5aa5@bell-sw.com> <5851a942b801889fa251148d452a750b05984b36.camel@redhat.com> Message-ID: <04200c97-f33b-1c5e-c138-8b41c4b66b02@bell-sw.com> That would be awesome, thanks! I also saw interpreter/interpreter.hpp and a couple of other unguarded modifications. While not strictly proving anything, guarding more code with #ifdefs might be an option to simplify verification. And further reduce risks for platforms that won't support Shenandoah (or help maintain compatibility for those who prefer to build without it). -Aleksei On 09/07/2020 15:17, Roman Kennke wrote: > We need to include "utilities/macros.hpp" wherever we use > INCLUDE_SHENANDOAHGC, that header should not define any code. Including > other headers would be a mistake. I'm currently trying to verify that > the resulting object files are identical. From rkennke at redhat.com Thu Jul 9 15:04:17 2020 From: rkennke at redhat.com (Roman Kennke) Date: Thu, 09 Jul 2020 17:04:17 +0200 Subject: RFR (11u, XXL): Upstream/backport Shenandoah to JDK11u In-Reply-To: <39b7b9292d64cfc77aa3b6ad48e5d922a4ceb5b4.camel@redhat.com> References: <3595e963-fa5b-bfce-c463-841b84d4103f@redhat.com> <6468CB08-3DB5-488E-9122-E3699A95218B@amazon.com> <33d0bd6339d4c9455c3309baa48e38e621fb9a79.camel@redhat.com> <4F72AD88-0165-497B-B32A-212A5AB076F0@azul.com> <7959350D-EABE-461E-90AB-167214C55D32@azul.com> <39b7b9292d64cfc77aa3b6ad48e5d922a4ceb5b4.camel@redhat.com> Message-ID: <76c375665982c3f9f41d38ff6019938eec281441.camel@redhat.com> > Now, about that proof: I will spend the day looking if we can do it > by > comparing object files of builds. E.g. do builds (-shenandoahgc) > before/after a change is applied, run checksums over each object > file, > and compare those checksums. Let's see if there's any pitfalls there. > I digged a little into this. I built JDK11u with and without the patch, and without debug-symbols (important), and compared the resulting objects. I number of object files differ in their generated code, and they all look like: mov $0x3e, %eax vs mov $0x44, %eax In other words, constant numbers are different. I very strongly suspect that this is line numbers that are caught by __LINE__, for example in guarantee(). There are a number of other places where __LINE__ is used. Line numbers are not expected to be equal before/after the patch. I could probably hack around that -- but would that still be in the spirit of what we want to achieve? I have my doubts. Roman From rkennke at redhat.com Thu Jul 9 15:06:40 2020 From: rkennke at redhat.com (Roman Kennke) Date: Thu, 09 Jul 2020 17:06:40 +0200 Subject: RFR (11u, XXL): Upstream/backport Shenandoah to JDK11u In-Reply-To: <04200c97-f33b-1c5e-c138-8b41c4b66b02@bell-sw.com> References: <3595e963-fa5b-bfce-c463-841b84d4103f@redhat.com> <6468CB08-3DB5-488E-9122-E3699A95218B@amazon.com> <33d0bd6339d4c9455c3309baa48e38e621fb9a79.camel@redhat.com> <4F72AD88-0165-497B-B32A-212A5AB076F0@azul.com> <7959350D-EABE-461E-90AB-167214C55D32@azul.com> <39b7b9292d64cfc77aa3b6ad48e5d922a4ceb5b4.camel@redhat.com> <92b8b029-c9ea-606f-4509-f6374dba5aa5@bell-sw.com> <5851a942b801889fa251148d452a750b05984b36.camel@redhat.com> <04200c97-f33b-1c5e-c138-8b41c4b66b02@bell-sw.com> Message-ID: On Thu, 2020-07-09 at 15:33 +0300, Aleksei Voitylov wrote: > That would be awesome, thanks! > > I also saw interpreter/interpreter.hpp Yes. I could wrap it in #if INCLUDE_SHENANDOAHGC, but it's actually a correct addition because Interpreter is used in that compilation unit. > and a couple of other unguarded > modifications. I only see some lone friend and class declarations, but they wouldn't do anything. If it's easier to review or reduces risks I can put #if INCLUDE_SHENANDOAHGC there too. Cheers, Roman From rkennke at redhat.com Thu Jul 9 17:36:31 2020 From: rkennke at redhat.com (Roman Kennke) Date: Thu, 09 Jul 2020 19:36:31 +0200 Subject: RFR (11u, XXL): Upstream/backport Shenandoah to JDK11u - the review thread Message-ID: <3db8e235f3ab515563802d23997f0ac32d699c48.camel@redhat.com> I'm starting a new thread with a new round of webrevs. Maybe we can keep in-principle discussions on the other thread and get some actual reviews here? Anyhow, small changes since the last one: - the lone include "interpreter.hpp" is gone, apparently it was a left- over that's no longer needed - the 2 friend class declarations are now wrapped in INCLUDE_SHENANDOAHGC too, just to be sure (even though compiler wouldn't compile any actual code) As noted in the other thread, there should not be any Shenandoah code leaking out when building with --with-jvm-features=-shenandoahgc (which is the default too). There's 2 exceptions: - The global UseShenandoahGC flag. When invoked but not built-in, it will print an error message and exit. - In: src/hotspot/share/jfr/periodic/jfrPeriodic.cpp I could not exclude the definition of: TRACE_REQUEST_FUNC(ShenandoahHeapRegionInformation) Because that would be called by generated code. Related to this: src/hotspot/share/jfr/metadata/metadata.xml This generates two event classes for Shenandaoh, but they would not be used. I don't know how to exclude them. It seems harmless to me. Shared-only changes: http://cr.openjdk.java.net/~rkennke/shenandoah-jdk11u-upstream/webrev.07-shared/ Full webrev: http://cr.openjdk.java.net/~rkennke/shenandoah-jdk11u-upstream/webrev.07-all/ Roman From goetz.lindenmaier at sap.com Thu Jul 9 18:55:39 2020 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Thu, 9 Jul 2020 18:55:39 +0000 Subject: [11u] RFR: 8249159: Downport test rework for SSLSocketTemplate from 8224650 Message-ID: Hi, I would like to bring this test rework from 14 to 11. It is part of https://bugs.openjdk.java.net/browse/JDK-8224650 . 8224650 contains the rework of SSLSocketTemplates.java and adds a new test. In the old version of SSLSocketTemplates.java certificates, names and keys are held in separate arrays that must correlate by indexes correctly. The new version uses enums to keep common information in one place. The new version is much easier to maintain. Also, downports that just add new enums are easier to handle. The new test NamedGroupsWithCipherSuite.java does not pass in 11 because the feature tested (JDK-8171279) is only in 13. I would like to downport the rework because it makes JDK-8246330 apply clean. I do not want to downport the new test as it fails. Therefore I opened an Enhancement issue of it's own for this, https://bugs.openjdk.java.net/browse/JDK-8249159 . Please review this webrev: http://cr.openjdk.java.net/~goetz/wr20/8249159-rework_SSLSocketTemplate-jdk11/01/ It contains the changes of SSLSocketTemplate from JDK-8224650. They applied clean. It does not contain the new test. SSLSocketTemplate.java passes with this test. Best regards, Goetz. From gil at azul.com Thu Jul 9 18:59:39 2020 From: gil at azul.com (Gil Tene) Date: Thu, 9 Jul 2020 18:59:39 +0000 Subject: RFR (11u, XXL): Upstream/backport Shenandoah to JDK11u In-Reply-To: <76c375665982c3f9f41d38ff6019938eec281441.camel@redhat.com> References: <3595e963-fa5b-bfce-c463-841b84d4103f@redhat.com> <6468CB08-3DB5-488E-9122-E3699A95218B@amazon.com> <33d0bd6339d4c9455c3309baa48e38e621fb9a79.camel@redhat.com> <4F72AD88-0165-497B-B32A-212A5AB076F0@azul.com> <7959350D-EABE-461E-90AB-167214C55D32@azul.com> <39b7b9292d64cfc77aa3b6ad48e5d922a4ceb5b4.camel@redhat.com> <76c375665982c3f9f41d38ff6019938eec281441.camel@redhat.com> Message-ID: > On Jul 9, 2020, at 8:04 AM, Roman Kennke wrote: > > >> Now, about that proof: I will spend the day looking if we can do it >> by >> comparing object files of builds. E.g. do builds (-shenandoahgc) >> before/after a change is applied, run checksums over each object >> file, >> and compare those checksums. Let's see if there's any pitfalls there. >> > > I digged a little into this. I built JDK11u with and without the patch, > and without debug-symbols (important), and compared the resulting > objects. I number of object files differ in their generated code, and > they all look like: > > mov $0x3e, %eax > vs > mov $0x44, %eax > > In other words, constant numbers are different. I very strongly suspect > that this is line numbers that are caught by __LINE__, for example in > guarantee(). There are a number of other places where __LINE__ is used. > Line numbers are not expected to be equal before/after the patch. I > could probably hack around that -- but would that still be in the > spirit of what we want to achieve? I have my doubts. I agree that a difference purely around line number constants should not be considered a "difference" for the purpose of the proof you are looking for. The question is how to identify that? Maybe instead of object files, we can go for differences in post-processed sources (basically the actual input to compilation)? I have a feeling that we'll run into issues there too, but at least there you can do some more textual analysis? > > Roman From rkennke at redhat.com Thu Jul 9 19:08:05 2020 From: rkennke at redhat.com (Roman Kennke) Date: Thu, 09 Jul 2020 21:08:05 +0200 Subject: RFR (11u, XXL): Upstream/backport Shenandoah to JDK11u In-Reply-To: References: <3595e963-fa5b-bfce-c463-841b84d4103f@redhat.com> <6468CB08-3DB5-488E-9122-E3699A95218B@amazon.com> <33d0bd6339d4c9455c3309baa48e38e621fb9a79.camel@redhat.com> <4F72AD88-0165-497B-B32A-212A5AB076F0@azul.com> <7959350D-EABE-461E-90AB-167214C55D32@azul.com> <39b7b9292d64cfc77aa3b6ad48e5d922a4ceb5b4.camel@redhat.com> <76c375665982c3f9f41d38ff6019938eec281441.camel@redhat.com> Message-ID: <611794ce054fff91804774c94863239448a43e5a.camel@redhat.com> On Thu, 2020-07-09 at 18:59 +0000, Gil Tene wrote: > > On Jul 9, 2020, at 8:04 AM, Roman Kennke > > wrote: > > > > > > > Now, about that proof: I will spend the day looking if we can do > > > it > > > by > > > comparing object files of builds. E.g. do builds (-shenandoahgc) > > > before/after a change is applied, run checksums over each object > > > file, > > > and compare those checksums. Let's see if there's any pitfalls > > > there. > > > > > > > I digged a little into this. I built JDK11u with and without the > > patch, > > and without debug-symbols (important), and compared the resulting > > objects. I number of object files differ in their generated code, > > and > > they all look like: > > > > mov $0x3e, %eax > > vs > > mov $0x44, %eax > > > > In other words, constant numbers are different. I very strongly > > suspect > > that this is line numbers that are caught by __LINE__, for example > > in > > guarantee(). There are a number of other places where __LINE__ is > > used. > > Line numbers are not expected to be equal before/after the patch. I > > could probably hack around that -- but would that still be in the > > spirit of what we want to achieve? I have my doubts. > > I agree that a difference purely around line number constants should > not > be considered a "difference" for the purpose of the proof you are > looking > for. The question is how to identify that? > > Maybe instead of object files, we can go for differences in post- > processed > sources (basically the actual input to compilation)? I have a feeling > that we'll > run into issues there too, but at least there you can do some more > textual > analysis? Yeah, I was thinking that too, but we have various places where we need to include "utilities/macros.hpp" in order to get INCLUDE_SHENANDOAH_GC and filenames, line-numbers, etc would be expanded into several files too. It wouldn't be clean and require manual review - at which point we can just as well review the original source changes, that seems easier to read. Roman From aph at redhat.com Fri Jul 10 08:32:12 2020 From: aph at redhat.com (Andrew Haley) Date: Fri, 10 Jul 2020 09:32:12 +0100 Subject: RFR (11u, XXL): Upstream/backport Shenandoah to JDK11u In-Reply-To: <7959350D-EABE-461E-90AB-167214C55D32@azul.com> References: <3595e963-fa5b-bfce-c463-841b84d4103f@redhat.com> <6468CB08-3DB5-488E-9122-E3699A95218B@amazon.com> <33d0bd6339d4c9455c3309baa48e38e621fb9a79.camel@redhat.com> <4F72AD88-0165-497B-B32A-212A5AB076F0@azul.com> <7959350D-EABE-461E-90AB-167214C55D32@azul.com> Message-ID: <6db9defc-88ec-37f0-bc97-2465a8e89148@redhat.com> On 09/07/2020 05:53, Gil Tene wrote: > That is the key point of agreement, and a powerful argument. It is a > way to remove my original concerns. > > To build to something useful from there, I'm looking to see how we > can both establish the "no change" situation (prove it), and to > maintain that situation as true over time (prove it for the next > update). While the latter is interesting and might be useful if it's not too onerous, I'm not going to insist on it as a condition for the pushing of this patch. -- Andrew Haley (he/him) Java Platform Lead Engineer Red Hat UK Ltd. https://keybase.io/andrewhaley EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From matthias.baesken at sap.com Fri Jul 10 10:45:43 2020 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Fri, 10 Jul 2020 10:45:43 +0000 Subject: 8236464: SO_LINGER option is ignored by SSLSocket in JDK 11 Message-ID: Hello G?tz, the patch itself looks good . However I noticed that in src/java.base/share/classes/sun/security/ssl/SSLSocketImpl.java void closeNotify(boolean useUserCanceled) throws IOException {..} there are some more diffs related to SO_LINGER/linger when comparing jdk/jdk and jdk11 . Have you checked those , do we want them in jdk11 too ? Best regards, Matthias --------------------------------- Hi, I am downporting this for parity with 11.0.9-oracle. I had to do quite some adaptions: http://cr.openjdk.java.net/~goetz/wr20/8236464-SSLSocket_LINGER-jdk11/01/ src/java.base/share/classes/sun/security/ssl/SSLSocketImpl.java The change renames a method that is not there in 11. I skipped the renaming and applied the functional change that is in the same chunk. src/java.base/share/classes/sun/security/ssl/TransportContext.java The code deleted differs, because in 15 explicit locking is used. I added a synchronized block in the method introduced. Please review. https://bugs.openjdk.java.net/browse/JDK-8236464 https://hg.openjdk.java.net/jdk/jdk/rev/c783b60f48db Best regards, Goetz From matthias.baesken at sap.com Fri Jul 10 11:08:55 2020 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Fri, 10 Jul 2020 11:08:55 +0000 Subject: [11u] RFR: 8249159: Downport test rework for SSLSocketTemplate from 8224650 Message-ID: I would like to bring this test rework from 14 to 11. It is part of https://bugs.openjdk.java.net/browse/JDK-8224650 . 8224650 contains the rework of SSLSocketTemplates.java and adds a new test. In the old version of SSLSocketTemplates.java certificates, names and keys are held in separate arrays that must correlate by indexes correctly. The new version uses enums to keep common information in one place. The new version is much easier to maintain. Also, downports that just add new enums are easier to handle. The new test NamedGroupsWithCipherSuite.java does not pass in 11 because the feature tested (JDK-8171279) is only in 13. I would like to downport the rework because it makes JDK-8246330 apply clean. I do not want to downport the new test as it fails. Therefore I opened an Enhancement issue of it's own for this, https://bugs.openjdk.java.net/browse/JDK-8249159 . Please review this webrev: http://cr.openjdk.java.net/~goetz/wr20/8249159-rework_SSLSocketTemplate-jdk11/01/ It contains the changes of SSLSocketTemplate from JDK-8224650. They applied clean. It does not contain the new test. SSLSocketTemplate.java passes with this test. Best regards, Goetz. --------------------------------------------------------- Hi G?tz , while the patch downport looks mostly fine, I wonder about one point. It looks to me like the certs tested changed a bit with your change . If this is really intentional , please confirm and consider your change as reviewed . Otherwise I wonder about for example this cert, I only find it in the new version ( did not check all the certs but noticed this one ). Best regards, Matthias http://cr.openjdk.java.net/~goetz/wr20/8249159-rework_SSLSocketTemplate-jdk11/01/test/jdk/javax/net/ssl/templates/SSLSocketTemplate.java.frames.html 636 CA_ECDSA_SECP384R1( 637 "EC", 638 // SHA384withECDSA, curve secp384r1 639 // Validity 640 // Not Before: Jun 24 08:15:06 2019 GMT 641 // Not After : Jun 19 08:15:06 2039 GMT 642 // Subject Key Identifier: 643 // 0a:93:a9:a0:bf:e7:d5:48:9d:4f:89:15:c6:51:98:80:05:51:4e:4e 644 "-----BEGIN CERTIFICATE-----\n" + 645 "MIICCDCCAY6gAwIBAgIUCpOpoL/n1UidT4kVxlGYgAVRTk4wCgYIKoZIzj0EAwMw\n" + 646 "OzELMAkGA1UEBhMCVVMxDTALBgNVBAoMBEphdmExHTAbBgNVBAsMFFN1bkpTU0Ug\n" + 647 "VGVzdCBTZXJpdmNlMB4XDTE5MDYyNDA4MTUwNloXDTM5MDYxOTA4MTUwNlowOzEL\n" + 648 "MAkGA1UEBhMCVVMxDTALBgNVBAoMBEphdmExHTAbBgNVBAsMFFN1bkpTU0UgVGVz\n" + 649 "dCBTZXJpdmNlMHYwEAYHKoZIzj0CAQYFK4EEACIDYgAENVQN1wXWFdgC6u/dDdiC\n" + 650 "y+WtMTF66oL/0BSm+1ZqsogamzCryawOcHgiuXgWzx5CQ3LuOC+tDFyXpGfHuCvb\n" + 651 "dkzxPrP5n9NrR8/uRPe5l1KOUbchviU8z9cTP+LZxnZDo1MwUTAdBgNVHQ4EFgQU\n" + 652 "SktSFArR1p/5mXV0kyo0RxIVa/UwHwYDVR0jBBgwFoAUSktSFArR1p/5mXV0kyo0\n" + 653 "RxIVa/UwDwYDVR0TAQH/BAUwAwEB/zAKBggqhkjOPQQDAwNoADBlAjBZvoNmq3/v\n" + 654 "RD2gBTyvxjS9h0rsMRLHDnvul/KWngytwGPTOBo0Y8ixQXSjdKoc3rkCMQDkiNgx\n" + 655 "IDxuHedmrLQKIPnVcthTmwv7//jHiqGoKofwChMo2a1P+DQdhszmeHD/ARQ=\n" + 656 "-----END CERTIFICATE-----", From matthias.baesken at sap.com Fri Jul 10 11:33:03 2020 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Fri, 10 Jul 2020 11:33:03 +0000 Subject: [11u] RFR: 8240169: javadoc fails to link to non-modular api docs Message-ID: Hello G?tz , the backport looks good to me . Best regards, Matthias --------------------- Hi I am downporting this for parity with 11.0.9-oracle. http://cr.openjdk.java.net/~goetz/wr20/8240169-javadoc-jdk11/01/ I had to do a row of adaptions to this change. src/jdk.javadoc/share/classes/jdk/javadoc/internal/doclets/toolkit/util/Extern.java The code differs slightly because in 11, configuration.utils is dereferenced everywhere, but in 15 utils is saved in a field of the object. Further down, put() is used in 11 where putIfAbsent() is used in 15. Further, code is formatted differently. Another chunk uses resources.getText, where in 11 it is configuration.getText. In checkLinkCompatibility() I added {} so it looks similar to 15. Besides resolving, I had to add "import javax.tools.Diagnostic.Kind;" JavadocTester.java 15 prints the faulty html file if the test fails. This was introduced in "8210031: implementation for JVM Constants API" which is huge. I would like to include this printing in this change, as it helps a lot to understand the issue if the test fails. It helped me to find out where I had resolved wrong. test/langtools/jdk/javadoc/doclet/testClassCrossReferences/TestClassCrossReferences.java The method is marked public in 15, therefore the patch does not apply. Further, I had to adapt the templates matched. In 11, "?is-external=true" is printed to the urls and "externalLink" is "external-link" in 15. Also, the "Overrides" heading is formatted differently. TestLinkOptionWithModule.java This needed similar adaptions of patterns as TestClassCrossReferences.java. Please review. https://bugs.openjdk.java.net/browse/JDK-8240169 https://hg.openjdk.java.net/jdk/jdk/rev/3b557aef43c4 Best regards, Goetz. From felix.yang at huawei.com Fri Jul 10 11:43:54 2020 From: felix.yang at huawei.com (Yangfei (Felix)) Date: Fri, 10 Jul 2020 11:43:54 +0000 Subject: [aarch64-port-dev ] RFR(S): 8248851: CMS: Missing memory fences between free chunk check and klass read In-Reply-To: <06646cf3-bc21-977f-96e2-1e9a974d1b00@redhat.com> References: <8957BB9E-617D-4E24-9CC4-31AF7D01D12E@oracle.com> <65012FED-2A93-4899-939E-887175AB07AC@oracle.com> <06646cf3-bc21-977f-96e2-1e9a974d1b00@redhat.com> Message-ID: Hi, > -----Original Message----- > From: Andrew Haley [mailto:aph at redhat.com] > Sent: Wednesday, July 8, 2020 5:48 PM > To: Kim Barrett ; Yangfei (Felix) > > Cc: jdk8u-dev at openjdk.java.net; hotspot-gc-dev at openjdk.java.net; > aarch64-port-dev at openjdk.java.net > Subject: Re: [aarch64-port-dev ] RFR(S): 8248851: CMS: Missing memory > fences between free chunk check and klass read > > On 08/07/2020 09:41, Kim Barrett wrote: > >> Since CMS is deprecated from JDK9, I am not sure if it's appropriate to fix > this issue for those JDK9+ versions. > > Deprecated != unsupported. > > > > Yes, it must be done. Thanks. CCing to jdk-updates-dev. I have prepared another two webrevs for jdk11u and jdk13u: Jdk11u-dev: http://cr.openjdk.java.net/~fyang/8248851-11u/ Jdk13u-dev: http://cr.openjdk.java.net/~fyang/8248851-13u/ The only difference lies in copyright year update. Both tiere1-3 tested on aarch64-linux-gnu. I will put jdk11u-fix-request and jdk13u-fix-request label on the issue if they are good. I have also prepared a new webrev for jdk8u incorporating copyright year update: Jdk8u-dev: http://cr.openjdk.java.net/~fyang/8248851-8u/ Thanks, Felix From goetz.lindenmaier at sap.com Mon Jul 13 06:22:01 2020 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Mon, 13 Jul 2020 06:22:01 +0000 Subject: 8236464: SO_LINGER option is ignored by SSLSocket in JDK 11 In-Reply-To: References: Message-ID: Hi Matthias, Thanks for your review. The further changes in this function stem from: 8221882: Use fiber-friendly java.util.concurrent.locks in JSSE 8224829: AsyncSSLSocketClose.java has timing issue Obviously, we don't need fiber preparation in 11. As I understand, the second issue is a fix for the first. So I think we should not bring these to 11. Best regards, Goetz. From: Baesken, Matthias Sent: Friday, July 10, 2020 12:46 PM To: Lindenmaier, Goetz ; 'jdk-updates-dev at openjdk.java.net' Subject: Re: 8236464: SO_LINGER option is ignored by SSLSocket in JDK 11 Hello G?tz, the patch itself looks good . However I noticed that in src/java.base/share/classes/sun/security/ssl/SSLSocketImpl.java void closeNotify(boolean useUserCanceled) throws IOException {..} there are some more diffs related to SO_LINGER/linger when comparing jdk/jdk and jdk11 . Have you checked those , do we want them in jdk11 too ? Best regards, Matthias --------------------------------- Hi, I am downporting this for parity with 11.0.9-oracle. I had to do quite some adaptions: http://cr.openjdk.java.net/~goetz/wr20/8236464-SSLSocket_LINGER-jdk11/01/ src/java.base/share/classes/sun/security/ssl/SSLSocketImpl.java The change renames a method that is not there in 11. I skipped the renaming and applied the functional change that is in the same chunk. src/java.base/share/classes/sun/security/ssl/TransportContext.java The code deleted differs, because in 15 explicit locking is used. I added a synchronized block in the method introduced. Please review. https://bugs.openjdk.java.net/browse/JDK-8236464 https://hg.openjdk.java.net/jdk/jdk/rev/c783b60f48db Best regards, Goetz From goetz.lindenmaier at sap.com Mon Jul 13 07:24:02 2020 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Mon, 13 Jul 2020 07:24:02 +0000 Subject: [11u] RFR: 8249159: Downport test rework for SSLSocketTemplate from 8224650 In-Reply-To: References: Message-ID: Hi Matthias, Yes, you are right. The old test contains the following certificates: In array trustedCertStrs SHA256withECDSA, curve prime256v1 SHA256withRSA, 2048 bits SHA256withDSA, 2048 bits In array endEntityCertStrs: SHA256withECDSA, curve prime256v1 SHA256withRSA, 2048 bits SHA256withRSA, curv prime256v1 SHA256withDSA, 2048 bits The new enum lists the following certificates. I marked those added by this change with +: CA_ECDSA_SECP256R1( SHA256withECDSA, curve secp256r1 + CA_ECDSA_SECP384R1( SHA384withECDSA, curve secp384r1 + CA_ECDSA_SECP521R1( SHA512withECDSA, curve secp521r1 CA_RSA_2048( SHA256withRSA, 2048 bits CA_DSA_2048( SHA256withDSA, 2048 bits EE_ECDSA_SECP256R1( SHA256withECDSA, curve secp256r1 + EE_ECDSA_SECP384R1( SHA384withECDSA, curve secp384r1 + EE_ECDSA_SECP521R1( SHA512withECDSA, curve secp521r1 EE_RSA_2048( SHA256withRSA, 2048 bits EE_EC_RSA_SECP256R1( SHA256withRSA, curve secp256r1 EE_DSA_2048( SHA256withDSA, 2048 bits Only the certificates that were in the old test are added to the arrays TRUSTED_CERTS and END_ENTITY_CERTS, so the test still exercises the same certificates. I double checked that the actual certificates equal, too. The others are used by the test which I did not downport. I'd like to keep them, having them does no harm but simplifies downports. What do you think? Best regards, Goetz. From: Baesken, Matthias Sent: Friday, July 10, 2020 1:09 PM To: Lindenmaier, Goetz ; 'jdk-updates-dev at openjdk.java.net' Subject: Re: [11u] RFR: 8249159: Downport test rework for SSLSocketTemplate from 8224650 I would like to bring this test rework from 14 to 11. It is part of https://bugs.openjdk.java.net/browse/JDK-8224650 . 8224650 contains the rework of SSLSocketTemplates.java and adds a new test. In the old version of SSLSocketTemplates.java certificates, names and keys are held in separate arrays that must correlate by indexes correctly. The new version uses enums to keep common information in one place. The new version is much easier to maintain. Also, downports that just add new enums are easier to handle. The new test NamedGroupsWithCipherSuite.java does not pass in 11 because the feature tested (JDK-8171279) is only in 13. I would like to downport the rework because it makes JDK-8246330 apply clean. I do not want to downport the new test as it fails. Therefore I opened an Enhancement issue of it's own for this, https://bugs.openjdk.java.net/browse/JDK-8249159 . Please review this webrev: http://cr.openjdk.java.net/~goetz/wr20/8249159-rework_SSLSocketTemplate-jdk11/01/ It contains the changes of SSLSocketTemplate from JDK-8224650. They applied clean. It does not contain the new test. SSLSocketTemplate.java passes with this test. Best regards, Goetz. --------------------------------------------------------- Hi G?tz , while the patch downport looks mostly fine, I wonder about one point. It looks to me like the certs tested changed a bit with your change . If this is really intentional , please confirm and consider your change as reviewed . Otherwise I wonder about for example this cert, I only find it in the new version ( did not check all the certs but noticed this one ). Best regards, Matthias http://cr.openjdk.java.net/~goetz/wr20/8249159-rework_SSLSocketTemplate-jdk11/01/test/jdk/javax/net/ssl/templates/SSLSocketTemplate.java.frames.html 636 CA_ECDSA_SECP384R1( 637 "EC", 638 // SHA384withECDSA, curve secp384r1 639 // Validity 640 // Not Before: Jun 24 08:15:06 2019 GMT 641 // Not After : Jun 19 08:15:06 2039 GMT 642 // Subject Key Identifier: 643 // 0a:93:a9:a0:bf:e7:d5:48:9d:4f:89:15:c6:51:98:80:05:51:4e:4e 644 "-----BEGIN CERTIFICATE-----\n" + 645 "MIICCDCCAY6gAwIBAgIUCpOpoL/n1UidT4kVxlGYgAVRTk4wCgYIKoZIzj0EAwMw\n" + 646 "OzELMAkGA1UEBhMCVVMxDTALBgNVBAoMBEphdmExHTAbBgNVBAsMFFN1bkpTU0Ug\n" + 647 "VGVzdCBTZXJpdmNlMB4XDTE5MDYyNDA4MTUwNloXDTM5MDYxOTA4MTUwNlowOzEL\n" + 648 "MAkGA1UEBhMCVVMxDTALBgNVBAoMBEphdmExHTAbBgNVBAsMFFN1bkpTU0UgVGVz\n" + 649 "dCBTZXJpdmNlMHYwEAYHKoZIzj0CAQYFK4EEACIDYgAENVQN1wXWFdgC6u/dDdiC\n" + 650 "y+WtMTF66oL/0BSm+1ZqsogamzCryawOcHgiuXgWzx5CQ3LuOC+tDFyXpGfHuCvb\n" + 651 "dkzxPrP5n9NrR8/uRPe5l1KOUbchviU8z9cTP+LZxnZDo1MwUTAdBgNVHQ4EFgQU\n" + 652 "SktSFArR1p/5mXV0kyo0RxIVa/UwHwYDVR0jBBgwFoAUSktSFArR1p/5mXV0kyo0\n" + 653 "RxIVa/UwDwYDVR0TAQH/BAUwAwEB/zAKBggqhkjOPQQDAwNoADBlAjBZvoNmq3/v\n" + 654 "RD2gBTyvxjS9h0rsMRLHDnvul/KWngytwGPTOBo0Y8ixQXSjdKoc3rkCMQDkiNgx\n" + 655 "IDxuHedmrLQKIPnVcthTmwv7//jHiqGoKofwChMo2a1P+DQdhszmeHD/ARQ=\n" + 656 "-----END CERTIFICATE-----", From rkennke at redhat.com Mon Jul 13 16:06:12 2020 From: rkennke at redhat.com (Roman Kennke) Date: Mon, 13 Jul 2020 18:06:12 +0200 Subject: RFR (11u, XXL): Upstream/backport Shenandoah to JDK11u - the review thread In-Reply-To: <3db8e235f3ab515563802d23997f0ac32d699c48.camel@redhat.com> References: <3db8e235f3ab515563802d23997f0ac32d699c48.camel@redhat.com> Message-ID: Hi all, So I did the exercise that we discussed in the other thread, and wrote a little script that compares object files of the baseline build with the object files of a patched builds with Shenandoah disabled. (I will clean up and post the script for this shortly.) In order for this to make any sense, I needed to patch debug.hpp to suppress inclusion of __LINE__ in a bunch of macros. I will clean it up and turn this into a proper configure flag and propose it for inclusion soon. I think it might be useful more generally (perhaps for JDK16 and then trickle down to jdk11u). This exercise indeed turned up a couple of (harmless) problems where Shenandoah code did indeed leak into non-Shenandoah builds: - We had a couple of unguarded UseShenandoahGC clauses in C2 code. - We generated ShenandoahBarrierNode declarations and node-query in node.hpp - ADLC build did not pass -DINCLUDE_SHENANDOAHGC=0 when Shenandoah is disabled, therefore it would pick up Shenandoah node declarations from classes.hpp, which in turn would leak into a couple of places like matcher.o, compile.o, dfa.o and a few others. - vmOperations.hpp would generate VM_Shenandoah* declarations which would get compiled into vmOperations.cpp - JFR generated a couple of Shenandoah related events and types, and this requires the definition of some empty methods too. I fixed the generator so that it can spread the JFR metadata.xml over several files, and include GC specific metadata only on-demand. I will extract this part and propose it for inclusion upstream in jdk16. With all those issues fixed, I can (and did) prove that Shenandoah does not leak anywhere in libjvm.so when Shenandoah is disabled. It does literally compile identical object files. The only exception is the inclusion of the global flag UseShenandoah and the code that prints an error and exists when trying to run with it. (If really needed, I could actually rip it out too, but it doesn't seem worth, nor very clean). Shenandoah is in-fact cleaner in this respect than any other GC in OpenJDK. :-) There are two places left where Shenandoah has potential impact to shared code: - Serviceability-agent: we needed to patch some Java code in order to provide support for Shenandoah in SA. As far as I can tell, it is dead code when running in a Shenandoah-disabled build. (In-fact, it should actually work to use the resulting JAR with a build that has Shenandoah enabled ;-). - Tests: a couple of tests have been amended with Shenandoah configurations. All those new configurations are disabled and will not be run in a build with Shenandoah disabled. This is also relatively easy to prove by actually running the affected tests with Shenandoah- disabled builds. Shared-code changes only: http://cr.openjdk.java.net/~rkennke/shenandoah-jdk11u-upstream/webrev.09-shared/ Full webrev: http://cr.openjdk.java.net/~rkennke/shenandoah-jdk11u-upstream/webrev.09-all/ What do you think? Roman On Thu, 2020-07-09 at 19:36 +0200, Roman Kennke wrote: > I'm starting a new thread with a new round of webrevs. Maybe we can > keep in-principle discussions on the other thread and get some actual > reviews here? > > Anyhow, small changes since the last one: > - the lone include "interpreter.hpp" is gone, apparently it was a > left- > over that's no longer needed > - the 2 friend class declarations are now wrapped in > INCLUDE_SHENANDOAHGC too, just to be sure (even though compiler > wouldn't compile any actual code) > > As noted in the other thread, there should not be any Shenandoah code > leaking out when building with --with-jvm-features=-shenandoahgc > (which > is the default too). There's 2 exceptions: > > - The global UseShenandoahGC flag. When invoked but not built-in, it > will print an error message and exit. > - In: > src/hotspot/share/jfr/periodic/jfrPeriodic.cpp > > I could not exclude the definition of: > TRACE_REQUEST_FUNC(ShenandoahHeapRegionInformation) > > Because that would be called by generated code. Related to this: > > src/hotspot/share/jfr/metadata/metadata.xml > > This generates two event classes for Shenandaoh, but they would not > be > used. I don't know how to exclude them. It seems harmless to me. > > Shared-only changes: > http://cr.openjdk.java.net/~rkennke/shenandoah-jdk11u-upstream/webrev.07-shared/ > > Full webrev: > http://cr.openjdk.java.net/~rkennke/shenandoah-jdk11u-upstream/webrev.07-all/ > > > Roman > From aleksei.voitylov at bell-sw.com Mon Jul 13 16:17:48 2020 From: aleksei.voitylov at bell-sw.com (Aleksei Voitylov) Date: Mon, 13 Jul 2020 19:17:48 +0300 Subject: RFR (11u, XXL): Upstream/backport Shenandoah to JDK11u - the review thread In-Reply-To: References: <3db8e235f3ab515563802d23997f0ac32d699c48.camel@redhat.com> Message-ID: <94e7c83f-97fd-4e50-1d42-43ad79bea82b@bell-sw.com> Roman, so it turned out to be a valuable exercise, indeed! Looking forward for the script to be able to replicate the check here. -Aleksei On 13/07/2020 19:06, Roman Kennke wrote: > Hi all, > > So I did the exercise that we discussed in the other thread, and wrote > a little script that compares object files of the baseline build with > the object files of a patched builds with Shenandoah disabled. (I will > clean up and post the script for this shortly.) > > In order for this to make any sense, I needed to patch debug.hpp to > suppress inclusion of __LINE__ in a bunch of macros. I will clean it up > and turn this into a proper configure flag and propose it for inclusion > soon. I think it might be useful more generally (perhaps for JDK16 and > then trickle down to jdk11u). > > This exercise indeed turned up a couple of (harmless) problems where > Shenandoah code did indeed leak into non-Shenandoah builds: > - We had a couple of unguarded UseShenandoahGC clauses in C2 code. > - We generated ShenandoahBarrierNode declarations and node-query in > node.hpp > - ADLC build did not pass -DINCLUDE_SHENANDOAHGC=0 when Shenandoah is > disabled, therefore it would pick up Shenandoah node declarations from > classes.hpp, which in turn would leak into a couple of places like > matcher.o, compile.o, dfa.o and a few others. > - vmOperations.hpp would generate VM_Shenandoah* declarations which > would get compiled into vmOperations.cpp > - JFR generated a couple of Shenandoah related events and types, and > this requires the definition of some empty methods too. I fixed the > generator so that it can spread the JFR metadata.xml over several > files, and include GC specific metadata only on-demand. I will extract > this part and propose it for inclusion upstream in jdk16. > > With all those issues fixed, I can (and did) prove that Shenandoah does > not leak anywhere in libjvm.so when Shenandoah is disabled. It does > literally compile identical object files. The only exception is the > inclusion of the global flag UseShenandoah and the code that prints an > error and exists when trying to run with it. (If really needed, I could > actually rip it out too, but it doesn't seem worth, nor very clean). > > Shenandoah is in-fact cleaner in this respect than any other GC in > OpenJDK. :-) > > There are two places left where Shenandoah has potential impact to > shared code: > - Serviceability-agent: we needed to patch some Java code in order to > provide support for Shenandoah in SA. As far as I can tell, it is dead > code when running in a Shenandoah-disabled build. (In-fact, it should > actually work to use the resulting JAR with a build that has Shenandoah > enabled ;-). > - Tests: a couple of tests have been amended with Shenandoah > configurations. All those new configurations are disabled and will not > be run in a build with Shenandoah disabled. This is also relatively > easy to prove by actually running the affected tests with Shenandoah- > disabled builds. > > Shared-code changes only: > http://cr.openjdk.java.net/~rkennke/shenandoah-jdk11u-upstream/webrev.09-shared/ > > Full webrev: > http://cr.openjdk.java.net/~rkennke/shenandoah-jdk11u-upstream/webrev.09-all/ > > What do you think? > > Roman > > > On Thu, 2020-07-09 at 19:36 +0200, Roman Kennke wrote: >> I'm starting a new thread with a new round of webrevs. Maybe we can >> keep in-principle discussions on the other thread and get some actual >> reviews here? >> >> Anyhow, small changes since the last one: >> - the lone include "interpreter.hpp" is gone, apparently it was a >> left- >> over that's no longer needed >> - the 2 friend class declarations are now wrapped in >> INCLUDE_SHENANDOAHGC too, just to be sure (even though compiler >> wouldn't compile any actual code) >> >> As noted in the other thread, there should not be any Shenandoah code >> leaking out when building with --with-jvm-features=-shenandoahgc >> (which >> is the default too). There's 2 exceptions: >> >> - The global UseShenandoahGC flag. When invoked but not built-in, it >> will print an error message and exit. >> - In: >> src/hotspot/share/jfr/periodic/jfrPeriodic.cpp >> >> I could not exclude the definition of: >> TRACE_REQUEST_FUNC(ShenandoahHeapRegionInformation) >> >> Because that would be called by generated code. Related to this: >> >> src/hotspot/share/jfr/metadata/metadata.xml >> >> This generates two event classes for Shenandaoh, but they would not >> be >> used. I don't know how to exclude them. It seems harmless to me. >> >> Shared-only changes: >> http://cr.openjdk.java.net/~rkennke/shenandoah-jdk11u-upstream/webrev.07-shared/ >> >> Full webrev: >> http://cr.openjdk.java.net/~rkennke/shenandoah-jdk11u-upstream/webrev.07-all/ >> >> >> Roman >> From goetz.lindenmaier at sap.com Mon Jul 13 17:17:17 2020 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Mon, 13 Jul 2020 17:17:17 +0000 Subject: [11u] RFR: 8240169: javadoc fails to link to non-modular api docs In-Reply-To: References: Message-ID: Hi Matthias, Thanks for the review! Best regards, Goetz. From: Baesken, Matthias Sent: Friday, July 10, 2020 1:33 PM To: 'jdk-updates-dev at openjdk.java.net' Cc: Lindenmaier, Goetz Subject: RE: [11u] RFR: 8240169: javadoc fails to link to non-modular api docs Hello G?tz , the backport looks good to me . Best regards, Matthias --------------------- Hi I am downporting this for parity with 11.0.9-oracle. http://cr.openjdk.java.net/~goetz/wr20/8240169-javadoc-jdk11/01/ I had to do a row of adaptions to this change. src/jdk.javadoc/share/classes/jdk/javadoc/internal/doclets/toolkit/util/Extern.java The code differs slightly because in 11, configuration.utils is dereferenced everywhere, but in 15 utils is saved in a field of the object. Further down, put() is used in 11 where putIfAbsent() is used in 15. Further, code is formatted differently. Another chunk uses resources.getText, where in 11 it is configuration.getText. In checkLinkCompatibility() I added {} so it looks similar to 15. Besides resolving, I had to add "import javax.tools.Diagnostic.Kind;" JavadocTester.java 15 prints the faulty html file if the test fails. This was introduced in "8210031: implementation for JVM Constants API" which is huge. I would like to include this printing in this change, as it helps a lot to understand the issue if the test fails. It helped me to find out where I had resolved wrong. test/langtools/jdk/javadoc/doclet/testClassCrossReferences/TestClassCrossReferences.java The method is marked public in 15, therefore the patch does not apply. Further, I had to adapt the templates matched. In 11, "?is-external=true" is printed to the urls and "externalLink" is "external-link" in 15. Also, the "Overrides" heading is formatted differently. TestLinkOptionWithModule.java This needed similar adaptions of patterns as TestClassCrossReferences.java. Please review. https://bugs.openjdk.java.net/browse/JDK-8240169 https://hg.openjdk.java.net/jdk/jdk/rev/3b557aef43c4 Best regards, Goetz. From rkennke at redhat.com Mon Jul 13 21:38:12 2020 From: rkennke at redhat.com (Roman Kennke) Date: Mon, 13 Jul 2020 23:38:12 +0200 Subject: RFR (11u, XXL): Upstream/backport Shenandoah to JDK11u - the review thread In-Reply-To: References: <3db8e235f3ab515563802d23997f0ac32d699c48.camel@redhat.com> Message-ID: And here comes the patch & script to verify the builds: 1. We need a patch to disable line numbers: http://cr.openjdk.java.net/~rkennke/disable-linenums.patch I opted to not change any build machinery and instead simply disable line numbers by configuring with --with-extra-cflags=- DDISABLE_LINE_NUMBERS This patch needs to be applied *both* for the baseline and patched build that the below script compiles. 2. The UseShenandoahGC flag should be included in both baseline and patched build, for the sake of this comparison, otherwise it will generate a bunch of extern symbols all over the place: http://cr.openjdk.java.net/~rkennke/shenandoahflag.patch 3. The actual big Shenandoah patch: http://cr.openjdk.java.net/~rkennke/shenandoah.patch (The webrev.09-all combines both Shenandoah patches.) 4. The script that builds with and without the Shenandoah patch and compares the resulting object files: http://cr.openjdk.java.net/~rkennke/compare-builds.sh Place this in the root of the jdk11u source directory. I am using Mercurial patch queues to handle the patches. hg qimport the patches above: hg qimport http://cr.openjdk.java.net/~rkennke/disable-linenums.patch hg qpush hg qimport http://cr.openjdk.java.net/~rkennke/shenandoahflag.patch hg qpush hg qimport http://cr.openjdk.java.net/~rkennke/shenandoah.patch Don't apply the big patch just yet. The compare script will do that for you. Invoke the script: ./compare-builds.sh It will make a clean build into build/baseline (including disable- linenums.patch and shenandoahflag.patch), and a patched build into build/patched (including all 3 patches), and compare the resulting object files. You should see that it complains about vm_version.o, that is because it captures the time-stamp of the build (it can be verified by objdump -s both objects and diffing the outputs). The remaining failure in gcConfig is the code that prints out the error and exists when invoked with -XX:+UseShenandoahGC (it could perhaps be extracted into shenandoahflag.patch with some more work). (For the record, I am doing this experiment with gcc 8.3.1. YMMV with other compilers) If you are trying this too, let me know how it goes. Roman On Mon, 2020-07-13 at 18:06 +0200, Roman Kennke wrote: > Hi all, > > So I did the exercise that we discussed in the other thread, and > wrote > a little script that compares object files of the baseline build with > the object files of a patched builds with Shenandoah disabled. (I > will > clean up and post the script for this shortly.) > > In order for this to make any sense, I needed to patch debug.hpp to > suppress inclusion of __LINE__ in a bunch of macros. I will clean it > up > and turn this into a proper configure flag and propose it for > inclusion > soon. I think it might be useful more generally (perhaps for JDK16 > and > then trickle down to jdk11u). > > This exercise indeed turned up a couple of (harmless) problems where > Shenandoah code did indeed leak into non-Shenandoah builds: > - We had a couple of unguarded UseShenandoahGC clauses in C2 code. > - We generated ShenandoahBarrierNode declarations and node-query in > node.hpp > - ADLC build did not pass -DINCLUDE_SHENANDOAHGC=0 when Shenandoah is > disabled, therefore it would pick up Shenandoah node declarations > from > classes.hpp, which in turn would leak into a couple of places like > matcher.o, compile.o, dfa.o and a few others. > - vmOperations.hpp would generate VM_Shenandoah* declarations which > would get compiled into vmOperations.cpp > - JFR generated a couple of Shenandoah related events and types, and > this requires the definition of some empty methods too. I fixed the > generator so that it can spread the JFR metadata.xml over several > files, and include GC specific metadata only on-demand. I will > extract > this part and propose it for inclusion upstream in jdk16. > > With all those issues fixed, I can (and did) prove that Shenandoah > does > not leak anywhere in libjvm.so when Shenandoah is disabled. It does > literally compile identical object files. The only exception is the > inclusion of the global flag UseShenandoah and the code that prints > an > error and exists when trying to run with it. (If really needed, I > could > actually rip it out too, but it doesn't seem worth, nor very clean). > > Shenandoah is in-fact cleaner in this respect than any other GC in > OpenJDK. :-) > > There are two places left where Shenandoah has potential impact to > shared code: > - Serviceability-agent: we needed to patch some Java code in order to > provide support for Shenandoah in SA. As far as I can tell, it is > dead > code when running in a Shenandoah-disabled build. (In-fact, it should > actually work to use the resulting JAR with a build that has > Shenandoah > enabled ;-). > - Tests: a couple of tests have been amended with Shenandoah > configurations. All those new configurations are disabled and will > not > be run in a build with Shenandoah disabled. This is also relatively > easy to prove by actually running the affected tests with Shenandoah- > disabled builds. > > Shared-code changes only: > http://cr.openjdk.java.net/~rkennke/shenandoah-jdk11u-upstream/webrev.09-shared/ > > Full webrev: > http://cr.openjdk.java.net/~rkennke/shenandoah-jdk11u-upstream/webrev.09-all/ > > What do you think? > > Roman > > > On Thu, 2020-07-09 at 19:36 +0200, Roman Kennke wrote: > > I'm starting a new thread with a new round of webrevs. Maybe we can > > keep in-principle discussions on the other thread and get some > > actual > > reviews here? > > > > Anyhow, small changes since the last one: > > - the lone include "interpreter.hpp" is gone, apparently it was a > > left- > > over that's no longer needed > > - the 2 friend class declarations are now wrapped in > > INCLUDE_SHENANDOAHGC too, just to be sure (even though compiler > > wouldn't compile any actual code) > > > > As noted in the other thread, there should not be any Shenandoah > > code > > leaking out when building with --with-jvm-features=-shenandoahgc > > (which > > is the default too). There's 2 exceptions: > > > > - The global UseShenandoahGC flag. When invoked but not built-in, > > it > > will print an error message and exit. > > - In: > > src/hotspot/share/jfr/periodic/jfrPeriodic.cpp > > > > I could not exclude the definition of: > > TRACE_REQUEST_FUNC(ShenandoahHeapRegionInformation) > > > > Because that would be called by generated code. Related to this: > > > > src/hotspot/share/jfr/metadata/metadata.xml > > > > This generates two event classes for Shenandaoh, but they would not > > be > > used. I don't know how to exclude them. It seems harmless to me. > > > > Shared-only changes: > > http://cr.openjdk.java.net/~rkennke/shenandoah-jdk11u-upstream/webrev.07-shared/ > > > > Full webrev: > > http://cr.openjdk.java.net/~rkennke/shenandoah-jdk11u-upstream/webrev.07-all/ > > > > > > Roman > > From matthias.baesken at sap.com Tue Jul 14 07:05:06 2020 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Tue, 14 Jul 2020 07:05:06 +0000 Subject: 8236464: SO_LINGER option is ignored by SSLSocket in JDK 11 In-Reply-To: References: Message-ID: > So I think we should not bring these to 11. Ok fine with me . Thanks for checking ! Best regards, Matthias From: Lindenmaier, Goetz Sent: Montag, 13. Juli 2020 08:22 To: Baesken, Matthias ; 'jdk-updates-dev at openjdk.java.net' Subject: RE: Re: 8236464: SO_LINGER option is ignored by SSLSocket in JDK 11 Hi Matthias, Thanks for your review. The further changes in this function stem from: 8221882: Use fiber-friendly java.util.concurrent.locks in JSSE 8224829: AsyncSSLSocketClose.java has timing issue Obviously, we don't need fiber preparation in 11. As I understand, the second issue is a fix for the first. So I think we should not bring these to 11. Best regards, Goetz. From: Baesken, Matthias > Sent: Friday, July 10, 2020 12:46 PM To: Lindenmaier, Goetz >; 'jdk-updates-dev at openjdk.java.net' > Subject: Re: 8236464: SO_LINGER option is ignored by SSLSocket in JDK 11 Hello G?tz, the patch itself looks good . However I noticed that in src/java.base/share/classes/sun/security/ssl/SSLSocketImpl.java void closeNotify(boolean useUserCanceled) throws IOException {..} there are some more diffs related to SO_LINGER/linger when comparing jdk/jdk and jdk11 . Have you checked those , do we want them in jdk11 too ? Best regards, Matthias --------------------------------- Hi, I am downporting this for parity with 11.0.9-oracle. I had to do quite some adaptions: http://cr.openjdk.java.net/~goetz/wr20/8236464-SSLSocket_LINGER-jdk11/01/ src/java.base/share/classes/sun/security/ssl/SSLSocketImpl.java The change renames a method that is not there in 11. I skipped the renaming and applied the functional change that is in the same chunk. src/java.base/share/classes/sun/security/ssl/TransportContext.java The code deleted differs, because in 15 explicit locking is used. I added a synchronized block in the method introduced. Please review. https://bugs.openjdk.java.net/browse/JDK-8236464 https://hg.openjdk.java.net/jdk/jdk/rev/c783b60f48db Best regards, Goetz From goetz.lindenmaier at sap.com Tue Jul 14 07:12:43 2020 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Tue, 14 Jul 2020 07:12:43 +0000 Subject: 8236464: SO_LINGER option is ignored by SSLSocket in JDK 11 In-Reply-To: References: Message-ID: Thanks for reviewing! Best regards, Goetz. From: Baesken, Matthias Sent: Tuesday, July 14, 2020 9:05 AM To: Lindenmaier, Goetz ; 'jdk-updates-dev at openjdk.java.net' Subject: RE: Re: 8236464: SO_LINGER option is ignored by SSLSocket in JDK 11 > So I think we should not bring these to 11. Ok fine with me . Thanks for checking ! Best regards, Matthias From: Lindenmaier, Goetz > Sent: Montag, 13. Juli 2020 08:22 To: Baesken, Matthias >; 'jdk-updates-dev at openjdk.java.net' > Subject: RE: Re: 8236464: SO_LINGER option is ignored by SSLSocket in JDK 11 Hi Matthias, Thanks for your review. The further changes in this function stem from: 8221882: Use fiber-friendly java.util.concurrent.locks in JSSE 8224829: AsyncSSLSocketClose.java has timing issue Obviously, we don't need fiber preparation in 11. As I understand, the second issue is a fix for the first. So I think we should not bring these to 11. Best regards, Goetz. From: Baesken, Matthias > Sent: Friday, July 10, 2020 12:46 PM To: Lindenmaier, Goetz >; 'jdk-updates-dev at openjdk.java.net' > Subject: Re: 8236464: SO_LINGER option is ignored by SSLSocket in JDK 11 Hello G?tz, the patch itself looks good . However I noticed that in src/java.base/share/classes/sun/security/ssl/SSLSocketImpl.java void closeNotify(boolean useUserCanceled) throws IOException {..} there are some more diffs related to SO_LINGER/linger when comparing jdk/jdk and jdk11 . Have you checked those , do we want them in jdk11 too ? Best regards, Matthias --------------------------------- Hi, I am downporting this for parity with 11.0.9-oracle. I had to do quite some adaptions: http://cr.openjdk.java.net/~goetz/wr20/8236464-SSLSocket_LINGER-jdk11/01/ src/java.base/share/classes/sun/security/ssl/SSLSocketImpl.java The change renames a method that is not there in 11. I skipped the renaming and applied the functional change that is in the same chunk. src/java.base/share/classes/sun/security/ssl/TransportContext.java The code deleted differs, because in 15 explicit locking is used. I added a synchronized block in the method introduced. Please review. https://bugs.openjdk.java.net/browse/JDK-8236464 https://hg.openjdk.java.net/jdk/jdk/rev/c783b60f48db Best regards, Goetz From goetz.lindenmaier at sap.com Tue Jul 14 08:21:53 2020 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Tue, 14 Jul 2020 08:21:53 +0000 Subject: [11u] RFR: 8249277: TestVerifyIterativeGVN.java is failing with timeout in OpenJDK 11 Message-ID: Hi This test times out on mac. I would like to increase the timeout. Please review: http://cr.openjdk.java.net/~goetz/wr20/8249277-TestVerifyIterativeGVN_timeout-jdk11/01/ Oracle did a similar change in the closed repo: https://bugs.openjdk.java.net/browse/JDK-8248521 but the change is not visible, so I crafted my own. New bug, open 11u only: https://bugs.openjdk.java.net/browse/JDK-8248521 Best regards, Goetz. From abrygin at azul.com Tue Jul 14 16:41:11 2020 From: abrygin at azul.com (Andrew Brygin) Date: Tue, 14 Jul 2020 19:41:11 +0300 Subject: [13u] RFR: jdk-13.0.4+7 & jdk-13.0.4+8 / jdk-13.0.4-ga Message-ID: <037ec2dc-7486-ecdc-4a61-5b0ac2536af4@azul.com> Hello, here are remaining changes for 13.0.4: Webrev: http://cr.openjdk.java.net/~bae/13u/13.0.4-webrev Changes in jdk-13.0.4+7: JDK-8230613: Better ASCII conversions JDK-8231800: Better listing of arrays JDK-8232014: Expand DTD support JDK-8233234: Better Zip Naming JDK-8233239: Enhance TIFF support JDK-8233255: Better Swing Buttons JDK-8234032: Improve basic calendar services JDK-8234042: Better factory production of certificates JDK-8234418: Better parsing with CertificateFactory JDK-8234836: Improve serialization handling JDK-8236191: Enhance OID processing JDK-8236867: Enhance Graal interface handling JDK-8237117: Better ForkJoinPool behavior JDK-8237592: Enhance certificate verification JDK-8238002: Better matrix operations JDK-8238013: Enhance String writing JDK-8238804: Enhance key handling process JDK-8238843: Enhanced font handing JDK-8238920: Better Buffer support JDK-8238925: Enhance WAV file playback JDK-8240119: Less Affine Transformations JDK-8240482: Improved WAV file playback JDK-8241379: Update JCEKS support JDK-8241522: Manifest improved jar headers redux JDK-8242136: Better XML namespace handling Changes in jdk-13.0.4+8: JDK-8248505: Unexpected NoSuchAlgorithmException when using secure random impl from BCFIPS provider The result is tagged as jdk-13.0.4-ga. This set of changes has been successfully built on Linux (i686, x86_64), Windows (i686, x86_64), and macOS (x86_64) platforms. Please take a look. Thanks, Andrew From gnu.andrew at redhat.com Tue Jul 14 17:00:42 2020 From: gnu.andrew at redhat.com (Andrew Hughes) Date: Tue, 14 Jul 2020 18:00:42 +0100 Subject: [RFR] [11u] 11.0.8+9 / 11.0.8+10 / 11.0.8-ga Message-ID: <5ae74928-4f66-fbcd-146d-c75c2a07b004@redhat.com> Here are the remaining changes for the jdk11u repository: Webrev: https://cr.openjdk.java.net/~andrew/openjdk11/11.0.8/ There are two new tags (plus jdk-11.0.8+10 is tagged as jdk-11.0.8-ga). JDK-8248505 was identified at the last minute and added for the jdk-11.0.8+10 tag (clean backport). Changes in jdk-11.0.8+9: - JDK-8230613: Better ASCII conversions - JDK-8231800: Better listing of arrays - JDK-8232014: Expand DTD support - JDK-8233234: Better Zip Naming - JDK-8233239: Enhance TIFF support - JDK-8233255: Better Swing Buttons - JDK-8234032: Improve basic calendar services - JDK-8234042: Better factory production of certificates - JDK-8234418: Better parsing with CertificateFactory - JDK-8234836: Improve serialization handling - JDK-8236191: Enhance OID processing - JDK-8236867: Enhance Graal interface handling - JDK-8237117: Better ForkJoinPool behavior - JDK-8237592: Enhance certificate verification - JDK-8238002: Better matrix operations - JDK-8238013: Enhance String writing - JDK-8238804: Enhance key handling process - JDK-8238843: Enhanced font handing - JDK-8238920, CVE-2020-14583: Better Buffer support - JDK-8238925: Enhance WAV file playback - JDK-8240119: Less Affine Transformations - JDK-8240482: Improved WAV file playback - JDK-8241379: Update JCEKS support - JDK-8241522: Manifest improved jar headers redux - JDK-8242136: Better XML namespace handling Changes in jdk-11.0.8+10: - JDK-8248505: Unexpected NoSuchAlgorithmException when using secure random impl from BCFIPS provider Successfully built on x86, x86_64, s390, s390x, ppc, ppc64, ppc64le & aarch64. Ok to push? Thanks, -- Andrew :) Senior Free Java Software Engineer Red Hat, Inc. (http://www.redhat.com) PGP Key: ed25519/0xCFDA0F9B35964222 (hkp://keys.gnupg.net) Fingerprint = 5132 579D D154 0ED2 3E04 C5A0 CFDA 0F9B 3596 4222 From shade at redhat.com Tue Jul 14 17:05:45 2020 From: shade at redhat.com (Aleksey Shipilev) Date: Tue, 14 Jul 2020 19:05:45 +0200 Subject: [RFR] [11u] 11.0.8+9 / 11.0.8+10 / 11.0.8-ga In-Reply-To: <5ae74928-4f66-fbcd-146d-c75c2a07b004@redhat.com> References: <5ae74928-4f66-fbcd-146d-c75c2a07b004@redhat.com> Message-ID: <18970f88-c2c7-1fe0-4c97-0a937d646657@redhat.com> On 7/14/20 7:00 PM, Andrew Hughes wrote: > Here are the remaining changes for the jdk11u repository: > > Webrev: https://cr.openjdk.java.net/~andrew/openjdk11/11.0.8/ > > There are two new tags (plus jdk-11.0.8+10 is tagged as jdk-11.0.8-ga). > JDK-8248505 was identified at the last minute and added for the > jdk-11.0.8+10 tag (clean backport). Looks fine to me. > Ok to push? Yes. -- Thanks, -Aleksey From yan at azul.com Tue Jul 14 17:28:42 2020 From: yan at azul.com (Yuri Nesterenko) Date: Tue, 14 Jul 2020 20:28:42 +0300 Subject: [13u] RFR: jdk-13.0.4+7 & jdk-13.0.4+8 / jdk-13.0.4-ga In-Reply-To: <037ec2dc-7486-ecdc-4a61-5b0ac2536af4@azul.com> References: <037ec2dc-7486-ecdc-4a61-5b0ac2536af4@azul.com> Message-ID: <0be9a07b-f4d6-260d-10e5-538a60af71cc@azul.com> Hi Andrew, I see nothing unexpected. Push it! --yan On 14.07.2020 19:41, Andrew Brygin wrote: > Hello, > > here are remaining changes for 13.0.4: > > Webrev: http://cr.openjdk.java.net/~bae/13u/13.0.4-webrev > > Changes in jdk-13.0.4+7: > JDK-8230613: Better ASCII conversions > JDK-8231800: Better listing of arrays > JDK-8232014: Expand DTD support > JDK-8233234: Better Zip Naming > JDK-8233239: Enhance TIFF support > JDK-8233255: Better Swing Buttons > JDK-8234032: Improve basic calendar services > JDK-8234042: Better factory production of certificates > JDK-8234418: Better parsing with CertificateFactory > JDK-8234836: Improve serialization handling > JDK-8236191: Enhance OID processing > JDK-8236867: Enhance Graal interface handling > JDK-8237117: Better ForkJoinPool behavior > JDK-8237592: Enhance certificate verification > JDK-8238002: Better matrix operations > JDK-8238013: Enhance String writing > JDK-8238804: Enhance key handling process > JDK-8238843: Enhanced font handing > JDK-8238920: Better Buffer support > JDK-8238925: Enhance WAV file playback > JDK-8240119: Less Affine Transformations > JDK-8240482: Improved WAV file playback > JDK-8241379: Update JCEKS support > JDK-8241522: Manifest improved jar headers redux > JDK-8242136: Better XML namespace handling > > Changes in jdk-13.0.4+8: > JDK-8248505: Unexpected NoSuchAlgorithmException when using secure > random impl from BCFIPS provider > > The result is tagged as jdk-13.0.4-ga. > > This set of changes has been successfully built on Linux (i686, > x86_64), Windows (i686, x86_64), and macOS (x86_64) platforms. > > Please take a look. > > Thanks, > Andrew > From rob.mckenna at oracle.com Tue Jul 14 18:56:54 2020 From: rob.mckenna at oracle.com (Rob McKenna) Date: Tue, 14 Jul 2020 19:56:54 +0100 Subject: [Updates Communication] 14.0.2 security patches pushed In-Reply-To: <20200415144830.GA2989@DESKTOP-JNNKIHU.localdomain> References: <20200415144830.GA2989@DESKTOP-JNNKIHU.localdomain> Message-ID: <20200714185654.GA1826@DESKTOP-JNNKIHU.localdomain> 14.0.2 security patches pushed to http://hg.openjdk.java.net/jdk-updates/jdk14u -Rob From sgehwolf at redhat.com Wed Jul 15 09:39:26 2020 From: sgehwolf at redhat.com (Severin Gehwolf) Date: Wed, 15 Jul 2020 11:39:26 +0200 Subject: [11u] RFR: 8226575: OperatingSystemMXBean should be made container aware Message-ID: <6f644db4cf5ebf8663b87eb7ba9496e581dddd0f.camel@redhat.com> Hi, Could I please get a review of this container awareness backport for OperatingSystemMXBean? I'd like to backport this since 8u will soon receive a similar update and this would avoid regressions when upgrading from JDK 8 to JDK 11 in the future. The JDK 14 patch doesn't exactly apply since for an 11u backport it's not appropriate to add new methods in OperatingSystemMXBean. Thus, for this backport the old methods now return container aware values on an OS which supports them (i.e. Linux only). I'm going to propse a test fix, JDK-8236617, as a follow-up. webrev: http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8226575/jdk11/02/webrev/ Bug: https://bugs.openjdk.java.net/browse/JDK-8226575 Corresponding CSR: https://bugs.openjdk.java.net/browse/JDK-8248804 (approved) Testing: tier 1, :jdk_management, docker tests on Linux x86_64. All pass. Thoughts? Thanks, Severin From sgehwolf at redhat.com Wed Jul 15 10:03:23 2020 From: sgehwolf at redhat.com (Severin Gehwolf) Date: Wed, 15 Jul 2020 12:03:23 +0200 Subject: [11u] RFR: 8236617: jtreg test containers/docker/TestMemoryAwareness.java fails after 8226575 Message-ID: <2db3f0712f66753d07ef764a5dff1a5d411b1f7b.camel@redhat.com> Hi, Please review this follow-up fix for JDK-8226575 (Container aware OperatingSystemMXBean) which is pending review for a 11u backport. It fixes a test issue seen on SAP's test cluster. The JDK 15 patch didn't apply cleanly since the backport doesn't add new method names and CheckOperatingSystemMXBean diagnostics don't print them. I've adjusted it manually. Bug: https://bugs.openjdk.java.net/browse/JDK-8236617 webrev: http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8236617/jdk11/01/webrev/ original change: https://hg.openjdk.java.net/jdk/jdk/rev/ab10165b4141 Testing: Docker tests on Linux x86_64. Passed. Once reviewed and approved I intend to push JDK-8226575 and JDK-8236617 together. Thanks, Severin From yan at azul.com Wed Jul 15 15:14:09 2020 From: yan at azul.com (Yuri Nesterenko) Date: Wed, 15 Jul 2020 18:14:09 +0300 Subject: OpenJDK 13.0.4 released Message-ID: <33ea4b7a-6bc5-6350-0ea0-ac737f1aa48f@azul.com> Let me announce the release of OpenJDK 13.0.4. The release sources are in the usual place, get them to build from https://hg.openjdk.java.net/jdk-updates/jdk13u Read-only git mirror is also available on GitHub thanks to the Skara team: https://github.com/openjdk/jdk13u A list of fixes please see below. Also, a nicely generated list could be found, again, at A. Shipilev's site: https://builds.shipilev.net/backports-monitor/release-notes-13.0.4.txt ============================================ Security fixes: JDK-8230613: Better ASCII conversions JDK-8231800: Better listing of arrays JDK-8232014: Expand DTD support JDK-8233234: Better Zip Naming JDK-8233239: Enhance TIFF support JDK-8233255: Better Swing Buttons JDK-8234032: Improve basic calendar services JDK-8234042: Better factory production of certificates JDK-8234418: Better parsing with CertificateFactory JDK-8234836: Improve serialization handling JDK-8236191: Enhance OID processing JDK-8236867: Enhance Graal interface handling JDK-8237117: Better ForkJoinPool behavior JDK-8237592: Enhance certificate verification JDK-8238002: Better matrix operations JDK-8238013: Enhance String writing JDK-8238804: Enhance key handling process JDK-8238843: Enhanced font handing JDK-8238920: Better Buffer support JDK-8238925: Enhance WAV file playback JDK-8240119: Less Affine Transformations JDK-8240482: Improved WAV file playback JDK-8241379: Update JCEKS support JDK-8241522: Manifest improved jar headers redux JDK-8242136: Better XML namespace handling Other fixes: JDK-8248505: Unexpected NoSuchAlgorithmException when using secure random impl from BCFIPS provider JDK-8248406: Some zipfs tests fail with AccessControlException JDK-8242141: New System Properties to configure the TLS signature schemes JDK-8229888: (zipfs) Updating an existing zip file does not preserve original permissions JDK-8239856: [ntintel] asserts about copying unaligned array element JDK-8246031: SSLSocket.getSession() doesn't close connection for timeout/ interrupts JDK-8246613: Choose the default SecureRandom algo based on registration ordering JDK-8237192: Generate stripped/public pdbs on Windows for jdk images JDK-8235563: [TESTBUG] appcds/CommandLineFlagComboNegative.java does not handle archive mapping failure JDK-8238738: AudioSystem.getMixerInfo() takes about 30 sec to report a gone audio device JDK-8243539: Copyright info (Year) should be updated for fix of 8241638 JDK-8241638: launcher time metrics always report 1 on Linux when _JAVA_LAUNCHER_DEBUG set JDK-8231713: x86_32 build failures after JDK-8226721 (Missing intrinsics for Math.ceil, floor, rint) JDK-8226721: Missing intrinsics for Math.ceil, floor, rint JDK-8244407: JVM crashes after transformation in C2 IdealLoopTree::split_fall_in JDK-8237045: JVM uses excessive memory with -XX:+EnableJVMCI -XX:JVMCICounterSize=2147483648 JDK-8225069: Remove Comodo root certificate that is expiring in May 2020 JDK-8225068: Remove DocuSign root certificate that is expiring in May 2020 JDK-8237474: Default SSLEngine should create in server role JDK-8232846: ProcessHandle.Info command with non-English shows question marks JDK-8214481: freetype path does not disable TrueType hinting with AA+FM hints JDK-8242626: enhance posix print_rlimit_info JDK-8244951: Missing entitlements for hardened runtime JDK-8240972: macOS codesign fail on macOS 10.13.5 or older JDK-8238534: Deep sign macOS bundles before bundle archive is being created JDK-8235585: Enable macOS codesigning for all libraries and executables JDK-8225126: Test SetBoundsPaintTest.html faild on Windows when desktop is scaled JDK-8240786: [TESTBUG] The test java/awt/Window/GetScreenLocation/GetScreenLocationTest.java fails on HiDPI screen JDK-8238575: DragSourceEvent.getLocation() returns wrong value on HiDPI screens (Windows) JDK-8040630: Popup menus and tooltips flicker with previous popup contents when first shown JDK-8232634: Problem List ICMColorDataTest.java JDK-8196181: sun/java2d/GdiRendering/InsetClipping.java fails JDK-8239055: Wrong implementation of VMState.hasListener JDK-8228687: [TESTBUG] exclude Container tests from hotspot_misc group JDK-8239915: Zero VM crashes when handling dynamic constant JDK-8241586: compiler/cpuflags/TestAESIntrinsicsOnUnsupportedConfig.java fails on aarch64 JDK-8237879: make 4.3 breaks build JDK-8233364: Fix undefined behavior in Canonicalizer::do_ShiftOp JDK-8231025: Incorrect method tag offset for big endian platform JDK-8242379: [TESTBUG] compiler/loopopts/TestLoopUnswitchingLostCastDependency.java fails with release VMs JDK-8241900: Loop unswitching may cause dependence on null check to be lost JDK-8153430: jdk regression test MletParserLocaleTest, ParserInfiniteLoopTest reduce default timeout JDK-8240286: [TESTBUG] Test command error in hotspot/jtreg/compiler/loopopts/superword/SumRedAbsNeg_Float.java JDK-8239819: XToolkit: Misread of screen information memory JDK-8242498: Invalid "sun.awt.TimedWindowEvent" object leads to JVM crash JDK-8242174: [macos] The NestedModelessDialogTest test make the macOS unstable JDK-8226806: [macOS 10.14] Methods of Java Robot should be called from appropriate thread JDK-8196019: java/awt/Window/Grab/GrabTest.java fails on Windows JDK-8240518: Incorrect JNU_ReleaseStringPlatformChars in Windows Print JDK-8233573: Toolkit.getScreenInsets(GraphicsConfiguration) may throw ClassCastException JDK-8239852: java/util/concurrent tests fail with -XX:+VerifyGraphEdges: assert(!VerifyGraphEdges) failed: verification should have failed JDK-8241808: [TESTBUG] The JDK-8039467 bug appeared on macOS JDK-8235153: [TESTBUG] [macos 10.15] java/awt/Graphics/DrawImageBG/SystemBgColorTest.java fails JDK-8176359: Frame#setMaximizedbounds not working properly in multi screen environments JDK-8234398: Replace ID2D1Factory::GetDesktopDpi with GetDeviceCaps JDK-8233707: systemScale.cpp could not compile with VS2019 JDK-8223260: NamingManager should cache InitialContextFactory JDK-8242470: Update Xerces to Version 2.12.1 JDK-8234809: set relro in linker flags when building with gcc JDK-8242108: Performance regression after fix for JDK-8229496 JDK-8234270: [REDO] JDK-8204128 NMT might report incorrect numbers for Compiler area JDK-8240223: Use consistent predicate order in and with PhaseIdealLoop::find_predicate JDK-8240220: IdealLoopTree::dump_head predicate printing is broken JDK-8238438: SuperWord::co_locate_pack picks memory state of first instead of last load JDK-8242541: Small charset issues (ISO8859-16, x-eucJP-Open, x-IBM834 and x-IBM949C) JDK-8239965: XMLEncoder/Test4625418.java fails due to "Error: Cp943 - can't read properly" JDK-8239976: Put JDK-8239965 on the ProblemList.txt JDK-8235834: IBM-943 charset encoder needs updating JDK-8022574: remove HaltNode code after uncommon trap calls JDK-8231720: Some perf regressions after 8225653 JDK-8225653: Provide more information when hitting SIGILL from HaltNode JDK-8156207: Resource allocated BitMaps are often cleared unnecessarily JDK-8227632: Incorrect PrintCompilation message: made not compilable on levels 0 1 2 3 4 JDK-8226879: Memory leak in Type::hashcons JDK-8225783: Incorrect use of binary operators on booleans in type.cpp JDK-8219904: ClassCastException when calling FlightRecorderMXBean#getRecordings() JDK-8233197: Invert JvmtiExport::post_vm_initialized() and Jfr:on_vm_start() start-up order for correct option parsing JDK-8233019: java.lang.Class.isPrimitive() (C1) returns wrong result if Klass* is aligned to 32bit JDK-8234824: java/nio/channels/SocketChannel/AdaptSocket.java fails on Windows 10 JDK-8220479: java/nio/channels/Selector/SelectWithConsumer.java failed at testTwoChannels() JDK-8220348: [ntintel] asserts about copying unaligned array JDK-8241568: (fs) UserPrincipalLookupService.lookupXXX failure with IOE "Operation not permitted" JDK-8234423: Modifying ArrayList.subList().subList() resets modCount of subList JDK-7143743: Potential memory leak with zip provider JDK-8239893: Windows handle Leak when starting processes using ProcessBuilder JDK-8232880: Update test documentation with additional settings for client UI tooltip tests JDK-8231351: Add notes for PKCS11 tests in the test doc JDK-8230079: Update test document by changing "TIMEOUT" to "TIMEOUT_FACTOR" JDK-8230813: Add JDK-8010500 to compiler/loopopts/superword/TestFuzzPreLoop.java bug list JDK-8240824: enhance print_full_memory_info on Linux by THP related information JDK-8238356: CodeHeap::blob_count() overestimates the number of blobs JDK-8209113: Use WeakReference for lastFontStrike for created Fonts JDK-8235403: Further cleanup to test serviceability/sa/ClhsdbCDSCore.java JDK-8201349: build broken when configured with --with-zlib=bundled on gcc 7.3 JDK-8233383: Various minor fixes JDK-8240529: CheckUnhandledOops breaks NULL check in Modules::define_module JDK-8240576: JVM crashes after transformation in C2 IdealLoopTree::merge_many_backedges JDK-8241556: Memory leak if -XX:CompileCommand is set JDK-8240905: assert(mem == (Node*)1 || mem == mem2) failed: multiple Memories being matched at once? JDK-8241296: Segfault in JNIHandleBlock::oops_do() JDK-8231779: crash HeapWord*ParallelScavengeHeap::failed_mem_allocate JDK-8235325: build failure on Linux after 8235243 JDK-8235243: handle VS2017 15.9 and VS2019 in abstract_vm_version JDK-8232056: GetOwnedMonitorInfoWithEATest.java fails with ZGC: Heap too small JDK-8238765: PhaseCFG::schedule_pinned_nodes cannot handle precedence edges from unmatched CFG nodes correctly JDK-8239005: [TESTBUG] test/hotspot/jtreg/runtime/StackGuardPages/TestStackGuardPages.java: exeinvoke.c: must initialize static state before calling do_overflow() JDK-8236759: ShouldNotReachHere in PhaseIdealLoop::verify_strip_mined_scheduling JDK-8238811: C2: assert(i >= req() || i == 0 || is_Region() || is_Phi()) with -XX:+VerifyGraphEdges JDK-8238756: C2: assert(((n) == __null || !VerifyIterativeGVN || !((n)->is_dead()))) failed: can not use dead node JDK-8237945: CTW: C2 compilation fails with assert(just_allocated_object(alloc_ctl) == ptr) failed: most recent allo JDK-8238591: CTW: Split applications/ctw/modules/jdk_localedata.java JDK-8238247: CTW runner should sweep nmethods more aggressively JDK-8238366: CTW runner closes standard output on exit JDK-8237086: assert(is_MachReturn()) running CTW with fix for JDK-8231291 JDK-8237951: CTW: C2 compilation fails with "malformed control flow" JDK-8183369: RFC unconformity of HttpURLConnection with proxy JDK-8227539: Replace wildcard address with loopback or local host in tests - part 20 JDK-8238721: Add failing client jtreg tests to the Problem List JDK-8214469: [macos] PIT: java/awt/Choice/ChoiceKeyEventReaction/ChoiceKeyEventReaction.java fails JDK-8194944: Regression automated test 'open/test/jdk/javax/swing/JInternalFrame/8145896/TestJInternalFrameMaximize.java' fails JDK-8196467: javax/swing/JInternalFrame/Test6325652.java fails JDK-8233649: Update ProblemList.txt to exclude failing headful tests on macos JDK-8232200: [macos 10.15] Windows in fullscreen tests jumps around the screen JDK-8238985: [TESTBUG] The arrow image is blue instead of green JDK-8224475: JTextPane does not show images in HTML rendering JDK-8231572: Use -lobjc instead of -fobjc-link-runtime in libosxsecurity JDK-8237396: JvmtiTagMap::weak_oops_do() should not trigger barriers JDK-8234968: check calloc rv in libinstrument InvocationAdapter JDK-8235332: TestInstanceCloneAsLoadsStores.java fails with -XX:+StressGCM JDK-8238190: [JVMCI] Fix single implementor speculation for diamond shapes. JDK-8210058: Algorithmic Italic font leans opposite angle in Printing JDK-8234625: hs test serviceability/sa/ClhsdbCDSCore.java fails on macOS 10.15 JDK-8235686: Add more custom hooks in Bundles.gmk JDK-8241660: Add virtualization information output to hs_err file on macOS JDK-8239224: libproc_impl.c previous_thr may be used uninitialized warning JDK-8239142: C2's UseUniqueSubclasses optimization is broken for array accesses JDK-8230388: Problemlist additional compiler/rtm tests JDK-8234323: NULL-check return value of SurfaceData_InitOps on macosx JDK-8235620: Broken merge between JDK-8006406 and JDK-8003559 JDK-4949105: Access Bridge lacks html tags parsing JDK-8230926: [macosx] Two apostrophes are entered instead of one with "U.S. International - PC" layout JDK-8240202: A few client tests leave mouse buttons pressed JDK-8226253: JAWS reports wrong number of radio buttons when buttons are hidden. JDK-8233696: [TESTBUG]Some jtreg tests fail when CAPS_LOCK is ON JDK-8234332: [TESTBUG] java/awt/Focus/DisposedWindow/DisposeDialogNotActivateOwnerTest/DisposeDialogNotActivateOwnerTest.java fails on linux-x64 nightly JDK-8234184: [TESTBUG] java/awt/Mouse/EnterExitEvents/ModalDialogEnterExitEventsTest.java fails in Windows JDK-8234786: Fix for JDK-8214578 breaks OS X 10.12 compatibility JDK-8214578: [macos] Problem with backslashes on macOS/JIS keyboard: Java ignores system settings JDK-8042383: [TEST_BUG] Test javax/swing/plaf/basic/BasicMenuUI/4983388/bug4983388.java fails with shortcuts on menus do not work JDK-8234288: Turkey Time Zone returns incorrect time zone name JDK-8230910: libsspi_bridge does not build on Windows 32bit JDK-8239456: vtable stub generation: assert failure (code size estimate) JDK-8239931: [win][x86] vtable stub generation: assert failure (code size estimate) follow-up JDK-8239091: Reversed arguments in call to strstr in freetype "debug" code. JDK-8221741: ClassCastException can happen when fontconfig.properties is used JDK-8235904: Infinite loop when rendering huge lines JDK-8236488: Support for configure option --with-native-debug-symbols=internal is impossible on Windows JDK-8212986: Make Visual Studio compiler check less strict JDK-8236953: [macos] JavaFX SwingNode is not rendered on macOS JDK-8216472: (se) Stack overflow during selection operation leads to crash (win) JDK-8239351: Give more meaningful InternalError messages in Deflater.c JDK-8237508: Simplify JarFile.isInitializing JDK-8234466: Class loading deadlock involving X509Factory#commitEvent() JDK-8235874: The ordering of Cipher Suites is not maintained provided through jdk.tls.client.cipherSuites and jdk.tls.server.cipherSuites system property. JDK-8234728: Some security tests should support TLSv1.3 JDK-8232357: Compare version info of Santuario to legal notice JDK-8238898: Missing hash characters for header on license file JDK-8233621: Mismatch in jsse.enableMFLNExtension property name JDK-8234501: remove obsolete NET_ReadV JDK-8232692: [TESTBUG] compiler/aot/fingerprint/SelfChangedCDS.java fails when cds is disabled JDK-8233491: Crash in AdapterHandlerLibrary::get_adapter with CDS due to code cache exhaustion JDK-8233820: Test crashed with assert(phi->operand_count() != 1 || phi->subst() != phi) failed: missed trivial simplification JDK-8214904: Test8004741.java failed due to "Too few ThreadDeath hits; expected at least 6 but saw only 5" JDK-8233920: MethodHandles::tryFinally generates illegal bytecode for long/double return type JDK-8232834: RunTest sometimes fails to produce valid exitcode.txt JDK-8232370: Refactor some com.sun.jdi tests to enable IDE integration JDK-8233608: Minimal build broken after JDK-8233494 JDK-8241445: Fix copyright in test/jdk/tools/launcher/ArgFileSyntax.java JDK-8240629: argfiles parsing broken for argfiles with comment cross 4096 bytes chunk JDK-8236700: Upgrading JSZip from v3.1.5 to v3.2.2 JDK-8218268: Javac treats Manifest Class-Path entries as Paths instead of URLs JDK-8232170: FSInfo#getJarClassPath throws an exception not declared in its throws clause JDK-8231863: Crash if classpath is read from @argument file and the main gets option argument JDK-8233466: aarch64: remove unnecessary load of mdo when profiling return and parameters type JDK-8233839: aarch64: missing memory barrier in NewObjectArrayStub and NewTypeArrayStub JDK-8229701: aarch64: C2 OSR compilation fails with "shouldn't process one node several times" in final graph reshaping JDK-8237055: [TESTBUG] compiler/c2/TestJumpTable.java fails with release VMs JDK-8237217: Incorrect G1StringDedupEntry type used in StringDedupTable destructor JDK-8229855: C2 fails with assert(false) failed: bad AD file JDK-8236709: struct SwitchRange in HS violates C++ One Definition Rule JDK-8236140: assert(!VerifyHashTableKeys || _hash_lock == 0) failed: remove node from hash table before modifying it JDK-8235984: C2: assert(out->in(PhiNode::Region) == head || out->in(PhiNode::Region) == slow_head) failed: phi must be either part of the slow or the fast loop JDK-8235998: [c2] Memory leaks during tracing after '8224193: stringStream should not use Resource Area'. JDK-8236179: C1 register allocation error with T_ADDRESS JDK-8235671: enhance print_rlimit_info in os_posix JDK-8235489: handle return values of sscanf calls in hotspot JDK-8233033: C2 produces wrong result while unswitching a loop due to lost control dependencies JDK-8235452: Strip mined loop verification fails with assert(is_OuterStripMinedLoop()) failed: invalid node class JDK-8235383: C1 compilation fails with -XX:+PrintIRDuringConstruction -XX:+Verbose JDK-8235510: java.util.zip.CRC32 performance drop after 8200067 JDK-8229994: assert(false) failed: Bad graph detected in get_early_ctrl_for_expensive JDK-8235288: AVX 512 instructions inadvertently used on Xeon for small vector width operations JDK-8234397: add OS uptime information to os::print_os_info output JDK-8231430: C2: Memory stomp in max_array_length() for T_ILLEGAL type JDK-8234741: enhance os::get_core_path on macOS JDK-8234617: C1: Incorrect result of field load due to missing narrowing conversion JDK-8230611: infinite loop in LogOutputList::wait_until_no_readers() JDK-8236470: Deal with ECDSA using ecdsa-with-SHA2 plus hash algorithm as AlgorithmId JDK-8225130: Add exception for expiring Comodo roots to VerifyCACerts test JDK-8225128: Add exception for expiring DocuSign root to VerifyCACerts test JDK-8215711: Missing key_share extension for (EC)DHE key exchange should alert missing_extension JDK-8238452: Keytool generates wrong expiration date if validity is set to 2050/01/01 JDK-8237962: give better error output for invalid OCSP response intervals in CertPathValidator checks JDK-8163251: Hard coded loop limit prevents reading of smart card data greater than 8k JDK-8242294: JSSE Client does not throw SSLException when an alert occurs during handshaking JDK-8230400: Missing constant pool entry for a method in stacktrace JDK-8226892: ActionListeners on JRadioButtons don't get notified when selection is changed with arrow keys JDK-8175984: ICC_Profile has un-needed, not-empty finalize method JDK-8223558: Java does not render Myanmar script correctly JDK-8215355: Object monitor deadlock with no threads holding the monitor (using jemalloc 5.1) JDK-8237869: exclude jtreg test security/infra/java/security/cert/CertPathValidator/certification/LuxTrustCA.java because of instabilities JDK-8234727: sun/security/ssl/X509TrustManagerImpl tests support TLSv1.3 JDK-8235263: Revert TLS 1.3 change that wrapped IOExceptions JDK-8234723: javax/net/ssl/TLS tests support TLSv1.3 JDK-8231810: javax/net/ssl/templates/SSLSocketSSLEngineTemplate.java fails intermittently with "java.lang.Exception: Unexpected EOF" JDK-8235255: ProblemList javax/net/ssl/templates/SSLSocketSSLEngineTemplate.java JDK-8234724: javax/net/ssl/templates/SSLSocketSSLEngineTemplate.java supports TLSv1.3 JDK-8239798: SSLSocket closes socket both socket endpoints on a SocketTimeoutException JDK-8238502: sunmscapi.dll causing EXCEPTION_ACCESS_VIOLATION JDK-8231081: TestMetadataRetention fails due to missing symbol id JDK-8230624: [TESTBUG] Problemlist JFR compiler/TestCodeSweeper.java JDK-8230238: Add another regression test for JDK-8134739 JDK-8230390: Problemlist SA tests with AOT JDK-8226779: [TESTBUG] Test JFR API from Java agent JDK-8231584: Deadlock with ClassLoader.findLibrary and System.loadLibrary call JDK-8225797: OldObjectSample event creates unexpected amount of checkpoint data JDK-8230398: Remove NULL checks before FREE_C_HEAP_ARRAY JDK-8234321: Call cache flush after generating trampoline. JDK-8233081: C1: PatchingStub for field access copies too much JDK-8233494: Avoid calling MallocTracker::record_malloc and record_free when NMT is off JDK-8233075: JFR - nmetods - misspelled in several places JDK-8209824: Improve the code coverage for ThreadLocal JDK-8233548: Update CUP to v0.11b JDK-8239557: [TESTBUG] VeryEarlyAssertTest.java validating "END." marker at lastline is not always true JDK-8234696: tools/jlink/plugins/VendorInfoPluginsTest.java times out JDK-8233291: [TESTBUG] tools/jlink/plugins/VendorInfoPluginsTest.java fails with debug or non-server VMs JDK-8233137: runtime/ErrorHandling/VeryEarlyAssertTest.java fails after 8232080 JDK-8232080: jlink plugins for vendor information and run-time options JDK-8224851: AArch64: fix warnings and errors with Clang and GCC 8.3 JDK-8232571: Add missing SIGINFO signal JDK-8232592: is shown in jstack mixed mode JDK-8232167: Visual Studio install found through --with-tools-dir value is discarded JDK-8223697: jfr tool can't format duration values greater than 1 minute JDK-8232207: Linux os::available_memory re-reads cgroup configuration on every invocation JDK-8216354: Syntax error in toolchain_windows.m4 JDK-8230677: Should disable Escape Analysis if JVMTI capability can_get_owned_monitor_info was taken JDK-8232052: use string literal for format string when handling PauseAtStartupFile JDK-8225694: Destination option missing in FlightRecorderMXBeanImpl JDK-8238942: Rendering artifacts with LCD text and fractional metrics JDK-8224109: Text spaced incorrectly by drawString under rotation with fractional metric JDK-8232226: [macos 10.15] test/jdk/java/awt/color/EqualityTest/EqualityTest.java may fail JDK-7124307: JSpinner and changing value by mouse JDK-8234137: The "AutoTestOnTop.java" test may run external applications JDK-8234769: Duplicate attribution in freetype.md JDK-8227324: Upgrade to freetype 2.10.1 JDK-8231243: [TESTBUG] CustomFont.java cannot find font file JDK-8232748: Build static versions of certain JDK libraries JDK-8232572: Add hooks for custom output dir in Bundles.gmk JDK-8235744: PIT: test/jdk/javax/swing/text/html/TestJLabelWithHTMLText.java times out in linux-x64 JDK-8223935: PIT: java/awt/font/WindowsIndicFonts.java fails on windows10 JDK-8235739: Rare NPE at WComponentPeer.getGraphics() JDK-8235638: NPE in LWWindowPeer.getOnscreenGraphics() JDK-8231438: [macOS] Dark mode for the desktop is not supported JDK-8234386: [macos] NPE was thrown at expanding Choice from maximized frame JDK-8232433: [macos 10.15] java/awt/Window/LocationAtScreenCorner/LocationAtScreenCorner.java may fail JDK-8233657: Intermittent NPE in Component.validate() JDK-8223108: Test java/awt/EventQueue/NonComponentSourcePost.java is unstable JDK-8146238: [macosx] Java2D Queue Flusher crash on OSX after switching between user accounts JDK-8231671: Fix copyright headers in hotspot (missing comma after year) JDK-8231503: [TESTBUG] compiler/{jvmci,aot} tests should not run with GCs that do not support JVMCI/AOT JDK-8231296: ZGC: vmTestbase/nsk/jvmti/Allocate/alloc001/ fails JDK-8016914: CoreDocumentImpl.setXmlVersion NPE JDK-8231294: ZGC: vmTestbase/nsk/jvmti/ResourceExhausted/resexhausted002 fails JDK-8223158: Docked MacBook cannot start any Java Swing applications JDK-8229483: Sinking load out of loop may trigger: assert(found_sfpt) failed: no node in loop that's not input to safepoint JDK-8231201: hs_err should print coalesced safepoint operations in Events section JDK-8230881: serviceability/sa/TestJmapCore tests fail with java.lang.RuntimeException: Could not find dump file JDK-8230711: ConnectionGraph::unique_java_object(Node* N) return NULL if n is not in the CG JDK-8230363: C2: Let ConnectionGraph::not_global_escape(Node* n) return false if n is not in the CG JDK-8231631: sun/net/ftp/FtpURLConnectionLeak.java fails intermittently with NPE JDK-8231213: Migrate SimpleDateFormatConstTest to JDK Repo JDK-8193596: java/net/DatagramPacket/ReuseBuf.java failed due to timeout JDK-8233018: Add a new test to verify that DatagramSocket is not interruptible JDK-8236873: Worker has a deadlock bug JDK-8235311: Tag mismatch may alert bad_record_mac JDK-8234339: replace JLI_StrTok in java_md_solinux.c JDK-8228725: AArch64: Purge method call format support JDK-8229352: Use of an uninitialized register in 32-bit ARM template interpreter JDK-8209802: Garbage collectors should register JFR types themselves to avoid build errors. JDK-8230431: Move G1 trace code from gcTrace* to G1 directory JDK-8227411: TestTimeMultiple.java failed "assert(!lease()) failed: invariant" JDK-8228757: Fail fast if the handshake type is unknown JDK-8229421: The logic of java/net/ipv6tests/TcpTest.java is flawed JDK-8191169: java/net/Authenticator/B4769350.java failed intermittently JDK-8230004: jdk/internal/jimage/JImageOpenTest.java runs no test JDK-8231387: java.security.Provider.getService returns random result due to race condition with mutating methods in the same class JDK-8176837: SunPKCS11 provider needs to check more details on PKCS11 Mechanism JDK-8217606: LdapContext#reconnect always opens a new connection JDK-8230376: [TESTBUG] runtime/StackTrace/HiddenFrameTest.java fails with release VM JDK-8216977: ShowHiddenFrames use in java_lang_StackTraceElement::fill_in appears broken JDK-8225430: Replace wildcard address with loopback or local host in tests - part 14 JDK-8228888: C2 compilation fails with assert "m has strange control" JDK-8193325: StackFrameInfo::getByteCodeIndex returns wrong value if bci > 32767 JDK-8228613: java.security.Provider#getServices order is no longer deterministic JDK-8189633: Missing -Xcheck:jni checking for DeleteWeakGlobalRef JDK-8227086: Use AS_NO_KEEPALIVE loads in HeapDumper JDK-8229158: make UseSwitchProfiling non-experimental or false by-default JDK-8229420: [Redo] jstat reports incorrect values for OU for CMS GC JDK-8229345: Memory leak due to vtable stubs not being shared on SPARC JDK-8228772: C2 compilation fails due to unschedulable graph if DominatorSearchLimit is reached JDK-8229169: False failure of GenericTaskQueue::pop_local on architectures with weak memory model JDK-8229020: Failure on CPUs allowing loads reordering: assert(_tasks[t] == 1) failed: What else? JDK-8231445: check ZALLOC return values in awt coding JDK-8225101: Crash at sun.awt.X11.XlibWrapper.XkbGetUpdatedMap when change keybord map JDK-8230480: check malloc/calloc results in java.desktop JDK-8232806: Introduce a system property to disable eager lambda initialization JDK-8225435: Upgrade IANA Language Subtag Registry to the latest for JDK14 JDK-8241750: x86_32 build failure after JDK-8227269 JDK-8227269: Slow class loading when running with JDWP JDK-8229022: BufferedReader performance can be improved by using StringBuilder JDK-8229899: Make java.io.File.isInvalid() less racy JDK-8231124: Missing closedir call with JDK-8223490 JDK-8223490: Optimize search algorithm for determining default time zone JDK-8232003: (fs) Files.write can leak file descriptor in the exception case JDK-8229236: CriticalJNINatives: dll handling should be done in native thread state JDK-8223769: Assert triggers with -XX:+StressReflectiveCode JDK-8227397: Add --with-extra-asflags configure option JDK-8228596: Class redefinition fails when condy instructions are removed JDK-8232713: Update BCEL version to 6.3.1 in license file JDK-8224157: BCEL: update to version 6.3.1 JDK-8228400: Remove built-in AArch64 simulator JDK-8239457: call ReleaseStringUTFChars before early returns in Java_sun_security_pkcs11_wrapper_PKCS11_connect JDK-8230900: missing ReleaseStringUTFChars in java.desktop native code JDK-8230901: missing ReleaseStringUTFChars in serviceability native code JDK-8236545: Compilation error in mach5 java/awt/FileDialog/MacOSGoToFolderCrash.java JDK-8234522: [macos] Crash with use of native file dialog JDK-8230769: BufImg_SetupICM add ReleasePrimitiveArrayCritical call in early return JDK-8134672: [TEST_BUG] Some tests should check isDisplayChangeSupported JDK-8228368: avoid incompatible pointer to integer conversion initializing gint in gtk2_interface JDK-8227645: Some tests in serviceability/sa run with fixed -Xmx values and risk running out of memory JDK-8229156: ProblemList gc/stress/gclocker/TestExcessGCLockerCollections.java JDK-8193042: NativeLookup::lookup_critical_entry() should only load shared library once JDK-8048556: Unnecessary GCLocker-initiated young GCs JDK-8226899: Problemlist compiler/rtm tests JDK-8229810: [macos] NullPointerException getting bounds of GraphicsConfiguration JDK-8229515: [macos] access to window property of NSView on wrong thread JDK-8243059: Build fails when --with-vendor-name contains a comma JDK-8232178: MacVolumesTest failed after upgrade to MacOS Catalina JDK-8231223: C2's conditional move optimization fails with assert(bol->Opcode() == Op_Bool) failed JDK-8231550: C2: ShouldNotReachHere() in verify_strip_mined_scheduling JDK-8231254: (fs) Add test for macOS Catalina changes to protect system software JDK-8227439: Turn off AOT by default JDK-8229406: ZGC: Fix incorrect statistics JDK-8228625: [TESTBUG] sun/tools/jhsdb/JShellHeapDumpTest.java fails with RuntimeException 'JShellToolProvider' missing from stdout/stderr JDK-8227834: build.log output from failing commands : include the hs_error file path in case of crashes in build JDK-8225715: jhsdb jmap fails to write binary heap dump of a jshell process JDK-8230466: check malloc/calloc results in jdk.hotspot.agent JDK-8228902: add os::dll_load to the unified logging os category JDK-8228501: java_props_macosx.c - provide missing CFRelease for CFLocaleCopyCurrent JDK-8238842: AIOOBE in GIFImageReader.initializeStringTable JDK-8243541: (tz) Upgrade time-zone data to tzdata2020a JDK-8221312: test/jdk/sanity/client/SwingSet/src/ColorChooserDemoTest.java failed JDK-8228479: Correct the format of ColorChooserDemoTest JDK-8226409: Enable argument profiling for sun.misc.Unsafe.put*/get* JDK-8226798: JVM crash in klassItable::initialize_itable_for_interface(int, InstanceKlass*, bool, Thread*) JDK-8227031: Print NMT statistics on fatal errors JDK-8227338: templateInterpreter.cpp: copy_table() needs to be safer JDK-8227032: MetaspaceUtils::print_report crashes when called before initialization JDK-8227035: JVM::printFlags fails in native OOM situations JDK-8225644: C1 dumps incorrect class name in ClassCastException message JDK-8230856: Java_java_net_NetworkInterface_getByName0 on unix misses ReleaseStringUTFChars in early return JDK-8224997: ChaCha20-Poly1305 TLS cipher suite decryption throws ShortBufferException JDK-8223727: com/sun/jndi/ldap/privconn/RunTest.java failed due to hang in LdapRequest.getReplyBer JDK-8213561: ZipFile/MultiThreadedReadTest.java timed out in tier1 JDK-8184157: (ch) AsynchronousFileChannel hangs with internal error when reading locked file JDK-8235637: jhsdb jmap from OpenJDK 11.0.5 doesn't work if prelink is enabled JDK-8225636: SA can't handle prelinked libraries JDK-8236996: Incorrect Roboto font rendering on Windows with subpixel antialiasing A team repository is open at: https://hg.openjdk.java.net/jdk-updates/jdk13u-dev The fixes from there go to 13.0.5 which is due on October 20, 2020 Thanks, --yan From gnu.andrew at redhat.com Wed Jul 15 16:08:49 2020 From: gnu.andrew at redhat.com (Andrew Hughes) Date: Wed, 15 Jul 2020 17:08:49 +0100 Subject: OpenJDK 11.0.8 Released Message-ID: <19418cc0-500d-b218-8e75-e80dfc8468c2@redhat.com> We are pleased to announce the release of OpenJDK 11.0.8. The source tarball is available from: * https://openjdk-sources.osci.io/openjdk11/openjdk-11.0.8-ga.tar.xz The tarball is accompanied by a digital signature available at: * https://openjdk-sources.osci.io/openjdk11/openjdk-11.0.8-ga.tar.xz.sig This is signed by our Red Hat OpenJDK key (openjdk at redhat.com): PGP Key: rsa4096/0x92EF8D39DC13168F (hkp://keys.gnupg.net) Fingerprint = CA5F 11C6 CE22 644D 42C6 AC44 92EF 8D39 DC13 168F SHA256 checksums: aa991f4a87c9b778800dea39988c556ea7d20582fab13ee11724557a29d2aa96 openjdk-11.0.8-ga.tar.xz 7112202237a5d7a58b49bedc5c7d21a0c6c95205c15707b048d58052d227ebce openjdk-11.0.8-ga.tar.xz.sig The checksums can be downloaded from: * https://openjdk-sources.osci.io/openjdk11/openjdk-11.0.8-ga.sha256 New in release OpenJDK 11.0.8 (2020-07-14): =========================================== Live versions of these release notes can be found at: * https://bitly.com/oj1108 * https://builds.shipilev.net/backports-monitor/release-notes-11.0.8.txt * Security fixes - JDK-8230613: Better ASCII conversions - JDK-8231800: Better listing of arrays - JDK-8232014: Expand DTD support - JDK-8233234: Better Zip Naming - JDK-8233239, CVE-2020-14562: Enhance TIFF support - JDK-8233255: Better Swing Buttons - JDK-8234032: Improve basic calendar services - JDK-8234042: Better factory production of certificates - JDK-8234418: Better parsing with CertificateFactory - JDK-8234836: Improve serialization handling - JDK-8236191: Enhance OID processing - JDK-8236867, CVE-2020-14573: Enhance Graal interface handling - JDK-8237117, CVE-2020-14556: Better ForkJoinPool behavior - JDK-8237592, CVE-2020-14577: Enhance certificate verification - JDK-8238002, CVE-2020-14581: Better matrix operations - JDK-8238013: Enhance String writing - JDK-8238804: Enhance key handling process - JDK-8238842: AIOOBE in GIFImageReader.initializeStringTable - JDK-8238843: Enhanced font handing - JDK-8238920, CVE-2020-14583: Better Buffer support - JDK-8238925: Enhance WAV file playback - JDK-8240119, CVE-2020-14593: Less Affine Transformations - JDK-8240482: Improved WAV file playback - JDK-8241379: Update JCEKS support - JDK-8241522: Manifest improved jar headers redux - JDK-8242136, CVE-2020-14621: Better XML namespace handling * Other changes - JDK-6933331: (d3d/ogl) java.lang.IllegalStateException: Buffers have not been created - JDK-7124307: JSpinner and changing value by mouse - JDK-8022574: remove HaltNode code after uncommon trap calls - JDK-8039082: [TEST_BUG] Test java/awt/dnd/BadSerializationTest/BadSerializationTest.java fails - JDK-8040630: Popup menus and tooltips flicker with previous popup contents when first shown - JDK-8044365: (dc) MulticastSendReceiveTests.java failing with ENOMEM when joining group (OS X 10.9) - JDK-8048215: [TESTBUG] java/lang/management/ManagementFactory/ThreadMXBeanProxy.java Expected non-null LockInfo - JDK-8051349: nsk/jvmti/scenarios/sampling/SP06/sp06t003 fails in nightly - JDK-8080353: JShell: Better error message on attempting to add default method - JDK-8139876: Exclude hanging nsk/stress/stack from execution with deoptimization enabled - JDK-8146090: java/lang/ref/ReachabilityFenceTest.java fails with -XX:+DeoptimizeALot - JDK-8153430: jdk regression test MletParserLocaleTest, ParserInfiniteLoopTest reduce default timeout - JDK-8156207: Resource allocated BitMaps are often cleared unnecessarily - JDK-8159740: JShell: corralled declarations do not have correct source to wrapper mapping - JDK-8175984: ICC_Profile has un-needed, not-empty finalize method - JDK-8176359: Frame#setMaximizedbounds not working properly in multi screen environments - JDK-8183369: RFC unconformity of HttpURLConnection with proxy - JDK-8187078: -XX:+VerifyOops finds numerous problems when running JPRT - JDK-8191169: java/net/Authenticator/B4769350.java failed intermittently - JDK-8191930: [Graal] emits unparseable XML into compile log - JDK-8193879: Java debugger hangs on method invocation - JDK-8196019: java/awt/Window/Grab/GrabTest.java fails on Windows - JDK-8196181: sun/java2d/GdiRendering/InsetClipping.java fails - JDK-8198000: java/awt/List/EmptyListEventTest/EmptyListEventTest.java debug assert on Windows - JDK-8198001: java/awt/Menu/WrongParentAfterRemoveMenu/WrongParentAfterRemoveMenu.java debug assert on Windows - JDK-8198339: Test javax/swing/border/Test6981576.java is unstable - JDK-8200701: jdk/jshell/ExceptionsTest.java fails on Windows, after JDK-8198801 - JDK-8203264: JNI exception pending in PlainDatagramSocketImpl.c:740 - JDK-8203672: JNI exception pending in PlainSocketImpl.c - JDK-8203673: JNI exception pending in DualStackPlainDatagramSocketImpl.c:398 - JDK-8204834: Fix confusing "allocate" naming in OopStorage - JDK-8205399: Set node color on pinned HashMap.TreeNode deletion - JDK-8205653: test/jdk/sun/management/jmxremote/bootstrap/RmiRegistrySslTest.java and RmiSslBootstrapTest.sh fail with handshake_failure - JDK-8206179: com/sun/management/OperatingSystemMXBean/GetCommittedVirtualMemorySize.java fails with Committed virtual memory size illegal value - JDK-8207334: VM times out in VM_HandshakeAllThreads::doit() with RunThese30M - JDK-8208277: Code cache heap (-XX:ReservedCodeCacheSize) doesn't work with 1GB LargePages - JDK-8209113: Use WeakReference for lastFontStrike for created Fonts - JDK-8209333: Socket reset issue for TLS 1.3 socket close - JDK-8209439: C2 library_call can potentially ignore Math.pow intrinsic or use null pointer - JDK-8209534: [TESTBUG]runtime/appcds/cacheObject/ArchivedModuleCompareTest.java fails with EnableJVMCI. - JDK-8210147: adjust some WSAGetLastError usages in windows network coding - JDK-8210284: "assert((av & 0x00000001) == 0) failed: unsupported V8" on Solaris 11.4 - JDK-8210303: VM_HandshakeAllThreads fails assert with "failed: blocked and not walkable" - JDK-8210515: [TESTBUG]CheckArchivedModuleApp.java needs to check if EnableJVMCI is set. - JDK-8210788: Javadoc for Thread.join(long, int) should specify that it waits forever when both arguments are zero - JDK-8211301: [macos] support full window content options - JDK-8211332: Space for stub routines (code_size2) is too small on new Skylake CPUs - JDK-8211339: NPE during SSL handshake caused by HostnameChecker - JDK-8211392: compiler/profiling/spectrapredefineclass_classloaders/Launcher.java times out in JDK12 CI - JDK-8211743: [AOT] crash in ScopeDesc::decode_body() when JVMTI walks AOT frames - JDK-8212154: [TESTBUG] CheckArchivedModuleApp fails with NPE when JVMCI is absent - JDK-8212167: JShell : Stack trace of exception has wrong line number - JDK-8212933: Thread-SMR: requesting a VM operation whilst holding a ThreadsListHandle can cause deadlocks - JDK-8212986: Make Visual Studio compiler check less strict - JDK-8213250: CDS archive creation aborts due to metaspace object allocation failure - JDK-8213516: jck test api/javax_accessibility/AccessibleState/fields.html fails intermittent - JDK-8213947: ARM32: failed check_simd should set UsePopCountInstruction to false - JDK-8214418: half-closed SSLEngine status may cause application dead loop - JDK-8214440: ldap over a TLS connection negotiate failed with "javax.net.ssl.SSLPeerUnverifiedException: hostname of the server '' does not match the hostname in the server's certificate" - JDK-8214444: Wrong strncat limits in dfa.cpp - JDK-8214481: freetype path does not disable TrueType hinting with AA+FM hints - JDK-8214571: -Xdoclint of array serialField gives "error: array type not allowed here" - JDK-8214856: Errors with JSZip in web console after upgrade to 3.1.5 - JDK-8214862: assert(proj != __null) at compile.cpp:3251 - JDK-8215369: Jcstress pollute /var/tmp with temporary files. - JDK-8215551: Missing case label in nmethod::reloc_string_for() - JDK-8215555: TieredCompilation C2 threads can excessively block handshakes - JDK-8215711: Missing key_share extension for (EC)DHE key exchange should alert missing_extension - JDK-8216151: [Graal] Module jdk.internal.vm.compiler.management has not been granted accessClassInPackage.org.graalvm.compiler.debug - JDK-8216154: C4819 warnings at HotSpot sources on Windows - JDK-8216541: CompiledICHolders of VM locked unloaded nmethods are released too late - JDK-8217230: assert(t == t_no_spec) failure in NodeHash::check_no_speculative_types() - JDK-8217404: --with-jvm-features doesn't work when multiple features are explicitly disabled - JDK-8217447: Develop flag TraceICs is broken - JDK-8217606: LdapContext#reconnect always opens a new connection - JDK-8218807: Compilation database (compile_commands.json) may contain obsolete items - JDK-8219214: Infinite Loop in CodeSection::dump() - JDK-8219904: ClassCastException when calling FlightRecorderMXBean#getRecordings() - JDK-8219991: New fix of the deadlock in sun.security.ssl.SSLSocketImpl - JDK-8221121: applications/microbenchmarks are encountering crashes in tier5 - JDK-8221445: FastSysexMessage constructor crashes MIDI receiption thread - JDK-8221482: Initialize VMRegImpl::regName[] earlier to prevent assert during PrintStubCode - JDK-8221741: ClassCastException can happen when fontconfig.properties is used - JDK-8221823: Requested JDialog width is ignored - JDK-8223108: Test java/awt/EventQueue/NonComponentSourcePost.java is unstable - JDK-8223935: PIT: java/awt/font/WindowsIndicFonts.java fails on windows10 - JDK-8224109: Text spaced incorrectly by drawString under rotation with fractional metric - JDK-8224632: testbug: java/awt/dnd/RemoveDropTargetCrashTest/RemoveDropTargetCrashTest.java fails on MacOS - JDK-8224793: os::die() does not honor CreateCoredumpOnCrash option - JDK-8224847: gc/stress/TestReclaimStringsLeaksMemory.java fails with reserved greater than expected - JDK-8224931: disable JAOTC invokedynamic support until 8223533 is fixed - JDK-8224997: ChaCha20-Poly1305 TLS cipher suite decryption throws ShortBufferException - JDK-8225068: Remove DocuSign root certificate that is expiring in May 2020 - JDK-8225069: Remove Comodo root certificate that is expiring in May 2020 - JDK-8225126: Test SetBoundsPaintTest.html faild on Windows when desktop is scaled - JDK-8225325: Add tests for redefining a class' private method during resolution of the bootstrap specifier - JDK-8225622: [AOT] runtime/SharedArchiveFile/TestInterpreterMethodEntries.java crashed with AOTed java.base - JDK-8225653: Provide more information when hitting SIGILL from HaltNode - JDK-8225783: Incorrect use of binary operators on booleans in type.cpp - JDK-8225789: Empty method parameter type should generate ClassFormatError - JDK-8226198: use of & instead of && in LibraryCallKit::arraycopy_restore_alloc_state - JDK-8226253: JAWS reports wrong number of radio buttons when buttons are hidden. - JDK-8226653: [accessibility] Can edit text cell correctly, but Accessibility Tool reads nothing about editor - JDK-8226806: [macOS 10.14] Methods of Java Robot should be called from appropriate thread - JDK-8226879: Memory leak in Type::hashcons - JDK-8227632: Incorrect PrintCompilation message: made not compilable on levels 0 1 2 3 4 - JDK-8228407: JVM crashes with shared archive file mismatch - JDK-8228482: fix xlc16/xlclang comparison of distinct pointer types and string literal conversion warnings - JDK-8228757: Fail fast if the handshake type is unknown - JDK-8229158: make UseSwitchProfiling non-experimental or false by-default - JDK-8229421: The logic of java/net/ipv6tests/TcpTest.java is flawed - JDK-8229855: C2 fails with assert(false) failed: bad AD file - JDK-8230591: AArch64: Missing intrinsics for Math.ceil, floor, rint - JDK-8231118: ARM32: Math tests failures - JDK-8231213: Migrate SimpleDateFormatConstTest to JDK Repo - JDK-8231243: [TESTBUG] CustomFont.java cannot find font file - JDK-8231438: [macOS] Dark mode for the desktop is not supported - JDK-8231550: C2: ShouldNotReachHere() in verify_strip_mined_scheduling - JDK-8231564: setMaximizedBounds is broken with large display scale and multiple monitors - JDK-8231572: Use -lobjc instead of -fobjc-link-runtime in libosxsecurity - JDK-8231631: sun/net/ftp/FtpURLConnectionLeak.java fails intermittently with NPE - JDK-8231671: Fix copyright headers in hotspot (missing comma after year) - JDK-8231720: Some perf regressions after 8225653 - JDK-8231779: crash HeapWord*ParallelScavengeHeap::failed_mem_allocate - JDK-8231863: Crash if classpath is read from @argument file and the main gets option argument - JDK-8232080: jlink plugins for vendor information and run-time options - JDK-8232106: [x86] C2: SIGILL due to usage of SSSE3 instructions on processors which don't support it - JDK-8232134: Change to Visual Studio 2017 15.9.16 for building on Windows at Oracle - JDK-8232226: [macos 10.15] test/jdk/java/awt/color/EqualityTest/EqualityTest.java may fail - JDK-8232357: Compare version info of Santuario to legal notice - JDK-8232572: Add hooks for custom output dir in Bundles.gmk - JDK-8232634: Problem List ICMColorDataTest.java - JDK-8232748: Build static versions of certain JDK libraries - JDK-8232846: ProcessHandle.Info command with non-English shows question marks - JDK-8233033: C2 produces wrong result while unswitching a loop due to lost control dependencies - JDK-8233137: runtime/ErrorHandling/VeryEarlyAssertTest.java fails after 8232080 - JDK-8233197: Invert JvmtiExport::post_vm_initialized() and Jfr:on_vm_start() start-up order for correct option parsing - JDK-8233291: [TESTBUG] tools/jlink/plugins/VendorInfoPluginsTest.java fails with debug or non-server VMs - JDK-8233364: Fix undefined behavior in Canonicalizer::do_ShiftOp - JDK-8233573: Toolkit.getScreenInsets(GraphicsConfiguration) may throw ClassCastException - JDK-8233608: Minimal build broken after JDK-8233494 - JDK-8233621: Mismatch in jsse.enableMFLNExtension property name - JDK-8233696: [TESTBUG]Some jtreg tests fail when CAPS_LOCK is ON - JDK-8233707: systemScale.cpp could not compile with VS2019 - JDK-8233801: GCMEmptyIv.java test fails on Solaris 11.4 - JDK-8233880: Support compilers with multi-digit major version numbers - JDK-8233920: MethodHandles::tryFinally generates illegal bytecode for long/double return type - JDK-8234137: The "AutoTestOnTop.java" test may run external applications - JDK-8234146: compiler/jsr292/ContinuousCallSiteTargetChange.java times out on SPARC - JDK-8234184: [TESTBUG] java/awt/Mouse/EnterExitEvents/ModalDialogEnterExitEventsTest.java fails in Windows - JDK-8234270: [REDO] JDK-8204128 NMT might report incorrect numbers for Compiler area - JDK-8234332: [TESTBUG] java/awt/Focus/DisposedWindow/DisposeDialogNotActivateOwnerTest/DisposeDialogNotActivateOwnerTest.java fails on linux-x64 nightly - JDK-8234398: Replace ID2D1Factory::GetDesktopDpi with GetDeviceCaps - JDK-8234522: [macos] Crash with use of native file dialog - JDK-8234691: Potential double-free in ParallelSPCleanupTask constructor - JDK-8234696: tools/jlink/plugins/VendorInfoPluginsTest.java times out - JDK-8234727: sun/security/ssl/X509TrustManagerImpl tests support TLSv1.3 - JDK-8234728: Some security tests should support TLSv1.3 - JDK-8234779: Provide idiom for declaring classes noncopyable - JDK-8234968: check calloc rv in libinstrument InvocationAdapter - JDK-8235153: [TESTBUG] [macos 10.15] java/awt/Graphics/DrawImageBG/SystemBgColorTest.java fails - JDK-8235183: Remove the "HACK CODE" in comment - JDK-8235263: Revert TLS 1.3 change that wrapped IOExceptions - JDK-8235311: Tag mismatch may alert bad_record_mac - JDK-8235332: TestInstanceCloneAsLoadsStores.java fails with -XX:+StressGCM - JDK-8235452: Strip mined loop verification fails with assert(is_OuterStripMinedLoop()) failed: invalid node class - JDK-8235584: UseProfiledLoopPredicate fails with assert(_phase->get_loop(c) == loop) failed: have to be in the same loop - JDK-8235620: Broken merge between JDK-8006406 and JDK-8003559 - JDK-8235638: NPE in LWWindowPeer.getOnscreenGraphics() - JDK-8235686: Add more custom hooks in Bundles.gmk - JDK-8235739: Rare NPE at WComponentPeer.getGraphics() - JDK-8235762: JVM crash in SWPointer during C2 compilation - JDK-8235834: IBM-943 charset encoder needs updating - JDK-8235874: The ordering of Cipher Suites is not maintained provided through jdk.tls.client.cipherSuites and jdk.tls.server.cipherSuites system property. - JDK-8235908: omit ThreadPriorityPolicy warning when value is set from image - JDK-8235984: C2: assert(out->in(PhiNode::Region) == head || out->in(PhiNode::Region) == slow_head) failed: phi must be either part of the slow or the fast loop - JDK-8236211: [Graal] compiler/graalunit/GraphTest.java is skipped in all testing - JDK-8236470: Deal with ECDSA using ecdsa-with-SHA2 plus hash algorithm as AlgorithmId - JDK-8236545: Compilation error in mach5 java/awt/FileDialog/MacOSGoToFolderCrash.java - JDK-8236700: Upgrading JSZip from v3.1.5 to v3.2.2 - JDK-8236759: ShouldNotReachHere in PhaseIdealLoop::verify_strip_mined_scheduling - JDK-8236897: Fix the copyright header for pkcs11gcm2.h - JDK-8236921: Add build target to produce a JDK image suitable for a Graal/SVM build - JDK-8236953: [macos] JavaFX SwingNode is not rendered on macOS - JDK-8236996: Incorrect Roboto font rendering on Windows with subpixel antialiasing - JDK-8237045: JVM uses excessive memory with -XX:+EnableJVMCI -XX:JVMCICounterSize=2147483648 - JDK-8237055: [TESTBUG] compiler/c2/TestJumpTable.java fails with release VMs - JDK-8237086: assert(is_MachReturn()) running CTW with fix for JDK-8231291 - JDK-8237192: Generate stripped/public pdbs on Windows for jdk images - JDK-8237396: JvmtiTagMap::weak_oops_do() should not trigger barriers - JDK-8237474: Default SSLEngine should create in server role - JDK-8237859: C2: Crash when loads float above range check - JDK-8237951: CTW: C2 compilation fails with "malformed control flow" - JDK-8237962: give better error output for invalid OCSP response intervals in CertPathValidator checks - JDK-8238190: [JVMCI] Fix single implementor speculation for diamond shapes. - JDK-8238356: CodeHeap::blob_count() overestimates the number of blobs - JDK-8238452: Keytool generates wrong expiration date if validity is set to 2050/01/01 - JDK-8238555: Allow Initialization of SunPKCS11 with NSS when there are external FIPS modules in the NSSDB - JDK-8238575: DragSourceEvent.getLocation() returns wrong value on HiDPI screens (Windows) - JDK-8238676: jni crashes on accessing it from process exit hook - JDK-8238721: Add failing client jtreg tests to the Problem List - JDK-8238738: AudioSystem.getMixerInfo() takes about 30 sec to report a gone audio device - JDK-8238756: C2: assert(((n) == __null || !VerifyIterativeGVN || !((n)->is_dead()))) failed: can not use dead node - JDK-8238765: PhaseCFG::schedule_pinned_nodes cannot handle precedence edges from unmatched CFG nodes correctly - JDK-8238898: Missing hash characters for header on license file - JDK-8238942: Rendering artifacts with LCD text and fractional metrics - JDK-8238985: [TESTBUG] The arrow image is blue instead of green - JDK-8239000: handle ContendedPaddingWidth in vm_version_ppc - JDK-8239055: Wrong implementation of VMState.hasListener - JDK-8239091: Reversed arguments in call to strstr in freetype "debug" code. - JDK-8239142: C2's UseUniqueSubclasses optimization is broken for array accesses - JDK-8239224: libproc_impl.c previous_thr may be used uninitialized warning - JDK-8239351: Give more meaningful InternalError messages in Deflater.c - JDK-8239365: ProcessBuilder test modifications for AIX execution - JDK-8239456: vtable stub generation: assert failure (code size estimate) - JDK-8239457: call ReleaseStringUTFChars before early returns in Java_sun_security_pkcs11_wrapper_PKCS11_connect - JDK-8239462: jdk.hotspot.agent misses some ReleaseStringUTFChars calls in case of early returns - JDK-8239557: [TESTBUG] VeryEarlyAssertTest.java validating "END." marker at lastline is not always true - JDK-8239787: AArch64: String.indexOf may incorrectly handle empty strings - JDK-8239792: Bump update version for OpenJDK: jdk-11.0.8 - JDK-8239798: SSLSocket closes socket both socket endpoints on a SocketTimeoutException - JDK-8239819: XToolkit: Misread of screen information memory - JDK-8239852: java/util/concurrent tests fail with -XX:+VerifyGraphEdges: assert(!VerifyGraphEdges) failed: verification should have failed - JDK-8239893: Windows handle Leak when starting processes using ProcessBuilder - JDK-8239915: Zero VM crashes when handling dynamic constant - JDK-8239931: [win][x86] vtable stub generation: assert failure (code size estimate) follow-up - JDK-8239976: Put JDK-8239965 on the ProblemList.txt - JDK-8240073: Fix 'test-make' build target in 11u - JDK-8240197: Cannot start JVM when $JAVA_HOME includes CJK characters - JDK-8240202: A few client tests leave mouse buttons pressed - JDK-8240220: IdealLoopTree::dump_head predicate printing is broken - JDK-8240223: Use consistent predicate order in and with PhaseIdealLoop::find_predicate - JDK-8240227: Loop predicates should be copied to unswitched loops - JDK-8240286: [TESTBUG] Test command error in hotspot/jtreg/compiler/loopopts/superword/SumRedAbsNeg_Float.java - JDK-8240518: Incorrect JNU_ReleaseStringPlatformChars in Windows Print - JDK-8240529: CheckUnhandledOops breaks NULL check in Modules::define_module - JDK-8240576: JVM crashes after transformation in C2 IdealLoopTree::merge_many_backedges - JDK-8240603: Windows 32bit compile error after 8238676 - JDK-8240629: argfiles parsing broken for argfiles with comment cross 4096 bytes chunk - JDK-8240711: TestJstatdPort.java failed due to "ExportException: Port already in use:" - JDK-8240786: [TESTBUG] The test java/awt/Window/GetScreenLocation/GetScreenLocationTest.java fails on HiDPI screen - JDK-8240824: enhance print_full_memory_info on Linux by THP related information - JDK-8240827: Downport SSLSocketImpl.java from "8221882: Use fiber-friendly java.util.concurrent.locks in JSSE" - JDK-8240905: assert(mem == (Node*)1 || mem == mem2) failed: multiple Memories being matched at once? - JDK-8240972: macOS codesign fail on macOS 10.13.5 or older - JDK-8241445: Fix copyright in test/jdk/tools/launcher/ArgFileSyntax.java - JDK-8241458: [JVMCI] add mark value to expose CodeOffsets::Frame_Complete - JDK-8241464: [11u] Backport: make rehashing be a needed guaranteed safepoint cleanup action - JDK-8241556: Memory leak if -XX:CompileCommand is set - JDK-8241568: (fs) UserPrincipalLookupService.lookupXXX failure with IOE "Operation not permitted" - JDK-8241586: compiler/cpuflags/TestAESIntrinsicsOnUnsupportedConfig.java fails on aarch64 - JDK-8241638: launcher time metrics always report 1 on Linux when _JAVA_LAUNCHER_DEBUG set - JDK-8241660: Add virtualization information output to hs_err file on macOS - JDK-8241808: [TESTBUG] The JDK-8039467 bug appeared on macOS - JDK-8241888: Mirror jdk.security.allowNonCaAnchor system property with a security one - JDK-8241900: Loop unswitching may cause dependence on null check to be lost - JDK-8241948: enhance list of environment variables printed in hs_err file - JDK-8241996: on linux set full relro in the linker flags - JDK-8242108: Performance regression after fix for JDK-8229496 - JDK-8242141: New System Properties to configure the TLS signature schemes - JDK-8242154: Backport parts of JDK-4947890 to OpenJDK 11u - JDK-8242174: [macos] The NestedModelessDialogTest test make the macOS unstable - JDK-8242239: [Graal] javax/management/generified/GenericTest.java fails: FAILED: queryMBeans sets same - JDK-8242294: JSSE Client does not throw SSLException when an alert occurs during handshaking - JDK-8242379: [TESTBUG] compiler/loopopts/TestLoopUnswitchingLostCastDependency.java fails with release VMs - JDK-8242470: Update Xerces to Version 2.12.1 - JDK-8242498: Invalid "sun.awt.TimedWindowEvent" object leads to JVM crash - JDK-8242541: Small charset issues (ISO8859-16, x-eucJP-Open, x-IBM834 and x-IBM949C) - JDK-8242626: enhance posix print_rlimit_info - JDK-8243059: Build fails when --with-vendor-name contains a comma - JDK-8243539: Copyright info (Year) should be updated for fix of 8241638 - JDK-8243541: (tz) Upgrade time-zone data to tzdata2020a - JDK-8244407: JVM crashes after transformation in C2 IdealLoopTree::split_fall_in - JDK-8244520: problemlist java/awt/font/Rotate/RotatedFontTest.java on linux - JDK-8244777: ClassLoaderStats VM Op uses constant hash value - JDK-8244853: The static build of libextnet is missing the JNI_OnLoad_extnet function - JDK-8244951: Missing entitlements for hardened runtime - JDK-8245047: [PPC64] C2: ReverseBytes + Load always match to unordered Load (acquire semantics missing) - JDK-8245649: Revert 8245397 backport of 8230591 - JDK-8246031: SSLSocket.getSession() doesn't close connection for timeout/ interrupts - JDK-8246613: Choose the default SecureRandom algo based on registration ordering - JDK-8248505: Unexpected NoSuchAlgorithmException when using secure random impl from BCFIPS provider Notes on individual issues: =========================== security-libs/java.security: JDK-8244167: Removal of Comodo Root CA Certificate ================================================== The following expired Comodo root CA certificate was removed from the `cacerts` keystore: + alias name "addtrustclass1ca [jdk]" Distinguished Name: CN=AddTrust Class 1 CA Root, OU=AddTrust TTP Network, O=AddTrust AB, C=SE JDK-8244166: Removal of DocuSign Root CA Certificate ==================================================== The following expired DocuSign root CA certificate was removed from the `cacerts` keystore: + alias name "keynectisrootca [jdk]" Distinguished Name: CN=KEYNECTIS ROOT CA, OU=ROOT, O=KEYNECTIS, C=FR security-libs/javax.crypto:pkcs11: JDK-8240191: Allow SunPKCS11 initialization with NSS when external FIPS modules are present in the Security Modules Database ========================================================== The SunPKCS11 security provider can now be initialized with NSS when FIPS-enabled external modules are configured in the Security Modules Database (NSSDB). Prior to this change, the SunPKCS11 provider would throw a RuntimeException with the message: "FIPS flag set for non-internal module" when such a library was configured for NSS in non-FIPS mode. This change allows the JDK to work properly with recent NSS releases in GNU/Linux operating systems when the system-wide FIPS policy is turned on. Further information can be found in JDK-8238555. security-libs/javax.net.ssl: JDK-8245077: Default SSLEngine Created in Server Role ===================================================== In JDK 11 and later, `javax.net.ssl.SSLEngine` by default used client mode when handshaking. As a result, the set of default enabled protocols may differ to what is expected. `SSLEngine` would usually be used in server mode. From this JDK release onwards, `SSLEngine` will default to server mode. The `javax.net.ssl.SSLEngine.setUseClientMode(boolean mode)` method may be used to configure the mode. JDK-8242147: New System Properties to Configure the TLS Signature Schemes ======================================================== Two new System Properties are added to customize the TLS signature schemes in JDK. `jdk.tls.client.SignatureSchemes` is added for TLS client side, and `jdk.tls.server.SignatureSchemes` is added for server side. Each System Property contains a comma-separated list of supported signature scheme names specifying the signature schemes that could be used for the TLS connections. The names are described in the "Signature Schemes" section of the *Java Security Standard Algorithm Names Specification*. Thanks, -- Andrew :) Senior Free Java Software Engineer OpenJDK Package Owner Red Hat, Inc. (http://www.redhat.com) PGP Key: ed25519/0xCFDA0F9B35964222 (hkp://keys.gnupg.net) Fingerprint = 5132 579D D154 0ED2 3E04 C5A0 CFDA 0F9B 3596 4222 From gnu.andrew at redhat.com Wed Jul 15 16:20:55 2020 From: gnu.andrew at redhat.com (Andrew Hughes) Date: Wed, 15 Jul 2020 17:20:55 +0100 Subject: OpenJDK 11.0.8 Released In-Reply-To: <19418cc0-500d-b218-8e75-e80dfc8468c2@redhat.com> References: <19418cc0-500d-b218-8e75-e80dfc8468c2@redhat.com> Message-ID: <9c5b9d90-ef8b-b583-1794-153deac49d50@redhat.com> On 15/07/2020 17:08, Andrew Hughes wrote: > New in release OpenJDK 11.0.8 (2020-07-14): > =========================================== > Live versions of these release notes can be found at: > * https://bitly.com/oj1108 * https://bitly.com/openjdk1108 I'll have to use 'openjdk' in full in future, as oj11* seems to keep being used by others :-( -- Andrew :) Senior Free Java Software Engineer OpenJDK Package Owner Red Hat, Inc. (http://www.redhat.com) PGP Key: ed25519/0xCFDA0F9B35964222 (hkp://keys.gnupg.net) Fingerprint = 5132 579D D154 0ED2 3E04 C5A0 CFDA 0F9B 3596 4222 From sgehwolf at redhat.com Wed Jul 15 16:50:07 2020 From: sgehwolf at redhat.com (Severin Gehwolf) Date: Wed, 15 Jul 2020 18:50:07 +0200 Subject: OpenJDK 11.0.8 Released In-Reply-To: <19418cc0-500d-b218-8e75-e80dfc8468c2@redhat.com> References: <19418cc0-500d-b218-8e75-e80dfc8468c2@redhat.com> Message-ID: <7853e005e8650786777b81b98dc839acae8fdf4f.camel@redhat.com> On Wed, 2020-07-15 at 17:08 +0100, Andrew Hughes wrote: > We are pleased to announce the release of OpenJDK 11.0.8. > > The source tarball is available from: > > * https://openjdk-sources.osci.io/openjdk11/openjdk-11.0.8-ga.tar.xz > > The tarball is accompanied by a digital signature available at: > > * https://openjdk-sources.osci.io/openjdk11/openjdk-11.0.8-ga.tar.xz.sig > > This is signed by our Red Hat OpenJDK key (openjdk at redhat.com): > > PGP Key: rsa4096/0x92EF8D39DC13168F (hkp://keys.gnupg.net) > Fingerprint = CA5F 11C6 CE22 644D 42C6 AC44 92EF 8D39 DC13 168F Corresponding binary release is here: https://adoptopenjdk.net/upstream.html?variant=openjdk11&ga=ga Thanks, Severin From matthias.baesken at sap.com Thu Jul 16 07:42:05 2020 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Thu, 16 Jul 2020 07:42:05 +0000 Subject: [11u] RFR: 8249277: TestVerifyIterativeGVN.java is failing with timeout in OpenJDK 11 In-Reply-To: References: Message-ID: Hi G?tz thanks for adjusting the timeout . Looks good to me ! Best regards, Matthias > -----Original Message----- > From: Lindenmaier, Goetz > Sent: Tuesday, July 14, 2020 10:22 AM > To: jdk-updates-dev at openjdk.java.net > Subject: [11u] RFR: 8249277: TestVerifyIterativeGVN.java is failing with > timeout in OpenJDK 11 > > Hi > > This test times out on mac. I would like to increase the timeout. > Please review: > http://cr.openjdk.java.net/~goetz/wr20/8249277- > TestVerifyIterativeGVN_timeout-jdk11/01/ > > Oracle did a similar change in the closed repo: > https://bugs.openjdk.java.net/browse/JDK-8248521 > but the change is not visible, so I crafted my own. > > New bug, open 11u only: > https://bugs.openjdk.java.net/browse/JDK-8248521 > > Best regards, > Goetz. From goetz.lindenmaier at sap.com Thu Jul 16 08:14:21 2020 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Thu, 16 Jul 2020 08:14:21 +0000 Subject: [11u] RFR: 8249159: Downport test rework for SSLSocketTemplate from 8224650 In-Reply-To: References: Message-ID: Hi Matthias, Thanks for the review. (I removed the rest of the mail, I guess the included cert blocks the mailing list. A previous mail did get stuck.) Best regards, Goetz. From: Baesken, Matthias Sent: Tuesday, July 14, 2020 9:32 AM To: Lindenmaier, Goetz ; 'jdk-updates-dev at openjdk.java.net' Subject: RE: Re: [11u] RFR: 8249159: Downport test rework for SSLSocketTemplate from 8224650 >I'd like to keep them, having them does no harm but simplifies >downports. What do you think? Hi Goetz, sounds good to me / lgtm ?? . Thanks for checking . Best regards, Matthias From felix.yang at huawei.com Thu Jul 16 09:01:48 2020 From: felix.yang at huawei.com (Yangfei (Felix)) Date: Thu, 16 Jul 2020 09:01:48 +0000 Subject: [aarch64-port-dev ] RFR(S): 8248851: CMS: Missing memory fences between free chunk check and klass read References: <8957BB9E-617D-4E24-9CC4-31AF7D01D12E@oracle.com> <65012FED-2A93-4899-939E-887175AB07AC@oracle.com> <06646cf3-bc21-977f-96e2-1e9a974d1b00@redhat.com> Message-ID: Ping ... Could someone please take another look? Especially the two webrevs for 11u & 13u. Thanks, Felix > -----Original Message----- > From: Yangfei (Felix) > Sent: Friday, July 10, 2020 7:44 PM > To: 'Andrew Haley' ; Kim Barrett > > Cc: jdk8u-dev at openjdk.java.net; hotspot-gc-dev at openjdk.java.net; > aarch64-port-dev at openjdk.java.net; jdk-updates-dev at openjdk.java.net > Subject: RE: [aarch64-port-dev ] RFR(S): 8248851: CMS: Missing memory > fences between free chunk check and klass read > > Hi, > > > -----Original Message----- > > From: Andrew Haley [mailto:aph at redhat.com] > > Sent: Wednesday, July 8, 2020 5:48 PM > > To: Kim Barrett ; Yangfei (Felix) > > > > Cc: jdk8u-dev at openjdk.java.net; hotspot-gc-dev at openjdk.java.net; > > aarch64-port-dev at openjdk.java.net > > Subject: Re: [aarch64-port-dev ] RFR(S): 8248851: CMS: Missing memory > > fences between free chunk check and klass read > > > > On 08/07/2020 09:41, Kim Barrett wrote: > > >> Since CMS is deprecated from JDK9, I am not sure if it's > > >> appropriate to fix > > this issue for those JDK9+ versions. > > > Deprecated != unsupported. > > > > > > > Yes, it must be done. Thanks. > > CCing to jdk-updates-dev. > > I have prepared another two webrevs for jdk11u and jdk13u: > Jdk11u-dev: http://cr.openjdk.java.net/~fyang/8248851-11u/ > Jdk13u-dev: http://cr.openjdk.java.net/~fyang/8248851-13u/ > > The only difference lies in copyright year update. Both tiere1-3 tested on > aarch64-linux-gnu. > I will put jdk11u-fix-request and jdk13u-fix-request label on the issue if they > are good. > > I have also prepared a new webrev for jdk8u incorporating copyright year > update: > Jdk8u-dev: http://cr.openjdk.java.net/~fyang/8248851-8u/ > > Thanks, > Felix > From david.holmes at oracle.com Thu Jul 16 09:16:04 2020 From: david.holmes at oracle.com (David Holmes) Date: Thu, 16 Jul 2020 19:16:04 +1000 Subject: [aarch64-port-dev ] RFR(S): 8248851: CMS: Missing memory fences between free chunk check and klass read In-Reply-To: References: <8957BB9E-617D-4E24-9CC4-31AF7D01D12E@oracle.com> <65012FED-2A93-4899-939E-887175AB07AC@oracle.com> <06646cf3-bc21-977f-96e2-1e9a974d1b00@redhat.com> Message-ID: <48d0abe9-29a1-a67f-b8f0-c823e15ce110@oracle.com> Hi Felix, Seems to me that you want OrderAccess::loadload() barriers to order the loads, not OrderAccess::acquire(). You should only use acquire semantics to pair with a corresponding release operation. Cheers, David On 16/07/2020 7:01 pm, Yangfei (Felix) wrote: > Ping ... > Could someone please take another look? Especially the two webrevs for 11u & 13u. > > Thanks, > Felix > >> -----Original Message----- >> From: Yangfei (Felix) >> Sent: Friday, July 10, 2020 7:44 PM >> To: 'Andrew Haley' ; Kim Barrett >> >> Cc: jdk8u-dev at openjdk.java.net; hotspot-gc-dev at openjdk.java.net; >> aarch64-port-dev at openjdk.java.net; jdk-updates-dev at openjdk.java.net >> Subject: RE: [aarch64-port-dev ] RFR(S): 8248851: CMS: Missing memory >> fences between free chunk check and klass read >> >> Hi, >> >>> -----Original Message----- >>> From: Andrew Haley [mailto:aph at redhat.com] >>> Sent: Wednesday, July 8, 2020 5:48 PM >>> To: Kim Barrett ; Yangfei (Felix) >>> >>> Cc: jdk8u-dev at openjdk.java.net; hotspot-gc-dev at openjdk.java.net; >>> aarch64-port-dev at openjdk.java.net >>> Subject: Re: [aarch64-port-dev ] RFR(S): 8248851: CMS: Missing memory >>> fences between free chunk check and klass read >>> >>> On 08/07/2020 09:41, Kim Barrett wrote: >>>>> Since CMS is deprecated from JDK9, I am not sure if it's >>>>> appropriate to fix >>> this issue for those JDK9+ versions. >>>> Deprecated != unsupported. >>>> >>> >>> Yes, it must be done. Thanks. >> >> CCing to jdk-updates-dev. >> >> I have prepared another two webrevs for jdk11u and jdk13u: >> Jdk11u-dev: http://cr.openjdk.java.net/~fyang/8248851-11u/ >> Jdk13u-dev: http://cr.openjdk.java.net/~fyang/8248851-13u/ >> >> The only difference lies in copyright year update. Both tiere1-3 tested on >> aarch64-linux-gnu. >> I will put jdk11u-fix-request and jdk13u-fix-request label on the issue if they >> are good. >> >> I have also prepared a new webrev for jdk8u incorporating copyright year >> update: >> Jdk8u-dev: http://cr.openjdk.java.net/~fyang/8248851-8u/ >> >> Thanks, >> Felix >> > From adinn at redhat.com Thu Jul 16 10:57:51 2020 From: adinn at redhat.com (Andrew Dinn) Date: Thu, 16 Jul 2020 11:57:51 +0100 Subject: [11u] RFR: 8226575: OperatingSystemMXBean should be made container aware In-Reply-To: <6f644db4cf5ebf8663b87eb7ba9496e581dddd0f.camel@redhat.com> References: <6f644db4cf5ebf8663b87eb7ba9496e581dddd0f.camel@redhat.com> Message-ID: <2fd5e940-f1ae-b0b2-964a-f455fc6bcb93@redhat.com> Hi Severin, On 15/07/2020 10:39, Severin Gehwolf wrote: > Could I please get a review of this container awareness backport for > OperatingSystemMXBean? I'd like to backport this since 8u will soon > receive a similar update and this would avoid regressions when > upgrading from JDK 8 to JDK 11 in the future. > > The JDK 14 patch doesn't exactly apply since for an 11u backport it's > not appropriate to add new methods in OperatingSystemMXBean. Thus, for > this backport the old methods now return container aware values on an > OS which supports them (i.e. Linux only). I'm going to propse a test > fix, JDK-8236617, as a follow-up. > > webrev: http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8226575/jdk11/02/webrev/ > Bug: https://bugs.openjdk.java.net/browse/JDK-8226575 > Corresponding CSR: https://bugs.openjdk.java.net/browse/JDK-8248804 (approved) > > Testing: tier 1, :jdk_management, docker tests on Linux x86_64. All pass. Yes, the backport looks fine to me. regards, Andrew Dinn ----------- Red Hat Distinguished Engineer Red Hat UK Ltd Registered in England and Wales under Company Registration No. 03798903 Directors: Michael Cunningham, Michael ("Mike") O'Neill From adinn at redhat.com Thu Jul 16 11:01:22 2020 From: adinn at redhat.com (Andrew Dinn) Date: Thu, 16 Jul 2020 12:01:22 +0100 Subject: [11u] RFR: 8236617: jtreg test containers/docker/TestMemoryAwareness.java fails after 8226575 In-Reply-To: <2db3f0712f66753d07ef764a5dff1a5d411b1f7b.camel@redhat.com> References: <2db3f0712f66753d07ef764a5dff1a5d411b1f7b.camel@redhat.com> Message-ID: <9d74fdc6-c79e-97af-a404-ee63f527bf7a@redhat.com> Hi Severin, On 15/07/2020 11:03, Severin Gehwolf wrote: > Please review this follow-up fix for JDK-8226575 (Container aware > OperatingSystemMXBean) which is pending review for a 11u backport. It > fixes a test issue seen on SAP's test cluster. The JDK 15 patch didn't > apply cleanly since the backport doesn't add new method names and > CheckOperatingSystemMXBean diagnostics don't print them. I've adjusted > it manually. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8236617 > webrev: http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8236617/jdk11/01/webrev/ > original change: https://hg.openjdk.java.net/jdk/jdk/rev/ab10165b4141 > > Testing: Docker tests on Linux x86_64. Passed. > > Once reviewed and approved I intend to push JDK-8226575 and > JDK-8236617 together. Yes, this also looks good. regards, Andrew Dinn ----------- Red Hat Distinguished Engineer Red Hat UK Ltd Registered in England and Wales under Company Registration No. 03798903 Directors: Michael Cunningham, Michael ("Mike") O'Neill From sgehwolf at redhat.com Thu Jul 16 11:36:45 2020 From: sgehwolf at redhat.com (Severin Gehwolf) Date: Thu, 16 Jul 2020 13:36:45 +0200 Subject: [11u] RFR: 8226575: OperatingSystemMXBean should be made container aware In-Reply-To: <2fd5e940-f1ae-b0b2-964a-f455fc6bcb93@redhat.com> References: <6f644db4cf5ebf8663b87eb7ba9496e581dddd0f.camel@redhat.com> <2fd5e940-f1ae-b0b2-964a-f455fc6bcb93@redhat.com> Message-ID: <08dbaeb8b90f1409fd1b5af7fd1e924dc1ea3472.camel@redhat.com> On Thu, 2020-07-16 at 11:57 +0100, Andrew Dinn wrote: > Hi Severin, > > On 15/07/2020 10:39, Severin Gehwolf wrote: > > Could I please get a review of this container awareness backport for > > OperatingSystemMXBean? I'd like to backport this since 8u will soon > > receive a similar update and this would avoid regressions when > > upgrading from JDK 8 to JDK 11 in the future. > > > > The JDK 14 patch doesn't exactly apply since for an 11u backport it's > > not appropriate to add new methods in OperatingSystemMXBean. Thus, for > > this backport the old methods now return container aware values on an > > OS which supports them (i.e. Linux only). I'm going to propse a test > > fix, JDK-8236617, as a follow-up. > > > > webrev: http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8226575/jdk11/02/webrev/ > > Bug: https://bugs.openjdk.java.net/browse/JDK-8226575 > > Corresponding CSR: https://bugs.openjdk.java.net/browse/JDK-8248804 (approved) > > > > Testing: tier 1, :jdk_management, docker tests on Linux x86_64. All pass. > Yes, the backport looks fine to me. Thanks for the review, Andrew! Cheers, Severin From sgehwolf at redhat.com Thu Jul 16 11:37:14 2020 From: sgehwolf at redhat.com (Severin Gehwolf) Date: Thu, 16 Jul 2020 13:37:14 +0200 Subject: [11u] RFR: 8236617: jtreg test containers/docker/TestMemoryAwareness.java fails after 8226575 In-Reply-To: <9d74fdc6-c79e-97af-a404-ee63f527bf7a@redhat.com> References: <2db3f0712f66753d07ef764a5dff1a5d411b1f7b.camel@redhat.com> <9d74fdc6-c79e-97af-a404-ee63f527bf7a@redhat.com> Message-ID: On Thu, 2020-07-16 at 12:01 +0100, Andrew Dinn wrote: > > Once reviewed and approved I intend to push JDK-8226575 and > > JDK-8236617 together. > > Yes, this also looks good. Thanks! Cheers, Severin From christian.hagedorn at oracle.com Thu Jul 16 13:30:57 2020 From: christian.hagedorn at oracle.com (Christian Hagedorn) Date: Thu, 16 Jul 2020 15:30:57 +0200 Subject: [11u] RFR: 8249277: TestVerifyIterativeGVN.java is failing with timeout in OpenJDK 11 In-Reply-To: References: Message-ID: Hi Goetz Looks good to me! Best regards, Christian On 16.07.20 09:42, Baesken, Matthias wrote: > Hi G?tz thanks for adjusting the timeout . Looks good to me ! > > Best regards, Matthias > > >> -----Original Message----- >> From: Lindenmaier, Goetz >> Sent: Tuesday, July 14, 2020 10:22 AM >> To: jdk-updates-dev at openjdk.java.net >> Subject: [11u] RFR: 8249277: TestVerifyIterativeGVN.java is failing with >> timeout in OpenJDK 11 >> >> Hi >> >> This test times out on mac. I would like to increase the timeout. >> Please review: >> http://cr.openjdk.java.net/~goetz/wr20/8249277- >> TestVerifyIterativeGVN_timeout-jdk11/01/ >> >> Oracle did a similar change in the closed repo: >> https://bugs.openjdk.java.net/browse/JDK-8248521 >> but the change is not visible, so I crafted my own. >> >> New bug, open 11u only: >> https://bugs.openjdk.java.net/browse/JDK-8248521 >> >> Best regards, >> Goetz. From aph at redhat.com Thu Jul 16 13:43:15 2020 From: aph at redhat.com (Andrew Haley) Date: Thu, 16 Jul 2020 14:43:15 +0100 Subject: [aarch64-port-dev ] RFR(S): 8248851: CMS: Missing memory fences between free chunk check and klass read In-Reply-To: <48d0abe9-29a1-a67f-b8f0-c823e15ce110@oracle.com> References: <8957BB9E-617D-4E24-9CC4-31AF7D01D12E@oracle.com> <65012FED-2A93-4899-939E-887175AB07AC@oracle.com> <06646cf3-bc21-977f-96e2-1e9a974d1b00@redhat.com> <48d0abe9-29a1-a67f-b8f0-c823e15ce110@oracle.com> Message-ID: <561611d4-18aa-f9d8-7302-3d89aff4ad81@redhat.com> On 16/07/2020 10:16, David Holmes wrote: > Seems to me that you want OrderAccess::loadload() barriers to order the > loads, not OrderAccess::acquire(). You should only use acquire semantics > to pair with a corresponding release operation. I agree, but it's unlikely to matter in practice. Having said that, I'm strongly of the opinion that if you see a naked StoreStore it may well be a bug, or at least you've got something very hard to analyse. I know of a few cases (e.g. zeroing an object) where this isn't true. https://www.hboehm.info/c++mm/no_write_fences.html, etc. But that's an argument for another day. -- Andrew Haley (he/him) Java Platform Lead Engineer Red Hat UK Ltd. https://keybase.io/andrewhaley EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From david.holmes at oracle.com Thu Jul 16 22:30:20 2020 From: david.holmes at oracle.com (David Holmes) Date: Fri, 17 Jul 2020 08:30:20 +1000 Subject: [aarch64-port-dev ] RFR(S): 8248851: CMS: Missing memory fences between free chunk check and klass read In-Reply-To: <561611d4-18aa-f9d8-7302-3d89aff4ad81@redhat.com> References: <8957BB9E-617D-4E24-9CC4-31AF7D01D12E@oracle.com> <65012FED-2A93-4899-939E-887175AB07AC@oracle.com> <06646cf3-bc21-977f-96e2-1e9a974d1b00@redhat.com> <48d0abe9-29a1-a67f-b8f0-c823e15ce110@oracle.com> <561611d4-18aa-f9d8-7302-3d89aff4ad81@redhat.com> Message-ID: On 16/07/2020 11:43 pm, Andrew Haley wrote: > On 16/07/2020 10:16, David Holmes wrote: >> Seems to me that you want OrderAccess::loadload() barriers to order the >> loads, not OrderAccess::acquire(). You should only use acquire semantics >> to pair with a corresponding release operation. > > I agree, but it's unlikely to matter in practice. In terms of what underlying hardware barriers get used, no it won't (likely) matter in practice. But from a code understandability perspective it matters very much IMHO. We have been actively trying to ensure that the right OrderAccess APIs are used, in the right way and only where actually needed. An acquire without a corresponding release shows a lack of understanding and leads to confusion for other developers. If you need ordered loads then use a loadload() barrier. If the loads need to be ordered you need to ensure there are not corresponding writes that also need to be ordered - which may show where release() is missing. > Having said that, I'm strongly of the opinion that if you see a naked > StoreStore it may well be a bug, or at least you've got something very > hard to analyse. I know of a few cases (e.g. zeroing an object) where > this isn't true. > > https://www.hboehm.info/c++mm/no_write_fences.html, etc. > > But that's an argument for another day. Indeed :) Cheers, David From felix.yang at huawei.com Fri Jul 17 03:05:03 2020 From: felix.yang at huawei.com (Yangfei (Felix)) Date: Fri, 17 Jul 2020 03:05:03 +0000 Subject: [aarch64-port-dev ] RFR(S): 8248851: CMS: Missing memory fences between free chunk check and klass read In-Reply-To: References: <8957BB9E-617D-4E24-9CC4-31AF7D01D12E@oracle.com> <65012FED-2A93-4899-939E-887175AB07AC@oracle.com> <06646cf3-bc21-977f-96e2-1e9a974d1b00@redhat.com> <48d0abe9-29a1-a67f-b8f0-c823e15ce110@oracle.com> <561611d4-18aa-f9d8-7302-3d89aff4ad81@redhat.com> Message-ID: Hi, Thanks for the suggestions. It makes sense to me. BTW: OrderAccess::loadload() and OrderAccess::acquire() both map to the same instruction for aarch64: dmb ishld. Updated webrev: http://cr.openjdk.java.net/~fyang/8248851-8u/webrev.01/ http://cr.openjdk.java.net/~fyang/8248851-11u/webrev.01/ http://cr.openjdk.java.net/~fyang/8248851-13u/webrev.01/ Performed the same test as before, result looks good. Does it look better? Felix > -----Original Message----- > From: David Holmes [mailto:david.holmes at oracle.com] > Sent: Friday, July 17, 2020 6:30 AM > To: Andrew Haley ; Yangfei (Felix) > ; Kim Barrett > Cc: jdk8u-dev at openjdk.java.net; hotspot-gc-dev at openjdk.java.net; > aarch64-port-dev at openjdk.java.net; jdk-updates-dev at openjdk.java.net > Subject: Re: [aarch64-port-dev ] RFR(S): 8248851: CMS: Missing memory > fences between free chunk check and klass read > > On 16/07/2020 11:43 pm, Andrew Haley wrote: > > On 16/07/2020 10:16, David Holmes wrote: > >> Seems to me that you want OrderAccess::loadload() barriers to order > >> the loads, not OrderAccess::acquire(). You should only use acquire > >> semantics to pair with a corresponding release operation. > > > > I agree, but it's unlikely to matter in practice. > > In terms of what underlying hardware barriers get used, no it won't > (likely) matter in practice. > > But from a code understandability perspective it matters very much IMHO. > We have been actively trying to ensure that the right OrderAccess APIs are > used, in the right way and only where actually needed. An acquire without a > corresponding release shows a lack of understanding and leads to confusion > for other developers. If you need ordered loads then use a > loadload() barrier. If the loads need to be ordered you need to ensure there > are not corresponding writes that also need to be ordered - which may show > where release() is missing. > > > Having said that, I'm strongly of the opinion that if you see a naked > > StoreStore it may well be a bug, or at least you've got something very > > hard to analyse. I know of a few cases (e.g. zeroing an object) where > > this isn't true. > > > > https://www.hboehm.info/c++mm/no_write_fences.html, etc. > > > > But that's an argument for another day. > > Indeed :) > > Cheers, > David From david.holmes at oracle.com Fri Jul 17 04:51:35 2020 From: david.holmes at oracle.com (David Holmes) Date: Fri, 17 Jul 2020 14:51:35 +1000 Subject: [aarch64-port-dev ] RFR(S): 8248851: CMS: Missing memory fences between free chunk check and klass read In-Reply-To: References: <8957BB9E-617D-4E24-9CC4-31AF7D01D12E@oracle.com> <65012FED-2A93-4899-939E-887175AB07AC@oracle.com> <06646cf3-bc21-977f-96e2-1e9a974d1b00@redhat.com> <48d0abe9-29a1-a67f-b8f0-c823e15ce110@oracle.com> <561611d4-18aa-f9d8-7302-3d89aff4ad81@redhat.com> Message-ID: Hi Felix, On 17/07/2020 1:05 pm, Yangfei (Felix) wrote: > Hi, > > Thanks for the suggestions. It makes sense to me. > BTW: OrderAccess::loadload() and OrderAccess::acquire() both map to the same instruction for aarch64: dmb ishld. > Updated webrev: > http://cr.openjdk.java.net/~fyang/8248851-8u/webrev.01/ > http://cr.openjdk.java.net/~fyang/8248851-11u/webrev.01/ > http://cr.openjdk.java.net/~fyang/8248851-13u/webrev.01/ > > Performed the same test as before, result looks good. > Does it look better? It certainly looks better to me. (This code still puzzles me somewhat but I'm not going to expend more time trying to understand it more broadly.) Thanks, David ----- > Felix > >> -----Original Message----- >> From: David Holmes [mailto:david.holmes at oracle.com] >> Sent: Friday, July 17, 2020 6:30 AM >> To: Andrew Haley ; Yangfei (Felix) >> ; Kim Barrett >> Cc: jdk8u-dev at openjdk.java.net; hotspot-gc-dev at openjdk.java.net; >> aarch64-port-dev at openjdk.java.net; jdk-updates-dev at openjdk.java.net >> Subject: Re: [aarch64-port-dev ] RFR(S): 8248851: CMS: Missing memory >> fences between free chunk check and klass read >> >> On 16/07/2020 11:43 pm, Andrew Haley wrote: >>> On 16/07/2020 10:16, David Holmes wrote: >>>> Seems to me that you want OrderAccess::loadload() barriers to order >>>> the loads, not OrderAccess::acquire(). You should only use acquire >>>> semantics to pair with a corresponding release operation. >>> >>> I agree, but it's unlikely to matter in practice. >> >> In terms of what underlying hardware barriers get used, no it won't >> (likely) matter in practice. >> >> But from a code understandability perspective it matters very much IMHO. >> We have been actively trying to ensure that the right OrderAccess APIs are >> used, in the right way and only where actually needed. An acquire without a >> corresponding release shows a lack of understanding and leads to confusion >> for other developers. If you need ordered loads then use a >> loadload() barrier. If the loads need to be ordered you need to ensure there >> are not corresponding writes that also need to be ordered - which may show >> where release() is missing. >> >>> Having said that, I'm strongly of the opinion that if you see a naked >>> StoreStore it may well be a bug, or at least you've got something very >>> hard to analyse. I know of a few cases (e.g. zeroing an object) where >>> this isn't true. >>> >>> https://www.hboehm.info/c++mm/no_write_fences.html, etc. >>> >>> But that's an argument for another day. >> >> Indeed :) >> >> Cheers, >> David > From christoph.langer at sap.com Fri Jul 17 08:36:32 2020 From: christoph.langer at sap.com (Langer, Christoph) Date: Fri, 17 Jul 2020 08:36:32 +0000 Subject: [11u] RFR: 8249277: TestVerifyIterativeGVN.java is failing with timeout in OpenJDK 11 In-Reply-To: References: Message-ID: Hi, the change looks good to me, too. I think, however, it should be submitted as backport of https://bugs.openjdk.java.net/browse/JDK-8248521. While it is not a literal backport of the fix for JDK-8248521, since that one is not available obviously (as it was done on Oracle's JDK11 release), it though aims to fix the reported problem. Having the fix as JBS backport item of JDK-8249277 will make it more obvious that the issue was addressed in OpenJDK, too. @Severin, aph or others, do you have an opinion here? Best regards Christoph > -----Original Message----- > From: jdk-updates-dev On > Behalf Of Christian Hagedorn > Sent: Donnerstag, 16. Juli 2020 15:31 > To: Baesken, Matthias ; Lindenmaier, Goetz > ; 'jdk-updates-dev at openjdk.java.net' updates-dev at openjdk.java.net> > Subject: Re: [11u] RFR: 8249277: TestVerifyIterativeGVN.java is failing with > timeout in OpenJDK 11 > > Hi Goetz > > Looks good to me! > > Best regards, > Christian > > On 16.07.20 09:42, Baesken, Matthias wrote: > > Hi G?tz thanks for adjusting the timeout . Looks good to me ! > > > > Best regards, Matthias > > > > > >> -----Original Message----- > >> From: Lindenmaier, Goetz > >> Sent: Tuesday, July 14, 2020 10:22 AM > >> To: jdk-updates-dev at openjdk.java.net > >> Subject: [11u] RFR: 8249277: TestVerifyIterativeGVN.java is failing with > >> timeout in OpenJDK 11 > >> > >> Hi > >> > >> This test times out on mac. I would like to increase the timeout. > >> Please review: > >> http://cr.openjdk.java.net/~goetz/wr20/8249277- > >> TestVerifyIterativeGVN_timeout-jdk11/01/ > >> > >> Oracle did a similar change in the closed repo: > >> https://bugs.openjdk.java.net/browse/JDK-8248521 > >> but the change is not visible, so I crafted my own. > >> > >> New bug, open 11u only: > >> https://bugs.openjdk.java.net/browse/JDK-8248521 > >> > >> Best regards, > >> Goetz. From felix.yang at huawei.com Fri Jul 17 08:40:58 2020 From: felix.yang at huawei.com (Yangfei (Felix)) Date: Fri, 17 Jul 2020 08:40:58 +0000 Subject: [aarch64-port-dev ] RFR(S): 8248851: CMS: Missing memory fences between free chunk check and klass read In-Reply-To: References: <8957BB9E-617D-4E24-9CC4-31AF7D01D12E@oracle.com> <65012FED-2A93-4899-939E-887175AB07AC@oracle.com> <06646cf3-bc21-977f-96e2-1e9a974d1b00@redhat.com> <48d0abe9-29a1-a67f-b8f0-c823e15ce110@oracle.com> <561611d4-18aa-f9d8-7302-3d89aff4ad81@redhat.com> Message-ID: Hi, Thanks for reviewing this and the great comments. I have put fix-request labels and accompanying comments on this issue for these three repos. Felix > -----Original Message----- > From: David Holmes [mailto:david.holmes at oracle.com] > Sent: Friday, July 17, 2020 12:52 PM > To: Yangfei (Felix) ; Andrew Haley > ; Kim Barrett > Cc: jdk8u-dev at openjdk.java.net; hotspot-gc-dev at openjdk.java.net; > aarch64-port-dev at openjdk.java.net; jdk-updates-dev at openjdk.java.net > Subject: Re: [aarch64-port-dev ] RFR(S): 8248851: CMS: Missing memory > fences between free chunk check and klass read > > Hi Felix, > > On 17/07/2020 1:05 pm, Yangfei (Felix) wrote: > > Hi, > > > > Thanks for the suggestions. It makes sense to me. > > BTW: OrderAccess::loadload() and OrderAccess::acquire() both map to > the same instruction for aarch64: dmb ishld. > > Updated webrev: > > http://cr.openjdk.java.net/~fyang/8248851-8u/webrev.01/ > > http://cr.openjdk.java.net/~fyang/8248851-11u/webrev.01/ > > http://cr.openjdk.java.net/~fyang/8248851-13u/webrev.01/ > > > > Performed the same test as before, result looks good. > > Does it look better? > > It certainly looks better to me. (This code still puzzles me somewhat but I'm > not going to expend more time trying to understand it more broadly.) > > Thanks, > David > ----- > > > Felix > > > >> -----Original Message----- > >> From: David Holmes [mailto:david.holmes at oracle.com] > >> Sent: Friday, July 17, 2020 6:30 AM > >> To: Andrew Haley ; Yangfei (Felix) > >> ; Kim Barrett > >> Cc: jdk8u-dev at openjdk.java.net; hotspot-gc-dev at openjdk.java.net; > >> aarch64-port-dev at openjdk.java.net; jdk-updates-dev at openjdk.java.net > >> Subject: Re: [aarch64-port-dev ] RFR(S): 8248851: CMS: Missing memory > >> fences between free chunk check and klass read > >> > >> On 16/07/2020 11:43 pm, Andrew Haley wrote: > >>> On 16/07/2020 10:16, David Holmes wrote: > >>>> Seems to me that you want OrderAccess::loadload() barriers to order > >>>> the loads, not OrderAccess::acquire(). You should only use acquire > >>>> semantics to pair with a corresponding release operation. > >>> > >>> I agree, but it's unlikely to matter in practice. > >> > >> In terms of what underlying hardware barriers get used, no it won't > >> (likely) matter in practice. > >> > >> But from a code understandability perspective it matters very much IMHO. > >> We have been actively trying to ensure that the right OrderAccess > >> APIs are used, in the right way and only where actually needed. An > >> acquire without a corresponding release shows a lack of understanding > >> and leads to confusion for other developers. If you need ordered > >> loads then use a > >> loadload() barrier. If the loads need to be ordered you need to > >> ensure there are not corresponding writes that also need to be > >> ordered - which may show where release() is missing. > >> > >>> Having said that, I'm strongly of the opinion that if you see a > >>> naked StoreStore it may well be a bug, or at least you've got > >>> something very hard to analyse. I know of a few cases (e.g. zeroing > >>> an object) where this isn't true. > >>> > >>> https://www.hboehm.info/c++mm/no_write_fences.html, etc. > >>> > >>> But that's an argument for another day. > >> > >> Indeed :) > >> > >> Cheers, > >> David > > From sgehwolf at redhat.com Fri Jul 17 09:05:36 2020 From: sgehwolf at redhat.com (Severin Gehwolf) Date: Fri, 17 Jul 2020 11:05:36 +0200 Subject: [11u] RFR: 8249277: TestVerifyIterativeGVN.java is failing with timeout in OpenJDK 11 In-Reply-To: References: Message-ID: Hi, On Fri, 2020-07-17 at 08:36 +0000, Langer, Christoph wrote: > Hi, > > the change looks good to me, too. > > I think, however, it should be submitted as backport of https://bugs.openjdk.java.net/browse/JDK-8248521. While it is not a literal backport of the fix for JDK-8248521, since that one is not available obviously (as it was done on Oracle's JDK11 release), it though aims to fix the reported problem. Having the fix as JBS backport item of JDK-8249277 will make it more obvious that the issue was addressed in OpenJDK, too. > > @Severin, aph or others, do you have an opinion here? Good question. I'd be fine either way. A slight preference being to use JDK-8248521 when pushing as it will likely help the filters to track parity work. If you go down the route of using JDK-8249277, please do mention JDK-8248521 via "Summary" in the commit message so it shows up when 'log -k'-ing for this change. Thanks, Severin > > > -----Original Message----- > > From: jdk-updates-dev On > > Behalf Of Christian Hagedorn > > Sent: Donnerstag, 16. Juli 2020 15:31 > > To: Baesken, Matthias ; Lindenmaier, Goetz > > ; 'jdk-updates-dev at openjdk.java.net' > updates-dev at openjdk.java.net> > > Subject: Re: [11u] RFR: 8249277: TestVerifyIterativeGVN.java is failing with > > timeout in OpenJDK 11 > > > > Hi Goetz > > > > Looks good to me! > > > > Best regards, > > Christian > > > > On 16.07.20 09:42, Baesken, Matthias wrote: > > > Hi G?tz thanks for adjusting the timeout . Looks good to me ! > > > > > > Best regards, Matthias > > > > > > > > > > -----Original Message----- > > > > From: Lindenmaier, Goetz > > > > Sent: Tuesday, July 14, 2020 10:22 AM > > > > To: jdk-updates-dev at openjdk.java.net > > > > Subject: [11u] RFR: 8249277: TestVerifyIterativeGVN.java is failing with > > > > timeout in OpenJDK 11 > > > > > > > > Hi > > > > > > > > This test times out on mac. I would like to increase the timeout. > > > > Please review: > > > > http://cr.openjdk.java.net/~goetz/wr20/8249277- > > > > TestVerifyIterativeGVN_timeout-jdk11/01/ > > > > > > > > Oracle did a similar change in the closed repo: > > > > https://bugs.openjdk.java.net/browse/JDK-8248521 > > > > but the change is not visible, so I crafted my own. > > > > > > > > New bug, open 11u only: > > > > https://bugs.openjdk.java.net/browse/JDK-8248521 > > > > > > > > Best regards, > > > > Goetz. From goetz.lindenmaier at sap.com Fri Jul 17 09:18:47 2020 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Fri, 17 Jul 2020 09:18:47 +0000 Subject: [11u] RFR: 8249277: TestVerifyIterativeGVN.java is failing with timeout in OpenJDK 11 In-Reply-To: References: Message-ID: Hi, I had thought how to handle this, but decided for a Bug/Enhancement, not a Downport. Looking at the bugs in JBS, a downport makes it look good, as it expresses that we have fixed the same issue. But: It also expresses that we have fixed the same issue the same way. This we don't know, as we don't see the original fix. A downport uses the BugID of the downported issue. This somehow expresses it's the very same patch modulo needed adaptions. Should I also push my patch with the BugID of the 11.0.9-oracle issue? Or should I push it with the BugID of the bug I opened (I can transform it into a downport issue). Reviewers: for a downported change the original reviews are available and valid. Here, it is not the case. Assignee: Who should be assignee in the JBS issue? Usually it?s the author of the original issue for downports. . But that would not be correct. If I add me, it looks as if I violated the rules... So I'm not sure... (All taking aside that this change is trivial, and Christian H. reviewed it) Best regards, Goetz. > -----Original Message----- > From: Langer, Christoph > Sent: Friday, July 17, 2020 10:37 AM > To: Lindenmaier, Goetz ; 'jdk-updates- > dev at openjdk.java.net' > Cc: Severin Gehwolf ; Andrew Haley > > Subject: RE: [11u] RFR: 8249277: TestVerifyIterativeGVN.java is failing with > timeout in OpenJDK 11 > > Hi, > > the change looks good to me, too. > > I think, however, it should be submitted as backport of > https://bugs.openjdk.java.net/browse/JDK-8248521. While it is not a literal > backport of the fix for JDK-8248521, since that one is not available obviously > (as it was done on Oracle's JDK11 release), it though aims to fix the reported > problem. Having the fix as JBS backport item of JDK-8249277 will make it > more obvious that the issue was addressed in OpenJDK, too. > > @Severin, aph or others, do you have an opinion here? > > Best regards > Christoph > > > -----Original Message----- > > From: jdk-updates-dev On > > Behalf Of Christian Hagedorn > > Sent: Donnerstag, 16. Juli 2020 15:31 > > To: Baesken, Matthias ; Lindenmaier, Goetz > > ; 'jdk-updates-dev at openjdk.java.net' > updates-dev at openjdk.java.net> > > Subject: Re: [11u] RFR: 8249277: TestVerifyIterativeGVN.java is failing with > > timeout in OpenJDK 11 > > > > Hi Goetz > > > > Looks good to me! > > > > Best regards, > > Christian > > > > On 16.07.20 09:42, Baesken, Matthias wrote: > > > Hi G?tz thanks for adjusting the timeout . Looks good to me ! > > > > > > Best regards, Matthias > > > > > > > > >> -----Original Message----- > > >> From: Lindenmaier, Goetz > > >> Sent: Tuesday, July 14, 2020 10:22 AM > > >> To: jdk-updates-dev at openjdk.java.net > > >> Subject: [11u] RFR: 8249277: TestVerifyIterativeGVN.java is failing with > > >> timeout in OpenJDK 11 > > >> > > >> Hi > > >> > > >> This test times out on mac. I would like to increase the timeout. > > >> Please review: > > >> http://cr.openjdk.java.net/~goetz/wr20/8249277- > > >> TestVerifyIterativeGVN_timeout-jdk11/01/ > > >> > > >> Oracle did a similar change in the closed repo: > > >> https://bugs.openjdk.java.net/browse/JDK-8248521 > > >> but the change is not visible, so I crafted my own. > > >> > > >> New bug, open 11u only: > > >> https://bugs.openjdk.java.net/browse/JDK-8248521 > > >> > > >> Best regards, > > >> Goetz. From sgehwolf at redhat.com Fri Jul 17 09:35:43 2020 From: sgehwolf at redhat.com (Severin Gehwolf) Date: Fri, 17 Jul 2020 11:35:43 +0200 Subject: [11u] RFR: 8249277: TestVerifyIterativeGVN.java is failing with timeout in OpenJDK 11 In-Reply-To: References: Message-ID: <7f4b4d04d8fa867b6cb674a7d85dbfa0ec7ceaba.camel@redhat.com> On Fri, 2020-07-17 at 09:18 +0000, Lindenmaier, Goetz wrote: > > (All taking aside that this change is trivial, and Christian H. > reviewed it) Yes, agreed. Because of that I suggest to to use the least-painful way for you as the backport dev. Use your good judgement. Thanks, Severin From goetz.lindenmaier at sap.com Fri Jul 17 09:36:11 2020 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Fri, 17 Jul 2020 09:36:11 +0000 Subject: [11u] RFR: 8249277: TestVerifyIterativeGVN.java is failing with timeout in OpenJDK 11 In-Reply-To: References: Message-ID: Hi Severin > If you go down the route of using JDK-8249277, please do > mention JDK-8248521 via "Summary" in the commit message so it shows up > when 'log -k'-ing for this change. Like this? http://cr.openjdk.java.net/~goetz/wr20/8249277-TestVerifyIterativeGVN_timeout-jdk11/02/ Best regards, Goetz. > -----Original Message----- > From: Severin Gehwolf > Sent: Friday, July 17, 2020 11:06 AM > To: Langer, Christoph ; Lindenmaier, Goetz > ; 'jdk-updates-dev at openjdk.java.net' updates-dev at openjdk.java.net> > Cc: Andrew Haley > Subject: Re: [11u] RFR: 8249277: TestVerifyIterativeGVN.java is failing with > timeout in OpenJDK 11 > > Hi, > > On Fri, 2020-07-17 at 08:36 +0000, Langer, Christoph wrote: > > Hi, > > > > the change looks good to me, too. > > > > I think, however, it should be submitted as backport of > https://bugs.openjdk.java.net/browse/JDK-8248521. While it is not a literal > backport of the fix for JDK-8248521, since that one is not available obviously > (as it was done on Oracle's JDK11 release), it though aims to fix the reported > problem. Having the fix as JBS backport item of JDK-8249277 will make it > more obvious that the issue was addressed in OpenJDK, too. > > > > @Severin, aph or others, do you have an opinion here? > > Good question. I'd be fine either way. A slight preference being to use > JDK-8248521 when pushing as it will likely help the filters to track > parity work. If you go down the route of using JDK-8249277, please do > mention JDK-8248521 via "Summary" in the commit message so it shows up > when 'log -k'-ing for this change. > > Thanks, > Severin > > > > > -----Original Message----- > > > From: jdk-updates-dev On > > > Behalf Of Christian Hagedorn > > > Sent: Donnerstag, 16. Juli 2020 15:31 > > > To: Baesken, Matthias ; Lindenmaier, > Goetz > > > ; 'jdk-updates-dev at openjdk.java.net' > > > updates-dev at openjdk.java.net> > > > Subject: Re: [11u] RFR: 8249277: TestVerifyIterativeGVN.java is failing > with > > > timeout in OpenJDK 11 > > > > > > Hi Goetz > > > > > > Looks good to me! > > > > > > Best regards, > > > Christian > > > > > > On 16.07.20 09:42, Baesken, Matthias wrote: > > > > Hi G?tz thanks for adjusting the timeout . Looks good to me ! > > > > > > > > Best regards, Matthias > > > > > > > > > > > > > -----Original Message----- > > > > > From: Lindenmaier, Goetz > > > > > Sent: Tuesday, July 14, 2020 10:22 AM > > > > > To: jdk-updates-dev at openjdk.java.net > > > > > Subject: [11u] RFR: 8249277: TestVerifyIterativeGVN.java is failing > with > > > > > timeout in OpenJDK 11 > > > > > > > > > > Hi > > > > > > > > > > This test times out on mac. I would like to increase the timeout. > > > > > Please review: > > > > > http://cr.openjdk.java.net/~goetz/wr20/8249277- > > > > > TestVerifyIterativeGVN_timeout-jdk11/01/ > > > > > > > > > > Oracle did a similar change in the closed repo: > > > > > https://bugs.openjdk.java.net/browse/JDK-8248521 > > > > > but the change is not visible, so I crafted my own. > > > > > > > > > > New bug, open 11u only: > > > > > https://bugs.openjdk.java.net/browse/JDK-8248521 > > > > > > > > > > Best regards, > > > > > Goetz. From sgehwolf at redhat.com Fri Jul 17 09:53:00 2020 From: sgehwolf at redhat.com (Severin Gehwolf) Date: Fri, 17 Jul 2020 11:53:00 +0200 Subject: [11u] RFR: 8249277: TestVerifyIterativeGVN.java is failing with timeout in OpenJDK 11 In-Reply-To: References: Message-ID: On Fri, 2020-07-17 at 09:36 +0000, Lindenmaier, Goetz wrote: > > If you go down the route of using JDK-8249277, please do > > mention JDK-8248521 via "Summary" in the commit message so it shows up > > when 'log -k'-ing for this change. > Like this? > http://cr.openjdk.java.net/~goetz/wr20/8249277-TestVerifyIterativeGVN_timeout-jdk11/02/ Yes. Ship it! Thanks, Severin From christoph.langer at sap.com Fri Jul 17 10:57:30 2020 From: christoph.langer at sap.com (Langer, Christoph) Date: Fri, 17 Jul 2020 10:57:30 +0000 Subject: [11u] RFR: 8249277: TestVerifyIterativeGVN.java is failing with timeout in OpenJDK 11 In-Reply-To: References: Message-ID: Hi, you could also add both messages to the commit like this: 8249277: TestVerifyIterativeGVN.java is failing with timeout in OpenJDK 11 8248521: TestVerifyIterativeGVN.java is failing with timeout That way, you'd have both, JDK-8249277 as a bug on its own and a generated backport bug for JDK-8248521. But, after all, changing JDK-8249277 into a backport bug would be my preferred choice. As for your questions regarding author/reviewer/assignee, I don't see anything wrong with Author: goetz Reviewer: chagedorn, mbaesken The assignee of the backport doesn't matter, but it should probably be you. Well, as Severin said, do whatever you feel best with, I've approved JDK-8249277 ?? Best regards Christoph > -----Original Message----- > From: Severin Gehwolf > Sent: Freitag, 17. Juli 2020 11:53 > To: Lindenmaier, Goetz ; Langer, Christoph > ; 'jdk-updates-dev at openjdk.java.net' updates-dev at openjdk.java.net> > Cc: Andrew Haley > Subject: Re: [11u] RFR: 8249277: TestVerifyIterativeGVN.java is failing with > timeout in OpenJDK 11 > > On Fri, 2020-07-17 at 09:36 +0000, Lindenmaier, Goetz wrote: > > > If you go down the route of using JDK-8249277, please do > > > mention JDK-8248521 via "Summary" in the commit message so it shows > up > > > when 'log -k'-ing for this change. > > Like this? > > http://cr.openjdk.java.net/~goetz/wr20/8249277- > TestVerifyIterativeGVN_timeout-jdk11/02/ > > Yes. Ship it! > > Thanks, > Severin From matthias.baesken at sap.com Fri Jul 17 14:23:31 2020 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Fri, 17 Jul 2020 14:23:31 +0000 Subject: [11u] RFR: 8243029: Rewrite javax/net/ssl/compatibility/Compatibility.java with a flexible interop test framework Message-ID: Hi Goetz, this looks good to me , However some remarks . I would like to downport this for parity with 11.0.9-oracle. This change caused me some headache: First, there were some files to resolve: Cert.java could not be deleted because it differs in certificate EC_RSA_PRIME256V1_SHA256 Compatibility.java could not be deleted because it differs in test compile instructions. The test in 11 forces Java 1.6 compatibility, while the test in the patch forces Java 1.7 compatibility. UseCase.java could not be deleted because it differs in one line in Object[][] PARAMS: jdk11u-dev: FULL_CASES ? CIPHER_SUITES : MANDATORY_CIPHER_SUITES, original change: FULL_CASES || FULL_CIPHER_SUITES ? CIPHER_SUITES : MANDATORY_CIPHER_SUITES, test/lib/jdk/test/lib/security/CertUtils.java patched fine after downporting "8228967: Trust/Key store and SSL context utilities for tests" as prerequisite. The test descriptions before required -source/target 1.6 and 1.7. I thought about keeping it that way, but that is not possible as the new code uses Java 8 features as Lambdas and Predicates. So the tests now require 1.8. HrrTest.java still would not work. I removed the test case for TLS_CHACHA20_POLY1305_SHA256 because it only came in 12 with "8140466: ChaCha20 and Poly1305 TLS Cipher Suites" -------------------------------------------------------------- Seems jdk11 does only know the name of TLS_CHACHA20_POLY1305_SHA256 but nothing more , the real support came with jdk12. Some other folks were struggeling with this too : https://jira.mariadb.org/browse/CONJ-721 Maybe there is a chance to backport it to jdk11 in future (of course it is a separate issue) ? README The readme of jdk11 mentions jdk 6 to 10. The source patched originally by the change mentioned jdk 7 to 10. Because the new test files contain Java 8 coding, I adapted the readme to mention 8 as oldest version. I ran the test as normal jtreg tests (testing with the VM built from the current repo). You need to pass -m to jtreg as the tests are marked as manual tests. This works fine. I also tried to run the test as described in the readme. This did not work. Among others, I get errors that certain enums naming Suites are not known. I tried to run the tests in jdk/jdk, there they do not compile. So I gave up on this. The readme anyways is inprecise and/or wrong, it mentions the wrong test file to call, and omits that '-m' must be set. So please review: http://cr.openjdk.java.net/~goetz/wr20/8243029-ssl_test_framework-jdk11/01/ Bug: https://bugs.openjdk.java.net/browse/JDK-8243029 https://hg.openjdk.java.net/jdk/jdk/rev/b6b4506a7827 -------------------------------------------------------------------------- Could you file a separate bug for the readme, I guess this needs fixing in jdk/jdk too . Best regards, Matthias From goetz.lindenmaier at sap.com Sun Jul 19 10:04:52 2020 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Sun, 19 Jul 2020 10:04:52 +0000 Subject: [11u] RFR: 8248348: Regression caused by the update to BCEL 6.0 Message-ID: Hi, I downport this for parity with 11.0.9-oracle. Please review: http://cr.openjdk.java.net/~goetz/wr20/8248348-BCEL_6_regression-jdk11/01/ I had to resolve the LastModified tags in the Javadoc comment. https://bugs.openjdk.java.net/browse/JDK-8248348 https://hg.openjdk.java.net/jdk/jdk/rev/458fa7cc82b3 Best regards, Goetz. From goetz.lindenmaier at sap.com Sun Jul 19 10:08:56 2020 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Sun, 19 Jul 2020 10:08:56 +0000 Subject: [11u] RFR: 8193367: Annotated type variable bounds crash javac Message-ID: Hi I am downporting this for 11.0.9-oracle parity. I had to resolve 3 files. 2 are copyrights. In LambdaToMethod.java, 11 compares with INTERSECTION, where the original calls a method. The selection of the bounds field is similar though, and was easy to adapt. http://cr.openjdk.java.net/~goetz/wr20/8193367-ann_type_var-jdk11/01/ Please review. https://bugs.openjdk.java.net/browse/JDK-8193367 http://hg.openjdk.java.net/jdk/jdk/rev/a772e65727c5 Best regards, Goetz. From matthias.baesken at sap.com Mon Jul 20 06:39:40 2020 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Mon, 20 Jul 2020 06:39:40 +0000 Subject: [11u] RFR: 8248348: Regression caused by the update to BCEL 6.0 Message-ID: Hi Goetz, looks good , thanks for backporting. Best regards, Matthias >I downport this for parity with 11.0.9-oracle. Please review: >http://cr.openjdk.java.net/~goetz/wr20/8248348-BCEL_6_regression-jdk11/01/ > >I had to resolve the LastModified tags in the Javadoc comment. > >https://bugs.openjdk.java.net/browse/JDK-8248348 >https://hg.openjdk.java.net/jdk/jdk/rev/458fa7cc82b3 > >Best regards, > Goetz. From thomas.stuefe at gmail.com Mon Jul 20 08:01:28 2020 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Mon, 20 Jul 2020 10:01:28 +0200 Subject: [11u] RFR: 8193367: Annotated type variable bounds crash javac In-Reply-To: References: Message-ID: Hi Goetz, LambdaToMethod.java: Your version: 2358 boolean interfaceParameterIsIntersectionOrUnionType() { 2359 List tl = tree.getDescriptorType(types).getParameterTypes(); 2360 for (; tl.nonEmpty(); tl = tl.tail) { 2361 Type pt = tl.head; 2362 switch (pt.getKind()) { 2363 case INTERSECTION: 2364 case UNION: 2365 return true; 2366 case TYPEVAR: 2367 TypeVar tv = (TypeVar) pt; 2368 if (tv.getUpperBound().getKind() == TypeKind.INTERSECTION) { 2369 return true; 2370 } 2371 } 2372 } 2373 return false; 2374 } does not seem equivalent to the original version: boolean isIntersectionOrUnionType(Type t) { switch (t.getKind()) { case INTERSECTION: case UNION: return true; case TYPEVAR: TypeVar tv = (TypeVar) t; return isIntersectionOrUnionType(tv.getUpperBound()); } return false; } for cases where tv.getUpperBound().kind == UNION. Yours returns false, original returns true. Best Regards, Thomas On Sun, Jul 19, 2020 at 12:27 PM Lindenmaier, Goetz < goetz.lindenmaier at sap.com> wrote: > Hi > > I am downporting this for 11.0.9-oracle parity. > I had to resolve 3 files. 2 are copyrights. > In LambdaToMethod.java, 11 compares with INTERSECTION, > where the original calls a method. The selection of the bounds field > is similar though, and was easy to adapt. > > http://cr.openjdk.java.net/~goetz/wr20/8193367-ann_type_var-jdk11/01/ > > Please review. > > https://bugs.openjdk.java.net/browse/JDK-8193367 > http://hg.openjdk.java.net/jdk/jdk/rev/a772e65727c5 > > Best regards, > Goetz. > From thomas.stuefe at gmail.com Mon Jul 20 08:02:55 2020 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Mon, 20 Jul 2020 10:02:55 +0200 Subject: [11u] RFR: 8248348: Regression caused by the update to BCEL 6.0 In-Reply-To: References: Message-ID: LGTM Cheers, Thomas On Sun, Jul 19, 2020 at 12:05 PM Lindenmaier, Goetz < goetz.lindenmaier at sap.com> wrote: > Hi, > > I downport this for parity with 11.0.9-oracle. Please review: > http://cr.openjdk.java.net/~goetz/wr20/8248348-BCEL_6_regression-jdk11/01/ > > I had to resolve the LastModified tags in the Javadoc comment. > > https://bugs.openjdk.java.net/browse/JDK-8248348 > https://hg.openjdk.java.net/jdk/jdk/rev/458fa7cc82b3 > > Best regards, > Goetz. > From goetz.lindenmaier at sap.com Mon Jul 20 08:07:40 2020 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Mon, 20 Jul 2020 08:07:40 +0000 Subject: [11u] RFR: 8193367: Annotated type variable bounds crash javac In-Reply-To: References: Message-ID: Hi Thomas, Yes, that is true. But this change is about retrieving the bounds. The difference you point out is not addressed by the change downported. I would say the downport is correct because I do preserve the old behavior wrt. the kind. Best regards, Goetz. From: Thomas St?fe Sent: Monday, July 20, 2020 10:01 AM To: Lindenmaier, Goetz Cc: jdk-updates-dev at openjdk.java.net Subject: Re: [11u] RFR: 8193367: Annotated type variable bounds crash javac Hi Goetz, LambdaToMethod.java: Your version: 2358 boolean interfaceParameterIsIntersectionOrUnionType() { 2359 List tl = tree.getDescriptorType(types).getParameterTypes(); 2360 for (; tl.nonEmpty(); tl = tl.tail) { 2361 Type pt = tl.head; 2362 switch (pt.getKind()) { 2363 case INTERSECTION: 2364 case UNION: 2365 return true; 2366 case TYPEVAR: 2367 TypeVar tv = (TypeVar) pt; 2368 if (tv.getUpperBound().getKind() == TypeKind.INTERSECTION) { 2369 return true; 2370 } 2371 } 2372 } 2373 return false; 2374 } does not seem equivalent to the original version: boolean isIntersectionOrUnionType(Type t) { switch (t.getKind()) { case INTERSECTION: case UNION: return true; case TYPEVAR: TypeVar tv = (TypeVar) t; return isIntersectionOrUnionType(tv.getUpperBound()); } return false; } for cases where tv.getUpperBound().kind == UNION. Yours returns false, original returns true. Best Regards, Thomas On Sun, Jul 19, 2020 at 12:27 PM Lindenmaier, Goetz > wrote: Hi I am downporting this for 11.0.9-oracle parity. I had to resolve 3 files. 2 are copyrights. In LambdaToMethod.java, 11 compares with INTERSECTION, where the original calls a method. The selection of the bounds field is similar though, and was easy to adapt. http://cr.openjdk.java.net/~goetz/wr20/8193367-ann_type_var-jdk11/01/ Please review. https://bugs.openjdk.java.net/browse/JDK-8193367 http://hg.openjdk.java.net/jdk/jdk/rev/a772e65727c5 Best regards, Goetz. From thomas.stuefe at gmail.com Mon Jul 20 08:34:36 2020 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Mon, 20 Jul 2020 10:34:36 +0200 Subject: [11u] RFR: 8193367: Annotated type variable bounds crash javac In-Reply-To: References: Message-ID: Hi Goetz, true. The behaviour was changed with https://bugs.openjdk.java.net/browse/JDK-8213703 (JDK-8213703 "LambdaConversionException: Invalid receiver type not a subtype of implementation type interface") Matthias as a backport request open for that one, maybe wait with this downport until Matthias is done? Cheers, Thomas On Mon, Jul 20, 2020 at 10:07 AM Lindenmaier, Goetz < goetz.lindenmaier at sap.com> wrote: > Hi Thomas, > > > > Yes, that is true. > > But this change is about retrieving the bounds. > > The difference you point out is not addressed by the > > change downported. I would say the downport is > > correct because I do preserve the old behavior wrt. > > the kind. > > > > Best regards, > > Goetz. > > > > > > > > > > *From:* Thomas St?fe > *Sent:* Monday, July 20, 2020 10:01 AM > *To:* Lindenmaier, Goetz > *Cc:* jdk-updates-dev at openjdk.java.net > *Subject:* Re: [11u] RFR: 8193367: Annotated type variable bounds crash > javac > > > > Hi Goetz, > > > > LambdaToMethod.java: > > > > Your version: > > 2358 boolean interfaceParameterIsIntersectionOrUnionType() { > > 2359 List tl = tree.getDescriptorType(types).getParameterTypes(); > > 2360 for (; tl.nonEmpty(); tl = tl.tail) { > > 2361 Type pt = tl.head; > > 2362 switch (pt.getKind()) { > > 2363 case INTERSECTION: > > 2364 case UNION: > > 2365 return true; > > 2366 case TYPEVAR: > > 2367 TypeVar tv = (TypeVar) pt; > > 2368 if (tv.getUpperBound().getKind() == TypeKind.INTERSECTION) { > > 2369 return true; > > 2370 } > > 2371 } > > 2372 } > > 2373 return false; > > 2374 } > > > > does not seem equivalent to the original version: > > > > boolean isIntersectionOrUnionType(Type t) { > switch (t.getKind()) { > case INTERSECTION: > case UNION: > return true; > case TYPEVAR: > TypeVar tv = (TypeVar) t; > return isIntersectionOrUnionType(tv.getUpperBound()); > } > return false; > } > > for cases where tv.getUpperBound().kind == UNION. Yours returns false, original returns true. > > > > Best Regards, Thomas > > > > > > > > > > On Sun, Jul 19, 2020 at 12:27 PM Lindenmaier, Goetz < > goetz.lindenmaier at sap.com> wrote: > > Hi > > I am downporting this for 11.0.9-oracle parity. > I had to resolve 3 files. 2 are copyrights. > In LambdaToMethod.java, 11 compares with INTERSECTION, > where the original calls a method. The selection of the bounds field > is similar though, and was easy to adapt. > > http://cr.openjdk.java.net/~goetz/wr20/8193367-ann_type_var-jdk11/01/ > > Please review. > > https://bugs.openjdk.java.net/browse/JDK-8193367 > http://hg.openjdk.java.net/jdk/jdk/rev/a772e65727c5 > > Best regards, > Goetz. > > From yan at azul.com Mon Jul 20 08:55:17 2020 From: yan at azul.com (Yuri Nesterenko) Date: Mon, 20 Jul 2020 11:55:17 +0300 Subject: [13u] RFR: 8248348: Regression caused by the update to BCEL 6.0 Message-ID: <14b9e489-28b4-25d5-90cb-0d1fc8e84203@azul.com> Hi, could you please take a look at this downport of https://bugs.openjdk.java.net/browse/JDK-8248348 ? Changes are, LastModified values in the javadoc comments (and removal of version id comment, was it there from SCCS times?). Other than that, patch applies cleanly: http://cr.openjdk.java.net/~yan/8248348/webrev.00/ Thanks, --yan From abrygin at azul.com Mon Jul 20 11:00:53 2020 From: abrygin at azul.com (Andrew Brygin) Date: Mon, 20 Jul 2020 14:00:53 +0300 Subject: [13u] RFR: 8248348: Regression caused by the update to BCEL 6.0 In-Reply-To: <14b9e489-28b4-25d5-90cb-0d1fc8e84203@azul.com> References: <14b9e489-28b4-25d5-90cb-0d1fc8e84203@azul.com> Message-ID: <55936460-c98b-1cd4-1384-d83549561f5c@azul.com> Hello Yuri, the change looks fine to me. Thanks, Andrew On 20/07/2020 11:55, Yuri Nesterenko wrote: > Hi, > > could you please take a look at this downport of > https://bugs.openjdk.java.net/browse/JDK-8248348 ? > > Changes are, LastModified values in the javadoc comments > (and removal of version id comment, was it there from SCCS times?). > Other than that, patch applies cleanly: > > http://cr.openjdk.java.net/~yan/8248348/webrev.00/ > > > Thanks, > --yan > From christoph.goettschkes at microdoc.com Mon Jul 20 12:07:32 2020 From: christoph.goettschkes at microdoc.com (=?UTF-8?Q?Christoph_G=c3=b6ttsches?=) Date: Mon, 20 Jul 2020 14:07:32 +0200 Subject: [13u] 8234535: Cross compilation fails due to missing CFLAGS for the BUILD_CC Message-ID: <32d6e85rw0-1@aserp2040.oracle.com> Hi, I requested the backport of 8234535 to 13u and it has been approved. Since I don't have commit access, could someone please sponsor this changeset for me? The patch applies cleanly to the jdk13u-dev repository. Bug: https://bugs.openjdk.java.net/browse/JDK-8234535 Original webrev: https://cr.openjdk.java.net/~cgo/8234535/webrev.01/ Commit: https://hg.openjdk.java.net/jdk/jdk/rev/eef0bf57357c Thanks, Christoph From yan at azul.com Mon Jul 20 12:17:19 2020 From: yan at azul.com (Yuri Nesterenko) Date: Mon, 20 Jul 2020 15:17:19 +0300 Subject: [13u] 8234535: Cross compilation fails due to missing CFLAGS for the BUILD_CC In-Reply-To: <32d6e85rw0-1@aserp2040.oracle.com> References: <32d6e85rw0-1@aserp2040.oracle.com> Message-ID: Hi Christoph, sure, I'll do. Thank you for the fix! --yan On 20.07.2020 15:07, Christoph G?ttsches wrote: > Hi, > > I requested the backport of 8234535 to 13u and it has been approved. Since I don't have commit access, > could someone please sponsor this changeset for me? The patch applies cleanly to the jdk13u-dev > repository. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8234535 > Original webrev: https://cr.openjdk.java.net/~cgo/8234535/webrev.01/ > Commit: https://hg.openjdk.java.net/jdk/jdk/rev/eef0bf57357c > > Thanks, > Christoph From christoph.goettschkes at microdoc.com Mon Jul 20 12:35:00 2020 From: christoph.goettschkes at microdoc.com (=?UTF-8?Q?Christoph_G=c3=b6ttsches?=) Date: Mon, 20 Jul 2020 14:35:00 +0200 Subject: [13u] 8234535: Cross compilation fails due to missing CFLAGS for the BUILD_CC In-Reply-To: References: <32d6e85rw0-1@aserp2040.oracle.com> Message-ID: <32d8h82ypd-1@aserp2050.oracle.com> Thank you for sponsoring this. -- Christoph On 2020-07-20 14:17, Yuri Nesterenko wrote: > Hi Christoph, > > sure, I'll do. > Thank you for the fix! > > --yan > > On 20.07.2020 15:07, Christoph G?ttsches wrote: >> Hi, >> >> I requested the backport of 8234535 to 13u and it has been approved. Since I don't have commit access, >> could someone please sponsor this changeset for me? The patch applies cleanly to the jdk13u-dev >> repository. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8234535 >> Original webrev: https://cr.openjdk.java.net/~cgo/8234535/webrev.01/ >> Commit: https://hg.openjdk.java.net/jdk/jdk/rev/eef0bf57357c >> >> Thanks, >> Christoph > From goetz.lindenmaier at sap.com Mon Jul 20 13:10:58 2020 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Mon, 20 Jul 2020 13:10:58 +0000 Subject: [11u] RFR: 8193367: Annotated type variable bounds crash javac In-Reply-To: References: Message-ID: Hi Thomas Matthias pushed his change, and now this one applies with Copyright changes only. http://cr.openjdk.java.net/~goetz/wr20/8193367-ann_type_var-jdk11/02/ Thanks for pointing me to this! Best regards, Goetz From: Thomas St?fe Sent: Monday, July 20, 2020 10:35 AM To: Lindenmaier, Goetz Cc: jdk-updates-dev at openjdk.java.net Subject: Re: [11u] RFR: 8193367: Annotated type variable bounds crash javac Hi Goetz, true. The behaviour was changed with https://bugs.openjdk.java.net/browse/JDK-8213703 (JDK-8213703 "LambdaConversionException: Invalid receiver type not a subtype of implementation type interface") Matthias as a backport request open for that one, maybe wait with this downport until Matthias is done? Cheers, Thomas On Mon, Jul 20, 2020 at 10:07 AM Lindenmaier, Goetz > wrote: Hi Thomas, Yes, that is true. But this change is about retrieving the bounds. The difference you point out is not addressed by the change downported. I would say the downport is correct because I do preserve the old behavior wrt. the kind. Best regards, Goetz. From: Thomas St?fe > Sent: Monday, July 20, 2020 10:01 AM To: Lindenmaier, Goetz > Cc: jdk-updates-dev at openjdk.java.net Subject: Re: [11u] RFR: 8193367: Annotated type variable bounds crash javac Hi Goetz, LambdaToMethod.java: Your version: 2358 boolean interfaceParameterIsIntersectionOrUnionType() { 2359 List tl = tree.getDescriptorType(types).getParameterTypes(); 2360 for (; tl.nonEmpty(); tl = tl.tail) { 2361 Type pt = tl.head; 2362 switch (pt.getKind()) { 2363 case INTERSECTION: 2364 case UNION: 2365 return true; 2366 case TYPEVAR: 2367 TypeVar tv = (TypeVar) pt; 2368 if (tv.getUpperBound().getKind() == TypeKind.INTERSECTION) { 2369 return true; 2370 } 2371 } 2372 } 2373 return false; 2374 } does not seem equivalent to the original version: boolean isIntersectionOrUnionType(Type t) { switch (t.getKind()) { case INTERSECTION: case UNION: return true; case TYPEVAR: TypeVar tv = (TypeVar) t; return isIntersectionOrUnionType(tv.getUpperBound()); } return false; } for cases where tv.getUpperBound().kind == UNION. Yours returns false, original returns true. Best Regards, Thomas On Sun, Jul 19, 2020 at 12:27 PM Lindenmaier, Goetz > wrote: Hi I am downporting this for 11.0.9-oracle parity. I had to resolve 3 files. 2 are copyrights. In LambdaToMethod.java, 11 compares with INTERSECTION, where the original calls a method. The selection of the bounds field is similar though, and was easy to adapt. http://cr.openjdk.java.net/~goetz/wr20/8193367-ann_type_var-jdk11/01/ Please review. https://bugs.openjdk.java.net/browse/JDK-8193367 http://hg.openjdk.java.net/jdk/jdk/rev/a772e65727c5 Best regards, Goetz. From christoph.goettschkes at microdoc.com Mon Jul 20 13:48:44 2020 From: christoph.goettschkes at microdoc.com (=?UTF-8?Q?Christoph_G=c3=b6ttschkes?=) Date: Mon, 20 Jul 2020 15:48:44 +0200 Subject: [11u] RFR 8234535: Cross compilation fails due to missing CFLAGS for the BUILD_CC Message-ID: <32d6e87j9y-1@aserp2040.oracle.com> Hi, please review this downport of JDK-8234535 to jdk11u-dev. The changeset does not apply cleanly because of unrelated differences in the flags-cflags.m4 file surrounding the patch. Manually applying the patch was trivial. Bug: https://bugs.openjdk.java.net/browse/JDK-8234535 Commit: https://hg.openjdk.java.net/jdk/jdk/rev/eef0bf57357c Webrev: https://cr.openjdk.java.net/~cgo/8234535/webrev-11u.00/ Thanks, Christoph From christoph.langer at sap.com Tue Jul 21 07:02:46 2020 From: christoph.langer at sap.com (Langer, Christoph) Date: Tue, 21 Jul 2020 07:02:46 +0000 Subject: [11u] RFR: JDK-8222074: Enhance auto vectorization for x86 In-Reply-To: References: Message-ID: Hi Paul, sorry for the delay but I eventually approved the prerequisites JDK-8222074 and JDK-8224234. Cheers Christoph > -----Original Message----- > From: jdk-updates-dev On > Behalf Of Hohensee, Paul > Sent: Mittwoch, 24. Juni 2020 19:45 > To: Lindenmaier, Goetz ; Lemmond, Dan > ; jdk-updates-dev at openjdk.java.net > Subject: RE: [11u] RFR: JDK-8222074: Enhance auto vectorization for x86 > > The 11.0.8 backport of JDK-8230591 was reverted by JDK-8245649 because it > required the backport dependencies Dan lists in his email. I'll file a redo JDK- > 8230591 backport request on Dan's behalf once the prerequisite backports > are done. > > We would like to backport aarch64 performance work because ARM64-based > server are being rapidly adopted for high performance floating point > applications. JDK 11 will be the primary Java execution platform for them for > some time to come. > > Thanks, > Paul > > ?On 6/22/20, 6:15 AM, "jdk-updates-dev on behalf of Lindenmaier, Goetz" > goetz.lindenmaier at sap.com> wrote: > > Hi Dan, > > I had a look at your change. > > The patch in the webrev is missing the original change > comments, please make sure to push it with the correct > information. > > Besides that the downport looks good, omitting the test coding > is fine. > > You are right, > "8224234 compiler/codegen/TestCharVect2.java fails in test_mulc" > needs to be pushed with this one. The other follow-up is already > downported. > > Best regards, > Goetz > > -----Original Message----- > From: jdk-updates-dev On > Behalf Of Lemmond, Dan > Sent: Samstag, 6. Juni 2020 00:09 > To: jdk-updates-dev at openjdk.java.net > Subject: [DMARC FAILURE] [11u] RFR: JDK-8222074: Enhance auto > vectorization for x86 > > Hi, > > I sent this review out previously but it was moderated. Moderators, feel > free to discard the old one. > > This backport is a prerequisite for backporting JDK- > 8230591. I plan to > backport JDK-8226721 > and JDK-8231713 in > the coming days before finally backporting JDK-8230591. > > This review is part of a set and will need to be pushed with the backport for > JDK-8224234 which I > will be sending out shortly. > > JBS: https://bugs.openjdk.java.net/browse/JDK-8222074 > Webrev: http://cr.openjdk.java.net/~phh/8222074/webrev.11u.00/ > Original review thread: https://mail.openjdk.java.net/pipermail/hotspot- > compiler-dev/2019-May/033561.html > > Please review this backport for JDK-8222074 to 11u. The changeset applied > cleanly for the most part, only requiring a manual fix for two hunks. > > The two hunks that needed to be fixed were hunk 3 in assembler_x86.hpp > and hunk 3 in x86.ad. > > The backport compiled without issue and successfully passed the Tier1 > Hotspot tests. > > Please note that the change to CheckGraalIntrinsics.java was omitted from > this backport as the check was ?isJDK13OrHigher? which is not relevant for > 11u. > > Thank you, > Dan > From christoph.langer at sap.com Tue Jul 21 07:30:29 2020 From: christoph.langer at sap.com (Langer, Christoph) Date: Tue, 21 Jul 2020 07:30:29 +0000 Subject: [11u] CSR JDK-8245689 Align some one-way conversion in MS950 charset with Windows In-Reply-To: <95414bbbed1aa20c0376bb90fe690e69@linux.vnet.ibm.com> References: <95414bbbed1aa20c0376bb90fe690e69@linux.vnet.ibm.com> Message-ID: Hi Ichiroh, since the CSR was processed and approved now, I've also approved the backport. Best regards Christoph > -----Original Message----- > From: Ichiroh Takiguchi > Sent: Mittwoch, 1. Juli 2020 07:49 > To: Langer, Christoph > Cc: jdk-updates-dev at openjdk.java.net > Subject: RE: [11u] CSR JDK-8245689 Align some one-way conversion in MS950 > charset with Windows > > Hello Christoph. > > I appreciate your suggestion and sorry for bad response. > > I created new CSR for 11u is JDK-8248305 [1] > (Base CSR was JDK-8233385 [2], I just copied the contents from > JDK-8233385 [2]) > > Could you review JDK-8248305 [1] ? > > [1] https://bugs.openjdk.java.net/browse/JDK-8248305 > [2] https://bugs.openjdk.java.net/browse/JDK-8233385 > > Thanks, > Ichiroh Takiguchi > > On 2020-06-23 13:38, Langer, Christoph wrote: > > Hi Ichiroh, > > > > I've just checked your request. > > > > The bug-item you created > > https://bugs.openjdk.java.net/browse/JDK-8245689 is of type > > "backport". I guess you've intuitively used "More" -> "Create > > Backport" of the original CSR. Unfortunately, that doesn't work > > correctly. > > So, I've made JDK-8245689 a backport of the original bug > > (https://bugs.openjdk.java.net/browse/JDK-8232161). It should get > > resolved once you actually push the backport (under its original bug > > id, of course). However, from that backport bug (JDK-8245689) you'll > > have to create (and edit) the backport CSR via "More" -> "Create CSR". > > Please let me know when you've done that and I can review. > > > > Cheers > > Christoph > > > >> -----Original Message----- > >> From: jdk-updates-dev > On > >> Behalf Of Ichiroh Takiguchi > >> Sent: Montag, 25. Mai 2020 03:55 > >> To: jdk-updates-dev at openjdk.java.net > >> Subject: [11u] CSR JDK-8245689 Align some one-way conversion in MS950 > >> charset with Windows > >> > >> Hello. > >> > >> Could you review JDK-8245689 [1] ? > >> > >> Parent CSR was JDK-8233385 Align some one-way conversion in MS950 > >> charset with Windows [2] > >> > >> Actual fix was JDK-8232161 Align some one-way conversion in MS950 > >> charset with Windows [3] > >> The fix can be applied against 11u-dev without any change. > >> > >> [1] https://bugs.openjdk.java.net/browse/JDK-8245689 > >> [2] https://bugs.openjdk.java.net/browse/JDK-8233385 > >> [3] https://bugs.openjdk.java.net/browse/JDK-8232161 > >> > >> Thanks, > >> Ichiroh Takiguchi > >> IBM Japan, Ltd. From matthias.baesken at sap.com Tue Jul 21 14:30:22 2020 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Tue, 21 Jul 2020 14:30:22 +0000 Subject: RFR [jdk11u] : 8249255 : Build fails if source code in cygwin home dir Message-ID: Hello, please review the jdk11 backport of 8249255 : Build fails if source code in cygwin home dir . The code change goes to another file in jdk11 ( make/autoconf/basics.m4 , not basic.m4 ) . Additionally , I had to use BASIC_FIXUP_PATH in jdk11 instead of UTIL_FIXUP_PATH in jdk/jdk . Bug/webrev : https://bugs.openjdk.java.net/browse/JDK-8249255 http://cr.openjdk.java.net/~mbaesken/webrevs/8249255_0_jdk11/ webrev from jdk/jdk : https://hg.openjdk.java.net/jdk/jdk15/rev/aba19f77c70d Thanks, Matthias From goetz.lindenmaier at sap.com Tue Jul 21 15:00:03 2020 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Tue, 21 Jul 2020 15:00:03 +0000 Subject: RFR [jdk11u] : 8249255 : Build fails if source code in cygwin home dir In-Reply-To: References: Message-ID: Hi Matthias, The change looks good. Thanks for backporting, Goetz. > -----Original Message----- > From: jdk-updates-dev On Behalf > Of Baesken, Matthias > Sent: Tuesday, July 21, 2020 4:30 PM > To: jdk-updates-dev at openjdk.java.net > Cc: Erik Joelsson > Subject: [CAUTION] RFR [jdk11u] : 8249255 : Build fails if source code in > cygwin home dir > > Hello, please review the jdk11 backport of 8249255 : Build fails if source > code in cygwin home dir . > The code change goes to another file in jdk11 ( make/autoconf/basics.m4 , > not basic.m4 ) . > Additionally , I had to use BASIC_FIXUP_PATH in jdk11 instead of > UTIL_FIXUP_PATH in jdk/jdk . > > > > > Bug/webrev : > > https://bugs.openjdk.java.net/browse/JDK-8249255 > > http://cr.openjdk.java.net/~mbaesken/webrevs/8249255_0_jdk11/ > > > webrev from jdk/jdk : > > https://hg.openjdk.java.net/jdk/jdk15/rev/aba19f77c70d > > > Thanks, Matthias From hohensee at amazon.com Tue Jul 21 17:18:52 2020 From: hohensee at amazon.com (Hohensee, Paul) Date: Tue, 21 Jul 2020 17:18:52 +0000 Subject: [11u] RFR: JDK-8222074: Enhance auto vectorization for x86 Message-ID: <7F36AE66-3A05-465C-A237-A072B8788B64@amazon.com> No problem. Thanks! Paul ?On 7/21/20, 12:03 AM, "Langer, Christoph" wrote: Hi Paul, sorry for the delay but I eventually approved the prerequisites JDK-8222074 and JDK-8224234. Cheers Christoph > -----Original Message----- > From: jdk-updates-dev On > Behalf Of Hohensee, Paul > Sent: Mittwoch, 24. Juni 2020 19:45 > To: Lindenmaier, Goetz ; Lemmond, Dan > ; jdk-updates-dev at openjdk.java.net > Subject: RE: [11u] RFR: JDK-8222074: Enhance auto vectorization for x86 > > The 11.0.8 backport of JDK-8230591 was reverted by JDK-8245649 because it > required the backport dependencies Dan lists in his email. I'll file a redo JDK- > 8230591 backport request on Dan's behalf once the prerequisite backports > are done. > > We would like to backport aarch64 performance work because ARM64-based > server are being rapidly adopted for high performance floating point > applications. JDK 11 will be the primary Java execution platform for them for > some time to come. > > Thanks, > Paul > > On 6/22/20, 6:15 AM, "jdk-updates-dev on behalf of Lindenmaier, Goetz" > goetz.lindenmaier at sap.com> wrote: > > Hi Dan, > > I had a look at your change. > > The patch in the webrev is missing the original change > comments, please make sure to push it with the correct > information. > > Besides that the downport looks good, omitting the test coding > is fine. > > You are right, > "8224234 compiler/codegen/TestCharVect2.java fails in test_mulc" > needs to be pushed with this one. The other follow-up is already > downported. > > Best regards, > Goetz > > -----Original Message----- > From: jdk-updates-dev On > Behalf Of Lemmond, Dan > Sent: Samstag, 6. Juni 2020 00:09 > To: jdk-updates-dev at openjdk.java.net > Subject: [DMARC FAILURE] [11u] RFR: JDK-8222074: Enhance auto > vectorization for x86 > > Hi, > > I sent this review out previously but it was moderated. Moderators, feel > free to discard the old one. > > This backport is a prerequisite for backporting JDK- > 8230591. I plan to > backport JDK-8226721 > and JDK-8231713 in > the coming days before finally backporting JDK-8230591. > > This review is part of a set and will need to be pushed with the backport for > JDK-8224234 which I > will be sending out shortly. > > JBS: https://bugs.openjdk.java.net/browse/JDK-8222074 > Webrev: http://cr.openjdk.java.net/~phh/8222074/webrev.11u.00/ > Original review thread: https://mail.openjdk.java.net/pipermail/hotspot- > compiler-dev/2019-May/033561.html > > Please review this backport for JDK-8222074 to 11u. The changeset applied > cleanly for the most part, only requiring a manual fix for two hunks. > > The two hunks that needed to be fixed were hunk 3 in assembler_x86.hpp > and hunk 3 in x86.ad. > > The backport compiled without issue and successfully passed the Tier1 > Hotspot tests. > > Please note that the change to CheckGraalIntrinsics.java was omitted from > this backport as the check was ?isJDK13OrHigher? which is not relevant for > 11u. > > Thank you, > Dan > From vkempik at azul.com Tue Jul 21 21:10:39 2020 From: vkempik at azul.com (Vladimir Kempik) Date: Tue, 21 Jul 2020 21:10:39 +0000 Subject: [11u] RFR 8238284: [macos] Zero VM build fails due to an obvious typo Message-ID: <290A2406-87A5-4029-A1D1-CBB5489F881B@azul.com> Hello Please review this backport of JDK-8238284 to jdk11u The patch didn?t apply cleanly. Original 8238284 fixed two issues in jdk15: JDK-8237437 - since jdk14 JDK-8165929 - since jdk11 Jdk11 only has 8165929 so only part of original fix needs to be backported to jdk11, that part applies cleanly. This is part2 out of 3 fixes needed to fix zero vm on macos. Fix has also part for linux but it wasn?t affected in my testing. Risks are low because the patch just adds const keyword to some vars for zerovm The bug: https://bugs.openjdk.java.net/browse/JDK-8238284 The webrev: http://cr.openjdk.java.net/~vkempik/8238284/webrev.11u.00/ Original review https://mail.openjdk.java.net/pipermail/hotspot-dev/2020-January/040681.html Other two mentioned bugs: https://bugs.openjdk.java.net/browse/JDK-8234737 , https://bugs.openjdk.java.net/browse/JDK-8165929 Thanks, Vladimir From goetz.lindenmaier at sap.com Wed Jul 22 06:07:59 2020 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Wed, 22 Jul 2020 06:07:59 +0000 Subject: [11u] RFR 8238284: [macos] Zero VM build fails due to an obvious typo In-Reply-To: <290A2406-87A5-4029-A1D1-CBB5489F881B@azul.com> References: <290A2406-87A5-4029-A1D1-CBB5489F881B@azul.com> Message-ID: Hi Vladimir, The change looks good. If the omitted part would be more complex, I would Propose to make a separate change for this, so that 8238284 is not marked as downported. But as the fix (!-->I) is quite obvious, and as the change introducing this typeo (8234737) is quite unlikely to be downported, I am fine with handling this as you do. Best regards, Goetz. > -----Original Message----- > From: jdk-updates-dev On Behalf > Of Vladimir Kempik > Sent: Tuesday, July 21, 2020 11:11 PM > To: jdk-updates-dev at openjdk.java.net > Subject: [11u] RFR 8238284: [macos] Zero VM build fails due to an obvious > typo > > Hello > Please review this backport of JDK-8238284 to jdk11u > > The patch didn?t apply cleanly. Original 8238284 fixed two issues in jdk15: > JDK-8237437 - since jdk14 > JDK-8165929 - since jdk11 > > Jdk11 only has 8165929 so only part of original fix needs to be backported to > jdk11, that part applies cleanly. > > This is part2 out of 3 fixes needed to fix zero vm on macos. > > Fix has also part for linux but it wasn?t affected in my testing. > > Risks are low because the patch just adds const keyword to some vars for > zerovm > > The bug: https://bugs.openjdk.java.net/browse/JDK-8238284 > The webrev: http://cr.openjdk.java.net/~vkempik/8238284/webrev.11u.00/ > Original review https://mail.openjdk.java.net/pipermail/hotspot-dev/2020- > January/040681.html > Other two mentioned bugs: https://bugs.openjdk.java.net/browse/JDK- > 8234737 , https://bugs.openjdk.java.net/browse/JDK-8165929 > > Thanks, Vladimir From vkempik at azul.com Wed Jul 22 11:19:08 2020 From: vkempik at azul.com (Vladimir Kempik) Date: Wed, 22 Jul 2020 11:19:08 +0000 Subject: [11u] RFR 8211296: Remove HotSpot deprecation warning suppression for Mac/clang Message-ID: Hello Please review this backport of JDK-8211296 to jdk11u The patch didn?t apply cleanly. Part of changes (for CompileJvm.gmk file) wasn?t needed. The motivation behind this backport is to prepare for aarch64 macos, finite macro is missing there and needs to be replaced with isfinite. Testing: tier1. The bug: https://bugs.openjdk.java.net/browse/JDK-8211296 The webrev: http://cr.openjdk.java.net/~vkempik/8211296/webrev.00.11u/ Original review https://mail.openjdk.java.net/pipermail/build-dev/2018-September/023485.html Original changeset: http://hg.openjdk.java.net/jdk/jdk/rev/286389b60292 Thanks, Vladimir From goetz.lindenmaier at sap.com Thu Jul 23 09:04:49 2020 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Thu, 23 Jul 2020 09:04:49 +0000 Subject: [11u] RFR: 8243029: Rewrite javax/net/ssl/compatibility/Compatibility.java with a flexible interop test framework In-Reply-To: References: Message-ID: Hi Matthias > Seems jdk11 does only know the name of TLS_CHACHA20_POLY1305_SHA256 but nothing more , the real support came with jdk12. > Some other folks were struggeling with this too : > https://jira.mariadb.org/browse/CONJ-721 > > Maybe there is a chance to backport it to jdk11 in future (of course it is a separate issue) ? On the one side, it would be good if all VMs supported the same certificates. On the other side, this is a functional change, e.g. as "8140466: ChaCha20 and Poly1305 TLS Cipher Suites" requires a CSR. As you probably noted, there are quite some resistances against downporting functional changes ... see the Shenandoah discussion. After my initial RFR, a fix for the test was pushed to jdk/jdk: https://bugs.openjdk.java.net/browse/JDK-8245005 javax/net/ssl/compatibility/BasicConnectTest.java failed with No enum constant With this fix the issues I see are fixed. It should be downported, too. > Could you file a separate bug for the readme, I guess this needs fixing in jdk/jdk too . The follow up fix also corrects the test name in the README I had complained about in my first RFR. Also, I figured that I need not specivy -m to run manual tests, it sufficies to remove -a from the command line. Still mentioning this in the README would be usedufl, but I don't feel like opening a change only for this. I did two adaptions to the change, though: http://cr.openjdk.java.net/~goetz/wr20/8243029-ssl_test_framework-jdk11/02/ Find the incremental patch below. Best regards, Goetz. --- a/test/jdk/javax/net/ssl//compatibility/HrrTest.java +++ b/test/jdk/javax/net/ssl//compatibility/HrrTest.java @@ -88,8 +88,7 @@ List> testCases = new ArrayList<>(); for (CipherSuite cipherSuite : new CipherSuite[] { CipherSuite.TLS_AES_128_GCM_SHA256, - CipherSuite.TLS_AES_256_GCM_SHA384, - CipherSuite.TLS_CHACHA20_POLY1305_SHA256}) { + CipherSuite.TLS_AES_256_GCM_SHA384}) { testCases.add(createTestCase(cipherSuite)); } return testCases; --- a/test/jdk/javax/net/ssl//compatibility/README +++ b/test/jdk/javax/net/ssl//compatibility/README @@ -21,9 +21,9 @@ ##### Summary ##### This test is used to check the interop compatibility on JSSE among different -JDK releases. The oldest version supported by the test is JDK 7. Some of Java +JDK releases. The oldest version supported by the test is JDK 8. Some of Java source files, like JdkInfoUtils.java, JdkProcServer.java and JdkProcClient.java, -use only JDK 7-compliant language features and APIs, in order to allowing +use only JDK 8-compliant language features and APIs, in order to allowing different JDK releases can load and run associated classes. ##### Usage ##### @@ -47,7 +47,6 @@ ##### Usage Examples ##### $ cat /path/to/jdkList -/path/to/jdk7 /path/to/jdk8 /path/to/jdk9 /path/to/jdk10 @@ -57,6 +56,6 @@ -Dtest.jdk.list.file=/path/to/jdkList \ $JDK_WS/jdk/test/javax/net/ssl/compatibility/Compatibility.java The above example uses a file "/path/to/jdkList" to contain the paths of local -different JDK builds through 7 to 10. The execution uses each of JDK builds as +different JDK builds through 8 to 10. The execution uses each of JDK builds as server and client respectively. And it enables SSL debug flag, and tests the full parameter value set. From christoph.langer at sap.com Thu Jul 23 09:46:44 2020 From: christoph.langer at sap.com (Langer, Christoph) Date: Thu, 23 Jul 2020 09:46:44 +0000 Subject: [11u] RFR: 8237182: Update copyright header for shenandoah and epsilon files Message-ID: Hi, please review this backport of a copyright header only change that Oracle has done in 11.0.8. Since Shenandoah does not exist (yet?;-)) in 11u, it boils down to touching the epsilon files only. Those updates apply cleanly, though. Should Shenandoah ever be integrated into 11u we should commit with the copyright headers in the correct form right away. Bug: https://bugs.openjdk.java.net/browse/JDK-8237182 Original Change: https://hg.openjdk.java.net/jdk/jdk/rev/14c78683c9f0 Webrev: http://cr.openjdk.java.net/~clanger/webrevs/8237182.11u.0/ Thanks Christoph From shade at redhat.com Thu Jul 23 09:59:09 2020 From: shade at redhat.com (Aleksey Shipilev) Date: Thu, 23 Jul 2020 11:59:09 +0200 Subject: [11u] RFR: 8237182: Update copyright header for shenandoah and epsilon files In-Reply-To: References: Message-ID: <7fd37e1a-78f0-37b8-d1d3-dcf1f205b46d@redhat.com> On 7/23/20 11:46 AM, Langer, Christoph wrote: > Bug: https://bugs.openjdk.java.net/browse/JDK-8237182 > Original Change: https://hg.openjdk.java.net/jdk/jdk/rev/14c78683c9f0 > Webrev: http://cr.openjdk.java.net/~clanger/webrevs/8237182.11u.0/ This looks fine, thank you. -- -Aleksey From matthias.baesken at sap.com Thu Jul 23 10:00:33 2020 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Thu, 23 Jul 2020 10:00:33 +0000 Subject: [11u] RFR: 8243029: Rewrite javax/net/ssl/compatibility/Compatibility.java with a flexible interop test framework In-Reply-To: References: Message-ID: Thanks for the explanation. >I did two adaptions to the change, though: >http://cr.openjdk.java.net/~goetz/wr20/8243029-ssl_test_framework-jdk11/02/ >Find the incremental patch below. Looks good . Best regards, Matthias -----Original Message----- From: Lindenmaier, Goetz Sent: Donnerstag, 23. Juli 2020 11:05 To: Baesken, Matthias ; 'jdk-updates-dev at openjdk.java.net' Subject: RE: RE: [11u] RFR: 8243029: Rewrite javax/net/ssl/compatibility/Compatibility.java with a flexible interop test framework Hi Matthias > Seems jdk11 does only know the name of TLS_CHACHA20_POLY1305_SHA256 but nothing more , the real support came with jdk12. > Some other folks were struggeling with this too : > https://jira.mariadb.org/browse/CONJ-721 > > Maybe there is a chance to backport it to jdk11 in future (of course it is a separate issue) ? On the one side, it would be good if all VMs supported the same certificates. On the other side, this is a functional change, e.g. as "8140466: ChaCha20 and Poly1305 TLS Cipher Suites" requires a CSR. As you probably noted, there are quite some resistances against downporting functional changes ... see the Shenandoah discussion. After my initial RFR, a fix for the test was pushed to jdk/jdk: https://bugs.openjdk.java.net/browse/JDK-8245005 javax/net/ssl/compatibility/BasicConnectTest.java failed with No enum constant With this fix the issues I see are fixed. It should be downported, too. > Could you file a separate bug for the readme, I guess this needs fixing in jdk/jdk too . The follow up fix also corrects the test name in the README I had complained about in my first RFR. Also, I figured that I need not specivy -m to run manual tests, it sufficies to remove -a from the command line. Still mentioning this in the README would be usedufl, but I don't feel like opening a change only for this. I did two adaptions to the change, though: http://cr.openjdk.java.net/~goetz/wr20/8243029-ssl_test_framework-jdk11/02/ Find the incremental patch below. Best regards, Goetz. --- a/test/jdk/javax/net/ssl//compatibility/HrrTest.java +++ b/test/jdk/javax/net/ssl//compatibility/HrrTest.java @@ -88,8 +88,7 @@ List> testCases = new ArrayList<>(); for (CipherSuite cipherSuite : new CipherSuite[] { CipherSuite.TLS_AES_128_GCM_SHA256, - CipherSuite.TLS_AES_256_GCM_SHA384, - CipherSuite.TLS_CHACHA20_POLY1305_SHA256}) { + CipherSuite.TLS_AES_256_GCM_SHA384}) { testCases.add(createTestCase(cipherSuite)); } return testCases; --- a/test/jdk/javax/net/ssl//compatibility/README +++ b/test/jdk/javax/net/ssl//compatibility/README @@ -21,9 +21,9 @@ ##### Summary ##### This test is used to check the interop compatibility on JSSE among different -JDK releases. The oldest version supported by the test is JDK 7. Some of Java +JDK releases. The oldest version supported by the test is JDK 8. Some of Java source files, like JdkInfoUtils.java, JdkProcServer.java and JdkProcClient.java, -use only JDK 7-compliant language features and APIs, in order to allowing +use only JDK 8-compliant language features and APIs, in order to allowing different JDK releases can load and run associated classes. ##### Usage ##### @@ -47,7 +47,6 @@ ##### Usage Examples ##### $ cat /path/to/jdkList -/path/to/jdk7 /path/to/jdk8 /path/to/jdk9 /path/to/jdk10 @@ -57,6 +56,6 @@ -Dtest.jdk.list.file=/path/to/jdkList \ $JDK_WS/jdk/test/javax/net/ssl/compatibility/Compatibility.java The above example uses a file "/path/to/jdkList" to contain the paths of local -different JDK builds through 7 to 10. The execution uses each of JDK builds as +different JDK builds through 8 to 10. The execution uses each of JDK builds as server and client respectively. And it enables SSL debug flag, and tests the full parameter value set. From goetz.lindenmaier at sap.com Thu Jul 23 11:40:01 2020 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Thu, 23 Jul 2020 11:40:01 +0000 Subject: [11u] RFR: 8243029: Rewrite javax/net/ssl/compatibility/Compatibility.java with a flexible interop test framework In-Reply-To: References: Message-ID: Hi Matthias, Thanks for reviewing! Best regards, Goetz. > -----Original Message----- > From: Baesken, Matthias > Sent: Thursday, July 23, 2020 12:01 PM > To: Lindenmaier, Goetz ; 'jdk-updates- > dev at openjdk.java.net' > Subject: RE: RE: [11u] RFR: 8243029: Rewrite > javax/net/ssl/compatibility/Compatibility.java with a flexible interop test > framework > > Thanks for the explanation. > > >I did two adaptions to the change, though: > >http://cr.openjdk.java.net/~goetz/wr20/8243029-ssl_test_framework- > jdk11/02/ > >Find the incremental patch below. > > Looks good . > > > Best regards, Matthias > > > -----Original Message----- > From: Lindenmaier, Goetz > Sent: Donnerstag, 23. Juli 2020 11:05 > To: Baesken, Matthias ; 'jdk-updates- > dev at openjdk.java.net' > Subject: RE: RE: [11u] RFR: 8243029: Rewrite > javax/net/ssl/compatibility/Compatibility.java with a flexible interop test > framework > > Hi Matthias > > > Seems jdk11 does only know the name of > TLS_CHACHA20_POLY1305_SHA256 but nothing more , the real support > came with jdk12. > > Some other folks were struggeling with this too : > > https://jira.mariadb.org/browse/CONJ-721 > > > > Maybe there is a chance to backport it to jdk11 in future (of course it is a > separate issue) ? > On the one side, it would be good if all VMs supported the same certificates. > On the other side, this is a functional change, e.g. as "8140466: ChaCha20 > and Poly1305 > TLS Cipher Suites" requires a CSR. As you probably noted, there are > quite some resistances against downporting functional changes ... > see the Shenandoah discussion. > > After my initial RFR, a fix for the test was pushed to jdk/jdk: > https://bugs.openjdk.java.net/browse/JDK-8245005 > javax/net/ssl/compatibility/BasicConnectTest.java failed with No enum > constant > With this fix the issues I see are fixed. It should be downported, too. > > > Could you file a separate bug for the readme, I guess this needs fixing in > jdk/jdk too . > The follow up fix also corrects the test name in the README I had > complained about in my first RFR. > Also, I figured that I need not specivy -m to run manual tests, it > sufficies to remove -a from the command line. > Still mentioning this in the README would be usedufl, but I don't > feel like opening a change only for this. > > I did two adaptions to the change, though: > http://cr.openjdk.java.net/~goetz/wr20/8243029-ssl_test_framework- > jdk11/02/ > Find the incremental patch below. > > Best regards, > Goetz. > > --- a/test/jdk/javax/net/ssl//compatibility/HrrTest.java > +++ b/test/jdk/javax/net/ssl//compatibility/HrrTest.java > @@ -88,8 +88,7 @@ > List> testCases = new ArrayList<>(); > for (CipherSuite cipherSuite : new CipherSuite[] { > CipherSuite.TLS_AES_128_GCM_SHA256, > - CipherSuite.TLS_AES_256_GCM_SHA384, > - CipherSuite.TLS_CHACHA20_POLY1305_SHA256}) { > + CipherSuite.TLS_AES_256_GCM_SHA384}) { > testCases.add(createTestCase(cipherSuite)); > } > return testCases; > --- a/test/jdk/javax/net/ssl//compatibility/README > +++ b/test/jdk/javax/net/ssl//compatibility/README > @@ -21,9 +21,9 @@ > > ##### Summary ##### > This test is used to check the interop compatibility on JSSE among different > -JDK releases. The oldest version supported by the test is JDK 7. Some of Java > +JDK releases. The oldest version supported by the test is JDK 8. Some of > Java > source files, like JdkInfoUtils.java, JdkProcServer.java and JdkProcClient.java, > -use only JDK 7-compliant language features and APIs, in order to allowing > +use only JDK 8-compliant language features and APIs, in order to allowing > different JDK releases can load and run associated classes. > > ##### Usage ##### > @@ -47,7 +47,6 @@ > > ##### Usage Examples ##### > $ cat /path/to/jdkList > -/path/to/jdk7 > /path/to/jdk8 > /path/to/jdk9 > /path/to/jdk10 > @@ -57,6 +56,6 @@ > -Dtest.jdk.list.file=/path/to/jdkList \ > $JDK_WS/jdk/test/javax/net/ssl/compatibility/Compatibility.java > The above example uses a file "/path/to/jdkList" to contain the paths of > local > -different JDK builds through 7 to 10. The execution uses each of JDK builds > as > +different JDK builds through 8 to 10. The execution uses each of JDK builds > as > server and client respectively. And it enables SSL debug flag, and tests the > full parameter value set. From rkennke at redhat.com Thu Jul 23 12:49:34 2020 From: rkennke at redhat.com (Roman Kennke) Date: Thu, 23 Jul 2020 14:49:34 +0200 Subject: RFR (11u, XXL): Upstream/backport Shenandoah to JDK11u In-Reply-To: <09D2175C-E013-404E-AAE3-C34FA3E8C394@amazon.com> References: <09D2175C-E013-404E-AAE3-C34FA3E8C394@amazon.com> Message-ID: <583d544e14e41367a45a24cfec3a3fc1fa00a602.camel@redhat.com> Hi Kelvin, On Thu, 2020-07-23 at 01:56 +0000, Nilsen, Kelvin wrote: > This is an unofficial review (I am not an official reviewer) of the > shared-code changes described in > http://cr.openjdk.java.net/~rkennke/shenandoah-jdk11u-upstream/webrev.09-shared/ > which were posted by "Roman Kennke" (rkennke at redhat.com) on Mon > Jul 13 16:06:12 UTC 2020. See > https://mail.openjdk.java.net/pipermail/jdk-updates-dev/2020-July/003483.html > for the original message. Thank you, Kelvin for reviewing it! You could have replied to the original email that you mentioned :-) > For the most part, these all look good and I concur that these do not > impact non-Shenandoah deployments except for the possible concerns > identified below: > > 1. In make/hotspot/gensrc/GensrcJfr.gmk, around line 68, it might be > good to list the following ExecuteWithLog arguments in the same order > as when Shenandoah is not enabled to make it more clear that these > parameters are "not affected": $(METADATA_XML) $(METADATA_XSD) > $(JFR_OUTPUTDIR) > This is not really possible. The patch also changes GenerateJfrFiles.java to accept multiple - and optional - arguments for metadata.xml, which means that it's most reasonable to place them at the end, and not first. BTW, I intend to upstream those build changes to jdk16 too. > 2. In src/hotspot/share/gc/shared/gc_globals.hpp, around line 200, it > looks like the lines: > product(bool, UseShenandoahGC, false, > "Use the Shenandoah garbage collector") > should be included only under #if INCLUDE_SHENANDOAHGC > conditional compilation control. That is not directly possible because I can't put an ifdef in the middle of a huge define. It can be done though: define the block under #if INCLUDE_SHENANDOAHGC above, and empty otherwise, and put that in the huge #define. I am doing that in the webrev.10 (see below). With that, the last remnants of Shenandoah are purged from libjvm.so when building without Shenandoah :-) On the (somewhat) downside, I also had to remove -XX:-UseShenandoahGC from TestDisableDefaultGC.java, that test would otherwise fail. (It does print 'unrecognized VM option' now, instead of "Garbage collector not selected"). > 3. In src/hotspot/share/opto/loopnode.cpp, around line 4514, it is > not clear from context that the Shenandoah change still includes all > of the same code that was present before the Shenandoah change. In > particular, it is not clear that !defined(PRODUCT) is universally > true. > Before the change, the rpo() method was only included when !PRODUCT. Shenandoah requires this method in PRODUCT-builds too, hence the change. I am not sure how to make this much clearer? > 4. Likewise, in src/hotspot/share/opto/loopnode.hpp, around line > 1327, the prototype for function void is introduced by Shenandoah > enhancements to the code even #if INCLUDE_SHENANDOAHGC in the case of > ifndef PRODUCT. See comment above. > 5. In src/hotspot/share/opto/type.hpp, around lines 1775 and 1822, do > we want to #if INCLUDE_SHENANDOAHGC the definitions of LoadXNode and > StoreXNode? Should be harmless as is. We could, but as you say it's harmless and doesn't generate any code per-se. I did it anyway in webrev.00. > 6. In src/hotspot/share/runtime/vmOperations.hpp, around line 108, do > we want to #if INCLUDE_SHENANDOAHGC the mentions of ShenandoahFullGC, > ShenandoahInitMark, ShenandoahFinalMarkStartEvac, > ShenadoahInitUpdateRefs, ShenandoahFinalUpdateRefs, > ShenandoahDegeneratedGC? Should be harmless as is. That is exactly what the block above with #if INCLUDE_SHENANDOAHGC does. We cannot #if INCLUDE_SHENANDOAHGC in the middle of a large #define. Hence the little stunt. (BTW, this is exactly the pattern how we could exclude the global UseShenandoahGC flag too, see above). > 7. In src/hotspot/share/utilities/globalDefinitions.hpp, around line > 98, do we want to put the UINT64_FORMAT_X_W(width) macro under #if > INCLUDE_SHENANDOAHGC? Harmless as is. We could, but as you say it's harmless and doesn't generate any code per-se. I did it anyway in webrev.00. > 8. In src/hotspot/share/utilities/macros.hpp, around line 238, should > the definition of NOT_SHENANDOAHGC_RETURN be { return; }? I don't think that is necessary. The definitions of other GCs are not doing it either, so let's keep it consistent with those, yes? > 9. I am not confident I understand how the file > src/jdk.jfr/share/conf/jfr/default.jfc is compiled. Should lines 423 > - 431 be under #if INCLUDE_SHENANDOAHGC control? Does this cause > unwanted Shenandoah code to be compiled in even when it's not > supposed to be? This is JFR profiles, it would not compile any code. We ensure that Shenandoah events are only compiled when Shenandoah is build-enabled (via the metadata.xml machinery mentioned above). Running JFR with such a profile on a JDK without Shenandoah support does work as usual. Those events are enabled, but simply not present. > 10. I am not confident I understand how regression suites are > configured. In test/hotspot/jtreg/TEST.ROOT and test/jdk/TEST.ROOT, > it appears that we are requiring vm.gc.Shenandoah in order to run > certain tests. If the JVM is compiled without support for > Shenandoah, will this cause these regression tests to not run. That > would not be good for someone who desires to build JDK 11 without > Shenandoah support. The test is specifically there to test if Shenandoah is built-in. Tests that execute Shenandoah code by enabling -XX:+UseShenandoahGC are all guarded by this, so they would not be run when Shenandoah is not built- in. See e.g.: http://cr.openjdk.java.net/~rkennke/shenandoah-jdk11u-upstream/webrev.09-shared/test/hotspot/jtreg/gc/TestHumongousReferenceObject.java.udiff.html > 11. In test/hotspot/jtreg/TEST.groups, it appears that there have > been many Shenandoah-specific tests added to the regression > suites. If someone builds the JDK11 without Shenandoah support and > runs the regression suite, will all of these tests now fail? I don't > see how the running of these tests is made conditional. (Perhaps my > issue 10 and issue 11 are complementary and between them, all of my > concerns are fully addressed.) See comment for #10. It is safe to run those suites with Shenandoah- build-disabled, it will only run a handful of tests then, and exclude all run configurations that have -XX:+UseShenandoahGC in them. Here's the updated webrevs, with UseShenandoahGC flag excluded on non- Shenandoah builds, and the two exclusions of macros: Shared-only: http://cr.openjdk.java.net/~rkennke/shenandoah-jdk11u-upstream/webrev.10-shared/ Full: http://cr.openjdk.java.net/~rkennke/shenandoah-jdk11u-upstream/webrev.10-all/ I did run hotspot_gc_shenandoah again on builds with and without Shenandoah, and it worked fine. From jcbeyler at google.com Thu Jul 23 14:43:20 2020 From: jcbeyler at google.com (Jean Christophe Beyler) Date: Thu, 23 Jul 2020 10:43:20 -0400 Subject: RFR (11u, S): backport JDK-8247615 Initialize the bytes left for the heap sampler Message-ID: Hi all, First time I request a backport so if I am missing anything, please let me know :) I am trying to port this webrev into 11u: Summary: This fixes a sampling bias for the first allocations and ensures that the sampling rate is correct from the start. It is safe to apply as the whole code is protected behind flags and only is used when JVMTI enables the heap monitoring. We added a test that tickles this code on purpose. Porting issues: Finally, the patch does not apply cleanly due to slight differences with the files. This webrev provides the clean to JDK11u: http://cr.openjdk.java.net/~jcbeyler/8247615/webrev.02_11/ - That webrev compiles and tests for HeapMonitor (with the new test) pass: make run-test TEST="test/hotspot/jtreg/serviceability/jvmti/HeapMonitor" compared to what I submitted to head: http://cr.openjdk.java.net/~jcbeyler/8247615/webrev.02/ Let me know what else I can do or provide! Thanks for all your help, Jc Ps: I sent this email before I subscribed so one is waiting in moderation; if this shows up in double, I apologize :) From rkennke at redhat.com Thu Jul 23 17:17:00 2020 From: rkennke at redhat.com (Roman Kennke) Date: Thu, 23 Jul 2020 19:17:00 +0200 Subject: RFR (11u, XXL): Upstream/backport Shenandoah to JDK11u In-Reply-To: <9C91C7F6-7209-4082-9036-305A9F5D2EE8@amazon.com> References: <9C91C7F6-7209-4082-9036-305A9F5D2EE8@amazon.com> Message-ID: <74dd2723182dc01b348eb23cb187cb735e5d8961.camel@redhat.com> Thanks for the reviews, Kelvin! Cheers, Roman > > ?On 7/23/20, 5:51 AM, "Roman Kennke" wrote: > > Hi Kelvin, > > On Thu, 2020-07-23 at 01:56 +0000, Nilsen, Kelvin wrote: > > This is an unofficial review (I am not an official reviewer) of > the > > shared-code changes described in > > > http://cr.openjdk.java.net/~rkennke/shenandoah-jdk11u-upstream/webrev.09-shared/ > > which were posted by "Roman Kennke" (rkennke at redhat.com) on > Mon > > Jul 13 16:06:12 UTC 2020. See > > > https://mail.openjdk.java.net/pipermail/jdk-updates-dev/2020-July/003483.html > > for the original message. > > Thank you, Kelvin for reviewing it! You could have replied to the > original email that you mentioned :-) > > Would have been much better, I realize. There were some email server > problems going on and I never received that original message. > > > For the most part, these all look good and I concur that these > do not > > impact non-Shenandoah deployments except for the possible > concerns > > identified below: > > > > 1. In make/hotspot/gensrc/GensrcJfr.gmk, around line 68, it > might be > > good to list the following ExecuteWithLog arguments in the same > order > > as when Shenandoah is not enabled to make it more clear that > these > > parameters are "not affected": $(METADATA_XML) $(METADATA_XSD) > > $(JFR_OUTPUTDIR) > > > > This is not really possible. The patch also changes > GenerateJfrFiles.java to accept multiple - and optional - > arguments for > metadata.xml, which means that it's most reasonable to place them > at > the end, and not first. > > BTW, I intend to upstream those build changes to jdk16 too. > > Thanks. I'm satisfied. > > > 2. In src/hotspot/share/gc/shared/gc_globals.hpp, around line > 200, it > > looks like the lines: > > product(bool, UseShenandoahGC, false, > > "Use the Shenandoah garbage collector") > > should be included only under #if INCLUDE_SHENANDOAHGC > > conditional compilation control. > > That is not directly possible because I can't put an ifdef in the > middle of a huge define. It can be done though: define the block > under > #if INCLUDE_SHENANDOAHGC above, and empty otherwise, and put that > in > the huge #define. I am doing that in the webrev.10 (see below). > With > that, the last remnants of Shenandoah are purged from libjvm.so > when > building without Shenandoah :-) On the (somewhat) downside, I > also had > to remove -XX:-UseShenandoahGC from TestDisableDefaultGC.java, > that > test would otherwise fail. (It does print 'unrecognized VM > option' now, > instead of "Garbage collector not selected"). > > Thanks. I'm satisfied. > > > 3. In src/hotspot/share/opto/loopnode.cpp, around line 4514, > it is > > not clear from context that the Shenandoah change still > includes all > > of the same code that was present before the Shenandoah > change. In > > particular, it is not clear that !defined(PRODUCT) is > universally > > true. > > > > Before the change, the rpo() method was only included when > !PRODUCT. > Shenandoah requires this method in PRODUCT-builds too, hence the > change. I am not sure how to make this much clearer? > > I see now. My side-by-side diffs were not showing me the condition > that controlled the block newly terminated by the inserted > #endif. Thanks for clarifying. I'm satisfied. > > > 4. Likewise, in src/hotspot/share/opto/loopnode.hpp, around > line > > 1327, the prototype for function void is introduced by > Shenandoah > > enhancements to the code even #if INCLUDE_SHENANDOAHGC in the > case of > > ifndef PRODUCT. > > See comment above. > > Satisfied now that I understand it better. > > > 5. In src/hotspot/share/opto/type.hpp, around lines 1775 and > 1822, do > > we want to #if INCLUDE_SHENANDOAHGC the definitions of > LoadXNode and > > StoreXNode? Should be harmless as is. > > We could, but as you say it's harmless and doesn't generate any > code > per-se. I did it anyway in webrev.00. > > Thanks. I'm satisfied. (Above, I think you mean webrev.10.) > > > 6. In src/hotspot/share/runtime/vmOperations.hpp, around line > 108, do > > we want to #if INCLUDE_SHENANDOAHGC the mentions of > ShenandoahFullGC, > > ShenandoahInitMark, ShenandoahFinalMarkStartEvac, > > ShenadoahInitUpdateRefs, ShenandoahFinalUpdateRefs, > > ShenandoahDegeneratedGC? Should be harmless as is. > > That is exactly what the block above with #if > INCLUDE_SHENANDOAHGC > does. We cannot #if INCLUDE_SHENANDOAHGC in the middle of a large > #define. Hence the little stunt. (BTW, this is exactly the > pattern how > we could exclude the global UseShenandoahGC flag too, see above). > > I see. Thanks. I'm satisfied. > > > 7. In src/hotspot/share/utilities/globalDefinitions.hpp, around > line > > 98, do we want to put the UINT64_FORMAT_X_W(width) macro under > #if > > INCLUDE_SHENANDOAHGC? Harmless as is. > > We could, but as you say it's harmless and doesn't generate any > code > per-se. I did it anyway in webrev.00. > > Thanks. Am satisfied with webrev.10. > > > 8. In src/hotspot/share/utilities/macros.hpp, around line 238, > should > > the definition of NOT_SHENANDOAHGC_RETURN be { return; }? > > I don't think that is necessary. The definitions of other GCs are > not > doing it either, so let's keep it consistent with those, yes? > > I see. Am satisfied. > > > 9. I am not confident I understand how the file > > src/jdk.jfr/share/conf/jfr/default.jfc is compiled. Should > lines 423 > > - 431 be under #if INCLUDE_SHENANDOAHGC control? Does this > cause > > unwanted Shenandoah code to be compiled in even when it's not > > supposed to be? > > This is JFR profiles, it would not compile any code. We ensure > that > Shenandoah events are only compiled when Shenandoah is build- > enabled > (via the metadata.xml machinery mentioned above). Running JFR > with such > a profile on a JDK without Shenandoah support does work as usual. > Those > events are enabled, but simply not present. > > Thanks for explaining. I am satisfied. > > > 10. I am not confident I understand how regression suites are > > configured. In test/hotspot/jtreg/TEST.ROOT and > test/jdk/TEST.ROOT, > > it appears that we are requiring vm.gc.Shenandoah in order to > run > > certain tests. If the JVM is compiled without support for > > Shenandoah, will this cause these regression tests to not > run. That > > would not be good for someone who desires to build JDK 11 > without > > Shenandoah support. > > The test is specifically there to test if Shenandoah is built-in. > Tests > that execute Shenandoah code by enabling -XX:+UseShenandoahGC are > all > guarded by this, so they would not be run when Shenandoah is not > built- > in. See e.g.: > > > http://cr.openjdk.java.net/~rkennke/shenandoah-jdk11u-upstream/webrev.09-shared/test/hotspot/jtreg/gc/TestHumongousReferenceObject.java.udiff.html > > Ok. I was concerned that this might be guarding other tests that did > not depend on Shenandoah. Thanks for explaining. I am satisfied. > > > 11. In test/hotspot/jtreg/TEST.groups, it appears that there > have > > been many Shenandoah-specific tests added to the regression > > suites. If someone builds the JDK11 without Shenandoah support > and > > runs the regression suite, will all of these tests now fail? I > don't > > see how the running of these tests is made > conditional. (Perhaps my > > issue 10 and issue 11 are complementary and between them, all > of my > > concerns are fully addressed.) > > See comment for #10. It is safe to run those suites with > Shenandoah- > build-disabled, it will only run a handful of tests then, and > exclude > all run configurations that have -XX:+UseShenandoahGC in them. > > Thanks for generating the new webrev. Looks good to me. > > Here's the updated webrevs, with UseShenandoahGC flag excluded on > non- > Shenandoah builds, and the two exclusions of macros: > > Shared-only: > > http://cr.openjdk.java.net/~rkennke/shenandoah-jdk11u-upstream/webrev.10-shared/ > Full: > > http://cr.openjdk.java.net/~rkennke/shenandoah-jdk11u-upstream/webrev.10-all/ > > I did run hotspot_gc_shenandoah again on builds with and without > Shenandoah, and it worked fine. > > > > From adityam at microsoft.com Fri Jul 24 02:43:05 2020 From: adityam at microsoft.com (Aditya Mandaleeka) Date: Fri, 24 Jul 2020 02:43:05 +0000 Subject: RFR (11u, XXL): Upstream/backport Shenandoah to JDK11u - the review thread In-Reply-To: References: <3db8e235f3ab515563802d23997f0ac32d699c48.camel@redhat.com> Message-ID: Hi Roman, I took a first pass at reviewing the shared code changes that are not under test/, with an eye toward ways in which the patch could affect Shenandoah-less builds. Things mostly looked good, and many of these are nitpicks. The review below is based on http://cr.openjdk.java.net/~rkennke/shenandoah-jdk11u-upstream/webrev.10-shared/ As a disclaimer, I'm not currently an official reviewer on the project. Thanks, Aditya Here it goes: ============================================================ src/hotspot/share/opto/escape.cpp, ~590: @@ -554,8 +567,8 @@ // Pointer stores in G1 barriers looks like unsafe access. // Ignore such stores to be able scalar replace non-escaping // allocations. -#if INCLUDE_G1GC - if (UseG1GC && adr->is_AddP()) { +#if INCLUDE_G1GC || INCLUDE_SHENANDOAHGC + if ((UseG1GC SHENANDOAHGC_ONLY(|| UseShenandoahGC)) && adr->is_AddP()) { Node* base = get_addp_base(adr); if (base->Opcode() == Op_LoadP && base->in(MemNode::Address)->is_AddP()) { @@ -563,7 +576,15 @@ Node* tls = get_addp_base(adr); if (tls->Opcode() == Op_ThreadLocal) { int offs = (int)igvn->find_intptr_t_con(adr->in(AddPNode::Offset), Type::OffsetBot); - if (offs == in_bytes(G1ThreadLocalData::satb_mark_queue_buffer_offset())) { +#if INCLUDE_G1GC && INCLUDE_SHENANDOAHGC + const int buf_offset = in_bytes(UseG1GC ? G1ThreadLocalData::satb_mark_queue_buffer_offset() + : ShenandoahThreadLocalData::satb_mark_queue_buffer_offset()); +#elif INCLUDE_G1GC + const int buf_offset = in_bytes(G1ThreadLocalData::satb_mark_queue_buffer_offset()); +#else + const int buf_offset = in_bytes(ShenandoahThreadLocalData::satb_mark_queue_buffer_offset()); +#endif + if (offs == buf_offset) { break; // G1 pre barrier previous oop value store. } if (offs == in_bytes(G1ThreadLocalData::dirty_card_queue_buffer_offset())) { ----- The outermost block has been relaxed to be under `#if INCLUDE_G1GC || INCLUDE_SHENANDOAHGC`, but I think that last line above (the call to G1ThreadLocalData::dirty_card_queue_buffer_offset()) won't compile if G1 is disabled, will it? I haven't verified it myself, but you may want to check that. I assume including Shenandoah and not G1 is a valid configuration based on the other checks you added. ============================================================ src/hotspot/cpu/x86/c1_LIRGenerator_x86.cpp, ~681: #ifdef ASSERT @@ -674,6 +675,11 @@ if (type == T_OBJECT || type == T_ARRAY) { cmp_value.load_item_force(FrameMap::rax_oop_opr); new_value.load_item(); +#if INCLUDE_SHENANDOAHGC + if (UseShenandoahGC) { + __ cas_obj(addr->as_address_ptr()->base(), cmp_value.result(), new_value.result(), new_register(T_OBJECT), new_register(T_OBJECT)); + } else +#endif __ cas_obj(addr->as_address_ptr()->base(), cmp_value.result(), new_value.result(), ill, ill); } else if (type == T_INT) { ----- The hanging brace-less else scares me a bit since someone might add something else after the endif and unknowingly cause problems for Shenandoah builds. ============================================================ src/hotspot/cpu/x86/c1_LIRGenerator_x86.cpp, ~708: @@ -699,6 +705,12 @@ // Because we want a 2-arg form of xchg and xadd __ move(value.result(), result); assert(type == T_INT || is_oop LP64_ONLY( || type == T_LONG ), "unexpected type"); +#if INCLUDE_SHENANDOAHGC + if (UseShenandoahGC) { + LIR_Opr tmp = is_oop ? new_register(type) : LIR_OprFact::illegalOpr; + __ xchg(addr, result, result, tmp); + } else +#endif __ xchg(addr, result, result, LIR_OprFact::illegalOpr); return result; } ----- This is another case like the above (and in the same file). I understand there's a motovation to keep the non-Shenandoah code delta minimal, but something like this seems much nicer: LIR_Opr tmp = LIR_OprFact::illegalOpr; #if INCLUDE_SHENANDOAHGC if (UseShenandoahGC && is_oop) { tmp = new_register(type); #endif __ xchg(addr, result, result, tmp); return result; ============================================================ src/hotspot/cpu/x86/sharedRuntime_x86_64.cpp, two places @@ -2234,6 +2245,14 @@ switch (in_sig_bt[i]) { case T_ARRAY: if (is_critical_native) { +#if INCLUDE_SHENANDOAHGC + // pin before unpack + if (UseShenandoahGC) { + assert(pinned_slot <= stack_slots, "overflow"); + ShenandoahBarrierSet::assembler()->pin_critical_native_array(masm, in_regs[i], pinned_slot); + pinned_args.append(i); + } +#endif @@ -2450,6 +2469,22 @@ default : ShouldNotReachHere(); } +#if INCLUDE_SHENANDOAHGC + if (UseShenandoahGC) { + // unpin pinned arguments + pinned_slot = oop_handle_offset; + if (pinned_args.length() > 0) { + // save return value that may be overwritten otherwise. + save_native_result(masm, ret_type, stack_slots); + for (int index = 0; index < pinned_args.length(); index ++) { + int i = pinned_args.at(index); + assert(pinned_slot <= stack_slots, "overflow"); + ShenandoahBarrierSet::assembler()->unpin_critical_native_array(masm, in_regs[i], pinned_slot); + } + restore_native_result(masm, ret_type, stack_slots); + } + } +#endif ----- Minor issue: There are tab characters used in indenting the ShenandoahBarrierSet* lines, inconsistent with the surrounding code. ============================================================ Various files - * Copyright (c) 1997, 2017, Oracle and/or its affiliates. All rights reserved. + * Copyright (c) 1997, 2018, Oracle and/or its affiliates. All rights reserved. ----- What's the normal policy around how these years are updated? If the change is being made now, should they be 2020? ============================================================ src/hotspot/share/gc/shared/gcConfig.cpp, ~60 + CMSGC_ONLY(static CMSArguments cmsArguments;) + EPSILONGC_ONLY(static EpsilonArguments epsilonArguments;) + G1GC_ONLY(static G1Arguments g1Arguments;) + PARALLELGC_ONLY(static ParallelArguments parallelArguments;) + SERIALGC_ONLY(static SerialArguments serialArguments;) +SHENANDOAHGC_ONLY(static ShenandoahArguments shenandoahArguments;) + ZGC_ONLY(static ZArguments zArguments;) ----- Minor issue: alignment of the others needs to be adjusted ============================================================ src/hotspot/share/gc/shared/gcConfig.cpp, ~98 @@ -90,14 +95,14 @@ bool GCConfig::_gc_selected_ergonomically = false; void GCConfig::fail_if_unsupported_gc_is_selected() { - NOT_CMSGC( FAIL_IF_SELECTED(UseConcMarkSweepGC, true)); - NOT_EPSILONGC( FAIL_IF_SELECTED(UseEpsilonGC, true)); - NOT_G1GC( FAIL_IF_SELECTED(UseG1GC, true)); - NOT_PARALLELGC(FAIL_IF_SELECTED(UseParallelGC, true)); - NOT_PARALLELGC(FAIL_IF_SELECTED(UseParallelOldGC, true)); - NOT_SERIALGC( FAIL_IF_SELECTED(UseSerialGC, true)); - NOT_SERIALGC( FAIL_IF_SELECTED(UseParallelOldGC, false)); - NOT_ZGC( FAIL_IF_SELECTED(UseZGC, true)); + NOT_CMSGC( FAIL_IF_SELECTED(UseConcMarkSweepGC, true)); + NOT_EPSILONGC( FAIL_IF_SELECTED(UseEpsilonGC, true)); + NOT_G1GC( FAIL_IF_SELECTED(UseG1GC, true)); + NOT_PARALLELGC( FAIL_IF_SELECTED(UseParallelGC, true)); + NOT_PARALLELGC( FAIL_IF_SELECTED(UseParallelOldGC, true)); + NOT_SERIALGC( FAIL_IF_SELECTED(UseSerialGC, true)); + NOT_SERIALGC( FAIL_IF_SELECTED(UseParallelOldGC, false)); + NOT_ZGC( FAIL_IF_SELECTED(UseZGC, true)); ----- Minor issue: These whitespace changes can be undone. (I guess a prior version had a Shenandoah thing added here which has now been removed?) ============================================================ src/hotspot/share/runtime/stackValue.cpp, ~114 - // Decode narrowoop and wrap a handle around the oop - Handle h(Thread::current(), CompressedOops::decode(value.noop)); + // Decode narrowoop + oop val = CompressedOops::decode(value.noop); + // Deoptimization must make sure all oops have passed load barriers +#if INCLUDE_SHENANDOAHGC + if (UseShenandoahGC) { + val = ShenandoahBarrierSet::barrier_set()->load_reference_barrier(val); + } +#endif + Handle h(Thread::current(), val); // Wrap a handle around the oop ----- I'd move the load barrier comment into the `if (UseShenandoahGC)` for better readability. ============================================================ src/hotspot/share/gc/shared/vmStructs_gc.hpp, three places SERIALGC_ONLY(VM_STRUCTS_SERIALGC(nonstatic_field, \ volatile_nonstatic_field, \ static_field)) \ + SHENANDOAHGC_ONLY(VM_STRUCTS_SHENANDOAH(nonstatic_field, \ + volatile_nonstatic_field, \ + static_field)) ----- Ultra nit-pick: The others have the second and third arguments aligned with the first. There are three places in this file where you've added the Shenandoah ones and they're not aligned this way. Hey, I warned you some of these comments are minor :). From nick.gasson at arm.com Fri Jul 24 05:57:27 2020 From: nick.gasson at arm.com (Nick Gasson) Date: Fri, 24 Jul 2020 13:57:27 +0800 Subject: [11u] RFR: 8248845: AArch64: stack corruption after spilling vector register Message-ID: <85imed1mm0.fsf@nicgas01-pc.shanghai.arm.com> Hi, Is anyone able to sponsor this backport for me? Bug: https://bugs.openjdk.java.net/browse/JDK-8248845 Original change: https://hg.openjdk.java.net/jdk/jdk15/rev/d5be95758352 Review thread: https://mail.openjdk.java.net/pipermail/hotspot-compiler-dev/2020-July/038886.html The patch does not apply cleanly on 11u as the ScheduleAndBundle function has been moved and renamed. I've prepared another webrev that applies on 11u: http://cr.openjdk.java.net/~ngasson/8248845/webrev.11u.0/ Tested tier1 on AArch64. -- Thanks, Nick From aph at redhat.com Fri Jul 24 09:48:21 2020 From: aph at redhat.com (Andrew Haley) Date: Fri, 24 Jul 2020 10:48:21 +0100 Subject: [11u] RFR: JDK-8222074: Enhance auto vectorization for x86 In-Reply-To: References: Message-ID: On 24/06/2020 18:44, Hohensee, Paul wrote: > We would like to backport aarch64 performance work because > ARM64-based server are being rapidly adopted for high performance > floating point applications. JDK 11 will be the primary Java > execution platform for them for some time to come. While I have some sympathy for this, please remember that JDK 11 is a long-term maintenance version. The ratio of reward to risk must be high, and therefore the benefits of large changes must be substantial. Many people depend on us not to break anything in production. AArch64 is, for most people, a new architecture and therefore it's reasonable (to some extent) for it to be enhanced in JDK 11, if only for parity with x86. The same cannot be said of x86, where the ultimate stability is essential. Multi-thousand-line backports for non-bugs should be very rare indeed. Is the gain really it? note that, whatever the motivation, this patch must be justified for x86. Also, although we really need to build confidence in the AArch64 platform, and there's nothing like regressions to destroy trust. Small performance improvements help a little, but crashes are disastrous. Be careful out there. -- Andrew Haley (he/him) Java Platform Lead Engineer Red Hat UK Ltd. https://keybase.io/andrewhaley EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From rkennke at redhat.com Fri Jul 24 11:50:55 2020 From: rkennke at redhat.com (Roman Kennke) Date: Fri, 24 Jul 2020 13:50:55 +0200 Subject: RFR (11u, XXL): Upstream/backport Shenandoah to JDK11u - the review thread In-Reply-To: References: <3db8e235f3ab515563802d23997f0ac32d699c48.camel@redhat.com> Message-ID: <811b7b06d3aea039b49b8880a2c80423e9c3c287.camel@redhat.com> Hi Aditya, thank you for reviewing! See my comments in-inline: > Hi Roman, > > I took a first pass at reviewing the shared code changes that are not > under test/, with an eye toward > ways in which the patch could affect Shenandoah-less builds. Things > mostly looked good, and many of these > are nitpicks. > > The review below is based on > http://cr.openjdk.java.net/~rkennke/shenandoah-jdk11u-upstream/webrev.10-shared/ > > As a disclaimer, I'm not currently an official reviewer on the > project. > > Thanks, > Aditya > > Here it goes: > > ============================================================ > src/hotspot/share/opto/escape.cpp, ~590: > > @@ -554,8 +567,8 @@ > // Pointer stores in G1 barriers looks like unsafe access. > // Ignore such stores to be able scalar replace non- > escaping > // allocations. > -#if INCLUDE_G1GC > - if (UseG1GC && adr->is_AddP()) { > +#if INCLUDE_G1GC || INCLUDE_SHENANDOAHGC > + if ((UseG1GC SHENANDOAHGC_ONLY(|| UseShenandoahGC)) && > adr->is_AddP()) { > Node* base = get_addp_base(adr); > if (base->Opcode() == Op_LoadP && > base->in(MemNode::Address)->is_AddP()) { > > @@ -563,7 +576,15 @@ > Node* tls = get_addp_base(adr); > if (tls->Opcode() == Op_ThreadLocal) { > int offs = (int)igvn->find_intptr_t_con(adr- > >in(AddPNode::Offset), Type::OffsetBot); > - if (offs == > in_bytes(G1ThreadLocalData::satb_mark_queue_buffer_offset())) { > +#if INCLUDE_G1GC && INCLUDE_SHENANDOAHGC > + const int buf_offset = in_bytes(UseG1GC ? > G1ThreadLocalData::satb_mark_queue_buffer_offset() > + : > ShenandoahThreadLocalData::satb_mark_queue_buffer_offset()); > +#elif INCLUDE_G1GC > + const int buf_offset = > in_bytes(G1ThreadLocalData::satb_mark_queue_buffer_offset()); > +#else > + const int buf_offset = > in_bytes(ShenandoahThreadLocalData::satb_mark_queue_buffer_offset()); > +#endif > + if (offs == buf_offset) { > break; // G1 pre barrier previous oop value store. > } > if (offs == > in_bytes(G1ThreadLocalData::dirty_card_queue_buffer_offset())) { > > ----- > > The outermost block has been relaxed to be under `#if > INCLUDE_G1GC || INCLUDE_SHENANDOAHGC`, but I think > that last line above (the call to > G1ThreadLocalData::dirty_card_queue_buffer_offset()) won't compile if > G1 > is disabled, will it? I haven't verified it myself, but you may > want to check that. I assume including > Shenandoah and not G1 is a valid configuration based on the other > checks you added. > I think you're missing the final #endif after the block that you showed here. The one that closes the #if INCLUDE_G1GC || INCLUDE_SHENANDOAHGC. I think it's ok. Also notice that G1ThreadLocalData::dirty_card_queue_buffer_offset() will never happen with Shenandoah, we don't use the dirty_card_queue. > ============================================================ > src/hotspot/cpu/x86/c1_LIRGenerator_x86.cpp, ~681: > > #ifdef ASSERT > @@ -674,6 +675,11 @@ > if (type == T_OBJECT || type == T_ARRAY) { > cmp_value.load_item_force(FrameMap::rax_oop_opr); > new_value.load_item(); > +#if INCLUDE_SHENANDOAHGC > + if (UseShenandoahGC) { > + __ cas_obj(addr->as_address_ptr()->base(), cmp_value.result(), > new_value.result(), new_register(T_OBJECT), new_register(T_OBJECT)); > + } else > +#endif > __ cas_obj(addr->as_address_ptr()->base(), cmp_value.result(), > new_value.result(), ill, ill); > } else if (type == T_INT) { > > > ----- > > The hanging brace-less else scares me a bit since someone might > add something else after the > endif and unknowingly cause problems for Shenandoah builds. > Yes. We've actually had what you propose below. For some reason it compiles slightly different code, and my comparison script would complain about it. The alternative would be to put in extra #if for the closing }, but that looks even messier. > ============================================================ > src/hotspot/cpu/x86/c1_LIRGenerator_x86.cpp, ~708: > > @@ -699,6 +705,12 @@ > // Because we want a 2-arg form of xchg and xadd > __ move(value.result(), result); > assert(type == T_INT || is_oop LP64_ONLY( || type == T_LONG ), > "unexpected type"); > +#if INCLUDE_SHENANDOAHGC > + if (UseShenandoahGC) { > + LIR_Opr tmp = is_oop ? new_register(type) : > LIR_OprFact::illegalOpr; > + __ xchg(addr, result, result, tmp); > + } else > +#endif > __ xchg(addr, result, result, LIR_OprFact::illegalOpr); > return result; > } > > ----- > > This is another case like the above (and in the same file). > > I understand there's a motovation to keep the non-Shenandoah code > delta minimal, but something like this seems much nicer: > > > LIR_Opr tmp = LIR_OprFact::illegalOpr; > > #if INCLUDE_SHENANDOAHGC > if (UseShenandoahGC && is_oop) { > tmp = new_register(type); > #endif > > __ xchg(addr, result, result, tmp); > return result; > Yes, but see above. > ============================================================ > src/hotspot/cpu/x86/sharedRuntime_x86_64.cpp, two places > > @@ -2234,6 +2245,14 @@ > switch (in_sig_bt[i]) { > case T_ARRAY: > if (is_critical_native) { > +#if INCLUDE_SHENANDOAHGC > + // pin before unpack > + if (UseShenandoahGC) { > + assert(pinned_slot <= stack_slots, "overflow"); > + ShenandoahBarrierSet::assembler()- > >pin_critical_native_array(masm, in_regs[i], pinned_slot); > + pinned_args.append(i); > + } > +#endif > > @@ -2450,6 +2469,22 @@ > default : ShouldNotReachHere(); > } > > +#if INCLUDE_SHENANDOAHGC > + if (UseShenandoahGC) { > + // unpin pinned arguments > + pinned_slot = oop_handle_offset; > + if (pinned_args.length() > 0) { > + // save return value that may be overwritten otherwise. > + save_native_result(masm, ret_type, stack_slots); > + for (int index = 0; index < pinned_args.length(); index ++) { > + int i = pinned_args.at(index); > + assert(pinned_slot <= stack_slots, "overflow"); > + ShenandoahBarrierSet::assembler()- > >unpin_critical_native_array(masm, in_regs[i], pinned_slot); > + } > + restore_native_result(masm, ret_type, stack_slots); > + } > + } > +#endif > > ----- > > Minor issue: There are tab characters used in indenting the > ShenandoahBarrierSet* lines, inconsistent > with the surrounding code. Good spot! Fixed in webrev.11 (see below). > ============================================================ > Various files > > - * Copyright (c) 1997, 2017, Oracle and/or its affiliates. All > rights reserved. > + * Copyright (c) 1997, 2018, Oracle and/or its affiliates. All > rights reserved. > > ----- > > What's the normal policy around how these years are updated? If > the change is being made now, should > they be 2020? Those copyright updates stem from when we actually made those changes in shenandoah/jdk11 repository. But we should decide what to do with this? Leave it like I proposed (i.e. according to original changes from shenandoah/jdk11), or remove all copyright changes, or update all touched files to 2020? Also, we should agree on the commit message if this gets approved. I currently set it to just 'Shenandoah: A Low-Pause-Time Garbage Collector'. Does it require a bug-ID? If yes, then which one? A backport-issue of what? The original Shenandoah JEP 189 implementation? A backport of the new JEP 379, which seems more fitting in the title, and also represents the current state of Shenandoah overall better, but a wholly different patch in jdk11u? > ============================================================ > src/hotspot/share/gc/shared/gcConfig.cpp, ~60 > > + CMSGC_ONLY(static CMSArguments cmsArguments;) > + EPSILONGC_ONLY(static EpsilonArguments epsilonArguments;) > + G1GC_ONLY(static G1Arguments g1Arguments;) > + PARALLELGC_ONLY(static ParallelArguments parallelArguments;) > + SERIALGC_ONLY(static SerialArguments serialArguments;) > +SHENANDOAHGC_ONLY(static ShenandoahArguments shenandoahArguments;) > + ZGC_ONLY(static ZArguments zArguments;) > > ----- > > Minor issue: alignment of the others needs to be adjusted Right. Fixed in webrev.11. > ============================================================ > src/hotspot/share/gc/shared/gcConfig.cpp, ~98 > > @@ -90,14 +95,14 @@ > bool GCConfig::_gc_selected_ergonomically = false; > > void GCConfig::fail_if_unsupported_gc_is_selected() { > - NOT_CMSGC( FAIL_IF_SELECTED(UseConcMarkSweepGC, true)); > - NOT_EPSILONGC( FAIL_IF_SELECTED(UseEpsilonGC, true)); > - NOT_G1GC( FAIL_IF_SELECTED(UseG1GC, true)); > - NOT_PARALLELGC(FAIL_IF_SELECTED(UseParallelGC, true)); > - NOT_PARALLELGC(FAIL_IF_SELECTED(UseParallelOldGC, true)); > - NOT_SERIALGC( FAIL_IF_SELECTED(UseSerialGC, true)); > - NOT_SERIALGC( FAIL_IF_SELECTED(UseParallelOldGC, false)); > - NOT_ZGC( FAIL_IF_SELECTED(UseZGC, true)); > + NOT_CMSGC( FAIL_IF_SELECTED(UseConcMarkSweepGC, true)); > + NOT_EPSILONGC( FAIL_IF_SELECTED(UseEpsilonGC, true)); > + NOT_G1GC( FAIL_IF_SELECTED(UseG1GC, true)); > + NOT_PARALLELGC( FAIL_IF_SELECTED(UseParallelGC, true)); > + NOT_PARALLELGC( FAIL_IF_SELECTED(UseParallelOldGC, true)); > + NOT_SERIALGC( FAIL_IF_SELECTED(UseSerialGC, true)); > + NOT_SERIALGC( FAIL_IF_SELECTED(UseParallelOldGC, false)); > + NOT_ZGC( FAIL_IF_SELECTED(UseZGC, true)); > > ----- > > Minor issue: These whitespace changes can be undone. (I guess a > prior version had a Shenandoah thing > added here which has now been removed?) Right. Fixed in webrev.11. > ============================================================ > src/hotspot/share/runtime/stackValue.cpp, ~114 > > - // Decode narrowoop and wrap a handle around the oop > - Handle h(Thread::current(), > CompressedOops::decode(value.noop)); > + // Decode narrowoop > + oop val = CompressedOops::decode(value.noop); > + // Deoptimization must make sure all oops have passed load > barriers > +#if INCLUDE_SHENANDOAHGC > + if (UseShenandoahGC) { > + val = ShenandoahBarrierSet::barrier_set()- > >load_reference_barrier(val); > + } > +#endif > + Handle h(Thread::current(), val); // Wrap a handle around the > oop > > ----- > > I'd move the load barrier comment into the `if (UseShenandoahGC)` > for better readability. > Right. Fixed in webrev.11. > ============================================================ > src/hotspot/share/gc/shared/vmStructs_gc.hpp, three places > > SERIALGC_ONLY(VM_STRUCTS_SERIALGC(nonstatic_field, > \ > volatile_nonstatic_field, > \ > static_field)) > \ > + SHENANDOAHGC_ONLY(VM_STRUCTS_SHENANDOAH(nonstatic_field, > \ > + volatile_nonstatic_field, > \ > + static_field)) > > ----- > > Ultra nit-pick: The others have the second and third arguments > aligned with the first. There > are three places in this file where you've added the Shenandoah > ones and they're not aligned > this way. Hey, I warned you some of these comments are minor :). Right. The last one doesn't quite fit, and I'm not going to change *all* of the trailing \s. Fixed in webrev.11. Full: Shared-only: Testing: re-did my object-file comparison. Still clean. Built with and without Shenandoah enabled, run hotspot_gc_shenandoah fine. Thank you, Roman From adityam at microsoft.com Fri Jul 24 22:23:09 2020 From: adityam at microsoft.com (Aditya Mandaleeka) Date: Fri, 24 Jul 2020 22:23:09 +0000 Subject: RFR (11u, XXL): Upstream/backport Shenandoah to JDK11u - the review thread In-Reply-To: <811b7b06d3aea039b49b8880a2c80423e9c3c287.camel@redhat.com> References: <3db8e235f3ab515563802d23997f0ac32d699c48.camel@redhat.com> <811b7b06d3aea039b49b8880a2c80423e9c3c287.camel@redhat.com> Message-ID: Hey Roman, Thanks for addressing the comments! Just following up on one of them: > > > The outermost block has been relaxed to be under `#if > > INCLUDE_G1GC || INCLUDE_SHENANDOAHGC`, but I think > > that last line above (the call to > > G1ThreadLocalData::dirty_card_queue_buffer_offset()) won't compile if > > G1 > > is disabled, will it? I haven't verified it myself, but you may > > want to check that. I assume including > > Shenandoah and not G1 is a valid configuration based on the other > > checks you added. > > > > I think you're missing the final #endif after the block that you showed > here. The one that closes the #if INCLUDE_G1GC || INCLUDE_SHENANDOAHGC. > I think it's ok. Also notice that > G1ThreadLocalData::dirty_card_queue_buffer_offset() will never happen > with Shenandoah, we don't use the dirty_card_queue. I think I should have explained the issue I see better. Here is the relevant code from escape.cpp again, after the patch: #if INCLUDE_G1GC || INCLUDE_SHENANDOAHGC if ((UseG1GC SHENANDOAHGC_ONLY(|| UseShenandoahGC)) && adr->is_AddP()) { Node* base = get_addp_base(adr); if (base->Opcode() == Op_LoadP && base->in(MemNode::Address)->is_AddP()) { adr = base->in(MemNode::Address); Node* tls = get_addp_base(adr); if (tls->Opcode() == Op_ThreadLocal) { int offs = (int)igvn->find_intptr_t_con(adr->in(AddPNode::Offset), Type::OffsetBot); #if INCLUDE_G1GC && INCLUDE_SHENANDOAHGC const int buf_offset = in_bytes(UseG1GC ? G1ThreadLocalData::satb_mark_queue_buffer_offset() : ShenandoahThreadLocalData::satb_mark_queue_buffer_offset()); #elif INCLUDE_G1GC const int buf_offset = in_bytes(G1ThreadLocalData::satb_mark_queue_buffer_offset()); #else const int buf_offset = in_bytes(ShenandoahThreadLocalData::satb_mark_queue_buffer_offset()); #endif if (offs == buf_offset) { break; // G1 pre barrier previous oop value store. } if (offs == in_bytes(G1ThreadLocalData::dirty_card_queue_buffer_offset())) { break; // G1 post barrier card address store. } } } } #endif That last call to G1ThreadLocalData::dirty_card_queue_buffer_offset() is guarded only by the `#if INCLUDE_G1GC || INCLUDE_SHENANDOAHGC` so it can be included in the compilation unit even when INCLUDE_G1GC is off (as long as INCLUDE_SHENANDOAHGC is on), and thus cause a build break. I think that part needs to also be guarded under #if INCLUDE_G1GC. -Aditya From rkennke at redhat.com Mon Jul 27 15:59:04 2020 From: rkennke at redhat.com (Roman Kennke) Date: Mon, 27 Jul 2020 17:59:04 +0200 Subject: RFR (11u, XXL): Upstream/backport Shenandoah to JDK11u - the review thread In-Reply-To: References: <3db8e235f3ab515563802d23997f0ac32d699c48.camel@redhat.com> <811b7b06d3aea039b49b8880a2c80423e9c3c287.camel@redhat.com> Message-ID: <3e5ffe74040ceeac11238482ef5c73c0cf00ee35.camel@redhat.com> On Fri, 2020-07-24 at 22:23 +0000, Aditya Mandaleeka wrote: > Hey Roman, > > Thanks for addressing the comments! Just following up on one of them: > > > > The outermost block has been relaxed to be under `#if > > > INCLUDE_G1GC || INCLUDE_SHENANDOAHGC`, but I think > > > that last line above (the call to > > > G1ThreadLocalData::dirty_card_queue_buffer_offset()) won't > > > compile if > > > G1 > > > is disabled, will it? I haven't verified it myself, but you > > > may > > > want to check that. I assume including > > > Shenandoah and not G1 is a valid configuration based on the > > > other > > > checks you added. > > > > > > > I think you're missing the final #endif after the block that you > > showed > > here. The one that closes the #if INCLUDE_G1GC || > > INCLUDE_SHENANDOAHGC. > > I think it's ok. Also notice that > > G1ThreadLocalData::dirty_card_queue_buffer_offset() will never > > happen > > with Shenandoah, we don't use the dirty_card_queue. > > I think I should have explained the issue I see better. > > Here is the relevant code from escape.cpp again, after the patch: > > #if INCLUDE_G1GC || INCLUDE_SHENANDOAHGC > if ((UseG1GC SHENANDOAHGC_ONLY(|| UseShenandoahGC)) > && adr->is_AddP()) { > Node* base = get_addp_base(adr); > if (base->Opcode() == Op_LoadP && > base->in(MemNode::Address)->is_AddP()) { > adr = base->in(MemNode::Address); > Node* tls = get_addp_base(adr); > if (tls->Opcode() == Op_ThreadLocal) { > int offs = (int)igvn->find_intptr_t_con(adr- > >in(AddPNode::Offset), Type::OffsetBot); > #if INCLUDE_G1GC && INCLUDE_SHENANDOAHGC > const int buf_offset = in_bytes(UseG1GC ? > G1ThreadLocalData::satb_mark_queue_buffer_offset() > : > ShenandoahThreadLocalData::satb_mark_queue_buffer_offset()); > #elif INCLUDE_G1GC > const int buf_offset = > in_bytes(G1ThreadLocalData::satb_mark_queue_buffer_offset()); > #else > const int buf_offset = > in_bytes(ShenandoahThreadLocalData::satb_mark_queue_buffer_offset()); > #endif > if (offs == buf_offset) { > break; // G1 pre barrier previous oop value > store. > } > if (offs == > in_bytes(G1ThreadLocalData::dirty_card_queue_buffer_offset())) { > break; // G1 post barrier card address > store. > } > } > } > } > #endif > > That last call to G1ThreadLocalData::dirty_card_queue_buffer_offset() > is guarded only by > the `#if INCLUDE_G1GC || INCLUDE_SHENANDOAHGC` so it can be included > in the compilation > unit even when INCLUDE_G1GC is off (as long as INCLUDE_SHENANDOAHGC > is on), and thus cause > a build break. I think that part needs to also be guarded under #if > INCLUDE_G1GC. Indeed! That mess of ifdefs is indeed a little complicated, it is so much nicer in JDK12+ where we have proper GC interfaces for that stuff. But that would be too much to backport now. (And who disables G1 and enables Shenandoah from their builds anyway? :-P ) Here comes webrev12, I only added the #if-block around G1-code: Shared-only: http://cr.openjdk.java.net/~rkennke/shenandoah-jdk11u-upstream/webrev.12-shared/ Full: http://cr.openjdk.java.net/~rkennke/shenandoah-jdk11u-upstream/webrev.12-all/ Thanks, Roman From rkennke at redhat.com Wed Jul 29 11:32:47 2020 From: rkennke at redhat.com (Roman Kennke) Date: Wed, 29 Jul 2020 13:32:47 +0200 Subject: RFR(11u): 8222079: Don't use memset to initialize fields decode_env constructor in disassembler.cpp Message-ID: Can I please get a review for the following 11u-backport: http://cr.openjdk.java.net/~rkennke/JDK-8222079-jdk11u/webrev.00/ ? The original change is considerably different (and more complicated because there are more constructors to take care of): http://hg.openjdk.java.net/jdk/jdk/rev/963924f1c891 The change is useful in 11u because it removes a warning (that is treated like an error by default) when compiling with GCC9+. Testing: tier1 & tier2 without regressions Is this ok for 11u? Thanks, Roman From volker.simonis at gmail.com Wed Jul 29 13:55:42 2020 From: volker.simonis at gmail.com (Volker Simonis) Date: Wed, 29 Jul 2020 15:55:42 +0200 Subject: [11u] RFR(XS): 8234011: (zipfs) Memory leak in ZipFileSystem.releaseDeflater() Message-ID: : Hi, can I please have a review for the following tiny downport? The fix itself itself is trivial and applies cleanly. For the test I had to change two lines because of a missing method in jdk11 compared to 14. Original bug and fix: https://bugs.openjdk.java.net/browse/JDK-8234011 https://hg.openjdk.java.net/jdk/jdk/rev/fe87a92570db Complete 11u downport: http://cr.openjdk.java.net/~simonis/webrevs/2020/8234011_11u/ Changes required for 11u downport: http://cr.openjdk.java.net/~simonis/webrevs/2020/8234011_11u_changes/ In jdk 11 we don't have the `FileSystems.newFileSystem(..)` variant which takes a `Path` as a first argument so we have to manually construct a corresponding URI first. Also notice the small change from `Map.of("create", true)` to `Map.of("create", "true")` which isn't strictly necessary for OpenJDK 11u but required for Oracle jdk 11 because they haven't downported JDK-8034802 yet (and we want to be nice to our Oracle colleagues just in case they want to take this downport as well :) Thank you and best regards, Volker [1] https://bugs.openjdk.java.net/browse/JDK-8034802 From aph at redhat.com Wed Jul 29 13:56:58 2020 From: aph at redhat.com (Andrew Haley) Date: Wed, 29 Jul 2020 14:56:58 +0100 Subject: RFR (11u, XXL): Upstream/backport Shenandoah to JDK11u - the review thread In-Reply-To: <3e5ffe74040ceeac11238482ef5c73c0cf00ee35.camel@redhat.com> References: <3db8e235f3ab515563802d23997f0ac32d699c48.camel@redhat.com> <811b7b06d3aea039b49b8880a2c80423e9c3c287.camel@redhat.com> <3e5ffe74040ceeac11238482ef5c73c0cf00ee35.camel@redhat.com> Message-ID: <34fd5bfb-0881-2b02-3311-22d5ee632042@redhat.com> On 27/07/2020 16:59, Roman Kennke wrote:> Indeed! That mess of ifdefs is indeed a little complicated, it is so > much nicer in JDK12+ where we have proper GC interfaces for that stuff. > But that would be too much to backport now. (And who disables G1 and > enables Shenandoah from their builds anyway? :-P ) > > Here comes webrev12, I only added the #if-block around G1-code: > > Shared-only: > http://cr.openjdk.java.net/~rkennke/shenandoah-jdk11u-upstream/webrev.12-shared/ > Full: > http://cr.openjdk.java.net/~rkennke/shenandoah-jdk11u-upstream/webrev.12-all/ Thanks. I think I may say without reasonable fear of successful contradiction that you've done enough to show that this patch does no harm. Also, by demonstrating that it's possible to add a feature in such a way that it has no effect (on the generated binary) unless enabled you've shown a way to solve a difficult problem. I don't wish to encourage more feature backports to JDK 11, and I don't want to see the source code turn into an unmaintainable mess of #ifdefs. However, in a few rare cases this technique well the closest thing to satisfying both the proponents and the opponents of a backport. You're good to go. Thank you for your hard work and patience. -- Andrew Haley (he/him) Java Platform Lead Engineer Red Hat UK Ltd. https://keybase.io/andrewhaley EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From aph at redhat.com Wed Jul 29 13:59:19 2020 From: aph at redhat.com (Andrew Haley) Date: Wed, 29 Jul 2020 14:59:19 +0100 Subject: RFR(11u): 8222079: Don't use memset to initialize fields decode_env constructor in disassembler.cpp In-Reply-To: References: Message-ID: On 29/07/2020 12:32, Roman Kennke wrote: > Can I please get a review for the following 11u-backport: > http://cr.openjdk.java.net/~rkennke/JDK-8222079-jdk11u/webrev.00/ > ? > > The original change is considerably different (and more complicated > because there are more constructors to take care of): > http://hg.openjdk.java.net/jdk/jdk/rev/963924f1c891 > > The change is useful in 11u because it removes a warning (that is > treated like an error by default) when compiling with GCC9+. I think it's worse than that, and it's actually UB. > Testing: tier1 & tier2 without regressions > > Is this ok for 11u? Yes. -- Andrew Haley (he/him) Java Platform Lead Engineer Red Hat UK Ltd. https://keybase.io/andrewhaley EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From hohensee at amazon.com Wed Jul 29 16:37:40 2020 From: hohensee at amazon.com (Hohensee, Paul) Date: Wed, 29 Jul 2020 16:37:40 +0000 Subject: [11u] RFR(XS): 8234011: (zipfs) Memory leak in ZipFileSystem.releaseDeflater() Message-ID: <937B1C30-220F-4C02-BF15-3927875C1850@amazon.com> Lgtm. :) Thanks, Paul ?On 7/29/20, 6:58 AM, "jdk-updates-dev on behalf of Volker Simonis" wrote: : Hi, can I please have a review for the following tiny downport? The fix itself itself is trivial and applies cleanly. For the test I had to change two lines because of a missing method in jdk11 compared to 14. Original bug and fix: https://bugs.openjdk.java.net/browse/JDK-8234011 https://hg.openjdk.java.net/jdk/jdk/rev/fe87a92570db Complete 11u downport: http://cr.openjdk.java.net/~simonis/webrevs/2020/8234011_11u/ Changes required for 11u downport: http://cr.openjdk.java.net/~simonis/webrevs/2020/8234011_11u_changes/ In jdk 11 we don't have the `FileSystems.newFileSystem(..)` variant which takes a `Path` as a first argument so we have to manually construct a corresponding URI first. Also notice the small change from `Map.of("create", true)` to `Map.of("create", "true")` which isn't strictly necessary for OpenJDK 11u but required for Oracle jdk 11 because they haven't downported JDK-8034802 yet (and we want to be nice to our Oracle colleagues just in case they want to take this downport as well :) Thank you and best regards, Volker [1] https://bugs.openjdk.java.net/browse/JDK-8034802 From rkennke at redhat.com Wed Jul 29 17:36:05 2020 From: rkennke at redhat.com (Roman Kennke) Date: Wed, 29 Jul 2020 19:36:05 +0200 Subject: RFR (11u, XXL): Upstream/backport Shenandoah to JDK11u - the review thread In-Reply-To: <34fd5bfb-0881-2b02-3311-22d5ee632042@redhat.com> References: <3db8e235f3ab515563802d23997f0ac32d699c48.camel@redhat.com> <811b7b06d3aea039b49b8880a2c80423e9c3c287.camel@redhat.com> <3e5ffe74040ceeac11238482ef5c73c0cf00ee35.camel@redhat.com> <34fd5bfb-0881-2b02-3311-22d5ee632042@redhat.com> Message-ID: <6484cf659fbf7ec42db35d9e1d619a0cf2e5210a.camel@redhat.com> On Wed, 2020-07-29 at 14:56 +0100, Andrew Haley wrote: > On 27/07/2020 16:59, Roman Kennke wrote:> Indeed! That mess of ifdefs > is indeed a little complicated, it is so > > much nicer in JDK12+ where we have proper GC interfaces for that > > stuff. > > But that would be too much to backport now. (And who disables G1 > > and > > enables Shenandoah from their builds anyway? :-P ) > > > > Here comes webrev12, I only added the #if-block around G1-code: > > > > Shared-only: > > http://cr.openjdk.java.net/~rkennke/shenandoah-jdk11u-upstream/webrev.12-shared/ > > Full: > > http://cr.openjdk.java.net/~rkennke/shenandoah-jdk11u-upstream/webrev.12-all/ > > Thanks. > > I think I may say without reasonable fear of successful contradiction > that you've done enough to show that this patch does no harm. Also, > by > demonstrating that it's possible to add a feature in such a way that > it has no effect (on the generated binary) unless enabled you've > shown > a way to solve a difficult problem. > > I don't wish to encourage more feature backports to JDK 11, and I > don't want to see the source code turn into an unmaintainable mess of > #ifdefs. However, in a few rare cases this technique well the closest > thing to satisfying both the proponents and the opponents of a > backport. > > You're good to go. Thank you for your hard work and patience. Thank you! For the record, the final patch/webrev that I'll be pushing is this: http://cr.openjdk.java.net/~rkennke/shenandoah-jdk11u-upstream/webrev.13/ I only adjusted the synopsis to reflect the bug-ID that I created for this backport: https://bugs.openjdk.java.net/browse/JDK-8250784 Thanks & cheers! Roman From jcbeyler at google.com Wed Jul 29 18:08:59 2020 From: jcbeyler at google.com (Jean Christophe Beyler) Date: Wed, 29 Jul 2020 14:08:59 -0400 Subject: RFR (11u, S): backport JDK-8247615 Initialize the bytes left for the heap sampler In-Reply-To: References: Message-ID: Hi all, Any chance to get a review on this? Did I perhaps missed something? Thanks again for your help! Jc On Thu, Jul 23, 2020 at 10:43 AM Jean Christophe Beyler wrote: > Hi all, > > First time I request a backport so if I am missing anything, please let me > know :) > > I am trying to port this webrev into 11u: > > Summary: > This fixes a sampling bias for the first allocations and ensures that the > sampling rate is correct from the start. > It is safe to apply as the whole code is protected behind flags and only > is used when JVMTI enables the heap monitoring. > We added a test that tickles this code on purpose. > > Porting issues: > Finally, the patch does not apply cleanly due to slight differences with > the files. This webrev provides the clean to JDK11u: > http://cr.openjdk.java.net/~jcbeyler/8247615/webrev.02_11/ > > - That webrev compiles and tests for HeapMonitor (with the new test) pass: > make run-test > TEST="test/hotspot/jtreg/serviceability/jvmti/HeapMonitor" > > compared to what I submitted to head: > http://cr.openjdk.java.net/~jcbeyler/8247615/webrev.02/ > > Let me know what else I can do or provide! > > Thanks for all your help, > Jc > > Ps: I sent this email before I subscribed so one is waiting in moderation; > if this shows up in double, I apologize :) > -- Thanks, Jc From volker.simonis at gmail.com Wed Jul 29 18:36:59 2020 From: volker.simonis at gmail.com (Volker Simonis) Date: Wed, 29 Jul 2020 20:36:59 +0200 Subject: RFR (11u, S): backport JDK-8247615 Initialize the bytes left for the heap sampler In-Reply-To: References: Message-ID: Hi Jc, the downport looks fine to me except the extra empty line you've inserted in threadHeapSampler.hpp: _collectors_present = 0; + + + // Call this after _rnd is initialized to initialize _bytes_until_sample. Thank you and best regards, Volker PS: as a general rule of thumb, a RFR for a downport should include the following: - a link to the original bug (JBS link) - a link to the original change (i.e. a link to the Mercurial change set, not to the original webrev) - a link to the webrev with the new change - if your changes to "original change" are significant an explanation why they are required. A diff in webrev format of the "new change" against "original change" might be helpful in such a case. On Wed, Jul 29, 2020 at 8:09 PM Jean Christophe Beyler wrote: > > Hi all, > > Any chance to get a review on this? Did I perhaps missed something? > > Thanks again for your help! > Jc > > On Thu, Jul 23, 2020 at 10:43 AM Jean Christophe Beyler > wrote: > > > Hi all, > > > > First time I request a backport so if I am missing anything, please let me > > know :) > > > > I am trying to port this webrev into 11u: > > > > Summary: > > This fixes a sampling bias for the first allocations and ensures that the > > sampling rate is correct from the start. > > It is safe to apply as the whole code is protected behind flags and only > > is used when JVMTI enables the heap monitoring. > > We added a test that tickles this code on purpose. > > > > Porting issues: > > Finally, the patch does not apply cleanly due to slight differences with > > the files. This webrev provides the clean to JDK11u: > > http://cr.openjdk.java.net/~jcbeyler/8247615/webrev.02_11/ > > > > - That webrev compiles and tests for HeapMonitor (with the new test) pass: > > make run-test > > TEST="test/hotspot/jtreg/serviceability/jvmti/HeapMonitor" > > > > compared to what I submitted to head: > > http://cr.openjdk.java.net/~jcbeyler/8247615/webrev.02/ > > > > Let me know what else I can do or provide! > > > > Thanks for all your help, > > Jc > > > > Ps: I sent this email before I subscribed so one is waiting in moderation; > > if this shows up in double, I apologize :) > > > > > -- > > Thanks, > Jc From martinrb at google.com Thu Jul 30 01:12:09 2020 From: martinrb at google.com (Martin Buchholz) Date: Wed, 29 Jul 2020 18:12:09 -0700 Subject: [11u] RFR(XS): 8234011: (zipfs) Memory leak in ZipFileSystem.releaseDeflater() In-Reply-To: References: Message-ID: lgtm On Wed, Jul 29, 2020 at 6:56 AM Volker Simonis wrote: > > : Hi, > > can I please have a review for the following tiny downport? > The fix itself itself is trivial and applies cleanly. For the test I > had to change two lines because of a missing method in jdk11 compared > to 14. > > Original bug and fix: > https://bugs.openjdk.java.net/browse/JDK-8234011 > https://hg.openjdk.java.net/jdk/jdk/rev/fe87a92570db > > Complete 11u downport: > http://cr.openjdk.java.net/~simonis/webrevs/2020/8234011_11u/ > Changes required for 11u downport: > http://cr.openjdk.java.net/~simonis/webrevs/2020/8234011_11u_changes/ > > In jdk 11 we don't have the `FileSystems.newFileSystem(..)` variant > which takes a `Path` as a first argument so we have to manually > construct a corresponding URI first. > > Also notice the small change from `Map.of("create", true)` to > `Map.of("create", "true")` which isn't strictly necessary for OpenJDK > 11u but required for Oracle jdk 11 because they haven't downported > JDK-8034802 yet (and we want to be nice to our Oracle colleagues just > in case they want to take this downport as well :) > > Thank you and best regards, > Volker > > [1] https://bugs.openjdk.java.net/browse/JDK-8034802 From volker.simonis at gmail.com Thu Jul 30 05:16:48 2020 From: volker.simonis at gmail.com (Volker Simonis) Date: Thu, 30 Jul 2020 07:16:48 +0200 Subject: [11u] RFR(XS): 8234011: (zipfs) Memory leak in ZipFileSystem.releaseDeflater() In-Reply-To: References: Message-ID: Thanks Martin! Martin Buchholz schrieb am Do., 30. Juli 2020, 03:12: > lgtm > > On Wed, Jul 29, 2020 at 6:56 AM Volker Simonis > wrote: > > > > : Hi, > > > > can I please have a review for the following tiny downport? > > The fix itself itself is trivial and applies cleanly. For the test I > > had to change two lines because of a missing method in jdk11 compared > > to 14. > > > > Original bug and fix: > > https://bugs.openjdk.java.net/browse/JDK-8234011 > > https://hg.openjdk.java.net/jdk/jdk/rev/fe87a92570db > > > > Complete 11u downport: > > http://cr.openjdk.java.net/~simonis/webrevs/2020/8234011_11u/ > > Changes required for 11u downport: > > http://cr.openjdk.java.net/~simonis/webrevs/2020/8234011_11u_changes/ > > > > In jdk 11 we don't have the `FileSystems.newFileSystem(..)` variant > > which takes a `Path` as a first argument so we have to manually > > construct a corresponding URI first. > > > > Also notice the small change from `Map.of("create", true)` to > > `Map.of("create", "true")` which isn't strictly necessary for OpenJDK > > 11u but required for Oracle jdk 11 because they haven't downported > > JDK-8034802 yet (and we want to be nice to our Oracle colleagues just > > in case they want to take this downport as well :) > > > > Thank you and best regards, > > Volker > > > > [1] https://bugs.openjdk.java.net/browse/JDK-8034802 > From sgehwolf at redhat.com Thu Jul 30 08:19:04 2020 From: sgehwolf at redhat.com (Severin Gehwolf) Date: Thu, 30 Jul 2020 10:19:04 +0200 Subject: [11u] RFR: 8250627: Use -XX:+/-UseContainerSupport for enabling/disabling Java container metrics Message-ID: <09ad2cf47ca7537be2dfee2fbeb549155745f9e5.camel@redhat.com> Hi, Could I please get a review of this OpenJDK 11u backport which allows one to disable Java container metrics via switch -XX:-UseContainerSupport. Since JDK-8226575 (OperatingSystemMXBean should be made container aware) is in 11.0.9, I'd like to also backport the switch which allows one to turn off this container awareness. The original patch doesn't apply cleanly since there is no cgroupsv2 support in JDK 11u yet. It's cgroupv1 only so far in 11u. Changes can be trivially resolved. Changes to the original fix were: * Moved the check for -XX:-/+UseContainerSupport flag into Metrics.initContainerSubSystems() since that's where the instance is created in JDK 11u * Adjusted natives accordingly. CgroupMetrics => j.p.i.cgroupv1.Metrics Bug: https://bugs.openjdk.java.net/browse/JDK-8250627 webrev: http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8250627/jdk11/01/webrev/ original-change: https://hg.openjdk.java.net/jdk/jdk/rev/c9ad4de69c32 Testing: new container test, existing container tests and tier1 on Linux x86_64 Thoughts? Thanks, Severin From zgu at redhat.com Thu Jul 30 14:05:22 2020 From: zgu at redhat.com (Zhengyu Gu) Date: Thu, 30 Jul 2020 10:05:22 -0400 Subject: [11u] RFR 8250827: Shenandoah: needs to reset/finish StringTable's dead count before/after parallel walk Message-ID: Please review this small patch that triggers StringTable cleaning and resizing after parallel walk. Bug: https://bugs.openjdk.java.net/browse/JDK-8250827 Webrev: http://cr.openjdk.java.net/~zgu/JDK-8250827/webrev.00/ Test: hotspot_gc_shenandoah Thanks, -Zhengyu From rkennke at redhat.com Thu Jul 30 14:47:54 2020 From: rkennke at redhat.com (Roman Kennke) Date: Thu, 30 Jul 2020 16:47:54 +0200 Subject: [11u] RFR 8250827: Shenandoah: needs to reset/finish StringTable's dead count before/after parallel walk In-Reply-To: References: Message-ID: <711f8132a93f628578f1578c092a325e56c98804.camel@redhat.com> Are you sure, this is 11u-specific? I know that we have concurrent stringtable processing in later JDKs, but some code paths still use the parallelCleaning code, or is it not relevant there? Roman On Thu, 2020-07-30 at 10:05 -0400, Zhengyu Gu wrote: > Please review this small patch that triggers StringTable cleaning > and > resizing after parallel walk. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8250827 > Webrev: http://cr.openjdk.java.net/~zgu/JDK-8250827/webrev.00/ > > > Test: > hotspot_gc_shenandoah > > Thanks, > > -Zhengyu > From zgu at redhat.com Thu Jul 30 15:00:56 2020 From: zgu at redhat.com (Zhengyu Gu) Date: Thu, 30 Jul 2020 11:00:56 -0400 Subject: [11u] RFR 8250827: Shenandoah: needs to reset/finish StringTable's dead count before/after parallel walk In-Reply-To: <711f8132a93f628578f1578c092a325e56c98804.camel@redhat.com> References: <711f8132a93f628578f1578c092a325e56c98804.camel@redhat.com> Message-ID: <8421a856-9e4b-e064-d490-6a4405a13328@redhat.com> No, we have similar problem with 14. But patch is not relevant, we have to do 14 specific fix. -Zhengyu On 7/30/20 10:47 AM, Roman Kennke wrote: > Are you sure, this is 11u-specific? I know that we have concurrent > stringtable processing in later JDKs, but some code paths still use the > parallelCleaning code, or is it not relevant there? > > Roman > > On Thu, 2020-07-30 at 10:05 -0400, Zhengyu Gu wrote: >> Please review this small patch that triggers StringTable cleaning >> and >> resizing after parallel walk. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8250827 >> Webrev: http://cr.openjdk.java.net/~zgu/JDK-8250827/webrev.00/ >> >> >> Test: >> hotspot_gc_shenandoah >> >> Thanks, >> >> -Zhengyu >> > From rkennke at redhat.com Thu Jul 30 15:28:00 2020 From: rkennke at redhat.com (Roman Kennke) Date: Thu, 30 Jul 2020 17:28:00 +0200 Subject: [11u] RFR 8250827: Shenandoah: needs to reset/finish StringTable's dead count before/after parallel walk In-Reply-To: <8421a856-9e4b-e064-d490-6a4405a13328@redhat.com> References: <711f8132a93f628578f1578c092a325e56c98804.camel@redhat.com> <8421a856-9e4b-e064-d490-6a4405a13328@redhat.com> Message-ID: Hmm ok then. The patch looks good. Cheers, Roman > No, we have similar problem with 14. But patch is not relevant, we > have > to do 14 specific fix. > > -Zhengyu > > On 7/30/20 10:47 AM, Roman Kennke wrote: > > Are you sure, this is 11u-specific? I know that we have concurrent > > stringtable processing in later JDKs, but some code paths still use > > the > > parallelCleaning code, or is it not relevant there? > > > > Roman > > > > On Thu, 2020-07-30 at 10:05 -0400, Zhengyu Gu wrote: > > > Please review this small patch that triggers StringTable cleaning > > > and > > > resizing after parallel walk. > > > > > > Bug: https://bugs.openjdk.java.net/browse/JDK-8250827 > > > Webrev: http://cr.openjdk.java.net/~zgu/JDK-8250827/webrev.00/ > > > > > > > > > Test: > > > hotspot_gc_shenandoah > > > > > > Thanks, > > > > > > -Zhengyu > > > From shade at redhat.com Fri Jul 31 10:53:34 2020 From: shade at redhat.com (Aleksey Shipilev) Date: Fri, 31 Jul 2020 12:53:34 +0200 Subject: [11u] RFR 8239477: jdk/jfr/jcmd/TestJcmdStartStopDefault.java fails -XX:+VerifyOops with "verify_oop: rsi: broken oop" Message-ID: Original fix: https://bugs.openjdk.java.net/browse/JDK-8239477 https://hg.openjdk.java.net/jdk/jdk/rev/1dfb43020070 CIs that run testing -XX:+VerifyOops with 11u fail without this. 11u fix is a bit different, because the context is a bit different around this line: __ branch(lir_cond_equal, T_OBJECT, L_end->label()); 11u webrev: http://cr.openjdk.java.net/~shade/8239477/webrev.11u.01/ Testing: affected test (used to fail, now it passes); tier{1,2}; jdk_jfr -- Thanks, -Aleksey From shade at redhat.com Fri Jul 31 11:01:21 2020 From: shade at redhat.com (Aleksey Shipilev) Date: Fri, 31 Jul 2020 13:01:21 +0200 Subject: [11u] RFR: 8250627: Use -XX:+/-UseContainerSupport for enabling/disabling Java container metrics In-Reply-To: <09ad2cf47ca7537be2dfee2fbeb549155745f9e5.camel@redhat.com> References: <09ad2cf47ca7537be2dfee2fbeb549155745f9e5.camel@redhat.com> Message-ID: <9ff58187-a787-84c0-d5f4-288fa90453ef@redhat.com> On 7/30/20 10:19 AM, Severin Gehwolf wrote: > Bug: https://bugs.openjdk.java.net/browse/JDK-8250627 > webrev: http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8250627/jdk11/01/webrev/ > original-change: https://hg.openjdk.java.net/jdk/jdk/rev/c9ad4de69c32 This looks good to me. I have a passing question about the original patch, though. Are we sure only symbols-unix needs to be updated? I.e. is JVM_IsUseContainerSupport ever called from Linux binaries, not from anywhere else? -- Thanks, -Aleksey From shade at redhat.com Fri Jul 31 11:15:16 2020 From: shade at redhat.com (Aleksey Shipilev) Date: Fri, 31 Jul 2020 13:15:16 +0200 Subject: [11u] RFR 8230870: (zipfs) Add a ZIP FS test that is similar to test/jdk/java/util/zip/EntryCount64k.java Message-ID: Original fix: https://bugs.openjdk.java.net/browse/JDK-8230870 https://hg.openjdk.java.net/jdk/jdk/rev/6a05019acb67 More tests for 11u! Unfortunately, it does not compile in 11u, because some FileSystem APIs are not there. 11u webrev: https://cr.openjdk.java.net/~shade/8230870/webrev.11u.01/ Testing: new test, tier{1,2} The fix for 11u was like this: diff -r f9225bbdf0c0 test/jdk/jdk/nio/zipfs/LargeEntriesTest.java --- a/test/jdk/jdk/nio/zipfs/LargeEntriesTest.java Tue Sep 17 14:00:36 2019 -0400 +++ b/test/jdk/jdk/nio/zipfs/LargeEntriesTest.java Fri Jul 31 10:45:51 2020 +0200 @@ -23,15 +23,17 @@ */ import org.testng.annotations.*; import java.io.*; +import java.net.URI; import java.nio.charset.StandardCharsets; import java.nio.file.FileSystem; import java.nio.file.*; import java.security.SecureRandom; import java.util.Arrays; +import java.util.Collections; import java.util.Map; import java.util.function.Consumer; import java.util.zip.ZipEntry; import java.util.zip.ZipFile; @@ -209,12 +211,14 @@ * @throws IOException If an error occurs while creating the ZIP file */ private void createZipFile(Path zipFile, Map env, int entries) throws IOException { System.out.printf("Creating file = %s%n", zipFile); + + URI uri = URI.create("jar:" + zipFile.toUri()); try (FileSystem zipfs = - FileSystems.newFileSystem(zipFile, env)) { + FileSystems.newFileSystem(uri, env)) { for (int i = 0; i < entries; i++) { Files.writeString(zipfs.getPath("Entry-" + i), ZIP_FILE_VALUE); } } @@ -238,12 +242,13 @@ + System.lineSeparator() + "Main-Class: " + MANIFEST_MAIN_CLASS + System.lineSeparator() + "Created-By: " + jdkVersion + " (" + jdkVendor + ")"; + URI uri = URI.create("jar:" + zipFile.toUri()); try (FileSystem zipfs = - FileSystems.newFileSystem(zipFile, env); + FileSystems.newFileSystem(uri, env); InputStream in = new ByteArrayInputStream(manifest.getBytes())) { // Get ZIP FS path to META-INF/MANIFEST.MF Path metadir = zipfs.getPath("/", "META-INF"); Path manifestFile = metadir.resolve("MANIFEST.MF"); @@ -329,11 +334,12 @@ assertTrue(Arrays.equals(bytes, ZIP_FILE_ENTRY)); } } } // check entries with FileSystem API - try (FileSystem fs = FileSystems.newFileSystem(zipfile)) { + URI uri = URI.create("jar:" + zipfile.toUri()); + try (FileSystem fs = FileSystems.newFileSystem(uri, Collections.emptyMap())) { // check entry count Path top = fs.getPath("/"); long count = Files.find(top, Integer.MAX_VALUE, (path, attrs) -> attrs.isRegularFile() || (attrs.isDirectory() && -- Thanks, -Aleksey From sgehwolf at redhat.com Fri Jul 31 11:19:35 2020 From: sgehwolf at redhat.com (Severin Gehwolf) Date: Fri, 31 Jul 2020 13:19:35 +0200 Subject: [11u] RFR: 8250627: Use -XX:+/-UseContainerSupport for enabling/disabling Java container metrics In-Reply-To: <9ff58187-a787-84c0-d5f4-288fa90453ef@redhat.com> References: <09ad2cf47ca7537be2dfee2fbeb549155745f9e5.camel@redhat.com> <9ff58187-a787-84c0-d5f4-288fa90453ef@redhat.com> Message-ID: <41ab05dbd7891fceaccc51ec367f7d7a6b808900.camel@redhat.com> On Fri, 2020-07-31 at 13:01 +0200, Aleksey Shipilev wrote: > On 7/30/20 10:19 AM, Severin Gehwolf wrote: > > Bug: https://bugs.openjdk.java.net/browse/JDK-8250627 > > webrev: http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8250627/jdk11/01/webrev/ > > original-change: https://hg.openjdk.java.net/jdk/jdk/rev/c9ad4de69c32 > > This looks good to me. Thanks for the review! > I have a passing question about the original patch, though. Are we sure only symbols-unix needs to > be updated? I.e. is JVM_IsUseContainerSupport ever called from Linux binaries, not from anywhere else? I believe so, yes. The only call site for JVM_IsUseContainerSupport is in a Linux-only native (also note that container support is a Linux only feature): JDK 11u: http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8250627/jdk11/01/webrev/src/java.base/linux/native/libjava/Metrics.c.html Original: https://hg.openjdk.java.net/jdk/jdk/rev/c9ad4de69c32#l5.37 Thanks, Severin From shade at redhat.com Fri Jul 31 11:28:57 2020 From: shade at redhat.com (Aleksey Shipilev) Date: Fri, 31 Jul 2020 13:28:57 +0200 Subject: [11u] RFR: 8250627: Use -XX:+/-UseContainerSupport for enabling/disabling Java container metrics In-Reply-To: <41ab05dbd7891fceaccc51ec367f7d7a6b808900.camel@redhat.com> References: <09ad2cf47ca7537be2dfee2fbeb549155745f9e5.camel@redhat.com> <9ff58187-a787-84c0-d5f4-288fa90453ef@redhat.com> <41ab05dbd7891fceaccc51ec367f7d7a6b808900.camel@redhat.com> Message-ID: <248a8756-f515-8c91-d5de-f07411812506@redhat.com> On 7/31/20 1:19 PM, Severin Gehwolf wrote: > JDK 11u: http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8250627/jdk11/01/webrev/src/java.base/linux/native/libjava/Metrics.c.html > Original: https://hg.openjdk.java.net/jdk/jdk/rev/c9ad4de69c32#l5.37 Okay then. -- Thanks, -Aleksey From yan at azul.com Fri Jul 31 14:16:54 2020 From: yan at azul.com (Yuri Nesterenko) Date: Fri, 31 Jul 2020 17:16:54 +0300 Subject: [13u notice] co-maintainer: Dmitry Cherepanov Message-ID: <4e436e4d-3eba-f8e5-c2be-0e656146bb1c@azul.com> Hi, here I'm appointing Dmitry Cherepanov https://openjdk.java.net/census#dcherepanov as a co-maintainer of jdk13u. Thanks, -yan From martinrb at google.com Fri Jul 31 17:27:44 2020 From: martinrb at google.com (Martin Buchholz) Date: Fri, 31 Jul 2020 10:27:44 -0700 Subject: [11u] RFR 8230870: (zipfs) Add a ZIP FS test that is similar to test/jdk/java/util/zip/EntryCount64k.java In-Reply-To: References: Message-ID: LGTM On Fri, Jul 31, 2020 at 4:15 AM Aleksey Shipilev wrote: > > Original fix: > https://bugs.openjdk.java.net/browse/JDK-8230870 > https://hg.openjdk.java.net/jdk/jdk/rev/6a05019acb67 > > More tests for 11u! Unfortunately, it does not compile in 11u, because some FileSystem APIs are not > there. 11u webrev: > https://cr.openjdk.java.net/~shade/8230870/webrev.11u.01/ > > Testing: new test, tier{1,2} > > The fix for 11u was like this: > > diff -r f9225bbdf0c0 test/jdk/jdk/nio/zipfs/LargeEntriesTest.java > --- a/test/jdk/jdk/nio/zipfs/LargeEntriesTest.java Tue Sep 17 14:00:36 2019 -0400 > +++ b/test/jdk/jdk/nio/zipfs/LargeEntriesTest.java Fri Jul 31 10:45:51 2020 +0200 > @@ -23,15 +23,17 @@ > */ > > import org.testng.annotations.*; > > import java.io.*; > +import java.net.URI; > import java.nio.charset.StandardCharsets; > import java.nio.file.FileSystem; > import java.nio.file.*; > import java.security.SecureRandom; > import java.util.Arrays; > +import java.util.Collections; > import java.util.Map; > import java.util.function.Consumer; > import java.util.zip.ZipEntry; > import java.util.zip.ZipFile; > > @@ -209,12 +211,14 @@ > * @throws IOException If an error occurs while creating the ZIP file > */ > private void createZipFile(Path zipFile, Map env, > int entries) throws IOException { > System.out.printf("Creating file = %s%n", zipFile); > + > + URI uri = URI.create("jar:" + zipFile.toUri()); > try (FileSystem zipfs = > - FileSystems.newFileSystem(zipFile, env)) { > + FileSystems.newFileSystem(uri, env)) { > > for (int i = 0; i < entries; i++) { > Files.writeString(zipfs.getPath("Entry-" + i), ZIP_FILE_VALUE); > } > } > @@ -238,12 +242,13 @@ > + System.lineSeparator() > + "Main-Class: " + MANIFEST_MAIN_CLASS > + System.lineSeparator() > + "Created-By: " + jdkVersion + " (" + jdkVendor + ")"; > > + URI uri = URI.create("jar:" + zipFile.toUri()); > try (FileSystem zipfs = > - FileSystems.newFileSystem(zipFile, env); > + FileSystems.newFileSystem(uri, env); > InputStream in = new ByteArrayInputStream(manifest.getBytes())) { > > // Get ZIP FS path to META-INF/MANIFEST.MF > Path metadir = zipfs.getPath("/", "META-INF"); > Path manifestFile = metadir.resolve("MANIFEST.MF"); > @@ -329,11 +334,12 @@ > assertTrue(Arrays.equals(bytes, ZIP_FILE_ENTRY)); > } > } > } > // check entries with FileSystem API > - try (FileSystem fs = FileSystems.newFileSystem(zipfile)) { > + URI uri = URI.create("jar:" + zipfile.toUri()); > + try (FileSystem fs = FileSystems.newFileSystem(uri, Collections.emptyMap())) { > > // check entry count > Path top = fs.getPath("/"); > long count = Files.find(top, Integer.MAX_VALUE, (path, attrs) -> > attrs.isRegularFile() || (attrs.isDirectory() && > > -- > Thanks, > -Aleksey >