RFR (11u, XXL): Upstream/backport Shenandoah to JDK11u

[Andrew Haley]

Before I go into this, I will make one thing clear: I am not
advocating for Shenandoah to go in to JDK 11.

In the end the decision is up to me, and I have not decided yet. I
can't simultaneously be the arbiter and an advocate for one of the
sides.

However, am I going to examine your argument in orderto find out for
myself how much of it stands up.

On 29/06/2020 20:04, Gil Tene wrote:
>
>> On Jun 29, 2020, at 10:10 AM, Andrew Haley <aph at redhat.com> wrote:
>>
>> On 29/06/2020 17:38, Gil Tene wrote:
>>>> Or is it that all changes to JDK11u are in-principle bad, regardless
>>>> of their actual risk, which we should not consider?
>>>
>>> All feature additions in updates for an already released version (as
>>> opposed to bug fixes and security updates, which those updates
>>> are the vehicle for) are in-principle bad IMO. Sometimes we have to
>>> do the bad thing because it is less bad than the alternative (the LTS
>>> will break for many people if we don't do it). But it is still bad.
>>
>> OK, so that is your position, and the in-principle argument is AFAICS
>> independent of the practical risk.
>
> Right. For non-neccesary *feature additions* to an existing LTS.

So, your position is based on the solid rock of a principle that is
entirely non-negotiable except for one thing: necessity, which
inevitably is a judgment call.

But this is not a principle that the project has ever followed
(AFAIK), so we'd have to decide to accept it in order for your
argument certainly to succeed. And right now I can't see any
convincing reason why we would want to bind ourselves in that way.

>> That fits with my understanding of your position: no matter how
>> helpful a feature is, or how low the risk of adding it is, it should
>> not happen. So, your conclusion is *entirely independent* of the
>> actual ratio of risk to reward. No evaluation of either is necessary
>> because the conclusion will always be the same: no.
>
> Exactly. For non-neccesary *feature additions* to an existing LTS.
>>
>> I hope that you will agree that your position on this matter is an
>> extreme one; in fact is the most extreme position it is possible for
>> anyone to take.
>
> I disagree. I don't think this is an extreme position at all. In fact, I
> think it is quite mainstream, and represents the most common
> understanding of what stable releases are, and what most consumers
> of updates to such releases think is going on.

It is extreme in the sense that it is hard to imagine a software
community accepting any more extreme position, even if that position
is a popular one. (It is surely possible for a majority to take an
extreme position!)

> But I think that where you think the vast majority of the community
> sits on this position being "extreme" or the opposite being the
> extreme one depends on who you think the community is.
>
> This list tends to be dominated by developers of OpenJDK.
>
> I claim that our community (for the updates projects) is the *users*
> of OpenJDK.

OK, but IMO the last thing this conversation needs is anyone claiming
to represent the "silent majority". We the developers are what I mean
when I talk about the OpenJDK community, and the OpenJDK community of
developers is the consensus I seek. Consensus does not mean unanimity:
that would give any developer an effective veto.

>> Please allow me to suggest a little thought experiment.
>>
>> Let's say that we could import Feature X (which is Shenandoah, but I'd
>> like to do make this discussion more general) without any risk. Sure,
>> risk is never zero, but let's say:
>>
>> 1. All changes to the source code are surrounded by compile-time
>>   #if USE_SHENANDOAH_GC.
>>
>> 2. USE_SHENANDOAH_GC is never set unless it is explicitly requested by
>>   a configure argument.
>>
>> 3. All changes to the source code within the USE_SHENANDOAH_GC markers
>>   are also guarded by runtime tests of the form
>>   if (UseShenandoahGC) { ...
>>
>> I maintain that no-one will ever be forced to use X. Firstly, no
>> downstream builder of our release JDK 11u will even build X unless
>> they explicitly choose to. Also, unless UseShenandoahGC is enabled on
>> the Java command line, no downstream user will be forced to use it
>> either.
>>
>> What, in your opinion, are the practical risks to production users of
>> the above? I'm assuming that we can confirm that the above plan works
>> and does not touch anything it shouldn't by a combination of tooling
>> and several sets of eyeballs. That's not perfect, but nothing is.
>
> Let me answer both in general, and then in the specific:
>
> General:
>
> First, my point i that the practical risk does not matter, as no necessity
> has been demonstrated or even argued for. And I claim that this is not
> an extreme position (and that if we want to label things as extreme,
> the opposite would be the extreme).
>
> Next, I'll certainly accept that with careful review of some things it is
> quite possible to keep risks very low. E.g. in cases where ALL
> code is protected by #ifdef the risk is somewhat mitigated.
> And that in cases where ALL non-ifdef-covered runtime flags
> protect code paths the risk mitigation is not quite as good, but
> stlll could be good.
>
> I'll even go farther and say that risk can (and should) be seriously
> reduced  even when the above stated protections are not practical.
> Deep review (with an eye to minimizing risk around change, as
> opposed to the typical review that focuses on correctness, code
> structure, cleanliness, elegance, reuse, maintainability, etc.) is key to
> minimizing risk when risk HAS to be taken. And as you well know, we
> are going through that excercize right now with TLS1.3 in 8u.

Indeed we are.

> None of these are good enough to overcome the "no necessity" bar
> IMO, because how good and low risk a change is is irrelevant
> (in an LTS update) when it is unnecessary.

OK, I understand that point.

> Specific:
> The above two are generic policy approaches. When it comes to the
> specific case here (the addition of a new collector to 11u), which touches
> a huge number of source files, some of the above description only apply
> "whereever reasonably possible", and actual code changes to common
> paths that are not protected by #ifdefs avoided statically via launch time
> configuration flags exist (i.e. every dynamic execution of some common
> code is now goigg through different logic that is considering the
> configuration flags, or worse, common code changes exist where no
> conditionals are applied at all).
> That is not a subtle difference.

Indeed not. That, if true, would be a powerful argument against
accepting Shenandoah. It would also be an argument based on
engineering issues rather than those of high-minded principle.

> With that said, this last argument is not a logic path I'm looking to go
> down, because I don't think the additional risk involved in something as
> big and as intrusive as adding Shenandoah actually matters here. I'd
> be making the same neccesity-based arguments for other features that
> don't share that level change and risk, so I don't want to rathole in
> discussing the specific code risks in this case.

I understand that too, and appealing to a hard-and-fast rule means
that we don't even have to examine the proposed patch to see how risky
it is.

-- 
Andrew Haley  (he/him)
Java Platform Lead Engineer
Red Hat UK Ltd. <https://www.redhat.com>
https://keybase.io/andrewhaley
EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671