RFR (11u, XXL): Upstream/backport Shenandoah to JDK11u

[Gil Tene]

> On Jul 1, 2020, at 8:10 AM, Andrew Haley <aph at redhat.com <mailto:aph at redhat.com>> wrote:
> 
> Before I go into this, I will make one thing clear: I am not
> advocating for Shenandoah to go in to JDK 11.
> 
> In the end the decision is up to me, and I have not decided yet. I
> can't simultaneously be the arbiter and an advocate for one of the
> sides.

Thank you for the reasoned consideration of the multiple
sides of this issue and of the more general "policy questions"
that it raises.

> 
> However, am I going to examine your argument in orderto find out for
> myself how much of it stands up.
> 
> On 29/06/2020 20:04, Gil Tene wrote:
>> 
>>> On Jun 29, 2020, at 10:10 AM, Andrew Haley <aph at redhat.com <mailto:aph at redhat.com>> wrote:
>>> 
>>> On 29/06/2020 17:38, Gil Tene wrote:
>>>>> Or is it that all changes to JDK11u are in-principle bad, regardless
>>>>> of their actual risk, which we should not consider?
>>>> 
>>>> All feature additions in updates for an already released version (as
>>>> opposed to bug fixes and security updates, which those updates
>>>> are the vehicle for) are in-principle bad IMO. Sometimes we have to
>>>> do the bad thing because it is less bad than the alternative (the LTS
>>>> will break for many people if we don't do it). But it is still bad.
>>> 
>>> OK, so that is your position, and the in-principle argument is AFAICS
>>> independent of the practical risk.
>> 
>> Right. For non-neccesary *feature additions* to an existing LTS.
> 
> So, your position is based on the solid rock of a principle that is
> entirely non-negotiable except for one thing: necessity, which
> inevitably is a judgment call.

Yes. gauging necessity is a judgement call. A hard one.
One that is likely debatable on a case by case basis. And one
that ultimately you, as the project lead, will have to make.

Focusing on that "neccesary or not"  judgment call and avoiding
dealing with things that don't rise to the necessity bar level that
*you* determine will make things simple. It will allows us to avoid
the various "but this is low risk, and my company really wants
it, and I know many people that would be happy to see it"
discussions that seek to increase value as an argument for
adding a feature in, and focus on a simple gate first.

In the thinking process I am suggesting, value should only matter
when a thing rises to a certain level of necessity. Above that level,
Both risk and value matter. But below it, value should be ignored,
and necessity should be the only thing being debated.

You can certainly use some of the value-building arguments to
argue for necessity as well, and that is a fine thing to do. It's just that
"I really like it and would appreciate it being added" arguments
would obviously not count. Let me show examples of things that
could make sense to argue for new feature inclusion:

- E.g. if we could show that without a new collector JDK 11
will show significant degradation from it's current behavior
on some new, here, or obviously-coming hardware configurations,
that could move this subject into the "gut wrenching" mode I
talk about earlier in the thread.
[Sort of where TLS 1.3 is for 8u. Sort of where my thinking about
aarch64 for 8u is.]

- E.g. if we identified some dramatic and degenerate
performance issue that is popping up in some newly common
field situations (e.g. logging, containers, cloud functions or KNative)
and can be addressed technically, it may cross into necessity.

 - E.g. if, during the "still struggling with adoption" phase of
the LTS we identified a performance issue that is a significant
regression from the performance of existing code an earlier LTS,
and is potentially hindering adoption as a result, and if that issue
can be addressed technically, it may cross into necessity.
[I think of 11u as still in that "still struggling with adoption" phase,
and of e.g. the log4j2 performance regressions due to stack
walking as a potential example, but I am hoping to see 11u move
out of the mode where fixing such things would be accepted as
necessity soon. We should hopefully not be fighting regresison
based adoption-preventing bugs e.g. 3+ years past an LTS GA date,
either because they are gone, or because adoption is good enough
that we don't care enough to gauge them necessary to fix].

There are obviously lots of other examples possible for necessary
feature inclusion. My point is that the argument for and against them
should be framed around what makes them necessary, not what
makes them valuable or low risk.

Would anyone like to put forward arguments for what makes this
change necessary?

> 
> But this is not a principle that the project has ever followed
> (AFAIK), so we'd have to decide to accept it in order for your
> argument certainly to succeed. And right now I can't see any
> convincing reason why we would want to bind ourselves in that way.
> 
>>> That fits with my understanding of your position: no matter how
>>> helpful a feature is, or how low the risk of adding it is, it should
>>> not happen. So, your conclusion is *entirely independent* of the
>>> actual ratio of risk to reward. No evaluation of either is necessary
>>> because the conclusion will always be the same: no.
>> 
>> Exactly. For non-neccesary *feature additions* to an existing LTS.
>>> 
>>> I hope that you will agree that your position on this matter is an
>>> extreme one; in fact is the most extreme position it is possible for
>>> anyone to take.
>> 
>> I disagree. I don't think this is an extreme position at all. In fact, I
>> think it is quite mainstream, and represents the most common
>> understanding of what stable releases are, and what most consumers
>> of updates to such releases think is going on.
> 
> It is extreme in the sense that it is hard to imagine a software
> community accepting any more extreme position, even if that position
> is a popular one. (It is surely possible for a majority to take an
> extreme position!)

There are multiple levels that would be "more extreme", and that some
might still find reasonable. For example:

- No feature additions of any kind. Regardless of necessity (e.g. not even
  a TLS 1.3 8u thing, no fixing to work on new hardware or OS versions,
  etc.) This is the "If you want that feature, no matter how necessary, move
  to a new version").

- Only include bug fixes that fix crashes with default flag settings.
  (e.g. don't fix bugs that crash with G1 on 8u, or with ParallelGC on 11u).
  Or more likely the same with a small set of "mainstream flags".
  (e.g. Obviously accept -Xloggc as a mainstream flag, accept several
   collectors as "will fix crashing bugs in", but don't do fixes for crashes
   under "uncommon combinations").

- Only include security fixes, and no other bug fixes, no matter how bad.
  (this is actually a common mode for the last legs in maintaining a project)

- Only include security fixes for issues with a CVSS score above 7.0, and
  no other bug fixes, no matter how bad.

Each of these level exists in various modes of maintenance for projects,
and I am not advocating for any of them for 11u (and obviously not even for
8u, as I'm one of the advocates of TLS1.3 work there).

Hence I claim that my position sits squarely in the moderate middle,
with plenty of more extreme positions possible on either side of this.
Those include the "don't fix non-security stuff" on one side and the
"add features that would make people happier but are not otherwise
necessary" on the other, and probably many other examples.

> 
>> But I think that where you think the vast majority of the community
>> sits on this position being "extreme" or the opposite being the
>> extreme one depends on who you think the community is.
>> 
>> This list tends to be dominated by developers of OpenJDK.
>> 
>> I claim that our community (for the updates projects) is the *users*
>> of OpenJDK.
> 
> OK, but IMO the last thing this conversation needs is anyone claiming
> to represent the "silent majority".

I am not claiming to be their representative. I am pointing out that they
exist, and are an important constituency that is affected by the decisions we
make here. And I am making arguments that IMO would be made by users
had they been active on the list. People can pick apart those arguments
on their merits.

The fact that my voice, at the point where I started making these "user centric"
arguments, seemed to be alone on that side (with a near consensus building
towards accepting such changes) does give me serious concern, but I am not
looking to claim some position as a proxy for the masses here. I'm hoping to
convince multiple others on this list to apply more of the "what would I think
if I were running 11u LTS in production and this change came along" thinking
in considering both this specific change, and the way we argue for or against
such changes in general.

> We the developers are what I mean
> when I talk about the OpenJDK community, and the OpenJDK community of
> developers is the consensus I seek. Consensus does not mean unanimity:
> that would give any developer an effective veto.

I don't have a veto, and I know it. I just have a sometimes loud voice.

— Gil.