Backporting features [was Re: RFR (11u, XXL): Upstream/backport Shenandoah to JDK11u]

[Andrew Haley]

On 2/13/20 10:00 PM, Andrew John Hughes wrote:

> In short, while I appreciate the concerns, I think upstream
> integration is the best solution available, short of convincing the
> people who want these features not to use 11u at all and switch to
> the latest OpenJDK.

(TL/DR: I haven't yet decided.)

When I applied to lead this project I wrote

  With regard to future plans, my Rule 0 is "first, do no harm":
  OpenJDK 11 is an increasingly important part of many organization's
  infrastructure, and we must not break it. I don't imagine that there
  will be many major feature backports, but I won't forbid them
  altogether.  I don't intend to be a dictator: we should reach
  consensus on such decisions.

Therefore, the default action when deciding on backporting a feature
should be to say "no" because of Rule 0, but if there is a compelling
reason to do the backport and we can do so without significant risk,
we should.

This is a very difficult call to make. While the argument about making
JDK 11u as useful as to the majority as we reasonably can and to make
it as well tested as possible is compelling, 11u is the upstream for
everyone, not just the majority. IMO, if a significant downstream
consumer feels that they cannot use a release because of the risk, the
project will have failed. Therefore, we must seek a broad consensus.

So what is a broad consensus? ISO defines it as

  General agreement, characterized by the absence of sustained
  opposition to substantial issues by any important part of the
  concerned interests and by a process that involves seeking to take
  into account the views of all parties concerned and to reconcile any
  conflicting arguments. Consensus need not imply unanimity.

This is the definition I will use.

It is in my view significant that we can guard the import of
Shenandoah in such a way that we can statically analyse it and prove
(beyond reasonable doubt) that, if not configured, it will have no
effect. We can also show, albeit not quite as convincingly, that if it
is not enabled with a runtime flag it will have no significant
effect. For that reason I believe that the import of Shenandoah is
less risky than many of the backport fixes that we do.

With regard to the TLS 1.3 and JFR backports to 8u, unless there is a
substantial risk we must do these because some proprietary Javas have
them and our mission is to make sure that our end users have the
choice of using free software. We don't want our users to be forced
into using proprietary software because our releases are missing some
important features. For that reason, although these backports are more
risky than Shenandoah would be, there is a broad consensus in favour
of them.

I am not convinced by the "HotSpot Express" model: as far as I can see
it does little more than replicating the effort that we have to make
today, and increasing the testing effort.

With regard to Gil Tene, I have to take it into account that he is,
with one or two exceptions, the most experienced HotSpot and Java
developer in this group. I do not believe that he is raising this
objection in order to reduce the free competition for Azul's Zing
product, but that he is solely motivated by the best interests of the
project.  Therefore, unless we reach a complete impasse that makes it
impossible to proceed, I want to discuss this matter further in the
hope that we can reach a consensus.

Gil says that "or every OpenJDK user that wants to see 11u or 8u have
new features (and is not willing to go to 13u or 14u for those), there
are at least 10 that simply want 11u and 8u to remain rock-solid, and
only take on changes that must be taken to improve stability and
security." He may be right about that, but it is also important to
minimize the workload and maximize the effective testing done by
downstream developers.

Finally, I believe that the underlying worry behind all of this is to
some extent that Shenandoah represents the thin end of the wedge, and
that if it is allowed in there will be a free-for-all. I don't think
that will happen. The need for broad consensus makes such a situation
unlikely, and I don't intend to allow it either.

The question in the end, then, is just how compelling Shenandoah is,
and how significant the risk is.

-- 
Andrew Haley  (he/him)
Java Platform Lead Engineer
Red Hat UK Ltd. <https://www.redhat.com>
https://keybase.io/andrewhaley
EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671