Hotspot Express (HSX)

Fri Feb 21 15:45:50 UTC 2020

Hi,

I want to start with some personal experience. Before joining Amazon
I've worked at SAP for many years and there, with SAP JVM as you many
know, we've taken HSX to the extreme. We've used the same HotSpot
(from JDK 8) for up to 5 different Java releases (4,5,6,7,8) in
parallel and still now SAP JVM 8 is running with an HS version from
JDK 10+. Our general experience was positive and the whole SAP Java
eco system running on these versions is a strong proof that this is a
viable approach.

One of the reasons we did HSX was that we had to support a lot of
custom features and ports which are not (or were not at that time)
part of OpenJDK. That's obviously a lot easier to do with a single HS
version. I also think it wouldn't have been easy to support that many
different Java releases in parallel with a restricted amount of
resources, without HotSpot Express. We obviously had to add some
switches and gates to the latest HS in order to make it work with
previous releases, but in our experience that was much less work than
down-porting, even if it was just down-porting of relevant bug fixes.
We obviously monitored performance carefully. For all but the latest
release, the impact of HSX is just positive and the older the JDK
version that uses HSX, the bigger the performance gain. But even for
the latest JDK release we never observed any measurable performance
regression compared to the corresponding OpenJDK/Oracle JDK because of
HSX.

Coming back to the here and now. I don't see why HotSpot shouldn't be
more modular and more independent of the JDK/class library. There's
simply no reason why major parts of the JVM like JIT or GC should be
tied to a specific Java /JDK version (and with JVMCI and the GC
interface we're moving in the right direction here). Some of the
reasons for HSX we had at SAP like new features (e.g. Shenandoah) or
ports (e.g. aarch64) maintained outside of the main update
repositories for LTS releases, are still relevant in OpenJDK today. By
the way, the discussion about a clean VM interface is as old as
OpenJDK itself and we even (still) have a special OpenJDK project
(sponsored by  the HS Group) dedicated to this topic: "Common VM
Interface" (http://openjdk.java.net/projects/cvmi). Anyone remembers
it :))

In the early OpenJDK days, a bunch of different VM implementations
(e.g. JamVM, JikesRVM, IKVM.NET) integrated with the OpenJDK class
library but even today we still have one  major, alternative JVM
implementation (called OpenJ9 :) which can work with the OpenJDK JDK -
and I think this is good for the project. I know that in the early
days, Sun "kind of" supported these alternative JVM implementations
while nowadays Oralce  is trying to prevent them with legal means
(i.e. withdrawing the OCTLA license), but let's not go into that
direction here and keep this discussion on a technical level :)

I don't agree with what's been said in this thread before, namely that
a strong coupling of VM and JDK "is good" for OpenJDK, helps
accelerating the development and improves the performance. In
contrary, I think it will lead to bad "ad-hoc" decisions and wrong
dependencies which might be helpful for the moment but which introduce
considerable maintenance costs in the long term.

I can understand that Oracle who is running the "tip" project, is not
interested to invest in HSX. On the other hand, this mail thread has
been initiated on "jdk-udates-dev" and not on "hotspot-dev". So maybe
some parties on this list who are already involved in maintaining
LTS/MTS releases, can agree on something like an "Hot Spot Express"
project which can evaluate the benefits and drawbacks of such an
approach in more detail (and in parallel and without impacting the
current "tip" development). If the experiments are successful, I can
imagine that some outcome which is not so controversial like for
example better (GC,JIT,etc..) interfaces, better modularization, a
better VM/JDK Interface can be upstreamed (simplifying the future
maintenance of HSX) while other parts may stay in the HSX project
forever. Such a project could then be used by every downstream
distribution as a drop-in replacement for the default LTS HotSpot but
also by projects which develop new features in "tip" to easily and
more quickly get access to a much wider user base.

Best regards,
Volker

On Thu, Feb 20, 2020 at 12:58 AM Hohensee, Paul <hohensee at amazon.com> wrote:
>
> Re JSX lack of adoption, the argument is that it's always taken at least a year to stabilize a new release, and with 6 months between releases none of the non-LTS releases will ever be stable. Doesn't matter whether that's true or not, it's what people believe based on previous experience with 8 and 11. The relatively large degree of incompatibility between 8 and 11 is another hurdle. People remember that and don't want to worry about language and library backward compatibility every 6 months, no matter how small those incompatibilities are. Doing a lot of backports is a response to that customer demand. Hotspot is "under the covers", so to speak, so HSX was easier to sell than is JSX.
>
> Paul
>
> On 2/19/20, 3:07 PM, "Claes Redestad" <claes.redestad at oracle.com> wrote:
>
>     On 2020-02-19 22:50, Hohensee, Paul wrote:
>     > Given your stance on backports and JSX adoption, I can see your point and even agree with it. The problem is that the vast majority of users don’t want to use JSX. That's definitely the case inside Amazon.
>
>     And why is that, Paul?
>
>     >
>     > HSX does complicate the code base a bit, but is that worse than doing 600+ backports and counting?
>
>     Yes. :-)
>
>     Or rather: Even if a vast majority of users only skip from LTS to LTS, I
>     believe only a small fraction of those backports have been *strictly
>     necessary*.
>
>     There are definitely resources that can be diverted from backporting
>     (which helps users stuck on some old release) to maintaining and
>     improving the tip (which brings *everyone* value sooner or later, be it
>     next quarter or in three years).
>
>     Granted, I've backported a few things that weren't strictly necessary,
>     too, such as various startup optimizations, but always with the
>     motivation that without a fix there would have been regressions caused
>     by other, more critical changes. I'm not religiously against backports.
>     :-)
>
>     But even if you don't agree the virtues of focusing engineering
>     resources on maintaining and working on the tip, we should stay well
>     clear of things that make maintaining the tip harder than it already is.
>
>     Because it is already really hard.
>
>     And HSX is one of the worse things you can do to make that maintenance
>     harder.
>
>     > Also, I'm curious about how removing HSX remnants has improved performance. Can anyone point me at specifics?
>
>     No. It's an accumulation of small improvements over time, including a
>     reduction in static footprint, many small startup improvements and
>     some overall performance improvements.
>
>     But what's really at stake here are all the future features, cleanups
>     and larger improvements that will be just that little bit *harder* to do
>     *right* if hsx was still a thing.
>
>     I won't go so far as to say it'd make any of the cool stuff happening at
>     or near the tip now impossible. But it would without a doubt slow down
>     progress to the detriment of everyone. Perhaps even enough to make some
>     throw their hands in the air and pick up ranching. Just sayin'.
>
>     /Claes
>
>