Call for Discussion: New Project: Metropolis

[John Rose]

On Sep 21, 2017, at 4:26 PM, Christian Thalinger <cthalinger at twitter.com> wrote:
> 
> Replying here then… 
> 
> You and I are talking about this for quite a while now and I
> welcome this becoming an official OpenJDK project now.

Yes, we have.  We have been watching Java-based JITs slowly mature
since Jikes, Maxine, JavaC1, C1X, and now Graal.  The stars have
aligned, I think, for a major shift.

(In more detail:  I have enjoyed watching the experimentation in JVM
implementation techniques by Hanspeter Mössenböck's students at JKU,
with collaboration by Sun/Oracle Labs engineers.  There is a wealth
of literature just from that department, and Graal was grown there.
A good place to connect into this history is Titzer et al in VEE'10.
People get inspired in crazy ways; I just noticed a paper on doing
JVMs in Haskell and FORTH: 
  http://wien.tomnetworks.com/uni/thesis_matevm_oneside.pdf )

(Ever further back:  Some of my happiest programming memories come
from hacking around in the Symbolics Lisp Machine OS, which was
extravagantly metacircular.  Anything not written in Lisp was written
in microcode and/or assembly code, in Lisp syntax, on a Lisp toolchain.
There was a flexible, hackable system.  Such flexibility never goes out
of style, I think, although there are always balancing concerns like
reliability and usability.)

> How do you envision people working on this project?  Should individual
> people just become contributors and then work on whichever module the
> want to replace?

This is not mainly up to me, but rather the guys you know:  Vladimir,
Igor, Thomas, Doug, Tom.  Whatever the group decides as a consensus
is where it will go.  Can we hope that you will be in the group???

That said, here are a few of my thoughts:

I notice that we are growing some tools and experience with branches
as a way to maintain multiple parallel experiments.  See Project Amber.
Maurizio's scripts keep the branches properly stacked into dependency
order and refreshed from their dependencies, and all of them from the
default branch.

If someone wanted to replace a relatively small, isolated component
like MethodLiveness, that would fit easily in a branch.

Trying to code up a simpler verifier in Java would be a worthy goal.
By simpler I don't mean a different algorithm, but rather a higher-level
expression of the same algorithm, more amenable to review and
validation, both human and robotic.

Anything is fair game:  The reflection runtime, the LinkResolver and
SystemDictionary, eventually the metadata itself (when we have more
control over object layout).  An especially tantalizing area would be
to find "hot spots" in the C++ code where JVM-specific optimizations
would improve the system performance more than a C++ optimizer
could do.  If we have a clever code shape we need generated for
the JVM, we can train Graal to do it for us, if we can figure out how
to replace the affected range of C++ code with Java code.  GC inner
loops are probably amenable to GC-specific optimization techniques.

The main driving function, I expect, will be making Graal compile itself
into a static-linked "C++ mode" (using SVM I hope), into a linkable Graal
JIT to supplant C2.  But that's just a driving function, not the endpoint,
of Java-on-Java.  An ambitious person could try to replace C1 or the
interpreter with Graal.

Did I mention this is likely to be a very long-running project?  :-)

— John