FINAL PROPOSAL: Virtual Machine Interface

[Andrew John Hughes]

This constitutes my final proposal for the OpenJDK challenge.

SUMMARY
======

The Virtual Machine (VM) interface within the OpenJDK [1] source code
forms the boundary between the class library and the virtual machine,
and allows the two to interact.  Within OpenJDK, the VM interface has
always been a means for HotSpot to access the J2SE classes and vice
versa.  As a result of this sole relationship between one VM and the
class libraries, the current interface, although abstracted somewhat
from the codebase, is undocumented and largely built around the needs
of HotSpot rather than being explicitly designed as a VM interface.

In contrast, within the Free Java space, the GNU Classpath [2] class
library has had to deal with multiple VMs from a range of areas,
including an Ahead-Of-Time (AOT) compiler (GCJ) [3],Java-in-Java VM
(JikesRVM) [4] and a Java-on-.NET VM (IKVM.net) [5].  This has
resulted in an explicit interface being constructed that, while there
is room for improvement, fulfills the role of abstracting the class
library from the virtual machines that use it.  One notable point
relating to its maintenance is that the VM interface has been
explicitly kept 1.4-friendly while the rest of the codebase has
shifted to the use of 1.5 language features such as generics.  In
general, native methods are housed in the VM layer rather than in the
class library so that a native implementation is optional.  For
example, native methods appear in java.lang.Class in the OpenJDK
whereas these methods are contained in the package-private
java.lang.VMClass in GNU Classpath.  This allows VMs to decide whether
to simply provide a native implementation of the methods in the
reference version of VMClass or provide their own VMClass more suited
to their needs.

This project proposes to try and find a happy medium between these two
worlds.  While GNU Classpath provides a more flexible VM interface,
the class library of OpenJDK is more stable, mature and is, of course,
the reference implementation for the Java platform.  As such, many
VMs, such as CACAO [6] and IKVM.net, that use GNU Classpath have
already begun to switch to OpenJDK.  However, for each VM to undertake
this effort alone results in a significant duplication of effort.  It
also means that the VM interface of OpenJDK remains in its current
undocumented and possibly volatile state.  We propose to provide a
documented VM interface for OpenJDK, which will significantly lower
the barrier for entry for both existing Classpath VMs and any other
VMs who may wish to use OpenJDK in the future.

BENEFITS
======

As outlined in the summary, the main benefit will be the existence of
a well-documented VM interface that can be used by a virtual machine
implementor to access the OpenJDK class libraries.  The immediate
benefit of this is that it will reduce the effort required for GNU
Classpath VMs to move to OpenJDK (or, even better, support both).  In
the future, it will allow a new VM to more easily utilise the OpenJDK
as its class library.  For users, it will mean increased choice.
While they may choose to remain with the reference HotSpot VM, they
may then also have the option of using an alternative implementation such
as GCJ for AOT compilation or JikesRVM for research on Java-based
virtual machines.

DELIVERABLES
=========

The ultimate goal of this project is to have a well-defined and
documented VM interface that is provided by the OpenJDK class library
and implemented by its virtual machine, HotSpot.  Ideally, it would
also be beneficial to have another virtual machine which currently
does not use the OpenJDK class library using this new interface to
access the OpenJDK library by the end of the project.

Concerning other virtual machines, while it would be good to see
GCJ/GIJ work with OpenJDK in the future, it's not an appropriate initial
target.  The criteria for such an initial target rests heavily on the VM
being quick and easy to modify for testing (GCJ can take an hour to
build) and on its existing interface use being relatively clean (GCJ
is still not fully integrated with GNU Classpath and maintains its own
local versions of some classes).  So while GCJ is a target for the
future, JamVM (due to its small size) and/or JikesRVM (its
implementation in Java being a proof of concept) seem like more
obvious initial targets.

At the very minimum, a VM interface should be produced that can
allow a VM to start up, even if some features are still missing.

PLAN
====

1. Become familiar with the VM interface by attempting to use it from
a non-implementing VM (see above for possible VMs)
2. Move towards a situation where the VM can be started using the
OpenJDK class library.
3. Fill in the gaps so as to provide a complete interface.
4. Implement this interface in HotSpot.

The initial stages of the project will be concerned with obtaining a
more detailed understanding of the current virtual machine interface.
The best approach to this is to take an existing GNU Classpath VM and
attempt to use it with the OpenJDK class library.  This will be used
as a way to find points of failure where the VM is not aware of the
current OpenJDK interface and to fix these to suit within either the
OpenJDK class library or the VM itself.  It is expected that some part
of this will involve moving the OpenJDK towards a GNU Classpath-esque
interface, but there will also be parts where the current GNU
Classpath VM interface is unsuitable or inadequate (e.g. some features
are still missing from GNU Classpath that are implemented in OpenJDK).

Documentation will be an ongoing part of this process, as will
continued interaction with the community.  This project will take
place completely in the open and will develop as a result of
discussion within the Free Java community.

SUN PROVISION
==========

As the source code bases for the OpenJDK class library and HotSpot are
already available, Sun's most significant contribution to this effort
has already been provided.  Ideally, we would ask that the project be
given space on the OpenJDK website and the means to host patches in
order to maintain the open nature of its development.  However, this
can be provided elsewhere if necessary.  We also expect some
interaction with the existing OpenJDK community, but the last year has
shown this to not be too problematic either.

CHALLENGES
========

The main challenge of this project is that, although moves from
Classpath to OpenJDK have been completed, the success of finding a
happy medium between the two is as yet unknown.  As a result, this
project is partly experimental in its aims and, while we expect it to
be fully feasible at this point, it may not completely reach fruition.
 In undertaking it, we expect that the eventual goal may change as
things progress and situations change which are beyond our control.

QUALIFICATION
=========

As a GNU Classpath developer for the last four years, during which
time I have maintained our experimental 1.5 branch and managed two
releases, I already have significant understanding of its VM interface
and of the needs of the VMs that use it.  I have also worked with the
JamVM, GCJ and JikesRVM codebases in the past (the last two as a
participant in Google's Summer of Code [7]) and as a result have some
knowledge of how these operate and the ways they interact with the
class library.

Regarding the OpenJDK, I have been working with the source code since
it was released in May last year (and the compiler before that in
November 2006).  While I don't profess to know everything about such a
large code base in such a short space of time, I am aware of how the
IcedTea project has worked with this codebase so far and how CACAO has
been integrated with it in order to allow its use as a virtual machine
for that class library.

[1] http://openjdk.java.net
[2] http://www.gnu.org/software/classpath
[3] http://gcc.gnu.org/java
[4] http://jikesrvm.org
[5] http://ikvm.net
[6] http://cacaovm.org
[7] http://code.google.com/soc
-- 
Andrew :-)

Document Freedom Day - March 26th
http://documentfreedom.org

Support Free Java!
Contribute to GNU Classpath and the OpenJDK
http://www.gnu.org/software/classpath
http://openjdk.java.net