Draft Proposal - Java foreign function interface

[Wayne Meissner]

I'll preface this one by saying I'm not sure if this proposal meets
the originality requirements (its mostly a re-implementation of
existing ideas), nor if it is desirable for OpenJDK (some might call
the idea heretical).

But, until other proposals start showing up, I thought I'd get the
ball rolling, and people could at least practice critiquing proposals
:-)

The proposal is far from complete, mostly at the "brain dump, slightly
reformatted" stage.

                   Draft Proposal - Java Foreign Function Interface

1.0 Outline

This proposal is for a project to provide Java programmers with easier access
to native libraries.

Although JNI already provides programmers with access to native libraries, it
has some drawbacks:
  a) It requires compilation of a native library for each potential target
     platform.  This places an extra burden on developers.
  b) The complexity of JNI programming is greater than that of a pure java
     project.  It is also usually more complex than the native APIs that the
     programmer wants to call.

For many tasks, being able to access native APIs using java syntax, with
all the hairy details of JNI programming hidden away would be a good thing.

2.0 Need.
Although everyone would like to have 100% pure java libraries and programs,
that is not practical if Java is to compete with e.g. C# as a modern desktop or
systems programming language.  There will always be native APIs that are not
exposed in the current JRE and there will always be a lag between the latest
JREand what the majority are using.

Although JNI can be used to access native APIs, it is somewhat more complex
than either java or the native APIs that programmers are trying to access, and
as such presents a higher barrier of entry than programmers would desire.

Although there is a viewpoint that this barrier encourages development of pure
java libraries, in the case of highly complex native frameworks (e.g. media
libraries), re-implementation in java is impractical.

Another need is from the JRuby and Jython communities - where the ability to
provide a more accurate implementation of the Ruby/Python environment to
programs written in those languages would be eased by the ability to access the
same native libraries that the original implementations of those languages
allow.  The benefit is not just restricted to implementation of JRuby/Jython
features in Java - these languages will also be able to directly call native
functions.

Even within the JDK itself, having easier access to native libraries might
enable faster feature implementation where access to native APIs is needed.

3.0 Concept Viability.
The concept of simplified native library is not new - there are several
projects, both commercial and open source.

For example, the JNA project[1] has been used to successfully build bindings
for the gstreamer multimedia framework.  The resulting bindings are pure
java (except for a dependency on JNA itself), and run without modification on
Linux/x86, Linux/amd64, Windows/x86, MacOSX/x86, Solaris(x86, sparc, sparcv9)
and FreeBSD/x86.

The gstreamer-java[2] bindings also demonstrate that it is possible to produce
reasonably performant native bindings using this method - the bindings can
render video into a lightweight swing component at better-than DVD resolution,
with no native code other than the common JNA stub.

The JRuby project is also now using JNA to provide access to native libraries
to Ruby programs so they can run un-altered on the JVM.

Although the idea of providing direct access to native resources from java code
might seem abhorrent to some, it is not without precedent.  The
sun.misc.Unsafe, java.nio.Bits classes already provide the ability to allocate,
read and write native memory directly from java.

4.0 Proposal Details.

This project proposes to implement a minimal set of java and JNI components that
facilitate calling native functions directly from Java, and to enable java code
to be called from native code (closures).

Although the project will not include higher level C language features
(e.g. structures, unions, custom mapping between java types and native types)
it will provide sufficient functionality to allow these features to be
implemented by other java libraries without requiring native code.

4.1 Dependencies
The proposal depends on libffi - part of gcc, but licensed under a more liberal
license than the GPL - for the low-level native dispatch functionality.


[
What else should go in this section?
  Technical details. e.g.
  - exact description of what the java interface would look like?
  - what native types are supported as arguments?
  - gory details of the java->native argument marshalling/unmarshalling?
  - perceived advantages vs existing implementations?

  I did have a bunch of technical stuff here, including the above information,
  but it was basically a brain dump, and somewhat incoherent.
]

5.0 References:
[1] JNA http://jna.dev.java.net
[2] gstreamer-java http://code.google.com/p/gstreamer-java