Scala and JSR 292

[Bill Venners]

Hi John,

You suggested we continue this on the mailing list, so here we go. For
the rest of you, the context is trying to see if the JVM could make it
easier to change a Scala trait without forcing a recompile of any
class that mixes in that trait. I've included the previous part of the
conversation below.

I wanted to ask a few questions about interface injection. First, if a
class already implements an interface, I assume the JVM won't try to
do the injection process. Is that correct?

I see your blog says that "If a method is injected with an interface,
it is not accessible except via that interface itself." So I think for
Scala to use this to solve the binary compatibility problem, every
method call would need to be done via an injectable interface. Here's
the scenario. In Version 1, we have:

trait Fred
class Bob with Fred

This is compiled and works fine. Neither Bob nor Fred have any methods
right now. If you call toString on a Bob instance, it would return
something like "Bob at 955a65".

trait Fred {
  override def toString = "hello, world!"
}

Now only Fred is recompiled. What will happen right now is that when
you call toString on a Bob instance it will just return a default
toString, such as "Bob at 955a65", but if you recompile Bob and call
toString again, you'll get "hello, world". So a changed trait can
change a method that exists already on the class.

So that's why it seems like to use interface injection to solve this
problem, all calls to any class would need to allow the class to
dynamically pick up a change on a trait. Thus any invocation would
need to go through some interface that can be injected?

I guess the realization I had was that this isn't just a method
missing behavior, because the method could be there already, but with
the wrong implementation. It almost seems like the first time any
class is used, Scala would need to be able to given a chance to add
new methods and change the implementation of existing methods based on
changes to traits the class implements. The class will already
implement an interface with the name of each trait, so it will already
implement all those interfaces. The problem is the behavior of the
methods declared in the class itself may need to be changed, and also
some extra methods that aren't currently declard in the class may need
to be added.

Bill

On Fri, Mar 20, 2009 at 12:19 AM, John Rose <John.Rose at sun.com> wrote:
> On Mar 12, 2009, at 9:50 AM, Bill Venners wrote:
>
>> Hi John,
>>
>> Several people (such as Mark Reinhold and Joe Darcy) have suggested we
>> contact you about a couple ways Scala could potentially benefit from
>> VM support, because people felt it fit in with JSR 292. I'm not sure
>> if Martin has spoken with you already about this. I wanted to make
>> sure the conversation got started.
>
> I'm delighted to hear from you!
>
> Do you mind if we move this conversation to mlvm-dev at openjdk.java.net?  It
> will be interesting to others there.
>
>> The main concern I believe is that in Scala, when you add a new
>> concrete method to a trait, you need to recompile classes that mix in
>> that trait. In other words it is a binary incompatible change to add a
>> new concrete method to an existing trait. The reason is that the Scala
>> compiler awards each class that mixes in a trait "forwarding methods"
>> whose signatures match any concrete methods provided in that trait.
>> These forwarding methods allow the method to be invoked on the class,
>> and forward the call to a static method defined elsewhere that
>> provides the implementation. (That's my understanding of how it works
>> anyway. Martin will correct me if needed.) So if you add a concrete
>> method to a trait, you also need to add a "forwarding method" to the
>> classes that mix it in. Well if the VM would provide some kind of plan
>> B if it finds it is trying to invoke a method on a class that isn't
>> defined in that class, to allow the class itself to provide an
>> implementation (kind of like "method missing" in dynamic languages),
>> then this would allow the Scala compiler to at least make this a
>> binary compatible change. Those methods might not invoke as
>> efficiently, but it wouldn't break code.
>
> Interface injection is designed to help with this sort of thing.
>
> Mapping this facility to traits would be, I think, pretty easy:  Abstract
> and concrete methods for trait T are declared in an injectable interface TI.
>  Concrete trait methods would be placed in an abstract class (perhaps in
> TI.Impl inner to the interface, just to be tidy).  The trait interface has
> an injector function which binds to a given concrete target class X,
> presumably one which Scala's type system has ensured has the desired methods
> (in some form).  In any case, the injector function matches up each target
> class X with the appropriate methods taken from X or from TI.Impl, as the
> case may be.
>
> Method handles are used to specify the mix-and-match.  Since they hide
> whether they are static or non-static, the decision to take from X or
> TI.Impl is a late-bound decision.  Also, adapter logic can be folded in,
> e.g., to box or unbox arguments if the signatures don't match exactly.  And
> of course the names don't need to match; method handles hide method names
> too.
>
> The effect is not of "method not found" but "interface not found", but it
> amounts to the same functionality.
>
> Refs:
>  http://blogs.sun.com/jrose/entry/interface_injection_in_the_vm
>  http://wikis.sun.com/display/mlvm/InterfaceInjection
>
>
>> Is that something that sounds like it might fit in with JSR 292?
>
> We are starting a prototype of this, and the 292 EG has discussed it
> briefly, but has not yet taken it up seriously.
>
>> One other issue that is less critical is simply that Scala makes
>> programmers very efficient at generating class files, because for
>> every anonymous function you get an anonymous class. In a couple
>> months of part time work I ended up with 12,000 class files of test
>> code. So there's a scaling issue here. One solution is the Scala
>> compiler could simply generate a JAR file, so you get one file. That's
>> probably a fine solution actually, but I heard JSR 292 may include
>> something like a method pointer. If there were a way that a compiler
>> could stick little "private classes" inside one class file to
>> represent some of those functions, then that could help reduce the
>> class file proliferation you get with Scala. This isn't really a big
>> deal, but I figured it is at least worthwhile to find out what those
>> method pointers are intended for, and see if they could be of any help
>> to Scala also.
>
> Method handles are pure functions, and so hide naming, static-ness, and
> access restriction.  (Access is checked when the handle is made, not when it
> is called as in reflection.)  With adapters, they can hide minor differences
> in signatures also.  Finally, they can be curried.  They are thus ideal for
> all sorts of small behavioral units.  Any class X can build any number and
> variety of method handles by defining private static methods and reifying
> them as needed as method handles.  The eventual user of the handle sees
> nothing about the original method, and has no special access rights beyond
> calling the method via the handle.
>
> The initial implementation of method handles has been under review for a few
> weeks and will soon be available (for x86/32bit only, at the beginning) in
> the OpenJDK JDK 7 builds.  The changeset under review, FYI, is:
>  http://cr.openjdk.java.net/~jrose/6655638/
>
> Meanwhile, the JSR 292 EG is seriously analysing the proposed APIs.  It is
> important that this work for all JVMs, not just Hotspot.
>
> Best wishes,
> -- John
>
>> Thanks.
>>
>> Bill
>> ----
>> Bill Venners
>> Artima, Inc.
>> http://www.artima.com
>
>