hg: mlvm/mlvm/jdk: meth: add proxy maker for closures

[Charles Oliver Nutter]

On Mon, May 24, 2010 at 8:22 PM, John Rose <john.r.rose at oracle.com> wrote:
> On May 18, 2010, at 1:18 PM, Charles Oliver Nutter wrote:
>
>> One important question for me is how multi-method interfaces would be
>> handled. I had originally tried to use Scala's "Function" interfaces
>> in Duby to represent closures, but they all have multiple abstract
>> methods that must be handled completely differently.
>
> Right.  Here are the degrees of freedom I see at this point:
>  - whether to accept SAM types which are not interfaces (default: yes)
>  - whether to allow a query API for recovering a method handle from a proxy (default: yes)
>  - whether to allow multiple methods to be associated with multiple MHs (default: no)

So you'd wire up a single handle that is dispatched through for all
incoming cases? Similar to how we (and presumably Groovy, when there
are no type hints) have a single JRuby "DynamicMethod" object for all
overloads of a given method name, and it makes a selection based on
actual types at runtime?

>  - whether to allow multiple super types in the proxy object (default: no)
>  - whether multiple methods must be individually closed, or can be mutually closed over another value
>
> The reason I "stopped at one" is there are more optimization options for the simplified case of one-type-one-method.
>
> Also, the query API in point 2 is harder to get right if you accept the other points.

I don't have a particular need for the query API, though I haven't
thought through the possibilities yet. For the most part, all I ever
need is the implementation of some interface or abstract class, and
after that it can "just" be a class to me.

> But I do agree there should be a mechanism supporting the additional degrees of freedom.
>
> It needs to be bulkier, though.  It probably needs multiple phases, like a builder.
>
> Also, the various parts (methods, receivers) can appear at various times.
>
> Here's a two-phase version:
>   class InstanceBuilder<T,R> {
>     InstanceBuilder(List<Class<? extends T>> supers, List<String> names, List<MethodType> types);
>     InstanceBuilder(List<Class<? extends T>> supers, List<String> names, List<MethodType> types, Class<R> receiver);
>     InstanceBuilder(List<Class<? extends T>> supers, List<String> names, List<MethodHandle> methods);
>     T newInstance(R receiver, MethodHandle... methods);
>     T newInstance(MethodHandle... methods); // no bound receiver
>     T newInstance(R receiver); // previously specified methods
>   }
>
> Here's chained multi-phase version:
>   class InstanceBuilder<T,R> {
>     InstanceBuilder(List<Class<? extends T>> supers, List<String> names, List<MethodType> types);
>     InstanceBuilder bindReceiver(R receiver);
>     InstanceBuilder bindMethod(String name, MethodHandle method);
>     T newInstance();
>   }
>
> And so on...

Yeah, these seem like the right general direction. I'd probably my
money behind the multi-phase, multi-call version.

To add another use case to the Scala Function interfaces (which have
multiple abstract methods): JRuby's own DynamicMethod has a number of
call paths of different arities; if it were possible to implement all
of them with handles we'd never have to generate another "invoker"
again.

> Wrapping method handles adds layers of indirection and boxing hazards as Remi points out.

To be sure...I'd love to move back into the realm of having no users
and JRuby being in its early research days, so I could commit to being
Java 7-only. It would make meeting performance challenges a vastly
simpler proposition. Unfortunately that's pretty hard to justify right
now, when we're busy trying to get paying customers (who stubbornly
refuse to use trunk builds of MLVM...imagine the nerve!). :)

> An unsolved problem with JSR 292 is how to mix method handles in with other supertypes.  An excellent solution would allow you to define your own supertypes and APIs, and have them more or less directly mapped to method handles when method handles were available.  This might increase the number of files which could be reused (just by recompilation or relinking) across JRuby implementations (indy or classic).

My current solution for that is infeasible given time and resources:
mutually-incompatible call paths loaded conditionally if you're
running on Java 7.

> I proposed JavaMethodHandle but as a fixed superclass it doesn't give much flexibility (not a mixin!).  And it constrains JVM implementations too much to have an inheritable subtype of MethodHandle.

Yes, having a java.dyn supertype doesn't help us reuse code from
non-indy builds. It certainly makes it easier to implement complicated
handle logic (assuming that code gets specialized for each inline
location...which I'm not sure of right now), but that's firmly
indy-specific.

> Another possibility is to ask for an implicit conversion from selected application types X to MethodHandle.  That in effect invents a new type relation: delegation.  So it's no small matter.  It's hard to keep that genie confined to a small bottle.

Being able to generate subclasses would definitely be the more
straightforward option here; we don't really care about the actual
type of *any* of our generated DynamicMethod subtypes...we just need a
way to stitch calls from A to B through a known interface without
defeating inlining.

>> Perhaps this also helps Java 7 closures work nicer by making it
>> possible to have a generic Closure or Function superclass completely
>> independent of java.dyn? Or by making it possible to implement
>> (faster) Java reflection with fewer generic frames under the covers
>> via a single abstract supertype populated by indy?
>
> The best way I know to accelerate Java reflective invocation is this:
>        java.lang.reflect.Method foo = ...;
>        try { z = MethodHandles.unreflect(foo).invokeGeneric(a, b, c, ...); }
>        catch { ... }
>
> For this to be faster, it assumes there is method handle caching on unreflect, which I haven't done.  But could be done.
>
> For example, you could preserve source compatibility by making a wrapper method to hide MethodHandles.

This would be similar to us lazily generating a direct handle object
for each reflected method, which we currently support but don't turn
on by default. In our case, there's an explosion of user-loaded
classes to deal with, as well as the stickiness of loading custom
classes that reference classes you might want to unload later...back
to the ClassLoader hard reference issue again.

But your trick is cleaner; we could potentially have a separate branch
that goes out to indy code if it's around, and that indy code would
use this logic. The ideal case would be that both indy and non-indy
dispatch through the same interface...and then we're back to needing a
way to implement an interface entirely with method handles again :)

- Charlie