MethodHandle vs function types

[John Rose]

On Feb 21, 2010, at 10:25 PM, Neal Gafter wrote:

> Approach (3) is outside the scope of project lambda to address (it is an
> issue for jsr292), but I think it would be a very good idea for us to
> encourage them to work on it.


As you know, the original JSR 292 EDR specifies that methods handle invocations are non-variant; the signatures of caller and callee must agree exactly.  Thus, a (String)->Object method handle cannot be invoked as (Object)->String, nor vice versa.

Based on comments from several people (including you–thanks!) the committee has agreed to provide a method handle invocation called "invokeGeneric" that provides on-the-fly reference conversions and boxing/unboxing conversions (N.B. but not primitive conversions).  The original "invoke" is now called "invokeExact", and never provides implicit conversions; it is useful for tuned library code, but unforgiving of function type shifts.  The names of these can be adjusted one more time if there are strong reasons.  I am now working on the OpenJDK reference implementation of invokeGeneric and related functionality.

The conversions provided by invokeGeneric are not enough to implement every conceivable "natural" function type conversion (as derived from Java's method calling conversions via substitution experiments).  But they are enough (I hope) to provide infrastructure for a useful system of implicit and explicit function type conversions. As with generics, there may need to be occasional unchecked operations.  But I think a static type system can be devised which will sit on top of method handle types and which can guarantee (for many kinds of useful code, and in the absence of unchecked operations) that an invokeGeneric will not throw CCE on any argument or return value.

Of course, arrays are (as always) a special case, with their special typing rules and special bytecodes which no one wants to revisit.  The 292 EG has seen no need (to date) to specify a mechanism for defining arrays of strongly typed method handles.  We probably cannot offer any help with integrating new types (of lambdas or methods handles or both) with arrays.  To me, extended array types would make more sense as a JVM support mechanism specified by Project Lambda and/or Project Coin.  It seems remotely possible that a scheme for extension methods, with modest JVM support, could integrate Object[] and List<T> enough to provide hybrid lists with array behavior, or hybrid arrays with list methods, which could then be programmed to perform the expected function type checks.  At present it's just speculation.

Throws are also a special case.  As with normal methods, exception declarations do not play a role in method handle linkage, and java.dyn.MethodType does not have a "throws" property, only "returnType" and "parameterTypes".  This will need some fudging since any reasonable static account of function types will have to check throws as well as returns.  Your choices are to erase throws, making dynamic type checks throw-agnostic, or to reify throws in MethodTypes or a related data structure.

The 292 EG is willing to consider incremental changes that would remove barriers to usability of method handles for lambdas, since it is obviously to everyone's advantage to unify these mechanisms where possible.

For throws, this might mean hardwiring a purely advisory attribute into MethodType.  More likely, we might provide some way to decorate method handle types with throws and other annotations.  This would require modest cooperation with MethodType, something like the way java.lang.Class cooperates with generics by implementing the interface Type.

For constant lambdas, JSR 292 can probably help with a new ldc type, which provides a compact way to "take the address of" any method.  Of course, non-constant lambdas (closures) can be created from constant (curried) functions by partial application.  Partial application is a fundamental feature of method handles; this is one of the key differentiators from java.lang.reflect.Method, which cannot be partially applied (or otherwise made into closures).  The partial application API (insertArgument, bind) is in the original JSR 292 EDR, and the new ldc types are in our current plans.

For SAM types (a contentious topic!) JSR 292 could perhaps help by defining library APIs (or other mechanisms?) for converting between method handles and SAM references.  Rather than requiring a stereotyped bytecode adapter for every statically detected SAM conversion, a library API would (a) provide a compact way to express such conversions and (b) a location for JVM vendors to concentrate optimization work.

With helpful best wishes,
-- John