[jsr-292-eg] Conversion of method reference to MethodHandle

[Remi Forax]

ping ...

On 12/31/2012 04:18 PM, Remi Forax wrote:
> On 12/31/2012 02:00 AM, Brian Goetz wrote:
>> To start with, the syntax that has been chosen for method references 
>> is inadequate for method handles. This is because of the point you 
>> raise -- there is no way to specify the type of the parameters, and 
>> therefore no reasonable way (creating a customized functional 
>> interface for each usage is not reasonable) to select between 
>> multiple overloads of the same name.  We have discussed explicit type 
>> parameters, but they turned out (or so we still think, jury is still 
>> out) that not adding this bit of syntax is adequate to what we are 
>> doing here.
>>
>> Failing completely in the presence of overloading makes this a 
>> crippled feature with even more sharp edges than it naturally has.
>
> I see that as a feature.
> People tend to use overloading where they should not. List::remove is 
> a good example, JPanel#add is another one.
> The only case where it's correctly used is PrintStream#println, 
> StringBuilder#append or Math::sqrt because all overloads have the same 
> semantics.
>
> More fundamentally method reference does an early binding, so even a 
> method reference with a functional interface is broken because the 
> overload resolution should appear when calling it and not when 
> creating it.
>
> The simple workaround is to create a bridge method, if you can't 
> change the class to not use overloading.
>   private static void myprintln(PrintStream stream, int value) {
>      return stream.println(value);
>   }
>
>>   But that's not even the real objection, 
>
> cool !
>
>> nor is this:
>>
>>> And because Brian will say that this is interesting but Oracle has 
>>> no resource that can work on this,
>>> here is the patch for the compiler* part based on lambda/langtools : 
>>> http://www-igm.univ-mlv.fr/~forax/tmp/method-ref-method-handle-langtools.diff
>>> and the patch for the jdk part based on lambda/jdk: 
>>> http://www-igm.univ-mlv.fr/~forax/tmp/method-ref-method-handle-jdk.diff
>>
>> My true objection is deeper than any of these.  After thinking about 
>> it for a long time, I think this feature just doesn't belong.  The 
>> set of people who need method handle literals is a tiny sliver of the 
>> Java developer base; complicating "everyone's Java" for the sake of a 
>> few advanced users (100% of whom are qualified to generate bytecode) 
>> is a bad stewardship choice.
>
> There are two points here:
> 1) the set of people interested by this feature is tiny.
> 2) it complicates Java for everybody
>
> Point 1.
> Perhaps, the set of people is actually not that big, it's the people 
> that write language runtimes i.e. people that really need to use 
> method handles.
> But there is a bigger set of people, the ones that will use that 
> feature, i.e. all the people that actually use java.lang.reflect are 
> people that will use that feature.
> The fact that there is no method reference literal make things like 
> creating bean properties, interceptors, proxy etc, more painful that 
> it should.
> So yes the actual set of people is not big, but the people that will 
> use it is far bigger.
>
> Point 2.
> It's just not true, it's the beauty of target typing. The syntax 
> already exists, the semantics is simple. The overhead is really tiny.
> You can compare it to the introduction of the hexadecimal syntax for 
> double in Java 5.
> The hexadecimal syntax already exists, after Java 5, the syntax can 
> used to specified double, not something really complicated.
>
>>
>> I'm one of those few who has to resort to writing ASM programs to use 
>> MHs when I want them.  So I know exactly how painful it is. And 
>> still, I'd rather impose that pain on myself and my peers than 
>> increase the complexity of Java for everyone for the benefit of a 
>> small few.
>
> see point 1.
>
>>
>> The way I solved this for myself is I wrote a tool which 
>> opportunistically mangles calls to Lookup.findXxx (with constant 
>> arguments) into LDCs using bytecode mangling.  From a 
>> programming-model perspective, this is only slightly more painful 
>> than what you propose -- and works more generally.  (It also works 
>> for indy.)  I much prefer an approach like that.
>
> I've also written this kind of codes, John Rose did too and every 
> people that have wanted to write JUnit Test using method handles have 
> written something similar.
> But focusing only on actual users, people that are forced to use 
> MethodHandle seems very wrong to me.
>
> Rémi
>
>>
>> On 12/30/2012 7:42 PM, Remi Forax wrote:
>>> [to lambda spec list and JSR292 EG list]
>>>
>>> I want to close one problem that we (the JSR 292 EG) have postponed to
>>> Java 8 when we were developing the JSR 292,
>>> a Java syntax to reference a method or a constructor as a method 
>>> handle.
>>> This item was postponed for two reasons, the first one is that the JSR
>>> 292 was a JSR allowing to change the VM spec and not the Java spec and
>>> the second one was that at that time the syntax for method reference 
>>> was
>>> in limbo and we didn't want to choose a syntax that may be incompatible
>>> with the upcoming lambda spec.
>>>
>>> Now that the syntax part of the lambda spec is frozen, we can solve 
>>> that
>>> issue.
>>>
>>> Just before starting, I want to briefly explain why this syntax is 
>>> needed.
>>> A runtime of a dynamic languages uses several well known function
>>> (usually specified as static method) implementing either part of the
>>> runtime logic like a function that check is the runtime class of an
>>> object is equals to a specified class or some global functions of the
>>> language it self (here is the file for JavaScript in Nashorn [1]).
>>> The main issue is that to be used in JSR 292 ecosystem, this functions
>>> must be handled by method handles and these method handles are defined
>>> are static final fields,
>>> because Java doesn't initialize these fields lazily, this greatly
>>> impacts the startup time of any dynamic runtime that runs on the JVM.
>>>
>>> So the goals of the syntax that convert a method reference to a method
>>> handle is:
>>>    - to move some runtime checks at compile time because creating a
>>> method handle involve using a string and a signature.
>>>    - to lazily create these method handle when needed (like the 
>>> lambda).
>>>
>>> The proposed syntax is to use the same syntax as the method reference,
>>> so no new syntax.
>>>    MethodHandle mh = String::length;
>>> the target type is a j.l.i.MethodHandle instead of being a functional
>>> interface
>>>
>>> For the semantics:
>>>    Unlike the method reference with a functional interface, only
>>> unbunded method reference are allowed, so by example,
>>>    a method reference starting with an expression or super are not 
>>> valid.
>>>    Moreover, because there is no type information that the compiler can
>>> extract from a MethodHandle, the semantics in case of several
>>> overloading methods, i.e. several applicable methods is simple, it's an
>>> ambiguity and is rejected by the compiler.
>>>    More formally, a method reference can be converted to a
>>> j.l.i.MethodHandle,
>>>      - only if it's form is
>>>         Type::name, with Type an existing type and name a valid Java
>>> identifier or 'new'.
>>>      - then from the type, the compiler gather all accessible methods
>>> declared in the type type or it's super type (class or interfaces),
>>>        if a method is override-equivalent to another one, the one from
>>> the supertype is discarded.
>>>        if the name is 'new', the compiler gather all constructors of 
>>> the
>>> type type.
>>>        if the type is an array, all public methods of j.l;Object are
>>> available plus clone()
>>>      - if there are more than one method in the set of gathered method,
>>> the reference is ambiguous.
>>>      - if there is no method, the reference is not available
>>>      - if there is one method, a method handle will be created from 
>>> this
>>> method.
>>>
>>> Here is some examples:
>>> class Test {
>>>    public static void foo() { }
>>> }
>>> ...
>>> MethodHandle mh1 = Test::foo;   // foo()V
>>> mh1.invokeExact();
>>>
>>> class Test
>>>    public void bar() { }
>>> }
>>> ...
>>> MethodHandle mh2 = Test::bar;   // bar(Test)V
>>> mh2.invokeExact(new Test());
>>>
>>> class Test {
>>>    interface I {
>>>      public void m();
>>>    }
>>>    static class A implements I {
>>>      public void m() { }
>>>    }
>>> }
>>> ...
>>> MethodHandle mh3 = I::m;   // m(I)V
>>> mh3.invokeExact((I)new A());
>>>
>>> class Test {
>>>    static class B {
>>>       public B(int value) { }
>>>       }
>>>    }
>>> }
>>> ...
>>> MethodHandle mh4 = B::new;   // B(int)
>>> B b = (B)mh4.invokeExact(3);
>>>
>>> class Test {
>>>    class C {  // inner class
>>>      public C() { }
>>>    }
>>> }
>>> ...
>>> MethodHandle mh5 = C::new;     // C(Test)
>>> C c = (C)mh5.invokeExact(new Test());
>>>
>>> class Test {
>>>    static class D {
>>>      Object foo() { }
>>>    }
>>>    static class E {
>>>      String foo() { }  // covariant return type
>>>    }
>>> }
>>> ...
>>> MethodHandle mh8 = E::foo;           // foo(E)String
>>> String s3 = (String)mh8.invokeExact(new E());
>>>
>>> class Test {
>>>    static class F {
>>>      private static void m() {  } // non visible method for the VM, ok
>>> in Java, so the compiler has to generate a bridge
>>>    }
>>> }
>>> ...
>>> MethodHandle mh9 = F::m;
>>> mh9.invoke();
>>>
>>> MethodHandle mh10 = String[]::clone;    // String[].clone returns
>>> Object, so it needs a bridge by the compiler
>>> String[] values = (String[])mh10.invoke(args);
>>>
>>> As you can see the syntax already exist, the semantics of the
>>> attribution (find the correct method) is simpler than the method call
>>> semantics (there is no inference) and after the attribution, the rules
>>> for the generation, bridge, etc. are the same as the one for the method
>>> reference with a functional interface as target.
>>>
>>> The compiler generates like the method reference an invokedynamic 
>>> with a
>>> different bootstrap method that takes the constant method handle as
>>> parameter and create a constant callsite with this constant method
>>> handle (so at runtime a method reference seen as a method handle is
>>> always constant).
>>>
>>> And because Brian will say that this is interesting but Oracle has no
>>> resource that can work on this,
>>> here is the patch for the compiler* part based on lambda/langtools :
>>> http://www-igm.univ-mlv.fr/~forax/tmp/method-ref-method-handle-langtools.diff 
>>>
>>>
>>> and the patch for the jdk part based on lambda/jdk:
>>> http://www-igm.univ-mlv.fr/~forax/tmp/method-ref-method-handle-jdk.diff
>>>
>>> cheers,
>>> Rémi
>>> * note, the mh9 doesn't work at runtime because it also doesn't work 
>>> for
>>> method reference with a functional interface.
>>>
>>> [1]
>>> http://hg.openjdk.java.net/nashorn/jdk8/nashorn/file/b4b05457b8b2/src/jdk/nashorn/internal/runtime/GlobalFunctions.java 
>>>
>>>
>>>
>