From forax at univ-mlv.fr Thu Dec 6 10:56:32 2012 From: forax at univ-mlv.fr (Remi Forax) Date: Thu, 06 Dec 2012 19:56:32 +0100 Subject: Lambda and method reference inference Message-ID: <50C0EA60.8030905@univ-mlv.fr> I think i've already raised this point in August during the face to face meeting and at that time nobody care, maybe this time, I will have more chance :) with interface A { T foo(T a, T a2); } this compile: A b = Integer::plus; // with plus defined as int plus(int,int) but not this one: A c = (int x, int y) -> x + y; Can we, please, have the same rules for both method references and lambdas that have their type of parameters specified ? cheers, R?mi From brian.goetz at oracle.com Thu Dec 6 13:03:06 2012 From: brian.goetz at oracle.com (Brian Goetz) Date: Thu, 06 Dec 2012 16:03:06 -0500 Subject: Lambda and method reference inference In-Reply-To: <50C0EA60.8030905@univ-mlv.fr> References: <50C0EA60.8030905@univ-mlv.fr> Message-ID: <50C1080A.4070009@oracle.com> We did care -- a lot -- but in the other direction :( This is one of those things that seems obvious at first but in reality has a poor cost-benefit balance. First of all, in your example, you can get what you want with: A c = (x, y) -> x + y; If you take the trouble to provide explicit parameter types, you are saying "I want these exact types." I can understand why you would want the compiler to "just do what I mean", but allowing this degree of flexibility unfortunately has a cost, and one of those costs is more inference failures in other cases. At the same time, the benefit is low -- to fix the problem, either leave out the types and let the compiler infer them, or use the exact types you need (there's a good chance your IDE will provide them for you anyway.) It's an easy and local fix. We do allow this flexibility for method references because the user has no control over the types of the referred-to method the way he does with lambda parameter types. And method references exhibit more inference failures. The semantics are modeled on the following intuition: - typing for lambda expressions is like overriding a method; - typing for method references is like calling a method. On 12/6/2012 1:56 PM, Remi Forax wrote: > I think i've already raised this point in August during the face to face > meeting and at that time nobody care, maybe this time, I will have more > chance :) > > with > interface A { > T foo(T a, T a2); > } > > this compile: > A b = Integer::plus; // with plus defined as int > plus(int,int) > > but not this one: > A c = (int x, int y) -> x + y; > > > Can we, please, have the same rules for both method references and > lambdas that have their type of parameters specified ? > > cheers, > R?mi > From forax at univ-mlv.fr Thu Dec 6 14:01:21 2012 From: forax at univ-mlv.fr (Remi Forax) Date: Thu, 06 Dec 2012 23:01:21 +0100 Subject: Lambda and method reference inference In-Reply-To: <50C1080A.4070009@oracle.com> References: <50C0EA60.8030905@univ-mlv.fr> <50C1080A.4070009@oracle.com> Message-ID: <50C115B1.8000904@univ-mlv.fr> On 12/06/2012 10:03 PM, Brian Goetz wrote: > We did care -- a lot -- but in the other direction :( > > This is one of those things that seems obvious at first but in reality > has a poor cost-benefit balance. > > First of all, in your example, you can get what you want with: > > A c = (x, y) -> x + y; it's not what I want, you can not do the same trick with: A c = (int x, int y) -> x == y; or A c = (int x, int y) -> { list.add(x, list.remove(y)); } or any calls that makes a difference between int and Integer. > > If you take the trouble to provide explicit parameter types, you are > saying "I want these exact types." I can understand why you would > want the compiler to "just do what I mean", but allowing this degree > of flexibility unfortunately has a cost, and one of those costs is > more inference failures in other cases. can you be a little more specific, which failures ? > > At the same time, the benefit is low -- to fix the problem, either > leave out the types and let the compiler infer them, or use the exact > types you need (there's a good chance your IDE will provide them for > you anyway.) It's an easy and local fix. The compiler infers the wrong type and I can't using the right type that exactly my problem. The best I can do is introduce a temporary method which is really non intuitive void foo(int x, int y) { list.add(x, list.remove(y)); } A c = (int x, int y) -> MyClass::foo; and sorry, I don't believe that the IDE will cope with that (at least not before 2 years). This 'feature' make even the job of IDE writer far more complex, by example refactoring a lambda to a method reference and vice-versa will have to deal with stupid discrepancies like this one. > > We do allow this flexibility for method references because the user > has no control over the types of the referred-to method the way he > does with lambda parameter types. And method references exhibit more > inference failures. > > The semantics are modeled on the following intuition: > - typing for lambda expressions is like overriding a method; > - typing for method references is like calling a method. I understand why when inferring a lambda with no specified type for formal parameter should be restricted to overriding rules but I don't understand why lambda with specified types for formal parameters has to have different rules different than the one for method reference. cheers, R?mi > > > On 12/6/2012 1:56 PM, Remi Forax wrote: >> I think i've already raised this point in August during the face to face >> meeting and at that time nobody care, maybe this time, I will have more >> chance :) >> >> with >> interface A { >> T foo(T a, T a2); >> } >> >> this compile: >> A b = Integer::plus; // with plus defined as int >> plus(int,int) >> >> but not this one: >> A c = (int x, int y) -> x + y; >> >> >> Can we, please, have the same rules for both method references and >> lambdas that have their type of parameters specified ? >> >> cheers, >> R?mi >> From brian.goetz at oracle.com Thu Dec 6 14:24:44 2012 From: brian.goetz at oracle.com (Brian Goetz) Date: Thu, 06 Dec 2012 17:24:44 -0500 Subject: Lambda and method reference inference In-Reply-To: <50C115B1.8000904@univ-mlv.fr> References: <50C0EA60.8030905@univ-mlv.fr> <50C1080A.4070009@oracle.com> <50C115B1.8000904@univ-mlv.fr> Message-ID: <50C11B2C.7010509@oracle.com> In the assignment case, there's usually not a problem. But nearly all the pain (which we're still not done with, by the way) has been where type inference and method overload selection interact. You've seen the "cyclic inference" errors in plenty of cases where it "should" be "obvious" what the right answer is. By giving more latitude to the compiler to consider box and unbox conversions, you deprive the compiler of type information with which to make better inference decisions. Consider the following simple example: Given overloads: m(IntIntFn) m(Fn) and a call m((int x, int y) -> ..., null) Under the current rules, with the explicit types, we can discard the second candidate early. The selection of the target type IntIntFn may further give us type information which can be used elsewhere in inference. If we have to allow for the idea that we might be calling either version of m, not only do we have less information, but we also are more likely to get into the stability problems that the EG quite broadly agreed we wanted to stay away from (where small changes here can silently change overload choices.) By picking the assignment case, you've ignored 99% of the complexity. All the complexity is in the interaction of inference and overload resolution. If this restriction gives us even 1% better results here -- and it does -- it's totally worth it in my book. On 12/6/2012 5:01 PM, Remi Forax wrote: > On 12/06/2012 10:03 PM, Brian Goetz wrote: >> We did care -- a lot -- but in the other direction :( >> >> This is one of those things that seems obvious at first but in reality >> has a poor cost-benefit balance. >> >> First of all, in your example, you can get what you want with: >> >> A c = (x, y) -> x + y; > > it's not what I want, you can not do the same trick with: > A c = (int x, int y) -> x == y; > or > A c = (int x, int y) -> { list.add(x, list.remove(y)); } > or any calls that makes a difference between int and Integer. > >> >> If you take the trouble to provide explicit parameter types, you are >> saying "I want these exact types." I can understand why you would >> want the compiler to "just do what I mean", but allowing this degree >> of flexibility unfortunately has a cost, and one of those costs is >> more inference failures in other cases. > > can you be a little more specific, which failures ? > >> >> At the same time, the benefit is low -- to fix the problem, either >> leave out the types and let the compiler infer them, or use the exact >> types you need (there's a good chance your IDE will provide them for >> you anyway.) It's an easy and local fix. > > The compiler infers the wrong type and I can't using the right type that > exactly my problem. > The best I can do is introduce a temporary method which is really non > intuitive > > void foo(int x, int y) { list.add(x, list.remove(y)); } > > A c = (int x, int y) -> MyClass::foo; > > and sorry, I don't believe that the IDE will cope with that (at least > not before 2 years). > This 'feature' make even the job of IDE writer far more complex, by > example refactoring a lambda to a method reference and vice-versa will > have to deal with stupid discrepancies like this one. > >> >> We do allow this flexibility for method references because the user >> has no control over the types of the referred-to method the way he >> does with lambda parameter types. And method references exhibit more >> inference failures. >> >> The semantics are modeled on the following intuition: >> - typing for lambda expressions is like overriding a method; >> - typing for method references is like calling a method. > > I understand why when inferring a lambda with no specified type for > formal parameter should be restricted to overriding rules but I don't > understand why lambda with specified types for formal parameters has to > have different rules different than the one for method reference. > > cheers, > R?mi > >> >> >> On 12/6/2012 1:56 PM, Remi Forax wrote: >>> I think i've already raised this point in August during the face to face >>> meeting and at that time nobody care, maybe this time, I will have more >>> chance :) >>> >>> with >>> interface A { >>> T foo(T a, T a2); >>> } >>> >>> this compile: >>> A b = Integer::plus; // with plus defined as int >>> plus(int,int) >>> >>> but not this one: >>> A c = (int x, int y) -> x + y; >>> >>> >>> Can we, please, have the same rules for both method references and >>> lambdas that have their type of parameters specified ? >>> >>> cheers, >>> R?mi >>> > From forax at univ-mlv.fr Thu Dec 6 15:37:21 2012 From: forax at univ-mlv.fr (Remi Forax) Date: Fri, 07 Dec 2012 00:37:21 +0100 Subject: Lambda and method reference inference In-Reply-To: <50C11B2C.7010509@oracle.com> References: <50C0EA60.8030905@univ-mlv.fr> <50C1080A.4070009@oracle.com> <50C115B1.8000904@univ-mlv.fr> <50C11B2C.7010509@oracle.com> Message-ID: <50C12C31.7010406@univ-mlv.fr> On 12/06/2012 11:24 PM, Brian Goetz wrote: > In the assignment case, there's usually not a problem. I've used assignment just because it's easier to see the target type so it's easier to read the mail. > But nearly all the pain (which we're still not done with, by the way) > has been where type inference and method overload selection interact. > You've seen the "cyclic inference" errors in plenty of cases where it > "should" be "obvious" what the right answer is. I claim there is no cyclic inference when you know the type of the formal parameters is know. Cyclic inference is another problem, and IMO should be tackle by considering the result type but that's another debate. We have 3 forms of pseudo lambdas: 1) the lambda with type of the parameters known, that the simplest form because from the type of the parameter you can find the return type by typeching the body using the classical algorithm (no inference here). When you have the return type, you can try all overloads using the same algorithm that the one used to invoke a method i.e first finding the applicable method (once without boxing and varags, once with boxing and no varargs and once with boxing an varargs). This is the very same algorithm as the one introduced by Java 5 (JLS2 if you prefer). So yes you can have ambiguities, exactly like you can have ambiguities when you call a method that is overloaded. 2) method reference, here because we don't specify the type of the parameter, the method reference can have several overloads and the method that takes the method reference can have several overloads. Here we can have cyclic inference problems. Despite that, method reference use the assignment conversion rules describe above. 3) lambda with no type of parameters, again here we can have cyclic inference. The inference is more complex. I understand why it's better to limit ourselves to only overriding conversions. > > By giving more latitude to the compiler to consider box and unbox > conversions, you deprive the compiler of type information with which > to make better inference decisions. Consider the following simple > example: > > Given overloads: > m(IntIntFn) > m(Fn) > > and a call > > m((int x, int y) -> ..., null) > > Under the current rules, with the explicit types, we can discard the > second candidate early. The rules of assignment conversions/invocation conversions will also discard the second candidate early because the first phase is to lookup for applicable method without considering boxing. So your point is moot. > The selection of the target type IntIntFn may further give us type > information which can be used elsewhere in inference. If we have to > allow for the idea that we might be calling either version of m, not > only do we have less information, but we also are more likely to get > into the stability problems that the EG quite broadly agreed we wanted > to stay away from (where small changes here can silently change > overload choices.) > > By picking the assignment case, you've ignored 99% of the complexity. > All the complexity is in the interaction of inference and overload > resolution. If this restriction gives us even 1% better results here > -- and it does -- it's totally worth it in my book. R?mi > > On 12/6/2012 5:01 PM, Remi Forax wrote: >> On 12/06/2012 10:03 PM, Brian Goetz wrote: >>> We did care -- a lot -- but in the other direction :( >>> >>> This is one of those things that seems obvious at first but in reality >>> has a poor cost-benefit balance. >>> >>> First of all, in your example, you can get what you want with: >>> >>> A c = (x, y) -> x + y; >> >> it's not what I want, you can not do the same trick with: >> A c = (int x, int y) -> x == y; >> or >> A c = (int x, int y) -> { list.add(x, list.remove(y)); } >> or any calls that makes a difference between int and Integer. >> >>> >>> If you take the trouble to provide explicit parameter types, you are >>> saying "I want these exact types." I can understand why you would >>> want the compiler to "just do what I mean", but allowing this degree >>> of flexibility unfortunately has a cost, and one of those costs is >>> more inference failures in other cases. >> >> can you be a little more specific, which failures ? >> >>> >>> At the same time, the benefit is low -- to fix the problem, either >>> leave out the types and let the compiler infer them, or use the exact >>> types you need (there's a good chance your IDE will provide them for >>> you anyway.) It's an easy and local fix. >> >> The compiler infers the wrong type and I can't using the right type that >> exactly my problem. >> The best I can do is introduce a temporary method which is really non >> intuitive >> >> void foo(int x, int y) { list.add(x, list.remove(y)); } >> >> A c = (int x, int y) -> MyClass::foo; >> >> and sorry, I don't believe that the IDE will cope with that (at least >> not before 2 years). >> This 'feature' make even the job of IDE writer far more complex, by >> example refactoring a lambda to a method reference and vice-versa will >> have to deal with stupid discrepancies like this one. >> >>> >>> We do allow this flexibility for method references because the user >>> has no control over the types of the referred-to method the way he >>> does with lambda parameter types. And method references exhibit more >>> inference failures. >>> >>> The semantics are modeled on the following intuition: >>> - typing for lambda expressions is like overriding a method; >>> - typing for method references is like calling a method. >> >> I understand why when inferring a lambda with no specified type for >> formal parameter should be restricted to overriding rules but I don't >> understand why lambda with specified types for formal parameters has to >> have different rules different than the one for method reference. >> >> cheers, >> R?mi >> >>> >>> >>> On 12/6/2012 1:56 PM, Remi Forax wrote: >>>> I think i've already raised this point in August during the face to >>>> face >>>> meeting and at that time nobody care, maybe this time, I will have >>>> more >>>> chance :) >>>> >>>> with >>>> interface A { >>>> T foo(T a, T a2); >>>> } >>>> >>>> this compile: >>>> A b = Integer::plus; // with plus defined as int >>>> plus(int,int) >>>> >>>> but not this one: >>>> A c = (int x, int y) -> x + y; >>>> >>>> >>>> Can we, please, have the same rules for both method references and >>>> lambdas that have their type of parameters specified ? >>>> >>>> cheers, >>>> R?mi >>>> >> From forax at univ-mlv.fr Thu Dec 6 23:46:00 2012 From: forax at univ-mlv.fr (Remi Forax) Date: Fri, 07 Dec 2012 08:46:00 +0100 Subject: Lambda and method reference inference In-Reply-To: <50C12C31.7010406@univ-mlv.fr> References: <50C0EA60.8030905@univ-mlv.fr> <50C1080A.4070009@oracle.com> <50C115B1.8000904@univ-mlv.fr> <50C11B2C.7010509@oracle.com> <50C12C31.7010406@univ-mlv.fr> Message-ID: <50C19EB8.2020603@univ-mlv.fr> On 12/06/2012 11:24 PM, Brian Goetz wrote: > In the assignment case, there's usually not a problem. I've used assignment just because it's easier to see the target type so it's easier to read the mail. > But nearly all the pain (which we're still not done with, by the way) > has been where type inference and method overload selection interact. > You've seen the "cyclic inference" errors in plenty of cases where it > "should" be "obvious" what the right answer is. I claim there is no cyclic inference when you know the type of the formal parameters is know. Cyclic inference is another problem, and IMO should be tackle by considering the result type but that's another debate. We have 3 forms of pseudo lambdas: 1) the lambda with type of the parameters known, that the simplest form because from the type of the parameter you can find the return type by typeching the body using the classical algorithm (no inference here). When you have the return type, you can try all overloads using the same algorithm that the one used to invoke a method i.e first finding the applicable method (once without boxing and varags, once with boxing and no varargs and once with boxing an varargs). This is the very same algorithm as the one introduced by Java 5 (JLS2 if you prefer). So yes you can have ambiguities, exactly like you can have ambiguities when you call a method that is overloaded. 2) method reference, here because we don't specify the type of the parameter, the method reference can have several overloads and the method that takes the method reference can have several overloads. Here we can have cyclic inference problems. Despite that, method reference use the assignment conversion rules describe above. 3) lambda with no type of parameters, again here we can have cyclic inference. The inference is more complex. I understand why it's better to limit ourselves to only overriding conversions. > > By giving more latitude to the compiler to consider box and unbox > conversions, you deprive the compiler of type information with which > to make better inference decisions. Consider the following simple > example: > > Given overloads: > m(IntIntFn) > m(Fn) > > and a call > > m((int x, int y) -> ..., null) > > Under the current rules, with the explicit types, we can discard the > second candidate early. The rules of assignment conversions/invocation conversions will also discard the second candidate early because the first phase is to lookup for applicable method without considering boxing. So your point is moot. > The selection of the target type IntIntFn may further give us type > information which can be used elsewhere in inference. If we have to > allow for the idea that we might be calling either version of m, not > only do we have less information, but we also are more likely to get > into the stability problems that the EG quite broadly agreed we wanted > to stay away from (where small changes here can silently change > overload choices.) > > By picking the assignment case, you've ignored 99% of the complexity. > All the complexity is in the interaction of inference and overload > resolution. If this restriction gives us even 1% better results here > -- and it does -- it's totally worth it in my book. R?mi > > On 12/6/2012 5:01 PM, Remi Forax wrote: >> On 12/06/2012 10:03 PM, Brian Goetz wrote: >>> We did care -- a lot -- but in the other direction >>> >>> This is one of those things that seems obvious at first but in reality >>> has a poor cost-benefit balance. >>> >>> First of all, in your example, you can get what you want with: >>> >>> A c = (x, y) -> x + y; >> >> it's not what I want, you can not do the same trick with: >> A c = (int x, int y) -> x == y; >> or >> A c = (int x, int y) -> { list.add(x, list.remove(y)); } >> or any calls that makes a difference between int and Integer. >> >>> >>> If you take the trouble to provide explicit parameter types, you are >>> saying "I want these exact types." I can understand why you would >>> want the compiler to "just do what I mean", but allowing this degree >>> of flexibility unfortunately has a cost, and one of those costs is >>> more inference failures in other cases. >> >> can you be a little more specific, which failures ? >> >>> >>> At the same time, the benefit is low -- to fix the problem, either >>> leave out the types and let the compiler infer them, or use the exact >>> types you need (there's a good chance your IDE will provide them for >>> you anyway.) It's an easy and local fix. >> >> The compiler infers the wrong type and I can't using the right type that >> exactly my problem. >> The best I can do is introduce a temporary method which is really non >> intuitive >> >> void foo(int x, int y) { list.add(x, list.remove(y)); } >> >> A c = (int x, int y) -> MyClass::foo; >> >> and sorry, I don't believe that the IDE will cope with that (at least >> not before 2 years). >> This 'feature' make even the job of IDE writer far more complex, by >> example refactoring a lambda to a method reference and vice-versa will >> have to deal with stupid discrepancies like this one. >> >>> >>> We do allow this flexibility for method references because the user >>> has no control over the types of the referred-to method the way he >>> does with lambda parameter types. And method references exhibit more >>> inference failures. >>> >>> The semantics are modeled on the following intuition: >>> - typing for lambda expressions is like overriding a method; >>> - typing for method references is like calling a method. >> >> I understand why when inferring a lambda with no specified type for >> formal parameter should be restricted to overriding rules but I don't >> understand why lambda with specified types for formal parameters has to >> have different rules different than the one for method reference. >> >> cheers, >> R?mi >> >>> >>> >>> On 12/6/2012 1:56 PM, Remi Forax wrote: >>>> I think i've already raised this point in August during the face to >>>> face >>>> meeting and at that time nobody care, maybe this time, I will have >>>> more >>>> chance >>>> >>>> with >>>> interface A { >>>> T foo(T a, T a2); >>>> } >>>> >>>> this compile: >>>> A b = Integer::plus; // with plus defined as int >>>> plus(int,int) >>>> >>>> but not this one: >>>> A c = (int x, int y) -> x + y; >>>> >>>> >>>> Can we, please, have the same rules for both method references and >>>> lambdas that have their type of parameters specified ? >>>> >>>> cheers, >>>> R?mi >>>> >> From daniel.smith at oracle.com Fri Dec 7 14:10:34 2012 From: daniel.smith at oracle.com (Dan Smith) Date: Fri, 7 Dec 2012 15:10:34 -0700 Subject: Lambda and method reference inference In-Reply-To: <50C0EA60.8030905@univ-mlv.fr> References: <50C0EA60.8030905@univ-mlv.fr> Message-ID: <1AF73124-AC30-479E-A051-BD2A6D864B1E@oracle.com> On Dec 6, 2012, at 11:56 AM, Remi Forax wrote: > I think i've already raised this point in August during the face to face meeting and at that time nobody care, maybe this time, I will have more chance :) > > with > interface A { > T foo(T a, T a2); > } > > this compile: > A b = Integer::plus; // with plus defined as int plus(int,int) > > but not this one: > A c = (int x, int y) -> x + y; > > > Can we, please, have the same rules for both method references and lambdas that have their type of parameters specified ? For what it's worth, I have been hanging on to this idea as something to circle back on. It would be useful if you could expand on your motivation. I'm guessing it is one (or both) of: 1) You don't like the model we have and prefer yours. 2) You want direct control over the signature of the method generated by the compiler, for performance reasons. Where we left this last is that we like the model ("we" meaning Oracle folks, plus I haven't heard much opposition from other EG members), but it's something to reconsider if we find that performant code requires some manual intervention for tuning (e.g., minimizing boxing). Also, it would be useful to define the scope of what you're after: 1) Parameter types can be boxed/unboxed 2) Parameter types can be widened (or anything else legal in an invocation context) 2') Thus, type parameters for generic descriptors can be widened to their bounds 3) Parameter types can be grouped via varargs Consistency with method references would suggest doing all of these... --- Addressing the model: Brian summarized the model succinctly?lambdas _override_ a functional interface method; method references are _invocations from_ a functional interface method. In other words, this: Integer::plus is shorthand for this: (x,y) -> Integer.plus(x, y) If the types of 'x' and 'y' are 'Integer', that's fine. It is not necessary that 'x' and 'y' have type 'int'. You point out that certain adaptations on the parameters can't be performed, and you have to do that yourself inside the body (e.g., cast Integer to int or call a method that will do the conversion automatically). In the rare circumstance in which such conversions have semantic consequences, that's true enough. We have a pretty good story for widening where the target type uses wildcards. For example: Predicate p = (Number n) -> ...; // legal Here, we infer that the instantiation of Predicate is Predicate. So probably the main practical impact of the problem, if there is one, is with respect to boxing/unboxing. If we eventually support primitives as type arguments, that problem may go away, too... Brian points out that equality is more useful for inference than convertibility. For example, the above wildcard inference problem essentially tries to find an alpha such that Predicate is a valid target type. With equality constraints, this means, trivially: alpha = Number (More generally, we could have something like a descriptor parameter type Foo> and a lambda parameter type Foo>; it's always just pattern matching to solve for alpha.) In contrast, with convertibility constraints, we get alpha :> Number And then we need to work out whether there are other constraints on alpha (e.g., the declared bounds), and there's a bunch of complexity to solve for the "best" alpha and hope we're right. Of course we know how to do this, because that's how we handle method invocations, but in the context of deciding what the descriptor of a target type is, it's sure nice to keep things simple. Similarly, we looked into contravariance for the most-specific algorithm. If you look at my write-up about overload resolution from a few months ago (jsr-335-eg list, subject "Method invocations & applicability", subheading "Functional Interface Parameter Types"), you'll see discussion about how both contravariance and covariance are problematic. So we settled on requiring that the parameter types be the same. Turns out, that was a really good choice. Because if the parameter types are different, we can't make any guarantees about the interpretation of the body. As we've explored the "most specific" analysis and decided, e.g., that we want to know whether certain lambda returns produce primitives or references, it's really nice to have a stable interpretation of the body with which to work. Summarizing, the idea that parameter types should match with something else pops up quite a bit in our design, and typically frees up the compiler to do more useful things. It's not just an anomaly in the lambda compatibility rules. ?Dan From daniel.smith at oracle.com Fri Dec 7 20:42:53 2012 From: daniel.smith at oracle.com (Dan Smith) Date: Fri, 7 Dec 2012 21:42:53 -0700 Subject: JSR 335 Lambda Specification, 0.6.0 Message-ID: <2F01A5BB-B2B5-477E-BAC5-6E1A6768D090@oracle.com> Attached is the 0.6.0 revision of the Lambda spec. I'm collecting feedback for a week or two, and then will push a revision out for public review. ?Dan ----- Change notes: Overall: Split Part F into Part F, Overload Resolution and Part G, Type Inference. Split the old Part G into Part H, Default Methods and Part J, Java Virtual Machine. Functional Interfaces: Defined intersections as a kind of functional interface type. Defined the descriptor of wildcard-parameterized types. Added the annotation. Lambda Expressions: Added the _ keyword and the -> operator to the grammar. Method References: Added qualified super references. Added array constructor references. Added the :: separator to the grammar. Poly Expressions: Allowed casts to intersection types. Refined the conditions under which a method invocation is considered a poly expression. Introduced rules for classifying and typing conditional expressions. Typing and Evaluation: Refined the presentation of inference of wildcard-parameterized type instantiations. Allowed arbitrary statement expressions in lambda bodies targeting a void return. Always interpret lambda returns in an assignment context. Defined runtime behavior when targeting an intersection functional interface type. Eliminated support for unbound inner class constructor references. Added support for qualified super references and array constructor references. Restricted supported forms of static method references. Overload Resolution: Defined potentially applicable for methods with functional interface parameter types targeted by lambda expressions. Revised most specific method analysis to consider the invocation argument expressions, compare descriptors of functional interface types, and minimize boxing/unboxing. Type Inference: Added a stub for most specific method inference. Default Methods: Modified syntax, treating default as a method modifier. Added support for static interface methods. Refined the inheritance rules for default and abstract methods. Enclosing instances of inner classes can be interfaces. Added support for the package access modifier. ----- Major things left until the next spec iteration: - Spec for inference enhancements - Refine JVM resolution rules; specify 'invokespecial' and 'invokestatic' - Spec for JVM bridge methods ----- -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.openjdk.java.net/pipermail/lambda-spec-experts/attachments/20121207/efec59e1/jsr335-0.6.0-0001.html -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.openjdk.java.net/pipermail/lambda-spec-experts/attachments/20121207/efec59e1/jsr335-0.6.0-diff-0001.html From forax at univ-mlv.fr Sat Dec 8 04:09:42 2012 From: forax at univ-mlv.fr (Remi Forax) Date: Sat, 08 Dec 2012 13:09:42 +0100 Subject: Lambda and method reference inference In-Reply-To: <1AF73124-AC30-479E-A051-BD2A6D864B1E@oracle.com> References: <50C0EA60.8030905@univ-mlv.fr> <1AF73124-AC30-479E-A051-BD2A6D864B1E@oracle.com> Message-ID: <50C32E06.3020306@univ-mlv.fr> On 12/07/2012 11:10 PM, Dan Smith wrote: > On Dec 6, 2012, at 11:56 AM, Remi Forax wrote: > >> I think i've already raised this point in August during the face to face meeting and at that time nobody care, maybe this time, I will have more chance :) >> >> with >> interface A { >> T foo(T a, T a2); >> } >> >> this compile: >> A b = Integer::plus; // with plus defined as int plus(int,int) >> >> but not this one: >> A c = (int x, int y) -> x + y; >> >> >> Can we, please, have the same rules for both method references and lambdas that have their type of parameters specified ? > For what it's worth, I have been hanging on to this idea as something to circle back on. > > It would be useful if you could expand on your motivation. I'm guessing it is one (or both) of: > 1) You don't like the model we have and prefer yours. > 2) You want direct control over the signature of the method generated by the compiler, for performance reasons. it's 2) but not for performance reason but for semantics reason, given that in Java, Integer and int doesn't have the same semantics, as a user I want the to specifies that a lambda takes two ints and not two integers. You can argue that users can insert cast to int where needed but from experience, users poorly understand where to put cast in the middle of the code and given that in our case types are implicit, it doesn't help. About performance, that true that if the signature only use primitive type, it will be easier for the VM to remove all boxing if the lambda proxies generated use method handles because the current optimizations done by Hotspot is enough, we don't have to rely on some hypothetical optimization not yet written. > > Where we left this last is that we like the model ("we" meaning Oracle folks, plus I haven't heard much opposition from other EG members), but it's something to reconsider if we find that performant code requires some manual intervention for tuning (e.g., minimizing boxing). see above, performance is part of the equation but not the important one. Being able to specify the right semantics is more important. > > Also, it would be useful to define the scope of what you're after: > 1) Parameter types can be boxed/unboxed > 2) Parameter types can be widened (or anything else legal in an invocation context) > 2') Thus, type parameters for generic descriptors can be widened to their bounds > 3) Parameter types can be grouped via varargs > > Consistency with method references would suggest doing all of these... yes, here consistency is the important word. The JLS defines two sets of rules for compatibility, invocation conversions and overriding conversions. I don't see why we need to invent a new one. The lambda spec should just make a difference between lambda with parameter types that should use invocation conversions and lambda without parameter types that should use overriding conversions. Or said differently, in a perfect word, lambda and method reference should all use invocation conversions but because we have to infer parameter type if lambda doesn't specify parameter types, we prefer to use overriding conversions to make the inference algorithm less surprising. > > --- > > Addressing the model: Brian summarized the model succinctly?lambdas _override_ a functional interface method; method references are _invocations from_ a functional interface method. In other words, this: > > Integer::plus > > is shorthand for this: > > (x,y) -> Integer.plus(x, y) > > If the types of 'x' and 'y' are 'Integer', that's fine. It is not necessary that 'x' and 'y' have type 'int'. > > You point out that certain adaptations on the parameters can't be performed, and you have to do that yourself inside the body (e.g., cast Integer to int or call a method that will do the conversion automatically). In the rare circumstance in which such conversions have semantic consequences, that's true enough. I disagree, it will not be that rare because features all goes in the same direction. - there is no way to declare a collection of primitive type => so user use wrapper type and usually it works - collection API use lambdas to filter/map so => so users use lambda to specify boolean expression like equality test - lambda doesn't require to specify type of parameters => so users will not see that the inferred type is boxed > > We have a pretty good story for widening where the target type uses wildcards. For example: > > Predicate p = (Number n) -> ...; // legal > > Here, we infer that the instantiation of Predicate is Predicate. > > So probably the main practical impact of the problem, if there is one, is with respect to boxing/unboxing. If we eventually support primitives as type arguments, that problem may go away, too... I think i will be dead before Java will truly support primitive types in parameter types. The problem is the VM/runtime not the compiler. > > Brian points out that equality is more useful for inference than convertibility. For example, the above wildcard inference problem essentially tries to find an alpha such that Predicate is a valid target type. With equality constraints, this means, trivially: > > alpha = Number > > (More generally, we could have something like a descriptor parameter type Foo> and a lambda parameter type Foo>; it's always just pattern matching to solve for alpha.) > > In contrast, with convertibility constraints, we get > > alpha :> Number > > And then we need to work out whether there are other constraints on alpha (e.g., the declared bounds), and there's a bunch of complexity to solve for the "best" alpha and hope we're right. Of course we know how to do this, because that's how we handle method invocations, but in the context of deciding what the descriptor of a target type is, it's sure nice to keep things simple. yes, when inferring the type of the parameters, it's better to keep the things simple for the user too. I don't know if pattern matching is enough but at least the error message will be understandable. > > Similarly, we looked into contravariance for the most-specific algorithm. If you look at my write-up about overload resolution from a few months ago (jsr-335-eg list, subject "Method invocations & applicability", subheading "Functional Interface Parameter Types"), you'll see discussion about how both contravariance and covariance are problematic. So we settled on requiring that the parameter types be the same. Turns out, that was a really good choice. Because if the parameter types are different, we can't make any guarantees about the interpretation of the body. As we've explored the "most specific" analysis and decided, e.g., that we want to know whether certain lambda returns produce primitives or references, it's really nice to have a stable interpretation of the body with which to work. > > Summarizing, the idea that parameter types should match with something else pops up quite a bit in our design, and typically frees up the compiler to do more useful things. It's not just an anomaly in the lambda compatibility rules. The idea that the Inference use less conversions than the one specified by the method invocation algorithm seems a good practical choice. I can not say if it's a good idea or not. But i don't see how it's related to the case that doesn't require inference of the parameter types. > > ?Dan R?mi From forax at univ-mlv.fr Sat Dec 8 15:47:26 2012 From: forax at univ-mlv.fr (Remi Forax) Date: Sun, 09 Dec 2012 00:47:26 +0100 Subject: JSR 335 Lambda Specification, 0.6.0 In-Reply-To: <2F01A5BB-B2B5-477E-BAC5-6E1A6768D090@oracle.com> References: <2F01A5BB-B2B5-477E-BAC5-6E1A6768D090@oracle.com> Message-ID: <50C3D18E.2070704@univ-mlv.fr> On 12/08/2012 05:42 AM, Dan Smith wrote: > Attached is the 0.6.0 revision of the Lambda spec. I'm collecting feedback for a week or two, and then will push a revision out for public review. section 3.9: I still don't see why we introduce a new keyword ('_') that will break compatibility for not using it. We can do that later if we want (or never). section 18.2.1: "If the expression is a class instance creation expression or a method invocation, and its type when treated as a standalone expression is /S/, the result is /S ?_c T/." There is nothing in the discussion that say why 1) it should not be an error instead of using Object as type argument (compatibility maybe ??) 2) why the inference can not be propagated and wait for the result of the poly expression. In section 18.2.1.2, in the discussion, the item 4, I don't understand why it doesn't work. javac agree with me :) interface Sequence { M mapReduce(M base, Function mapper, BinaryOperator op); } interface Person { public String name(); } public static void main(String[] args) { Sequence ss = null; String names = ss.mapReduce("", p -> p.name(), (s1, s2) -> s1+", "+s2); // ok } E is person because ss is a Sequence of Person, M is a String because base="", so where is the issue ?? Section 18.2.1.3 I'm lost here. Why do you erase the type of the method reference considering it as a lambda without parameter types ?? Section 18.2.5 I agree that checked exception should not be part of the applicability of a method. Section 18.5.1 and section 18.5.2, I think you should add a shortcut in the applicability rules. If there is only one method with the correct name and number of parameters, the inference should use the target type. Or said differently, if finding the applicable method fail because a type argument is not found, the compiler should try again inserting the fact that the target type is known, and if only one method is applicable, it's the most specific method. so it will not solve this case like this that we rule out during the face to face meeting: Future future = executor.submit(() -> list.add("foo")); // a Callable or a Runnable but solve ArrayList list = stream.into(new ArrayList<>()); > > ?Dan R?mi > > ----- > > Change notes: > > Overall: Split Part F into Part F, Overload Resolution and Part G, Type Inference. Split the old Part G into Part H, Default Methods and Part J, Java Virtual Machine. > > Functional Interfaces: Defined intersections as a kind of functional interface type. Defined the descriptor of wildcard-parameterized types. Added the annotation. > > Lambda Expressions: Added the _ keyword and the -> operator to the grammar. > > Method References: Added qualified super references. Added array constructor references. Added the :: separator to the grammar. > > Poly Expressions: Allowed casts to intersection types. Refined the conditions under which a method invocation is considered a poly expression. Introduced rules for classifying and typing conditional expressions. > > Typing and Evaluation: Refined the presentation of inference of wildcard-parameterized type instantiations. Allowed arbitrary statement expressions in lambda bodies targeting a void return. Always interpret lambda returns in an assignment context. Defined runtime behavior when targeting an intersection functional interface type. Eliminated support for unbound inner class constructor references. Added support for qualified super references and array constructor references. Restricted supported forms of static method references. > > Overload Resolution: Defined potentially applicable for methods with functional interface parameter types targeted by lambda expressions. Revised most specific method analysis to consider the invocation argument expressions, compare descriptors of functional interface types, and minimize boxing/unboxing. > > Type Inference: Added a stub for most specific method inference. > > Default Methods: Modified syntax, treating default as a method modifier. Added support for static interface methods. Refined the inheritance rules for default and abstract methods. Enclosing instances of inner classes can be interfaces. Added support for the package access modifier. > > ----- > > Major things left until the next spec iteration: > - Spec for inference enhancements > - Refine JVM resolution rules; specify 'invokespecial' and 'invokestatic' > - Spec for JVM bridge methods > > ----- > From daniel.smith at oracle.com Mon Dec 10 12:38:37 2012 From: daniel.smith at oracle.com (Dan Smith) Date: Mon, 10 Dec 2012 13:38:37 -0700 Subject: Lambda and method reference inference In-Reply-To: <50C32E06.3020306@univ-mlv.fr> References: <50C0EA60.8030905@univ-mlv.fr> <1AF73124-AC30-479E-A051-BD2A6D864B1E@oracle.com> <50C32E06.3020306@univ-mlv.fr> Message-ID: <4CE01260-6415-41A3-A24A-96D64244949C@oracle.com> On Dec 8, 2012, at 5:09 AM, Remi Forax wrote: > On 12/07/2012 11:10 PM, Dan Smith wrote: >> It would be useful if you could expand on your motivation. I'm guessing it is one (or both) of: >> 1) You don't like the model we have and prefer yours. >> 2) You want direct control over the signature of the method generated by the compiler, for performance reasons. > > it's 2) but not for performance reason but for semantics reason, given that in Java, Integer and int doesn't have the same semantics, as a user I want the to specifies that a lambda takes two ints and not two integers. You can argue that users can insert cast to int where needed but from experience, users poorly understand where to put cast in the middle of the code and given that in our case types are implicit, it doesn't help. > > About performance, that true that if the signature only use primitive type, it will be easier for the VM to remove all boxing if the lambda proxies generated use method handles because the current optimizations done by Hotspot is enough, we don't have to rely on some hypothetical optimization not yet written. > >> >> Where we left this last is that we like the model ("we" meaning Oracle folks, plus I haven't heard much opposition from other EG members), but it's something to reconsider if we find that performant code requires some manual intervention for tuning (e.g., minimizing boxing). > > see above, performance is part of the equation but not the important one. Being able to specify the right semantics is more important. I would describe what you're saying as (1)?you feel like modeling the lambda invocation as delegation (functional interface method calls the lambda) rather than overriding (functional interface method _is_ the lambda) is nicer. >> You point out that certain adaptations on the parameters can't be performed, and you have to do that yourself inside the body (e.g., cast Integer to int or call a method that will do the conversion automatically). In the rare circumstance in which such conversions have semantic consequences, that's true enough. > > I disagree, it will not be that rare because features all goes in the same direction. > - there is no way to declare a collection of primitive type > => so user use wrapper type and usually it works > - collection API use lambdas to filter/map so > => so users use lambda to specify boolean expression like equality test > - lambda doesn't require to specify type of parameters > => so users will not see that the inferred type is boxed Boxing/unboxing will not be rare. But use of lambda parameters in contexts in which it matters whether they're boxed/unboxed or not (e.g., as you point out, '==' or 'List.remove') will be rare. ?Dan From daniel.smith at oracle.com Mon Dec 10 13:30:26 2012 From: daniel.smith at oracle.com (Dan Smith) Date: Mon, 10 Dec 2012 14:30:26 -0700 Subject: JSR 335 Lambda Specification, 0.6.0 In-Reply-To: <50C3D18E.2070704@univ-mlv.fr> References: <2F01A5BB-B2B5-477E-BAC5-6E1A6768D090@oracle.com> <50C3D18E.2070704@univ-mlv.fr> Message-ID: Thanks for the feedback! Caveat on Part G: I haven't touched it since the last round, and need to revise it to get up to speed with our latest work on inference. That's near the top of my list now... On Dec 8, 2012, at 4:47 PM, Remi Forax wrote: > section 18.2.1: > "If the expression is a class instance creation expression or a method invocation, > and its type when treated as a standalone expression is /S/, the result is /S ?_c T/." > > There is nothing in the discussion that say why > 1) it should not be an error instead of using Object as type argument (compatibility maybe ??) > 2) why the inference can not be propagated and wait for the result of the poly expression. This will be addressed. Keep in mind that this clause only applies where T contains inference variables. We _do_ use the target in simpler cases in which we don't have to infer the target first (and this is a change from 7, which never used targets in invocation contexts). The new plan for inference is to expand on this, supporting inference variables in generic method invocation targets by hoisting constraints from the nested invocation into the outer invocation, and solving them all together. I think that's what you're after. Your comment about 'Object' could apply in other contexts, too. Are you suggesting that inference should never use 'Object' as a fallback result, and should produce errors instead? That's a plausible approach (but with some compatibility issues); what's your motivation? > In section 18.2.1.2, in the discussion, the item 4, I don't understand why it doesn't work. > javac agree with me :) > > interface Sequence { > M mapReduce(M base, Function mapper, BinaryOperator op); > } > interface Person { > public String name(); > } > public static void main(String[] args) { > Sequence ss = null; > String names = ss.mapReduce("", p -> p.name(), (s1, s2) -> s1+", "+s2); // ok > } > > E is person because ss is a Sequence of Person, M is a String because base="", so where is the issue ?? This will also be addressed. This approach to inference variables in lambda parameters?we just don't support it?was a baseline. The compiler has experimented with various approaches to resolving inference variables as we go; we have a solid plan now, but it needs to be written up. > Section 18.2.1.3 > I'm lost here. Why do you erase the type of the method reference considering it as a lambda without parameter types ?? Not sure what you mean. There's no erasure in 18.2.1.3. As the discussion points out, the handling of method references mimics both of the previous two areas you asked about: first, like an implicitly-typed lambda, we can't resolve a method reference with inference variables in the parameter types; second, like a nested method invocation, we can't use its target type if the target contains inference variables. Both of our new inference strategies will come into play here. > Section 18.2.5 > I agree that checked exception should not be part of the applicability of a method. Good. :-) In our overload resolution strategy (which I'm planning to send a separate post about), that's where we've ended up. > Section 18.5.1 and section 18.5.2, > > I think you should add a shortcut in the applicability rules. > If there is only one method with the correct name and number of parameters, the inference should use the target type. > Or said differently, if finding the applicable method fail because a type argument is not found, the compiler should try again inserting the fact that the target type is known, > and if only one method is applicable, it's the most specific method. > > so it will not solve this case like this that we rule out during the face to face meeting: > Future future = executor.submit(() -> list.add("foo")); // a Callable or a Runnable > but solve > ArrayList list = stream.into(new ArrayList<>()); To get what you want, you'd also need some interleaving of reduction and resolution: we need to solve alpha=String before looking at the argument to 'into'. We considered an approach like this, but found it to be too ad hoc, so we settled on our current plan. We'll be able to handle the nested 'new ArrayList<>'. Callable vs. Runnable turns out to be a fairly easy problem to solve: in the most specific test, we just prefer functional interfaces that produce values over functional interfaces that produce 'void'. We already do a subtyping check, so this is just an extra thing in the same place (informally, in this context, 'void' is a supertype of everything). See Part F, 15.12.2.5. ?Dan From forax at univ-mlv.fr Mon Dec 10 16:01:37 2012 From: forax at univ-mlv.fr (Remi Forax) Date: Tue, 11 Dec 2012 01:01:37 +0100 Subject: JSR 335 Lambda Specification, 0.6.0 In-Reply-To: References: <2F01A5BB-B2B5-477E-BAC5-6E1A6768D090@oracle.com> <50C3D18E.2070704@univ-mlv.fr> Message-ID: <50C677E1.8050408@univ-mlv.fr> On 12/10/2012 10:30 PM, Dan Smith wrote: > Thanks for the feedback! > > Caveat on Part G: I haven't touched it since the last round, and need to revise it to get up to speed with our latest work on inference. That's near the top of my list now... ok. > > On Dec 8, 2012, at 4:47 PM, Remi Forax wrote: > >> section 18.2.1: >> "If the expression is a class instance creation expression or a method invocation, >> and its type when treated as a standalone expression is /S/, the result is /S ?_c T/." >> >> There is nothing in the discussion that say why >> 1) it should not be an error instead of using Object as type argument (compatibility maybe ??) >> 2) why the inference can not be propagated and wait for the result of the poly expression. > This will be addressed. > > Keep in mind that this clause only applies where T contains inference variables. We _do_ use the target in simpler cases in which we don't have to infer the target first (and this is a change from 7, which never used targets in invocation contexts). yes, I know. > > The new plan for inference is to expand on this, supporting inference variables in generic method invocation targets by hoisting constraints from the nested invocation into the outer invocation, and solving them all together. I think that's what you're after. yes > > Your comment about 'Object' could apply in other contexts, too. Are you suggesting that inference should never use 'Object' as a fallback result, and should produce errors instead? yes > That's a plausible approach (but with some compatibility issues); what's your motivation? The issue is that if the inference pick Object and fail later, the error message doesn't even reference the location where the inference fail so it's hard to debug. You have to read the error, find why you don"t have the correct signature and backtrack to the place where the inference was not able to infer correctly the type variable. So it defeat of the cherish principle of Java, fail early. One month ago a student of mine summary this problem by saying that even the inference algorithm doesn't understand the inference algorithm and give up boldly. Now, because the inference will also use the target type in 8, maybe there will be less cases where the inference algorithm will pick Object. And I have no idea about the impact of the compatibility issues. > >> In section 18.2.1.2, in the discussion, the item 4, I don't understand why it doesn't work. >> javac agree with me :) >> >> interface Sequence { >> M mapReduce(M base, Function mapper, BinaryOperator op); >> } >> interface Person { >> public String name(); >> } >> public static void main(String[] args) { >> Sequence ss = null; >> String names = ss.mapReduce("", p -> p.name(), (s1, s2) -> s1+", "+s2); // ok >> } >> >> E is person because ss is a Sequence of Person, M is a String because base="", so where is the issue ?? > This will also be addressed. > > This approach to inference variables in lambda parameters?we just don't support it?was a baseline. The compiler has experimented with various approaches to resolving inference variables as we go; we have a solid plan now, but it needs to be written up. Ok, cool. > >> Section 18.2.1.3 >> I'm lost here. Why do you erase the type of the method reference considering it as a lambda without parameter types ?? > Not sure what you mean. There's no erasure in 18.2.1.3. > > As the discussion points out, the handling of method references mimics both of the previous two areas you asked about: first, like an implicitly-typed lambda, we can't resolve a method reference with inference variables in the parameter types; second, like a nested method invocation, we can't use its target type if the target contains inference variables. > > Both of our new inference strategies will come into play here. Erasure was not the best term here. Suppose we have class MyClass { public int myMethod(int t) { ... } public long myMethod(long t) { ... } } The question is why do you want to translate System.out.println(MyClass::myMethod) to System.out.println(e -> MyClass.myMethod(e)) instead of translating it into System.out.println(int e -> MyClass.myMethod(e)) or System.out.println(long e -> MyClass.myMethod(e)). the overloads of myMethod will give you all the combination that are possible, you have to see if there are applicable and then try to find the most specific one. > >> Section 18.2.5 >> I agree that checked exception should not be part of the applicability of a method. > Good. :-) In our overload resolution strategy (which I'm planning to send a separate post about), that's where we've ended up. > >> Section 18.5.1 and section 18.5.2, >> >> I think you should add a shortcut in the applicability rules. >> If there is only one method with the correct name and number of parameters, the inference should use the target type. >> Or said differently, if finding the applicable method fail because a type argument is not found, the compiler should try again inserting the fact that the target type is known, >> and if only one method is applicable, it's the most specific method. >> >> so it will not solve this case like this that we rule out during the face to face meeting: >> Future future = executor.submit(() -> list.add("foo")); // a Callable or a Runnable >> but solve >> ArrayList list = stream.into(new ArrayList<>()); > To get what you want, you'd also need some interleaving of reduction and resolution: we need to solve alpha=String before looking at the argument to 'into'. > > We considered an approach like this, but found it to be too ad hoc, so we settled on our current plan. We'll be able to handle the nested 'new ArrayList<>'. yes, it's to ad-hocy, i'm glad that you have find a better way. > > Callable vs. Runnable turns out to be a fairly easy problem to solve: in the most specific test, we just prefer functional interfaces that produce values over functional interfaces that produce 'void'. We already do a subtyping check, so this is just an extra thing in the same place (informally, in this context, 'void' is a supertype of everything). See Part F, 15.12.2.5. I will take a look tomorrow. > > ?Dan thanks, R?mi From forax at univ-mlv.fr Tue Dec 11 00:27:02 2012 From: forax at univ-mlv.fr (Remi Forax) Date: Tue, 11 Dec 2012 09:27:02 +0100 Subject: Lambda and method reference inference In-Reply-To: <4CE01260-6415-41A3-A24A-96D64244949C@oracle.com> References: <50C0EA60.8030905@univ-mlv.fr> <1AF73124-AC30-479E-A051-BD2A6D864B1E@oracle.com> <50C32E06.3020306@univ-mlv.fr> <4CE01260-6415-41A3-A24A-96D64244949C@oracle.com> Message-ID: <50C6EE56.5040305@univ-mlv.fr> On 12/10/2012 09:38 PM, Dan Smith wrote: > On Dec 8, 2012, at 5:09 AM, Remi Forax wrote: > >> On 12/07/2012 11:10 PM, Dan Smith wrote: >>> It would be useful if you could expand on your motivation. I'm guessing it is one (or both) of: >>> 1) You don't like the model we have and prefer yours. >>> 2) You want direct control over the signature of the method generated by the compiler, for performance reasons. >> it's 2) but not for performance reason but for semantics reason, given that in Java, Integer and int doesn't have the same semantics, as a user I want the to specifies that a lambda takes two ints and not two integers. You can argue that users can insert cast to int where needed but from experience, users poorly understand where to put cast in the middle of the code and given that in our case types are implicit, it doesn't help. >> >> About performance, that true that if the signature only use primitive type, it will be easier for the VM to remove all boxing if the lambda proxies generated use method handles because the current optimizations done by Hotspot is enough, we don't have to rely on some hypothetical optimization not yet written. >> >>> Where we left this last is that we like the model ("we" meaning Oracle folks, plus I haven't heard much opposition from other EG members), but it's something to reconsider if we find that performant code requires some manual intervention for tuning (e.g., minimizing boxing). >> see above, performance is part of the equation but not the important one. Being able to specify the right semantics is more important. > I would describe what you're saying as (1)?you feel like modeling the lambda invocation as delegation (functional interface method calls the lambda) rather than overriding (functional interface method _is_ the lambda) is nicer. It's not nicer, what user want is to not think too much about boxing, so if they specify that their lambda takes ints, it should just work. Note that I don't propose to have the *full* invocation semantics because i don't think that allowing varargs call is a good idea. So if you want you can see it as the overriding semantics + boxing conversions. > >>> You point out that certain adaptations on the parameters can't be performed, and you have to do that yourself inside the body (e.g., cast Integer to int or call a method that will do the conversion automatically). In the rare circumstance in which such conversions have semantic consequences, that's true enough. >> I disagree, it will not be that rare because features all goes in the same direction. >> - there is no way to declare a collection of primitive type >> => so user use wrapper type and usually it works >> - collection API use lambdas to filter/map so >> => so users use lambda to specify boolean expression like equality test >> - lambda doesn't require to specify type of parameters >> => so users will not see that the inferred type is boxed > Boxing/unboxing will not be rare. But use of lambda parameters in contexts in which it matters whether they're boxed/unboxed or not (e.g., as you point out, '==' or 'List.remove') will be rare. == is a filter will not be rare. > > ?Dan R?mi From daniel.smith at oracle.com Tue Dec 11 09:52:03 2012 From: daniel.smith at oracle.com (Dan Smith) Date: Tue, 11 Dec 2012 10:52:03 -0700 Subject: JSR 335 Lambda Specification, 0.6.0 In-Reply-To: <50C677E1.8050408@univ-mlv.fr> References: <2F01A5BB-B2B5-477E-BAC5-6E1A6768D090@oracle.com> <50C3D18E.2070704@univ-mlv.fr> <50C677E1.8050408@univ-mlv.fr> Message-ID: On Dec 10, 2012, at 5:01 PM, Remi Forax wrote: >>> Section 18.2.1.3 >>> I'm lost here. Why do you erase the type of the method reference considering it as a lambda without parameter types ?? >> Not sure what you mean. There's no erasure in 18.2.1.3. >> >> As the discussion points out, the handling of method references mimics both of the previous two areas you asked about: first, like an implicitly-typed lambda, we can't resolve a method reference with inference variables in the parameter types; second, like a nested method invocation, we can't use its target type if the target contains inference variables. >> >> Both of our new inference strategies will come into play here. > > Erasure was not the best term here. > Suppose we have > class MyClass { > public int myMethod(int t) { ... } > public long myMethod(long t) { ... } > } > > The question is why do you want to translate System.out.println(MyClass::myMethod) to > System.out.println(e -> MyClass.myMethod(e)) instead of translating it into > System.out.println(int e -> MyClass.myMethod(e)) or System.out.println(long e -> MyClass.myMethod(e)). > > the overloads of myMethod will give you all the combination that are possible, you have to see if there are applicable > and then try to find the most specific one. I think I still might be missing your point. If you're talking about the process we use to disambiguate between 'myMethod' candidates, we decide which ones are applicable based on the target type. Your example ('System.out.println') doesn't have a functional interface target, so it would be an error. If the target had parameter type 'long' or 'Long', then only the 'long' version of 'myMethod' would be applicable. If the target had parameter type 'int', 'Integer', 'short', etc., then both the 'int' and the 'long' versions of 'myMethod' would be applicable, and we'd use the most-specific analysis to pick one (the 'int' version). If the target had a to-be-inferred parameter type, then the resolution would depend on how that parameter type is ultimately inferred. It would be premature to try to resolve 'myMethod' before we resolve the target parameter type. (Otherwise, the answer we get during inference could be different than the answer we get after inference, and that's never a good thing.) It _is_ safe to eagerly resolve the method reference if there is only one potentially-applicable method. That's something we're considering. It would avoid some ambiguity errors and also provide extra information to inference (e.g., we could get constraints on the target return type from the method reference return before resolving inference vars in the target parameter types). ?Dan From daniel.smith at oracle.com Tue Dec 11 12:36:20 2012 From: daniel.smith at oracle.com (Dan Smith) Date: Tue, 11 Dec 2012 13:36:20 -0700 Subject: JSR 335 Lambda Specification, 0.6.0 In-Reply-To: References: <2F01A5BB-B2B5-477E-BAC5-6E1A6768D090@oracle.com> Message-ID: Ah, thanks. It seems that the mailing list stripped the zip file attachment from my message. It contained separate files for each section. If anybody wants the zipped directory, please contact me directly. There was also a bug in my script, so it failed to concatenate the last two sections when creating the one-page version. Here is the one-page version, fixed. (Note that part J is incomplete, as described in the "Unfinished Items" list.) ?Dan -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.openjdk.java.net/pipermail/lambda-spec-experts/attachments/20121211/1d34f5bc/jsr335-0.6.0-0001.html -------------- next part -------------- On Dec 11, 2012, at 1:19 PM, Daniel Heidinga wrote: > These sections appear to be missing from the document: > * Part H: Default Methods: Describes the syntax and inheritance behavior of default methods, which are members of interfaces. > * Part J: Java Virtual Machine: Enhances the JVM specification to support code-carrying methods in interfaces. > > Can you send out a version with these sections? From brian.goetz at oracle.com Tue Dec 11 20:35:00 2012 From: brian.goetz at oracle.com (Brian Goetz) Date: Tue, 11 Dec 2012 23:35:00 -0500 Subject: Fwd: Annotation on lambda formal parameter? In-Reply-To: <50C7CF6B.3080304@oracle.com> References: <50C7CF6B.3080304@oracle.com> Message-ID: <50C80974.4050902@oracle.com> Received from the -comments list. Note that this is largely a query of "what does the Lambda EG think the JSR-308 EG should do". -------- Original Message -------- Subject: Annotation on lambda formal parameter? Date: Tue, 11 Dec 2012 16:27:23 -0800 From: Alex Buckley Organization: Oracle Corporation To: lambda-spec-comments at openjdk.java.net CC: Michael Ernst Lambda EG, JSR 308 "Annotations on Java Types" will allow annotations on the manifest type of a lambda expression's formal parameter, and on the type and/or type argument in a method/constructor reference expression. Is there any interest in having 308 allow an annotation on a lambda formal parameter itself? That is, on the declaration of the parameter, rather than its type. A declaration annotation is conceptually valid even if the parameter declaration has no manifest type on which to write a type annotation. This is somewhat in scope for 308 because it already allows annotations on declarations of type parameters (filling a hole from Java SE 5.0) and it has the sub-method-level class file attributes needed to store annotations on declarations of individual lambda parameters. Alex From spullara at gmail.com Wed Dec 12 10:05:14 2012 From: spullara at gmail.com (Sam Pullara) Date: Wed, 12 Dec 2012 10:05:14 -0800 Subject: Inference error Message-ID: <82E491A2-A04E-4AE8-9573-2EAC601B6634@sampullara> The code itself might be a bad idea, just experimenting, but: https://github.com/spullara/java-future-jdk8/commit/2fa4321a450ca56e3a0958214c7cdcd9bca1ad84 The third test fails to compile with this error: java: no suitable method found for curry(map::put,java.lang.String) method spullara.util.Currier.curry(spullara.util.Currier.C2,U) is not applicable (Cannot instantiate inference variables U,V because of an inference loop) method spullara.util.Currier.curry(spullara.util.Currier.C2,U,V) is not applicable (cannot infer type-variable(s) T,U,V (actual and formal argument lists differ in length)) method spullara.util.Currier.curry(spullara.util.Currier.C1,U) is not applicable (cannot infer type-variable(s) T,U (argument mismatch; invalid method reference; method put in interface java.util.Map cannot be applied to given types required: java.lang.String,java.lang.Integer found: java.lang.String reason: actual and formal argument lists differ in length)) Seems like this should work to me. Sam From daniel.smith at oracle.com Wed Dec 12 12:29:03 2012 From: daniel.smith at oracle.com (Dan Smith) Date: Wed, 12 Dec 2012 13:29:03 -0700 Subject: Inference error In-Reply-To: <82E491A2-A04E-4AE8-9573-2EAC601B6634@sampullara> References: <82E491A2-A04E-4AE8-9573-2EAC601B6634@sampullara> Message-ID: <8632ECB1-5838-48D3-B1F7-BD83494851E2@oracle.com> On Dec 12, 2012, at 11:05 AM, Sam Pullara wrote: > The code itself might be a bad idea, just experimenting, but: > > https://github.com/spullara/java-future-jdk8/commit/2fa4321a450ca56e3a0958214c7cdcd9bca1ad84 > > The third test fails to compile with this error: > > java: no suitable method found for curry(map::put,java.lang.String) > method spullara.util.Currier.curry(spullara.util.Currier.C2,U) is not applicable > (Cannot instantiate inference variables U,V because of an inference loop) Here's what inference in this case looks like. interface C2 { T invoke(U u, V v); } interface C1 { T invoke(U u); } C1 curry(C2 get, U u); Map map = new HashMap<>(); C1 c1 = curry(map::put, "test"); For applicability testing, there are two inference constraints: map::put -> C2 "test" -> beta --- The currently-implemented strategy infers beta :> String and then resolves beta = String But we know nothing about gamma at this point, and the first constraint can't be resolved until we have concrete parameter types (instantiations for beta and gamma) to work with. So inference is stuck, and an error occurs. --- What we plan to implement (this is alluded to in the 0.6.0 spec, but not explained in the inference section yet; I'm going to send a high-level summary soon) is this: "test" -> beta reduces to beta :> String while map::put -> C2 is "stuck" and set aside for later. This 'curry' method is considered "provisionally applicable". _After_ overload resolution is done, we can come back and look at a new constraint: C1 <: C1 giving us alpha = Integer gamma = Integer And then we can resolve beta=String and gamma=Integer, updating the 'map::put' constraint to map::put -> C2 Which works out. --- Except that your example makes interesting use of overloading: C0 curry(C1 get, U u); // don't want this C1 curry(C2 get, U u); // want this With a naive approach, both of these would be provisionally applicable, since the method reference constraint would be "stuck" in both cases, and so there would be an ambiguity. We're looking at three different ideas that would avoid that problem: 1) Enhance "potentially applicable" for method references, so that the first candidate is dismissed because there's no unary 'put' method. 2) Since there is only one possible interpretation of 'map::put', handle that method reference bottom-up rather than the usual top-down approach. 3) Eagerly resolve U before overload resolution, since it doesn't show up in the return type. This would "unstuck" the constraint targeting C1 in the first candidate, which would be enough to demonstrate that the first candidate is inapplicable. Some or all of these may make the cut. We've been focusing our efforts on lambda expressions, not method references, as we get the framework in place, and then these things can be tweaked. ?Dan From daniel.smith at oracle.com Wed Dec 12 16:21:52 2012 From: daniel.smith at oracle.com (Dan Smith) Date: Wed, 12 Dec 2012 17:21:52 -0700 Subject: JSR 335 Lambda Specification, 0.6.0 In-Reply-To: References: <2F01A5BB-B2B5-477E-BAC5-6E1A6768D090@oracle.com> Message-ID: For easier access, I've uploaded the spec here: http://cr.openjdk.java.net/~dlsmith/jsr335-0.6.0/ (Added some legal stuff in order to make it publishable for public access.) You can browse at the root (~dlsmith) to get a zip, diff, or single-page version. ?Dan On Dec 11, 2012, at 1:36 PM, Dan Smith wrote: > Ah, thanks. It seems that the mailing list stripped the zip file attachment from my message. It contained separate files for each section. If anybody wants the zipped directory, please contact me directly. > > There was also a bug in my script, so it failed to concatenate the last two sections when creating the one-page version. Here is the one-page version, fixed. > > (Note that part J is incomplete, as described in the "Unfinished Items" list.) > > ?Dan > > > > On Dec 11, 2012, at 1:19 PM, Daniel Heidinga wrote: > >> These sections appear to be missing from the document: >> * Part H: Default Methods: Describes the syntax and inheritance behavior of default methods, which are members of interfaces. >> * Part J: Java Virtual Machine: Enhances the JVM specification to support code-carrying methods in interfaces. >> >> Can you send out a version with these sections? > From daniel.smith at oracle.com Thu Dec 13 13:18:37 2012 From: daniel.smith at oracle.com (Dan Smith) Date: Thu, 13 Dec 2012 14:18:37 -0700 Subject: Annotation on lambda formal parameter? In-Reply-To: <50C80974.4050902@oracle.com> References: <50C7CF6B.3080304@oracle.com> <50C80974.4050902@oracle.com> Message-ID: Didn't we already address this with "Modifiers on Inferred-Type Lambda Parameters", which we rejected (in email "Some rejected ideas" 5 Jul 2012, then in the August meeting)? Or am I misunderstanding the question? ?Dan On Dec 11, 2012, at 9:35 PM, Brian Goetz wrote: > Received from the -comments list. Note that this is largely a query of "what does the Lambda EG think the JSR-308 EG should do". > > > -------- Original Message -------- > Subject: Annotation on lambda formal parameter? > Date: Tue, 11 Dec 2012 16:27:23 -0800 > From: Alex Buckley > Organization: Oracle Corporation > To: lambda-spec-comments at openjdk.java.net > CC: Michael Ernst > > Lambda EG, > > JSR 308 "Annotations on Java Types" will allow annotations on the > manifest type of a lambda expression's formal parameter, and on the type > and/or type argument in a method/constructor reference expression. > > Is there any interest in having 308 allow an annotation on a lambda > formal parameter itself? That is, on the declaration of the parameter, > rather than its type. A declaration annotation is conceptually valid > even if the parameter declaration has no manifest type on which to write > a type annotation. > > This is somewhat in scope for 308 because it already allows annotations > on declarations of type parameters (filling a hole from Java SE 5.0) and > it has the sub-method-level class file attributes needed to store > annotations on declarations of individual lambda parameters. > > Alex > > From brian.goetz at oracle.com Thu Dec 13 13:32:30 2012 From: brian.goetz at oracle.com (Brian Goetz) Date: Thu, 13 Dec 2012 16:32:30 -0500 Subject: Annotation on lambda formal parameter? In-Reply-To: References: <50C7CF6B.3080304@oracle.com> <50C80974.4050902@oracle.com> Message-ID: <50CA496E.6050200@oracle.com> I think we did, but the JSR-308 EG is asking us for that opinion. While one might say "sure, annotations should be allowed everywhere, what's the harm", the reality is that it is not going to be easy for tools to *find* annotations on lambda formal parameters, since reflection only goes down to the method level and not the AST level. Also, this query is not about type annotations on the types of explicit lambda parameters; it's about annotations on the parameters themselves. While containers use parmeter annotations for things like injection, I am having a hard time seeing a use case that is both sensible and desirable. On 12/13/2012 4:18 PM, Dan Smith wrote: > Didn't we already address this with "Modifiers on Inferred-Type Lambda Parameters", which we rejected (in email "Some rejected ideas" 5 Jul 2012, then in the August meeting)? Or am I misunderstanding the question? > > ?Dan > > On Dec 11, 2012, at 9:35 PM, Brian Goetz wrote: > >> Received from the -comments list. Note that this is largely a query of "what does the Lambda EG think the JSR-308 EG should do". >> >> >> -------- Original Message -------- >> Subject: Annotation on lambda formal parameter? >> Date: Tue, 11 Dec 2012 16:27:23 -0800 >> From: Alex Buckley >> Organization: Oracle Corporation >> To: lambda-spec-comments at openjdk.java.net >> CC: Michael Ernst >> >> Lambda EG, >> >> JSR 308 "Annotations on Java Types" will allow annotations on the >> manifest type of a lambda expression's formal parameter, and on the type >> and/or type argument in a method/constructor reference expression. >> >> Is there any interest in having 308 allow an annotation on a lambda >> formal parameter itself? That is, on the declaration of the parameter, >> rather than its type. A declaration annotation is conceptually valid >> even if the parameter declaration has no manifest type on which to write >> a type annotation. >> >> This is somewhat in scope for 308 because it already allows annotations >> on declarations of type parameters (filling a hole from Java SE 5.0) and >> it has the sub-method-level class file attributes needed to store >> annotations on declarations of individual lambda parameters. >> >> Alex >> >> > From daniel.smith at oracle.com Thu Dec 13 13:49:39 2012 From: daniel.smith at oracle.com (Dan Smith) Date: Thu, 13 Dec 2012 14:49:39 -0700 Subject: Annotation on lambda formal parameter? In-Reply-To: <50CA496E.6050200@oracle.com> References: <50C7CF6B.3080304@oracle.com> <50C80974.4050902@oracle.com> <50CA496E.6050200@oracle.com> Message-ID: My understanding is: 1) Java 7 already allows annotations on method parameters (JLS 8.4.1) 2) Explicitly-typed lambdas allow the same annotations as method parameters (Lambda Spec Part B, 15.27 & 15.27.1) 3) Implicitly-typed lambdas do not allow any modifiers, including annotations (Lambda Spec again; also, the design discussions I referenced) I thought the question was about (3), but maybe it's also about whether we _don't_ want (2)? (FWIW, I note that exception parameters can have annotations in Java 7. I can't seem to find the line in the spec that says what kind of ElementType they support, but some reverse engineering tells me they're PARAMETERs, not LOCAL_VARIABLEs.) ?Dan On Dec 13, 2012, at 2:32 PM, Brian Goetz wrote: > I think we did, but the JSR-308 EG is asking us for that opinion. > > While one might say "sure, annotations should be allowed everywhere, what's the harm", the reality is that it is not going to be easy for tools to *find* annotations on lambda formal parameters, since reflection only goes down to the method level and not the AST level. > > Also, this query is not about type annotations on the types of explicit lambda parameters; it's about annotations on the parameters themselves. While containers use parmeter annotations for things like injection, I am having a hard time seeing a use case that is both sensible and desirable. > > On 12/13/2012 4:18 PM, Dan Smith wrote: >> Didn't we already address this with "Modifiers on Inferred-Type Lambda Parameters", which we rejected (in email "Some rejected ideas" 5 Jul 2012, then in the August meeting)? Or am I misunderstanding the question? >> >> ?Dan >> >> On Dec 11, 2012, at 9:35 PM, Brian Goetz wrote: >> >>> Received from the -comments list. Note that this is largely a query of "what does the Lambda EG think the JSR-308 EG should do". >>> >>> >>> -------- Original Message -------- >>> Subject: Annotation on lambda formal parameter? >>> Date: Tue, 11 Dec 2012 16:27:23 -0800 >>> From: Alex Buckley >>> Organization: Oracle Corporation >>> To: lambda-spec-comments at openjdk.java.net >>> CC: Michael Ernst >>> >>> Lambda EG, >>> >>> JSR 308 "Annotations on Java Types" will allow annotations on the >>> manifest type of a lambda expression's formal parameter, and on the type >>> and/or type argument in a method/constructor reference expression. >>> >>> Is there any interest in having 308 allow an annotation on a lambda >>> formal parameter itself? That is, on the declaration of the parameter, >>> rather than its type. A declaration annotation is conceptually valid >>> even if the parameter declaration has no manifest type on which to write >>> a type annotation. >>> >>> This is somewhat in scope for 308 because it already allows annotations >>> on declarations of type parameters (filling a hole from Java SE 5.0) and >>> it has the sub-method-level class file attributes needed to store >>> annotations on declarations of individual lambda parameters. >>> >>> Alex >>> >>> >> From forax at univ-mlv.fr Thu Dec 13 14:20:47 2012 From: forax at univ-mlv.fr (Remi Forax) Date: Thu, 13 Dec 2012 23:20:47 +0100 Subject: Annotation on lambda formal parameter? In-Reply-To: References: <50C7CF6B.3080304@oracle.com> <50C80974.4050902@oracle.com> <50CA496E.6050200@oracle.com> Message-ID: <50CA54BF.1070703@univ-mlv.fr> On 12/13/2012 10:49 PM, Dan Smith wrote: > My understanding is: > > 1) Java 7 already allows annotations on method parameters (JLS 8.4.1) > > 2) Explicitly-typed lambdas allow the same annotations as method parameters (Lambda Spec Part B, 15.27 & 15.27.1) > > 3) Implicitly-typed lambdas do not allow any modifiers, including annotations (Lambda Spec again; also, the design discussions I referenced) > > I thought the question was about (3), but maybe it's also about whether we _don't_ want (2)? > > (FWIW, I note that exception parameters can have annotations in Java 7. I can't seem to find the line in the spec that says what kind of ElementType they support, but some reverse engineering tells me they're PARAMETERs, not LOCAL_VARIABLEs.) > > ?Dan Re-reading the mail of Alex, the question is about (3) that as you said we rule this out. R?mi > > On Dec 13, 2012, at 2:32 PM, Brian Goetz wrote: > >> I think we did, but the JSR-308 EG is asking us for that opinion. >> >> While one might say "sure, annotations should be allowed everywhere, what's the harm", the reality is that it is not going to be easy for tools to *find* annotations on lambda formal parameters, since reflection only goes down to the method level and not the AST level. >> >> Also, this query is not about type annotations on the types of explicit lambda parameters; it's about annotations on the parameters themselves. While containers use parmeter annotations for things like injection, I am having a hard time seeing a use case that is both sensible and desirable. >> >> On 12/13/2012 4:18 PM, Dan Smith wrote: >>> Didn't we already address this with "Modifiers on Inferred-Type Lambda Parameters", which we rejected (in email "Some rejected ideas" 5 Jul 2012, then in the August meeting)? Or am I misunderstanding the question? >>> >>> ?Dan >>> >>> On Dec 11, 2012, at 9:35 PM, Brian Goetz wrote: >>> >>>> Received from the -comments list. Note that this is largely a query of "what does the Lambda EG think the JSR-308 EG should do". >>>> >>>> >>>> -------- Original Message -------- >>>> Subject: Annotation on lambda formal parameter? >>>> Date: Tue, 11 Dec 2012 16:27:23 -0800 >>>> From: Alex Buckley >>>> Organization: Oracle Corporation >>>> To: lambda-spec-comments at openjdk.java.net >>>> CC: Michael Ernst >>>> >>>> Lambda EG, >>>> >>>> JSR 308 "Annotations on Java Types" will allow annotations on the >>>> manifest type of a lambda expression's formal parameter, and on the type >>>> and/or type argument in a method/constructor reference expression. >>>> >>>> Is there any interest in having 308 allow an annotation on a lambda >>>> formal parameter itself? That is, on the declaration of the parameter, >>>> rather than its type. A declaration annotation is conceptually valid >>>> even if the parameter declaration has no manifest type on which to write >>>> a type annotation. >>>> >>>> This is somewhat in scope for 308 because it already allows annotations >>>> on declarations of type parameters (filling a hole from Java SE 5.0) and >>>> it has the sub-method-level class file attributes needed to store >>>> annotations on declarations of individual lambda parameters. >>>> >>>> Alex >>>> >>>> From brian.goetz at oracle.com Tue Dec 18 14:33:59 2012 From: brian.goetz at oracle.com (Brian Goetz) Date: Tue, 18 Dec 2012 17:33:59 -0500 Subject: Fwd: lambda expression parameters In-Reply-To: <6930E92B-352A-4616-B2CE-7BF03C3CF031@agiledeveloper.com> References: <6930E92B-352A-4616-B2CE-7BF03C3CF031@agiledeveloper.com> Message-ID: <50D0EF57.4010603@oracle.com> I like this suggestion that Venkat made on lambda-dev. However, it would have to be a little more restrictive than that; basically, that lambda parameters are *always* final. The only thing where I can imagine this being an issue is in refactoring from inner classes, and even there, if you are modifying method parameters, it is likely that you need to do additional work to adapt this inner class to a lambda anyway. So the suggestion is: lambda formals are always final. What do people think? -------- Original Message -------- Subject: lambda expression parameters Date: Tue, 18 Dec 2012 14:40:25 -0700 From: Venkat Subramaniam To: lambda-dev at openjdk.java.net Hi Is there plans to make the inferred parameter of a lambda expression final by default. Right now, we can't specify the parameter is final unless we provide it with the type. In the spirit of leaning towards immutability, which is a better practice, would it be possible to (a) make all inferred parameters of lambda expressions final, and (b) to make it non-final, force the developers to explicitly request (like the mutable in F#). It's more of a wish-list, if that's something that can be considered. Thanks, Venkat From spullara at gmail.com Tue Dec 18 14:40:05 2012 From: spullara at gmail.com (Sam Pullara) Date: Tue, 18 Dec 2012 14:40:05 -0800 Subject: lambda expression parameters In-Reply-To: <50D0EF57.4010603@oracle.com> References: <6930E92B-352A-4616-B2CE-7BF03C3CF031@agiledeveloper.com> <50D0EF57.4010603@oracle.com> Message-ID: <0D74EADD-9B7C-486E-921C-733CFFDEF97F@gmail.com> I agree with Remi that immutability and final are somewhat orthogonal. However, I am all for making the lambda parameters always final. Sam On Dec 18, 2012, at 2:33 PM, Brian Goetz wrote: > I like this suggestion that Venkat made on lambda-dev. However, it would have to be a little more restrictive than that; basically, that lambda parameters are *always* final. The only thing where I can imagine this being an issue is in refactoring from inner classes, and even there, if you are modifying method parameters, it is likely that you need to do additional work to adapt this inner class to a lambda anyway. > > So the suggestion is: lambda formals are always final. > > What do people think? > > > -------- Original Message -------- > Subject: lambda expression parameters > Date: Tue, 18 Dec 2012 14:40:25 -0700 > From: Venkat Subramaniam > To: lambda-dev at openjdk.java.net > > Hi > > Is there plans to make the inferred parameter of a lambda expression final by default. > Right now, we can't specify the parameter is final unless we provide it with the type. > > In the spirit of leaning towards immutability, which is a better practice, would it be possible to > > (a) make all inferred parameters of lambda expressions final, and > (b) to make it non-final, force the developers to explicitly request (like the mutable in F#). > > It's more of a wish-list, if that's something that can be considered. > > Thanks, > > Venkat > > > From forax at univ-mlv.fr Tue Dec 18 14:41:12 2012 From: forax at univ-mlv.fr (Remi Forax) Date: Tue, 18 Dec 2012 23:41:12 +0100 Subject: Fwd: lambda expression parameters In-Reply-To: <50D0EF57.4010603@oracle.com> References: <6930E92B-352A-4616-B2CE-7BF03C3CF031@agiledeveloper.com> <50D0EF57.4010603@oracle.com> Message-ID: <50D0F108.8040703@univ-mlv.fr> On 12/18/2012 11:33 PM, Brian Goetz wrote: > I like this suggestion that Venkat made on lambda-dev. However, it > would have to be a little more restrictive than that; basically, that > lambda parameters are *always* final. The only thing where I can > imagine this being an issue is in refactoring from inner classes, and > even there, if you are modifying method parameters, it is likely that > you need to do additional work to adapt this inner class to a lambda > anyway. > > So the suggestion is: lambda formals are always final. > > What do people think? Brian, I don't see why parameter should be final. You explain a reason why they should not, but not why they should. R?mi > > > -------- Original Message -------- > Subject: lambda expression parameters > Date: Tue, 18 Dec 2012 14:40:25 -0700 > From: Venkat Subramaniam > To: lambda-dev at openjdk.java.net > > Hi > > Is there plans to make the inferred parameter of a lambda expression > final by default. > Right now, we can't specify the parameter is final unless we provide > it with the type. > > In the spirit of leaning towards immutability, which is a better > practice, would it be possible to > > (a) make all inferred parameters of lambda expressions final, and > (b) to make it non-final, force the developers to explicitly request > (like the mutable in F#). > > It's more of a wish-list, if that's something that can be considered. > > Thanks, > > Venkat > > > From brian.goetz at oracle.com Tue Dec 18 14:51:18 2012 From: brian.goetz at oracle.com (Brian Goetz) Date: Tue, 18 Dec 2012 17:51:18 -0500 Subject: Fwd: lambda expression parameters In-Reply-To: <50D0F108.8040703@univ-mlv.fr> References: <6930E92B-352A-4616-B2CE-7BF03C3CF031@agiledeveloper.com> <50D0EF57.4010603@oracle.com> <50D0F108.8040703@univ-mlv.fr> Message-ID: <50D0F366.3090301@oracle.com> Imagine we were designing a language from scratch, knowing what we have learned in the last 17 years about mutability being the root of all evil. We would do many things to gently encourage people away from avoidable mutability. Of all places to additionally discourage mutability as a way to do things, and where it would likely have the least harm, lambda parameters seem an obvious choice. The only reason we'd even consider letting them be mutable is that Java already has methods that have parameters that are mutable by default, and lambdas are like methods. On 12/18/2012 5:41 PM, Remi Forax wrote: > On 12/18/2012 11:33 PM, Brian Goetz wrote: >> I like this suggestion that Venkat made on lambda-dev. However, it >> would have to be a little more restrictive than that; basically, that >> lambda parameters are *always* final. The only thing where I can >> imagine this being an issue is in refactoring from inner classes, and >> even there, if you are modifying method parameters, it is likely that >> you need to do additional work to adapt this inner class to a lambda >> anyway. >> >> So the suggestion is: lambda formals are always final. >> >> What do people think? > > Brian, > I don't see why parameter should be final. You explain a reason why they > should not, but not why they should. > > R?mi > >> >> >> -------- Original Message -------- >> Subject: lambda expression parameters >> Date: Tue, 18 Dec 2012 14:40:25 -0700 >> From: Venkat Subramaniam >> To: lambda-dev at openjdk.java.net >> >> Hi >> >> Is there plans to make the inferred parameter of a lambda expression >> final by default. >> Right now, we can't specify the parameter is final unless we provide >> it with the type. >> >> In the spirit of leaning towards immutability, which is a better >> practice, would it be possible to >> >> (a) make all inferred parameters of lambda expressions final, and >> (b) to make it non-final, force the developers to explicitly request >> (like the mutable in F#). >> >> It's more of a wish-list, if that's something that can be considered. >> >> Thanks, >> >> Venkat >> >> >> > From forax at univ-mlv.fr Tue Dec 18 14:57:23 2012 From: forax at univ-mlv.fr (Remi Forax) Date: Tue, 18 Dec 2012 23:57:23 +0100 Subject: lambda expression parameters In-Reply-To: <0D74EADD-9B7C-486E-921C-733CFFDEF97F@gmail.com> References: <6930E92B-352A-4616-B2CE-7BF03C3CF031@agiledeveloper.com> <50D0EF57.4010603@oracle.com> <0D74EADD-9B7C-486E-921C-733CFFDEF97F@gmail.com> Message-ID: <50D0F4D3.9000701@univ-mlv.fr> On 12/18/2012 11:40 PM, Sam Pullara wrote: > I agree with Remi that immutability and final are somewhat orthogonal. However, I am all for making the lambda parameters always final. > > Sam Same answer as Brian, why ? Brian and you seems to always forget that each time that you consider that the lambda semantics is different from the classical method semantics, you make the whole Java mental model more complex. This kind of changes are far from free. Currently, the way to solve the scope of parameters for lambda and the scope of parameter for a method are different, for no reason in my opinion, at least not good enough reason, and you are asking to go a step further. So I fully agree with your proposal, but instead of just making lambda parameters always final, because it's really that important, why not making all method parameters always final. R?mi BTW, The primary purpose of a programming language is to get things done, without bothering the user and both of you are adding fences and traps. > On Dec 18, 2012, at 2:33 PM, Brian Goetz wrote: > >> I like this suggestion that Venkat made on lambda-dev. However, it would have to be a little more restrictive than that; basically, that lambda parameters are *always* final. The only thing where I can imagine this being an issue is in refactoring from inner classes, and even there, if you are modifying method parameters, it is likely that you need to do additional work to adapt this inner class to a lambda anyway. >> >> So the suggestion is: lambda formals are always final. >> >> What do people think? >> >> >> -------- Original Message -------- >> Subject: lambda expression parameters >> Date: Tue, 18 Dec 2012 14:40:25 -0700 >> From: Venkat Subramaniam >> To: lambda-dev at openjdk.java.net >> >> Hi >> >> Is there plans to make the inferred parameter of a lambda expression final by default. >> Right now, we can't specify the parameter is final unless we provide it with the type. >> >> In the spirit of leaning towards immutability, which is a better practice, would it be possible to >> >> (a) make all inferred parameters of lambda expressions final, and >> (b) to make it non-final, force the developers to explicitly request (like the mutable in F#). >> >> It's more of a wish-list, if that's something that can be considered. >> >> Thanks, >> >> Venkat >> >> >> From forax at univ-mlv.fr Tue Dec 18 15:09:49 2012 From: forax at univ-mlv.fr (Remi Forax) Date: Wed, 19 Dec 2012 00:09:49 +0100 Subject: Fwd: lambda expression parameters In-Reply-To: <50D0F366.3090301@oracle.com> References: <6930E92B-352A-4616-B2CE-7BF03C3CF031@agiledeveloper.com> <50D0EF57.4010603@oracle.com> <50D0F108.8040703@univ-mlv.fr> <50D0F366.3090301@oracle.com> Message-ID: <50D0F7BD.8070305@univ-mlv.fr> On 12/18/2012 11:51 PM, Brian Goetz wrote: > Imagine we were designing a language from scratch, knowing what we > have learned in the last 17 years about mutability being the root of > all evil. We would do many things to gently encourage people away > from avoidable mutability. sorry, side effects are the root of all evil. > > Of all places to additionally discourage mutability as a way to do > things, and where it would likely have the least harm, I don't like cargo cult. > lambda parameters seem an obvious choice. and as I said, it doesn't worth the pain for people to understand why it works with methods and not with lambda. > The only reason we'd even consider letting them be mutable is that > Java already has methods that have parameters that are mutable by > default, and lambdas are like methods. lambdas are anonymous methods, simple to explain, simple to understand, but wait, the scoping rules ... R?mi > > On 12/18/2012 5:41 PM, Remi Forax wrote: >> On 12/18/2012 11:33 PM, Brian Goetz wrote: >>> I like this suggestion that Venkat made on lambda-dev. However, it >>> would have to be a little more restrictive than that; basically, that >>> lambda parameters are *always* final. The only thing where I can >>> imagine this being an issue is in refactoring from inner classes, and >>> even there, if you are modifying method parameters, it is likely that >>> you need to do additional work to adapt this inner class to a lambda >>> anyway. >>> >>> So the suggestion is: lambda formals are always final. >>> >>> What do people think? >> >> Brian, >> I don't see why parameter should be final. You explain a reason why they >> should not, but not why they should. >> >> R?mi >> >>> >>> >>> -------- Original Message -------- >>> Subject: lambda expression parameters >>> Date: Tue, 18 Dec 2012 14:40:25 -0700 >>> From: Venkat Subramaniam >>> To: lambda-dev at openjdk.java.net >>> >>> Hi >>> >>> Is there plans to make the inferred parameter of a lambda expression >>> final by default. >>> Right now, we can't specify the parameter is final unless we provide >>> it with the type. >>> >>> In the spirit of leaning towards immutability, which is a better >>> practice, would it be possible to >>> >>> (a) make all inferred parameters of lambda expressions final, and >>> (b) to make it non-final, force the developers to explicitly request >>> (like the mutable in F#). >>> >>> It's more of a wish-list, if that's something that can be considered. >>> >>> Thanks, >>> >>> Venkat >>> >>> >>> >> From sam at sampullara.com Sat Dec 29 19:05:26 2012 From: sam at sampullara.com (Sam Pullara) Date: Sat, 29 Dec 2012 19:05:26 -0800 Subject: New and exciting error in the latest compiler Message-ID: <76F2296B-D252-4312-A62F-5EC90041B26E@sampullara.com> Here is the code: https://github.com/spullara/java-future-jdk8/blob/master/src/main/java/spullara/util/Currier.java https://github.com/spullara/java-future-jdk8/blob/master/src/test/java/spullara/util/CurrierTest.java Gives an ambiguity when I think it should be straightforward to pick the right one based on arity of the method reference: error: reference to curry is ambiguous both method curry(C2,U#1) in Currier and method curry(C1,U#2) in Currier match where T#1,U#1,V,T#2,U#2 are type-variables: T#1 extends Object declared in method curry(C2,U#1) U#1 extends Object declared in method curry(C2,U#1) V extends Object declared in method curry(C2,U#1) T#2 extends Object declared in method curry(C1,U#2) U#2 extends Object declared in method curry(C1,U#2) error: reference to curry is ambiguous both method curry(C2,U#1) in Currier and method curry(C1,U#2) in Currier match where T#1,U#1,V,T#2,U#2 are type-variables: T#1 extends Object declared in method curry(C2,U#1) U#1 extends Object declared in method curry(C2,U#1) V extends Object declared in method curry(C2,U#1) T#2 extends Object declared in method curry(C1,U#2) U#2 extends Object declared in method curry(C1,U#2) And those error messages have to be fixed, they are brutal :) Sam From brian.goetz at oracle.com Sat Dec 29 19:19:45 2012 From: brian.goetz at oracle.com (Brian Goetz) Date: Sat, 29 Dec 2012 22:19:45 -0500 Subject: New and exciting error in the latest compiler In-Reply-To: <76F2296B-D252-4312-A62F-5EC90041B26E@sampullara.com> References: <76F2296B-D252-4312-A62F-5EC90041B26E@sampullara.com> Message-ID: <50DFB2D1.1070505@oracle.com> > And those error messages have to be fixed, they are brutal :) The error messages are deliberately tuned to give maximal information, because right now we're still developing the language and its important to have all the information to determine whether the code is in error or the compiler. For that purpose, they're great! We'll turn the knobs down once we're satisfied that things are working as we expect. (For example, there's no need to list out candidates who are not applicable because of the wrong arity.) From forax at univ-mlv.fr Sun Dec 30 16:42:09 2012 From: forax at univ-mlv.fr (Remi Forax) Date: Mon, 31 Dec 2012 01:42:09 +0100 Subject: Conversion of method reference to MethodHandle Message-ID: <50E0DF61.3060608@univ-mlv.fr> [to lambda spec list and JSR292 EG list] I want to close one problem that we (the JSR 292 EG) have postponed to Java 8 when we were developing the JSR 292, a Java syntax to reference a method or a constructor as a method handle. This item was postponed for two reasons, the first one is that the JSR 292 was a JSR allowing to change the VM spec and not the Java spec and the second one was that at that time the syntax for method reference was in limbo and we didn't want to choose a syntax that may be incompatible with the upcoming lambda spec. Now that the syntax part of the lambda spec is frozen, we can solve that issue. Just before starting, I want to briefly explain why this syntax is needed. A runtime of a dynamic languages uses several well known function (usually specified as static method) implementing either part of the runtime logic like a function that check is the runtime class of an object is equals to a specified class or some global functions of the language it self (here is the file for JavaScript in Nashorn [1]). The main issue is that to be used in JSR 292 ecosystem, this functions must be handled by method handles and these method handles are defined are static final fields, because Java doesn't initialize these fields lazily, this greatly impacts the startup time of any dynamic runtime that runs on the JVM. So the goals of the syntax that convert a method reference to a method handle is: - to move some runtime checks at compile time because creating a method handle involve using a string and a signature. - to lazily create these method handle when needed (like the lambda). The proposed syntax is to use the same syntax as the method reference, so no new syntax. MethodHandle mh = String::length; the target type is a j.l.i.MethodHandle instead of being a functional interface For the semantics: Unlike the method reference with a functional interface, only unbunded method reference are allowed, so by example, a method reference starting with an expression or super are not valid. Moreover, because there is no type information that the compiler can extract from a MethodHandle, the semantics in case of several overloading methods, i.e. several applicable methods is simple, it's an ambiguity and is rejected by the compiler. More formally, a method reference can be converted to a j.l.i.MethodHandle, - only if it's form is Type::name, with Type an existing type and name a valid Java identifier or 'new'. - then from the type, the compiler gather all accessible methods declared in the type type or it's super type (class or interfaces), if a method is override-equivalent to another one, the one from the supertype is discarded. if the name is 'new', the compiler gather all constructors of the type type. if the type is an array, all public methods of j.l;Object are available plus clone() - if there are more than one method in the set of gathered method, the reference is ambiguous. - if there is no method, the reference is not available - if there is one method, a method handle will be created from this method. Here is some examples: class Test { public static void foo() { } } ... MethodHandle mh1 = Test::foo; // foo()V mh1.invokeExact(); class Test public void bar() { } } ... MethodHandle mh2 = Test::bar; // bar(Test)V mh2.invokeExact(new Test()); class Test { interface I { public void m(); } static class A implements I { public void m() { } } } ... MethodHandle mh3 = I::m; // m(I)V mh3.invokeExact((I)new A()); class Test { static class B { public B(int value) { } } } } ... MethodHandle mh4 = B::new; // B(int) B b = (B)mh4.invokeExact(3); class Test { class C { // inner class public C() { } } } ... MethodHandle mh5 = C::new; // C(Test) C c = (C)mh5.invokeExact(new Test()); class Test { static class D { Object foo() { } } static class E { String foo() { } // covariant return type } } ... MethodHandle mh8 = E::foo; // foo(E)String String s3 = (String)mh8.invokeExact(new E()); class Test { static class F { private static void m() { } // non visible method for the VM, ok in Java, so the compiler has to generate a bridge } } ... MethodHandle mh9 = F::m; mh9.invoke(); MethodHandle mh10 = String[]::clone; // String[].clone returns Object, so it needs a bridge by the compiler String[] values = (String[])mh10.invoke(args); As you can see the syntax already exist, the semantics of the attribution (find the correct method) is simpler than the method call semantics (there is no inference) and after the attribution, the rules for the generation, bridge, etc. are the same as the one for the method reference with a functional interface as target. The compiler generates like the method reference an invokedynamic with a different bootstrap method that takes the constant method handle as parameter and create a constant callsite with this constant method handle (so at runtime a method reference seen as a method handle is always constant). And because Brian will say that this is interesting but Oracle has no resource that can work on this, here is the patch for the compiler* part based on lambda/langtools : http://www-igm.univ-mlv.fr/~forax/tmp/method-ref-method-handle-langtools.diff and the patch for the jdk part based on lambda/jdk: http://www-igm.univ-mlv.fr/~forax/tmp/method-ref-method-handle-jdk.diff cheers, R?mi * note, the mh9 doesn't work at runtime because it also doesn't work for method reference with a functional interface. [1] http://hg.openjdk.java.net/nashorn/jdk8/nashorn/file/b4b05457b8b2/src/jdk/nashorn/internal/runtime/GlobalFunctions.java From brian.goetz at oracle.com Sun Dec 30 17:00:58 2012 From: brian.goetz at oracle.com (Brian Goetz) Date: Sun, 30 Dec 2012 20:00:58 -0500 Subject: Conversion of method reference to MethodHandle In-Reply-To: <50E0DF61.3060608@univ-mlv.fr> References: <50E0DF61.3060608@univ-mlv.fr> Message-ID: <50E0E3CA.4020005@oracle.com> To start with, the syntax that has been chosen for method references is inadequate for method handles. This is because of the point you raise -- there is no way to specify the type of the parameters, and therefore no reasonable way (creating a customized functional interface for each usage is not reasonable) to select between multiple overloads of the same name. We have discussed explicit type parameters, but they turned out (or so we still think, jury is still out) that not adding this bit of syntax is adequate to what we are doing here. Failing completely in the presence of overloading makes this a crippled feature with even more sharp edges than it naturally has. But that's not even the real objection, nor is this: > And because Brian will say that this is interesting but Oracle has no resource that can work on this, > here is the patch for the compiler* part based on lambda/langtools : http://www-igm.univ-mlv.fr/~forax/tmp/method-ref-method-handle-langtools.diff > and the patch for the jdk part based on lambda/jdk: http://www-igm.univ-mlv.fr/~forax/tmp/method-ref-method-handle-jdk.diff My true objection is deeper than any of these. After thinking about it for a long time, I think this feature just doesn't belong. The set of people who need method handle literals is a tiny sliver of the Java developer base; complicating "everyone's Java" for the sake of a few advanced users (100% of whom are qualified to generate bytecode) is a bad stewardship choice. I'm one of those few who has to resort to writing ASM programs to use MHs when I want them. So I know exactly how painful it is. And still, I'd rather impose that pain on myself and my peers than increase the complexity of Java for everyone for the benefit of a small few. The way I solved this for myself is I wrote a tool which opportunistically mangles calls to Lookup.findXxx (with constant arguments) into LDCs using bytecode mangling. From a programming-model perspective, this is only slightly more painful than what you propose -- and works more generally. (It also works for indy.) I much prefer an approach like that. On 12/30/2012 7:42 PM, Remi Forax wrote: > [to lambda spec list and JSR292 EG list] > > I want to close one problem that we (the JSR 292 EG) have postponed to > Java 8 when we were developing the JSR 292, > a Java syntax to reference a method or a constructor as a method handle. > This item was postponed for two reasons, the first one is that the JSR > 292 was a JSR allowing to change the VM spec and not the Java spec and > the second one was that at that time the syntax for method reference was > in limbo and we didn't want to choose a syntax that may be incompatible > with the upcoming lambda spec. > > Now that the syntax part of the lambda spec is frozen, we can solve that > issue. > > Just before starting, I want to briefly explain why this syntax is needed. > A runtime of a dynamic languages uses several well known function > (usually specified as static method) implementing either part of the > runtime logic like a function that check is the runtime class of an > object is equals to a specified class or some global functions of the > language it self (here is the file for JavaScript in Nashorn [1]). > The main issue is that to be used in JSR 292 ecosystem, this functions > must be handled by method handles and these method handles are defined > are static final fields, > because Java doesn't initialize these fields lazily, this greatly > impacts the startup time of any dynamic runtime that runs on the JVM. > > So the goals of the syntax that convert a method reference to a method > handle is: > - to move some runtime checks at compile time because creating a > method handle involve using a string and a signature. > - to lazily create these method handle when needed (like the lambda). > > The proposed syntax is to use the same syntax as the method reference, > so no new syntax. > MethodHandle mh = String::length; > the target type is a j.l.i.MethodHandle instead of being a functional > interface > > For the semantics: > Unlike the method reference with a functional interface, only > unbunded method reference are allowed, so by example, > a method reference starting with an expression or super are not valid. > Moreover, because there is no type information that the compiler can > extract from a MethodHandle, the semantics in case of several > overloading methods, i.e. several applicable methods is simple, it's an > ambiguity and is rejected by the compiler. > More formally, a method reference can be converted to a > j.l.i.MethodHandle, > - only if it's form is > Type::name, with Type an existing type and name a valid Java > identifier or 'new'. > - then from the type, the compiler gather all accessible methods > declared in the type type or it's super type (class or interfaces), > if a method is override-equivalent to another one, the one from > the supertype is discarded. > if the name is 'new', the compiler gather all constructors of the > type type. > if the type is an array, all public methods of j.l;Object are > available plus clone() > - if there are more than one method in the set of gathered method, > the reference is ambiguous. > - if there is no method, the reference is not available > - if there is one method, a method handle will be created from this > method. > > Here is some examples: > class Test { > public static void foo() { } > } > ... > MethodHandle mh1 = Test::foo; // foo()V > mh1.invokeExact(); > > class Test > public void bar() { } > } > ... > MethodHandle mh2 = Test::bar; // bar(Test)V > mh2.invokeExact(new Test()); > > class Test { > interface I { > public void m(); > } > static class A implements I { > public void m() { } > } > } > ... > MethodHandle mh3 = I::m; // m(I)V > mh3.invokeExact((I)new A()); > > class Test { > static class B { > public B(int value) { } > } > } > } > ... > MethodHandle mh4 = B::new; // B(int) > B b = (B)mh4.invokeExact(3); > > class Test { > class C { // inner class > public C() { } > } > } > ... > MethodHandle mh5 = C::new; // C(Test) > C c = (C)mh5.invokeExact(new Test()); > > class Test { > static class D { > Object foo() { } > } > static class E { > String foo() { } // covariant return type > } > } > ... > MethodHandle mh8 = E::foo; // foo(E)String > String s3 = (String)mh8.invokeExact(new E()); > > class Test { > static class F { > private static void m() { } // non visible method for the VM, ok > in Java, so the compiler has to generate a bridge > } > } > ... > MethodHandle mh9 = F::m; > mh9.invoke(); > > MethodHandle mh10 = String[]::clone; // String[].clone returns > Object, so it needs a bridge by the compiler > String[] values = (String[])mh10.invoke(args); > > As you can see the syntax already exist, the semantics of the > attribution (find the correct method) is simpler than the method call > semantics (there is no inference) and after the attribution, the rules > for the generation, bridge, etc. are the same as the one for the method > reference with a functional interface as target. > > The compiler generates like the method reference an invokedynamic with a > different bootstrap method that takes the constant method handle as > parameter and create a constant callsite with this constant method > handle (so at runtime a method reference seen as a method handle is > always constant). > > And because Brian will say that this is interesting but Oracle has no > resource that can work on this, > here is the patch for the compiler* part based on lambda/langtools : > http://www-igm.univ-mlv.fr/~forax/tmp/method-ref-method-handle-langtools.diff > > and the patch for the jdk part based on lambda/jdk: > http://www-igm.univ-mlv.fr/~forax/tmp/method-ref-method-handle-jdk.diff > > cheers, > R?mi > * note, the mh9 doesn't work at runtime because it also doesn't work for > method reference with a functional interface. > > [1] > http://hg.openjdk.java.net/nashorn/jdk8/nashorn/file/b4b05457b8b2/src/jdk/nashorn/internal/runtime/GlobalFunctions.java > > From blackdrag at gmx.org Mon Dec 31 04:16:02 2012 From: blackdrag at gmx.org (Jochen Theodorou) Date: Mon, 31 Dec 2012 13:16:02 +0100 Subject: [jsr-292-eg] Conversion of method reference to MethodHandle In-Reply-To: <50E0DF61.3060608@univ-mlv.fr> References: <50E0DF61.3060608@univ-mlv.fr> Message-ID: <50E18202.7080908@gmx.org> Am 31.12.2012 01:42, schrieb Remi Forax: [...] > For the semantics: > Unlike the method reference with a functional interface, only > unbunded method reference are allowed, so by example, > a method reference starting with an expression or super are not valid. > Moreover, because there is no type information that the compiler can > extract from a MethodHandle, the semantics in case of several > overloading methods, i.e. several applicable methods is simple, it's an > ambiguity and is rejected by the compiler. > More formally, a method reference can be converted to a > j.l.i.MethodHandle, > - only if it's form is > Type::name, with Type an existing type and name a valid Java > identifier or 'new'. > - then from the type, the compiler gather all accessible methods > declared in the type type or it's super type (class or interfaces), > if a method is override-equivalent to another one, the one from > the supertype is discarded. > if the name is 'new', the compiler gather all constructors of the > type type. > if the type is an array, all public methods of j.l;Object are > available plus clone() > - if there are more than one method in the set of gathered method, > the reference is ambiguous. > - if there is no method, the reference is not available > - if there is one method, a method handle will be created from this > method. I think not allowing overloaded methods will restrict the usage quite a lot. You say this should be able to reference constructors, but having more than one constructor is a very common thing. Also you don't specify visibility. what if for example there are two X::foo, but one is private the other not? does it still count as overloaded? Will we be allowed to call the other? Since you talk of simplified method selection logic I would assume it is not seen as overloaded, if the private method is not visible. That would at least in same cases save the usage for this. But I still think you should be able to give the parameter types. Actually being able to give the return type too would be even better for languages that are not Java. X:foo could be a short form if there is no overloading... or is x:foo(String) difficult to implement? bye Jochen -- Jochen "blackdrag" Theodorou - Groovy Project Tech Lead blog: http://blackdragsview.blogspot.com/ german groovy discussion newsgroup: de.comp.lang.misc For Groovy programming sources visit http://groovy-lang.org From forax at univ-mlv.fr Mon Dec 31 07:18:04 2012 From: forax at univ-mlv.fr (Remi Forax) Date: Mon, 31 Dec 2012 16:18:04 +0100 Subject: Conversion of method reference to MethodHandle In-Reply-To: <50E0E3CA.4020005@oracle.com> References: <50E0DF61.3060608@univ-mlv.fr> <50E0E3CA.4020005@oracle.com> Message-ID: <50E1ACAC.8030806@univ-mlv.fr> On 12/31/2012 02:00 AM, Brian Goetz wrote: > To start with, the syntax that has been chosen for method references > is inadequate for method handles. This is because of the point you > raise -- there is no way to specify the type of the parameters, and > therefore no reasonable way (creating a customized functional > interface for each usage is not reasonable) to select between multiple > overloads of the same name. We have discussed explicit type > parameters, but they turned out (or so we still think, jury is still > out) that not adding this bit of syntax is adequate to what we are > doing here. > > Failing completely in the presence of overloading makes this a > crippled feature with even more sharp edges than it naturally has. I see that as a feature. People tend to use overloading where they should not. List::remove is a good example, JPanel#add is another one. The only case where it's correctly used is PrintStream#println, StringBuilder#append or Math::sqrt because all overloads have the same semantics. More fundamentally method reference does an early binding, so even a method reference with a functional interface is broken because the overload resolution should appear when calling it and not when creating it. The simple workaround is to create a bridge method, if you can't change the class to not use overloading. private static void myprintln(PrintStream stream, int value) { return stream.println(value); } > But that's not even the real objection, cool ! > nor is this: > >> And because Brian will say that this is interesting but Oracle has no >> resource that can work on this, >> here is the patch for the compiler* part based on lambda/langtools : >> http://www-igm.univ-mlv.fr/~forax/tmp/method-ref-method-handle-langtools.diff >> and the patch for the jdk part based on lambda/jdk: >> http://www-igm.univ-mlv.fr/~forax/tmp/method-ref-method-handle-jdk.diff > > My true objection is deeper than any of these. After thinking about > it for a long time, I think this feature just doesn't belong. The set > of people who need method handle literals is a tiny sliver of the Java > developer base; complicating "everyone's Java" for the sake of a few > advanced users (100% of whom are qualified to generate bytecode) is a > bad stewardship choice. There are two points here: 1) the set of people interested by this feature is tiny. 2) it complicates Java for everybody Point 1. Perhaps, the set of people is actually not that big, it's the people that write language runtimes i.e. people that really need to use method handles. But there is a bigger set of people, the ones that will use that feature, i.e. all the people that actually use java.lang.reflect are people that will use that feature. The fact that there is no method reference literal make things like creating bean properties, interceptors, proxy etc, more painful that it should. So yes the actual set of people is not big, but the people that will use it is far bigger. Point 2. It's just not true, it's the beauty of target typing. The syntax already exists, the semantics is simple. The overhead is really tiny. You can compare it to the introduction of the hexadecimal syntax for double in Java 5. The hexadecimal syntax already exists, after Java 5, the syntax can used to specified double, not something really complicated. > > I'm one of those few who has to resort to writing ASM programs to use > MHs when I want them. So I know exactly how painful it is. And still, > I'd rather impose that pain on myself and my peers than increase the > complexity of Java for everyone for the benefit of a small few. see point 1. > > The way I solved this for myself is I wrote a tool which > opportunistically mangles calls to Lookup.findXxx (with constant > arguments) into LDCs using bytecode mangling. From a > programming-model perspective, this is only slightly more painful than > what you propose -- and works more generally. (It also works for > indy.) I much prefer an approach like that. I've also written this kind of codes, John Rose did too and every people that have wanted to write JUnit Test using method handles have written something similar. But focusing only on actual users, people that are forced to use MethodHandle seems very wrong to me. R?mi > > On 12/30/2012 7:42 PM, Remi Forax wrote: >> [to lambda spec list and JSR292 EG list] >> >> I want to close one problem that we (the JSR 292 EG) have postponed to >> Java 8 when we were developing the JSR 292, >> a Java syntax to reference a method or a constructor as a method handle. >> This item was postponed for two reasons, the first one is that the JSR >> 292 was a JSR allowing to change the VM spec and not the Java spec and >> the second one was that at that time the syntax for method reference was >> in limbo and we didn't want to choose a syntax that may be incompatible >> with the upcoming lambda spec. >> >> Now that the syntax part of the lambda spec is frozen, we can solve that >> issue. >> >> Just before starting, I want to briefly explain why this syntax is >> needed. >> A runtime of a dynamic languages uses several well known function >> (usually specified as static method) implementing either part of the >> runtime logic like a function that check is the runtime class of an >> object is equals to a specified class or some global functions of the >> language it self (here is the file for JavaScript in Nashorn [1]). >> The main issue is that to be used in JSR 292 ecosystem, this functions >> must be handled by method handles and these method handles are defined >> are static final fields, >> because Java doesn't initialize these fields lazily, this greatly >> impacts the startup time of any dynamic runtime that runs on the JVM. >> >> So the goals of the syntax that convert a method reference to a method >> handle is: >> - to move some runtime checks at compile time because creating a >> method handle involve using a string and a signature. >> - to lazily create these method handle when needed (like the lambda). >> >> The proposed syntax is to use the same syntax as the method reference, >> so no new syntax. >> MethodHandle mh = String::length; >> the target type is a j.l.i.MethodHandle instead of being a functional >> interface >> >> For the semantics: >> Unlike the method reference with a functional interface, only >> unbunded method reference are allowed, so by example, >> a method reference starting with an expression or super are not >> valid. >> Moreover, because there is no type information that the compiler can >> extract from a MethodHandle, the semantics in case of several >> overloading methods, i.e. several applicable methods is simple, it's an >> ambiguity and is rejected by the compiler. >> More formally, a method reference can be converted to a >> j.l.i.MethodHandle, >> - only if it's form is >> Type::name, with Type an existing type and name a valid Java >> identifier or 'new'. >> - then from the type, the compiler gather all accessible methods >> declared in the type type or it's super type (class or interfaces), >> if a method is override-equivalent to another one, the one from >> the supertype is discarded. >> if the name is 'new', the compiler gather all constructors of the >> type type. >> if the type is an array, all public methods of j.l;Object are >> available plus clone() >> - if there are more than one method in the set of gathered method, >> the reference is ambiguous. >> - if there is no method, the reference is not available >> - if there is one method, a method handle will be created from this >> method. >> >> Here is some examples: >> class Test { >> public static void foo() { } >> } >> ... >> MethodHandle mh1 = Test::foo; // foo()V >> mh1.invokeExact(); >> >> class Test >> public void bar() { } >> } >> ... >> MethodHandle mh2 = Test::bar; // bar(Test)V >> mh2.invokeExact(new Test()); >> >> class Test { >> interface I { >> public void m(); >> } >> static class A implements I { >> public void m() { } >> } >> } >> ... >> MethodHandle mh3 = I::m; // m(I)V >> mh3.invokeExact((I)new A()); >> >> class Test { >> static class B { >> public B(int value) { } >> } >> } >> } >> ... >> MethodHandle mh4 = B::new; // B(int) >> B b = (B)mh4.invokeExact(3); >> >> class Test { >> class C { // inner class >> public C() { } >> } >> } >> ... >> MethodHandle mh5 = C::new; // C(Test) >> C c = (C)mh5.invokeExact(new Test()); >> >> class Test { >> static class D { >> Object foo() { } >> } >> static class E { >> String foo() { } // covariant return type >> } >> } >> ... >> MethodHandle mh8 = E::foo; // foo(E)String >> String s3 = (String)mh8.invokeExact(new E()); >> >> class Test { >> static class F { >> private static void m() { } // non visible method for the VM, ok >> in Java, so the compiler has to generate a bridge >> } >> } >> ... >> MethodHandle mh9 = F::m; >> mh9.invoke(); >> >> MethodHandle mh10 = String[]::clone; // String[].clone returns >> Object, so it needs a bridge by the compiler >> String[] values = (String[])mh10.invoke(args); >> >> As you can see the syntax already exist, the semantics of the >> attribution (find the correct method) is simpler than the method call >> semantics (there is no inference) and after the attribution, the rules >> for the generation, bridge, etc. are the same as the one for the method >> reference with a functional interface as target. >> >> The compiler generates like the method reference an invokedynamic with a >> different bootstrap method that takes the constant method handle as >> parameter and create a constant callsite with this constant method >> handle (so at runtime a method reference seen as a method handle is >> always constant). >> >> And because Brian will say that this is interesting but Oracle has no >> resource that can work on this, >> here is the patch for the compiler* part based on lambda/langtools : >> http://www-igm.univ-mlv.fr/~forax/tmp/method-ref-method-handle-langtools.diff >> >> >> and the patch for the jdk part based on lambda/jdk: >> http://www-igm.univ-mlv.fr/~forax/tmp/method-ref-method-handle-jdk.diff >> >> cheers, >> R?mi >> * note, the mh9 doesn't work at runtime because it also doesn't work for >> method reference with a functional interface. >> >> [1] >> http://hg.openjdk.java.net/nashorn/jdk8/nashorn/file/b4b05457b8b2/src/jdk/nashorn/internal/runtime/GlobalFunctions.java >> >> >>