From c.m.bockisch at cs.utwente.nl Thu Jul 1 06:06:11 2010 From: c.m.bockisch at cs.utwente.nl (Christoph Bockisch) Date: Thu, 01 Jul 2010 15:06:11 +0200 Subject: Invitation to submit to VMIL'10 Message-ID: <4C2C92C3.9050509@cs.utwente.nl> Dear Colleagues: It gives us great pleasure to invite you to submit your contributions to **the fourth international workshop on Virtual Machines and Intermediate Languages (VMIL 2010)**, which will be co-located with SPLASH 2010 (former OOPSLA). This workshop is a forum for research in virtual machines (VM) and intermediate languages (IL). It is dedicated to identifying programming mechanisms and constructs that are currently realized as code transformations or implemented in libraries but should rather be supported at VM and IL level. ------------------------------------------------------------ **Invited Talks** In the tradition of past VMIL workshops, the 2010 edition will feature high-quality, on-topic invited talks. At this moment, talks by Cliff Click and Kathryn McKinley are confirmed. ------------------------------------------------------------ **Due Dates** Papers for VMIL are due Aug 9, 2010. Abstract submission is not required but recommended until Aug 2, 2010. Notification of acceptance will be on August 30, 2010. ------------------------------------------------------------ **Program Committee** We have once again assembled an excellent program committee which consists of Walter Binder, Steve Blackburn, Erik Ernst, Naveen Kumar, Doug Simon, Roel Wuyts, and all workshop organizers. ------------------------------------------------------------ For more details please see: http://www.cs.iastate.edu/~design/vmil/ We look forward to your submissions to VMIL 2010. Best regards, Hridesh Rajan, Christoph Bockisch, Michael Haupt and Robert Dyer VMIL 2010 Organizers From brian.goetz at oracle.com Thu Jul 1 16:38:45 2010 From: brian.goetz at oracle.com (Brian Goetz) Date: Thu, 01 Jul 2010 19:38:45 -0400 Subject: Reminder -- 2010 JVM Language Summit Message-ID: <4C2D2705.1040200@oracle.com> We're less than four weeks away from the JVM Language Summit! The 2010 JVM Language Summit to be held at Oracle's Santa Clara campus on July 26-28, 2010. Registration is still open for general attendance. The JVM Language Summit is an open technical collaboration among language designers, compiler writers, tool builders, runtime engineers, and VM architects. We will share our experiences as creators of programming languages for the JVM and of the JVM itself. We also welcome non-JVM developers on similar technologies to attend or speak on their runtime, VM, or language of choice. The format this year will be similar to last year; we've divided the schedule equally between traditional presentations (we're limiting most presentations to 30 minutes) and "Workshops". Workshops are informal, facilitated discussion groups among smaller, self-selected participants, and should enable "deeper dives" into the subject matter. There will also be impromptu "lightning talks". The Summit is being organized by language and JVM engineers; no marketers involved! So bring your slide rules and be prepared for some seriously geeky discussions. The registration page is now open at: http://registration.jvmlangsummit.com/ If you have any questions, send inquiries to inquire at jvmlangsummit.com. We hope to see you there! From peter.levart at marand.si Fri Jul 2 05:25:02 2010 From: peter.levart at marand.si (Peter Levart) Date: Fri, 2 Jul 2010 14:25:02 +0200 Subject: SAM conversion In-Reply-To: <20100628122832.88BB1475CD@hg.openjdk.java.net> References: <20100628122832.88BB1475CD@hg.openjdk.java.net> Message-ID: <201007021425.02494.peter.levart@marand.si> Hello Maurizio! #1 - I played a little with current prototype and noticed that this code: package lambda; public class SamTest1 { public static abstract class AbstractRunnable1 implements Runnable { } public static abstract class AbstractRunnable2 implements Runnable { } public static void doRun1(AbstractRunnable1 r) { r.run(); } public static void doRun2(AbstractRunnable2 r) { r.run(); } public static void main(String[] args) { #void() lambda = #() { System.out.println("Hello!"); }; doRun1(lambda); doRun2(lambda); } } ...produces verification error when trying to run it: Exception in thread "main" java.lang.VerifyError: Bad type on operand stack in method lambda.SamTest1.main([Ljava/lang/String;)V at offset 42 at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:186) at com.intellij.rt.execution.application.AppMain.main(AppMain.java:107) I noticed this when I wanted to test the state of current SAM conversion functionality. As I found out it follows the straw-man proposal which has a number of problems that were discussed on this list. I'll just summarize two of them: #2 - the following code: package lambda; public class SamTest2 { public static abstract class CountingRunnable implements Runnable { public int counter; } public static void doRun(CountingRunnable r) { System.out.print("Run #" + (++r.counter) + ": "); r.run(); } public static void main(String[] args) { #void() lambda = #() { System.out.println("Hello!"); }; Runnable r1 = lambda; Runnable r2 = lambda; System.out.println("r1 == r2: " + (r1 == r2)); doRun(lambda); doRun(lambda); } } ...prints: r1 == r2: false Run #1: Hello! Run #1: Hello! #3 - the following code: package lambda; import java.io.IOException; public class SamTest3 { public static abstract class SneakyRunnable implements Runnable { public SneakyRunnable() throws IOException { throw new IOException("sneaked!"); } } public static void main(String[] args) { #void() lambda = #() { System.out.println("Hello!"); }; SneakyRunnable r = lambda; } } ...throws: Exception in thread "main" java.io.IOException: sneaked! at lambda.SamTest3$SneakyRunnable.(SamTest3.java:15) at lambda.SamTest3$1.(Unknown Source) at lambda.SamTest3.main(SamTest3.java:25) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:613) at com.intellij.rt.execution.application.AppMain.main(AppMain.java:110) Do you have any information about the direction Oracle will take to solve these problems? As these are problems of the specification, I can see why prototype is not solving them yet. Anyway, I think they can be split into "light" and "serious" problems: The "r1 == r2: false" is in the "light" category. We already have the same problem with auto-boxing of primitives and it is not causing any noticeable errors in practice. The other two: "Run #1: Hello!, Run #1: Hello!" and "IOException: sneaked!" are more serious and I think that the specification will sooner or later have to take a stand on how to solve them. The specification could take the stand that "any implicit conversion of function typed value to a SAM type always produces new instance of SAM object". This would simply mean that problems #2 are not acknowledged problems and problem #3 could be solved by having the compiler infer that "SneakyRunnable r = lambda;" statement can throw IOException. Many people would find this acceptable. I don't know yet if I'm one of them. Alternately specification could take care of the problems by making the function type -> SAM type conversion "explicitly visible" with new syntax. This would, for the price of new syntax and some more typing, make browsing the code easier, since known code patterns would preserve semantics and new code patterns would explicitly speak their own semantics. Take for example the following syntax: SamTypedExpression: "new" FunctionTypedExpression For example: #void() lambda = #() { System.out.println("Hello!"); }; Runnable r1 = new lambda; Runnable r2 = new lambda; doSomethingWithSneakyRunnable( new #(){ System.out.println("Hello!"); } ); This syntax has it's drawbacks. It's ambiguous in the following example: public class ExplicitSamConversion { public static class Task implements Runnable { public void run() { System.out.println("Hello from class!"); } } public static #void() Task() { return #() { System.out.println("Hello from lambda!"); }; } public static void main(String[] args) { Runnable r = new Task(); } } ...but it could be disambiguated by compiler giving precedence to class instantiation when both class name and method name are in scope. The work-around for invoking method would simply be placing parentheses around method invocation: Runnable r = new (Task()); In practice such clashes would be rare for those following Java naming conventions. So, how is lambda specification doing so far? Regards, Peter From nathan.bryant at linkshare.com Fri Jul 2 13:34:00 2010 From: nathan.bryant at linkshare.com (Nathan Bryant) Date: Sat, 3 Jul 2010 05:34:00 +0900 Subject: First draft of translation document References: <7FDA6630E1822F448C97A48D5D73309474E4CE@EXVMSTOR302.intra.rakuten.co.jp> <4BF55612.30508@oracle.com> Message-ID: <7FDA6630E1822F448C97A48D5D733094F21FCA@EXVMSTOR302.intra.rakuten.co.jp> Brian Goetz wrote: > This may not seem like a big deal, but partial application gives the VM way > more information -- type information, nullity information, array bounds > information -- with which it can perform all sorts of optimizations (type > sharpening which leads to better inlining, null check elimination, array > bounds check elimination, dead code elimination, etc). I'm not sure where these optimizations terminate. It seems like they don't occur outside of lambdas. For example, in ParallelArray, the things I'd think we'd most want to optimize are inner loops like these, to make it more likely that "procedure" can be inlined into leafApply: final void leafApply(int lo, int hi, Procedure procedure) { final Object[] a = this.array; for (int i = lo; i < hi; ++i) procedure.op(a[i]); } Obviously, the VM can't simply inline all the way from the caller of the HOF through to the worker thread. Scalar replacement doesn't happen; the "procedure.op" call site is, therefore, virtual and likely megamorphic for any nontrivial program. Your description seems to imply that MethodHandle.bind is the fundamental primitive that enables the desired optimizations; without it they don't take place; so they don't happen for code written in the classic OO style. So unless I'm misreading you, then, because there is currently nothing that would cause a bind operation /surrounding/ leafApply, those optimizations won't happen /within/ leafApply, and the programmer might conceivably have to rewrite the code that calls it along these lines to force a bind operation to take place: static final class FJOApply extends FJBase { final #void(int l, int h) atLeaf; // instead of storing Procedure here as was previously done FJOApply(AbstractParallelAnyArray pap, int lo, int hi, FJBase next, Procedure procedure) { super(pap, lo, hi, next); this.atLeaf = #(int l, int h)( pap.leafApply(l, h, procedure) ); // have to hope that pap.leafApply also gets inlined, in this case } FJOApply(AbstractParallelAnyArray pap, int lo, int hi, FJBase next, #void(int l, int h) atLeaf) { super(pap, lo, hi, next); this.atLeaf = atLeaf; } FJBase newSubtask(int l, int h, FJBase r) { return new FJOApply(pap, l, h, r, atLeaf); // instead of passing Procedure } void atLeaf(int l, int h) { // overrides a method in FJBase atleaf.(l, h); } } It seems advantageous for the library code to perform the binding operation "#(int l, int h)( pap.leafApply(l, h, procedure) )" as early as possible, since creating this method handle is likely to be costly. From brian.goetz at oracle.com Wed Jul 7 09:32:04 2010 From: brian.goetz at oracle.com (Brian Goetz) Date: Wed, 07 Jul 2010 12:32:04 -0400 Subject: State of the Lambda Message-ID: <4C34AC04.9090109@oracle.com> I have posted a "State of the Lambda" document at: http://cr.openjdk.java.net/~briangoetz/lambda/lambda-state-2.html For convenience, the text is reproduced here (in "markdown" source). This is an updated proposal to add lambda expressions (informally, "closures") to the Java programming language. This sketch is built on the [straw-man proposal][strawman] made by Mark Reinhold in December 2009. 1. Background; SAM classes --------------------------- The Java programming language already has a form of closures: anonymous inner classes. There are a number of reasons these are considered [imperfect closures][jrose_bc], primarily: - Bulky syntax - Inability to capture non-final local variables - Transparency issues surrounding the meaning of return, break, continue, and 'this' - No nonlocal control flow operators It is *not* a goal of Project Lambda to address *all* of these issues. The standard way for Java APIs to define callbacks is to use an interface representing the callback method, such as: public interface CallbackHandler { public void callback(Context c); } The CallbackHandler interface has a useful property: it has a *single abstract method*. Many common interfaces and abstract classes have this property, such as Runnable, Callable, EventHandler, or Comparator. We call these classes *SAM classes*. The biggest pain point for anonymous inner classes is bulkiness. To call a method taking a CallbackHandler, one typically creates an anonymous inner class: foo.doSomething(new CallbackHandler() { public void callback(Context c) { System.out.println("pippo"); } }); The anonymous inner class here is what some might call a "vertical problem": five lines of source code to encapsulate a single statement. > Astute readers will notice that the syntax used for examples in this document differ from that expressed in the straw-man proposal. This does *not* reflect a final decision on syntax; we are still experimenting with various candidate syntax options. 2. Lambda expressions ---------------------- Lambda expressions are anonymous functions, aimed at addressing the "vertical problem" by replacing the machinery of anonymous inner classes with a simpler mechanism. One way to do that would be to add function types to the language, but this has several disadvantages: - Mixing of structural and nominal types; - Divergence of library styles (some libraries would continue to use callback objects, while others would use function types). So, we have instead chosen to take the path of making it easier to create instances of callback objects. Here are some examples of lambda expressions: { -> 42 } { int x -> x + 1 } The first expression takes no arguments, and returns the integer 42; the second takes a single integer argument, named x, and returns x+1. Lambda expressions are distinguished from ordinary statement blocks by the presence of a (possibly empty) formal parameter list and the -> token. The lambda expressions shown so far are a simplified form containing a single expression; there is also a multi-statement form that can contain one or more statements. 3. SAM conversion ------------------ One can describe a SAM type by its return type, parameter types, and checked exception types. Similarly, one can describe the type of a lambda expression by its return type, parameter types, and exception types. Informally, a lambda expression e is *convertible-to* a SAM type S if an anonymous inner class that is a subtype of S and that declares a method with the same name as S's abstract method and a signature and return type corresponding to the lambda expressions signature and return type would be considered assignment-compatible with S. The return type and exception types of a lambda expression are inferred by the compiler; the parameter types may be explicitly specified or they may be inferred from the assignment context (see *Target Typing*, below.) When a lambda expression is converted to a SAM type, invoking the single abstract method of the SAM instance causes the body of the lambda expression to be invoked. For example, SAM conversion will happen in the context of assignment: CallbackHandler cb = { Context c -> System.out.println("pippo") }; In this case, the lambda expression has a single Context parameter, has void return type, and throws no checked exceptions, and is therefore compatible with the SAM type CallbackHandler. 4. Target Typing ----------------- Lambda expressions can *only* appear in context where it will be converted to a variable of SAM type; the type of 'this' inside the lambda expression is (a subtype of) the SAM type to which the lambda expression is being converted. So the following code will print "Yes": Runnable r = { -> if (this instanceof Runnable) System.out.println("Yes"); }; r.run(); The following use of lambda expressions is forbidden because it does not appear in a SAM-convertible context: Object o = { -> 42 }; In a method invocation context, the target type for a lambda expression used as a method parameter is inferred by examining the set of possible compatible method signatures for the method being invoked. This entails some additional complexity in method selection; ordinarily the types of all parameters are computed, and then the set of compatible methods is computed, and a most specific method is selected if possible. Inference of the target type for lambda-valued actual parameters happens after the types of the other parameters is computed but before method selection; method selection then happens using the inferred target types for the lambda-valued parameters. The type of the formal parameters to the lambda expression can also be inferred from the target type of the lambda expression. So we can abbreviate our callback handler as: CallbackHandler cb = { c -> System.out.println("pippo") }; as the type of the parameter c can be inferred from the target type of the lambda expression. Allowing the formal parameter types to be inferred in this way furthers a desirable design goal: "Don't turn a vertical problem into a horizontal problem." We wish that the reader of the code have to wade through as little code as possible before arriving at the "meat" of the lambda expression. The user can explicitly choose a target type by specifying a type name. This might be for clarity, or might be because there are multiple overloaded methods and the compiler cannot correctly chose the target type. For example: executor.submit(Callable { -> "foo" }); If the target type is an abstract class, it is an open question as to whether we want to permit an argument list so a constructor other than the no-arg constructor can be used. 5. Lambda bodies ----------------- In addition to the simplified expression form of a lambda body, a lambda body can also contain a list of statements, similar to a method body, with several differences: the break, return, and continue statements are not permitted, and a "yield" statement, whose form is similar to to the return statement, is permitted instead of a return statement. The type of a multi-statement lambda expression is inferred by unifying the type of the values yielded by the set of yield statements. As with method bodies, every control path through a multi-statement lambda expression must either yield a value, yield no value, or throw an exception. Expressions after a yield statement are unreachable. The complete syntax is given by: lambda-exp := "{" arg-list "->" lambda-body "}" arg-list := "(" args ")" | args args := arg | arg "," args arg := [ type ] identifier lambda-body := expression | statement-list [ ";" ] statement-list := statement | statement ";" statement-list 6. Instance capture -------------------- Once the target type of a lambda expression is determined, the body of a lambda expression is treated largely the same way as an anonymous inner class whose parent is the target type. The 'this' variable refers to the SAM-converted lambda (whose type is a subtype of the target type). Variables of the form OuterClassName.this refer to the instances of lexically enclosing classes, just as with inner classes. Unqualified names may refer to members of the SAM class (if it is a class and not an interface), or to members of lexically enclosing classes, using the same rules as for inner classes. For members of lexically enclosing instanaces, member capture is treated as if the references were desugared to use the appropriate "Outer.this" qualifier and Outer.this is captured as if it were a local final variable. 7. Local variable capture -------------------------- The current rules for capturing local variables of enclosing contexts in inner classes are quite restrictive; only final variables may be captured. For lambda expressions (and for consistency, probably inner class instances as well), we relax these rules to also allow for capture of *effectively final* local variables. (Informally, a local variable is effectively final if making it final would not cause a compilation failure.) It is likely that we will *not* permit capture of mutable local variables. The reason is that idioms like this: int sum = 0; list.forEach({ Element e -> sum += e.size(); }); are fundamentally serial; it is quite difficult to write lambda bodies like this that do not have race conditions. Unless we are willing to enforce (preferably statically) that such lambdas not escape their capturing thread, such a feature may likely cause more trouble than it solves. 8. Exception transparency -------------------------- A separate document on [exception transparency][etrans] proposes our strategy for amending generics to allow abstraction over thrown checked exception types. 9. Method references --------------------- SAM conversion allows us to take an anonymous method body and treat it as if it were a SAM type. It is often desirable to do the same with an existing method (such as when a class has multiple methods that are signature-compatible with Comparable.compareTo().) Method references are expressions which have the same treatment as lambda expressions (i.e., they can only be SAM-converted), but instead of providing a method body they refer to a method of an existing class or object instance. For example, consider a Person class that can be sorted by name or by age: class Person { private final String name; private final int age; public static int compareByAge(Person a, Person b) { ... } public static int compareByName(Person a, Person b) { ... } } Person[] people = ... Arrays.sort(people, #Person.compareByAge); Here, the expression #Person.compareByAge is sugar for a lambda expression whose formal argument list is copied from the method Person.compareByAge, and whose body calls Person.compareByAge. This lambda expression will then get SAM-converted to a Comparator. If the method being referenced is overloaded, it can be disambiguated by providing a list of argument types: Arrays.sort(people, #Person.compareByAge(Person, Person)); Instance methods can be referenced as well, by providing a receiver variable: Arrays.sort(people, #comparatorHolder.comparePersonByAge); In this case, the implicit lambda expression would capture a final copy of the "comparatorHolder" reference and the body would invoke the comparePersonByAge using that as the receiver. We may choose to restrict the forms that the receiver can take, rather than allowing arbitrary object-valued expressions like "#foo(bar).moo", when capturing instance method references. 10. Extension methods ---------------------- A separate document on [*defender methods*][defender] proposes our strategy for extending existing interfaces with virtual extension methods. [strawman]: http://cr.openjdk.java.net/~mr/lambda/straw-man/ [jrose_bc]: http://blogs.sun.com/jrose/entry/better_closures [defender]: http://cr.openjdk.java.net/~darcy/DefenderMethods.pdf [etrans]: http://blogs.sun.com/briangoetz/entry/exception_transparency_in_java From neal at gafter.com Wed Jul 7 09:59:37 2010 From: neal at gafter.com (Neal Gafter) Date: Wed, 7 Jul 2010 09:59:37 -0700 Subject: State of the Lambda In-Reply-To: <4C34AC04.9090109@oracle.com> References: <4C34AC04.9090109@oracle.com> Message-ID: On Wed, Jul 7, 2010 at 9:32 AM, Brian Goetz wrote: > I have posted a "State of the Lambda" document at: > > http://cr.openjdk.java.net/~briangoetz/lambda/lambda-state-2.html > > For convenience, the text is reproduced here (in "markdown" source). > A few comments: A primary reason that inner classes are imperfect closures is that they are not lexically scoped, but this reason is not even mentioned. It isn't clear where function types fit into this. If you're considering dropping them, that should be more explicit. For a variety of reasons, I believe that would be a mistake. It isn't clear from this document which contexts would allow inference of the lambda parameter type. I suggest it should be supported everywhere. Disallowing lambda parameter type inference when the lambda is an argument to an overloaded method will cripple the API designer in a number of common situations. Scala and C# (among other languages) have both method overloading and lambdas in which the lambda argument type may be inferred. The combination is allowed, and as a practical matter is quite useful. The main "disadvantage" is that it is possible to write programs that are NP-hard to compile < http://blogs.msdn.com/b/ericlippert/archive/2007/03/28/lambda-expressions-vs-anonymous-methods-part-five.aspx>. The hard (slow to compile) cases do not arise in practice, so the disadvantage does not impact the programmer. The hard part for the compiler-writer (as usual) is producing good diagnostics. Adding a "yield" statement doesn't improve the transparency issue with "return", it just pushes the issue somewhere else. Your "yield" statement is not transparent. It isn't clear how a lambda body is to be analyzed. Specifically, there is a chicken-and-egg issue with the type of "this" and overload resolution (and exception analysis). The situation is made worse by extension methods and class SAMs. For example, consider f({ -> this.foo(); }); The lambda takes no arguments and returns "void", but the exceptions thrown by "foo" affects the exceptions thrown by the lambda, and therefore affects the results of overload resolution for "f". But the results of overload resolution for "f" defines the type of "this", and therefore controls the meaning of "foo" and the exceptions thrown by "this.foo()". As you can see, the type of "this" affects the type of the lambda even when not used in a result expression. I have no idea what ordering will be implied by the specification for semantic analysis. Your statement list syntax is messed up. It only allows infinite lists. It has duplicate semicolons (there is a semicolon as a statement terminator and a second one between statements). I can't figure out why you would put an optional ";" in the lambda body when it follows a statement list; a statement list can already end with an "empty" statement, so that part of your syntax introduces an ambiguity. The logic for disallowing capture of mutable variables doesn't make sense: It is likely that we will *not* permit capture of mutable local variables. The reason is that idioms like this: int sum = 0; list.forEach({ Element e -> sum += e.size(); }); are fundamentally serial; it is quite difficult to write lambda bodies like this that do not have race conditions. Presuming list.forEach is not concurrent (I believe it very unlikely that it would be) this code has no race conditions. Lambda expressions are very useful for improving the organization of parts of the program that are not concurrent, and that benefit is undermined by restrictions that are justified by reference concurrency. Forcing the user to use a final Reference or int[] for the sum variable here doesn't address a race condition, it simply inconveniences the programmer. In short, lambdas are orthogonal to concurrency (they benefit concurrent scenarios as well as serial ones). From scolebourne at joda.org Wed Jul 7 10:18:14 2010 From: scolebourne at joda.org (Stephen Colebourne) Date: Wed, 7 Jul 2010 18:18:14 +0100 Subject: Removal of function types Message-ID: The latest (6 July) lambda document removes function types. I support this (even though they were in FCM). - Removing them reduces the scope, thus increasing the delivery likelihood - Function types look pig ugly with exception transparancy - Function types are a very different concept to the rest of Java, which is very name based - Function type don't visually look much like types (which are names everywhere else) - SAMs centralise the definition of Javadoc and give a meaningful name - function types don't - Function types were causing trouble when in a list/array due to generic erasure - Removing them removes some of the most contentious syntax debates - Function types can be added later if the right approach is taken (as in the doc) of declaring lambda expressions to have no expressible type. I suggest using this thread to comment on the removal of function types :-) Stephen From scolebourne at joda.org Wed Jul 7 10:30:02 2010 From: scolebourne at joda.org (Stephen Colebourne) Date: Wed, 7 Jul 2010 18:30:02 +0100 Subject: Type inference of parameters Message-ID: The latest (6 July) lambda document includes type inference of formal parameters (section 4). I support this decision. Without this, I fear we risk creating something akin to the wide lines of generics that are being simplified with the diamond operator. So long as a user can choose to write the type if they wish, I suspect most users will be happy. The exact mechanism for calculating the type inference does seem tricky as Neal indicates. I'd want to see more detail on how it would work in complex cases around method overloading. Like Neal, I'd want type inference to work in all cases (as corner cases are what users complain about). Can I suggest we use this thread for discussions on type inference? Stephen From neal at gafter.com Wed Jul 7 10:32:30 2010 From: neal at gafter.com (Neal Gafter) Date: Wed, 7 Jul 2010 10:32:30 -0700 Subject: Removal of function types In-Reply-To: References: Message-ID: Reducing the scope and making things pretty is nice, but not very useful if the resulting construct doesn't actually address the needs of API designers. The list/array issues are no worse or better for having eliminated function types, as SAMs were always part of the equation and generic SAMs have the same issues. Function types cannot reasonably be added later if APIs have been designed in the interim that would have been better designed using function types. I'm specifically thinking of aggregate APIs and extension methods on collections. Finally, adding function types to Java provides a nice interoperability basis for lambdas among all of the JVM languages. There is no straightforward way to do that using SAMs. On Wed, Jul 7, 2010 at 10:18 AM, Stephen Colebourne wrote: > The latest (6 July) lambda document removes function types. I support > this (even though they were in FCM). > > - Removing them reduces the scope, thus increasing the delivery likelihood > - Function types look pig ugly with exception transparancy > - Function types are a very different concept to the rest of Java, > which is very name based > - Function type don't visually look much like types (which are names > everywhere else) > - SAMs centralise the definition of Javadoc and give a meaningful name > - function types don't > - Function types were causing trouble when in a list/array due to > generic erasure > - Removing them removes some of the most contentious syntax debates > - Function types can be added later if the right approach is taken (as > in the doc) of declaring lambda expressions to have no expressible > type. > > I suggest using this thread to comment on the removal of function types :-) > > Stephen > > From scolebourne at joda.org Wed Jul 7 10:52:22 2010 From: scolebourne at joda.org (Stephen Colebourne) Date: Wed, 7 Jul 2010 18:52:22 +0100 Subject: Transparancy Message-ID: The latest (6 July) lambda document has big changes to the "transparancy" features. - this is still scoped to the lambda - Outer.this is used for lexical scoping - return is dropped - yield is added The problem with the approach is that it is schizophrenic. The previous approach with "this" and "return" had a certain consistency such that it was the same as an inner class. Since this proposal is *more* like CICE (simplified inner classes), one would have expected to have seen both "this and "return" remain. Instead "yield" appears completely at random. Firstly, the document doesn't really give us enough info on "yield" to be completely clear, but it sounds like "return" but in a lambda. But WHY? - it hasn't made the code more transparent or readable, just different - its a context keyword, which has negative implications in parsing and interactions with variables - it doesn't fit the "simpler inner class" mental model As is well known, I believe strongly that 'this' should be lexically scoped. Its just so much the standard for closures in other languages that IMO it is quite amazing that anything else is actually being considered. The only mental model that would justify lambda-scoped 'this' is if the project were renamed to "shorter syntax for inner classes" (or CICE) and the syntax adjusted to be something obviously based on inner classes (ie. just adopt the CICE proposal!!!) I also strongly oppose the 'yield' keyword. It is an unecessary addition to the language when the 'return' keyword performs a similarly suitable role. The only justification for not using 'return' is a desire to use it later for long-returns. In the original closure debates I showed many cases why this was a generally bad idea, as it introduces new exceptions at the core language level and simply isn't what most users expect. I thought I had sufficiently proven the case of short-returns during that debate when the original straw-man was produced. Do I really have to demonstrate it again? The new syntax is an offshoot of this change I suspect, but as I've indicated its the wrong offshoot. The proposal is now a CICE-like proposal, but in the body of BGGA. This combination makes no conceptual sense I'm afraid, and is frankly rather a frankenstein monster. Stephen From brian.goetz at oracle.com Wed Jul 7 11:16:00 2010 From: brian.goetz at oracle.com (Brian Goetz) Date: Wed, 07 Jul 2010 14:16:00 -0400 Subject: Removal of function types In-Reply-To: References: Message-ID: <4C34C460.800@oracle.com> > The latest (6 July) lambda document removes function types. I support > this (even though they were in FCM). Indeed it does. People will ask why, so let me share my thinking here. Among other reasons: 1. There are two basic approaches to typing: nominal and structural. The identity of a nominal is based on its name; the identity of a structural type is based on what it is composed of (such as "tuple of int, int" or "function from int to float".) Most languages pick mostly nominal or mostly structural; there are not a lot of languages that successfully mix nominal and structural typing except "around the edges." Java is almost entirely nominal (with a few exceptions: arrays are a structural type, but at the bottom there is always a nominal element type; generics have a mix of nominal and structural too, and this is in fact part of the source of many of people's complaints about generics.) Grafting a structural type system (function types) onto Java's nominal type system means new complexity and edge cases. Is the benefit of function types worth this? 2. We've been using SAM types for years, and Java developers are comfortable with them. If we added function types, it would immediately bifurcate the libraries (both JDK and external) into "old style" libraries (those that use SAM types) and "new style" libraries (those that use function types.) That's a big ugly seam in the middle of the platform. 3. No reification. There was a long thread about how useful it would be for function types to be reified. Without reification, function types are hobbled. For these reasons and others, we would prefer to leave out function types now. > - Removing them reduces the scope, thus increasing the delivery likelihood > - Function types look pig ugly with exception transparancy > - Function types are a very different concept to the rest of Java, > which is very name based > - Function type don't visually look much like types (which are names > everywhere else) > - SAMs centralise the definition of Javadoc and give a meaningful name > - function types don't > - Function types were causing trouble when in a list/array due to > generic erasure > - Removing them removes some of the most contentious syntax debates > - Function types can be added later if the right approach is taken (as > in the doc) of declaring lambda expressions to have no expressible > type. > > I suggest using this thread to comment on the removal of function types :-) > > Stephen > From int19h at gmail.com Wed Jul 7 11:17:26 2010 From: int19h at gmail.com (Pavel Minaev) Date: Wed, 7 Jul 2010 11:17:26 -0700 Subject: Transparancy In-Reply-To: References: Message-ID: On Wed, Jul 7, 2010 at 10:52 AM, Stephen Colebourne wrote: > The latest (6 July) lambda document has big changes to the > "transparancy" features. > > - this is still scoped to the lambda > - Outer.this is used for lexical scoping > - return is dropped > - yield is added > The other two that are surprising are the restrictions on "break" and "continue" in the body of the lambda. The text is not very clear on whether it is a restriction only on top level of the lambda (i.e. meaning strictly that one can't use break/continue to leave its body), or on the entire body, but as "break" and "continue" are listed alongside "return", and lambda body is contrasted with method body, it would seem that the restriction is universal. This seems to be very excessive - there's no ambiguity or uncertainty when the loop to which break/continue belongs is itself contained entirely within the body of the lambda, so why ban that? From brian.goetz at oracle.com Wed Jul 7 11:18:14 2010 From: brian.goetz at oracle.com (Brian Goetz) Date: Wed, 07 Jul 2010 14:18:14 -0400 Subject: Type inference of parameters In-Reply-To: References: Message-ID: <4C34C4E6.6040609@oracle.com> > Can I suggest we use this thread for discussions on type inference? Yes. I, for one, am in favor of type inference :) Type inference is a key underlying motivator for a number of JDK 7 language changes, including lambda and diamond. We are all about static typing, but that doesn't mean that the user has to laboriously write out every single type declaration when it is obvious from the context. From pbenedict at apache.org Wed Jul 7 11:19:53 2010 From: pbenedict at apache.org (Paul Benedict) Date: Wed, 7 Jul 2010 13:19:53 -0500 Subject: Transparancy In-Reply-To: References: Message-ID: Here's my -1 vote on the yield. On Wed, Jul 7, 2010 at 12:52 PM, Stephen Colebourne wrote: > The latest (6 July) lambda document has big changes to the > "transparancy" features. > > - this is still scoped to the lambda > - Outer.this is used for lexical scoping > - return is dropped > - yield is added > > The problem with the approach is that it is schizophrenic. The > previous approach with "this" and "return" had a certain consistency > such that it was the same as an inner class. Since this proposal is > *more* like CICE (simplified inner classes), one would have expected > to have seen both "this and "return" remain. Instead "yield" appears > completely at random. > > Firstly, the document doesn't really give us enough info on "yield" to > be completely clear, but it sounds like "return" but in a lambda. But > WHY? > - it hasn't made the code more transparent or readable, just different > - its a context keyword, which has negative implications in parsing > and interactions with variables > - it doesn't fit the "simpler inner class" mental model > > As is well known, I believe strongly that 'this' should be lexically > scoped. Its just so much the standard for closures in other languages > that IMO it is quite amazing that anything else is actually being > considered. > > The only mental model that would justify lambda-scoped 'this' is if > the project were renamed to "shorter syntax for inner classes" (or > CICE) and the syntax adjusted to be something obviously based on inner > classes (ie. just adopt the CICE proposal!!!) > > I also strongly oppose the 'yield' keyword. It is an unecessary > addition to the language when the 'return' keyword performs a > similarly suitable role. > > The only justification for not using 'return' is a desire to use it > later for long-returns. In the original closure debates I showed many > cases why this was a generally bad idea, as it introduces new > exceptions at the core language level and simply isn't what most users > expect. I thought I had sufficiently proven the case of short-returns > during that debate when the original straw-man was produced. Do I > really have to demonstrate it again? > > The new syntax is an offshoot of this change I suspect, but as I've > indicated its the wrong offshoot. The proposal is now a CICE-like > proposal, but in the body of BGGA. This combination makes no > conceptual sense I'm afraid, and is frankly rather a frankenstein > monster. > > Stephen > > From brian.goetz at oracle.com Wed Jul 7 11:26:06 2010 From: brian.goetz at oracle.com (Brian Goetz) Date: Wed, 07 Jul 2010 14:26:06 -0400 Subject: Transparancy In-Reply-To: References: Message-ID: <4C34C6BE.5010101@oracle.com> > - this is still scoped to the lambda Yes, that's true. In fact, a better way to think about these lambda expressions is that they are really "SAM expressions". They are often more compact than CICE (since the target type can be inferred) but lambdas do not (at this time) stand alone. > The problem with the approach is that it is schizophrenic. The > previous approach with "this" and "return" had a certain consistency > such that it was the same as an inner class. Since this proposal is > *more* like CICE (simplified inner classes), one would have expected > to have seen both "this and "return" remain. Instead "yield" appears > completely at random. It is indeed *more* like CICE (and therefore an inner class), it is really something in between. The quibbling over the semantics of return are a reflection of trying to force lambdas to either be exactly like inner classes or something completely new. The reality is that they are somewhere in the middle. Using "return" now complicates our ability to provide nonlocal control flow in the future. > As is well known, I believe strongly that 'this' should be lexically > scoped. Its just so much the standard for closures in other languages > that IMO it is quite amazing that anything else is actually being > considered. If you were designing a language from scratch, you would almost certainly take that path. But adding features to a mature language like Java without breaking either compatibility or the mental model of the 12M existing users requires taking into account all the decisions that have been made so far, and often leads you to places that you would never get if you started with a blank sheet of paper. From alex.blewitt at gmail.com Wed Jul 7 11:31:59 2010 From: alex.blewitt at gmail.com (Alex Blewitt) Date: Wed, 7 Jul 2010 19:31:59 +0100 Subject: Transparancy In-Reply-To: References: Message-ID: > The other two that are surprising are the restrictions on "break" and > "continue" in the body of the lambda. One would hope you could use break and continue within for/while/switch statements within the lambda itself, but that a break/continue on its own would not be permitted. Is this something that will be corrected in a subsequent version of the draft? Alex From grev at miginfocom.com Wed Jul 7 11:36:37 2010 From: grev at miginfocom.com (Mikael Grev) Date: Wed, 7 Jul 2010 20:36:37 +0200 Subject: Transparancy In-Reply-To: References: Message-ID: Yeah, yield is not a good solution. I don't think using return will confuse anyone except maybe the first time. -1 for yield. Basically, and a bit boring maybe, +1 for all of Stephen's comments and reasons below. Cheers, Mikael Grev On Jul 7, 2010, at 20:19 PM, Paul Benedict wrote: > Here's my -1 vote on the yield. > > On Wed, Jul 7, 2010 at 12:52 PM, Stephen Colebourne > wrote: >> The latest (6 July) lambda document has big changes to the >> "transparancy" features. >> >> - this is still scoped to the lambda >> - Outer.this is used for lexical scoping >> - return is dropped >> - yield is added >> >> The problem with the approach is that it is schizophrenic. The >> previous approach with "this" and "return" had a certain consistency >> such that it was the same as an inner class. Since this proposal is >> *more* like CICE (simplified inner classes), one would have expected >> to have seen both "this and "return" remain. Instead "yield" appears >> completely at random. >> >> Firstly, the document doesn't really give us enough info on "yield" to >> be completely clear, but it sounds like "return" but in a lambda. But >> WHY? >> - it hasn't made the code more transparent or readable, just different >> - its a context keyword, which has negative implications in parsing >> and interactions with variables >> - it doesn't fit the "simpler inner class" mental model >> >> As is well known, I believe strongly that 'this' should be lexically >> scoped. Its just so much the standard for closures in other languages >> that IMO it is quite amazing that anything else is actually being >> considered. >> >> The only mental model that would justify lambda-scoped 'this' is if >> the project were renamed to "shorter syntax for inner classes" (or >> CICE) and the syntax adjusted to be something obviously based on inner >> classes (ie. just adopt the CICE proposal!!!) >> >> I also strongly oppose the 'yield' keyword. It is an unecessary >> addition to the language when the 'return' keyword performs a >> similarly suitable role. >> >> The only justification for not using 'return' is a desire to use it >> later for long-returns. In the original closure debates I showed many >> cases why this was a generally bad idea, as it introduces new >> exceptions at the core language level and simply isn't what most users >> expect. I thought I had sufficiently proven the case of short-returns >> during that debate when the original straw-man was produced. Do I >> really have to demonstrate it again? >> >> The new syntax is an offshoot of this change I suspect, but as I've >> indicated its the wrong offshoot. The proposal is now a CICE-like >> proposal, but in the body of BGGA. This combination makes no >> conceptual sense I'm afraid, and is frankly rather a frankenstein >> monster. >> >> Stephen >> >> From grev at miginfocom.com Wed Jul 7 11:37:35 2010 From: grev at miginfocom.com (Mikael Grev) Date: Wed, 7 Jul 2010 20:37:35 +0200 Subject: Transparancy In-Reply-To: <4C34C6BE.5010101@oracle.com> References: <4C34C6BE.5010101@oracle.com> Message-ID: > Using "return" now complicates our ability to provide nonlocal > control flow in the future. Only if you expect to ever use only "return" as long return. I would never want to see that happen anyway, even "return return" is better, since that is the new "keyword" for a new concept. I think it is a bit backwards to introduce something new for a current thing to keep an option to have something current mean something new in the future. Cheers, Mikael Grev From brian.goetz at oracle.com Wed Jul 7 11:39:19 2010 From: brian.goetz at oracle.com (Brian Goetz) Date: Wed, 07 Jul 2010 14:39:19 -0400 Subject: Transparancy In-Reply-To: References: Message-ID: <4C34C9D7.2000204@oracle.com> Break and continue are allowed within the context of a loop inside a lambda. They are only banned at the "top level". On 7/7/2010 2:17 PM, Pavel Minaev wrote: > On Wed, Jul 7, 2010 at 10:52 AM, Stephen Colebournewrote: > >> The latest (6 July) lambda document has big changes to the >> "transparancy" features. >> >> - this is still scoped to the lambda >> - Outer.this is used for lexical scoping >> - return is dropped >> - yield is added >> > > The other two that are surprising are the restrictions on "break" and > "continue" in the body of the lambda. The text is not very clear on whether > it is a restriction only on top level of the lambda (i.e. meaning strictly > that one can't use break/continue to leave its body), or on the entire body, > but as "break" and "continue" are listed alongside "return", and lambda body > is contrasted with method body, it would seem that the restriction is > universal. This seems to be very excessive - there's no ambiguity or > uncertainty when the loop to which break/continue belongs is itself > contained entirely within the body of the lambda, so why ban that? > From neal at gafter.com Wed Jul 7 11:40:15 2010 From: neal at gafter.com (Neal Gafter) Date: Wed, 7 Jul 2010 11:40:15 -0700 Subject: Transparancy In-Reply-To: <4C34C6BE.5010101@oracle.com> References: <4C34C6BE.5010101@oracle.com> Message-ID: On Wed, Jul 7, 2010 at 11:26 AM, Brian Goetz wrote: > Using "return" now complicates our ability to provide nonlocal > control flow in the future. > The goal isn't nonlocal return so much as control transparency. Using "return" now complicates our ability to provide control transparency in the future. Using "yield" now complicates our ability to provide control transparency in the future. > As is well known, I believe strongly that 'this' should be lexically > > scoped. Its just so much the standard for closures in other languages > > that IMO it is quite amazing that anything else is actually being > > considered. > > If you were designing a language from scratch, you would almost certainly > take > that path. But adding features to a mature language like Java without > breaking either compatibility or the mental model of the 12M existing users > requires taking into account all the decisions that have been made so far, > and > often leads you to places that you would never get if you started with a > blank > sheet of paper. > By definition, one does not add language features to a "mature" language. The result of language extension should be a language that is coherent. The kinds of compromises you appear to be making undermine that. From brian.goetz at oracle.com Wed Jul 7 11:42:23 2010 From: brian.goetz at oracle.com (Brian Goetz) Date: Wed, 07 Jul 2010 14:42:23 -0400 Subject: Transparancy In-Reply-To: References: <4C34C6BE.5010101@oracle.com> Message-ID: <4C34CA8F.4050402@oracle.com> > I think it is a bit backwards to introduce something new for a current thing to keep an option to have something current mean something new in the future. Seriously? It would be outright incompetence for a language steward to *not* consider what might happen in the future. But your point is taken. In future, perhaps I should not be sharing my thoughts on the future direction of the language with the public, because its obviously too confusing. From neal at gafter.com Wed Jul 7 11:41:49 2010 From: neal at gafter.com (Neal Gafter) Date: Wed, 7 Jul 2010 11:41:49 -0700 Subject: Transparancy In-Reply-To: References: <4C34C6BE.5010101@oracle.com> Message-ID: On Wed, Jul 7, 2010 at 11:37 AM, Mikael Grev wrote: > > Using "return" now complicates our ability to provide nonlocal > > control flow in the future. > > Only if you expect to ever use only "return" as long return. I would never > want to see that happen anyway, even "return return" is better, since that > is the new "keyword" for a new concept. > > I think it is a bit backwards to introduce something new for a current > thing to keep an option to have something current mean something new in the > future. > "return" currently means return from the innermost method or constructor. Supporting return across lambda boundaries would not change that. From int19h at gmail.com Wed Jul 7 11:49:00 2010 From: int19h at gmail.com (Pavel Minaev) Date: Wed, 7 Jul 2010 11:49:00 -0700 Subject: Transparancy In-Reply-To: <4C34C6BE.5010101@oracle.com> References: <4C34C6BE.5010101@oracle.com> Message-ID: On Wed, Jul 7, 2010 at 11:26 AM, Brian Goetz wrote: > The quibbling over the semantics of return are a > reflection of trying to force lambdas to either be exactly like inner classes > or something completely new. ?The reality is that they are somewhere in the > middle. ?Using "return" now complicates our ability to provide nonlocal > control flow in the future. I don't see why it would complicate the ability to provide non-local returns in general. It would certainly complicate the ability to do so for lambdas which retain the rest of the presented syntax, but this isn't a requirement for future development. In general, the ability of the lambda to return (and otherwise transfer control) nonlocally should be captured in its type, anyway, and giving a different top-level syntax for that new lambda type would allow the semantics of return in the lambda to be redefined specifically for that syntax later on, if desired. From grev at miginfocom.com Wed Jul 7 11:59:01 2010 From: grev at miginfocom.com (Mikael Grev) Date: Wed, 7 Jul 2010 20:59:01 +0200 Subject: Transparancy In-Reply-To: <4C34CA8F.4050402@oracle.com> References: <4C34C6BE.5010101@oracle.com> <4C34CA8F.4050402@oracle.com> Message-ID: Brian, Either you read it wrong or, maybe more probable, my "swedish" english isn't tuned enough to express what I meant. I actually meant to think about the future, but don't make the present bad just to keep an option for a (IMO) bad solution open in the future. What I mean is: New things (long returns) should look and feel new. Things that works as before should look the same. So "return" should be a local return. As Neal mentioned just now that isn't maybe 100% accurate for a language expert, but I, and I think must developers, will see the body of the lambda as a method body and expect the return from it to just leave that body. Hence a simple "return" will do what's expected and yield would just confuse. Cheers, Mikael On Jul 7, 2010, at 20:42 PM, Brian Goetz wrote: >> I think it is a bit backwards to introduce something new for a current thing to keep an option to have something current mean something new in the future. > > Seriously? It would be outright incompetence for a language steward to *not* consider what might happen in the future. > > But your point is taken. In future, perhaps I should not be sharing my thoughts on the future direction of the language with the public, because its obviously too confusing. From mthornton at optrak.co.uk Wed Jul 7 12:10:38 2010 From: mthornton at optrak.co.uk (Mark Thornton) Date: Wed, 07 Jul 2010 20:10:38 +0100 Subject: Transparancy In-Reply-To: References: <4C34C6BE.5010101@oracle.com> <4C34CA8F.4050402@oracle.com> Message-ID: <4C34D12E.2050901@optrak.co.uk> Mikael Grev wrote: > New things (long returns) should look and feel new. Things that works as before should look the same. > > So "return" should be a local return. As Neal mentioned just now that isn't maybe 100% accurate for a language expert, but I, and I think must developers, will see the body of the lambda as a method body and expect the return from it to just leave that body. Hence a simple "return" will do what's expected and yield would just confuse. > > But, depending to some extent on syntax choice, it doesn't look like a method body. The current 'method' cues are missing. Mark Thornton From pbenedict at apache.org Wed Jul 7 12:12:15 2010 From: pbenedict at apache.org (Paul Benedict) Date: Wed, 7 Jul 2010 14:12:15 -0500 Subject: Transparancy In-Reply-To: References: <4C34C6BE.5010101@oracle.com> <4C34CA8F.4050402@oracle.com> Message-ID: In jest, simply introduce the "long return" syntax. Those are already keywords! :) On Wed, Jul 7, 2010 at 1:59 PM, Mikael Grev wrote: > Brian, > > Either you read it wrong or, maybe more probable, my "swedish" english isn't tuned enough to express what I meant. I actually meant to think about the future, but don't make the present bad just to keep an option for a (IMO) bad solution open in the future. > > What I mean is: > > New things (long returns) should look and feel new. Things that works as before should look the same. > > So "return" should be a local return. As Neal mentioned just now that isn't maybe 100% accurate for a language expert, but I, and I think must developers, will see the body of the lambda as a method body and expect the return from it to just leave that body. Hence a simple "return" will do what's expected and yield would just confuse. > > > Cheers, > Mikael > > > > On Jul 7, 2010, at 20:42 PM, Brian Goetz wrote: > >>> I think it is a bit backwards to introduce something new for a current thing to keep an option to have something current mean something new in the future. >> >> Seriously? ?It would be outright incompetence for a language steward to *not* consider what might happen in the future. >> >> But your point is taken. ?In future, perhaps I should not be sharing my thoughts on the future direction of the language with the public, because its obviously too confusing. > > > From alex.blewitt at gmail.com Wed Jul 7 12:20:23 2010 From: alex.blewitt at gmail.com (Alex Blewitt) Date: Wed, 7 Jul 2010 20:20:23 +0100 Subject: Transparancy In-Reply-To: References: <4C34C6BE.5010101@oracle.com> Message-ID: On 7 Jul 2010, at 19:41, Neal Gafter wrote: > On Wed, Jul 7, 2010 at 11:37 AM, Mikael Grev wrote: > >>> Using "return" now complicates our ability to provide nonlocal >>> control flow in the future. >> >> Only if you expect to ever use only "return" as long return. I would never >> want to see that happen anyway, even "return return" is better, since that >> is the new "keyword" for a new concept. >> >> I think it is a bit backwards to introduce something new for a current >> thing to keep an option to have something current mean something new in the >> future. >> > > "return" currently means return from the innermost method or constructor. > Supporting return across lambda boundaries would not change that. If these lambdas are shorthand for creating a SAM, then "return" would work in both places to mean return from this execution context: new Thread(new Runnable() { void run() { return; } ); new Thread( {-> return;}); This would keep the meaning of "return" sane to existing Java developers who think of lambdas as synonymous with inner classes, especially as "this" has exactly the same meaning for both under the current proposal. For those cases where you want to GOTO some other part of the program, I suggest we invent a new (or previously reserved keyword) for that purpose. Of course, some people may consider it harmful. Alex From grev at miginfocom.com Wed Jul 7 12:21:24 2010 From: grev at miginfocom.com (Mikael Grev) Date: Wed, 7 Jul 2010 21:21:24 +0200 Subject: Transparancy In-Reply-To: References: <4C34C6BE.5010101@oracle.com> Message-ID: Most developers seem to think that a lambda body is comparable to a SAM/AIC implementation body, which is a method. Cheers, Mikael On Jul 7, 2010, at 20:41 PM, Neal Gafter wrote: > On Wed, Jul 7, 2010 at 11:37 AM, Mikael Grev wrote: > > Using "return" now complicates our ability to provide nonlocal > > control flow in the future. > > Only if you expect to ever use only "return" as long return. I would never want to see that happen anyway, even "return return" is better, since that is the new "keyword" for a new concept. > > I think it is a bit backwards to introduce something new for a current thing to keep an option to have something current mean something new in the future. > > "return" currently means return from the innermost method or constructor. Supporting return across lambda boundaries would not change that. From neal at gafter.com Wed Jul 7 12:26:21 2010 From: neal at gafter.com (Neal Gafter) Date: Wed, 7 Jul 2010 12:26:21 -0700 Subject: Transparancy In-Reply-To: References: <4C34C6BE.5010101@oracle.com> Message-ID: On Wed, Jul 7, 2010 at 12:21 PM, Mikael Grev wrote: > Most developers seem to think that a lambda body is comparable to a SAM/AIC > implementation body, which is a method. > Most Java developers have not used lambdas in Java. Those who have used them in other languages rarely think of them as a method body. From jkuhnert at gmail.com Wed Jul 7 12:39:47 2010 From: jkuhnert at gmail.com (Jesse Kuhnert) Date: Wed, 7 Jul 2010 15:39:47 -0400 Subject: Transparancy In-Reply-To: References: <4C34C6BE.5010101@oracle.com> Message-ID: Yep. Never noticed anyone freaking out when coming across them for the first time in other languages either. :/ On Wed, Jul 7, 2010 at 3:26 PM, Neal Gafter wrote: > On Wed, Jul 7, 2010 at 12:21 PM, Mikael Grev wrote: > >> Most developers seem to think that a lambda body is comparable to a SAM/AIC >> implementation body, which is a method. >> > > Most Java developers have not used lambdas in Java. ?Those who have used > them in other languages rarely think of them as a method body. > > From int19h at gmail.com Wed Jul 7 12:41:14 2010 From: int19h at gmail.com (Pavel Minaev) Date: Wed, 7 Jul 2010 12:41:14 -0700 Subject: Transparancy In-Reply-To: References: <4C34C6BE.5010101@oracle.com> Message-ID: On Wed, Jul 7, 2010 at 12:26 PM, Neal Gafter wrote: > On Wed, Jul 7, 2010 at 12:21 PM, Mikael Grev wrote: > >> Most developers seem to think that a lambda body is comparable to a SAM/AIC >> implementation body, which is a method. > > Most Java developers have not used lambdas in Java. Nonetheless, the existence of SAM conversion, and the immediate usability of lambdas in all contexts where anonymous inner classes were previously used, makes a clear case for lambdas as syntactic sugar for inner classes for many (I'd argue most, but that is subjective) Java developers. >?Those who have used them in other languages rarely think of them as a method body. And again I have to disagree. Most C# programmers, for one, would think of them in precisely that way, and they alone make up a huge part of "those who have used them in other languages", given the popularity of C#. Another very huge - probably even bigger, actually - group with "foreign lambda background" would be JavaScript developers, and they also tend to think of a lambda as some kind of function. Indeed, can you give an example of a popular language which 1) has explicit "return", 2) has lambdas, and either 3) does not allow for "return" in a lambda, or 4) allows for it, but only in the meaning of non-local return? The only one I can think of right off the bat is Smalltalk, and also Ruby to some extent (as it has different kinds of lambdas, with both local and non-local returns). From neal at gafter.com Wed Jul 7 12:47:57 2010 From: neal at gafter.com (Neal Gafter) Date: Wed, 7 Jul 2010 12:47:57 -0700 Subject: Transparancy In-Reply-To: References: <4C34C6BE.5010101@oracle.com> Message-ID: On Wed, Jul 7, 2010 at 12:41 PM, Pavel Minaev wrote: > > Those who have used them in other languages rarely think of them as a > method body. > > And again I have to disagree. Most C# programmers, for one, would > think of them in precisely that way, and they alone make up a huge > part of "those who have used them in other languages", given the > popularity of C#. My experience is exactly the opposite. In C# "this" is inherited from the enclosing scope. No members are inherited from the delegate object type into the body of the lambda expression. > Another very huge - probably even bigger, actually - > group with "foreign lambda background" would be JavaScript developers, > and they also tend to think of a lambda as some kind of function. > In Javascript, the meaning of "this" can be either lexical at the point of definition of the lambda or inherited from an object to which it is attached. That's confusing enough that I don't think it is something to emulate. Indeed, can you give an example of a popular language which 1) has > explicit "return", 2) has lambdas, and either 3) does not allow for > "return" in a lambda, or 4) allows for it, but only in the meaning of > non-local return? The only one I can think of right off the bat is > Smalltalk, and also Ruby to some extent (as it has different kinds of > lambdas, with both local and non-local returns). > Besides the two you named I'd add Scala. From grev at miginfocom.com Wed Jul 7 12:53:32 2010 From: grev at miginfocom.com (Mikael Grev) Date: Wed, 7 Jul 2010 21:53:32 +0200 Subject: Transparancy In-Reply-To: References: <4C34C6BE.5010101@oracle.com> Message-ID: <18818B44-DC50-436C-AE79-3649663F99E7@miginfocom.com> Correct. But it is Java we are enhancing and that legacy to think about. Otherwise, anything goes. On Jul 7, 2010, at 21:26 PM, Neal Gafter wrote: > On Wed, Jul 7, 2010 at 12:21 PM, Mikael Grev wrote: > Most developers seem to think that a lambda body is comparable to a SAM/AIC implementation body, which is a method. > > Most Java developers have not used lambdas in Java. Those who have used them in other languages rarely think of them as a method body. From int19h at gmail.com Wed Jul 7 12:59:31 2010 From: int19h at gmail.com (Pavel Minaev) Date: Wed, 7 Jul 2010 12:59:31 -0700 Subject: Transparancy In-Reply-To: References: <4C34C6BE.5010101@oracle.com> Message-ID: On Wed, Jul 7, 2010 at 12:47 PM, Neal Gafter wrote: > On Wed, Jul 7, 2010 at 12:41 PM, Pavel Minaev wrote: >> >> >?Those who have used them in other languages rarely think of them as a >> > method body. >> >> And again I have to disagree. Most C# programmers, for one, would >> think of them in precisely that way, and they alone make up a huge >> part of "those who have used them in other languages", given the >> popularity of C#. > > My experience is exactly the opposite.? In C# "this" is inherited from the > enclosing scope.? No members are inherited from the delegate object type > into the body of the lambda expression. This doesn't mean that C# programmers don't think of lambdas as methods. It means that they think of them as methods on the class, in the body of which the lambda is written (rather than a delegate type) - but a method nonetheless. Consequently, they expect "return" to work in the body of the lambda in the same manner as in any other method. Also, my comments were only with respect to the meaning of (and restrictions on) "return". The meaning of "this" is a separate, albeit related, issue. I personally don't like how it refers to the lambda object itself in the proposal, but at the same time I see how it can make some sense given existing Java semantics for anonymous inner classes, and the obvious parallels between them and lambdas. So I have no clear _objective_ position on this issue, and wouldn't want to argue either for or against it on a ground as shaky as a mere personal preference. From alex.blewitt at gmail.com Wed Jul 7 12:59:30 2010 From: alex.blewitt at gmail.com (Alex Blewitt) Date: Wed, 7 Jul 2010 20:59:30 +0100 Subject: Transparancy In-Reply-To: References: <4C34C6BE.5010101@oracle.com> Message-ID: <9F5E1D8E-FADE-43B4-A82E-658916305185@gmail.com> On 7 Jul 2010, at 20:47, Neal Gafter wrote: >> Another very huge - probably even bigger, actually - >> group with "foreign lambda background" would be JavaScript developers, >> and they also tend to think of a lambda as some kind of function. > > In Javascript, the meaning of "this" can be either lexical at the point of > definition of the lambda or inherited from an object to which it is > attached. That's confusing enough that I don't think it is something to > emulate. But the point is that return is as expected, i.e. it returns from the lambda, right? > >> The only one I can think of right off the bat is >> Smalltalk, and also Ruby to some extent (as it has different kinds of >> lambdas, with both local and non-local returns). > > Besides the two you named I'd add Scala. Scala has difficulties in even being compatible with itself. It's hardly an example of how to design (or extend) a mature language. It's worth pointing out that Objective-C's blocks, which arguably has a much wider following than Smalltalk ever did, treats blocks as syntactic sugar around a block object and yet still permits return to come out of the block and not any enclosing scope. You can still achieve non-local returns using #define macros if you really want - but all of the major APIs are now becoming oak focussed. Alex From neal at gafter.com Wed Jul 7 13:12:31 2010 From: neal at gafter.com (Neal Gafter) Date: Wed, 7 Jul 2010 13:12:31 -0700 Subject: Transparancy In-Reply-To: References: <4C34C6BE.5010101@oracle.com> Message-ID: On Wed, Jul 7, 2010 at 12:59 PM, Pavel Minaev wrote: > On Wed, Jul 7, 2010 at 12:47 PM, Neal Gafter wrote: > > My experience is exactly the opposite. In C# "this" is inherited from > the > > enclosing scope. No members are inherited from the delegate object type > > into the body of the lambda expression. > > This doesn't mean that C# programmers don't think of lambdas as > methods. It means that they think of them as methods on the class, in > the body of which the lambda is written (rather than a delegate type) > - but a method nonetheless. Consequently, they expect "return" to work > in the body of the lambda in the same manner as in any other method. > Even if I accept this logic, this is quite different from what is proposed for project lambda. From jjb at google.com Wed Jul 7 14:56:16 2010 From: jjb at google.com (Joshua Bloch) Date: Wed, 7 Jul 2010 14:56:16 -0700 Subject: State of the Lambda In-Reply-To: <4C34AC04.9090109@oracle.com> References: <4C34AC04.9090109@oracle.com> Message-ID: Brian, Thanks for the revised draft. On the whole, I like the changes because they make things simpler. There is one notable exception, and a caveat or two. The substitution of the (new) yield keyword for return in lambda bodies seems like gratuitous complexity. Java programmers already know what return does, and this draft simply creates a synonym and forces programmers to use it. It's very much like the Monty Python "Buying a Bed" sketch: you can ask people to say "dog kennel" whenever they mean "mattress" but all it does is to cause confusion. Usually they'll say mattress (return) and the loony (compiler/IDE) will start behaving badly. I would definitely revert this part of the specification to use return. It has many advantages and (so far as I can tell) no disadvantages. The proposed target typing in method invocation contexts scares me. Method invocation (overload resolution and dynamic dispatch) are already among the most complicated parts of the language and rich source of Puzzler material. I understand the desire to do type inference here, but I'm scared of the dragons that lie in wait. The syntax for specifying the target type in a method invocation context seems awfully confusing. The draft says: executor.submit(Callable { -> "foo"} ); That looks a heck of a lot like a cast, except it's missing the parens. People will therefore tend to put them in by accident: executor.submit((Callable) { -> "foo"} ); And I suppose that will be legal too (assuming a cast to a SAM type is included in the definition of "a context where [an expression] will be converted to a variable of a SAM type"). I'd guess I'm not the only one who finds this confusing. If we do end up supporting fancy type-inference with target typing, I suspect people will want to use it in other contexts besides actual parameters. I'm reminded of the current mess with type parameter inferencing, where it's *not* legal in an actual parameter context, so you end up having to introduce a local variable to avoid specifying a type parameter explicitly (which would, in turn, cause God to kill a kitten). I realize that I'm not proposing a solution to this constellation of issues, but merely voicing a vague unease. Section 5 introduces lambdas with statement lists in place of expressions, but such a lambda was already shown in Section 4 (the Runnable containing the if-statement). (This is just a pedagogy thing.) The optional semicolon in the lambda-body syntax seems wrong. Also the syntax doesn't appear to admit the explicit target type, but since that's restricted to actual parameters, presumably its syntax is described elsewhere. At the beginning of Section 9, Comparable.compareTo() should be replaced by Comparator.compare. (Later in the section, the discussion does switch to Comparator.) I'm a little bit worried that method references with receiver variables do an copy. This is probably in line with people's expectations, but if the receiver variable that's being copied is a field reference, the behavior can be very different from that of a lambda expression where the field is referred to from within the body: if the value of the field changes between the creation of the lambda and its execution, the latter form will reflect the changed value, while the former (which captures the value at lambda-create time) won't. A possible solution is to require such receiver variables to be (implicitly) final. Regards, Josh On Wed, Jul 7, 2010 at 9:32 AM, Brian Goetz wrote: > I have posted a "State of the Lambda" document at: > > http://cr.openjdk.java.net/~briangoetz/lambda/lambda-state-2.html > > For convenience, the text is reproduced here (in "markdown" source). > > > > This is an updated proposal to add lambda expressions (informally, > "closures") to the Java programming language. This sketch is built on > the [straw-man proposal][strawman] made by Mark Reinhold in December > 2009. > > > 1. Background; SAM classes > --------------------------- > > The Java programming language already has a form of closures: > anonymous inner classes. There are a number of reasons these are > considered [imperfect closures][jrose_bc], primarily: > > - Bulky syntax > - Inability to capture non-final local variables > - Transparency issues surrounding the meaning of return, break, > continue, and 'this' > - No nonlocal control flow operators > > It is *not* a goal of Project Lambda to address *all* of these issues. > > The standard way for Java APIs to define callbacks is to use an > interface representing the callback method, such as: > > public interface CallbackHandler { > public void callback(Context c); > } > > The CallbackHandler interface has a useful property: it has a *single > abstract method*. Many common interfaces and abstract classes have > this property, such as Runnable, Callable, EventHandler, or > Comparator. We call these classes *SAM classes*. > > The biggest pain point for anonymous inner classes is bulkiness. To > call a method taking a CallbackHandler, one typically creates an > anonymous inner class: > > foo.doSomething(new CallbackHandler() { > public void callback(Context c) { > System.out.println("pippo"); > } > }); > > The anonymous inner class here is what some might call a "vertical > problem": five lines of source code to encapsulate a single statement. > > > Astute readers will notice that the syntax used for examples in this > document differ from that expressed in the straw-man proposal. This > does *not* reflect a final decision on syntax; we are still > experimenting with various candidate syntax options. > > > 2. Lambda expressions > ---------------------- > > Lambda expressions are anonymous functions, aimed at addressing the > "vertical problem" by replacing the machinery of anonymous inner > classes with a simpler mechanism. One way to do that would be to add > function types to the language, but this has several disadvantages: > - Mixing of structural and nominal types; > - Divergence of library styles (some libraries would continue to use > callback objects, while others would use function types). > > So, we have instead chosen to take the path of making it easier to > create instances of callback objects. > > Here are some examples of lambda expressions: > > { -> 42 } > > { int x -> x + 1 } > > The first expression takes no arguments, and returns the integer 42; > the second takes a single integer argument, named x, and returns x+1. > > Lambda expressions are distinguished from ordinary statement blocks by > the presence of a (possibly empty) formal parameter list and the -> > token. The lambda expressions shown so far are a simplified form > containing a single expression; there is also a multi-statement form > that can contain one or more statements. > > > 3. SAM conversion > ------------------ > > One can describe a SAM type by its return type, parameter types, and > checked exception types. Similarly, one can describe the type of a > lambda expression by its return type, parameter types, and exception > types. > > Informally, a lambda expression e is *convertible-to* a SAM type S if > an anonymous inner class that is a subtype of S and that declares a > method with the same name as S's abstract method and a signature and > return type corresponding to the lambda expressions signature and > return type would be considered assignment-compatible with S. > > The return type and exception types of a lambda expression are > inferred by the compiler; the parameter types may be explicitly > specified or they may be inferred from the assignment context (see > *Target Typing*, below.) > > When a lambda expression is converted to a SAM type, invoking the > single abstract method of the SAM instance causes the body of the > lambda expression to be invoked. > > For example, SAM conversion will happen in the context of assignment: > > CallbackHandler cb = { Context c -> System.out.println("pippo") }; > > In this case, the lambda expression has a single Context parameter, > has void return type, and throws no checked exceptions, and is > therefore compatible with the SAM type CallbackHandler. > > > 4. Target Typing > ----------------- > > Lambda expressions can *only* appear in context where it will be > converted to a variable of SAM type; the type of 'this' inside the > lambda expression is (a subtype of) the SAM type to which the lambda > expression is being converted. So the following code will print > "Yes": > > Runnable r = { -> > if (this instanceof Runnable) > System.out.println("Yes"); > }; > r.run(); > > The following use of lambda expressions is forbidden because it does > not appear in a SAM-convertible context: > > Object o = { -> 42 }; > > In a method invocation context, the target type for a lambda > expression used as a method parameter is inferred by examining the set > of possible compatible method signatures for the method being invoked. > This entails some additional complexity in method selection; > ordinarily the types of all parameters are computed, and then the set > of compatible methods is computed, and a most specific method is > selected if possible. Inference of the target type for lambda-valued > actual parameters happens after the types of the other parameters is > computed but before method selection; method selection then happens > using the inferred target types for the lambda-valued parameters. > > The type of the formal parameters to the lambda expression can also be > inferred from the target type of the lambda expression. So we can > abbreviate our callback handler as: > > CallbackHandler cb = { c -> System.out.println("pippo") }; > > as the type of the parameter c can be inferred from the target type > of the lambda expression. > > Allowing the formal parameter types to be inferred in this way > furthers a desirable design goal: "Don't turn a vertical problem into > a horizontal problem." We wish that the reader of the code have to > wade through as little code as possible before arriving at the "meat" > of the lambda expression. > > The user can explicitly choose a target type by specifying a type > name. This might be for clarity, or might be because there are > multiple overloaded methods and the compiler cannot correctly chose > the target type. For example: > > executor.submit(Callable { -> "foo" }); > > If the target type is an abstract class, it is an open question as to > whether we want to permit an argument list so a constructor other than > the no-arg constructor can be used. > > > 5. Lambda bodies > ----------------- > > In addition to the simplified expression form of a lambda body, a > lambda body can also contain a list of statements, similar to a method > body, with several differences: the break, return, and continue > statements are not permitted, and a "yield" statement, whose form is > similar to to the return statement, is permitted instead of a return > statement. The type of a multi-statement lambda expression is > inferred by unifying the type of the values yielded by the set of > yield statements. As with method bodies, every control path through a > multi-statement lambda expression must either yield a value, yield no > value, or throw an exception. Expressions after a yield statement are > unreachable. > > The complete syntax is given by: > > lambda-exp := "{" arg-list "->" lambda-body "}" > arg-list := "(" args ")" | args > args := arg | arg "," args > arg := [ type ] identifier > lambda-body := expression | statement-list [ ";" ] > statement-list := statement | statement ";" statement-list > > > 6. Instance capture > -------------------- > > Once the target type of a lambda expression is determined, the body of > a lambda expression is treated largely the same way as an anonymous > inner class whose parent is the target type. The 'this' variable > refers to the SAM-converted lambda (whose type is a subtype of the > target type). Variables of the form OuterClassName.this refer to the > instances of lexically enclosing classes, just as with inner classes. > Unqualified names may refer to members of the SAM class (if it is a > class and not an interface), or to members of lexically enclosing > classes, using the same rules as for inner classes. > > For members of lexically enclosing instanaces, member capture is > treated as if the references were desugared to use the appropriate > "Outer.this" qualifier and Outer.this is captured as if it were a > local final variable. > > > 7. Local variable capture > -------------------------- > > The current rules for capturing local variables of enclosing contexts > in inner classes are quite restrictive; only final variables may be > captured. For lambda expressions (and for consistency, probably inner > class instances as well), we relax these rules to also allow for > capture of *effectively final* local variables. (Informally, a local > variable is effectively final if making it final would not cause a > compilation failure.) > > It is likely that we will *not* permit capture of mutable local > variables. The reason is that idioms like this: > > int sum = 0; > list.forEach({ Element e -> sum += e.size(); }); > > are fundamentally serial; it is quite difficult to write lambda bodies > like this that do not have race conditions. Unless we are willing to > enforce (preferably statically) that such lambdas not escape their > capturing thread, such a feature may likely cause more trouble than it > solves. > > > 8. Exception transparency > -------------------------- > > A separate document on [exception transparency][etrans] proposes our > strategy > for amending generics to allow abstraction over thrown checked > exception types. > > > 9. Method references > --------------------- > > SAM conversion allows us to take an anonymous method body and treat it > as if it were a SAM type. It is often desirable to do the same with > an existing method (such as when a class has multiple methods that are > signature-compatible with Comparable.compareTo().) > > Method references are expressions which have the same treatment as > lambda expressions (i.e., they can only be SAM-converted), but instead > of providing a method body they refer to a method of an existing class > or object instance. > > For example, consider a Person class that can be sorted by name or by > age: > > class Person { > private final String name; > private final int age; > > public static int compareByAge(Person a, Person b) { ... } > > public static int compareByName(Person a, Person b) { ... } > } > > Person[] people = ... > Arrays.sort(people, #Person.compareByAge); > > Here, the expression #Person.compareByAge is sugar for a lambda > expression whose formal argument list is copied from the method > Person.compareByAge, and whose body calls Person.compareByAge. This > lambda expression will then get SAM-converted to a Comparator. > > If the method being referenced is overloaded, it can be disambiguated > by providing a list of argument types: > > Arrays.sort(people, #Person.compareByAge(Person, Person)); > > Instance methods can be referenced as well, by providing a receiver > variable: > > Arrays.sort(people, #comparatorHolder.comparePersonByAge); > > In this case, the implicit lambda expression would capture a final > copy of the "comparatorHolder" reference and the body would invoke > the comparePersonByAge using that as the receiver. > > We may choose to restrict the forms that the receiver can take, rather > than allowing arbitrary object-valued expressions like > "#foo(bar).moo", when capturing instance method references. > > > 10. Extension methods > ---------------------- > > A separate document on [*defender methods*][defender] proposes our > strategy for extending existing interfaces with virtual extension > methods. > > [strawman]: http://cr.openjdk.java.net/~mr/lambda/straw-man/ > [jrose_bc]: http://blogs.sun.com/jrose/entry/better_closures > [defender]: http://cr.openjdk.java.net/~darcy/DefenderMethods.pdf > [etrans]: > http://blogs.sun.com/briangoetz/entry/exception_transparency_in_java > > > From bobfoster at gmail.com Wed Jul 7 16:01:24 2010 From: bobfoster at gmail.com (Bob Foster) Date: Wed, 7 Jul 2010 16:01:24 -0700 Subject: Transparency Message-ID: Neal Gafter wrote: > Most Java developers have not used lambdas in Java. Those who have used > them in other languages rarely think of them as a method body. A tautology followed by an unprovable assertion, but never mind that. This discussion suffers a definitional problem: there is no consensus on what 'lambda' means. Instead, some people are pushing lambda as in lisp, others are trying to redefine lambda to be Smalltalk blocks or lambda as in ad hoc feature in some language that happens to call the feature lambda. I will just state for the record that I have used lambda in other languages and I think of them as anonymous functions. I truly doubt that puts me in the 'rarely' category. But let's put it on the table and stop jawboning about what return means. What does lambda mean? Bob From dl at cs.oswego.edu Wed Jul 7 16:05:38 2010 From: dl at cs.oswego.edu (Doug Lea) Date: Wed, 07 Jul 2010 19:05:38 -0400 Subject: Removal of function types In-Reply-To: References: Message-ID: <4C350842.1050800@cs.oswego.edu> On 07/07/10 13:18, Stephen Colebourne wrote: > I suggest using this thread to comment on the removal of function types :-) > As one of the instigators-by-counter-example of function types, I do wonder what the plan is for providing dozens if not hundreds of SAM types for (parallel) aggregate operations. As in my infamous workarounds at: http://gee.cs.oswego.edu/dl/jsr166/dist/extra166ydocs/extra166y/Ops.html -Doug From maurizio.cimadamore at oracle.com Wed Jul 7 16:06:30 2010 From: maurizio.cimadamore at oracle.com (maurizio cimadamore) Date: Thu, 08 Jul 2010 00:06:30 +0100 Subject: Type inference of parameters In-Reply-To: References: Message-ID: <4C350876.7060509@oracle.com> On 07/07/2010 18:30, Stephen Colebourne wrote: > The latest (6 July) lambda document includes type inference of formal > parameters (section 4). I support this decision. > > Without this, I fear we risk creating something akin to the wide lines > of generics that are being simplified with the diamond operator. So > long as a user can choose to write the type if they wish, I suspect > most users will be happy. > > The exact mechanism for calculating the type inference does seem > tricky as Neal indicates. I'd want to see more detail on how it would > work in complex cases around method overloading. Like Neal, I'd want > type inference to work in all cases (as corner cases are what users > complain about). > I think an example of problematic overload resolution would be the following: interface SAM1 { void m1(Integer n) throws Exception; } interface SAM2 { void m2(Integer n); } void call(SAM1 sam) { ... } void call(SAM2 sam) { ... } call({ x -> System.out.println(x); }); In the general case, it is not clear how the compiler should attribute the lambda body when there is no unique target for the SAM conversion; one option would be to do a trial-and-error attribution, as described in [1]. Instead of solving a problem by introducing another one (as Josh said in a separate post, Java method resolution is one of the most complicated parts of the language), we decided to ban this kind of type-inference - eventually the user will be able to disambiguate at the call site in a very java-ish fashion: call(SAM1 { x -> System.out.println(x); }); Maurizio [1] - http://blogs.msdn.com/b/ericlippert/archive/2007/03/26/lambda-expressions-vs-anonymous-methods-part-four.aspx From brian.goetz at oracle.com Wed Jul 7 16:13:05 2010 From: brian.goetz at oracle.com (Brian Goetz) Date: Wed, 07 Jul 2010 19:13:05 -0400 Subject: Removal of function types In-Reply-To: <4C350842.1050800@cs.oswego.edu> References: <4C350842.1050800@cs.oswego.edu> Message-ID: <4C350A01.70005@oracle.com> The underlying problem with this counterexample is the need for primitive types. In lots of situations, it might be reasonable to live with boxing and then you only need k*9^1 SAM types, but when you really need to abstract over primitives in this way, it becomes k*9^n for n > 1. I don't have a good answer. But I don't think adding function types meets the cost/benefit analysis as a means of solving this problem either. On 7/7/2010 7:05 PM, Doug Lea wrote: > On 07/07/10 13:18, Stephen Colebourne wrote: >> I suggest using this thread to comment on the removal of function types :-) >> > > As one of the instigators-by-counter-example of function types, > I do wonder what the plan is for providing dozens if not > hundreds of SAM types for (parallel) aggregate operations. > As in my infamous workarounds at: > http://gee.cs.oswego.edu/dl/jsr166/dist/extra166ydocs/extra166y/Ops.html > > -Doug > > From nathan.bryant at linkshare.com Wed Jul 7 16:17:06 2010 From: nathan.bryant at linkshare.com (Nathan Bryant) Date: Thu, 8 Jul 2010 08:17:06 +0900 Subject: Removal of function types References: <4C350842.1050800@cs.oswego.edu> Message-ID: <7FDA6630E1822F448C97A48D5D733094FB7A93@EXVMSTOR302.intra.rakuten.co.jp> Perhaps there is an alternative middle ground, that would provide function types but make them look like SAM's: variadic generics. Wink wink, nudge nudge. -----Original Message----- From: lambda-dev-bounces at openjdk.java.net [mailto:lambda-dev-bounces at openjdk.java.net] On Behalf Of Doug Lea Sent: Wednesday, July 07, 2010 7:06 PM To: lambda-dev at openjdk.java.net Subject: Re: Removal of function types On 07/07/10 13:18, Stephen Colebourne wrote: > I suggest using this thread to comment on the removal of function types :-) > As one of the instigators-by-counter-example of function types, I do wonder what the plan is for providing dozens if not hundreds of SAM types for (parallel) aggregate operations. As in my infamous workarounds at: http://gee.cs.oswego.edu/dl/jsr166/dist/extra166ydocs/extra166y/Ops.html -Doug From brian.goetz at oracle.com Wed Jul 7 16:26:23 2010 From: brian.goetz at oracle.com (Brian Goetz) Date: Wed, 07 Jul 2010 19:26:23 -0400 Subject: Removal of function types In-Reply-To: <7FDA6630E1822F448C97A48D5D733094FB7A93@EXVMSTOR302.intra.rakuten.co.jp> References: <4C350842.1050800@cs.oswego.edu> <7FDA6630E1822F448C97A48D5D733094FB7A93@EXVMSTOR302.intra.rakuten.co.jp> Message-ID: <4C350D1F.5080005@oracle.com> What you mean, presumably, is being able to generify over primitives, likely via specialization (we're already addressing variadic generics in the context of exception transparency), as .NET does. Duh. Why didn't we think of that? :) Would love to do it, but is way out of scope for JDK 7. On 7/7/2010 7:17 PM, Nathan Bryant wrote: > Perhaps there is an alternative middle ground, that would provide > function types but make them look like SAM's: variadic generics. > > Wink wink, nudge nudge. > > > -----Original Message----- > From: lambda-dev-bounces at openjdk.java.net > [mailto:lambda-dev-bounces at openjdk.java.net] On Behalf Of Doug Lea > Sent: Wednesday, July 07, 2010 7:06 PM > To: lambda-dev at openjdk.java.net > Subject: Re: Removal of function types > > On 07/07/10 13:18, Stephen Colebourne wrote: >> I suggest using this thread to comment on the removal of function > types :-) >> > > As one of the instigators-by-counter-example of function types, > I do wonder what the plan is for providing dozens if not > hundreds of SAM types for (parallel) aggregate operations. > As in my infamous workarounds at: > http://gee.cs.oswego.edu/dl/jsr166/dist/extra166ydocs/extra166y/Ops.html > > -Doug > > > From reinier at zwitserloot.com Wed Jul 7 16:35:33 2010 From: reinier at zwitserloot.com (Reinier Zwitserloot) Date: Thu, 8 Jul 2010 01:35:33 +0200 Subject: Removal of function types In-Reply-To: <7FDA6630E1822F448C97A48D5D733094FB7A93@EXVMSTOR302.intra.rakuten.co.jp> References: <4C350842.1050800@cs.oswego.edu> <7FDA6630E1822F448C97A48D5D733094FB7A93@EXVMSTOR302.intra.rakuten.co.jp> Message-ID: How timely, I was just about to post with the same thing. This latest round of lambda proposals obviously has my stamp of approval, as it's a virtual carbon copy of a proposal I put out on the lombok mailing list about a year ago, though I admit the inference of parameter types part of it is nifty and in retrospect perhaps an obvious next move once one has already accepted the burden of mandatory immediate conversion to an existing SAM type. The obvious solution to address situations such as the ParallelArrays.Ops explosion is to allow primitives in generics. This isn't _that_ difficult; I posted a proposal to this very list a while ago with a relatively simple scheme, that would nevertheless fit both primary requirements for PA.Ops: - It allows you to eliminate the vast majority of PA.Ops's SAM types, leaving only the basic set (Reducer, Filter, etc, no more specializations such as LongReducer). - Using a primitive PA with e.g. a Reducer instead of LongReducer will achieve roughly the same performance, both CPU and memory wise. NB: And as I'm writing this, Brian chimes in that it would be way out of scope for JDK7. Well, specialization might be, but specialization isn't necessary. I'll reread my own proposal, update it where necessary, and repost it. --Reinier Zwitserloot On Thu, Jul 8, 2010 at 1:17 AM, Nathan Bryant wrote: > Perhaps there is an alternative middle ground, that would provide > function types but make them look like SAM's: variadic generics. > > Wink wink, nudge nudge. > > > -----Original Message----- > From: lambda-dev-bounces at openjdk.java.net > [mailto:lambda-dev-bounces at openjdk.java.net] On Behalf Of Doug Lea > Sent: Wednesday, July 07, 2010 7:06 PM > To: lambda-dev at openjdk.java.net > Subject: Re: Removal of function types > > On 07/07/10 13:18, Stephen Colebourne wrote: > > I suggest using this thread to comment on the removal of function > types :-) > > > > As one of the instigators-by-counter-example of function types, > I do wonder what the plan is for providing dozens if not > hundreds of SAM types for (parallel) aggregate operations. > As in my infamous workarounds at: > http://gee.cs.oswego.edu/dl/jsr166/dist/extra166ydocs/extra166y/Ops.html > > -Doug > > > > From reinier at zwitserloot.com Wed Jul 7 17:20:25 2010 From: reinier at zwitserloot.com (Reinier Zwitserloot) Date: Thu, 8 Jul 2010 02:20:25 +0200 Subject: Type inference of parameters In-Reply-To: <4C350876.7060509@oracle.com> References: <4C350876.7060509@oracle.com> Message-ID: In general I'm looking for some clarification on the topic of resolution. Here's how I currently believe it works, but this idea is a bit more specific than the state-of-lambda document: Given: a({some lambda literal}, b, c, d), the following occurs: 1. the types of b, c, and d are calculated as usual, and for the lambda literal, "[some sort of lambda literal]" is used as placeholder type. This literal contains no further signature. 2. the compiler looks through the lexical stack for any methods named 'a' that match signature ({some lambda literal}, type-of-b, type-of-c, and type-of-d). Where a method definitely cannot match, it is discarded, but if it can, it is tracked for the next step. 3. If there's only one method left, the lambda literal is now retrofitted to whatever SAM type is dictated by the first parameter of the one remaining method. If this is not possible because, for example, the lambda literal throws checked exceptions that aren't allowed or its signature doesn't fit, a compiler error is generated. 4. If there is more than one method left, either 4A or 4B happens. The spec seems to lean towards 4A but I'm not sure that's its intent. 4A. The compiler emits a compiler error listing all possible alternatives. The programmer can fix the problem by specifying the type in the lambda literal. 4B. The compiler starts using certain known facts of the lambda literal's signature to eliminate more methods. For example, the number of arguments of the lambda literals is indisputable and is not subject to e.g. the recursive type inference (which could occur if one writes "yield this;" for example). It can do similar things for any parameters which have an explicitly specified type. The compiler can also approach from the 'other side' so to speak, and for example eliminate all remaining methods in the list for which the type of the parameter where in the invocation there's the lambda literal which are NOT sam types. This process can in theory go on forever in trying to make both ends meet. In practice at some point the compiler stops trying, emits a list of remaining target methods, and suggests explicitly specifying a type, at which point resolution is exactly like java 1.6's resolution. --Reinier Zwitserloot On Thu, Jul 8, 2010 at 1:06 AM, maurizio cimadamore < maurizio.cimadamore at oracle.com> wrote: > On 07/07/2010 18:30, Stephen Colebourne wrote: > > The latest (6 July) lambda document includes type inference of formal > > parameters (section 4). I support this decision. > > > > Without this, I fear we risk creating something akin to the wide lines > > of generics that are being simplified with the diamond operator. So > > long as a user can choose to write the type if they wish, I suspect > > most users will be happy. > > > > The exact mechanism for calculating the type inference does seem > > tricky as Neal indicates. I'd want to see more detail on how it would > > work in complex cases around method overloading. Like Neal, I'd want > > type inference to work in all cases (as corner cases are what users > > complain about). > > > I think an example of problematic overload resolution would be the > following: > > interface SAM1 { > void m1(Integer n) throws Exception; > } > > interface SAM2 { > void m2(Integer n); > } > > void call(SAM1 sam) { ... } > void call(SAM2 sam) { ... } > > call({ x -> System.out.println(x); }); > > > In the general case, it is not clear how the compiler should attribute > the lambda body when there is no unique target for the SAM conversion; > one option would be to do a trial-and-error attribution, as described in > [1]. Instead of solving a problem by introducing another one (as Josh > said in a separate post, Java method resolution is one of the most > complicated parts of the language), we decided to ban this kind of > type-inference - eventually the user will be able to disambiguate at the > call site in a very java-ish fashion: > > call(SAM1 { x -> System.out.println(x); }); > > Maurizio > > [1] - > > http://blogs.msdn.com/b/ericlippert/archive/2007/03/26/lambda-expressions-vs-anonymous-methods-part-four.aspx > > > From reinier at zwitserloot.com Wed Jul 7 17:33:03 2010 From: reinier at zwitserloot.com (Reinier Zwitserloot) Date: Thu, 8 Jul 2010 02:33:03 +0200 Subject: Transparancy In-Reply-To: References: <4C34C6BE.5010101@oracle.com> Message-ID: Given that I espoused the benefits of dropping function types altogether state-of-lambda is like music to my ears, but still, a note on yield vs. this. I too feel the combination makes rather little sense. To preserve (future) transparency, yield is used instead of return. Okay. It has a secondary benefit, even, in that folks looking at java code definitely won't confuse an 'inner' return with an 'outer' one, as inner returns are now called 'yield'. However, if future transparency is the goal, with the further solution to treat return as definitely returning from a method, and not from the nearest returnable thing in lexical scope, then why is 'this' instead defined as a reference to the nearest object in lexical scope? Consider too that unqualified references (x instead of this.x) already work via lexical scoping. i.e. they would prefer the local SAM type's members before the containing class. This means that there's virtually no need to *EVER* use inner.this. I can't think of a single compelling use case for it. Even the previous last hope of a use case for it, recursion, does not apply here: If one wants to call oneself, you do have a name for this; all lambdas are always fitted to a SAM type, and the one abstract method in a SAM type has a name. That's self, and you can call it from inside a lambda literal without messing up resolution. Thus: Comparator c = {a, b -> a.startsWith("foo") ? -1 : compare(a.toLowerCase(), b)}; The above example uses recursion; the call to 'compare' is in fact a call to self. The only remaining use case is passing self as a reference to for example another method from inside the lambda. This can be solved by allowing an "X.this" syntax, where X takes the name of the SAM type. This wasn't really feasible with function types but it works great when all lambda literals must fit to a SAM type: Comparator c = {a, b -> compare(Comparator.this.toString(), a + b)}; This concept can only go wrong when trying to define a lambda literal that will fit to SAM type FOO in code that is itself in an instance method of FOO, but that seems like an extremely academic (read: so rare it can be ignored as unsolved edge case) problem that can be ignored safely by defining arbitrary but specified behaviour (such as: specified this references use lexical scoping, i.e. it would refer to the closure). --Reinier Zwitserloot On Wed, Jul 7, 2010 at 10:12 PM, Neal Gafter wrote: > On Wed, Jul 7, 2010 at 12:59 PM, Pavel Minaev wrote: > > > On Wed, Jul 7, 2010 at 12:47 PM, Neal Gafter wrote: > > > My experience is exactly the opposite. In C# "this" is inherited from > > the > > > enclosing scope. No members are inherited from the delegate object > type > > > into the body of the lambda expression. > > > > This doesn't mean that C# programmers don't think of lambdas as > > methods. It means that they think of them as methods on the class, in > > the body of which the lambda is written (rather than a delegate type) > > - but a method nonetheless. Consequently, they expect "return" to work > > in the body of the lambda in the same manner as in any other method. > > > > Even if I accept this logic, this is quite different from what is proposed > for project lambda. > > From reinier at zwitserloot.com Wed Jul 7 17:39:59 2010 From: reinier at zwitserloot.com (Reinier Zwitserloot) Date: Thu, 8 Jul 2010 02:39:59 +0200 Subject: Primitives in Generics Message-ID: State of the Lambda has introduced dropping function types entirely. This concept was in fact the driving force behind the "Primitives in Generics" I posted back in March on this mailing list. Allowing generics in primitives is very important when function types are removed from project lambda, because without doing so, PA's 132 SAM types defined in its Ops container will remain, and presumably this was the original use case for why function types were needed! I've only had to update a few lines. The executive summary is basically this: This ISNT a proposal for specialization, instead List is effectively syntax sugar for List, there are some troubles arising with T[] in method signatures that I haven't solved yet, so far as I can tell all other issues have been solved, and the superior performance of primitives is ensured via a JVM optimization that eliminates back-to-back box/unbox operations, like the JRockit VM already does. Section 1 is justification for / suggestion to drop function types altogether. As this idea has already been established by State-of-the-Lambda, I suggest you skip it unless you're interested in reading up on why implementing primitives in generics is in my opinion a better plan compared to backpedalling and adding function types. The full proposal is available in somewhat easier to look at form here: http://projectlombok.org/primitive_generics/ and has been reproduced in full below: ---------------- # Primitives in generics proposal v1.1 by Reinier Zwitserloot ## 0 Changelog * v1.0 - Original * v1.1 - Update in response to Brian Goetz' State-of-the-Lambda of July 6th 2010. Changed: * Last part of section 1 (relation to Project Lambda) to reflect changes in Project Lambda. * Paragraph 2.3 has been rewritten, because the old version lacked clarity. * A tiny clarification added to 2.10 to highlight that one method declaration can override two separate methods from a supertype in java 1.6. * Small addition to the end of 2.13 to highlight that the specialization strategy wouldn't help either. ## 1 Preamble and necessity of the feature The goal of this proposal is to allow primitives in generics. To simplify the proposal, `int` is used in all examples and in all text. All the other 7 primitive types are of course also allowed as type arguments. In particular, a closure strategy for JDK7's [Project Lambda][] to always _box_ any closure into a so-called SAM type is a far more realistic option if the used SAM types can accept primitives in their generics. (a type that contains exactly 1 abstract method, usually an interface. For example, `java.util.Comparator` is a SAM type. So is `java.lang.Runnable`). For example, [jsr166y-extra - Parallel Arrays][jsr166yx], the standard usecase example for Project Lambda, currently lists 132 SAM interface types in its [Ops]( http://gee.cs.oswego.edu/dl/jsr166/dist/extra166ydocs/extra166y/Ops.html) container class, but most of those are simply to provide a primitive-based version of a few central concepts. If primitive generics were possible, this list of 132 would be reduced to a much more manageable set. To be precise: Ops.Op Ops.BinaryOp Ops.Predicate Ops.BinaryPredicate Ops.Procedure Ops.Generator Ops.Reducer extends BinaryOp java.util.Comparator Ops.Action These names are more meaningful than their equivalent purely structural function type constructs. For example, jsr166yx includes the concept of a `Ops.Reducer` even though the signature of a reducer is 100% equivalent to the more general `Op` class. It is self-evident then that the designers of jsr166yx find this kind of purely name-based distinction important enough to dedicate 4 of Ops' 132 SAM types to it even though they didn't have to. Furthermore, as the inclusion of `java.util.Comparator` foreshadowed, sticking with names for function types fits better with existing java code. jsr166yx as it stands uses `java.util`'s version of Comparator for its Object based `ParallelArray` but defined its own versions for the primitives, such as `Ops.LongComparator`. If function types were introduced instead, then jsr166yx would obviously replace its `Ops.LongComparator` with `#int(long, long)` but it would face a dilemma for its `Object` based `ParallelArray`: Should it take `#int(T, T)`, should it take `java.util.Comparator`, or should it take both? None of these three options are fully consistent. This isn't a rare coincidence; the amount of SAM types that are in active use across the java community is staggering, and introducing function types to solve the primitives problem is going to introduce a lot of inconsistency as APIs evolve to accept closures but keep their old SAM based methods for backwards compatibility. The solution is as clear as it is obvious: Make primitives allowable as type argument. Together with a relatively minor and proven (by JRockit which already includes it) JVM optimization, both the API and the performance issues can be solved. As a secondary step, one could start questioning if, once this feature has been added, function types are even necessary. As recent traffic on the [Project Lambda mailing list]( http://mail.openjdk.java.net/mailman/listinfo/lambda-dev) has indicated, there are a host of issues with function types, from syntax ambiguities to problems around allowing arrays of function types. By sticking with SAM types, no new ideas are introduced for the java community to learn; they presumably already know that for example the expression `new Comparator[10]` is not legal. Last but certainly not least: Primitives as type arguments are a great feature for the java language all by themselves and not only as a part of a closure proposal. For example, if this proposal is implemented in JDK7, the following could be allowed: `List intList = new IntList();` and be virtually as fast as a purely `int` array based list implementation such as the ones that can be found in [GNU Trove]( http://trove4j.sourceforge.net/javadocs/gnu/trove/TIntArrayList.html), but which is API compatible with `java.util.List`. This v1.1 of the proposal does not highlight which sections of the JLS are involved or need to be changed. Its primary intention is to start a discussion, explore the feasibility, if it has clear solutions for all potential issues, and if it can justify Brian Goetz' [State of the Lambda - July 6th 2010][stateOfLambda] simplification of eliminating function types altogether. [Project Lambda]: http://openjdk.java.net/projects/lambda/ "Project Lambda Mailing List" [jsr166yx]: http://gee.cs.oswego.edu/dl/concurrency-interest/ "jsr166y - Parallel Arrays" [stateOfLambda]: http://cr.openjdk.java.net/~briangoetz/lambda/lambda-state-2.html ## 2 Proposal ### 2.1 Type Variables Type variables are the `T` in `public class Foo {}`. Type variables can only include actual types in their bounds, such as ``. As `T extends int` doesn't really convey a useful construct, we simply won't allow it. This means type variables themselves don't have to change at all. ### 2.2 Type Arguments Type Arguments come in two flavours: Just the type, as in: `List`, or as a bound, such as `List`. As primitive types don't really have subtype relations, there is again not much sense in allowing either `List` or `List`. It is therefore not allowed; *only* `List` would be legal. To be precise, `List>` would be allowed, but `List>` would not be. ### 2.3 Syntax Sugar The JVM barely supports generics (instances don't preserve their type arguments, and method signatures don't either), and changing the JVM is a major undertaking so this proposal does not include any required changes to the JVM. In other words, a common way to implement primitives in generics called _specialization_ is *NOT* what this proposal suggests. Instead, `List x;` is primarily syntax sugar for `List x;` with some extra compile-time checked caveats. ### 2.4 Legality of primitive type arguments. A primitive can be bound to a type argument anywhere its wrapped type would be legal. Therefore, `int` is valid when the bound is `T extends Number`, but it would not be if the bound was, for example, `T extends InputStream`. ### 2.5 The *typearg_int* imaginary type. Anytime a primitive type is encountered as a type argument, the compiler needs to keep track of the notion that the primitive type is used as type bound. For example, the compiler will usually have to inject synthetic methods that replace the primitive type with its wrapper type. To help clarify the actions that the compiler needs to take, we'll call any `int` that appears in a location where it is being used to fit a type variable as the imaginary `typearg_int` type. This isn't a real type, just a virtual construct of the compiler. This concept already exists in java. If, for example, you write the following code: public abstract class Example implements List { @Override public boolean void add(String x) { return true; } } and inspect the class produced by javac, you'll notice 2, and not 1, `add` method; you'll see `add(Object)` as well as `add(String)` in the generated class file. The `add(Object)` simply passes the call to the `add(String x)` method, which involves a type cast. For primitive types, this concept is no different: public abstract class Example implements List { @Override public boolean void add(int x) { return true; } } would result in not just `add(int)` but also `add(Object)` which simply passes the call on. In addition to a type cast (to `java.lang.Integer`), this wrapper method will also auto-unbox the object. ### Sugar #### 2.6 Field Read Anytime a field of type "int" is read, the compiler emits bytecode: GETFIELD/GETSTATIC "java/awt/Point" "x" "I" However, if a field of type "typearg_int" is read, it is unboxed first, so, the compiler emits: GETFIELD/GETSTATIC "java/awt/Point" "x" "Ljava/lang/Object;" INVOKEVIRTUAL "java/lang/Integer" "intValue" "()I" In other words, the field is assumed to be of the erased typevar's type (Object in the example, but could also be e.g. `Number`), is assumed to contain an `Integer` object, and `intValue()` is called on it. This act may result in the usual `ClassCastException` if heap pollution has occurred, but can additionally also result in a `NullPointerException`, even without heap pollution. This is analogous to auto-unboxing causing an NPE, which is in the current Java6 also possible even if heap pollution has not taken place. The java community has embraced autoboxing/unboxing even with this shortcoming in the typing system, hence this similar case, which is not readily solvable, can be considered the same way: A problem, but not a show-stopper. #### 2.7 Field Write Analogous to field read; instead of generating a `PUTFIELD` of type "I", the erased typevar's type is used instead, and a call to `INVOKESTATIC "java/lang/Integer" "valueOf" "(I)Ljava/lang/Integer;"` is inserted before the `PUTFIELD` instruction. Unlike the field read, there's no NullPointerException issue here. #### 2.8 Creating a field with a "typearg_int" type. This isn't possible; Fields cannot be overridden, they can only be shadowed. Fields *can* of course be defined as having a type variable as type, but a type variable's actual type is unknown. For example: public class Foo { T x; } The `T` can be anything and is in practice erased to `java.lang.Object`. This erasure will not change because of this proposal, and an erasure to a primitive type isn't possible, because `` is not legal. #### 2.9 Method invocation When invoking a method where one or more parameters is of type `typearg_int`, then during the creation of the bytecode to load the parameters on the stack, an `INVOKESTATIC` to `Integer.valueOf(int)` is generated to box the primitive. Similarly, the signature generated in the `INVOKEVIRTUAL`/`INVOKESTATIC` call is of the erased typevar's type and not `I`. If the invoked method's return type is of type `typearg_int`, the return type in the signature is as usual replaced with the erased typevar's type. Unless the return value is discarded because the method invocation appears as expression statement, an `INVOKEVIRTUAL` call to `Integer.intValue()` is inserted immediately after the `INVOKEVIRTUAL`/`INVOKESTATIC` instruction. #### 2.10 Method declaration (Preamble) This part of the spec gets somewhat complicated, but this complication is not new! For example, imagine the following two types: public class A { void foo(T in) {} } public class B extends A { void foo(String in) {} } The current javac compiler solves this scenario by generating a synthetic wrapper with the signature: `void foo(Object)`, as `A`'s `foo` method has that erased type (`Object` is `T`'s type bound in class `A`). This synthetic method wraps the call to `B`. You can actually see this in action by looking at the stack trace you get when you throw an exception from `B.foo` and then calling it via: `A x = new B(); x.foo("");` The current java gets even more complicated if the supertype contains both `void foo(T)` and `void foo(String)`, which is of course legal as `T` erases to `Object`, not `String`. When defining `void foo(String)` in `B`, *BOTH* methods are overridden by the *ONE* method declaration of `void foo(String)` in `B`! Furthermore, trying to call `foo("")` on a variable of type `A` is not possible; a *this call is ambiguous* error is generated. This error cannot be avoided by casting the argument; only by casting the receiver (an instance of `A`) to a raw type can `foo` even be called in such a circumstance. The same problems occur when primitives are legal type bounds for type variables, and the same solutions suffice as well. There is some friction here, as the above java puzzler-esque exercise shows, but I'm not aware of any proposal for e.g. Project Coin to adress this problem, and thus one can safely conclude that this oddity is not that important. #### 2.11 Method Declaration (parameter) Let's say we have: public abstract class IntList implements List { public boolean add(int v) {return false;} } In this case, both the actual method is generated, as well as a synthetic method where every `typearg_int` is replaced with `Ljava/lang/Object;` (the erased type of typevar), i.e. both of these: add(I)Z add(Ljava/lang/Object;)Z are generated, with the second method typecasting its argument to `Integer`, unboxing it, and calling the first. Both `ClassCastException` and `NullPointerException` can occur as part of this operation, but the former only if heap pollution has taken place. Note there is no risk of an explosion of method signatures when a large number of `typearg_int` typed parameters show up. In a hypothetical method `add(T1 a, T2 b, T3 c)` with all 3 parameters a type variable, when all type variables are bound to a primitive type, then only two methods are generated: all 3 as primitive, and all 3 as wrapper type. Entirely analogous to the convoluted example at the beginning of this section, if this method is accessed as a `List`, the wrapper type version will be called, but code that works directly with the `IntList` type will generate a call to the primitive version. #### 2.12 Method Declaration (return type) Let's say we have: public abstract class IntList implements List { public int get(int idx) {return 0;} } In this case, again two versions are generated: get(I)I get(I)Ljava/lang/Object; Where `Object` comes from the erased type of this particular version of `typearg_int`. The second method simply calls on the first, boxes the result, and returns that. If a method contains both `typearg_int` in its parameter list as well as its return type, still only 2 methods are produced; one with all-primitives, and one with all-wrappers. The wrapper will unbox parameters and box the returned value. ### 2.13 The JRockit gambit It is virtually impossible to make something like this: List list = new ArrayList(); as efficient as an `int[]` without reification; `ArrayList`'s implementation creates an `Object[]` to store the list's elements in, and once it does this, the performance game is automatically lost - it must create an `int[]` array to get performance that is comparable to the performance of a raw `int` array. However, it couldn't possibly do this; when `ArrayList`'s constructor is called, its type parameter has been erased, so `ArrayList` simply cannot tell what kind of array it should make. However, all hope is not lost here, as this can be made quite efficient: List list = new IntArrayList(); or, to avoid up to 8 new classes in java.util, these specialized classes can be turned into inner classes of ArrayList, and ArrayList can gain a new "static" constructor: List list = ArrayList.with(int.class); The above example is poor man's reification: This way ArrayList's "with" static method can in fact create a new IntArrayList by runtime-inspecting the `int.class` parameter. However, even with the receiver (`IntArrayList`) being specifically built with the `int` primitive type in mind, and the caller (the one that declared `List list`) also operating solely on the primitives level, the above compilation strategy still means that the following code: list.add(10); results in the following bytecode: ICONST 10 INVOKESTATIC java/lang/Integer valueOf(I)Ljava/lang/Integer; INVOKEVIRTUAL java/util/List add(Ljava/lang/Object;)Z .... execution moves to `IntArrayList`'s code INVOKEVIRTUAL java/lang/Integer intValue()I INVOKEVIRTUAL java/util/IntArrayList add(I)Z .... The hotspot compiler will eventually inline the static call and the last 2 calls (as they are to small final methods), but nevertheless there's an object on the heap involved (the boxed integer), and two calls, even if inlined. However, the JRockit VM already eliminates sequential box/unbox calls even across invocations, which eliminates the heap object entirely. Together with inlining the pass-through call from add(Object) to add(int) this call is then just as fast as calling the *GNU Trove* `TIntArrayList`. See < http://blogs.oracle.com/ohrstrom/2009/05/pulling_a_machine_code_rabbit.html> for a treatise on how the JRockit VM handles eliminating sequential box/unbox operations. In situations where either the receiver (such as `java.util.ArrayList`) or the sender (some generic code that created a new list of its own type variable `E`) is not working exclusively with `int` this optimization obviously won't work, _but_, this optimization would be impossible without at least reification, which is not feasible for JDK7. Also, the specialization strategy wouldn't allow more performant code to be generated here either. ### 2.14 Generics Conversion A type such as `Map` can be treated as a `Map` implicitly. The same property is granted to primitives; a primitive will readily convert to its wrapper type, which includes conversion to a (bound) wildcard as `Integer` has a place in the type hierarchy. In other words, a `Map` will implicitly convert to a `Map`, for example. This leads automatically to a dilemma; it means `List` can be treated as a `List`, and in that way, `null` can be added to this list which isn't technically compatible with primitives, and as a result, treating this list as a `List` again can result in a `NullPointerException`. This type conversion feels like autoboxing/unboxing which suffers from the same problem, and as covered this omission in the type system has been deemed acceptable before. As has been stated, `int` as a type bound is legal anywhere `Integer` would be, which implies that converting from for example `List` to `List` is also legal. ### 2.15 The benefit to the java community - recap Even without the JRockit optimization (which can at any time be added without requiring changes to this spec, which is focussed on the JLS and the compiler, and not the VM), this proposal is highly beneficial to the java community. Generified libraries virtually never offer primitive-optimized versions. For example, there is no ArrayList-like class for primitive ints in the java runtime; one would have to reach for third party libraries such as GNU Trove to find them. The large amount of code duplication that is required to produce a primitive-optimized version for all 8 primitive types is also very inpractical. It is clear, then, that the java community has decided to accept the performance penalty, and more problematically, the readability penalty, by choosing to use for example `List`. The readily available alternative, arrays, are continuously problematic in new proposals, as they don't work well together with generics and have different variance rules. This proposal would also help greatly in APIs which are designed with primitive arrays in mind. For example, the java.util.Arrays class has a utility method to sort an int[], but only by the natural order of integers. There is no `Comparator` based method for `int[]` in `java.util.Arrays`; there is only such a method for sorting `T[]`, and `T` can currently only stand for non-primitive types. With this feature, the following code could be legal: int[] array = {1, -3, 2, 8, 6}; Arrays.sort(array, new Comparator { public int compare(int a, int b) { a = Math.abs(a); b = Math.abs(b); return a < b ? -1 : a > b ? 1 : 0; } }); or, with closure support: Arrays.sort(array, #(int a, int b) { a = Math.abs(a); b = Math.abs(b); return a < b ? -1 : a > b ? 1 : 0; }); In other words, convenient syntax, fast execution, while *keeping* the `Comparator` concept. Comparator cannot obviously be removed (it would not be backwards compatible), and it would be inconsistent for the sorting APIs to work with `Comparator` when sorting Objects, but to work with `#int(int, int)` closures when trying to sort an array of primitive ints. ### 3 Curses! What about arrays! There's one case that this proposal has not yet covered: What happens when `typearg_int` is used in an array? Let's say we have: public class Foo { protected T[] array = null; public abstract void method(T[] param); public abstract T[] returnType(); } What would happen if we tried to write: public class Bar extends Foo { @Override public void method(int[] param) { this.array = param; } @Override public int[] returnType() { return param; } } This question seems academic, in that generics and arrays almost never meet, but that isn't quite true: `java.util.Collection` itself contains `public abstract T[] toList(T[] in);`. It is important to realize that because arrays and generics are already convoluted when used together in java6, the `T` in that signature is not related to the `E` of List itself (the `E` in `public interface Collection`!). For usage in method declarations the wrapper methods can create a new array and copy each element, autoboxing/unboxing on the way. This is mind-bogglingly inefficient, but it is simple. In practice the authors strongly suggest the implementer of `IntArrayList` write a specific method to get a straight `int[]` array without this convolution which obviously cannot be optimized by the VM the way a single `int` to/from `Integer` conversion can be. For array access, the proposal suggests treating this concept as extremely rare and thus coming up with the simplest thing that could possibly work: Access to the array (read/write) is allowed and involves boxing as usual, but a reference to `Foo.array` itself will simply remain its erased type and emits a warning. Therefore: public abstract class Bar extends Foo { void test() { Object[] o = array; //warning but legal int[] p = array; //error } } Nevertheless this concept introduces an API incompatibility: The javadoc for `toList` states that if the input array has enough room, the same array will be returned, and this cannot of course be done if the input array is first number-for-number copied into a `Object[]` array, and then after the copy operation has completed, this array is translated back to `int[]`. This proposal offers no solution to this dilemma. It can only note that as `int` is currently illegal in generics, it is technically backwards compatible to add a note to the javadoc that explains the same array is *not* returned even if it has enough room if the type bound is a primitive. It also shows a gap in the API of a hypothetical `IntArrayList`: The signature (as mandated by the `Collection` interface) must be `public T[] toArray(T[] in)`, and as `T[]`'s erasure, which is `Object[]`, is not compatible with `int[]`, this method cannot be made fast even for a specific subtype such as `IntArrayList`. This proposal offers no pragmatic solution other than to create a custom method in `IntArrayList` as well as a static utility method in `java.util.Collections` that can translate any list to `int[]`, which uses an instanceof check to pick up `java.util.ArrayList.IntArrayList`s special method and use that. The second issue raised is, in `public T[] toArray(T[] in)`, what happens if the calling code ends up binding `int` to `T`? This is a particularly painful process: Javac needs to generate the conversion from `int[]` to `Integer[]` prior to calling the method, and then the method will most likely convert back to `int[]`, and on the return phase the same double convert occurs for a grand total of 4 conversions. To avoid accidental triggering of this potentially very slow operation, a warning should be emitted anytime the compiler generates conversion code to convert a primitive array to a non-primitive array in order to fit with the erased signature of an invoked method. (For fields, as mentioned, the arrays remain their erased type, and an attempt to treat them as primitive array results in an error). --Reinier Zwitserloot From jjb at google.com Wed Jul 7 17:48:29 2010 From: jjb at google.com (Joshua Bloch) Date: Wed, 7 Jul 2010 17:48:29 -0700 Subject: Transparancy In-Reply-To: References: <4C34C6BE.5010101@oracle.com> Message-ID: Reinier, On Wed, Jul 7, 2010 at 5:33 PM, Reinier Zwitserloot wrote: > . > > To preserve (future) transparency, yield is used instead of return. I believe this is an abuse of the word "transparency." There's nothing transparent about making the "return" keyword do double duty to mean "long return." It's a new concept, and if and when it is to be supported in Java, it deserves a new keyword. > Okay. It > has a secondary benefit, even, in that folks looking at java code > definitely > won't confuse an 'inner' return with an 'outer' one, as inner returns are > now called 'yield'. This makes little sense to me. I don't think anyone would confuse a return from within a lambda with a "naked return." People don't have that problem with nested classes today. > However, if future transparency is the goal... When you say "future transparency," I believe that what you mean is "allowing lambdas to be used to emulate control constructs in the future." I think you should call it that. Josh From jkuhnert at gmail.com Wed Jul 7 17:58:01 2010 From: jkuhnert at gmail.com (Jesse Kuhnert) Date: Wed, 7 Jul 2010 20:58:01 -0400 Subject: Removal of function types In-Reply-To: References: <4C350842.1050800@cs.oswego.edu> <7FDA6630E1822F448C97A48D5D733094FB7A93@EXVMSTOR302.intra.rakuten.co.jp> Message-ID: Well that clenches it, if reineir likes it there must be something wrong. =p (just kidding, mostly..) On Wednesday, July 7, 2010, Reinier Zwitserloot wrote: > How timely, I was just about to post with the same thing. > > This latest round of lambda proposals obviously has my stamp of approval, as > it's a virtual carbon copy of a proposal I put out on the lombok mailing > list about a year ago, though I admit the inference of parameter types part > of it is nifty and in retrospect perhaps an obvious next move once one has > already accepted the burden of mandatory immediate conversion to an existing > SAM type. > > The obvious solution to address situations such as the ParallelArrays.Ops > explosion is to allow primitives in generics. > > This isn't _that_ difficult; I posted a proposal to this very list a while > ago with a relatively simple scheme, that would nevertheless fit both > primary requirements for PA.Ops: > > ?- It allows you to eliminate the vast majority of PA.Ops's SAM types, > leaving only the basic set (Reducer, Filter, etc, no more specializations > such as LongReducer). > > ?- Using a primitive PA with e.g. a Reducer instead of LongReducer > will achieve roughly the same performance, both CPU and memory wise. > > NB: And as I'm writing this, Brian chimes in that it would be way out of > scope for JDK7. Well, specialization might be, but specialization isn't > necessary. I'll reread my own proposal, update it where necessary, and > repost it. > > --Reinier Zwitserloot > > > > On Thu, Jul 8, 2010 at 1:17 AM, Nathan Bryant > wrote: > >> Perhaps there is an alternative middle ground, that would provide >> function types but make them look like SAM's: variadic generics. >> >> Wink wink, nudge nudge. >> >> >> -----Original Message----- >> From: lambda-dev-bounces at openjdk.java.net >> [mailto:lambda-dev-bounces at openjdk.java.net] On Behalf Of Doug Lea >> Sent: Wednesday, July 07, 2010 7:06 PM >> To: lambda-dev at openjdk.java.net >> Subject: Re: Removal of function types >> >> On 07/07/10 13:18, Stephen Colebourne wrote: >> > I suggest using this thread to comment on the removal of function >> types :-) >> > >> >> As one of the instigators-by-counter-example of function types, >> I do wonder what the plan is for providing dozens if not >> hundreds of SAM types for (parallel) aggregate operations. >> As in my infamous workarounds at: >> http://gee.cs.oswego.edu/dl/jsr166/dist/extra166ydocs/extra166y/Ops.html >> >> -Doug >> >> >> >> > > From reinier at zwitserloot.com Wed Jul 7 18:04:17 2010 From: reinier at zwitserloot.com (Reinier Zwitserloot) Date: Thu, 8 Jul 2010 03:04:17 +0200 Subject: Transparancy In-Reply-To: References: <4C34C6BE.5010101@oracle.com> Message-ID: Fair enough, I should stop using Neal's random vocabulary. You're indeed entirely correct in that I was referring to allowing lambdas to be treated as the block to a custom control construct. It's perhaps true that a return in an Anonymous Inner Class Literal is today rarely mistaken for a return from the containing method, but then the boilerplate and indents surrounding such a return statement are far more indicative of the usual marked for the nearest scope of a return statement: A complete method signature one indent up, and an AICL constructor invocation two indents up. This clear marker would disappear with lambdas. Perhaps the rather unique arrow of Brian's new example syntax, or the # of the original strawman serve as enough of a marker that one won't make the mistake of misreading a return statement. --Reinier Zwitserloot On Thu, Jul 8, 2010 at 2:48 AM, Joshua Bloch wrote: > Reinier, > > On Wed, Jul 7, 2010 at 5:33 PM, Reinier Zwitserloot < > reinier at zwitserloot.com> wrote: > >> . >> >> To preserve (future) transparency, yield is used instead of return. > > > > I believe this is an abuse of the word "transparency." There's nothing > transparent about making the "return" keyword do double duty to mean "long > return." It's a new concept, and if and when it is to be supported in Java, > it deserves a new keyword. > > >> Okay. It >> has a secondary benefit, even, in that folks looking at java code >> definitely >> won't confuse an 'inner' return with an 'outer' one, as inner returns are >> now called 'yield'. > > > This makes little sense to me. I don't think anyone would confuse a return > from within a lambda with a "naked return." People don't have that problem > with nested classes today. > > >> However, if future transparency is the goal... > > > When you say "future transparency," I believe that what you mean is > "allowing lambdas to be used to emulate control constructs in the future." > I think you should call it that. > > Josh > > From jkuhnert at gmail.com Wed Jul 7 18:12:09 2010 From: jkuhnert at gmail.com (Jesse Kuhnert) Date: Wed, 7 Jul 2010 21:12:09 -0400 Subject: Transparancy In-Reply-To: References: <4C34C6BE.5010101@oracle.com> Message-ID: On Wednesday, July 7, 2010, Reinier Zwitserloot wrote: > Fair enough, I should stop using Neal's random vocabulary. You're indeed > entirely correct in that I was referring to allowing lambdas to be treated > as the block to a custom control construct. > Or rather you should stop using his vocabulary to make yourself sound smart when you don't understand the meaning behind it. > > --Reinier Zwitserloot > > > > On Thu, Jul 8, 2010 at 2:48 AM, Joshua Bloch wrote: > >> Reinier, >> >> On Wed, Jul 7, 2010 at 5:33 PM, Reinier Zwitserloot < >> reinier at zwitserloot.com> wrote: >> >>> . >>> >>> To preserve (future) transparency, yield is used instead of return. >> >> >> >> I believe this is an abuse of the word "transparency." ?There's nothing >> transparent about making the "return" keyword do double duty to mean "long >> return." ?It's a new concept, and if and when it is to be supported in Java, >> it deserves a new keyword. >> >> >>> Okay. It >>> has a secondary benefit, even, in that folks looking at java code >>> definitely >>> won't confuse an 'inner' return with an 'outer' one, as inner returns are >>> now called 'yield'. >> >> >> This makes little sense to me. ?I don't think anyone would confuse a return >> from within a lambda with a "naked return." ?People don't have that problem >> with nested classes today. >> >> >>> However, if future transparency is the goal... >> >> >> When you say "future transparency," I believe that what you mean is >> "allowing lambdas to be used to emulate control constructs in the future." >> ?I think you should call it that. >> >> ? ? ? ? ?Josh >> >> > > From reinier at zwitserloot.com Wed Jul 7 20:11:39 2010 From: reinier at zwitserloot.com (Reinier Zwitserloot) Date: Thu, 8 Jul 2010 05:11:39 +0200 Subject: state-of-the-lambda syntax for non-SAM types such as WindowAdapter. Message-ID: An idea: The state-of-the-lambda document allows lambda literals to be explicitly typed: Comparator {a, b -> return a.length() - b.length()}; What if one could also append the method to this? Then these closures can extend to non-SAM types: frame.addWindowListener(WindowAdapter.windowClosed {e -> .... }); Just a thought. --Reinier Zwitserloot From neal at gafter.com Wed Jul 7 21:27:42 2010 From: neal at gafter.com (Neal Gafter) Date: Wed, 7 Jul 2010 21:27:42 -0700 Subject: Removal of function types In-Reply-To: <7FDA6630E1822F448C97A48D5D733094FB7A93@EXVMSTOR302.intra.rakuten.co.jp> References: <4C350842.1050800@cs.oswego.edu> <7FDA6630E1822F448C97A48D5D733094FB7A93@EXVMSTOR302.intra.rakuten.co.jp> Message-ID: <7B695633-97E6-4790-B44E-9AC3F3491709@gafter.com> "veridic genetics" is an interesting pair of words, but I have no idea what that would mean as a language construct, or how it would help address any of the issues here. Please tell us more. -Neal On Jul 7, 2010, at 4:17 PM, "Nathan Bryant" wrote: > Perhaps there is an alternative middle ground, that would provide > function types but make them look like SAM's: variadic generics. > > Wink wink, nudge nudge. > > > -----Original Message----- > From: lambda-dev-bounces at openjdk.java.net > [mailto:lambda-dev-bounces at openjdk.java.net] On Behalf Of Doug Lea > Sent: Wednesday, July 07, 2010 7:06 PM > To: lambda-dev at openjdk.java.net > Subject: Re: Removal of function types > > On 07/07/10 13:18, Stephen Colebourne wrote: >> I suggest using this thread to comment on the removal of function > types :-) >> > > As one of the instigators-by-counter-example of function types, > I do wonder what the plan is for providing dozens if not > hundreds of SAM types for (parallel) aggregate operations. > As in my infamous workarounds at: > http://gee.cs.oswego.edu/dl/jsr166/dist/extra166ydocs/extra166y/Ops.html > > -Doug > > > From int19h at gmail.com Wed Jul 7 21:40:38 2010 From: int19h at gmail.com (Pavel Minaev) Date: Wed, 7 Jul 2010 21:40:38 -0700 Subject: Removal of function types In-Reply-To: <7B695633-97E6-4790-B44E-9AC3F3491709@gafter.com> References: <4C350842.1050800@cs.oswego.edu> <7FDA6630E1822F448C97A48D5D733094FB7A93@EXVMSTOR302.intra.rakuten.co.jp> <7B695633-97E6-4790-B44E-9AC3F3491709@gafter.com> Message-ID: I think the meaning is something along the lines of C++ template parameter packs & their expansions in various context, which would allow you to write something like: interface Func { R invoke(A... a); } Func f0; Func f1; Func f2; ... I don't see how it would help, though, so long as type parameters cannot be primitive types. You'd still need IntAndIntToLong and friends.And it doesn't really buy much over Func0, Func1, ..., up to some sufficiently high N (what is it these days in .NET for System.Func, 16 arguments? I haven't heard complaints about it being too low). On Wed, Jul 7, 2010 at 9:27 PM, Neal Gafter wrote: > > "veridic genetics" is an interesting pair of words, but I have no idea what that would mean as a language construct, or how it would help address any of the issues here. ?Please tell us more. > > -Neal > > On Jul 7, 2010, at 4:17 PM, "Nathan Bryant" wrote: > > > Perhaps there is an alternative middle ground, that would provide > > function types but make them look like SAM's: variadic generics. > > > > Wink wink, nudge nudge. > > > > > > -----Original Message----- > > From: lambda-dev-bounces at openjdk.java.net > > [mailto:lambda-dev-bounces at openjdk.java.net] On Behalf Of Doug Lea > > Sent: Wednesday, July 07, 2010 7:06 PM > > To: lambda-dev at openjdk.java.net > > Subject: Re: Removal of function types > > > > On 07/07/10 13:18, Stephen Colebourne wrote: > >> I suggest using this thread to comment on the removal of function > > types :-) > >> > > > > As one of the instigators-by-counter-example of function types, > > I do wonder what the plan is for providing dozens if not > > hundreds of SAM types for (parallel) aggregate operations. > > As in my infamous workarounds at: > > http://gee.cs.oswego.edu/dl/jsr166/dist/extra166ydocs/extra166y/Ops.html > > > > -Doug > > > > > > > From peter.levart at gmail.com Wed Jul 7 22:46:40 2010 From: peter.levart at gmail.com (Peter Levart) Date: Thu, 8 Jul 2010 07:46:40 +0200 Subject: state-of-the-lambda syntax for non-SAM types such as WindowAdapter. In-Reply-To: References: Message-ID: <201007080746.41341.peter.levart@gmail.com> On Thursday, July 08, 2010 05:11:39 am Reinier Zwitserloot wrote: > An idea: > > The state-of-the-lambda document allows lambda literals to be explicitly > typed: > > Comparator {a, b -> return a.length() - b.length()}; > > > What if one could also append the method to this? Then these closures can > extend to non-SAM types: > > frame.addWindowListener(WindowAdapter.windowClosed {e -> .... }); > > Just a thought. > > --Reinier Zwitserloot You don't need special syntax for that. For example: public class WindowAdapters { interface Fn { void invoke(WindowEvent e); } public static WindowAdapter windowOpened(final Fn fn) { return new WindowAdapter() { @Override public void windowOpened(WindowEvent e) { fn.invoke(e); } }; } public static WindowAdapter windowClosed(final Fn fn) { return new WindowAdapter() { @Override public void windowClosed(WindowEvent e) { fn.invoke(e); } }; } // etc... } // and with an additional pair of parens you can write: frame.addWindowListener(WindowAdapters.windowClosed({e -> .... })); Regards, Peter From john.r.rose at oracle.com Wed Jul 7 23:25:06 2010 From: john.r.rose at oracle.com (John Rose) Date: Wed, 7 Jul 2010 23:25:06 -0700 Subject: State of the Lambda In-Reply-To: References: <4C34AC04.9090109@oracle.com> Message-ID: <232B8A26-69CE-4BCC-B545-F654BFDBA486@oracle.com> On Jul 7, 2010, at 2:56 PM, Joshua Bloch wrote: > I'm a little bit worried that method references with receiver variables do > an copy. ... A possible solution is to require such receiver > variables to be (implicitly) final. This is one of those times where syntax might imply something deep about an expression. Brian's sketch uses a prefix "#" to introduce a method lifted to a function (and perhaps partially applied to a receiver). The prefix looks like it might govern the evaluation of the expression to the right. An infix operator would much more clearly express that the left hand side of the expression (at least) is evaluated in the normal left-to-right order of Java expressions. (Some infix operators, ?: && || have special rules for evaluation order, but it only applies to the right of the operator.) An infix formulation would make it clearer that the qualifier LHS is a normal expression: foo # bar baz[++n].bang() # bar This means that foo#bar is not equivalent to { (T x) -> foo.bar(x) } if foo contains side effects or is affected by them. -- John P.S. FWIW, I prototyped the foo#bar syntax as a test for JSR 292, which includes an extended 'ldc' bytecode which can materialize arbitrary method constants. See the bottom of the patch for a unit test file which exercises a similar syntax. http://hg.openjdk.java.net/mlvm/mlvm/langtools/file/tip/meth-ldc-6939203.patch From mcnepp02 at googlemail.com Thu Jul 8 00:42:27 2010 From: mcnepp02 at googlemail.com (Gernot Neppert) Date: Thu, 8 Jul 2010 07:42:27 +0000 Subject: Primitives in Generics In-Reply-To: References: Message-ID: Reinier, I think your proposal is propably off-topic with regards to this mailing list, nevertheless I could not resist to respond ;-) I like the idea of making efficient use of generic classes with primitive type parameters possible. However, I'm worried that solely relying on a JVM optimization will not be sufficient. Wouldn't it be possible to gain efficiency on every VM if something along the following could be adopted: 1. Let's assume the existence of a generic interface L { void add(T element); T get(int index); } 2. The declaration of a variable of type L

where p is one of the 8 primitive types is possible. If the compiler encounters such a variable, it assumes that L

actually is equivalent to following 'extended' interface L'

{ void add(P element); void add(p element); P get(int index); p get(int index); } where 'P' is the wrapper type corresponding to 'p'. 3. It would still remain illegal to declare an interface such as the above 'by hand', since overriding by return type is illegal unless the return types are covariant. However, the VM can cope with such overrides; and the compiler will encounter no ambiguities, as paragraph 4. will show. 4. With the assumptions made above, the following code would compile to the most efficient bytecode, invoking the primitive versions of 'add' and 'get': L list; list.add(42); int value = list.get(0); 5. There has to be a means of instantiating objects of type L

. With the assumption made in 2., the solution is the same as with other abstract classes: A non-abstract class that implements L

for a primitive type p must implement those methods from the synthetic L'

that use the primitive type p, while the compiler will generate the necessary bridge methods with the wrapper type P: public class IntList implements L { public void add(int element) { // efficient int-based implementation } public int get(int index) { // efficient int-based implementation. } } 6. The types L

and L

will be assignment compatible in only one direction: It will be possible to assign an instance of type L to L, but not vice versa. L list1 = new IntList(); // OK L list2 = new ArrayList(); // Will not compile 7. If the assignment from L

to L

is forced through an unsafe cast, the attempt to invoke a method of L

using the primitive type p will result in a java.lang.NoSuchMethodException at runtime. This is comparable to the ClassCastException that can be thrown arbitrarily when unsafe casts are involved. L list = (List)(List)new ArrayList(); list.add(42); // Will throw NoSuchMethodException. From fweimer at bfk.de Thu Jul 8 01:46:45 2010 From: fweimer at bfk.de (Florian Weimer) Date: Thu, 08 Jul 2010 08:46:45 +0000 Subject: Transparancy In-Reply-To: (Pavel Minaev's message of "Wed\, 7 Jul 2010 12\:41\:14 -0700") References: <4C34C6BE.5010101@oracle.com> Message-ID: <82iq4qqs22.fsf@mid.bfk.de> * Pavel Minaev: > Indeed, can you give an example of a popular language which 1) has > explicit "return", 2) has lambdas, and either 3) does not allow for > "return" in a lambda, or 4) allows for it, but only in the meaning of > non-local return? Perl: sub foo (@) { map {return} @_; print "map did not return non-locally\n"; } print "calling foo\n"; foo; print "calling foo\n"; foo 1; Anonymous subs have proper returns, though. I can ask around if this is considered good language design. (I think the consensus with regard to Perl's non-closing behavior of named nested subprograms is that it's not a good idea at all.) By the way, all Perl subs have non-local break/continue (which are called last/next in Perl, in both local and non-local cases): sub my_break { last; } sub foo { while (1) { print "foo\n"; my_break; } } foo; However, this coding style is discouraged, and the interpreter will print a warning. I think Common Lisp has got non-local returns within lexical scope on downward closures (but Common Lisp was not conceived as a safe language, so the trade-offs are quite different). Obviously, FORTH has got some unsafe construct for that purpose, too. I also remember using something else which had a similar "return from" mechanism, but I don't recall which language that was (it certainly wasn't something I would consider popular, perhaps HP-48 User RPL or something like that). -- Florian Weimer BFK edv-consulting GmbH http://www.bfk.de/ Kriegsstra?e 100 tel: +49-721-96201-1 D-76133 Karlsruhe fax: +49-721-96201-99 From scolebourne at joda.org Thu Jul 8 02:31:00 2010 From: scolebourne at joda.org (Stephen Colebourne) Date: Thu, 8 Jul 2010 10:31:00 +0100 Subject: Type inference of parameters In-Reply-To: <4C350876.7060509@oracle.com> References: <4C350876.7060509@oracle.com> Message-ID: On 8 July 2010 00:06, maurizio cimadamore wrote: > I think an example of problematic overload resolution would be the > following: > ... > In the general case, it is not clear how the compiler should attribute the > lambda body when there is no unique target for the SAM conversion; one > option would be to do a trial-and-error attribution, as described in [1]. > Instead of solving a problem by introducing another one (as Josh said in a > separate post, Java method resolution is one of the most complicated parts > of the language), we decided to ban this kind of type-inference - eventually > the user will be able to disambiguate at the call site in a very java-ish > fashion: > > call(SAM1 { x -> System.out.println(x); }); I find the conservative approach to be reasonable here, as I don't think it will occur that often, and is simple to understand and explain. Were the inference to extend to counting the number of parameters in the lambda, I would also support that, but lub type calculations seem a step too far. BTW, I really don't like the new syntax suggestion, and I believe there were obvious ways to extend the previous syntax to support the newly required prefix type. Stephen From scolebourne at joda.org Thu Jul 8 02:32:54 2010 From: scolebourne at joda.org (Stephen Colebourne) Date: Thu, 8 Jul 2010 10:32:54 +0100 Subject: state-of-the-lambda syntax for non-SAM types such as WindowAdapter. In-Reply-To: References: Message-ID: On 8 July 2010 04:11, Reinier Zwitserloot wrote: > An idea: > > The state-of-the-lambda document allows lambda literals to be explicitly > typed: > > Comparator {a, b -> return a.length() - b.length()}; > > > What if one could also append the method to this? Then these closures can > extend to non-SAM types: > > frame.addWindowListener(WindowAdapter.windowClosed {e -> .... }); We had this in early FCM versions, but took it out as it complicates matters for a small use case. Stephen From scolebourne at joda.org Thu Jul 8 05:02:16 2010 From: scolebourne at joda.org (Stephen Colebourne) Date: Thu, 8 Jul 2010 13:02:16 +0100 Subject: Transparancy In-Reply-To: <4C34C6BE.5010101@oracle.com> References: <4C34C6BE.5010101@oracle.com> Message-ID: On 7 July 2010 19:26, Brian Goetz wrote: >> - this is still scoped to the lambda > > Yes, that's true. ?In fact, a better way to think about these lambda > expressions is that they are really "SAM expressions". ?They are often more > compact than CICE (since the target type can be inferred) but lambdas do not > (at this time) stand alone. Fine. My concern is conceptual integrity. Simplified SAMs has conceptual integrity, but requires local 'return' (not 'yield') and a syntax that actually looks like a simplified SAM. That means parameters in round brackets and no arrow symbol. ie. the strawman syntax was good. Its also vital for there to be a hook, something that the eye can easily see that is used to indicate where the 'return' will return to. Both CICE and FCM explored this area, with FCM treating 'this' lexically simply because this is a new language feature and it is what is most useful (SAM class lambdas are not especially useful and are a confusing distraction. A second conceptual integrity is "full closures", where 'return' is used for long-returns, 'this' is lexical, there might be no local-returns, and the syntax looks nothing whatsoever like a SAM. The BGGA proposals explored this area. While I see some manouvering to allow for this (or maybe just control abstraction) in the future, the latest proposal is insufficient to be useful for this in the future. Just scoping 'this' locally completely breaks any form of control abstraction, and the use of the latest syntax blocks a likely syntax for control abstraction. Your reply appears to indicate that the team's thinking is the former approach - simpler SAMs. If so, then this should be made crystal clear, and all design should focus on that space. Essentially I'm saying that the team should mostly ignore the control abstraction case - that should be thought of as a completely different language feature (implementable via another closure type, or macros, or only in a limited way via some other way such as a 'throws return' combined keyword). The only concern is not to use a syntax that would confuse in the future. Unfortunately, the current document follows much of simplified SAMs but inserts the weird 'yield' for no logical reason (given the arguments above) and uses a syntax that isn't appropriate for simplified SAMs. Once the team gets back to a core CICE-like worldview of simpler SAMs as a solid starting point, its then much easier to take a single opinion on the pros and cons of SAM classes and lexical 'this', and a sensible syntax tends to just drop out. > It is indeed *more* like CICE (and therefore an inner class), it is really > something in between. ?The quibbling over the semantics of return are a > reflection of trying to force lambdas to either be exactly like inner > classes or something completely new. ?The reality is that they are somewhere > in the middle. Unfortunately, I struggled to understand this part. Here's the question - In what way are they "something in the middle". Why are they /not/ just simplified SAMs? What else are you trying to achieve? (Its OK to have something else - like lexical 'this' ;-) > Using "return" now complicates our ability to provide > nonlocal control flow in the future. No it doesn't, as you've got non-lexical 'this'. And a new keyword 'yield' that is only usable in some scopes. Please stop trying to allow for the control flow feature in this way and focus on making the best simpler SAM solution! We can deal with control flow if and well it is felt to be a real issue (I suspect it will never come up). Stephen From howard.lovatt at gmail.com Thu Jul 8 05:47:36 2010 From: howard.lovatt at gmail.com (Howard Lovatt) Date: Thu, 8 Jul 2010 19:47:36 +0700 Subject: State of the Lambda Message-ID: @Brian, Really like the new draft. There are a few 'controversial' points, this is my 3 pennies worth: 1. Yield: stick with return, for long return use methodOrBlockName.return 2. Type parameter inference, great idea if it can be done (can it be extended to any overridden method in the future?) 3. Effectively final, I know you would like to do this for inner classes as well and would just like to encourage this further As I said, a real step forward. -- Howard. From reinier at zwitserloot.com Thu Jul 8 06:36:35 2010 From: reinier at zwitserloot.com (Reinier Zwitserloot) Date: Thu, 8 Jul 2010 15:36:35 +0200 Subject: state-of-the-lambda syntax for non-SAM types such as WindowAdapter. In-Reply-To: <201007080746.41341.peter.levart@gmail.com> References: <201007080746.41341.peter.levart@gmail.com> Message-ID: Heh, that's a nice trick. There's an alternative version for it that would work just as well: public interface WindowListener { public abstract class WindowClosed extends WindowAdapter { public abstract void windowClosed(WindowEvent e); } } which re-abstracts windowClosed, turning WindowListener.WindowClosed into a SAM: window.addWindowListener(WindowListener.WindowClosed {e -> ...}); between that and your example, this use case is served acceptably. --Reinier Zwitserloot On Thu, Jul 8, 2010 at 7:46 AM, Peter Levart wrote: > On Thursday, July 08, 2010 05:11:39 am Reinier Zwitserloot wrote: > > An idea: > > > > The state-of-the-lambda document allows lambda literals to be explicitly > > typed: > > > > Comparator {a, b -> return a.length() - b.length()}; > > > > > > What if one could also append the method to this? Then these closures can > > extend to non-SAM types: > > > > frame.addWindowListener(WindowAdapter.windowClosed {e -> .... }); > > > > Just a thought. > > > > --Reinier Zwitserloot > > You don't need special syntax for that. For example: > > public class WindowAdapters > { > interface Fn > { > void invoke(WindowEvent e); > } > > public static WindowAdapter windowOpened(final Fn fn) > { > return new WindowAdapter() > { > @Override > public void windowOpened(WindowEvent e) > { > fn.invoke(e); > } > }; > } > > public static WindowAdapter windowClosed(final Fn fn) > { > return new WindowAdapter() > { > @Override > public void windowClosed(WindowEvent e) > { > fn.invoke(e); > } > }; > } > > // etc... > } > > > // and with an additional pair of parens you can write: > > frame.addWindowListener(WindowAdapters.windowClosed({e -> .... })); > > > Regards, Peter > From reinier at zwitserloot.com Thu Jul 8 06:48:50 2010 From: reinier at zwitserloot.com (Reinier Zwitserloot) Date: Thu, 8 Jul 2010 15:48:50 +0200 Subject: Primitives in Generics In-Reply-To: References: Message-ID: It's quite relevant, because state-of-the-lambda suffers from a rather serious problem, which is highlighted by a core use-case driving project lambda: ParallelArray's Ops container with its 132 SAM types. This proposal would solve the issue and reduce that set of 132 to 8. If a JVM optimization can be proven to be trivial to implement (JRockit), and fail to apply only in those cases where specialization wouldn't work either, then I don't see how relying on this optimization is not sufficient. Your idea is an intriguing one which would indeed drop the need for the JRockit boxing elimination optimization, though I don't think it can work. For example, the following 2 code samples would seem to be perfectly valid, but as far as I can tell the end result will be a NoSuchMethodError because the produced instance will not actually have the primitive version of the method. How will the compiler stop you from doing this, and, even if it does, how is a java programmer to know these snippets won't work? Snippet #1: new ArrayList().add(10); and to show this problem cannot be remedied just by detecting such situations and disallowing them: Snippet #2: public static List createList(Class type) { return new ArrayList(); } public void test() { List list = createList(int.class); list.add(10); } --Reinier Zwitserloot On Thu, Jul 8, 2010 at 9:42 AM, Gernot Neppert wrote: > Reinier, I think your proposal is propably off-topic with regards to > this mailing list, nevertheless I could not resist to respond ;-) > > I like the idea of making efficient use of generic classes with > primitive type parameters possible. However, I'm worried that solely > relying on a JVM optimization will not be sufficient. Wouldn't it be > possible to gain efficiency on every VM if something along the > following could be adopted: > > 1. Let's assume the existence of a generic interface > > L { > void add(T element); > T get(int index); > } > > 2. The declaration of a variable of type L

where p is one of the 8 > primitive types is possible. > If the compiler encounters such a variable, it assumes that L

> actually is equivalent to following 'extended' interface > > L'

> { > void add(P element); > void add(p element); > P get(int index); > p get(int index); > } > > where 'P' is the wrapper type corresponding to 'p'. > > 3. It would still remain illegal to declare an interface such as the > above 'by hand', since overriding by return type is illegal unless the > return types are covariant. However, the VM can cope with such > overrides; and the compiler will encounter no ambiguities, as > paragraph 4. will show. > > 4. With the assumptions made above, the following code would compile > to the most efficient bytecode, invoking the primitive versions of > 'add' and 'get': > > L list; > list.add(42); > int value = list.get(0); > > > 5. There has to be a means of instantiating objects of type L

. With > the assumption made in 2., the solution is the same as with other > abstract classes: > > A non-abstract class that implements L

for a primitive type p must > implement those methods from the synthetic L'

that use the > primitive type p, while the compiler will generate the necessary > bridge methods with the wrapper type P: > > public class IntList implements L > { > public void add(int element) > { > // efficient int-based implementation > } > > > public int get(int index) > { > // efficient int-based implementation. > } > > } > > 6. The types L

and L

will be assignment compatible in only one > direction: It will be possible to assign an instance of type L to > L, but not vice versa. > > L list1 = new IntList(); // OK > L list2 = new ArrayList(); // Will not compile > > 7. If the assignment from L

to L

is forced through an unsafe > cast, the attempt to invoke a method of L

using the primitive type > p will result in a java.lang.NoSuchMethodException at runtime. This is > comparable to the ClassCastException that can be thrown arbitrarily > when unsafe casts are involved. > > L list = (List)(List)new ArrayList(); > list.add(42); // Will throw NoSuchMethodException. > From neal at gafter.com Thu Jul 8 08:09:39 2010 From: neal at gafter.com (Neal Gafter) Date: Thu, 8 Jul 2010 08:09:39 -0700 Subject: Removal of function types In-Reply-To: References: <4C350842.1050800@cs.oswego.edu> <7FDA6630E1822F448C97A48D5D733094FB7A93@EXVMSTOR302.intra.rakuten.co.jp> <7B695633-97E6-4790-B44E-9AC3F3491709@gafter.com> Message-ID: On Wed, Jul 7, 2010 at 9:40 PM, Pavel Minaev wrote: > I think the meaning is something along the lines of C++ template > parameter packs & their expansions in various context > If we're talking about turning generics into macro-expanded templates, that sounds like a huge change. From neal at gafter.com Thu Jul 8 08:13:55 2010 From: neal at gafter.com (Neal Gafter) Date: Thu, 8 Jul 2010 08:13:55 -0700 Subject: Type inference of parameters In-Reply-To: <4C350876.7060509@oracle.com> References: <4C350876.7060509@oracle.com> Message-ID: On Wed, Jul 7, 2010 at 4:06 PM, maurizio cimadamore < maurizio.cimadamore at oracle.com> wrote: > In the general case, it is not clear how the compiler should attribute > the lambda body when there is no unique target for the SAM conversion; > one option would be to do a trial-and-error attribution, as described in > [1]. Instead of solving a problem by introducing another one (as Josh > said in a separate post, Java method resolution is one of the most > complicated parts of the language), we decided to ban this kind of > type-inference - eventually the user will be able to disambiguate at the > call site in a very java-ish fashion: > > call(SAM1 { x -> System.out.println(x); }); > Solving the compiler-writer's problem by pushing the problem onto the programmer is tried and true, but exactly backwards. The point of a language and compiler is to simplify the programmer's life, at the expense of complicating the compiler-writer's job[CWFEA86]. -Neal [CWFEA86] The Compiler-Writer's Full Employment Act of 1986. From leonardoavs at gmail.com Thu Jul 8 10:04:15 2010 From: leonardoavs at gmail.com (Leonardo Vargas Sandoval) Date: Thu, 8 Jul 2010 17:04:15 +0000 Subject: Removal of function types In-Reply-To: References: Message-ID: No me gusta que los remuevan solo por discusiones que no tienen sentido y nunca llegan a un fin es un error por que esto provoca que las APIs tengan docenas de interfaces o quien sabe que para hacer un clousure para mi esta es la direcci?n incorrecta as? como sucedi? con los generics. Pero esta vez la decisi?n equivocada sale de que nadie se pone de acuerdo en cuanto las desiciones. Yo prefiero function types que tener unt ipo por cada function type que tenga y mas a un si necesito hacer una interfaz por function type, y por ultimo lo siguiente son simples escusas: - Removing them reduces the scope, thus increasing the delivery likelihood. *(Pongan m?s programadores)* - Function types look pig ugly with exception transparancy *(Tomen la desici?n no basado en lo que se piensa personalmente si no en lo que se debe de hacer)* - Function types are a very different concept to the rest of Java, which is very name based *(Esto es una escusa sinceramente yo he trabajado toda mi vida con Java y no veo por que no incluirlos solo por que Java es "name based" como en punto net todo es un cambio y que unas librer?as utilicen functions types y otras interfaces es un problema de arquitectura del software no del compilador )* - Function type don't visually look much like types (which are names everywhere else). *Eso es lo bueno no son tipos como tal lo que puede reducir que no tengo que hacer una interfaz en un archivo .java que sea publica para poder utilizar la funcion que se debe de ejecutar. * - SAMs centralise the definition of Javadoc and give a meaningful name. *Es un punto pero yo digo por que no podemos documentar los funtions types cuando sea necesario. Osea otro argumento que no me gusta. * - function types don't - Function types were causing trouble when in a list/array due to generic erasure.* Tomen una desici?n*. - Removing them removes some of the most contentious syntax debates. *Ponganse de acuerdo y dejen los pensamientos personales y creencias y busquen lo mejor para el lenguaje dejar functions types por fuera por solo esto me suena a que no se esta pensando bien. * - Function types can be added later if the right approach is taken (as in the doc) of declaring lambda expressions to have no expressible type. *As? como la reified generics nunca llego los functions types no llegaran si no se hacen ahora. * 2010/7/7 Stephen Colebourne > The latest (6 July) lambda document removes function types. I support > this (even though they were in FCM). > > - Removing them reduces the scope, thus increasing the delivery likelihood > - Function types look pig ugly with exception transparancy > - Function types are a very different concept to the rest of Java, > which is very name based > - Function type don't visually look much like types (which are names > everywhere else) > - SAMs centralise the definition of Javadoc and give a meaningful name > - function types don't > - Function types were causing trouble when in a list/array due to > generic erasure > - Removing them removes some of the most contentious syntax debates > - Function types can be added later if the right approach is taken (as > in the doc) of declaring lambda expressions to have no expressible > type. > > I suggest using this thread to comment on the removal of function types :-) > > Stephen > > -- Leonardo Andr?s Vargas Sandoval. From brian.goetz at oracle.com Thu Jul 8 10:20:14 2010 From: brian.goetz at oracle.com (Brian Goetz) Date: Thu, 08 Jul 2010 13:20:14 -0400 Subject: Transparancy In-Reply-To: References: <4C34C6BE.5010101@oracle.com> Message-ID: <4C3608CE.1080909@oracle.com> > Most developers seem to think that a lambda body is comparable to a SAM/AIC implementation body, which is a method. This is true, but only because they have no other experience. The reality is that they are mostly like method bodies, but not entirely like method bodies. Such first-order approximations are very useful in forming mental models but should not be taken too far! Here are some contexts in Java-like languages where a control flow statement stops executing in the current context and resumes in another: A Return from a method. B Local return from a closure. C Lexical return from a closure. D Incremental production from a coroutine. E Terminal production from a coroutine. F Value production from a local block (i.e.: var x = { var y = 3; yield 3*y } G Lexical return from a local block. Java currently only has A. So our thinking about "what return means" is heavily conditioned by our experience with A. BGGA has both B and C. The current SotL draft has "yield" for B and currently nothing for "C". I think we can all agree that "return" is absurd for "F". (You might not like "yield" because it is used for something else (D/E) in C# and yet something else (and weird!) in Ruby, but "yield" is a fine candidate for F.) So "return" is obviously right for A and obviously wrong for F. B and C have something in common with methods, and something in common with local blocks. We're letting the fact that Java has methods and does not have local blocks color our thinking that closures are "just like" methods when in fact they are "mostly like" methods. (We can disagree about whether it's 98/2 or 90/10, but it's obviously not 100/0.) Imagine you grew up in a world where the only animals were dogs (methods). Your mental model of animal would be heavily influenced by the characteristics of dogs -- for example, animals have four legs*. Then one day ants (local blocks) were introduced into your planet. They are obviously not dogs, but they have more legs than you and are shorter than you, so they are more like dogs than like humans, so we redefine the world as "there are people and animals; animals are dogs and ants." Having thus classified things, you are again comfortable with the world. Two hundred years later, a cat (closure) arrives on your planet. Most people in this situation will think "what a fancy dog!" The point is that "return" is only obvious for B if you have grown up in the world of A-but-not-F. And if we're sure that C/D/E/F/G will *never* be added to Java, then we are perfectly justified in ignoring all the cat-like aspects of closures and lump them in with dogs, and choose "return" for B and never think about it again. But if there's any reasonable chance that F (or even C) will ever become part of Java, then all of a sudden these dogs that say "meow" start to seem really weird. That said, this forum is for lambda discussions, so this is probably not the place for a discussion about which of these forms Java should have (beyond A/B/C). However, considerations of where Java might or might not go in future releases that do not even have numbers yet do (and should!) inform our thoughts. *Yes, everyone has a three-legged dog story. From mcnepp02 at googlemail.com Thu Jul 8 11:08:59 2010 From: mcnepp02 at googlemail.com (Gernot Neppert) Date: Thu, 8 Jul 2010 20:08:59 +0200 Subject: Primitives in Generics In-Reply-To: References: Message-ID: 2010/7/8 Reinier Zwitserloot : > Your idea is an intriguing one which would indeed drop the need for the > JRockit boxing elimination optimization, though?I don't think it can work. > For example, the following 2 code samples would seem to be perfectly valid, > but as far as I can tell the end result will be a NoSuchMethodError because > the produced instance will not actually have the primitive version of the > method. How will the compiler stop you from doing this, and, even if it > does, how is a java programmer to know these snippets won't work? > Snippet #1:?new ArrayList().add(10); This would not compile according to my proposal. ArrayList does not implement the int-based versions of the generic methods. According to the proposal (paragraph 5.), the only way of implementing a non-abstract class for a primitive-typed generic baseclass or interface would be to supply all primitive-typed functions manually. The compiler would generate the necessary bridge methods as 'synthetic' methods. This means that you could not do this: public class IntList extends AbstractList { public int get(int index) { // efficient int-based implementation } } because the method public Integer get(int index) has already been defined explicitly in AbstractList and you could not overload it differing in return type only. But the compiler can - thererfore you'd have to do: public class IntList implements List { public boolean add(int element) { // efficient int-based implementation } } in which case the compiler would generate the synthetic method Integer get(int index). A little more work to do, but asmall price to pay, I'd say! From bobfoster at gmail.com Thu Jul 8 11:17:05 2010 From: bobfoster at gmail.com (Bob Foster) Date: Thu, 8 Jul 2010 11:17:05 -0700 Subject: State of the Lambda Message-ID: Howard Lovett wrote: > 1. Yield: stick with return, for long return use methodOrBlockName.return Yes, plain 'return', 'break' and 'continue' should have the same meaning they would in a SAM implemented in an inner class. Scala got 'return' wrong. Bob From forax at univ-mlv.fr Thu Jul 8 11:20:09 2010 From: forax at univ-mlv.fr (=?ISO-8859-1?Q?R=E9mi_Forax?=) Date: Thu, 08 Jul 2010 20:20:09 +0200 Subject: Removal of function types In-Reply-To: <4C34C460.800@oracle.com> References: <4C34C460.800@oracle.com> Message-ID: <4C3616D9.2080902@univ-mlv.fr> ahhhh, you kill my lambda :) Le 07/07/2010 20:16, Brian Goetz a ?crit : >> The latest (6 July) lambda document removes function types. I support >> this (even though they were in FCM). >> > Indeed it does. People will ask why, so let me share my thinking here. Among > other reasons: > > 1. There are two basic approaches to typing: nominal and structural. The > identity of a nominal is based on its name; the identity of a structural type > is based on what it is composed of (such as "tuple of int, int" or "function > from int to float".) > > Most languages pick mostly nominal or mostly structural; there are not a lot > of languages that successfully mix nominal and structural typing except > "around the edges." Java is almost entirely nominal (with a few exceptions: > arrays are a structural type, but at the bottom there is always a nominal > element type; generics have a mix of nominal and structural too, and this is > in fact part of the source of many of people's complaints about generics.) > They mostly complain about generics because there is no reification (see your point 3) and because they doesn't understand where to put ? super/extends. Function types have several advantages over SAMs. (remember that to be useful SAMs must be generified, see my point below). - no need to be have to understand long error message because you forget a ? extends or a ? super somewhere. - no need to box/unbox things. - no need to find a creative name for a god damn function. > Grafting a structural type system (function types) onto Java's nominal type > system means new complexity and edge cases. Is the benefit of function types > worth this? > yes. > 2. We've been using SAM types for years, and Java developers are comfortable > with them. Java developers are perhaps comfortable with SAMs but the main drawback is that there is lot of SAMs that represent exactly same thing just because you have to give it a name. By example, you can count the number of filter/predicate SAM you have in the JDK: javax.swing.RowFilter, java.nio.file.DirectoryStream.Filter, javax.lang.model.util.ElementFilter, javax.stream.EventFilter, java.io.FileFilter, javax.swing.filechooser.FileFilter, javax.swing.filechooser.FileNameExtensionFilter, java.io.FilenameFilter, java.util.logging.Filter, javax.imageio.spi.ServiceRegistryFilter, javax.xml.stream.StreamFilter, javax.sql.rowset.Predicate. There is no interopt between them, no way to create a filter that will returns true if one of the two filters taken as argument is true (the generic orFilter) without having to write adapters, proxies or decorators. Function types are important because they allow to write combinators and to reuse them at different places of a program. > If we added function types, it would immediately bifurcate the > libraries (both JDK and external) into "old style" libraries (those that use > SAM types) and "new style" libraries (those that use function types.) That's > a big ugly seam in the middle of the platform. > JDK already contains bifurcations, it has more than 10 ways to represent a function that takes an object and returns a boolean. > 3. No reification. There was a long thread about how useful it would be for > function types to be reified. Without reification, function types are hobbled. > Function type can't be reified like SAMs can't be reified because argument of type variable aren't available at runtime. That's it. R?mi From int19h at gmail.com Thu Jul 8 11:28:59 2010 From: int19h at gmail.com (Pavel Minaev) Date: Thu, 8 Jul 2010 11:28:59 -0700 Subject: Removal of function types In-Reply-To: <4C3616D9.2080902@univ-mlv.fr> References: <4C34C460.800@oracle.com> <4C3616D9.2080902@univ-mlv.fr> Message-ID: On Thu, Jul 8, 2010 at 11:20 AM, R?mi Forax wrote: > Java developers are perhaps comfortable with SAMs but > the main drawback is that there is lot of SAMs that represent > exactly same thing just because you have to give it a name. > > By example, you can count the number of filter/predicate SAM > you have in the JDK: > javax.swing.RowFilter, > java.nio.file.DirectoryStream.Filter, > javax.lang.model.util.ElementFilter, > javax.stream.EventFilter, > java.io.FileFilter, > javax.swing.filechooser.FileFilter, > javax.swing.filechooser.FileNameExtensionFilter, > java.io.FilenameFilter, > java.util.logging.Filter, > javax.imageio.spi.ServiceRegistryFilter, > javax.xml.stream.StreamFilter, > javax.sql.rowset.Predicate. This situation is similar to that in C#, which also has nominal function types only, and there was a proliferation of identical types throughout the class library. The pragmatical solution to this was to provide a set of generic types such as Func<...> - a pure library solution, very easy to implement, and yet it was surprisingly effective for all the inelegance. Java doesn't get off the hook that easy because its generics don't play well with primitives; but then the set of primitives is non-extensible. So, even though you need many more definitions to cover all (or even reasonable - e.g. no short, just int/long) ground, it's a work that can be done once, with code automatically generated for as many arguments as deemed reasonable, and placed into the standard library for everyone to reuse. Yes, it's even less elegant than the .NET solution, but it's still a workable one, and doesn't require any changes from the language aside from those already in the proposal. So perhaps it's just good enough? From brian.goetz at oracle.com Thu Jul 8 12:02:47 2010 From: brian.goetz at oracle.com (Brian Goetz) Date: Thu, 08 Jul 2010 15:02:47 -0400 Subject: State of the Lambda In-Reply-To: References: <4C34AC04.9090109@oracle.com> Message-ID: <4C3620D7.3020000@oracle.com> > The syntax for specifying the target type in a method invocation context > seems awfully confusing. The draft says: > > executor.submit(Callable { -> "foo"} ); > > That looks a heck of a lot like a cast, except it's missing the parens. > People will therefore tend to put them in by accident: > > executor.submit((Callable) { -> "foo"} ); Indeed, the (CICE-inspired) approach of "stick the type in front of the block" looks an awful lot like a cast. We went back and forth a few times on this. The main reasons in favor of this approach are: - We have left undefined the type of a naked lambda expression, because it is target-typed. Most of the JLS describes casting rules in terms of from-A-to-B. If we don't know the type of A, it is not really a cast. - If we intend to support abstract classes as SAM types (and we do) and we intend to support use of non-default-constructor (which we might), there needs to be a place to put the constructor arguments. One choice is to punt and say "use anonymous classes there." Another is to do something really CICE-like: executor.submit(new Callable(args) { -> "foo"} ); ("new" is optional here; everyone will have opinions.) A cast does not address this problem. From reinier at zwitserloot.com Thu Jul 8 12:04:40 2010 From: reinier at zwitserloot.com (Reinier Zwitserloot) Date: Thu, 8 Jul 2010 21:04:40 +0200 Subject: Removal of function types In-Reply-To: <4C3616D9.2080902@univ-mlv.fr> References: <4C34C460.800@oracle.com> <4C3616D9.2080902@univ-mlv.fr> Message-ID: On Thu, Jul 8, 2010 at 8:20 PM, R?mi Forax wrote: > Function types have several advantages over SAMs. > (remember that to be useful SAMs must be generified, see my point below). > - no need to be have to understand long error message because > you forget a ? extends or a ? super somewhere. > I do not understand this complaint. type parameters are as much a part of the abandoned function type concept as they are for SAM types. In fact, function types use _MORE_ generics than SAM types; #int(String, String) is translated to FunctionIOO internally, which is extra generics. This implementation detail leaks, for example because an array of #int(String, String) types doesn't work very well, whereas an array of a SAM type will work just fine. I expect that most SAM types will have some type parameters of their own making this point moot, but if either of these two proposals needs to be faulted for causing confusion based around generics, shouldn't it be function types? > - no need to box/unbox things. > SAM fitting isn't "boxing" or "unboxing". Specifically, any lambda literal is not first treated as an unnamed/untyped function, then boxed into the appropriate SAM depending on how you use it. Instead, the use site of the lambda literal is inspected to figure out what is required, and the lambda literal is then read by the compiler with this target type in mind. That's entirely different from boxing/unboxing. The old function type based proposal that had the ability to automatically box them into SAM types was appropriate usage of the 'box' term I'd say, but this is quite different. For example, by figuring out the target SAM type _before_ attempting to understand the signature of the lambda literal, parameter types become optional. > - no need to find a creative name for a god damn function. > Not sure if the hyperbole is warranted here. Names are a good thing far more often than they are a bad thing. At least, it seems to be that way if I look at Parallel Arrays, where, if we assume for a moment the primitives-and-generics issue is fixed, there are 9 named SAM types, which each can carry useful javadoc: Op, BinaryOp, Reducer, Generator, Procedure, Comparator, Predicate, BinaryPredicate, and Action. I admit there are rare situations where naming the SAM type is superfluous, but even in this situations, it's only a burden to the library developer, not the user of the library. The word "Comparator" can't be found anywhere in the following snippet: Collections.sort(stringList, {x, y -> x.length() - y.length()}); > By example, you can count the number of filter/predicate SAM > you have in the JDK: > And none of those can go away; java must remain backwards compatible. It's absolutely true that these are a problem, but I don't see how you can fault the SAM based proposal for these any more than you can the strawman proposal. It was never the intention in the strawman proposal that one could treat a javax.swing.RowFilter as a java.io.FileFilter. A #boolean(T) might box to either depending on the circumstance, but I don't see how that solves the problem of so many copies of filter. Perhaps you meant that forcing to SAM type results in further duplicates in the future. Maybe. But I can't think any other examples of duplication other than Filter and Runnable / Executable. Once predicates and ops arrive in rt.jar via either Parallel Arrays or new filter and map methods for collections, I'd expect no further duplication. The same argument can also be used to call a Camera and a Gun not being interoperable as problematic, as they both have a shoot() method. It may be true that in practice Camera/Gun occurs less often than 2 incompatible classes whose structure nevertheless does match, which do represent the same semantical concept, but that's not how java works today, and I rather doubt turning java into a structurally typed language is in scope for project lambda, let alone desired. > > JDK already contains bifurcations, it has more than 10 ways to represent > a function that takes an object and returns a boolean. > Yes. Let's not make it worse. > Function type can't be reified like SAMs can't be reified because > argument of type variable aren't available at runtime. > That's it. > SAMs are already reified. I'm not sure what you mean when you say they cannot be: Runnable r = {-> System.out.println("Hello, World!");} one can introspect at runtime that r's type is in fact "Runnable". --Reinier Zwitserloot From reinier at zwitserloot.com Thu Jul 8 12:06:53 2010 From: reinier at zwitserloot.com (Reinier Zwitserloot) Date: Thu, 8 Jul 2010 21:06:53 +0200 Subject: Primitives in Generics In-Reply-To: References: Message-ID: You didn't mention my second snippet, which is a shame, because I included it because that's the one I'm confused about - how could you possibly write a compiler that detects there's a problem there? --Reinier Zwitserloot On Thu, Jul 8, 2010 at 8:08 PM, Gernot Neppert wrote: > 2010/7/8 Reinier Zwitserloot : > > Your idea is an intriguing one which would indeed drop the need for the > > JRockit boxing elimination optimization, though I don't think it can > work. > > For example, the following 2 code samples would seem to be perfectly > valid, > > but as far as I can tell the end result will be a NoSuchMethodError > because > > the produced instance will not actually have the primitive version of the > > method. How will the compiler stop you from doing this, and, even if it > > does, how is a java programmer to know these snippets won't work? > > > Snippet #1: new ArrayList().add(10); > > This would not compile according to my proposal. ArrayList does not > implement the int-based versions of the generic methods. > According to the proposal (paragraph 5.), the only way of implementing > a non-abstract class for a primitive-typed generic baseclass or > interface would be to supply all primitive-typed functions manually. > The compiler would generate the necessary bridge methods as > 'synthetic' methods. > This means that you could not do this: > > public class IntList extends AbstractList > { > public int get(int index) > { > // efficient int-based implementation > } > } > > because the method public Integer get(int index) has already been > defined explicitly in AbstractList and you could not overload it > differing in return type only. But the compiler can - thererfore you'd > have to do: > > public class IntList implements List > { > public boolean add(int element) > { > // efficient int-based implementation > } > } > > in which case the compiler would generate the synthetic method Integer > get(int index). > A little more work to do, but asmall price to pay, I'd say! > From jjb at google.com Thu Jul 8 12:25:02 2010 From: jjb at google.com (Joshua Bloch) Date: Thu, 8 Jul 2010 12:25:02 -0700 Subject: State of the Lambda In-Reply-To: <4C3620D7.3020000@oracle.com> References: <4C34AC04.9090109@oracle.com> <4C3620D7.3020000@oracle.com> Message-ID: Brian, If the cast is illegal in that position, rather than subtly different, then I'm much happier. Josh On Thu, Jul 8, 2010 at 12:02 PM, Brian Goetz wrote: > The syntax for specifying the target type in a method invocation context >> seems awfully confusing. The draft says: >> >> executor.submit(Callable { -> "foo"} ); >> >> That looks a heck of a lot like a cast, except it's missing the parens. >> People will therefore tend to put them in by accident: >> >> executor.submit((Callable) { -> "foo"} ); >> > > Indeed, the (CICE-inspired) approach of "stick the type in front of the > block" looks an awful lot like a cast. We went back and forth a few times > on this. The main reasons in favor of this approach are: > > - We have left undefined the type of a naked lambda expression, because it > is target-typed. Most of the JLS describes casting rules in terms of > from-A-to-B. If we don't know the type of A, it is not really a cast. > > - If we intend to support abstract classes as SAM types (and we do) and we > intend to support use of non-default-constructor (which we might), there > needs to be a place to put the constructor arguments. One choice is to punt > and say "use anonymous classes there." Another is to do something really > CICE-like: > > executor.submit(new Callable(args) { -> "foo"} ); > > ("new" is optional here; everyone will have opinions.) > > A cast does not address this problem. > > From john at milsson.nu Thu Jul 8 12:39:11 2010 From: john at milsson.nu (John Nilsson) Date: Thu, 8 Jul 2010 21:39:11 +0200 Subject: State of the Lambda In-Reply-To: <4C34AC04.9090109@oracle.com> References: <4C34AC04.9090109@oracle.com> Message-ID: All in all I like the approach you are taking, do agree with the -1 on yield though. One question wrt method references: do you intend to support "unbound" references to to instance methods? The use case being a shorter syntax to define expression lists in query like expression. I.e. With the current proposal I guess I could have an expression like this: select({ p -> p.getName() }, { p -> p.getAge() }).from(persons) A use case I imagine won't be to uncommon. F.ex. in our system we have resorted to strings and reflection to select properties from objects. select("name","age").from(persons) By "unbound" method reference I thus mean any syntactic sugar that shortens the syntactic distance between the two expressions. BR, John From brian.goetz at oracle.com Thu Jul 8 12:44:12 2010 From: brian.goetz at oracle.com (Brian Goetz) Date: Thu, 08 Jul 2010 15:44:12 -0400 Subject: State of the Lambda In-Reply-To: References: <4C34AC04.9090109@oracle.com> Message-ID: <4C362A8C.4070902@oracle.com> > One question wrt method references: do you intend to support "unbound" > references to to instance methods? The use case being a shorter syntax > to define expression lists in query like expression. I.e. With the > current proposal I guess I could have an expression like this: > > select({ p -> p.getName() }, { p -> p.getAge() }).from(persons) > > A use case I imagine won't be to uncommon. F.ex. in our system we have > resorted to strings and reflection to select properties from objects. > > select("name","age").from(persons) > > By "unbound" method reference I thus mean any syntactic sugar that > shortens the syntactic distance between the two expressions. What I think you are asking for is something like "field references" to go along with "method references". I can definitely see the utility of such a thing. We did indeed consider this when we considered method references, but chose to leave it out at this time and revisit in the future. From forax at univ-mlv.fr Thu Jul 8 13:52:52 2010 From: forax at univ-mlv.fr (=?UTF-8?B?UsOpbWkgRm9yYXg=?=) Date: Thu, 08 Jul 2010 22:52:52 +0200 Subject: Removal of function types In-Reply-To: References: <4C34C460.800@oracle.com> <4C3616D9.2080902@univ-mlv.fr> Message-ID: <4C363AA4.1040608@univ-mlv.fr> Le 08/07/2010 21:04, Reinier Zwitserloot a ?crit : > On Thu, Jul 8, 2010 at 8:20 PM, R?mi Forax > wrote: > > Function types have several advantages over SAMs. > (remember that to be useful SAMs must be generified, see my point > below). > - no need to be have to understand long error message because > you forget a ? extends or a ? super somewhere. > > > > I do not understand this complaint. type parameters are as much a part > of the abandoned function type concept as they are for SAM types. In > fact, function types use _MORE_ generics than SAM types; #int(String, > String) is translated to FunctionIOO internally, which > is extra generics. This implementation detail leaks, for example > because an array of #int(String, String) types doesn't work very well, > whereas an array of a SAM type will work just fine. I expect that most > SAM types will have some type parameters of their own making this > point moot, but if either of these two proposals needs to be faulted > for causing confusion based around generics, shouldn't it be function > types? I mean if function types are erased to java.dyn.MethodHandle. > - no need to box/unbox things. > > > SAM fitting isn't "boxing" or "unboxing". Specifically, any lambda > literal is not first treated as an unnamed/untyped function, then > boxed into the appropriate SAM depending on how you use it. Instead, > the use site of the lambda literal is inspected to figure out what is > required, and the lambda literal is then read by the compiler with > this target type in mind. That's entirely different from > boxing/unboxing. The old function type based proposal that had the > ability to automatically box them into SAM types was appropriate usage > of the 'box' term I'd say, but this is quite different. For example, > by figuring out the target SAM type _before_ attempting to understand > the signature of the lambda literal, parameter types become optional. > > - no need to find a creative name for a god damn function. > > > Not sure if the hyperbole is warranted here. Names are a good thing > far more often than they are a bad thing. At least, it seems to be > that way if I look at Parallel Arrays, where, if we assume for a > moment the primitives-and-generics issue is fixed, there are 9 named > SAM types, which each can carry useful javadoc: Op, BinaryOp, Reducer, > Generator, Procedure, Comparator, Predicate, BinaryPredicate, and Action. > > I admit there are rare situations where naming the SAM type is > superfluous, but even in this situations, it's only a burden to the > library developer, not the user of the library. The word "Comparator" > can't be found anywhere in the following snippet: > > Collections.sort(stringList, {x, y -> x.length() - y.length()}); I don't follow you here. Usually, we use type at declaration site. > > > > By example, you can count the number of filter/predicate SAM > you have in the JDK: > > > And none of those can go away; java must remain backwards compatible. > It's absolutely true that these are a problem, but I don't see how you > can fault the SAM based proposal for these any more than you can the > strawman proposal. It was never the intention in the strawman proposal > that one could treat a javax.swing.RowFilter as a java.io.FileFilter. > A #boolean(T) might box to either depending on the circumstance, but I > don't see how that solves the problem of so many copies of filter. My point is because we don't have function type, we (all of us) create SAMs that are incompatible each others. So we can't write reusable high order methods. > > Perhaps you meant that forcing to SAM type results in further > duplicates in the future. Maybe. But I can't think any other examples > of duplication other than Filter and Runnable / Executable. Once > predicates and ops arrive in rt.jar via either Parallel Arrays or new > filter and map methods for collections, I'd expect no further duplication. > > The same argument can also be used to call a Camera and a Gun not > being interoperable as problematic, as they both have a shoot() > method. It may be true that in practice Camera/Gun occurs less often > than 2 incompatible classes whose structure nevertheless does match, > which do represent the same semantical concept, but that's not how > java works today, and I rather doubt turning java into a structurally > typed language is in scope for project lambda, let alone desired. > > > JDK already contains bifurcations, it has more than 10 ways to > represent > a function that takes an object and returns a boolean. > > > Yes. Let's not make it worse. > > > Function type can't be reified like SAMs can't be reified because > argument of type variable aren't available at runtime. > That's it. > > > SAMs are already reified. I'm not sure what you mean when you say they > cannot be: > > Runnable r = {-> System.out.println("Hello, World!");} > > one can introspect at runtime that r's type is in fact "Runnable". I think you have written something stupid :) > > --Reinier Zwitserloot > R?mi From brian.goetz at oracle.com Thu Jul 8 14:37:22 2010 From: brian.goetz at oracle.com (Brian Goetz) Date: Thu, 08 Jul 2010 17:37:22 -0400 Subject: State of the Lambda In-Reply-To: <232B8A26-69CE-4BCC-B545-F654BFDBA486@oracle.com> References: <4C34AC04.9090109@oracle.com> <232B8A26-69CE-4BCC-B545-F654BFDBA486@oracle.com> Message-ID: <4C364512.1050703@oracle.com> >> I'm a little bit worried that method references with receiver variables do >> an copy. ... A possible solution is to require such receiver >> variables to be (implicitly) final. > > This is one of those times where syntax might imply something deep about an expression. Brian's sketch uses a prefix "#" to introduce a method lifted to a function (and perhaps partially applied to a receiver). > > The prefix looks like it might govern the evaluation of the expression to the right. An infix operator would much more clearly express that the left hand side of the expression (at least) is evaluated in the normal left-to-right order of Java expressions. (Some infix operators, ?:&& || have special rules for evaluation order, but it only applies to the right of the operator.) My intention for method refs for non-static methods is that the receiver reference is evaluated and copied (likely as part of the method handle binding); this is a binding of a method handle with a specific receiver, not syntactic sugar for "whatever the receiver happens to be at the time the lambda is invoked, call it." From brian.goetz at oracle.com Thu Jul 8 14:42:52 2010 From: brian.goetz at oracle.com (Brian Goetz) Date: Thu, 08 Jul 2010 17:42:52 -0400 Subject: Primitives in Generics In-Reply-To: References: Message-ID: <4C36465C.6090809@oracle.com> Its worth noting that ParallelArray is really the *only* well-known bad case here. Given the invasiveness to the language that function types represent, it seems a bad trade to take on such a big solution for a single ill-behaved library that some mad professor cooked up :) :) :) ^ Lots of smileys here! On 7/8/2010 9:48 AM, Reinier Zwitserloot wrote: > It's quite relevant, because state-of-the-lambda suffers from a rather > serious problem, which is highlighted by a core use-case driving project > lambda: ParallelArray's Ops container with its 132 SAM types. This proposal > would solve the issue and reduce that set of 132 to 8. > > If a JVM optimization can be proven to be trivial to implement (JRockit), > and fail to apply only in those cases where specialization wouldn't work > either, then I don't see how relying on this optimization is not sufficient. > > Your idea is an intriguing one which would indeed drop the need for the > JRockit boxing elimination optimization, though I don't think it can work. > For example, the following 2 code samples would seem to be perfectly valid, > but as far as I can tell the end result will be a NoSuchMethodError because > the produced instance will not actually have the primitive version of the > method. How will the compiler stop you from doing this, and, even if it > does, how is a java programmer to know these snippets won't work? > > Snippet #1: new ArrayList().add(10); > > and to show this problem cannot be remedied just by detecting such > situations and disallowing them: > > Snippet #2: > > public static List createList(Class type) { return new ArrayList(); > } > > public void test() { > List list = createList(int.class); > list.add(10); > } > > > --Reinier Zwitserloot > > > > On Thu, Jul 8, 2010 at 9:42 AM, Gernot Neppertwrote: > >> Reinier, I think your proposal is propably off-topic with regards to >> this mailing list, nevertheless I could not resist to respond ;-) >> >> I like the idea of making efficient use of generic classes with >> primitive type parameters possible. However, I'm worried that solely >> relying on a JVM optimization will not be sufficient. Wouldn't it be >> possible to gain efficiency on every VM if something along the >> following could be adopted: >> >> 1. Let's assume the existence of a generic interface >> >> L { >> void add(T element); >> T get(int index); >> } >> >> 2. The declaration of a variable of type L

where p is one of the 8 >> primitive types is possible. >> If the compiler encounters such a variable, it assumes that L

>> actually is equivalent to following 'extended' interface >> >> L'

>> { >> void add(P element); >> void add(p element); >> P get(int index); >> p get(int index); >> } >> >> where 'P' is the wrapper type corresponding to 'p'. >> >> 3. It would still remain illegal to declare an interface such as the >> above 'by hand', since overriding by return type is illegal unless the >> return types are covariant. However, the VM can cope with such >> overrides; and the compiler will encounter no ambiguities, as >> paragraph 4. will show. >> >> 4. With the assumptions made above, the following code would compile >> to the most efficient bytecode, invoking the primitive versions of >> 'add' and 'get': >> >> L list; >> list.add(42); >> int value = list.get(0); >> >> >> 5. There has to be a means of instantiating objects of type L

. With >> the assumption made in 2., the solution is the same as with other >> abstract classes: >> >> A non-abstract class that implements L

for a primitive type p must >> implement those methods from the synthetic L'

that use the >> primitive type p, while the compiler will generate the necessary >> bridge methods with the wrapper type P: >> >> public class IntList implements L >> { >> public void add(int element) >> { >> // efficient int-based implementation >> } >> >> >> public int get(int index) >> { >> // efficient int-based implementation. >> } >> >> } >> >> 6. The types L

and L

will be assignment compatible in only one >> direction: It will be possible to assign an instance of type L to >> L, but not vice versa. >> >> L list1 = new IntList(); // OK >> L list2 = new ArrayList(); // Will not compile >> >> 7. If the assignment from L

to L

is forced through an unsafe >> cast, the attempt to invoke a method of L

using the primitive type >> p will result in a java.lang.NoSuchMethodException at runtime. This is >> comparable to the ClassCastException that can be thrown arbitrarily >> when unsafe casts are involved. >> >> L list = (List)(List)new ArrayList(); >> list.add(42); // Will throw NoSuchMethodException. >> > From neal at gafter.com Thu Jul 8 14:49:29 2010 From: neal at gafter.com (Neal Gafter) Date: Thu, 8 Jul 2010 14:49:29 -0700 Subject: Primitives in Generics In-Reply-To: <4C36465C.6090809@oracle.com> References: <4C36465C.6090809@oracle.com> Message-ID: ParallelArray is really the *only* well-known example of the kind of aggregate operations API that project lambda is intended to enable. If project lambda can't do a good job there, it is hard to see how we can consider it to have fulfilled its design criteria. On Thu, Jul 8, 2010 at 2:42 PM, Brian Goetz wrote: > Its worth noting that ParallelArray is really the *only* well-known bad > case > here. Given the invasiveness to the language that function types > represent, > it seems a bad trade to take on such a big solution for a single > ill-behaved > library that some mad professor cooked up :) :) :) > ^ Lots of smileys here! > > > > On 7/8/2010 9:48 AM, Reinier Zwitserloot wrote: > > It's quite relevant, because state-of-the-lambda suffers from a rather > > serious problem, which is highlighted by a core use-case driving project > > lambda: ParallelArray's Ops container with its 132 SAM types. This > proposal > > would solve the issue and reduce that set of 132 to 8. > > > > If a JVM optimization can be proven to be trivial to implement (JRockit), > > and fail to apply only in those cases where specialization wouldn't work > > either, then I don't see how relying on this optimization is not > sufficient. > > > > Your idea is an intriguing one which would indeed drop the need for the > > JRockit boxing elimination optimization, though I don't think it can > work. > > For example, the following 2 code samples would seem to be perfectly > valid, > > but as far as I can tell the end result will be a NoSuchMethodError > because > > the produced instance will not actually have the primitive version of the > > method. How will the compiler stop you from doing this, and, even if it > > does, how is a java programmer to know these snippets won't work? > > > > Snippet #1: new ArrayList().add(10); > > > > and to show this problem cannot be remedied just by detecting such > > situations and disallowing them: > > > > Snippet #2: > > > > public static List createList(Class type) { return new > ArrayList(); > > } > > > > public void test() { > > List list = createList(int.class); > > list.add(10); > > } > > > > > > --Reinier Zwitserloot > > > > > > > > On Thu, Jul 8, 2010 at 9:42 AM, Gernot Neppert >wrote: > > > >> Reinier, I think your proposal is propably off-topic with regards to > >> this mailing list, nevertheless I could not resist to respond ;-) > >> > >> I like the idea of making efficient use of generic classes with > >> primitive type parameters possible. However, I'm worried that solely > >> relying on a JVM optimization will not be sufficient. Wouldn't it be > >> possible to gain efficiency on every VM if something along the > >> following could be adopted: > >> > >> 1. Let's assume the existence of a generic interface > >> > >> L { > >> void add(T element); > >> T get(int index); > >> } > >> > >> 2. The declaration of a variable of type L

where p is one of the 8 > >> primitive types is possible. > >> If the compiler encounters such a variable, it assumes that L

> >> actually is equivalent to following 'extended' interface > >> > >> L'

> >> { > >> void add(P element); > >> void add(p element); > >> P get(int index); > >> p get(int index); > >> } > >> > >> where 'P' is the wrapper type corresponding to 'p'. > >> > >> 3. It would still remain illegal to declare an interface such as the > >> above 'by hand', since overriding by return type is illegal unless the > >> return types are covariant. However, the VM can cope with such > >> overrides; and the compiler will encounter no ambiguities, as > >> paragraph 4. will show. > >> > >> 4. With the assumptions made above, the following code would compile > >> to the most efficient bytecode, invoking the primitive versions of > >> 'add' and 'get': > >> > >> L list; > >> list.add(42); > >> int value = list.get(0); > >> > >> > >> 5. There has to be a means of instantiating objects of type L

. With > >> the assumption made in 2., the solution is the same as with other > >> abstract classes: > >> > >> A non-abstract class that implements L

for a primitive type p must > >> implement those methods from the synthetic L'

that use the > >> primitive type p, while the compiler will generate the necessary > >> bridge methods with the wrapper type P: > >> > >> public class IntList implements L > >> { > >> public void add(int element) > >> { > >> // efficient int-based implementation > >> } > >> > >> > >> public int get(int index) > >> { > >> // efficient int-based implementation. > >> } > >> > >> } > >> > >> 6. The types L

and L

will be assignment compatible in only one > >> direction: It will be possible to assign an instance of type L to > >> L, but not vice versa. > >> > >> L list1 = new IntList(); // OK > >> L list2 = new ArrayList(); // Will not compile > >> > >> 7. If the assignment from L

to L

is forced through an unsafe > >> cast, the attempt to invoke a method of L

using the primitive type > >> p will result in a java.lang.NoSuchMethodException at runtime. This is > >> comparable to the ClassCastException that can be thrown arbitrarily > >> when unsafe casts are involved. > >> > >> L list = (List)(List)new ArrayList(); > >> list.add(42); // Will throw NoSuchMethodException. > >> > > > > From john.r.rose at oracle.com Thu Jul 8 14:50:41 2010 From: john.r.rose at oracle.com (John Rose) Date: Thu, 8 Jul 2010 14:50:41 -0700 Subject: State of the Lambda In-Reply-To: <4C362A8C.4070902@oracle.com> References: <4C34AC04.9090109@oracle.com> <4C362A8C.4070902@oracle.com> Message-ID: <6A29BC79-6E21-4AC8-ADB1-BB50CDC0CBA5@oracle.com> On Jul 8, 2010, at 12:44 PM, Brian Goetz wrote: > What I think you are asking for is something like "field references" to go > along with "method references". No, I think he means the difference between the unbound String#length() which is of type (String)->int and the bound "myString"#length(), which is of type ()->int. The latter is derived by a partial-application operation from the former. John's example uses the unbound version, while other use cases (event handlers, e.g.) would use the bound version. > select({ p -> p.getName() }, { p -> p.getAge() }).from(persons) => select(Person#getName(), Person#getAge()).from(persons) The method reference Foo#bar() is the basic building block. The derived reference myFoo#bar() starts with Foo#bar() and partially applies the receiver argument myFoo. The binding could be neatly performed by an inserted call to MethodHandle.bindTo or an similar static runtime routine. The basic unbound method reference could be materialized by a JSR 292 ldc/MethodHandle, perhaps with a library routine call (asSAM) to wrap it in the SAM type. Compilers would be able to readily optimize the bytecodes into one heap node when or if desirable. -- John BTW, in my prototype, I made the trailing parentheses *required* precisely to leave room for the analogous (but less useful) notion of field references. (Both bound and unbound, both setters and getters: all were implemented; see previous reference.) Requiring the trailing parens makes it slightly more clear that a method is being mentioned, since identifier-followed-by-paren is a cue Java programmers know very well. I chose a wildcard-like notation Foo#bar(...) to mean "find the unique 'bar' method". But that's a relatively unimportant detail, compared to getting the bound-vs.-unbound distinction right. From john.r.rose at oracle.com Thu Jul 8 14:53:23 2010 From: john.r.rose at oracle.com (John Rose) Date: Thu, 8 Jul 2010 14:53:23 -0700 Subject: State of the Lambda In-Reply-To: <4C364512.1050703@oracle.com> References: <4C34AC04.9090109@oracle.com> <232B8A26-69CE-4BCC-B545-F654BFDBA486@oracle.com> <4C364512.1050703@oracle.com> Message-ID: On Jul 8, 2010, at 2:37 PM, Brian Goetz wrote: > My intention for method refs for non-static methods is that the receiver reference is evaluated and copied (likely as part of the method handle binding); this is a binding of a method handle with a specific receiver, not syntactic sugar for "whatever the receiver happens to be at the time the lambda is invoked, call it." Right; partial application must not be confused with making a closure. (Since they are two distinct ideas!) That's why I sympathize with Josh's qualms about the lambda-like syntax #foo.bar and prefer foo#bar, foo.#bar, foo.&bar or some other infix syntax. -- John From brian.goetz at oracle.com Thu Jul 8 14:57:01 2010 From: brian.goetz at oracle.com (Brian Goetz) Date: Thu, 08 Jul 2010 17:57:01 -0400 Subject: State of the Lambda In-Reply-To: <6A29BC79-6E21-4AC8-ADB1-BB50CDC0CBA5@oracle.com> References: <4C34AC04.9090109@oracle.com> <4C362A8C.4070902@oracle.com> <6A29BC79-6E21-4AC8-ADB1-BB50CDC0CBA5@oracle.com> Message-ID: <4C3649AD.4090606@oracle.com> > No, I think he means the difference between the unbound String#length() which is of type (String)->int and the bound "myString"#length(), which is of type ()->int. The latter is derived by a partial-application operation from the former. John's example uses the unbound version, while other use cases (event handlers, e.g.) would use the bound version. > >> select({ p -> p.getName() }, { p -> p.getAge() }).from(persons) > => > select(Person#getName(), Person#getAge()).from(persons) Ah, I see. Nice idea. > Requiring the trailing parens makes it slightly more clear that a method is being mentioned, since identifier-followed-by-paren is a cue Java programmers know very well. I chose a wildcard-like notation Foo#bar(...) to mean "find the unique 'bar' method". But that's a relatively unimportant detail, compared to getting the bound-vs.-unbound distinction right. Yep, and not too burdensome. From grev at miginfocom.com Thu Jul 8 14:59:36 2010 From: grev at miginfocom.com (Mikael Grev) Date: Thu, 8 Jul 2010 23:59:36 +0200 Subject: Transparancy In-Reply-To: <4C3608CE.1080909@oracle.com> References: <4C34C6BE.5010101@oracle.com> <4C3608CE.1080909@oracle.com> Message-ID: <60C317AE-9932-4A62-8A15-5DB0179EFB88@miginfocom.com> I actually don't find return in F very strange. It works for me. In the perfect world you can always make the decisions on pure technical merits and all will follow and accept. What I mean is that there is a tradeoff here. People will hate the yield keyword simply because so few will understand the IMO very little benefit of holding the return keyword for, again IMO, a bad future possibility. You fare a big risk of being nuked in the blogosphere and that means more than any technical merits it may have. Think generics here, how may understand why they are as they are, yet they love to hate them? I think many will mean that if you add a keywords that easily why not for other things? Sun has been really reluctant to add keywords before, for better reasons ("property" comes to mind). Well, I don't think we will get any further in this thread. Make a decision and move on. :) Cheers, Mikael On Jul 8, 2010, at 19:20 PM, Brian Goetz wrote: >> Most developers seem to think that a lambda body is comparable to a SAM/AIC implementation body, which is a method. > > This is true, but only because they have no other experience. The reality is that they are mostly like method bodies, but not entirely like method bodies. Such first-order approximations are very useful in forming mental models but should not be taken too far! > > Here are some contexts in Java-like languages where a control flow statement stops executing in the current context and resumes in another: > > A Return from a method. > B Local return from a closure. > C Lexical return from a closure. > D Incremental production from a coroutine. > E Terminal production from a coroutine. > F Value production from a local block (i.e.: > var x = { var y = 3; yield 3*y } > G Lexical return from a local block. > > Java currently only has A. So our thinking about "what return means" is heavily conditioned by our experience with A. > > BGGA has both B and C. The current SotL draft has "yield" for B and currently nothing for "C". > > I think we can all agree that "return" is absurd for "F". (You might not like "yield" because it is used for something else (D/E) in C# and yet something else (and weird!) in Ruby, but "yield" is a fine candidate for F.) > > So "return" is obviously right for A and obviously wrong for F. B and C have something in common with methods, and something in common with local blocks. We're letting the fact that Java has methods and does not have local blocks color our thinking that closures are "just like" methods when in fact they are "mostly like" methods. (We can disagree about whether it's 98/2 or 90/10, but it's obviously not 100/0.) > > Imagine you grew up in a world where the only animals were dogs (methods). Your mental model of animal would be heavily influenced by the characteristics of dogs -- for example, animals have four legs*. Then one day ants (local blocks) were introduced into your planet. They are obviously not dogs, but they have more legs than you and are shorter than you, so they are more like dogs than like humans, so we redefine the world as "there are people and animals; animals are dogs and ants." Having thus classified things, you are again comfortable with the world. Two hundred years later, a cat (closure) arrives on your planet. Most people in this situation will think "what a fancy dog!" > > The point is that "return" is only obvious for B if you have grown up in the world of A-but-not-F. And if we're sure that C/D/E/F/G will *never* be added to Java, then we are perfectly justified in ignoring all the cat-like aspects of closures and lump them in with dogs, and choose "return" for B and never think about it again. But if there's any reasonable chance that F (or even C) will ever become part of Java, then all of a sudden these dogs that say "meow" start to seem really weird. > > That said, this forum is for lambda discussions, so this is probably not the place for a discussion about which of these forms Java should have (beyond A/B/C). However, considerations of where Java might or might not go in future releases that do not even have numbers yet do (and should!) inform our thoughts. > > > > > *Yes, everyone has a three-legged dog story. From nathan.bryant at linkshare.com Thu Jul 8 15:02:20 2010 From: nathan.bryant at linkshare.com (Nathan Bryant) Date: Fri, 9 Jul 2010 07:02:20 +0900 Subject: State of the Lambda References: <4C34AC04.9090109@oracle.com><232B8A26-69CE-4BCC-B545-F654BFDBA486@oracle.com> <4C364512.1050703@oracle.com> Message-ID: <7FDA6630E1822F448C97A48D5D733094FB82BA@EXVMSTOR302.intra.rakuten.co.jp> So, #foo.setBar is sugar for { Bar bar -> foo.setBar(bar) } An alternate desugared form, perhaps more convenient for the runtime: { Foo x -> { Bar bar -> x.setBar(bar) } } (foo) // Is this legal? If the first desugaring is accurate, then "foo" must be effectively final. If the second is accurate, then it seems foo need not be, but would be copied -----Original Message----- From: lambda-dev-bounces at openjdk.java.net [mailto:lambda-dev-bounces at openjdk.java.net] On Behalf Of Brian Goetz Sent: Thursday, July 08, 2010 5:37 PM To: John Rose Cc: lambda-dev at openjdk.java.net Subject: Re: State of the Lambda >> I'm a little bit worried that method references with receiver variables do >> an copy. ... A possible solution is to require such receiver >> variables to be (implicitly) final. > > This is one of those times where syntax might imply something deep about an expression. Brian's sketch uses a prefix "#" to introduce a method lifted to a function (and perhaps partially applied to a receiver). > > The prefix looks like it might govern the evaluation of the expression to the right. An infix operator would much more clearly express that the left hand side of the expression (at least) is evaluated in the normal left-to-right order of Java expressions. (Some infix operators, ?:&& || have special rules for evaluation order, but it only applies to the right of the operator.) My intention for method refs for non-static methods is that the receiver reference is evaluated and copied (likely as part of the method handle binding); this is a binding of a method handle with a specific receiver, not syntactic sugar for "whatever the receiver happens to be at the time the lambda is invoked, call it." From john at milsson.nu Thu Jul 8 15:07:31 2010 From: john at milsson.nu (John Nilsson) Date: Fri, 9 Jul 2010 00:07:31 +0200 Subject: State of the Lambda In-Reply-To: <6A29BC79-6E21-4AC8-ADB1-BB50CDC0CBA5@oracle.com> References: <4C34AC04.9090109@oracle.com> <4C362A8C.4070902@oracle.com> <6A29BC79-6E21-4AC8-ADB1-BB50CDC0CBA5@oracle.com> Message-ID: On Thu, Jul 8, 2010 at 11:50 PM, John Rose wrote: > On Jul 8, 2010, at 12:44 PM, Brian Goetz wrote: > >> What I think you are asking for is something like "field references" to go >> along with "method references". > > No, I think he means the difference between the unbound String#length() which is of type (String)->int and the bound "myString"#length(), which is of type ()->int. ?The latter is derived by a partial-application operation from the former. Ah, yes, that was what I meant. Thanks for the clarification! :) The cost, for us, of having to resort to strings and reflection is the lack of IDE-support to find the references when doing refactorings. Even with the somewhat verbose syntax the current proposal uses it would be worth it to get rid of that source of problems. Anything that can be done to make it leaner to the eye would be welcome though. BR, John From mbien at fh-landshut.de Thu Jul 8 15:18:21 2010 From: mbien at fh-landshut.de (Michael Bien) Date: Fri, 09 Jul 2010 00:18:21 +0200 Subject: Primitives in Generics In-Reply-To: <4C36465C.6090809@oracle.com> References: <4C36465C.6090809@oracle.com> Message-ID: <4C364EAD.1040803@fh-landshut.de> I wouldn't bet money on that :). E.g we generate various kinds of primitive HashMaps just to prevent autoboxing which caused problems in some situations. http://jogamp.org/deployment/gluegen-next/javadoc/com/jogamp/common/util/package-frame.html (gluegen is the binding lib used for jogl, jocl and joal) sure its unrelated to project lambda but its basically the same issue as in Ops.Foo (ParallelArray). regards, michael On 07/08/2010 11:42 PM, Brian Goetz wrote: > Its worth noting that ParallelArray is really the *only* well-known bad case > here. Given the invasiveness to the language that function types represent, > it seems a bad trade to take on such a big solution for a single ill-behaved > library that some mad professor cooked up :) :) :) > ^ Lots of smileys here! > > > > On 7/8/2010 9:48 AM, Reinier Zwitserloot wrote: >> It's quite relevant, because state-of-the-lambda suffers from a rather >> serious problem, which is highlighted by a core use-case driving project >> lambda: ParallelArray's Ops container with its 132 SAM types. This proposal >> would solve the issue and reduce that set of 132 to 8. >> >> If a JVM optimization can be proven to be trivial to implement (JRockit), >> and fail to apply only in those cases where specialization wouldn't work >> either, then I don't see how relying on this optimization is not sufficient. >> >> Your idea is an intriguing one which would indeed drop the need for the >> JRockit boxing elimination optimization, though I don't think it can work. >> For example, the following 2 code samples would seem to be perfectly valid, >> but as far as I can tell the end result will be a NoSuchMethodError because >> the produced instance will not actually have the primitive version of the >> method. How will the compiler stop you from doing this, and, even if it >> does, how is a java programmer to know these snippets won't work? >> >> Snippet #1: new ArrayList().add(10); >> >> and to show this problem cannot be remedied just by detecting such >> situations and disallowing them: >> >> Snippet #2: >> >> public static List createList(Class type) { return new ArrayList(); >> } >> >> public void test() { >> List list = createList(int.class); >> list.add(10); >> } >> >> >> --Reinier Zwitserloot >> >> >> >> On Thu, Jul 8, 2010 at 9:42 AM, Gernot Neppertwrote: >> >>> Reinier, I think your proposal is propably off-topic with regards to >>> this mailing list, nevertheless I could not resist to respond ;-) >>> >>> I like the idea of making efficient use of generic classes with >>> primitive type parameters possible. However, I'm worried that solely >>> relying on a JVM optimization will not be sufficient. Wouldn't it be >>> possible to gain efficiency on every VM if something along the >>> following could be adopted: >>> >>> 1. Let's assume the existence of a generic interface >>> >>> L { >>> void add(T element); >>> T get(int index); >>> } >>> >>> 2. The declaration of a variable of type L

where p is one of the 8 >>> primitive types is possible. >>> If the compiler encounters such a variable, it assumes that L

>>> actually is equivalent to following 'extended' interface >>> >>> L'

>>> { >>> void add(P element); >>> void add(p element); >>> P get(int index); >>> p get(int index); >>> } >>> >>> where 'P' is the wrapper type corresponding to 'p'. >>> >>> 3. It would still remain illegal to declare an interface such as the >>> above 'by hand', since overriding by return type is illegal unless the >>> return types are covariant. However, the VM can cope with such >>> overrides; and the compiler will encounter no ambiguities, as >>> paragraph 4. will show. >>> >>> 4. With the assumptions made above, the following code would compile >>> to the most efficient bytecode, invoking the primitive versions of >>> 'add' and 'get': >>> >>> L list; >>> list.add(42); >>> int value = list.get(0); >>> >>> >>> 5. There has to be a means of instantiating objects of type L

. With >>> the assumption made in 2., the solution is the same as with other >>> abstract classes: >>> >>> A non-abstract class that implements L

for a primitive type p must >>> implement those methods from the synthetic L'

that use the >>> primitive type p, while the compiler will generate the necessary >>> bridge methods with the wrapper type P: >>> >>> public class IntList implements L >>> { >>> public void add(int element) >>> { >>> // efficient int-based implementation >>> } >>> >>> >>> public int get(int index) >>> { >>> // efficient int-based implementation. >>> } >>> >>> } >>> >>> 6. The types L

and L

will be assignment compatible in only one >>> direction: It will be possible to assign an instance of type L to >>> L, but not vice versa. >>> >>> L list1 = new IntList(); // OK >>> L list2 = new ArrayList(); // Will not compile >>> >>> 7. If the assignment from L

to L

is forced through an unsafe >>> cast, the attempt to invoke a method of L

using the primitive type >>> p will result in a java.lang.NoSuchMethodException at runtime. This is >>> comparable to the ClassCastException that can be thrown arbitrarily >>> when unsafe casts are involved. >>> >>> L list = (List)(List)new ArrayList(); >>> list.add(42); // Will throw NoSuchMethodException. >>> > -- http://michael-bien.com/ From forax at univ-mlv.fr Thu Jul 8 15:38:56 2010 From: forax at univ-mlv.fr (=?ISO-8859-1?Q?R=E9mi_Forax?=) Date: Fri, 09 Jul 2010 00:38:56 +0200 Subject: Primitives in Generics In-Reply-To: <4C364EAD.1040803@fh-landshut.de> References: <4C36465C.6090809@oracle.com> <4C364EAD.1040803@fh-landshut.de> Message-ID: <4C365380.6030303@univ-mlv.fr> ParallelArray is not the *only* bad case. Trove was created more than 10 years ago. http://trove4j.sourceforge.net/javadocs/ R?mi Le 09/07/2010 00:18, Michael Bien a ?crit : > I wouldn't bet money on that :). E.g we generate various kinds of > primitive HashMaps just to prevent autoboxing which caused problems in > some situations. > http://jogamp.org/deployment/gluegen-next/javadoc/com/jogamp/common/util/package-frame.html > (gluegen is the binding lib used for jogl, jocl and joal) > > sure its unrelated to project lambda but its basically the same issue as > in Ops.Foo (ParallelArray). > > regards, > michael > > On 07/08/2010 11:42 PM, Brian Goetz wrote: > >> Its worth noting that ParallelArray is really the *only* well-known bad case >> here. Given the invasiveness to the language that function types represent, >> it seems a bad trade to take on such a big solution for a single ill-behaved >> library that some mad professor cooked up :) :) :) >> ^ Lots of smileys here! >> >> >> >> On 7/8/2010 9:48 AM, Reinier Zwitserloot wrote: >> >>> It's quite relevant, because state-of-the-lambda suffers from a rather >>> serious problem, which is highlighted by a core use-case driving project >>> lambda: ParallelArray's Ops container with its 132 SAM types. This proposal >>> would solve the issue and reduce that set of 132 to 8. >>> >>> If a JVM optimization can be proven to be trivial to implement (JRockit), >>> and fail to apply only in those cases where specialization wouldn't work >>> either, then I don't see how relying on this optimization is not sufficient. >>> >>> Your idea is an intriguing one which would indeed drop the need for the >>> JRockit boxing elimination optimization, though I don't think it can work. >>> For example, the following 2 code samples would seem to be perfectly valid, >>> but as far as I can tell the end result will be a NoSuchMethodError because >>> the produced instance will not actually have the primitive version of the >>> method. How will the compiler stop you from doing this, and, even if it >>> does, how is a java programmer to know these snippets won't work? >>> >>> Snippet #1: new ArrayList().add(10); >>> >>> and to show this problem cannot be remedied just by detecting such >>> situations and disallowing them: >>> >>> Snippet #2: >>> >>> public static List createList(Class type) { return new ArrayList(); >>> } >>> >>> public void test() { >>> List list = createList(int.class); >>> list.add(10); >>> } >>> >>> >>> --Reinier Zwitserloot >>> >>> >>> >>> On Thu, Jul 8, 2010 at 9:42 AM, Gernot Neppertwrote: >>> >>> >>>> Reinier, I think your proposal is propably off-topic with regards to >>>> this mailing list, nevertheless I could not resist to respond ;-) >>>> >>>> I like the idea of making efficient use of generic classes with >>>> primitive type parameters possible. However, I'm worried that solely >>>> relying on a JVM optimization will not be sufficient. Wouldn't it be >>>> possible to gain efficiency on every VM if something along the >>>> following could be adopted: >>>> >>>> 1. Let's assume the existence of a generic interface >>>> >>>> L { >>>> void add(T element); >>>> T get(int index); >>>> } >>>> >>>> 2. The declaration of a variable of type L

where p is one of the 8 >>>> primitive types is possible. >>>> If the compiler encounters such a variable, it assumes that L

>>>> actually is equivalent to following 'extended' interface >>>> >>>> L'

>>>> { >>>> void add(P element); >>>> void add(p element); >>>> P get(int index); >>>> p get(int index); >>>> } >>>> >>>> where 'P' is the wrapper type corresponding to 'p'. >>>> >>>> 3. It would still remain illegal to declare an interface such as the >>>> above 'by hand', since overriding by return type is illegal unless the >>>> return types are covariant. However, the VM can cope with such >>>> overrides; and the compiler will encounter no ambiguities, as >>>> paragraph 4. will show. >>>> >>>> 4. With the assumptions made above, the following code would compile >>>> to the most efficient bytecode, invoking the primitive versions of >>>> 'add' and 'get': >>>> >>>> L list; >>>> list.add(42); >>>> int value = list.get(0); >>>> >>>> >>>> 5. There has to be a means of instantiating objects of type L

. With >>>> the assumption made in 2., the solution is the same as with other >>>> abstract classes: >>>> >>>> A non-abstract class that implements L

for a primitive type p must >>>> implement those methods from the synthetic L'

that use the >>>> primitive type p, while the compiler will generate the necessary >>>> bridge methods with the wrapper type P: >>>> >>>> public class IntList implements L >>>> { >>>> public void add(int element) >>>> { >>>> // efficient int-based implementation >>>> } >>>> >>>> >>>> public int get(int index) >>>> { >>>> // efficient int-based implementation. >>>> } >>>> >>>> } >>>> >>>> 6. The types L

and L

will be assignment compatible in only one >>>> direction: It will be possible to assign an instance of type L to >>>> L, but not vice versa. >>>> >>>> L list1 = new IntList(); // OK >>>> L list2 = new ArrayList(); // Will not compile >>>> >>>> 7. If the assignment from L

to L

is forced through an unsafe >>>> cast, the attempt to invoke a method of L

using the primitive type >>>> p will result in a java.lang.NoSuchMethodException at runtime. This is >>>> comparable to the ClassCastException that can be thrown arbitrarily >>>> when unsafe casts are involved. >>>> >>>> L list = (List)(List)new ArrayList(); >>>> list.add(42); // Will throw NoSuchMethodException. >>>> >>>> >> > From jjb at google.com Thu Jul 8 16:16:35 2010 From: jjb at google.com (Joshua Bloch) Date: Thu, 8 Jul 2010 16:16:35 -0700 Subject: State of the Lambda In-Reply-To: <232B8A26-69CE-4BCC-B545-F654BFDBA486@oracle.com> References: <4C34AC04.9090109@oracle.com> <232B8A26-69CE-4BCC-B545-F654BFDBA486@oracle.com> Message-ID: John, Yes; your proposed syntax would alleviate this concern. Josh On Wed, Jul 7, 2010 at 11:25 PM, John Rose wrote: > On Jul 7, 2010, at 2:56 PM, Joshua Bloch wrote: > > > I'm a little bit worried that method references with receiver variables > do > > an copy. ... A possible solution is to require such receiver > > variables to be (implicitly) final. > > This is one of those times where syntax might imply something deep about an > expression. Brian's sketch uses a prefix "#" to introduce a method lifted > to a function (and perhaps partially applied to a receiver). > > The prefix looks like it might govern the evaluation of the expression to > the right. An infix operator would much more clearly express that the left > hand side of the expression (at least) is evaluated in the normal > left-to-right order of Java expressions. (Some infix operators, ?: && || > have special rules for evaluation order, but it only applies to the right of > the operator.) > > An infix formulation would make it clearer that the qualifier LHS is a > normal expression: > > foo # bar > baz[++n].bang() # bar > > This means that foo#bar is not equivalent to { (T x) -> foo.bar(x) } if foo > contains side effects or is affected by them. > > -- John > > P.S. FWIW, I prototyped the foo#bar syntax as a test for JSR 292, which > includes an extended 'ldc' bytecode which can materialize arbitrary method > constants. See the bottom of the patch for a unit test file which exercises > a similar syntax. > > http://hg.openjdk.java.net/mlvm/mlvm/langtools/file/tip/meth-ldc-6939203.patch > > From john.r.rose at oracle.com Thu Jul 8 20:15:16 2010 From: john.r.rose at oracle.com (John Rose) Date: Thu, 8 Jul 2010 20:15:16 -0700 Subject: State of the Lambda In-Reply-To: <4C3620D7.3020000@oracle.com> References: <4C34AC04.9090109@oracle.com> <4C3620D7.3020000@oracle.com> Message-ID: On Jul 8, 2010, at 12:02 PM, Brian Goetz wrote: > Another is to do something really CICE-like: > > executor.submit(new Callable(args) { -> "foo"} ); Cute: Starts like an anonymous inner class (so it shouts "I'm a class!") but ends with a closure body (which murmurs, "here's your code, sir"). But, just because something can be compressed doesn't mean it should be. A pattern needs to be common in order for its compression to pay off. Are there numbers that show that inner classes with constructor arguments are common enough to make this extension of CICE worth the effort? I suspect not. -- John P.S. I'll throw out a general question that is related: How common is access to 'protected' names from inner classes that implement SAM types? This is one of the peculiar strengths of CICE, but I'm curious how often it would be used. I suspect it would be infrequently used. From jkuhnert at gmail.com Thu Jul 8 20:46:23 2010 From: jkuhnert at gmail.com (Jesse Kuhnert) Date: Thu, 8 Jul 2010 23:46:23 -0400 Subject: State of the Lambda In-Reply-To: References: <4C34AC04.9090109@oracle.com> <4C3620D7.3020000@oracle.com> Message-ID: Yes, saving characters is nice and all but what we really need are better building blocks. The community is holding it's breath, java7 means either continued life / shelved language for better or more modern languages to run on top of / or just something to abandon altogether. I'm sure Oracle has it's own goals and needs, but some decisions live on long after any one employer. On Thursday, July 8, 2010, John Rose wrote: > On Jul 8, 2010, at 12:02 PM, Brian Goetz wrote: > >> Another is to do something really CICE-like: >> >> ? ? ? executor.submit(new Callable(args) { -> "foo"} ); > > Cute: ?Starts like an anonymous inner class (so it shouts "I'm a class!") but ends with a closure body (which murmurs, "here's your code, sir"). > > But, just because something can be compressed doesn't mean it should be. ?A pattern needs to be common in order for its compression to pay off. ?Are there numbers that show that inner classes with constructor arguments are common enough to make this extension of CICE worth the effort? ?I suspect not. > > -- John > > P.S. I'll throw out a general question that is related: ?How common is access to 'protected' names from inner classes that implement SAM types? ?This is one of the peculiar strengths of CICE, but I'm curious how often it would be used. ?I suspect it would be infrequently used. > > From reinier at zwitserloot.com Thu Jul 8 20:58:21 2010 From: reinier at zwitserloot.com (Reinier Zwitserloot) Date: Fri, 9 Jul 2010 05:58:21 +0200 Subject: State of the Lambda In-Reply-To: References: <4C34AC04.9090109@oracle.com> Message-ID: With static imports you can do something rather similar based around the current proposal: from(persons).select({p -> p.getName()}, {p -> p.getAge()}); from would be something like: public static Selector from(Collection c) { ... } and select would be: public interface Transformer { R transform(A in); } public class Selector { public List select(Transformer... transformers) { ... } } By moving the 'from' to the front, type inference on the list can eliminate the need for a mandatory mention of the "Person" type; the compiler can figure out that the select method can only take a varargs list of Transformer instances, which means all those lambda expressions will be fitted to that SAM type, which results in the compiler being able to infer both Transformer itself as well as the parameter type (Person). --Reinier Zwitserloot On Thu, Jul 8, 2010 at 9:39 PM, John Nilsson wrote: > All in all I like the approach you are taking, do agree with the -1 on > yield though. > > One question wrt method references: do you intend to support "unbound" > references to to instance methods? The use case being a shorter syntax > to define expression lists in query like expression. I.e. With the > current proposal I guess I could have an expression like this: > > select({ p -> p.getName() }, { p -> p.getAge() }).from(persons) > > A use case I imagine won't be to uncommon. F.ex. in our system we have > resorted to strings and reflection to select properties from objects. > > select("name","age").from(persons) > > By "unbound" method reference I thus mean any syntactic sugar that > shortens the syntactic distance between the two expressions. > > BR, > John > > From mcnepp02 at googlemail.com Thu Jul 8 23:28:03 2010 From: mcnepp02 at googlemail.com (Gernot Neppert) Date: Fri, 9 Jul 2010 06:28:03 +0000 Subject: Primitives in Generics In-Reply-To: References: Message-ID: 2010/7/8 Reinier Zwitserloot : > You didn't mention my second snippet, which is a shame, because I included > it because that's the one I'm confused about - how could you possibly write > a compiler that detects there's a problem there? > --Reinier Zwitserloot >Snippet #2: >public static List createList(Class type) { return new ArrayList(); } >public void test() { > List list = createList(int.class); > list.add(10); >} OK, looks like you've found the big weak spot in my proposal ;-) Even though the exact code from snippet #2 would not compile (int.class is of type Class, therefore the assignment to List is not valid), the snippet shows of course the fundamental problem: The use of primitive type parameters in all contexts is incompatible with Type Erasure. Thus, one would have to restrict the use; maybe something like this would do: 8. Type parameters of primitive type parameters may only be used in the following contexts: a) when declaring variables. Local variables or members may be declared using a primitive type parameter, and optionally initialized with null. Example: List list = null; b) during implementation of a generic interface or a generic baseclass by a class that specifies a concrete type for the type parameter. Example: public class IntList implements List { } c) generic functions will never make use of primitive type parameters. If you pass an instance of a generic class L

for a primitive type p to a generic function paramerized by T over L, the instance will be automatically treated as L

with P the corresponding wrapper type for p. (See paragraph 6. for assignment rules) Example: import java.util.Collections; List list = new IntList(); List sorted = Collections.sort(list); // Will not compile List empty = Collections.emptyList(); // Will not compile List sorted = Collections.sort(list); // OK, list is passed as List here. From scolebourne at joda.org Thu Jul 8 23:32:25 2010 From: scolebourne at joda.org (Stephen Colebourne) Date: Fri, 9 Jul 2010 07:32:25 +0100 Subject: Transparancy In-Reply-To: <4C3608CE.1080909@oracle.com> References: <4C34C6BE.5010101@oracle.com> <4C3608CE.1080909@oracle.com> Message-ID: Thanks for a good explanation. I believe you missed a key element when you said java only has A. Inner classes. These may be a method in terms of the language spec, but they are used as a closure (B). When i write an anon inner class for a filter or comparator, i don't think of it in class terms, and i suspect i'm not alone in that. And it is still possible to use return for long-return C. This simply requires a language feature separate from closures, where the block must be executed before the end of the method, a la macros. So, the trick is retaining enough syntax space for a macro-like block in the future with long-return. Stephen BTW, does anyone outside oracle actually like 'yield'?? All the mails have opposed it so far IIRC From scolebourne at joda.org Thu Jul 8 23:35:33 2010 From: scolebourne at joda.org (Stephen Colebourne) Date: Fri, 9 Jul 2010 07:35:33 +0100 Subject: Transparancy In-Reply-To: <4C3608CE.1080909@oracle.com> References: <4C34C6BE.5010101@oracle.com> <4C3608CE.1080909@oracle.com> Message-ID: Thanks for a good explanation. I believe you missed a key element when you said java only has A. Inner classes. These may be a method in terms of the language spec, but they are used as a closure (B). When i write an anon inner class for a filter or comparator, i don't think of it in class terms, and i suspect i'm not alone in that. And it is still possible to use return for long-return C. This simply requires a language feature separate from closures, where the block must be executed before the end of the method, a la macros. So, the trick is retaining enough syntax space for a macro-like block in the future with long-return. Stephen BTW, does anyone outside oracle actually like 'yield'?? All the mails have opposed it so far IIRC From scolebourne at joda.org Fri Jul 9 00:01:12 2010 From: scolebourne at joda.org (Stephen Colebourne) Date: Fri, 9 Jul 2010 08:01:12 +0100 Subject: Method refs [was Re: State of the Lambda] Message-ID: +1 to all John said on bound and unbound method refs and associated syntax (# infix). Java frameworks today have a big need for both forms. Look at the play framework for one example. BTW, since method references do not have a denotable type, a useful addition to the spec for comparibility with older systems would be target type conversion to j.u.r.Method. I don't believe there are any complications in doing that. This then strengthens the case for field references. Stephen On 08/07/2010, John Rose wrote: > On Jul 8, 2010, at 12:44 PM, Brian Goetz wrote: > >> What I think you are asking for is something like "field references" to go >> >> along with "method references". > > No, I think he means the difference between the unbound String#length() > which is of type (String)->int and the bound "myString"#length(), which is > of type ()->int. The latter is derived by a partial-application operation > from the former. John's example uses the unbound version, while other use > cases (event handlers, e.g.) would use the bound version. > >> select({ p -> p.getName() }, { p -> p.getAge() >> }).from(persons) > => > select(Person#getName(), Person#getAge()).from(persons) > > The method reference Foo#bar() is the basic building block. The derived > reference myFoo#bar() starts with Foo#bar() and partially applies the > receiver argument myFoo. The binding could be neatly performed by an > inserted call to MethodHandle.bindTo or an similar static runtime routine. > The basic unbound method reference could be materialized by a JSR 292 > ldc/MethodHandle, perhaps with a library routine call (asSAM) to wrap it in > the SAM type. Compilers would be able to readily optimize the bytecodes > into one heap node when or if desirable. > > -- John > > BTW, in my prototype, I made the trailing parentheses *required* precisely > to leave room for the analogous (but less useful) notion of field > references. (Both bound and unbound, both setters and getters: all were > implemented; see previous reference.) > > Requiring the trailing parens makes it slightly more clear that a method is > being mentioned, since identifier-followed-by-paren is a cue Java > programmers know very well. I chose a wildcard-like notation Foo#bar(...) > to mean "find the unique 'bar' method". But that's a relatively unimportant > detail, compared to getting the bound-vs.-unbound distinction right. > > From forax at univ-mlv.fr Fri Jul 9 02:54:51 2010 From: forax at univ-mlv.fr (=?ISO-8859-1?Q?R=E9mi_Forax?=) Date: Fri, 09 Jul 2010 11:54:51 +0200 Subject: Transparancy In-Reply-To: References: <4C34C6BE.5010101@oracle.com> <4C3608CE.1080909@oracle.com> Message-ID: <4C36F1EB.4060305@univ-mlv.fr> Le 09/07/2010 08:32, Stephen Colebourne a ?crit : [...] > Stephen > > BTW, does anyone outside oracle actually like 'yield'?? All the mails > have opposed it so far IIRC > And there is another problem with 'yield', it means something different in most of the language that implements coroutine. http://en.wikipedia.org/wiki/Coroutine Moreover, there is already a prototype of coroutine in mlvm repository. Even if this prototype doesn't require the 'yield' keyword, i think 'yield' should be reserved for that usage. R?mi From john at milsson.nu Fri Jul 9 06:47:21 2010 From: john at milsson.nu (John Nilsson) Date: Fri, 9 Jul 2010 15:47:21 +0200 Subject: State of the Lambda In-Reply-To: References: <4C34AC04.9090109@oracle.com> Message-ID: Somewhat how LINQ does it, isn't it? Usually we define the projection before there exists a collection to select from though so I would then have to provide Person.class or something as argument. Actually I guess there's some other ways to do this that might make more sense than specific syntax. 1. With a lambda returning the entire list directly: { Person p -> select(p.getName(),p.getAge()) } 2. Generate meta data to define the projection: Query.persons({ p -> select(p.name,p.age) }); The idea in 2 being that the type of p is a MetaPerson. BR, John On Fri, Jul 9, 2010 at 5:58 AM, Reinier Zwitserloot wrote: > With static imports you can do something rather similar based around the > current proposal: > from(persons).select({p -> p.getName()}, {p -> p.getAge()}); > from would be something like: > public static Selector from(Collection c) { ... } > and select would be: > public interface Transformer { R transform(A in); } > public class Selector { > ?? ?public List select(Transformer... transformers) { > ?? ? ? ?... > ?? ?} > } > By moving the 'from' to the front, type inference on the list can eliminate > the need for a mandatory mention of the "Person" type; the compiler can > figure out that the select method can only take a varargs list of > Transformer instances, which means all those lambda expressions will > be fitted to that SAM type, which results in the compiler being able to > infer both Transformer itself as well as the parameter type > (Person). > ?--Reinier Zwitserloot > > > > On Thu, Jul 8, 2010 at 9:39 PM, John Nilsson wrote: >> >> All in all I like the approach you are taking, do agree with the -1 on >> yield though. >> >> One question wrt method references: do you intend to support "unbound" >> references to to instance methods? The use case being a shorter syntax >> to define expression lists in query like expression. I.e. With the >> current proposal I guess I could have an expression like this: >> >> ? ?select({ p -> p.getName() }, { p -> p.getAge() }).from(persons) >> >> A use case I imagine won't be to uncommon. F.ex. in our system we have >> resorted to strings and reflection to select properties from objects. >> >> ? ?select("name","age").from(persons) >> >> By "unbound" method reference I thus mean any syntactic sugar that >> shortens the syntactic distance between the two expressions. >> >> BR, >> John >> > > From dl at cs.oswego.edu Fri Jul 9 12:30:51 2010 From: dl at cs.oswego.edu (Doug Lea) Date: Fri, 09 Jul 2010 15:30:51 -0400 Subject: Primitives in Generics In-Reply-To: <4C36465C.6090809@oracle.com> References: <4C36465C.6090809@oracle.com> Message-ID: <4C3778EB.60405@cs.oswego.edu> On 07/08/10 17:42, Brian Goetz wrote: > Its worth noting that ParallelArray is really the *only* well-known bad case > here. Given the invasiveness to the language that function types represent, > it seems a bad trade to take on such a big solution for a single ill-behaved > library that some mad professor cooked up :) :) :) > ^ Lots of smileys here! > > My take is that you have three choices: A) Support function types covering primitives B) Support primitives in generics C) Do something that doesn't restrict your options to reconsider (A) or (B) in Java8 in response to all the complaints about sucky parallel performance. -Doug From brian.goetz at oracle.com Fri Jul 9 12:38:29 2010 From: brian.goetz at oracle.com (Brian Goetz) Date: Fri, 09 Jul 2010 15:38:29 -0400 Subject: Primitives in Generics In-Reply-To: <4C3778EB.60405@cs.oswego.edu> References: <4C36465C.6090809@oracle.com> <4C3778EB.60405@cs.oswego.edu> Message-ID: <4C377AB5.7050205@oracle.com> >> ... some mad professor cooked up :) :) :) > > My take is that you have three choices: > A) Support function types covering primitives > B) Support primitives in generics > C) Do something that doesn't restrict your options to reconsider > (A) or (B) in Java8 in response to all the complaints about > sucky parallel performance. Indeed so. We believe we have chosen (C). If we have not, please say something! From jkuhnert at gmail.com Fri Jul 9 12:51:58 2010 From: jkuhnert at gmail.com (Jesse Kuhnert) Date: Fri, 9 Jul 2010 15:51:58 -0400 Subject: Primitives in Generics In-Reply-To: <4C377AB5.7050205@oracle.com> References: <4C36465C.6090809@oracle.com> <4C3778EB.60405@cs.oswego.edu> <4C377AB5.7050205@oracle.com> Message-ID: Really? On Fri, Jul 9, 2010 at 3:38 PM, Brian Goetz wrote: >>> ... some mad professor cooked up :) ?:) ?:) >> >> My take is that you have three choices: >> ? ?A) Support function types covering primitives >> ? ?B) Support primitives in generics >> ? ?C) Do something that doesn't restrict your options to reconsider >> ? ? ? (A) or (B) in Java8 in response to all the complaints about >> ? ? ? sucky parallel performance. > > Indeed so. ?We believe we have chosen (C). ?If we have not, please say > something! > > From jkuhnert at gmail.com Fri Jul 9 13:33:54 2010 From: jkuhnert at gmail.com (Jesse Kuhnert) Date: Fri, 9 Jul 2010 16:33:54 -0400 Subject: Primitives in Generics In-Reply-To: References: <4C36465C.6090809@oracle.com> <4C3778EB.60405@cs.oswego.edu> <4C377AB5.7050205@oracle.com> Message-ID: Sorry, to clarify. It "feels" like most people who like to tinker in other languages - even if on the side or smaller portions of projects - tend to not take anything seriously that doesn't have common language support for things like "closures" and parallel capabilities. Given this choice of C you might start to assume Java is just going to kind of limp along but Oracle isn't seriously attempting do anything but minimal / easy (from a user acceptance / scary unknown / meeting deadlines point of view) changes. Given that thought, how much longer can people continue to look at the jvm as a viable runtime? (well, realistically right now probably for a while still.. ;) ) There are more projects coming out for this as well http://llvm.org/ . I'm probably just frustrated and pouting, but seems kind of weird to have seen all of this work only to to finish it up like this. Guessing "someone from above" made the decision for you guys. Or hoping. On Fri, Jul 9, 2010 at 3:51 PM, Jesse Kuhnert wrote: > Really? > > On Fri, Jul 9, 2010 at 3:38 PM, Brian Goetz wrote: >>>> ... some mad professor cooked up :) ?:) ?:) >>> >>> My take is that you have three choices: >>> ? ?A) Support function types covering primitives >>> ? ?B) Support primitives in generics >>> ? ?C) Do something that doesn't restrict your options to reconsider >>> ? ? ? (A) or (B) in Java8 in response to all the complaints about >>> ? ? ? sucky parallel performance. >> >> Indeed so. ?We believe we have chosen (C). ?If we have not, please say >> something! >> >> > From brian.goetz at oracle.com Fri Jul 9 13:59:20 2010 From: brian.goetz at oracle.com (Brian Goetz) Date: Fri, 09 Jul 2010 16:59:20 -0400 Subject: Primitives in Generics In-Reply-To: References: <4C36465C.6090809@oracle.com> <4C3778EB.60405@cs.oswego.edu> <4C377AB5.7050205@oracle.com> Message-ID: <4C378DA8.7060409@oracle.com> At the risk of stating the blindingly obvious... We are constrained by many things: - Source and/or binary compatibility - "Spiritual" compatibility (i.e., don't too badly violate the common man's idea of "what Java is" or make Java too complex for the market segment in which it has succeeded) - Specifiability, Testability, Implementability, Teachability - Budget constraints - Time constraints - Political constraints - Technical constraints - Quality constraints - Cross-dependencies with other initiatives going on inside the JDK development effort or other initiatives at Oracle - etc etc etc Some of these constraints we can talk about publicly (e.g., compatibility); others are confidential (e.g., budgets). On the other hand, the majority of the participants on this list are constrained only by: - What they want Java to be if they had infinite time and resources Now, even if I agreed 100% with someone else's vision for what Java should be, reality frequently forces us to accept less in the name of shipping software in our lifetime. This isn't because we don't understand the vision, or don't care about Java, or simply are morons, but because at the end of the day, what matters is what you ship, not what you talk about shipping in the future. Without triage decisions of what features to leave out, we would never ship -- the set of "worthwhile" features is infinite! And not shipping is bad not only for Oracle, but for the Java community as a whole, since they never get the benefit of any innovation, just lots of promises about how good tomorrow will be. In short, you probably are just frustrated and pouting. Frustrated is OK; life is imperfect. As to this specific decision, to not solve A or B in JDK 7 and instead leave it to be tomorrow's problem -- of course we would like to see it solved now just as much as you do (duh!) You can blame it on the "guys upstairs" if you like; I think it is better to solve something in 7 and something more in 8 than to never ship anything. So let's please focus on the half-full part of the glass. We've got a plan here for: - lambda expressions - SAM conversion - Exception transparency - Interface evolution - Method references That sounds like some good stuff to me. On 7/9/2010 4:33 PM, Jesse Kuhnert wrote: > Sorry, to clarify. > > It "feels" like most people who like to tinker in other languages - > even if on the side or smaller portions of projects - tend to not take > anything seriously that doesn't have common language support for > things like "closures" and parallel capabilities. > > Given this choice of C you might start to assume Java is just going to > kind of limp along but Oracle isn't seriously attempting do anything > but minimal / easy (from a user acceptance / scary unknown / meeting > deadlines point of view) changes. Given that thought, how much longer > can people continue to look at the jvm as a viable runtime? (well, > realistically right now probably for a while still.. ;) ) There are > more projects coming out for this as well http://llvm.org/ . > > I'm probably just frustrated and pouting, but seems kind of weird to > have seen all of this work only to to finish it up like this. > Guessing "someone from above" made the decision for you guys. Or > hoping. > > On Fri, Jul 9, 2010 at 3:51 PM, Jesse Kuhnert wrote: >> Really? >> >> On Fri, Jul 9, 2010 at 3:38 PM, Brian Goetz wrote: >>>>> ... some mad professor cooked up :) :) :) >>>> >>>> My take is that you have three choices: >>>> A) Support function types covering primitives >>>> B) Support primitives in generics >>>> C) Do something that doesn't restrict your options to reconsider >>>> (A) or (B) in Java8 in response to all the complaints about >>>> sucky parallel performance. >>> >>> Indeed so. We believe we have chosen (C). If we have not, please say >>> something! >>> >>> >> From jkuhnert at gmail.com Fri Jul 9 14:13:34 2010 From: jkuhnert at gmail.com (Jesse Kuhnert) Date: Fri, 9 Jul 2010 17:13:34 -0400 Subject: Primitives in Generics In-Reply-To: <4C378DA8.7060409@oracle.com> References: <4C36465C.6090809@oracle.com> <4C3778EB.60405@cs.oswego.edu> <4C377AB5.7050205@oracle.com> <4C378DA8.7060409@oracle.com> Message-ID: Those are Nice things, just thought we were getting closures. On Friday, July 9, 2010, Brian Goetz wrote: > At the risk of stating the blindingly obvious... > > We are constrained by many things: > ?- Source and/or binary compatibility > ?- "Spiritual" compatibility (i.e., don't too badly violate the common man's idea of "what Java is" or make Java too complex for the market segment in which it has succeeded) > ?- Specifiability, Testability, Implementability, Teachability > ?- Budget constraints > ?- Time constraints > ?- Political constraints > ?- Technical constraints > ?- Quality constraints > ?- Cross-dependencies with other initiatives going on inside the JDK development effort or other initiatives at Oracle > ?- etc etc etc > > Some of these constraints we can talk about publicly (e.g., compatibility); others are confidential (e.g., budgets). > > On the other hand, the majority of the participants on this list are constrained only by: > > ?- What they want Java to be if they had infinite time and resources > > Now, even if I agreed 100% with someone else's vision for what Java should be, reality frequently forces us to accept less in the name of shipping software in our lifetime. ?This isn't because we don't understand the vision, or don't care about Java, or simply are morons, but because at the end of the day, what matters is what you ship, not what you talk about shipping in the future. > > Without triage decisions of what features to leave out, we would never ship -- the set of "worthwhile" features is infinite! ?And not shipping is bad not only for Oracle, but for the Java community as a whole, since they never get the benefit of any innovation, just lots of promises about how good tomorrow will be. > > In short, you probably are just frustrated and pouting. ?Frustrated is OK; life is imperfect. > > As to this specific decision, to not solve A or B in JDK 7 and instead leave it to be tomorrow's problem -- of course we would like to see it solved now just as much as you do (duh!) ?You can blame it on the "guys upstairs" if you like; I think it is better to solve something in 7 and something more in 8 than to never ship anything. > > So let's please focus on the half-full part of the glass. ?We've got a plan here for: > ?- lambda expressions > ?- SAM conversion > ?- Exception transparency > ?- Interface evolution > ?- Method references > > That sounds like some good stuff to me. > > > On 7/9/2010 4:33 PM, Jesse Kuhnert wrote: > > Sorry, to clarify. > > It "feels" like most people who like to tinker in other languages - > even if on the side or smaller portions of projects - tend to not take > anything seriously that doesn't have common language support for > things like "closures" and parallel capabilities. > > Given this choice of C you might start to assume Java is just going to > kind of limp along but Oracle isn't seriously attempting do anything > but minimal / easy (from a user acceptance / scary unknown / meeting > deadlines point of view) changes. Given that thought, how much longer > can people continue to look at the jvm as a viable runtime? (well, > realistically right now probably for a while still.. ;) ) There are > more projects coming out for this as well http://llvm.org/ . > > I'm probably just frustrated and pouting, but seems kind of weird to > have seen all of this work only to to finish it up like this. > Guessing "someone from above" made the decision for you guys. Or > hoping. > > On Fri, Jul 9, 2010 at 3:51 PM, Jesse Kuhnert ?wrote: > > Really? > > On Fri, Jul 9, 2010 at 3:38 PM, Brian Goetz ?wrote: > > ... some mad professor cooked up :) ?:) ?:) > > > My take is that you have three choices: > ? ?A) Support function types covering primitives > ? ?B) Support primitives in generics > ? ?C) Do something that doesn't restrict your options to reconsider > ? ? ? (A) or (B) in Java8 in response to all the complaints about > ? ? ? sucky parallel performance. > > > Indeed so. ?We believe we have chosen (C). ?If we have not, please say > something! > > > > > > From alex.blewitt at gmail.com Fri Jul 9 14:18:19 2010 From: alex.blewitt at gmail.com (Alex Blewitt) Date: Fri, 9 Jul 2010 22:18:19 +0100 Subject: Yield considered harmful Message-ID: <2A48108C-0A72-4EC7-B4F6-1A4542F1DC2B@gmail.com> As has been noted elsewhere, the inconsistency between 'this' as a reference to the SAM instance and 'yield' as a way of return-from-lambda is not a good development. Given that 'this' is already defiend, why not use 'return' as well? Almost all Java developers will understand this immediately; and it doesn't prevent the kind of macro use that has been described about elsewhere in the future. Furthermore, 'yield' already has an existant meaning in Java: http://download.oracle.com/docs/cd/E17476_01/javase/1.5.0/docs/api/java/lang/Thread.html#yield() "Causes the currently executing thread object to temporarily pause and allow other threads to execute." Having a keyword mean something completely different from a well defined term in multi-threading (and one that's been there since Java 1.0 days) seems folly at best. Furthermore, note that many people will want to take advantage of the smaller syntax in existing filter operations. For example, Google's guava provides Predicate http://guava-libraries.googlecode.com/svn/trunk/javadoc/com/google/common/base/Predicate.html Examples like: Sets.filter(set, new Predicate() { boolean apply(Object o) { if (o instanceof String && ((String)o).contains("http") ) return true; return false; } } ); could easily take advantage of the new syntax: Sets.filter(set, { o -> if(o instanceof String && ((String)o).contains( "http")) yield true; yield false } ) however, note that as well as removing the boilerplate, we also have to re-write the 'return' for 'yield' for no practically obvious reason. Even if non-local returns are added at a later stage, there's no reason that that could not use a new syntax and avoid the complications of misusing return now. Alex From neal at gafter.com Fri Jul 9 14:23:57 2010 From: neal at gafter.com (Neal Gafter) Date: Fri, 9 Jul 2010 14:23:57 -0700 Subject: Primitives in Generics In-Reply-To: References: <4C36465C.6090809@oracle.com> <4C3778EB.60405@cs.oswego.edu> <4C377AB5.7050205@oracle.com> <4C378DA8.7060409@oracle.com> Message-ID: Jesse: "closures" is slang for lambda expressions, which means lexically-scoped function-valued expressions. Unfortunately, what is Brian is now calling a lambda expression is neither lexically scoped nor function-valued. Cheers, Neal On Fri, Jul 9, 2010 at 2:13 PM, Jesse Kuhnert wrote: > Those are Nice things, just thought we were getting closures. > > On Friday, July 9, 2010, Brian Goetz wrote: > > At the risk of stating the blindingly obvious... > > > > We are constrained by many things: > > - Source and/or binary compatibility > > - "Spiritual" compatibility (i.e., don't too badly violate the common > man's idea of "what Java is" or make Java too complex for the market segment > in which it has succeeded) > > - Specifiability, Testability, Implementability, Teachability > > - Budget constraints > > - Time constraints > > - Political constraints > > - Technical constraints > > - Quality constraints > > - Cross-dependencies with other initiatives going on inside the JDK > development effort or other initiatives at Oracle > > - etc etc etc > > > > Some of these constraints we can talk about publicly (e.g., > compatibility); others are confidential (e.g., budgets). > > > > On the other hand, the majority of the participants on this list are > constrained only by: > > > > - What they want Java to be if they had infinite time and resources > > > > Now, even if I agreed 100% with someone else's vision for what Java > should be, reality frequently forces us to accept less in the name of > shipping software in our lifetime. This isn't because we don't understand > the vision, or don't care about Java, or simply are morons, but because at > the end of the day, what matters is what you ship, not what you talk about > shipping in the future. > > > > Without triage decisions of what features to leave out, we would never > ship -- the set of "worthwhile" features is infinite! And not shipping is > bad not only for Oracle, but for the Java community as a whole, since they > never get the benefit of any innovation, just lots of promises about how > good tomorrow will be. > > > > In short, you probably are just frustrated and pouting. Frustrated is > OK; life is imperfect. > > > > As to this specific decision, to not solve A or B in JDK 7 and instead > leave it to be tomorrow's problem -- of course we would like to see it > solved now just as much as you do (duh!) You can blame it on the "guys > upstairs" if you like; I think it is better to solve something in 7 and > something more in 8 than to never ship anything. > > > > So let's please focus on the half-full part of the glass. We've got a > plan here for: > > - lambda expressions > > - SAM conversion > > - Exception transparency > > - Interface evolution > > - Method references > > > > That sounds like some good stuff to me. > > > > > > On 7/9/2010 4:33 PM, Jesse Kuhnert wrote: > > > > Sorry, to clarify. > > > > It "feels" like most people who like to tinker in other languages - > > even if on the side or smaller portions of projects - tend to not take > > anything seriously that doesn't have common language support for > > things like "closures" and parallel capabilities. > > > > Given this choice of C you might start to assume Java is just going to > > kind of limp along but Oracle isn't seriously attempting do anything > > but minimal / easy (from a user acceptance / scary unknown / meeting > > deadlines point of view) changes. Given that thought, how much longer > > can people continue to look at the jvm as a viable runtime? (well, > > realistically right now probably for a while still.. ;) ) There are > > more projects coming out for this as well http://llvm.org/ . > > > > I'm probably just frustrated and pouting, but seems kind of weird to > > have seen all of this work only to to finish it up like this. > > Guessing "someone from above" made the decision for you guys. Or > > hoping. > > > > On Fri, Jul 9, 2010 at 3:51 PM, Jesse Kuhnert > wrote: > > > > Really? > > > > On Fri, Jul 9, 2010 at 3:38 PM, Brian Goetz > wrote: > > > > ... some mad professor cooked up :) :) :) > > > > > > My take is that you have three choices: > > A) Support function types covering primitives > > B) Support primitives in generics > > C) Do something that doesn't restrict your options to reconsider > > (A) or (B) in Java8 in response to all the complaints about > > sucky parallel performance. > > > > > > Indeed so. We believe we have chosen (C). If we have not, please say > > something! > > > > > > > > > > > > > > From nathan.bryant at linkshare.com Fri Jul 9 14:26:41 2010 From: nathan.bryant at linkshare.com (Nathan Bryant) Date: Sat, 10 Jul 2010 06:26:41 +0900 Subject: Yield considered harmful References: <2A48108C-0A72-4EC7-B4F6-1A4542F1DC2B@gmail.com> Message-ID: <7FDA6630E1822F448C97A48D5D7330940102078A@EXVMSTOR302.intra.rakuten.co.jp> > Sets.filter(set, new Predicate() { > boolean apply(Object o) { > if (o instanceof String && ((String)o).contains("http") ) > return true; > return false; > } > } ); > could easily take advantage of the new syntax: > Sets.filter(set, { o -> if(o instanceof String && ((String)o).contains( "http")) yield true; yield false } ) Not necessary, when one can write it like this: Sets.filter(set, { o -> o instanceof String && ((String)o).contains( "http") } ) Or in the more common case where your Set is a Set, you can write: Sets.filter(set, { o -> o.contains("http") } ) That said, yield is bad. -1 on yield. (If an additional -1 vote is even necessary now that you've pointed out Thread.yield) From int19h at gmail.com Fri Jul 9 14:38:49 2010 From: int19h at gmail.com (Pavel Minaev) Date: Fri, 9 Jul 2010 14:38:49 -0700 Subject: Primitives in Generics In-Reply-To: References: <4C36465C.6090809@oracle.com> <4C3778EB.60405@cs.oswego.edu> <4C377AB5.7050205@oracle.com> <4C378DA8.7060409@oracle.com> Message-ID: On Fri, Jul 9, 2010 at 2:23 PM, Neal Gafter wrote: > "closures" is slang for lambda expressions, which means lexically-scoped > function-valued expressions. ?Unfortunately, what is Brian is now calling a > lambda expression is neither lexically scoped nor function-valued. Pragmatically, "closures" is slang for "those cool function-like things like in JS/Ruby/C#/...". Few programmers in the wild are actually interested in conceptual purity. It doesn't matter if some bits are not "properly" lexically scoped, if all (or even most) of those that matter are. By the way, why do you say that the proposed lambda expressions aren't function-valued? Is that because they don't have an expressible type? If so, then do you imply that C# lambdas aren't closures, either? From mthornton at optrak.co.uk Fri Jul 9 14:44:23 2010 From: mthornton at optrak.co.uk (Mark Thornton) Date: Fri, 09 Jul 2010 22:44:23 +0100 Subject: Yield considered harmful In-Reply-To: <2A48108C-0A72-4EC7-B4F6-1A4542F1DC2B@gmail.com> References: <2A48108C-0A72-4EC7-B4F6-1A4542F1DC2B@gmail.com> Message-ID: <4C379837.1020301@optrak.co.uk> Alex Blewitt wrote: > As has been noted elsewhere, the inconsistency between 'this' as a reference to the SAM instance and 'yield' as a way of return-from-lambda is not a good development. Given that 'this' is already defiend, why not use 'return' as well? Almost all Java developers will understand this immediately; and it doesn't prevent the kind of macro use that has been described about elsewhere in the future. > > Furthermore, 'yield' already has an existant meaning in Java: > > http://download.oracle.com/docs/cd/E17476_01/javase/1.5.0/docs/api/java/lang/Thread.html#yield() > > "Causes the currently executing thread object to temporarily pause and allow other threads to execute." > > Having a keyword mean something completely different from a well defined term in multi-threading (and one that's been there since Java 1.0 days) seems folly at best. > On the other hand Thread.yield() is so rarely used that very few Java programmers are likely to be bothered by the additional meaning. I wonder how many people have used it to any practical effect in the last five years. That said, I'm also not keen on the proposed use of yield. Mark Thornton From lk at teamten.com Fri Jul 9 14:49:25 2010 From: lk at teamten.com (Lawrence Kesteloot) Date: Fri, 9 Jul 2010 14:49:25 -0700 Subject: Yield considered harmful In-Reply-To: <2A48108C-0A72-4EC7-B4F6-1A4542F1DC2B@gmail.com> References: <2A48108C-0A72-4EC7-B4F6-1A4542F1DC2B@gmail.com> Message-ID: On Fri, Jul 9, 2010 at 2:18 PM, Alex Blewitt wrote: > Furthermore, 'yield' already has an existant meaning in Java: Additionally, in Python "yield" is used in generators. If we ever want those in Java we'll be sad to not have a good keyword for it. > Even if non-local returns are added at a later stage, there's no reason that that could not use a new syntax and avoid the complications of misusing return now. That would violate what Neal calls "transparency": for (String word : words ) { if (word.contains("http")) { return word; } } would be re-written as: words.forEach() { word -> if (word.contains("http")) { long return word; } } To use your phrase, we have to re-write the "return" for "long return" for no practically obvious reason. Broadly speaking it seems like we have at least three options: 1. "yield" or similar for local return, "return" for lexical return. Has the problems you point out. 2. "return" for local return, "long return" for lexical return. Violates transparency. 3. Naked expression for local return, "return" for lexical return. Has some of the problems you point out and is (IMO) awkward-looking in all but the simplest examples (like Nathan's in this thread). Given these trade-offs, I feel like option 2 is best, possibly even to the point of having "long continue" and "long break" later. Well, to be honest, my actual preference is to use CICE as-is for project lambda, which makes it very clear that "return" should be local since it looks like a function definition (the name of the SAM and parentheses). Later if we introduce blocks, we can treat those very differently because they won't look anything like a function -- they'll look more like a block and a plain "return" there can be lexical. But it seems like the desire to remove the SAM name from lambdas is too strong and we're forced into a situation which will certainly be confusing to programmers down the line, whatever we choose now. Lawrence From int19h at gmail.com Fri Jul 9 15:13:32 2010 From: int19h at gmail.com (Pavel Minaev) Date: Fri, 9 Jul 2010 15:13:32 -0700 Subject: Yield considered harmful In-Reply-To: References: <2A48108C-0A72-4EC7-B4F6-1A4542F1DC2B@gmail.com> Message-ID: On Fri, Jul 9, 2010 at 2:49 PM, Lawrence Kesteloot wrote: > On Fri, Jul 9, 2010 at 2:18 PM, Alex Blewitt wrote: >> Furthermore, 'yield' already has an existant meaning in Java: > > Additionally, in Python "yield" is used in generators. If we ever want > those in Java we'll be sad to not have a good keyword for it. > >> Even if non-local returns are added at a later stage, there's no reason that that could not use a new syntax and avoid the complications of misusing return now. > > That would violate what Neal calls "transparency": > > for (String word : words ) { > ? ?if (word.contains("http")) { > ? ? ? ?return word; > ? ?} > } > > would be re-written as: > > words.forEach() { word -> > ? ?if (word.contains("http")) { > ? ? ? ?long return word; > ? ?} > } The hypothetical new syntax does not need to apply to "return" - it can apply to the lambda as a whole. E.g.: words.forEach() {# word -> ? ?if (word.contains("http")) { ? ? ? ?return word; ? ?} } where # is a placeholder for something that distinguishes this form of lambda (a "block" in Ruby parlance) from a proper function. The meaning of "return" in the body of such a lambda is then adjusted, and "break" and "continue" are permitted. Such a lambda expression could furthermore have a distinct type, which could, for example, be used to prevent it from escaping the scope in which "break" and "continue" are meaningful (by restricting that type for return values, fields, captured mutable locals etc). In the following list, that would be a distinct option #4. > 1. "yield" or similar for local return, "return" for lexical return. > Has the problems you point out. > 2. "return" for local return, "long return" for lexical return. > Violates transparency. > 3. Naked expression for local return, "return" for lexical return. Has > some of the problems you point out and is (IMO) awkward-looking in all > but the simplest examples (like Nathan's in this thread). From neal at gafter.com Fri Jul 9 15:23:47 2010 From: neal at gafter.com (Neal Gafter) Date: Fri, 9 Jul 2010 15:23:47 -0700 Subject: Primitives in Generics In-Reply-To: References: <4C36465C.6090809@oracle.com> <4C3778EB.60405@cs.oswego.edu> <4C377AB5.7050205@oracle.com> <4C378DA8.7060409@oracle.com> Message-ID: On Fri, Jul 9, 2010 at 2:38 PM, Pavel Minaev wrote: > On Fri, Jul 9, 2010 at 2:23 PM, Neal Gafter wrote: > > "closures" is slang for lambda expressions, which means lexically-scoped > > function-valued expressions. Unfortunately, what is Brian is now calling > a > > lambda expression is neither lexically scoped nor function-valued. > > Pragmatically, "closures" is slang for "those cool function-like > things like in JS/Ruby/C#/...". Few programmers in the wild are > actually interested in conceptual purity. It doesn't matter if some > bits are not "properly" lexically scoped, if all (or even most) of > those that matter are. > Well, perhaps nothing really matters at all. Few programmers in the wild are actually interested in the language specification until the issues begin to interfere with their effective use of the language. It is the language designer's job to make sure that doesn't happen. By the way, why do you say that the proposed lambda expressions aren't > function-valued? Is that because they don't have an expressible type? > It is because they aren't of a function type, which is because function types have been dropped. > If so, then do you imply that C# lambdas aren't closures, either? > C# lambdas are indeed of "delegate" types, which are function types. From int19h at gmail.com Fri Jul 9 15:36:26 2010 From: int19h at gmail.com (Pavel Minaev) Date: Fri, 9 Jul 2010 15:36:26 -0700 Subject: Primitives in Generics In-Reply-To: References: <4C36465C.6090809@oracle.com> <4C3778EB.60405@cs.oswego.edu> <4C377AB5.7050205@oracle.com> <4C378DA8.7060409@oracle.com> Message-ID: On Fri, Jul 9, 2010 at 3:23 PM, Neal Gafter wrote: >> By the way, why do you say that the proposed lambda expressions aren't >> function-valued? Is that because they don't have an expressible type? > > It is because they aren't of a function type, which is because function > types have been dropped. >> >> If so, then do you imply that C# lambdas aren't closures, either? > > C# lambdas are indeed of "delegate" types, which are function types. C# lambda expressions and anonymous delegate expressions are not of delegate types. C# 4.0 language specification, section 7.15 "Anonymous function expressions": "An anonymous function is an expression that represents an ?in-line? method definition. An anonymous function does not have a value or type in and of itself, but is convertible to a compatible delegate or expression tree type." Consequently, you cannot e.g. write: object f = (() => 123); and VC# compiler will even state the reason rather bluntly: error CS1660: Cannot convert lambda expression to type 'object' because it is not a delegate type Indeed, how could it be any other way, given that C# delegate types are nominal (and not synthesized), and there could be many (or none!) delegate types declared with a signature matching the lambda? So far as I can see, C# has precisely the same limitations here as the proposed no-function-types-but-SAM-conversion for Java. If this isn't good enough to be properly called "closures", fine - though then the definition of "closure" used is clearly different (and quite possibly more correct, I don't want to argue this point) from the popular one. The point, anyway, is that C# has had this construct, whatever the name, for 5 years now, and it has a success story to tell. If Java "closure-like construct" is on par with that, it's good enough for me and all the people who have been happy with that aspect of C#. From forax at univ-mlv.fr Fri Jul 9 15:42:49 2010 From: forax at univ-mlv.fr (=?ISO-8859-1?Q?R=E9mi_Forax?=) Date: Sat, 10 Jul 2010 00:42:49 +0200 Subject: Primitives in Generics In-Reply-To: <4C377AB5.7050205@oracle.com> References: <4C36465C.6090809@oracle.com> <4C3778EB.60405@cs.oswego.edu> <4C377AB5.7050205@oracle.com> Message-ID: <4C37A5E9.8040404@univ-mlv.fr> Le 09/07/2010 21:38, Brian Goetz a ?crit : >>> ... some mad professor cooked up :) :) :) >>> >> My take is that you have three choices: >> A) Support function types covering primitives >> B) Support primitives in generics >> C) Do something that doesn't restrict your options to reconsider >> (A) or (B) in Java8 in response to all the complaints about >> sucky parallel performance. >> > Indeed so. We believe we have chosen (C). If we have not, please say > something! > > something! :) The latest proposal don't tell the story to go from C to A or B, and I doubt it's possible. Let suppose, we want to add 'apply' to collection API, with the proposed scheme, we will have something like this: interface Function { void invoke(T element); } interface Collection { ... extension void apply(Function function) default Collections.apply; } class Collections { ... static void apply(Collection c, Function function) { for(T element: c) { function.invoke(element); } } } I don't see a *binary* backward compatible way to introduce function types later. R?mi From neal at gafter.com Fri Jul 9 16:00:10 2010 From: neal at gafter.com (Neal Gafter) Date: Fri, 9 Jul 2010 16:00:10 -0700 Subject: Primitives in Generics In-Reply-To: References: <4C36465C.6090809@oracle.com> <4C3778EB.60405@cs.oswego.edu> <4C377AB5.7050205@oracle.com> <4C378DA8.7060409@oracle.com> Message-ID: On Fri, Jul 9, 2010 at 3:36 PM, Pavel Minaev wrote: > C# lambda expressions and anonymous delegate expressions are not of > delegate types. C# 4.0 language specification, section 7.15 "Anonymous > function expressions": > > "An anonymous function is an expression that represents an > ?in-line? method definition. An anonymous function does not have a > value or type in and of itself, but is convertible to a compatible > delegate or expression tree type." > That is compatible with what I described. Any given lambda expression in any given valid C# program has a delegate type which is selected by target-typing. But the target type is required (i.e., the lambda does not have a type in and of itself independent of the target of the conversion). BGGA works that way too; that aspect of project lambda does not bother me at all. So far as I can see, C# has precisely the same limitations here as the > proposed no-function-types-but-SAM-conversion for Java. C# lambdas bind names lexically. The proposed lambdas do not. C# lambdas bind "this" lexically. The proposed lambdas do not. C# "function types" (delegates) are typically defined to be covariant in return types and contravariant in argument types. SAMs cannot be so defined. C#'s generic delegate types such as Func can specialize over primitives and over different numbers of arguments. SAMs cannot. C# lambdas can bind mutable local variables. The proposed lambdas do not. C# lambdas can infer lambda argument types even when passed as an argument to an overloaded method. The proposed lambdas do not. The point, anyway, is that C# has had this construct, whatever the > name, for 5 years now, and it has a success story to tell. If Java > "closure-like construct" is on par with that, it's good enough for me > and all the people who have been happy with that aspect of C#. > It doesn't appear that it is likely to be on par. From jkuhnert at gmail.com Fri Jul 9 16:08:37 2010 From: jkuhnert at gmail.com (Jesse Kuhnert) Date: Fri, 9 Jul 2010 19:08:37 -0400 Subject: Primitives in Generics In-Reply-To: References: <4C36465C.6090809@oracle.com> <4C3778EB.60405@cs.oswego.edu> <4C377AB5.7050205@oracle.com> <4C378DA8.7060409@oracle.com> Message-ID: Indeed, I have a feeling that anyone with a keyboard and search engine is going to have a hard time matching up the current proposal with the promise of closures - if and when it does make it out into the wild. Java developers get ragged on enough as it is, no one can say for ~sure~ what the reaction will be except maybe time and loss of market share/respect. Certainly not the most glorious first major release for Oracle to show the world what the new deal will be with java, but the market has rewarded them so far so why would it stop? ( of course there's that pesky issue of not having sun around anymore to feed off of but that probably won't matter) On Friday, July 9, 2010, Neal Gafter wrote: > On Fri, Jul 9, 2010 at 3:36 PM, Pavel Minaev wrote: > > > C# lambda expressions and anonymous delegate expressions are not of > delegate types. C# 4.0 language specification, section 7.15 "Anonymous > function expressions": > > ? ?"An anonymous function is an expression that represents an > ?in-line? method definition. An anonymous function does not have a > value or type in and of itself, but is convertible to a compatible > delegate or expression tree type." > > That is compatible with what I described.? Any given lambda expression in any given valid C# program has a > delegate type which is selected by target-typing.? But the target type is required (i.e., the lambda does not have a type in and of itself independent of the target of the conversion).? BGGA works that way too; that aspect of project lambda does not bother me at all. > > > So far as I can see, C# has precisely the same limitations here as the > proposed no-function-types-but-SAM-conversion for Java. > C# lambdas bind names lexically.? The proposed lambdas do not. > C# lambdas bind "this" lexically.? The proposed lambdas do not. > C# "function types" (delegates) are typically defined to be covariant in return types and contravariant in argument types.? SAMs cannot be so defined. > C#'s generic delegate types such as Func can specialize over primitives and over different numbers of arguments.? SAMs cannot. > C# lambdas can bind mutable local variables.? The proposed lambdas do not. > C# lambdas can infer lambda argument types even when passed as an argument to an overloaded method.? The proposed lambdas do not. > > > > The point, anyway, is that C# has had this construct, whatever the > name, for 5 years now, and it has a success story to tell. If Java > "closure-like construct" is on par with that, it's good enough for me > and all the people who have been happy with that aspect of C#. > > It doesn't appear that it is likely to be on par. > > From int19h at gmail.com Fri Jul 9 16:18:08 2010 From: int19h at gmail.com (Pavel Minaev) Date: Fri, 9 Jul 2010 16:18:08 -0700 Subject: Primitives in Generics In-Reply-To: References: <4C36465C.6090809@oracle.com> <4C3778EB.60405@cs.oswego.edu> <4C377AB5.7050205@oracle.com> <4C378DA8.7060409@oracle.com> Message-ID: On Fri, Jul 9, 2010 at 4:00 PM, Neal Gafter wrote: > On Fri, Jul 9, 2010 at 3:36 PM, Pavel Minaev wrote: >> >> C# lambda expressions and anonymous delegate expressions are not of >> delegate types. C# 4.0 language specification, section 7.15 "Anonymous >> function expressions": >> >> ? ?"An anonymous function is an expression that represents an >> ?in-line? method definition. An anonymous function does not have a >> value or type in and of itself, but is convertible to a compatible >> delegate or expression tree type." > > That is compatible with what I described.? Any given lambda expression in > any given valid C# program has a delegate type which is selected by > target-typing.? But the target type is required (i.e., the lambda does not > have a type in and of itself independent of the target of the conversion). > BGGA works that way too; that aspect of project lambda does not bother me at > all. You previously wrote: > what is Brian is now calling a lambda expression is neither lexically scoped nor function-valued ... > because they aren't of a function type, which is because function types have been dropped. We have already established that neither proposed Java nor C# lambdas have function types per se, but obtain them via target-typing. So your claim then boils down to "SAMs are not function types". And why aren't they? Why shouldn't C# delegate types rather be treated as mere syntactic sugar for a single-method interface or abstract class? >> So far as I can see, C# has precisely the same limitations here as the >> proposed no-function-types-but-SAM-conversion for Java. > > C# lambdas bind names lexically.? The proposed lambdas do not. Do you mean SAM type members? > C# lambdas bind "this" lexically.? The proposed lambdas do not. I don't like it, either, and hope it will be changed by the time of release, but it's far from being a major issue, in my opinion. > C# "function types" (delegates) are typically defined to be covariant in > return types and contravariant in argument types.? SAMs cannot be so > defined. That boils down to declaration-site variance in C# vs use-site variance in Java. SAMs cannot be so defined, but nothing prevents you from using wildcards when referencing them to achieve the same effect. > C#'s generic delegate types such as Func can specialize over > primitives and over different numbers of arguments.? SAMs cannot. That is a separate problem which is also much broader, and should be solved as such (i.e. reify generics, then you get that for function types for free - any other solution is going to be partial, anyway). > C# lambdas can bind mutable local variables.? The proposed lambdas do not. > C# lambdas can infer lambda argument types even when passed as an argument > to an overloaded method.? The proposed lambdas do not. Both are negative, I agree. But, anyway, I do not want to get into the discussion on all those points, here. I only wanted to clarify the claim that dropping structural function types somehow changes the status of lambdas such that they are no longer proper closures. From neal at gafter.com Fri Jul 9 16:31:25 2010 From: neal at gafter.com (Neal Gafter) Date: Fri, 9 Jul 2010 16:31:25 -0700 Subject: Primitives in Generics In-Reply-To: References: <4C36465C.6090809@oracle.com> <4C3778EB.60405@cs.oswego.edu> <4C377AB5.7050205@oracle.com> <4C378DA8.7060409@oracle.com> Message-ID: On Fri, Jul 9, 2010 at 4:18 PM, Pavel Minaev wrote: > We have already established that neither proposed Java nor C# lambdas > have function types per se, but obtain them via target-typing. So your > claim then boils down to "SAMs are not function types". And why aren't > they? Why shouldn't C# delegate types rather be treated as mere > syntactic sugar for a single-method interface or abstract class? > Are you proposing a language change for C#? If it would introduce into C# all of the disadvantages of project lambda's specification, I don't think that would be worth doing. > >> So far as I can see, C# has precisely the same limitations here as the > >> proposed no-function-types-but-SAM-conversion for Java. > > > > C# lambdas bind names lexically. The proposed lambdas do not. > > Do you mean SAM type members? > I don't understand your question. I'm referring to the binding of free names appearing in a lambda expression, which in the proposal are first sought in the scope of the target type (i.e. not lexically). > > C# "function types" (delegates) are typically defined to be covariant in > > return types and contravariant in argument types. SAMs cannot be so > > defined. > > That boils down to declaration-site variance in C# vs use-site > variance in Java. SAMs cannot be so defined, but nothing prevents you > from using wildcards when referencing them to achieve the same effect. > Nothing prevents you from programming in assembly language either. It isn't very practical to expect programmers to do so correctly or consistently; the purpose of a programming language extension is to make things easier for the programmer, not harder. > > C#'s generic delegate types such as Func can specialize over > > primitives and over different numbers of arguments. SAMs cannot. > > That is a separate problem which is also much broader, and should be > solved as such (i.e. reify generics, then you get that for function > types for free - any other solution is going to be partial, anyway). > This has nothing to do with reification. BGGA addresses the issue with its function types but does not reify. From int19h at gmail.com Fri Jul 9 16:41:16 2010 From: int19h at gmail.com (Pavel Minaev) Date: Fri, 9 Jul 2010 16:41:16 -0700 Subject: Primitives in Generics In-Reply-To: References: <4C36465C.6090809@oracle.com> <4C3778EB.60405@cs.oswego.edu> <4C377AB5.7050205@oracle.com> <4C378DA8.7060409@oracle.com> Message-ID: On Fri, Jul 9, 2010 at 4:31 PM, Neal Gafter wrote: > On Fri, Jul 9, 2010 at 4:18 PM, Pavel Minaev wrote: >> >> We have already established that neither proposed Java nor C# lambdas >> have function types per se, but obtain them via target-typing. So your >> claim then boils down to "SAMs are not function types". And why aren't >> they? Why shouldn't C# delegate types rather be treated as mere >> syntactic sugar for a single-method interface or abstract class? > > Are you proposing a language change for C#?? If it would introduce into C# > all of the disadvantages of project lambda's specification, I don't think > that would be worth doing. No, I'm asking what is the practical difference between a delegate type and a single-method interface, in the context of function types. >> >> So far as I can see, C# has precisely the same limitations here as the >> >> proposed no-function-types-but-SAM-conversion for Java. >> > >> > C# lambdas bind names lexically.? The proposed lambdas do not. >> >> Do you mean SAM type members? > > I don't understand your question.? I'm referring to the binding of free > names appearing in a lambda expression, which in the proposal are first > sought in the scope of the target type (i.e. not lexically). That is what I meant. In the typical case of a single-method interface (which would directly correspond to C# delegates; anything above that is "extra" on Java side), we're talking about a single name here that will be found in that scope. Again this seems to be a fairly minor issue - and it has a positive side effect of enabling self-recursive lambdas without the need to pass the lambda to itself as an argument. >> > C# "function types" (delegates) are typically defined to be covariant in >> > return types and contravariant in argument types.? SAMs cannot be so >> > defined. >> >> That boils down to declaration-site variance in C# vs use-site >> variance in Java. SAMs cannot be so defined, but nothing prevents you >> from using wildcards when referencing them to achieve the same effect. > > Nothing prevents you from programming in assembly language either.? It isn't > very practical to expect programmers to do so correctly or consistently; the > purpose of a programming language extension is to make things easier for the > programmer, not harder. Perhaps, but, again, this should be an argument for declaration-site variance for generics in general, and not just for function types. >> > C#'s generic delegate types such as Func can specialize over >> > primitives and over different numbers of arguments.? SAMs cannot. >> >> That is a separate problem which is also much broader, and should be >> solved as such (i.e. reify generics, then you get that for function >> types for free - any other solution is going to be partial, anyway). > > This has nothing to do with reification.? BGGA addresses the issue with its > function types but does not reify. I was under the impression that you need reification to make it properly work for generics _in general_ (and I am quite possibly wrong here, so please correct me if I am). Regardless, the point is that once you make it work for generics in general, it also applies to SAM types under the current proposal - and the utility of such a solution would be far greater. On the other hand, enabling primitives for function types while leaving the same limitations for generics would definitely lead to the question of "why did they solve that problem for the new thing, but didn't also fix it for the old one?". From jkuhnert at gmail.com Fri Jul 9 16:59:52 2010 From: jkuhnert at gmail.com (Jesse Kuhnert) Date: Fri, 9 Jul 2010 19:59:52 -0400 Subject: Primitives in Generics In-Reply-To: References: <4C36465C.6090809@oracle.com> <4C3778EB.60405@cs.oswego.edu> <4C377AB5.7050205@oracle.com> <4C378DA8.7060409@oracle.com> Message-ID: Is that really what they'd ask? Maybe they would, as they happily moved forward with all their newfound productivity and application performance boosts. With the current proposal however I think the questions may be a bit broader and more extreme. Only in cases where they've discovered modern (or more popular now) programming concepts though - which will be almost everyone except for those poor souls who really do fit the java developer stereotype and maybe don't like programming all that much anyway. No matter how it's packaged. > Regardless, the point is that once you make it work for generics in > general, it also applies to SAM types under the current proposal - and > the utility of such a solution would be far greater. On the other > hand, enabling primitives for function types while leaving the same > limitations for generics would definitely lead to the question of "why > did they solve that problem for the new thing, but didn't also fix it > for the old one?". > From jkuhnert at gmail.com Fri Jul 9 18:21:30 2010 From: jkuhnert at gmail.com (Jesse Kuhnert) Date: Fri, 9 Jul 2010 21:21:30 -0400 Subject: Primitives in Generics In-Reply-To: References: <4C36465C.6090809@oracle.com> <4C3778EB.60405@cs.oswego.edu> <4C377AB5.7050205@oracle.com> <4C378DA8.7060409@oracle.com> Message-ID: And quite frankly, this whole mailing list feels a little bit like one of the elves trying to bestow the secrets of the ring before leaving middle earth and getting questions like "but why do you live in trees? That's kind of wierd." The rest of the elves (bracha, Steele, gosling, etc) are already there in various forms. It's kind of insulting to see so many people unable to accept a genuine desire (and knowledge/experience to back it up) to better the language with their own false sense of their own idea of what is right - even if that rightness is mostly about supporting an idea that has long since lost any coherent basis as a real actual good thing in computer science / humane design terms but is mostly there for self ego. ( which is also human but far from the generous spirit of accepting when you are wrong) On Friday, July 9, 2010, Jesse Kuhnert wrote: > Is that really what they'd ask? Maybe they would, as they happily > moved forward with all their newfound productivity and application > performance boosts. > > With the current proposal however I think the questions may be a bit > broader and more extreme. Only in cases where they've discovered > modern (or more popular now) programming concepts though - which will > be almost everyone except for those poor souls who really do fit the > java developer stereotype and maybe don't like programming all that > much anyway. No matter how it's packaged. > >> Regardless, the point is that once you make it work for generics in >> general, it also applies to SAM types under the current proposal - and >> the utility of such a solution would be far greater. On the other >> hand, enabling primitives for function types while leaving the same >> limitations for generics would definitely lead to the question of "why >> did they solve that problem for the new thing, but didn't also fix it >> for the old one?". >> > From john at milsson.nu Sat Jul 10 04:52:04 2010 From: john at milsson.nu (John Nilsson) Date: Sat, 10 Jul 2010 13:52:04 +0200 Subject: Primitives in Generics In-Reply-To: References: <4C36465C.6090809@oracle.com> <4C3778EB.60405@cs.oswego.edu> <4C377AB5.7050205@oracle.com> <4C378DA8.7060409@oracle.com> Message-ID: On Sat, Jul 10, 2010 at 1:31 AM, Neal Gafter wrote: > I'm referring to the binding of free > names appearing in a lambda expression, which in the proposal are first > sought in the scope of the target type (i.e. not lexically). Are you arguing that this is a bad thing? If so: why? To me this seems like a feature that could be quite useful. It's a way to introduce an implicit scope. Maybe it could come in handy for defining small DSLs in Java? >> ?> C#'s generic delegate types such as Func can specialize over >> > primitives and over different numbers of arguments. ?SAMs cannot. >> >> That is a separate problem which is also much broader, and should be >> solved as such (i.e. reify generics, then you get that for function >> types for free - any other solution is going to be partial, anyway). >> > > This has nothing to do with reification. ?BGGA addresses the issue with its > function types but does not reify. There has been a lot of talk about reification to solve some problems. Maybe this is off topic for lambda-dev but I have to ask if my intuition is way of here: To me reification is about bringing static information to the dynamic domain to enable logic based on this information. But if the logic is supposed to be based on static information, wouldn't it be better to bring the logic into the static domain? Why add additional overhead at runtime that could be resolved at compile time? BR, John From dl at cs.oswego.edu Sat Jul 10 11:34:12 2010 From: dl at cs.oswego.edu (Doug Lea) Date: Sat, 10 Jul 2010 14:34:12 -0400 Subject: Primitives in Generics In-Reply-To: <4C377AB5.7050205@oracle.com> References: <4C36465C.6090809@oracle.com> <4C3778EB.60405@cs.oswego.edu> <4C377AB5.7050205@oracle.com> Message-ID: <4C38BD24.2070403@cs.oswego.edu> On 07/09/10 15:38, Brian Goetz wrote: >> My take is that you have three choices: >> A) Support function types covering primitives >> B) Support primitives in generics >> C) Do something that doesn't restrict your options to reconsider >> (A) or (B) in Java8 in response to all the complaints about >> sucky parallel performance. > > Indeed so. We believe we have chosen (C). If we have not, please say something! > On only a little thought, I don't know the answer. If someone could provide a rough sketch for how either of these (presumably only (B)?) could be done in Java8 given the Java7 plans, it would be a big comfort. For one example of a generics system that accommodates both A and B see the X10 specs: http://dist.codehaus.org/x10/documentation/languagespec/x10-latest.pdf Scala also comes close with upcoming "specialize" directives. See http://www.scala-lang.org/sites/default/files/sids/dragos/Mon,%202009-11-16,%2017:34/sid-spec.pdf or maybe some more recent docs? (Also, to some extent, C#.) I worry a bit because (1) the base generic type systems for both are fairly different than Javas and (2) both implementations are still in pre-release mode running on JVMs. -Doug From nathan.bryant at linkshare.com Sat Jul 10 18:06:39 2010 From: nathan.bryant at linkshare.com (Nathan Bryant) Date: Sun, 11 Jul 2010 10:06:39 +0900 Subject: Primitives in Generics References: <4C36465C.6090809@oracle.com><4C3778EB.60405@cs.oswego.edu> <4C377AB5.7050205@oracle.com> <4C38BD24.2070403@cs.oswego.edu> Message-ID: <7FDA6630E1822F448C97A48D5D733094010208CB@EXVMSTOR302.intra.rakuten.co.jp> Doug Lea wrote: > For one example of a generics system that accommodates both A and B > see the X10 specs: > http://dist.codehaus.org/x10/documentation/languagespec/x10-latest.pdf > Scala also comes close with upcoming "specialize" directives. See > http://www.scala-lang.org/sites/default/files/sids/dragos/Mon,%202009-11 -16,%2017:34/sid-spec.pdf > or maybe some more recent docs? > (Also, to some extent, C#.) They come close, yet they're so far. If you've tried using @specalized for anything serious in Scala, you'll start to see what I mean: scalac starts generating incorrect bytecode for nontrivial cases. (As of whatever release candidate or nightly build I most recently checked, which was a month or two ago.) But even if they fix that... based on what I could dig up, it sounds like part of the problem is that it doesn't really preserve transitivity for @specialized, so it seems likely that if the code generation issues are fixed, it is likely to remain problematic in other respects. So for the near term, it'll be great if the JVM *cough* MLVM MethodHandle inlining is good enough to (a) transport closures to another thread, (b) inline them there in a way that exposes autoboxing elimination transformations, and (c) do all this in the presence of megamorphic call sites, but I'm not holding my breath. All this stuff seems novel. > I worry a bit because (1) the base generic type systems for > both are fairly different than Javas and (2) both implementations > are still in pre-release mode running on JVMs. Hmm, I'm not sure what you mean by this. Scala's type system seems like a straightforward mapping on top of Java generics. If an Int parameter is being used polymorphically as a subclass of Any, which should also be the case for the generic erasure, it's going to be boxed to java.lang.Integer. Scala's type system does seem to map better to the CLR than to the JVM, and that may turn out to be an advantage in the distant future. Random thought about inlining heuristics: it seems like the JVM shouldn't just consider the size of the method to be inlined. There could be an advantage in considering the size of the method containing the call site! And, perhaps, whether or not boxed primitives are present in the signature. Nathan > -Doug From neal at gafter.com Mon Jul 12 15:08:11 2010 From: neal at gafter.com (Neal Gafter) Date: Mon, 12 Jul 2010 15:08:11 -0700 Subject: Transparancy In-Reply-To: References: <4C34C6BE.5010101@oracle.com> Message-ID: On Wed, Jul 7, 2010 at 5:48 PM, Joshua Bloch wrote: > I believe this is an abuse of the word "transparency." There's nothing > transparent about making the "return" keyword do double duty to mean "long > return." It's a new concept, and if and when it is to be supported in > Java, > it deserves a new keyword. > "return" currently means return from the nearest enclosing method or constructor. It is an existing concept using an existing keyword. A lambda expression is a new concept, so yielding a value from a lambda expression is therefore a new concept. The idea of transparency is that the boundaries of a lambda expression do not change the meaning of the program elements within the lambda compared to what those program elements would mean outside the lambda ("transparent" because constructs appear unchanged by the boundary of the lambda except that they are delayed). The boundaries of the lambda are said to be "transparent" if this property holds. There are numerous advantages to designing lambda expressions that are transparent. For constructs such as "return", "break", etc, which are control constructs, this is "control transparency". For a longer list of the various kinds of transparency worth considering, see the bullet list in < http://gafter.blogspot.com/2007/01/definition-of-closures.html>. > However, if future transparency is the goal... > > When you say "future transparency," I believe that what you mean is > "allowing lambdas to be used to emulate control constructs in the future." > I think you should call it that. > That ability is one of the many advantages of designing lambda expressions to be transparent. I don't think we should call a design principle by the name of a single use case that it might (or might not) enable. From alessiostalla at gmail.com Mon Jul 12 16:59:06 2010 From: alessiostalla at gmail.com (Alessio Stalla) Date: Tue, 13 Jul 2010 01:59:06 +0200 Subject: Transparency In-Reply-To: References: Message-ID: On Thu, Jul 8, 2010 at 1:01 AM, Bob Foster wrote: > Neal Gafter wrote: >> Most Java developers have not used lambdas in Java. ?Those who have used >> them in other languages rarely think of them as a method body. > > A tautology followed by an unprovable assertion, but never mind that. > > This discussion suffers a definitional problem: there is no consensus > on what 'lambda' means. Instead, some people are pushing lambda as in > lisp, others are trying to redefine lambda to be Smalltalk blocks or > lambda as in ad hoc feature in some language that happens to call the > feature lambda. > > I will just state for the record that I have used lambda in other > languages and I think of them as anonymous functions. I truly doubt > that puts me in the 'rarely' category. But let's put it on the table > and stop jawboning about what return means. What does lambda mean? I think your question is very appropriate, also given Neal Gafter's last reply to the "Transparency" thread (liberally quoting: "There are numerous advantages to designing lambda expressions that are transparent"). The term "lambda" comes from functional languages and in particular from prof. McCarthy's LISP, which borrowed it from lambda calculus. In that context, "lambda" simply means "anonymous function" - whatever a function exactly is in the language being considered. Now, Java has no functions. Instead, it has methods. Methods are not first-class objects; however, SAM types can be used to simulate first-class methods. We can say that in Java an instance of a SAM class is the closest equivalent to a first-class function object as found in typical functional languages. We can also say that in a sense Java already has both named and unnamed "functions", since "unnamed" classes extending a SAM type can be declared. Keeping that in mind, it is just natural that many people - myself included - when hearing "lambda" and "Java" together first think about simpler syntax for anonymous "functions", i.e. for anonymous inner classes which extend a SAM class. Then naturally people will associate "lambda" more generically with first-class functions as found in the languages they know; for example, people coming from Lisp will expect functions (and thus lambdas) to be full lexical closures. However, strictly speaking that goes beyond the concept of lambda, which is simply that of an anonymous function - again, whatever "function" concretely means. Neal's concept of "lambda" does not appear to me to resemble the concept of "function" but that of first-class code block instead. The key difference being that functions are safe to pass around and call, while code blocks can only safely be called in a certain context, or control transfer will fail. Corollary differences are return vs yield, and yes or no to long control transfer using break/continue/return, which are related. Morale: without stating clearly what "lambda" is supposed to mean, further discussion is not very useful. I propose to use either the expression "anonymous function" or "code block" to avoid confusion. Regards, Alessio Stalla From neal at gafter.com Mon Jul 12 17:34:29 2010 From: neal at gafter.com (Neal Gafter) Date: Mon, 12 Jul 2010 17:34:29 -0700 Subject: Transparency In-Reply-To: References: Message-ID: On Mon, Jul 12, 2010 at 4:59 PM, Alessio Stalla wrote: > On Thu, Jul 8, 2010 at 1:01 AM, Bob Foster wrote: > > Neal Gafter wrote: > >> Most Java developers have not used lambdas in Java. Those who have used > >> them in other languages rarely think of them as a method body. > > > > A tautology followed by an unprovable assertion, but never mind that. > > > > This discussion suffers a definitional problem: there is no consensus > > on what 'lambda' means. Instead, some people are pushing lambda as in > > lisp, others are trying to redefine lambda to be Smalltalk blocks or > > lambda as in ad hoc feature in some language that happens to call the > > feature lambda. > > > > I will just state for the record that I have used lambda in other > > languages and I think of them as anonymous functions. I truly doubt > > that puts me in the 'rarely' category. But let's put it on the table > > and stop jawboning about what return means. What does lambda mean? > > I think your question is very appropriate, also given Neal Gafter's > last reply to the "Transparency" thread (liberally quoting: "There are > numerous advantages to designing lambda expressions that are > transparent"). > > The term "lambda" comes from functional languages and in particular > from prof. McCarthy's LISP, which borrowed it from lambda calculus. In > that context, "lambda" simply means "anonymous function" - whatever a > function exactly is in the language being considered. > > Now, Java has no functions. Instead, it has methods. Methods are not > first-class objects; however, SAM types can be used to simulate > first-class methods. We can say that in Java an instance of a SAM > class is the closest equivalent to a first-class function object as > found in typical functional languages. We can also say that in a sense > Java already has both named and unnamed "functions", since "unnamed" > classes extending a SAM type can be declared. > > Keeping that in mind, it is just natural that many people - myself > included - when hearing "lambda" and "Java" together first think about > simpler syntax for anonymous "functions", i.e. for anonymous inner > classes which extend a SAM class. > Then naturally people will associate "lambda" more generically with > first-class functions as found in the languages they know; for > example, people coming from Lisp will expect functions (and thus > lambdas) to be full lexical closures. However, strictly speaking that > goes beyond the concept of lambda, which is simply that of an > anonymous function - again, whatever "function" concretely means. > > Neal's concept of "lambda" does not appear to me to resemble the > concept of "function" but that of first-class code block instead. The > key difference being that functions are safe to pass around and call, > while code blocks can only safely be called in a certain context, or > control transfer will fail. Corollary differences are return vs yield, > and yes or no to long control transfer using break/continue/return, > which are related. > > Morale: without stating clearly what "lambda" is supposed to mean, > further discussion is not very useful. I propose to use either the > expression "anonymous function" or "code block" to avoid confusion. > I'm not sure either captures what you intend. Methods that return void or that have side-effects are not usually termed "functions". Excluding the possibility of failure in a language that has failure (exceptions) as a first-class concept is not constructive. What you call "code block" would likely be used in most circumstances without any contained control-flow constructs. In other words, you appear to be naming distinct use cases rather than distinct language constructs. Many languages that have "functions" that can nest, side-effects, and acknowledge failure also have transparent "return" (lisp, smalltalk, scala, etc). You allude to the proverbial correspondence between objects (as a collection of lambdas closed over each other) and lambdas (as objects with one member) to suggest that lambdas should be thought as the method of an object with one member. But it is not possible to have an object in Java with just one member. Lambdas are the simpler of the two concepts due to the purely lexical scoping. Designing a new language construct to do exactly what you can already do is not the best way to increase the flexibility of the language. In any case, my suggested definition appears here: < http://gafter.blogspot.com/2007/01/definition-of-closures.html>. Feel free to substitute "lambda" for "closure" in most of that. That post also has pointers to a number of papers that explain in more detail the utility of lambdas in the sense defined there. From howard.lovatt at gmail.com Tue Jul 13 00:33:19 2010 From: howard.lovatt at gmail.com (Howard Lovatt) Date: Tue, 13 Jul 2010 15:33:19 +0800 Subject: Virtual extension methods -- second draft In-Reply-To: <6238C68B-A526-457E-9F55-C4EC8BB16186@mac.com> References: <6238C68B-A526-457E-9F55-C4EC8BB16186@mac.com> Message-ID: @Andrew, I would write the code like you jokingly suggest, OK I wouldn't use AnnoyingAndPoinlessInnerClass for the name I would use Trait, because I like to have the definition and the declaration in the same file. One of the annoying things in C++ is having to have headers and bodies. Cheers, -- Howard. On 13 July 2010 10:38, Andrew Thompson wrote: > > On Jun 11, 2010, at 8:07 PM, Howard Lovatt wrote: > >> I don't think that it would confuse people and would probably become the >> norm, instead of abstract classes (which would be a good thing since it >> would eliminate state). With the current proposal you have to make an >> interface and a class of static methods, for *most* use cases this is >> verbose and a pain since you separate the implementation from the >> declaration. > > > Yes. I agree with Howard Please don't make me open another file to find the implementation. If you do that to me, I'm going to be horribly tempted to write this > > public interface Foo { > ? ?public void method(); > ? ?public extension void method2() default AnnoyingAndPoinlessInnerClass.method2(); > > ? public static final class AnnoyingAndPoinlessInnerClass { > ? ? ? ?public void method2() { > ? ? ? ? ? ? ? ? ... > ? ? ? ?} > ? ?} > } > > Which certainly isn't better than: > > public interface Foo { > ? ?public void method(); > ? ?public extension void method2() default { > ? ? ? ? ? ? ? ? ... > ? ?} > } > > At least I've avoided 2 files, but no one could love the first example. > > AndyT (lordpixel - the cat who walks through walls) > A little bigger on the inside > > ? ? ? ?(see you later space cowboy, you can't take the sky from me) > > > -- ? -- Howard. From howard.lovatt at gmail.com Tue Jul 13 00:51:35 2010 From: howard.lovatt at gmail.com (Howard Lovatt) Date: Tue, 13 Jul 2010 15:51:35 +0800 Subject: Transparency Message-ID: I favour using return instead of yield or another keyword for lambdas since I think that it is a better fit with Java, i.e you get the behaviour of an inner class which is the most natural way. Additionally, people will be continually refactoring between inner classes and lambdas and having different behaviour will be most confusing. The most import ant point are the two above, but additional points in favour of return and inner class behaviour are: 1. Just because some other language has particular semantics doesn't mean Java should, it is much more important that Java is self consistent. 2. The term lambda comes primarily from Church's Lambda calculus, which didn't have return, break, continue, objects, methods, etc. so I find the argument that it must behave exactly like Church's construct weak since that would imply that we should eliminate return, break, continue, objects, methods, etc. and no one would want that. 3. If you change Java to allow long returns, say "methodName.return value;", and allow this to always be used and say that an unqualified return is a shorthand that returns from the inner most method or lambda then this is transparent (so long as you use the named return). Therefore long returns and transparency can be added in future if required (I suspect they wouldn't be required). 4. To me the odd constructs are the loops, if, and blocks, because they don't return a value. I am not suggesting this change for Java because it would be too hard, but imagine that you used return inside all blocks like you do a method to early terminate the block and to optionally return a value. Then you wouldn't require the ?: operator or continue (return would do this). To me, this use of return would be much more consistent than the Lisp/Smalltalk type blocks anyway. -- ? -- Howard. From alessiostalla at gmail.com Tue Jul 13 01:45:14 2010 From: alessiostalla at gmail.com (Alessio Stalla) Date: Tue, 13 Jul 2010 10:45:14 +0200 Subject: Transparency In-Reply-To: References: Message-ID: On Tue, Jul 13, 2010 at 2:34 AM, Neal Gafter wrote: > On Mon, Jul 12, 2010 at 4:59 PM, Alessio Stalla > wrote: >> >> On Thu, Jul 8, 2010 at 1:01 AM, Bob Foster wrote: >> > Neal Gafter wrote: >> >> Most Java developers have not used lambdas in Java. ?Those who have >> >> used >> >> them in other languages rarely think of them as a method body. >> > >> > A tautology followed by an unprovable assertion, but never mind that. >> > >> > This discussion suffers a definitional problem: there is no consensus >> > on what 'lambda' means. Instead, some people are pushing lambda as in >> > lisp, others are trying to redefine lambda to be Smalltalk blocks or >> > lambda as in ad hoc feature in some language that happens to call the >> > feature lambda. >> > >> > I will just state for the record that I have used lambda in other >> > languages and I think of them as anonymous functions. I truly doubt >> > that puts me in the 'rarely' category. But let's put it on the table >> > and stop jawboning about what return means. What does lambda mean? >> >> I think your question is very appropriate, also given Neal Gafter's >> last reply to the "Transparency" thread (liberally quoting: "There are >> numerous advantages to designing lambda expressions that are >> transparent"). >> >> The term "lambda" comes from functional languages and in particular >> from prof. McCarthy's LISP, which borrowed it from lambda calculus. In >> that context, "lambda" simply means "anonymous function" - whatever a >> function exactly is in the language being considered. >> >> Now, Java has no functions. Instead, it has methods. Methods are not >> first-class objects; however, SAM types can be used to simulate >> first-class methods. We can say that in Java an instance of a SAM >> class is the closest equivalent to a first-class function object as >> found in typical functional languages. We can also say that in a sense >> Java already has both named and unnamed "functions", since "unnamed" >> classes extending a SAM type can be declared. >> >> Keeping that in mind, it is just natural that many people - myself >> included - when hearing "lambda" and "Java" together first think about >> simpler syntax for anonymous "functions", i.e. for anonymous inner >> classes which extend a SAM class. >> Then naturally people will associate "lambda" more generically with >> first-class functions as found in the languages they know; for >> example, people coming from Lisp will expect functions (and thus >> lambdas) to be full lexical closures. However, strictly speaking that >> goes beyond the concept of lambda, which is simply that of an >> anonymous function - again, whatever "function" concretely means. >> >> Neal's concept of "lambda" does not appear to me to resemble the >> concept of "function" but that of first-class code block instead. The >> key difference being that functions are safe to pass around and call, >> while code blocks can only safely be called in a certain context, or >> control transfer will fail. Corollary differences are return vs yield, >> and yes or no to long control transfer using break/continue/return, >> which are related. >> >> Morale: without stating clearly what "lambda" is supposed to mean, >> further discussion is not very useful. I propose to use either the >> expression "anonymous function" or "code block" to avoid confusion. > > I'm not sure either captures what you intend.? Methods that return void or > that have side-effects are not usually termed "functions". They're not functions in the mathematical sense, but they're functions as they are commonly named in functional programming languages. Call them procedures or subroutines if you like. > Excluding the possibility of failure in a language that has failure (exceptions) as a > first-class concept is not constructive. I'm not proposing that. > What you call "code block" would likely be used in most circumstances without any contained control-flow > constructs.? In other words, you appear to be naming distinct use cases > rather than distinct language constructs. No, because the two "use cases" give the language different semantics. With lambdas-as-functions one can copy-paste the body of a method in the body of a closure and it will "just work". With lambdas-as-blocks that doesn't hold. > Many languages that have "functions" that can nest, side-effects, and acknowledge failure also have > transparent "return" (lisp, smalltalk, scala, etc). I don't know Smalltalk nor Scala, but Lisp surely has no transparent return by default. You have to tell it explicitly to do a long return with (return-from ), and that's a seldom-used feature (but Lisp has macros, making the use of closures for control structures unnecessary). Languages in the ML family also don't have long return, at least not by default. > You allude to the proverbial correspondence between objects (as a collection > of lambdas closed over each other) and lambdas (as objects with one member) > to suggest that lambdas should be thought as the method of an object with > one member.? But it is not possible to have an object in Java with just one > member.? Lambdas are the simpler of the two concepts due to the purely > lexical scoping. No, I didn't mean to allude to that. I meant that SAM types are commonly used to simulate first-class functions, not that they are proper closures. > Designing a new language construct to do exactly what you can already do is > not the best way to increase the flexibility of the language. > > In any case, my suggested definition appears here: > .? Feel free > to substitute "lambda" for "closure" in most of that.? That post also has > pointers to a number of papers that explain in more detail the utility of > lambdas in the sense defined there. Lambda and closure are two orthogonal concepts. Lambda means anonymous function, where "function" does not necessarily imply "closure". Closure means a function that captures the lexical environment it's been defined in, and can be named or unnamed (lambda). In that sense, lambdas can be mapped to anonymous SAM types in Java, while closures cannot (since anonymous inner classes only capture a fraction of their outer lexical environment, final local variables). In any case, long control transfer by default ihmo contrasts with the concept of function altogether, so neither lambda nor closure would be a fitting term. I'd really keep the two concepts separate; the same object cannot be both a function to be mapped over the elements of a collection and the body of a user-defined control structure. Mixing the two unnecessarily complicates the language. This, I think, is the root of the problem. Best, Alessio From scolebourne at joda.org Tue Jul 13 04:41:52 2010 From: scolebourne at joda.org (Stephen Colebourne) Date: Tue, 13 Jul 2010 12:41:52 +0100 Subject: Yield considered harmful In-Reply-To: References: <2A48108C-0A72-4EC7-B4F6-1A4542F1DC2B@gmail.com> Message-ID: On 9 July 2010 23:13, Pavel Minaev wrote: > The hypothetical new syntax does not need to apply to "return" - it > can apply to the lambda as a whole. E.g.: > > ?words.forEach() {# word -> > ?? ?if (word.contains("http")) { > ?? ? ? ?return word; > ?? ?} > ?} > > where # is a placeholder for something that distinguishes this form of > lambda (a "block" in Ruby parlance) from a proper function. The > meaning of "return" in the body of such a lambda is then adjusted, and > "break" and "continue" are permitted. > > Such a lambda expression could furthermore have a distinct type, which > could, for example, be used to prevent it from escaping the scope in > which "break" and "continue" are meaningful (by restricting that type > for return values, fields, captured mutable locals etc). The option FCM-JCA outlined followed the strategy of using syntax to enable two uses of "return" Syntax A example: forEach(list, #(str) { // looks more like an inner class if (str.length() > 3) return; // local-return System.out.println(str); }); Syntax B example: list.each() for (String str) { // looks like a code block if (str.length() > 3) return; // lexical return System.out.println(str); } In syntax A (ie. the current project Lambda), "return" is a local-return. In syntax B (some future language change) "return" is lexical (and the block might not even be implemented using a "closure-like" technique). Thus, the two pieces of code are not equivalent. The point here is that using "return" now is not limiting to the future addition of lexical "return" - so long as the *syntax* used *now* is obvious (inner class like) and allows enough syntax space for a future separate change (code abstraction like). And any desire for adding lexical returns from syntax A in a future release would use a new syntax, of which many have been proposed (throw return, long return, methodname.return ....) In summary, using "return" now does not constrain future options hence "yield" is a distraction. The trick to keep future options open is the right syntax choice. Stephen From reinier at zwitserloot.com Tue Jul 13 06:56:12 2010 From: reinier at zwitserloot.com (Reinier Zwitserloot) Date: Tue, 13 Jul 2010 15:56:12 +0200 Subject: Primitives in Generics In-Reply-To: <4C38BD24.2070403@cs.oswego.edu> References: <4C36465C.6090809@oracle.com> <4C3778EB.60405@cs.oswego.edu> <4C377AB5.7050205@oracle.com> <4C38BD24.2070403@cs.oswego.edu> Message-ID: Doesn't the OP of this thread show a way forward to get to (B) given SotL is implemented for java7? Nothing in SotL so far seems to clash with my proposal. However, omitting primitives-in-generics for java7 would mean the flurry of development that'll ensue to take advantage of lambdas may result in a number of specialization SAMs (IntComparator would for example be a specialization of Comparator), which will then have to be supported in perpetuity when java8 deprecates all of them by offering direct support of either primitives in generics or specialization. Case in point: Given that PA was the driving use case for Project Lambda, one would imagine that if SotL goes ahead, PA will be added to the core runtime libraries of java7. Presumably with the full 132 SAMs of PA.Ops. Those would then have to remain there forever. I can imagine worse things; they are small interfaces that can easily be corralled into a module. Nevertheless, a point worth thinking about, because I doubt PA.Ops will be the only library that ends up shipping with specialization SAMs. If in the end we're stuck with some 1000-odd specialization SAM types amongst commonly used java libraries, that's a sizable price to pay now for omitting the feature. NB: Other than arrays-of-parameterized-types I've yet to hear of a fundamental problem with my proposal. Arrays remain popular in large part _because_ primitives in generics doesn't work, so while arrays-of-paramerized-types is never going to work particularly well in my proposal when the parameterized type is a primitive, on the plus side it'll reduce the number of times folks are messing with T[] in the first place. --Reinier Zwitserloot On Sat, Jul 10, 2010 at 8:34 PM, Doug Lea
wrote: > On 07/09/10 15:38, Brian Goetz wrote: > > >> My take is that you have three choices: > >> A) Support function types covering primitives > >> B) Support primitives in generics > >> C) Do something that doesn't restrict your options to reconsider > >> (A) or (B) in Java8 in response to all the complaints about > >> sucky parallel performance. > > > > Indeed so. We believe we have chosen (C). If we have not, please say > something! > > > > On only a little thought, I don't know the answer. If someone could > provide a rough sketch for how either of these (presumably only (B)?) > could be done in Java8 given the Java7 plans, it would be a big comfort. > > For one example of a generics system that accommodates both A and B > see the X10 specs: > http://dist.codehaus.org/x10/documentation/languagespec/x10-latest.pdf > Scala also comes close with upcoming "specialize" directives. See > > http://www.scala-lang.org/sites/default/files/sids/dragos/Mon,%202009-11-16,%2017:34/sid-spec.pdf > or maybe some more recent docs? > (Also, to some extent, C#.) > > I worry a bit because (1) the base generic type systems for > both are fairly different than Javas and (2) both implementations > are still in pre-release mode running on JVMs. > > -Doug > > > > From neal at gafter.com Tue Jul 13 08:23:12 2010 From: neal at gafter.com (Neal Gafter) Date: Tue, 13 Jul 2010 08:23:12 -0700 Subject: Transparency In-Reply-To: References: Message-ID: On Tue, Jul 13, 2010 at 1:45 AM, Alessio Stalla wrote: > > What you call "code block" would likely be used in most circumstances > without any contained control-flow > > constructs. In other words, you appear to be naming distinct use cases > > rather than distinct language constructs. > > No, because the two "use cases" give the language different semantics. > With lambdas-as-functions one can copy-paste the body of a method in > the body of a closure and it will "just work". With lambdas-as-blocks > that doesn't hold. > With lambdas-as-blocks you can just copy-paste a block of code into a lambda to delay its execution. With lambdas-as-method-bodies that doesn't hold. > > > Many languages that have "functions" that can nest, side-effects, and > acknowledge failure also have > > transparent "return" (lisp, smalltalk, scala, etc). > > I don't know Smalltalk nor Scala, but Lisp surely has no transparent > return by default. Common lisp's "return" will pass transparently though any number of intervening lambda boundaries. > You have to tell it explicitly to do a long return > with (return-from ), and that's a > seldom-used feature (but Lisp has macros, making the use of closures > for control structures unnecessary). Macros and transparent lambdas are incomparable in expressive power. Languages in the ML family also > don't have long return, at least not by default. > Not a "local" return. But you can define a returning construct, which would then be transparent. > > Designing a new language construct to do exactly what you can already do > is > > not the best way to increase the flexibility of the language. > > > > In any case, my suggested definition appears here: > > . Feel > free > > to substitute "lambda" for "closure" in most of that. That post also has > > pointers to a number of papers that explain in more detail the utility of > > lambdas in the sense defined there. > > Lambda and closure are two orthogonal concepts. Lambda means anonymous > function, where "function" does not necessarily imply "closure". > Closure means a function that captures the lexical environment it's > been defined in, and can be named or unnamed (lambda). In that sense, > lambdas can be mapped to anonymous SAM types in Java, while closures > cannot (since anonymous inner classes only capture a fraction of their > outer lexical environment, final local variables). Lambdas are *capable* of capturing the lexical environment; when they do, they are closures. You've found one of many reasons that anonymous inner classes are neither lambdas nor full closures. In any case, long > control transfer by default ihmo contrasts with the concept of > function altogether, so neither lambda nor closure would be a fitting > term. Indeed, a void-returning thing contrasts with the concept of function altogether as well. > I'd really keep the two concepts separate; the same object > cannot be both a function to be mapped over the elements of a > collection and the body of a user-defined control structure. Huh? Why not? The only difference is whether or not it yields a new value or not (void result type). From pbenedict at apache.org Tue Jul 13 08:48:28 2010 From: pbenedict at apache.org (Paul Benedict) Date: Tue, 13 Jul 2010 10:48:28 -0500 Subject: Transparency In-Reply-To: References: Message-ID: If I may ask a broad question... Since Neal provided a working prototype of Closures, why did OpenJDK 7 not begin its coding efforts from his work? Being a side-line judge here (excuse the metaphor), it appears his prototype already solves some of the questions being discussed, and there were enough open issues for healthy community feedback. Wouldn't it be wise to start from something -- augmenting and disassemble as you please -- rather than code the official solution from square one? On Tue, Jul 13, 2010 at 10:23 AM, Neal Gafter wrote: > On Tue, Jul 13, 2010 at 1:45 AM, Alessio Stalla wrote: > >> > What you call "code block" would likely be used in most circumstances >> without any contained control-flow >> > constructs. ?In other words, you appear to be naming distinct use cases >> > rather than distinct language constructs. >> >> No, because the two "use cases" give the language different semantics. >> With lambdas-as-functions one can copy-paste the body of a method in >> the body of a closure and it will "just work". With lambdas-as-blocks >> that doesn't hold. >> > > With lambdas-as-blocks you can just copy-paste a block of code into a lambda > to delay its execution. ?With lambdas-as-method-bodies that doesn't hold. > >> >> > Many languages that have "functions" that can nest, side-effects, and >> acknowledge failure also have >> > transparent "return" (lisp, smalltalk, scala, etc). >> >> I don't know Smalltalk nor Scala, but Lisp surely has no transparent >> return by default. > > > Common lisp's "return" will pass transparently though any number of > intervening lambda boundaries. > > >> You have to tell it explicitly to do a long return >> with (return-from ), and that's a >> seldom-used feature (but Lisp has macros, making the use of closures >> for control structures unnecessary). > > > Macros and transparent lambdas are incomparable in expressive power. > > Languages in the ML family also >> don't have long return, at least not by default. >> > > Not a "local" return. ?But you can define a returning construct, which would > then be transparent. > > >> ?> Designing a new language construct to do exactly what you can already do >> is >> > not the best way to increase the flexibility of the language. >> > >> > In any case, my suggested definition appears here: >> > . ?Feel >> free >> > to substitute "lambda" for "closure" in most of that. ?That post also has >> > pointers to a number of papers that explain in more detail the utility of >> > lambdas in the sense defined there. >> >> Lambda and closure are two orthogonal concepts. Lambda means anonymous >> function, where "function" does not necessarily imply "closure". >> Closure means a function that captures the lexical environment it's >> been defined in, and can be named or unnamed (lambda). In that sense, >> lambdas can be mapped to anonymous SAM types in Java, while closures >> cannot (since anonymous inner classes only capture a fraction of their >> outer lexical environment, final local variables). > > > Lambdas are *capable* of capturing the lexical environment; when they do, > they are closures. ?You've found one of many reasons that anonymous inner > classes are neither lambdas nor full closures. > > In any case, long >> control transfer by default ihmo contrasts with the concept of >> function altogether, so neither lambda nor closure would be a fitting >> term. > > > Indeed, a void-returning thing contrasts with the concept of function > altogether as well. > > >> I'd really keep the two concepts separate; the same object >> cannot be both a function to be mapped over the elements of a >> collection and the body of a user-defined control structure. > > > Huh? ?Why not? ?The only difference is whether or not it yields a new value > or not (void result type). > > From alessiostalla at gmail.com Tue Jul 13 09:10:32 2010 From: alessiostalla at gmail.com (Alessio Stalla) Date: Tue, 13 Jul 2010 18:10:32 +0200 Subject: Transparency In-Reply-To: References: Message-ID: On Tue, Jul 13, 2010 at 5:23 PM, Neal Gafter wrote: > On Tue, Jul 13, 2010 at 1:45 AM, Alessio Stalla > wrote: >> >> > What you call "code block" would likely be used in most circumstances >> > without any contained control-flow >> > constructs.? In other words, you appear to be naming distinct use cases >> > rather than distinct language constructs. >> >> No, because the two "use cases" give the language different semantics. >> With lambdas-as-functions one can copy-paste the body of a method in >> the body of a closure and it will "just work". With lambdas-as-blocks >> that doesn't hold. > > With lambdas-as-blocks you can just copy-paste a block of code into a lambda > to delay its execution.? With lambdas-as-method-bodies that doesn't hold. I agree! That's why the two are different language constructs, not simply different use cases for the same construct. >> > Many languages that have "functions" that can nest, side-effects, and >> > acknowledge failure also have >> > transparent "return" (lisp, smalltalk, scala, etc). >> >> I don't know Smalltalk nor Scala, but Lisp surely has no transparent >> return by default. > > Common lisp's "return" will pass transparently though any number of > intervening lambda boundaries. Not really. First, "return" is not the default way to return a value in Lisp; most of the time you don't use an explicit return, and the value of the last expression in the function body is returned. Second, Common Lisp's return is equivalent to (return-from nil), i.e., it returns from the lexically closest block named nil. Functions do not introduce such a block, so by default, return indeed will pass through a function call (note that I said function, not lambda). However, the user can introduce blocks and limit the extent of the return. In short, CL's return does not exactly map to any concept in Java, already present or proposed here. >> >> You have to tell it explicitly to do a long return >> with (return-from ), and that's a >> seldom-used feature (but Lisp has macros, making the use of closures >> for control structures unnecessary). > > Macros and transparent lambdas are incomparable in expressive power. Yes, it's just that macros are often used to implement control structures that, without macros, would likely be implemented with "transparent lambdas". > >> Languages in the ML family also >> don't have long return, at least not by default. > > Not a "local" return.? But you can define a returning construct, which would > then be transparent. i.e., it's not the default. >> >> > Designing a new language construct to do exactly what you can already do >> > is >> > not the best way to increase the flexibility of the language. >> > >> > In any case, my suggested definition appears here: >> > .? Feel >> > free >> > to substitute "lambda" for "closure" in most of that.? That post also >> > has >> > pointers to a number of papers that explain in more detail the utility >> > of >> > lambdas in the sense defined there. >> >> Lambda and closure are two orthogonal concepts. Lambda means anonymous >> function, where "function" does not necessarily imply "closure". >> Closure means a function that captures the lexical environment it's >> been defined in, and can be named or unnamed (lambda). In that sense, >> lambdas can be mapped to anonymous SAM types in Java, while closures >> cannot (since anonymous inner classes only capture a fraction of their >> outer lexical environment, final local variables). > > Lambdas are *capable* of capturing the lexical environment; when they do, > they are closures.? You've found one of many reasons that anonymous inner > classes are neither lambdas nor full closures. Lambdas are capable iff functions are capable. It is conceivable to have a language with lambda but without lexical closures; early, dynamically scoped Lisps were like that. Of course, functions which are also closures are the norm nowadays, for good reasons. However, if one considers SAM classes as approximations of first-class functions, then Java's "functions" are not closures. So if you speak of adding closures to Java, one can and often will interpret it as making SAMs be real closures. That's why I'm advocating to use different words than lambda and closure, if you have different objectives. >> In any case, long >> control transfer by default ihmo contrasts with the concept of >> function altogether, so neither lambda nor closure would be a fitting >> term. > > Indeed, a void-returning thing contrasts with the concept of function > altogether as well. I think you're constantly missing the "by default" part. Yes, if "functions" returned void by default, and returning a value was the exception, they wouldn't be really functions. If they return a value by default, but have the option to return no value, and to make long returns, they're still functions, with a few more features. That's subjective, to a certain degree, but programming language concepts are rarely black-or-white. You can write purely imperative code in Java, but that doesn't make it less OO. You can write OO code in C, but that doesn't make it less imperative. You can write all your functions with long return in ML or Lisp, but that won't change the nature of functions in those languages. >> >> I'd really keep the two concepts separate; the same object >> cannot be both a function to be mapped over the elements of a >> collection and the body of a user-defined control structure. > > Huh?? Why not?? The only difference is whether or not it yields a new value > or not (void result type). And whether, when it yields a value, it does so by default with a local or long return. And note that the yield keyword is not the default way to return a value in Java. I think that returning void or not is much less significant; after all, if something other than void is returned when no-one expects a value, it will be discarded. If void is returned when someone expects a value, a compile-time error is raised. That's how Java works today. Bye, Alessio From neal at gafter.com Tue Jul 13 09:59:28 2010 From: neal at gafter.com (Neal Gafter) Date: Tue, 13 Jul 2010 09:59:28 -0700 Subject: Transparency In-Reply-To: References: Message-ID: On Jul 13, 2010, at 9:10 AM, Alessio Stalla wrote: > On Tue, Jul 13, 2010 at 5:23 PM, Neal Gafter wrote: >> On Tue, Jul 13, 2010 at 1:45 AM, Alessio Stalla >> wrote: >>> >>>> What you call "code block" would likely be used in most circumstances >>>> without any contained control-flow >>>> constructs. In other words, you appear to be naming distinct use cases >>>> rather than distinct language constructs. >>> >>> No, because the two "use cases" give the language different semantics. >>> With lambdas-as-functions one can copy-paste the body of a method in >>> the body of a closure and it will "just work". With lambdas-as-blocks >>> that doesn't hold. >> >> With lambdas-as-blocks you can just copy-paste a block of code into a lambda >> to delay its execution. With lambdas-as-method-bodies that doesn't hold. > > I agree! That's why the two are different language constructs, not > simply different use cases for the same construct. Indeed, that is the crux of our agreement: whether we're trying to add to Java something it already has (methods or anonymous instances) or something that enables things not possible today. From bobfoster at gmail.com Tue Jul 13 13:14:12 2010 From: bobfoster at gmail.com (Bob Foster) Date: Tue, 13 Jul 2010 13:14:12 -0700 Subject: Transparency Message-ID: Neal Gafter wrote: > Indeed, that is the crux of our agreement: whether we're trying to add to Java something it already has > (methods or anonymous instances) or something that enables things not possible today. Like multiple threads returning non-locally to the same method. Bob From nathan.bryant at linkshare.com Tue Jul 13 13:24:16 2010 From: nathan.bryant at linkshare.com (Nathan Bryant) Date: Wed, 14 Jul 2010 05:24:16 +0900 Subject: Transparency References: Message-ID: <7FDA6630E1822F448C97A48D5D733094010A91FB@EXVMSTOR302.intra.rakuten.co.jp> Perhaps too much hay is being made out of the parallel use case. Having a decent set of combinators in collections frameworks may result in programmers being more likely to use bulk operations where appropriate, minimizing the tendency to do things like coding application logic in SQL, nesting SQL in loops... That being said, I think it's going to be a lot harder (i.e., nearly impossible) to get primitives-in-generics and specialization right, than to get function types right. The problem for both primitives-in-generics and specialization is they're somewhat hobbled by lack of reification... and it's too late for reification. -----Original Message----- From: lambda-dev-bounces at openjdk.java.net [mailto:lambda-dev-bounces at openjdk.java.net] On Behalf Of Bob Foster Sent: Tuesday, July 13, 2010 4:14 PM To: lambda-dev at openjdk.java.net; alessiostalla at gmail.com; neal at gafter.com Subject: Re: Transparency Neal Gafter wrote: > Indeed, that is the crux of our agreement: whether we're trying to add to Java something it already has > (methods or anonymous instances) or something that enables things not possible today. Like multiple threads returning non-locally to the same method. Bob From neal at gafter.com Tue Jul 13 13:28:41 2010 From: neal at gafter.com (Neal Gafter) Date: Tue, 13 Jul 2010 13:28:41 -0700 Subject: Transparency In-Reply-To: References: Message-ID: On Tue, Jul 13, 2010 at 1:14 PM, Bob Foster wrote: > Neal Gafter wrote: > > Indeed, that is the crux of our agreement: whether we're trying to add to > Java something it already has > > (methods or anonymous instances) or something that enables things not > possible today. > > Like multiple threads returning non-locally to the same method. > Sounds like a red herring. Nobody has proposed any extension by which multiple threads could return non-locally to the same method. From mthornton at optrak.co.uk Tue Jul 13 13:39:06 2010 From: mthornton at optrak.co.uk (Mark Thornton) Date: Tue, 13 Jul 2010 21:39:06 +0100 Subject: Transparency In-Reply-To: <7FDA6630E1822F448C97A48D5D733094010A91FB@EXVMSTOR302.intra.rakuten.co.jp> References: <7FDA6630E1822F448C97A48D5D733094010A91FB@EXVMSTOR302.intra.rakuten.co.jp> Message-ID: <4C3CCEEA.3070203@optrak.co.uk> Nathan Bryant wrote: > Perhaps too much hay is being made out of the parallel use case. > > Having a decent set of combinators in collections frameworks may result > in programmers being more likely to use bulk operations where > appropriate, minimizing the tendency to do things like coding > application logic in SQL, nesting SQL in loops... > > That being said, I think it's going to be a lot harder (i.e., nearly > impossible) to get primitives-in-generics and specialization right, than > to get function types right. > > The problem for both primitives-in-generics and specialization is > they're somewhat hobbled by lack of reification... and it's too late for > reification. > I don't think it is too late for reification, but it does seem to be necessary to support both reified and non reified (erased) classes, and possibly classes where some but not all type parameters are reified. Primitive type parameters are unavoidably reified (as far as I can see), so not supporting reification for other types would be peculiar. Mark Thornton From john at milsson.nu Tue Jul 13 13:59:23 2010 From: john at milsson.nu (John Nilsson) Date: Tue, 13 Jul 2010 22:59:23 +0200 Subject: Transparency In-Reply-To: <4C3CCEEA.3070203@optrak.co.uk> References: <7FDA6630E1822F448C97A48D5D733094010A91FB@EXVMSTOR302.intra.rakuten.co.jp> <4C3CCEEA.3070203@optrak.co.uk> Message-ID: On Tue, Jul 13, 2010 at 10:39 PM, Mark Thornton wrote: > Nathan Bryant wrote: >> The problem for both primitives-in-generics and specialization is >> they're somewhat hobbled by lack of reification... and it's too late for >> reification. >> > I don't think it is too late for reification, but it does seem to be > necessary to support both reified and non reified (erased) classes, and > possibly classes where some but not all type parameters are reified. > Primitive type parameters are unavoidably reified (as far as I can see), > so not supporting reification for other types would be peculiar. What exactly is it that requires reification to work? BR, John From mthornton at optrak.co.uk Tue Jul 13 14:01:13 2010 From: mthornton at optrak.co.uk (Mark Thornton) Date: Tue, 13 Jul 2010 22:01:13 +0100 Subject: Transparency In-Reply-To: References: <7FDA6630E1822F448C97A48D5D733094010A91FB@EXVMSTOR302.intra.rakuten.co.jp> <4C3CCEEA.3070203@optrak.co.uk> Message-ID: <4C3CD419.6090304@optrak.co.uk> John Nilsson wrote: > On Tue, Jul 13, 2010 at 10:39 PM, Mark Thornton wrote: > >> Nathan Bryant wrote: >> >>> The problem for both primitives-in-generics and specialization is >>> they're somewhat hobbled by lack of reification... and it's too late for >>> reification. >>> >>> >> I don't think it is too late for reification, but it does seem to be >> necessary to support both reified and non reified (erased) classes, and >> possibly classes where some but not all type parameters are reified. >> Primitive type parameters are unavoidably reified (as far as I can see), >> so not supporting reification for other types would be peculiar. >> > > What exactly is it that requires reification to work? > > BR, > John > Simply that you can't erase primitives. Mark From nathan.bryant at linkshare.com Tue Jul 13 14:08:20 2010 From: nathan.bryant at linkshare.com (Nathan Bryant) Date: Wed, 14 Jul 2010 06:08:20 +0900 Subject: Transparency References: <7FDA6630E1822F448C97A48D5D733094010A91FB@EXVMSTOR302.intra.rakuten.co.jp><4C3CCEEA.3070203@optrak.co.uk> Message-ID: <7FDA6630E1822F448C97A48D5D733094010A9261@EXVMSTOR302.intra.rakuten.co.jp> John, it's just array creation class Foo { T[] arr = new T[SIZE]; // Can't write this without reification } class Foo { Object[] arr = new Object[SIZE]; void set(int i, T t) { arr[i] = t; } // Either causes autoboxing unbeknownst to the sorry fool who wrote new Foo, or becomes illegal } John Nilsson wrote: > What exactly is it that requires reification to work? BR, John From john at milsson.nu Tue Jul 13 14:58:32 2010 From: john at milsson.nu (John Nilsson) Date: Tue, 13 Jul 2010 23:58:32 +0200 Subject: Transparency In-Reply-To: <7FDA6630E1822F448C97A48D5D733094010A9261@EXVMSTOR302.intra.rakuten.co.jp> References: <7FDA6630E1822F448C97A48D5D733094010A91FB@EXVMSTOR302.intra.rakuten.co.jp> <4C3CCEEA.3070203@optrak.co.uk> <7FDA6630E1822F448C97A48D5D733094010A9261@EXVMSTOR302.intra.rakuten.co.jp> Message-ID: On Tue, Jul 13, 2010 at 11:08 PM, Nathan Bryant wrote: > ? ? ? ?void set(int i, T t) { arr[i] = t; } // Either causes autoboxing > unbeknownst to the sorry fool who wrote new Foo, or becomes illegal How about just erasing T[] to Object and let API-developers worry about picking an array implementation? BR, John From nathan.bryant at linkshare.com Tue Jul 13 15:16:24 2010 From: nathan.bryant at linkshare.com (Nathan Bryant) Date: Wed, 14 Jul 2010 07:16:24 +0900 Subject: Transparency References: <7FDA6630E1822F448C97A48D5D733094010A91FB@EXVMSTOR302.intra.rakuten.co.jp><4C3CCEEA.3070203@optrak.co.uk><7FDA6630E1822F448C97A48D5D733094010A9261@EXVMSTOR302.intra.rakuten.co.jp> Message-ID: <7FDA6630E1822F448C97A48D5D733094010A92EE@EXVMSTOR302.intra.rakuten.co.jp> John, If you mean what I think you mean, that seems to result in a generics that is useful only for interface definition. Either it's not possible to write: foo = new ArrayList() Or doing so results in autoboxing to Integer, unbeknownst to the user, which is a serious violation of the principle of least surprise. One way around this is to add an annotation that declares that a type parameter "accepts primitives", but this would render a lot of existing interface definitions unable to work with the scheme, rendering them nearly as obsolete as adding function types to the language would do. Either way, adding combinators to the standard library during Java7 will force the choice of either SAMs or function types. If SAMs are the choice, and are implemented without the abovementioned annotation, it might have to be added later. If it's added later, and "accepts primitives" is to mean "reified", then that's most likely a binary incompatible change. -----Original Message----- From: John Nilsson [mailto:john at milsson.nu] Sent: Tuesday, July 13, 2010 5:59 PM To: Nathan Bryant Cc: Mark Thornton; lambda-dev at openjdk.java.net Subject: Re: Transparency On Tue, Jul 13, 2010 at 11:08 PM, Nathan Bryant wrote: > ? ? ? ?void set(int i, T t) { arr[i] = t; } // Either causes autoboxing > unbeknownst to the sorry fool who wrote new Foo, or becomes illegal How about just erasing T[] to Object and let API-developers worry about picking an array implementation? BR, John From reinier at zwitserloot.com Wed Jul 14 07:17:17 2010 From: reinier at zwitserloot.com (Reinier Zwitserloot) Date: Wed, 14 Jul 2010 16:17:17 +0200 Subject: Transparency In-Reply-To: <7FDA6630E1822F448C97A48D5D733094010A9261@EXVMSTOR302.intra.rakuten.co.jp> References: <7FDA6630E1822F448C97A48D5D733094010A91FB@EXVMSTOR302.intra.rakuten.co.jp> <4C3CCEEA.3070203@optrak.co.uk> <7FDA6630E1822F448C97A48D5D733094010A9261@EXVMSTOR302.intra.rakuten.co.jp> Message-ID: Indeed, having: new ArrayList(); be as fast as an int array requires reification; once ArrayList creates a backing array of type Object[], the performance game is lost. Nothing short of reification can address this issue. To be specific, specialization won't solve it any more than primitives-in-generics does. However, this: List list = Lists.of(int.class); list.add(10); or: List list = new IntList(); list.add(10); *CAN* be made fast. Specialization, interestingly enough, cannot do this*, but my primitives in generics proposal can. Which partly makes me wonder why we're talking about the far more complicated specialization concept in the first place. I know scala uses this, but they either introduced their collections API with specialization from day one, or they created a backwards incompatible retrofit later. *) list.add(10), is, as far as the compiler knows, a call to add(T) on a List. If this is to be fast under specialization, it would therefore have to generate a call to an add method with signature (I)Z and not with signature (Ljava/lang/Object;)Z like it would today (and autobox to make the int fit as an object). However, if it is to generate such a call, then *ANY* list, no matter how it was obtained, must have an add (I)Z method, but lists written today do not because that's not part of the List.java interface. We can't add methods to interfaces or we'd break backwards compatibility, and breaking backwards compatibility is not acceptable. Therefore, in list implementations that lack add(I)Z, the JVM or the classloader has to generate these. There's precedence of course: Brian Goetz' proposal for defender methods does something quite similar. Still, this is far from trivial, and I'd like to see a proposal on how one could update List.java so that the JVM / ClassLoader considers the specialization methods as defenderish, but the object-based one itself not. Primitives-in-generics keeps things simple by letting the call to (Ljava/lang/Object;)Z stand, and letting the JVM hotspot compiler eliminate the inefficiency instead. --Reinier Zwitserloot On Tue, Jul 13, 2010 at 11:08 PM, Nathan Bryant wrote: > John, it's just array creation > > class Foo { > T[] arr = new T[SIZE]; // Can't write this without reification > } > > class Foo { > Object[] arr = new Object[SIZE]; > > void set(int i, T t) { arr[i] = t; } // Either causes autoboxing > unbeknownst to the sorry fool who wrote new Foo, or becomes illegal > } > > > John Nilsson wrote: > > > What exactly is it that requires reification to work? > > BR, > John > > From opinali at gmail.com Wed Jul 14 09:16:29 2010 From: opinali at gmail.com (Osvaldo Doederlein) Date: Wed, 14 Jul 2010 13:16:29 -0300 Subject: Transparency In-Reply-To: References: <7FDA6630E1822F448C97A48D5D733094010A91FB@EXVMSTOR302.intra.rakuten.co.jp> <4C3CCEEA.3070203@optrak.co.uk> <7FDA6630E1822F448C97A48D5D733094010A9261@EXVMSTOR302.intra.rakuten.co.jp> Message-ID: A concrete suggestion: we could support this via bytecode transformation, e.g. ArrayList list = Magic.reify(ArrayList.class, int.class); ...where reify() will basically change the type of ArrayList's internal Object[] backing store to int[]. This would probably benefit some help from the original collection, e.g. annotations that list which fields/parameters/returns/locals must be changed (or not); but this mapping could be provided on a configuration-by-exception manner. The bytecode transformation is mostly simple, it just needs also to translate some API calls (e.g. to Arrays.copyOf(T[], int, int)) to equivalent calls for the required primitive-array type (or fail the whole thing, if it hits some call that is unknown to the transformer or has no primitive-friendly overloads). Several reifications of the same origin class could be arranged in a hierarchy that's designed to both reflect the original hierarchy and to share methods that don't directly depend on the element type (e.g. isEmpty() { return size()==0; }). So we get synthetic classes like: AbstractCollection$ extends AbstractCollection AbstractCollection$Int extends AbstractCollection$ AbstractList$Int extends AbstractCollection$Int ArrayList$Int extends AbstractList$Int The new classes can even inherit the top-level class from the original hierarchy, as long as that class has no fields or at least no "container" Object[] fields. Returning a type like ArrayList means no changes in syntax, method signatures, classfile format, reflection etc. to support ArrayList. So the proposal is a simple library without any exceptional demands - which is the only kind of proposal that we could hope adding to JDK 7, at this late time. The tradeoff is that we're stuck with signatures like add(I)Z, so we depend on HotSpot to optimize out the boxing/unboxing that happens in each method call. At least for inlined calls I guess this optimization is already done; the remaining work should be feasible, and it could wait for some 7uXX maintenance update if there's no time do do that until FCS. There is no Object[] field to worry about... but still, some Object[]s in APIs like toArray(). In that case there is no significant extra conversion cost (toArray() already does a defensive copy so we just do a different copy from int[] to Object[]). The reified class could implement some extra, type-specific helper interface that adds methods like int[] toArrayInt(); this would probably make the transformer more tightly-bound to the Java SE Collections API, or require extra annotations for custom additions; not pretty, but not hard to implement. And while we're at it, I would love to add equivalent methods that DON'T do a defensive copy, just return the internal array, or assign an external array to the collection's field. Perf-critical code often benefits from that; I have personally had this problem. When your code is already optimized enough, redundant data copies always bubble up to the top of the performance profile. This is why estimating the initial size is everybody's favorite collections optimization, as it avoids lots of allocation and copy from growing. Finally, it's also easy to implement as the bytecode transformation, full reification for any non-public method. So we can't touch add(Object), but we certainly can do that for all private and even protected methods, and also entire non-public helper classes like AbstractList's Itr, ListItr, SubList, and RandomAccessSubList. I don't like ideas like new IntList(), or some Lists.of() that's just a factory to hide types like IntList. This would add enormous bloat to the core, and/or would be very limited - there's a hefty number of collection classes in the core (not to mention third-party code) and I may need primitive-optimized versions of ANY of these classes. If I need a List of int, I may also need the specific features or O(N) traits of particular implementations like ArrayList, LinkedList, or even CopyOnWriteArrayList. Don't make me choose between optimal element storage (primitive reification) and optimal API/structure (compact array, linked nodes, concurrency etc.). A partial solution is sometimes worse than no solution at all - it avoids a much bigger mess, e.g. using a core collection for ArrayList-of-int, and GNU Trove for LinkedList-of-int or whatever. As a final remark, this implementation would be valid: T reify (T collection, Class elementType) { return collection; } ..it just wouldn't gain any performance. But it's good enough, for example, to add (as a compatibility feature) to platforms where code footprint is a higher concern, like the lowest Java ME profiles. Or even to the first release of JDK 7, if there is no time for a full implementation of this API. A+ Osvaldo 2010/7/14 Reinier Zwitserloot > Indeed, having: new ArrayList(); be as fast as an int array requires > reification; once ArrayList creates a backing array of type Object[], the > performance game is lost. > > Nothing short of reification can address this issue. To be specific, > specialization won't solve it any more than primitives-in-generics does. > However, this: > > List list = Lists.of(int.class); > list.add(10); > > or: > > List list = new IntList(); > list.add(10); > > > *CAN* be made fast. Specialization, interestingly enough, cannot do this*, > but my primitives in generics proposal can. Which partly makes me wonder > why > we're talking about the far more complicated specialization concept in the > first place. I know scala uses this, but they either introduced their > collections API with specialization from day one, or they created a > backwards incompatible retrofit later. > > *) list.add(10), is, as far as the compiler knows, a call to add(T) on a > List. If this is to be fast under specialization, it would therefore > have to generate a call to an add method with signature (I)Z and not with > signature (Ljava/lang/Object;)Z like it would today (and autobox to make > the > int fit as an object). However, if it is to generate such a call, then > *ANY* > list, no matter how it was obtained, must have an add (I)Z method, but > lists > written today do not because that's not part of the List.java interface. We > can't add methods to interfaces or we'd break backwards compatibility, and > breaking backwards compatibility is not acceptable. Therefore, in list > implementations that lack add(I)Z, the JVM or the classloader has to > generate these. There's precedence of course: Brian Goetz' proposal for > defender methods does something quite similar. Still, this is far from > trivial, and I'd like to see a proposal on how one could update List.java > so > that the JVM / ClassLoader considers the specialization methods as > defenderish, but the object-based one itself not. Primitives-in-generics > keeps things simple by letting the call to (Ljava/lang/Object;)Z stand, and > letting the JVM hotspot compiler eliminate the inefficiency instead. > > --Reinier Zwitserloot > > > > On Tue, Jul 13, 2010 at 11:08 PM, Nathan Bryant < > nathan.bryant at linkshare.com > > wrote: > > > John, it's just array creation > > > > class Foo { > > T[] arr = new T[SIZE]; // Can't write this without reification > > } > > > > class Foo { > > Object[] arr = new Object[SIZE]; > > > > void set(int i, T t) { arr[i] = t; } // Either causes autoboxing > > unbeknownst to the sorry fool who wrote new Foo, or becomes illegal > > } > > > > > > John Nilsson wrote: > > > > > What exactly is it that requires reification to work? > > > > BR, > > John > > > > > > From forax at univ-mlv.fr Wed Jul 14 10:37:01 2010 From: forax at univ-mlv.fr (=?ISO-8859-1?Q?R=E9mi_Forax?=) Date: Wed, 14 Jul 2010 19:37:01 +0200 Subject: Transparency In-Reply-To: References: <7FDA6630E1822F448C97A48D5D733094010A91FB@EXVMSTOR302.intra.rakuten.co.jp> <4C3CCEEA.3070203@optrak.co.uk> <7FDA6630E1822F448C97A48D5D733094010A9261@EXVMSTOR302.intra.rakuten.co.jp> Message-ID: <4C3DF5BD.6070900@univ-mlv.fr> Osvaldo, in my opinion, you should read: http://linkinghub.elsevier.com/retrieve/pii/S1571066105051376 and http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.22.3706&rep=rep1&type=pdf and for the support of wildcard http://portal.acm.org/citation.cfm?id=1244286 (you can also take a look to the name of the first author of this paper). cheers, R?mi Le 14/07/2010 18:16, Osvaldo Doederlein a ?crit : > A concrete suggestion: we could support this via bytecode transformation, > e.g. > > ArrayList list = Magic.reify(ArrayList.class, int.class); > > ...where reify() will basically change the type of ArrayList's internal > Object[] backing store to int[]. This would probably benefit some help from > the original collection, e.g. annotations that list which > fields/parameters/returns/locals must be changed (or not); but this mapping > could be provided on a configuration-by-exception manner. The bytecode > transformation is mostly simple, it just needs also to translate some API > calls (e.g. to Arrays.copyOf(T[], int, int)) to equivalent calls for the > required primitive-array type (or fail the whole thing, if it hits some call > that is unknown to the transformer or has no primitive-friendly overloads). > > Several reifications of the same origin class could be arranged in a > hierarchy that's designed to both reflect the original hierarchy and to > share methods that don't directly depend on the element type (e.g. isEmpty() > { return size()==0; }). So we get synthetic classes like: > > AbstractCollection$ extends AbstractCollection > AbstractCollection$Int extends AbstractCollection$ > AbstractList$Int extends AbstractCollection$Int > ArrayList$Int extends AbstractList$Int > > The new classes can even inherit the top-level class from the original > hierarchy, as long as that class has no fields or at least no "container" > Object[] fields. > > Returning a type like ArrayList means no changes in syntax, method > signatures, classfile format, reflection etc. to support ArrayList. So > the proposal is a simple library without any exceptional demands - which is > the only kind of proposal that we could hope adding to JDK 7, at this late > time. The tradeoff is that we're stuck with signatures like add(I)Z, so we > depend on HotSpot to optimize out the boxing/unboxing that happens in each > method call. At least for inlined calls I guess this optimization is already > done; the remaining work should be feasible, and it could wait for some 7uXX > maintenance update if there's no time do do that until FCS. > > There is no Object[] field to worry about... but still, some Object[]s in > APIs like toArray(). In that case there is no significant extra conversion > cost (toArray() already does a defensive copy so we just do a different copy > from int[] to Object[]). The reified class could implement some extra, > type-specific helper interface that adds methods like int[] toArrayInt(); > this would probably make the transformer more tightly-bound to the Java SE > Collections API, or require extra annotations for custom additions; not > pretty, but not hard to implement. And while we're at it, I would love to > add equivalent methods that DON'T do a defensive copy, just return the > internal array, or assign an external array to the collection's field. > Perf-critical code often benefits from that; I have personally had this > problem. When your code is already optimized enough, redundant data copies > always bubble up to the top of the performance profile. This is why > estimating the initial size is everybody's favorite collections > optimization, as it avoids lots of allocation and copy from growing. > > Finally, it's also easy to implement as the bytecode transformation, full > reification for any non-public method. So we can't touch add(Object), but we > certainly can do that for all private and even protected methods, and also > entire non-public helper classes like AbstractList's Itr, ListItr, SubList, > and RandomAccessSubList. > > I don't like ideas like new IntList(), or some Lists.of() that's just a > factory to hide types like IntList. This would add enormous bloat to the > core, and/or would be very limited - there's a hefty number of collection > classes in the core (not to mention third-party code) and I may need > primitive-optimized versions of ANY of these classes. If I need a List of > int, I may also need the specific features or O(N) traits of particular > implementations like ArrayList, LinkedList, or even CopyOnWriteArrayList. > Don't make me choose between optimal element storage (primitive reification) > and optimal API/structure (compact array, linked nodes, concurrency etc.). A > partial solution is sometimes worse than no solution at all - it avoids a > much bigger mess, e.g. using a core collection for ArrayList-of-int, and GNU > Trove for LinkedList-of-int or whatever. > > As a final remark, this implementation would be valid: > > T reify (T collection, Class elementType) { > return collection; > } > > ..it just wouldn't gain any performance. But it's good enough, for example, > to add (as a compatibility feature) to platforms where code footprint is a > higher concern, like the lowest Java ME profiles. Or even to the first > release of JDK 7, if there is no time for a full implementation of this API. > > A+ > Osvaldo > > > 2010/7/14 Reinier Zwitserloot > > >> Indeed, having: new ArrayList(); be as fast as an int array requires >> reification; once ArrayList creates a backing array of type Object[], the >> performance game is lost. >> >> Nothing short of reification can address this issue. To be specific, >> specialization won't solve it any more than primitives-in-generics does. >> However, this: >> >> List list = Lists.of(int.class); >> list.add(10); >> >> or: >> >> List list = new IntList(); >> list.add(10); >> >> >> *CAN* be made fast. Specialization, interestingly enough, cannot do this*, >> but my primitives in generics proposal can. Which partly makes me wonder >> why >> we're talking about the far more complicated specialization concept in the >> first place. I know scala uses this, but they either introduced their >> collections API with specialization from day one, or they created a >> backwards incompatible retrofit later. >> >> *) list.add(10), is, as far as the compiler knows, a call to add(T) on a >> List. If this is to be fast under specialization, it would therefore >> have to generate a call to an add method with signature (I)Z and not with >> signature (Ljava/lang/Object;)Z like it would today (and autobox to make >> the >> int fit as an object). However, if it is to generate such a call, then >> *ANY* >> list, no matter how it was obtained, must have an add (I)Z method, but >> lists >> written today do not because that's not part of the List.java interface. We >> can't add methods to interfaces or we'd break backwards compatibility, and >> breaking backwards compatibility is not acceptable. Therefore, in list >> implementations that lack add(I)Z, the JVM or the classloader has to >> generate these. There's precedence of course: Brian Goetz' proposal for >> defender methods does something quite similar. Still, this is far from >> trivial, and I'd like to see a proposal on how one could update List.java >> so >> that the JVM / ClassLoader considers the specialization methods as >> defenderish, but the object-based one itself not. Primitives-in-generics >> keeps things simple by letting the call to (Ljava/lang/Object;)Z stand, and >> letting the JVM hotspot compiler eliminate the inefficiency instead. >> >> --Reinier Zwitserloot >> >> >> >> On Tue, Jul 13, 2010 at 11:08 PM, Nathan Bryant< >> nathan.bryant at linkshare.com >> >>> wrote: >>> >> >>> John, it's just array creation >>> >>> class Foo { >>> T[] arr = new T[SIZE]; // Can't write this without reification >>> } >>> >>> class Foo { >>> Object[] arr = new Object[SIZE]; >>> >>> void set(int i, T t) { arr[i] = t; } // Either causes autoboxing >>> unbeknownst to the sorry fool who wrote new Foo, or becomes illegal >>> } >>> >>> >>> John Nilsson wrote: >>> >>> >>>> What exactly is it that requires reification to work? >>>> >>> BR, >>> John >>> >>> >>> >> >> > From opinali at gmail.com Wed Jul 14 11:23:10 2010 From: opinali at gmail.com (Osvaldo Doederlein) Date: Wed, 14 Jul 2010 15:23:10 -0300 Subject: Transparency In-Reply-To: <4C3DF5BD.6070900@univ-mlv.fr> References: <7FDA6630E1822F448C97A48D5D733094010A91FB@EXVMSTOR302.intra.rakuten.co.jp> <4C3CCEEA.3070203@optrak.co.uk> <7FDA6630E1822F448C97A48D5D733094010A9261@EXVMSTOR302.intra.rakuten.co.jp> <4C3DF5BD.6070900@univ-mlv.fr> Message-ID: Thanks R?mi! I don't remember of any of these recent papers, will check them. But I did read the much older research from Ole Agesen ( http://portal.acm.org/citation.cfm?id=263720); my proposal is obviously inspired by Ole's approach... although I don't remember the fine details now, that's been many years. BTW, that stuff is _so_ old that the current status quo, wrt generics and primitive types, is bewildering for me. But perhaps it's not yet too late to adopt solutions that were well-researched 13+ years ago. A+ Osvaldo 2010/7/14 R?mi Forax > Osvaldo, > in my opinion, you should read: > http://linkinghub.elsevier.com/retrieve/pii/S1571066105051376 > and > > http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.22.3706&rep=rep1&type=pdf > and for the support of wildcard > http://portal.acm.org/citation.cfm?id=1244286 > (you can also take a look to the name of the first author of this paper). > > cheers, > R?mi > > Le 14/07/2010 18:16, Osvaldo Doederlein a ?crit : > > A concrete suggestion: we could support this via bytecode transformation, > > e.g. > > > > ArrayList list = Magic.reify(ArrayList.class, int.class); > > > > ...where reify() will basically change the type of ArrayList's internal > > Object[] backing store to int[]. This would probably benefit some help > from > > the original collection, e.g. annotations that list which > > fields/parameters/returns/locals must be changed (or not); but this > mapping > > could be provided on a configuration-by-exception manner. The bytecode > > transformation is mostly simple, it just needs also to translate some API > > calls (e.g. to Arrays.copyOf(T[], int, int)) to equivalent calls for the > > required primitive-array type (or fail the whole thing, if it hits some > call > > that is unknown to the transformer or has no primitive-friendly > overloads). > > > > Several reifications of the same origin class could be arranged in a > > hierarchy that's designed to both reflect the original hierarchy and to > > share methods that don't directly depend on the element type (e.g. > isEmpty() > > { return size()==0; }). So we get synthetic classes like: > > > > AbstractCollection$ extends AbstractCollection > > AbstractCollection$Int extends AbstractCollection$ > > AbstractList$Int extends AbstractCollection$Int > > ArrayList$Int extends AbstractList$Int > > > > The new classes can even inherit the top-level class from the original > > hierarchy, as long as that class has no fields or at least no "container" > > Object[] fields. > > > > Returning a type like ArrayList means no changes in syntax, > method > > signatures, classfile format, reflection etc. to support ArrayList. > So > > the proposal is a simple library without any exceptional demands - which > is > > the only kind of proposal that we could hope adding to JDK 7, at this > late > > time. The tradeoff is that we're stuck with signatures like add(I)Z, so > we > > depend on HotSpot to optimize out the boxing/unboxing that happens in > each > > method call. At least for inlined calls I guess this optimization is > already > > done; the remaining work should be feasible, and it could wait for some > 7uXX > > maintenance update if there's no time do do that until FCS. > > > > There is no Object[] field to worry about... but still, some Object[]s in > > APIs like toArray(). In that case there is no significant extra > conversion > > cost (toArray() already does a defensive copy so we just do a different > copy > > from int[] to Object[]). The reified class could implement some extra, > > type-specific helper interface that adds methods like int[] toArrayInt(); > > this would probably make the transformer more tightly-bound to the Java > SE > > Collections API, or require extra annotations for custom additions; not > > pretty, but not hard to implement. And while we're at it, I would love to > > add equivalent methods that DON'T do a defensive copy, just return the > > internal array, or assign an external array to the collection's field. > > Perf-critical code often benefits from that; I have personally had this > > problem. When your code is already optimized enough, redundant data > copies > > always bubble up to the top of the performance profile. This is why > > estimating the initial size is everybody's favorite collections > > optimization, as it avoids lots of allocation and copy from growing. > > > > Finally, it's also easy to implement as the bytecode transformation, full > > reification for any non-public method. So we can't touch add(Object), but > we > > certainly can do that for all private and even protected methods, and > also > > entire non-public helper classes like AbstractList's Itr, ListItr, > SubList, > > and RandomAccessSubList. > > > > I don't like ideas like new IntList(), or some Lists.of() that's just a > > factory to hide types like IntList. This would add enormous bloat to the > > core, and/or would be very limited - there's a hefty number of collection > > classes in the core (not to mention third-party code) and I may need > > primitive-optimized versions of ANY of these classes. If I need a List of > > int, I may also need the specific features or O(N) traits of particular > > implementations like ArrayList, LinkedList, or even CopyOnWriteArrayList. > > Don't make me choose between optimal element storage (primitive > reification) > > and optimal API/structure (compact array, linked nodes, concurrency > etc.). A > > partial solution is sometimes worse than no solution at all - it avoids a > > much bigger mess, e.g. using a core collection for ArrayList-of-int, and > GNU > > Trove for LinkedList-of-int or whatever. > > > > As a final remark, this implementation would be valid: > > > > T reify (T collection, Class elementType) { > > return collection; > > } > > > > ..it just wouldn't gain any performance. But it's good enough, for > example, > > to add (as a compatibility feature) to platforms where code footprint is > a > > higher concern, like the lowest Java ME profiles. Or even to the first > > release of JDK 7, if there is no time for a full implementation of this > API. > > > > A+ > > Osvaldo > > > > > > 2010/7/14 Reinier Zwitserloot > > > > > >> Indeed, having: new ArrayList(); be as fast as an int array > requires > >> reification; once ArrayList creates a backing array of type Object[], > the > >> performance game is lost. > >> > >> Nothing short of reification can address this issue. To be specific, > >> specialization won't solve it any more than primitives-in-generics does. > >> However, this: > >> > >> List list = Lists.of(int.class); > >> list.add(10); > >> > >> or: > >> > >> List list = new IntList(); > >> list.add(10); > >> > >> > >> *CAN* be made fast. Specialization, interestingly enough, cannot do > this*, > >> but my primitives in generics proposal can. Which partly makes me wonder > >> why > >> we're talking about the far more complicated specialization concept in > the > >> first place. I know scala uses this, but they either introduced their > >> collections API with specialization from day one, or they created a > >> backwards incompatible retrofit later. > >> > >> *) list.add(10), is, as far as the compiler knows, a call to add(T) on a > >> List. If this is to be fast under specialization, it would > therefore > >> have to generate a call to an add method with signature (I)Z and not > with > >> signature (Ljava/lang/Object;)Z like it would today (and autobox to make > >> the > >> int fit as an object). However, if it is to generate such a call, then > >> *ANY* > >> list, no matter how it was obtained, must have an add (I)Z method, but > >> lists > >> written today do not because that's not part of the List.java interface. > We > >> can't add methods to interfaces or we'd break backwards compatibility, > and > >> breaking backwards compatibility is not acceptable. Therefore, in list > >> implementations that lack add(I)Z, the JVM or the classloader has to > >> generate these. There's precedence of course: Brian Goetz' proposal for > >> defender methods does something quite similar. Still, this is far from > >> trivial, and I'd like to see a proposal on how one could update > List.java > >> so > >> that the JVM / ClassLoader considers the specialization methods as > >> defenderish, but the object-based one itself not. Primitives-in-generics > >> keeps things simple by letting the call to (Ljava/lang/Object;)Z stand, > and > >> letting the JVM hotspot compiler eliminate the inefficiency instead. > >> > >> --Reinier Zwitserloot > >> > >> > >> > >> On Tue, Jul 13, 2010 at 11:08 PM, Nathan Bryant< > >> nathan.bryant at linkshare.com > >> > >>> wrote: > >>> > >> > >>> John, it's just array creation > >>> > >>> class Foo { > >>> T[] arr = new T[SIZE]; // Can't write this without reification > >>> } > >>> > >>> class Foo { > >>> Object[] arr = new Object[SIZE]; > >>> > >>> void set(int i, T t) { arr[i] = t; } // Either causes > autoboxing > >>> unbeknownst to the sorry fool who wrote new Foo, or becomes > illegal > >>> } > >>> > >>> > >>> John Nilsson wrote: > >>> > >>> > >>>> What exactly is it that requires reification to work? > >>>> > >>> BR, > >>> John > >>> > >>> > >>> > >> > >> > > > > > From john at milsson.nu Wed Jul 14 13:24:32 2010 From: john at milsson.nu (John Nilsson) Date: Wed, 14 Jul 2010 22:24:32 +0200 Subject: Transparency In-Reply-To: <7FDA6630E1822F448C97A48D5D733094010A92EE@EXVMSTOR302.intra.rakuten.co.jp> References: <7FDA6630E1822F448C97A48D5D733094010A91FB@EXVMSTOR302.intra.rakuten.co.jp> <4C3CCEEA.3070203@optrak.co.uk> <7FDA6630E1822F448C97A48D5D733094010A9261@EXVMSTOR302.intra.rakuten.co.jp> <7FDA6630E1822F448C97A48D5D733094010A92EE@EXVMSTOR302.intra.rakuten.co.jp> Message-ID: On Wed, Jul 14, 2010 at 12:16 AM, Nathan Bryant wrote: > John, > > If you mean what I think you mean, that seems to result in a generics that is useful only for interface definition. Either it's not possible to write: > > foo = new ArrayList() > > Or doing so results in autoboxing to Integer, unbeknownst to the user, which is a serious violation of the principle of least surprise. Or there is some clever hacks in ArrayList to handle the issue. An empty array is just empty, so no need to have any specific array-type. If the first element added is an int or Integer you could optimistically assume that all input will be ints and thus allocate an int[], should that assumption turnout to be wrong, in later adds you allocate an Object[]. I can see how this could be problematic for uninitialized values, return null or 0? Maybe this could be solved by adding a generic "zero"-value that is valid for all types? I'm not saying that this is particularily elegent though... BR, John From opinali at gmail.com Wed Jul 14 15:57:25 2010 From: opinali at gmail.com (Osvaldo Doederlein) Date: Wed, 14 Jul 2010 19:57:25 -0300 Subject: Transparency In-Reply-To: References: <7FDA6630E1822F448C97A48D5D733094010A91FB@EXVMSTOR302.intra.rakuten.co.jp> <4C3CCEEA.3070203@optrak.co.uk> <7FDA6630E1822F448C97A48D5D733094010A9261@EXVMSTOR302.intra.rakuten.co.jp> <7FDA6630E1822F448C97A48D5D733094010A92EE@EXVMSTOR302.intra.rakuten.co.jp> Message-ID: 2010/7/14 John Nilsson > On Wed, Jul 14, 2010 at 12:16 AM, Nathan Bryant > wrote: > > John, > > > > If you mean what I think you mean, that seems to result in a generics > that is useful only for interface definition. Either it's not possible to > write: > > > > foo = new ArrayList() > > > > Or doing so results in autoboxing to Integer, unbeknownst to the user, > which is a serious violation of the principle of least surprise. > > Or there is some clever hacks in ArrayList to handle the issue. An > empty array is just empty, so no need to have any specific array-type. > If the first element added is an int or Integer you could > optimistically assume that all input will be ints and thus allocate an > int[], should that assumption turnout to be wrong, in later adds you > allocate an Object[]. > This would create some extra effort - instanceof, typecast and branches - in many critical methods like add(), get() etc. For collections of primitives, this would be much better than the cost of boxing; but for collections of reference types, we would make existing code (close to 100% of current code) slower, even if just a bit... which is not acceptable, breaks the "first, do no harm" principle. (Collections are critical enough that any 1% slowdown in the general/dominant usage will only pass with some big fight.) > I can see how this could be problematic for uninitialized values, > return null or 0? Maybe this could be solved by adding a generic > "zero"-value that is valid for all types? > This is not a problem. If the first value added is int and later we transition int[]->Object, any unused position (slack due to the exponential growing strategy) becomes nulls, but these values are not really contained by the collection (they are in positions >= size()). If we ever transition Object[]->int[], slack elements are initialized with 0 but once again they're not contained. If the collection (in Object[] state) does contains nulls, then that collection simply cannot transition to the optimized primitive-array form. A+ Osvaldo > > I'm not saying that this is particularily elegent though... > > BR, > John > > From collin.fagan at gmail.com Wed Jul 14 19:53:16 2010 From: collin.fagan at gmail.com (Collin Fagan) Date: Wed, 14 Jul 2010 21:53:16 -0500 Subject: Fwd: Transparency In-Reply-To: References: <7FDA6630E1822F448C97A48D5D733094010A91FB@EXVMSTOR302.intra.rakuten.co.jp> <4C3CCEEA.3070203@optrak.co.uk> <7FDA6630E1822F448C97A48D5D733094010A9261@EXVMSTOR302.intra.rakuten.co.jp> <7FDA6630E1822F448C97A48D5D733094010A92EE@EXVMSTOR302.intra.rakuten.co.jp> Message-ID: It could be my imperfect understanding your proposal, but when I hear you talk about ArrayList and not the interface List, List. Well, I get the feeling that you are talking about an implementation of primitives in generics for a select few JDK classes only and this mechanism would not be available to any old programmer who wants to use primitives. Also while I'm commenting on this idea I'd like to mention that the auto-boxing genie is out of the bottle. As an example my friend was working with some code that was backed by a bunch of Map> objects. The methods to populate this map took parameters declared as just long. In this case auto-boxing causes a massive amounts of new Long objects to be created. Personally I feel autoboxing ArrayList to ArrayList is no more dangerous or surprising then the current behavior. Which in my experience is very surprising to people. I think it would be a real benefit if one could minimize the number of objects created by autoboxing in general. Then something like this won't be as harmful. To see just how bad memory allocation was under this model my friend and I hacked up a quick Long cache. 3/4 of a gig of ram for a heap that was about 8 gig. He is championing a campaign to remove all primitive longs from his companies code base, but it's an uphill battle. Collin On Wed, Jul 14, 2010 at 5:57 PM, Osvaldo Doederlein wrote: > 2010/7/14 John Nilsson > > > On Wed, Jul 14, 2010 at 12:16 AM, Nathan Bryant > > wrote: > > > John, > > > > > > If you mean what I think you mean, that seems to result in a generics > > that is useful only for interface definition. Either it's not possible to > > write: > > > > > > foo = new ArrayList() > > > > > > Or doing so results in autoboxing to Integer, unbeknownst to the user, > > which is a serious violation of the principle of least surprise. > > > > Or there is some clever hacks in ArrayList to handle the issue. An > > empty array is just empty, so no need to have any specific array-type. > > If the first element added is an int or Integer you could > > optimistically assume that all input will be ints and thus allocate an > > int[], should that assumption turnout to be wrong, in later adds you > > allocate an Object[]. > > > > This would create some extra effort - instanceof, typecast and branches - > in > many critical methods like add(), get() etc. For collections of primitives, > this would be much better than the cost of boxing; but for collections of > reference types, we would make existing code (close to 100% of current > code) > slower, even if just a bit... which is not acceptable, breaks the "first, > do > no harm" principle. (Collections are critical enough that any 1% slowdown > in > the general/dominant usage will only pass with some big fight.) > > > > I can see how this could be problematic for uninitialized values, > > return null or 0? Maybe this could be solved by adding a generic > > "zero"-value that is valid for all types? > > > > This is not a problem. If the first value added is int and later we > transition int[]->Object, any unused position (slack due to the exponential > growing strategy) becomes nulls, but these values are not really contained > by the collection (they are in positions >= size()). If we ever transition > Object[]->int[], slack elements are initialized with 0 but once again > they're not contained. If the collection (in Object[] state) does contains > nulls, then that collection simply cannot transition to the optimized > primitive-array form. > > A+ > Osvaldo > > > > > > I'm not saying that this is particularily elegent though... > > > > BR, > > John > > > > > > From opinali at gmail.com Thu Jul 15 06:21:04 2010 From: opinali at gmail.com (Osvaldo Doederlein) Date: Thu, 15 Jul 2010 10:21:04 -0300 Subject: Transparency In-Reply-To: References: <7FDA6630E1822F448C97A48D5D733094010A91FB@EXVMSTOR302.intra.rakuten.co.jp> <4C3CCEEA.3070203@optrak.co.uk> <7FDA6630E1822F448C97A48D5D733094010A9261@EXVMSTOR302.intra.rakuten.co.jp> <7FDA6630E1822F448C97A48D5D733094010A92EE@EXVMSTOR302.intra.rakuten.co.jp> Message-ID: 2010/7/14 Collin Fagan > > It could be my imperfect understanding your proposal, but when I hear you > talk about ArrayList and not the interface List, List. Well, I get the > feeling that you are talking about an implementation of primitives in > generics for a select few JDK classes only and this mechanism would not be > available to any old programmer who wants to use primitives. > My use of ArrayList, not List, was not intentional to mean anything, but you've spotted a bug in my proposal: a call like Magic.reify(ArrayList.class, int.class) should obviously return a subtype of ArrayList. This is not compatible with my proposed draft of implementation hierarchy. But of course, we don't want the reified collection to be a subclass of ArrayList, remarkably because this would carry together ArrayList's fields, and we surely don't need its stinkin' Object[] field. Well, that's not a big deal if this field can just be left uninitialized. But forced subtyping of ArrayList still has the problem of making harder to create an implementation hierarchy that favors code reuse (and this is an important concern in any template expansion-like mechanism). What I really want is: use the ArrayList class as an implementation template, but return something that just implements the same interfaces (remarkably List and RandomAccess). In that case, the reify() API could become: public class Collections { public static List reifiedCollection (Class> template, Class primitive); public static List reifiedList (Class template, Class primitive); public static Map reifiedMap (Class template, Class primitive); } ... List list = Collections.reifiedList(ArrayList.class, int.class); The reifiedXxx() methods can be just facades to a single transformer; they return each of the core Collections interfaces, allowing invocations to be warning-free. Their signature also makes explicit that the returned object is just a Collection/List/Map, not necessarily a subtype of the template class. Notice that the runtime should not depend on static knowledge of implementation types to perform the proposed optimizations. Making this more explicit, say you have the list variable allocated above, being used somewhere else (in a method too far away, so the compiler cannot use flow analysis to find that its type is always the reified ArrayList$Int). It doesn't matter. If we do this: list.add(10); thjat code is compiled as usual, as a full-blown invokeinterface to add(Object). But that doesn't matter. The method that's actually invoked is the reified method, like this: public boolean add (Object e) { int reified_e = ((Integer)e).intValue(); ensureCapacity(size + 1); elementData[size++] = e; return true; } ...where elementData has static type int[], only the first line contains unboxing cost. But if and when the VM inlines our call, it's easy to get rid of that cost, both the boxing right before the CALL and the unboxing right after it. In fact I think current HotSpot can already do this kind of optimization (comes for free with inlining + escape analysis + scalar replacement). Even for non-inlined / JIT-compiled methods, maybe it's possible to do some good speedup. The interpreter could have new quickened bytecode to invoke methods that have this boxing/CALL/unboxing pattern. The JIT compiler could emit two entry points for the same method, so a call-site that has an int to pass can just call the unboxed entry point directly, without boxing before the call and without landing in prolog code that needs to unbox - this would be good when the call cannot be inlined for any reason. But these tricks could be added in later releases, they are "implementation detail" that doesn't depend on public APIs or bytecode format changes. An initial implementation that removes all boxing overhead only for "hot" methods, only in HotSpot Server, would already be a big step forward. > Also while I'm commenting on this idea I'd like to mention that the > auto-boxing genie is out of the bottle. As an example my friend was working > with some code that was backed by a bunch of Map> objects. > The methods to populate this map took parameters declared as just long. In > this case auto-boxing causes a massive amounts of new Long objects to be > created. > > Personally I feel autoboxing ArrayList to ArrayList is no > more dangerous or surprising then the current behavior. Which in my > experience is very surprising to people. I think it would be a real benefit > if one could minimize the number of objects created by autoboxing in > general. Then something like this won't be as harmful. > > To see just how bad memory allocation was under this model my friend and I > hacked up a quick Long cache. 3/4 of a gig of ram for a heap that was about > 8 gig. He is championing a campaign to remove all primitive longs from his > companies code base, but it's an uphill battle. > Removing usage 'long' won't buy much advantage to your friend. If you have, say, a database where all PKs are longs but in practice their values always fit in an int, you can just implement a Long cache that contains all possible 2^31 positive int values (as negative values are not usual for PKs). Such cache is just 2X bigger than the all-values Integer cache that you need if all longs are replaced by ints. For huge collections of primitive values or keys, the per-element overhead is often enormous even without boxing, e.g. a big Map.Entry object for every entry of a HashMap. In extreme cases (millions of elements and up) even the reified java.util collections won't do the trick; you need alternative collection implementations (you can implement a hashtable without per-element containers like Entry). A+ Osvaldo > > Collin > > > On Wed, Jul 14, 2010 at 5:57 PM, Osvaldo Doederlein wrote: > >> 2010/7/14 John Nilsson >> >> > On Wed, Jul 14, 2010 at 12:16 AM, Nathan Bryant >> > wrote: >> > > John, >> > > >> > > If you mean what I think you mean, that seems to result in a generics >> > that is useful only for interface definition. Either it's not possible >> to >> > write: >> > > >> > > foo = new ArrayList() >> > > >> > > Or doing so results in autoboxing to Integer, unbeknownst to the user, >> > which is a serious violation of the principle of least surprise. >> > >> > Or there is some clever hacks in ArrayList to handle the issue. An >> > empty array is just empty, so no need to have any specific array-type. >> > If the first element added is an int or Integer you could >> > optimistically assume that all input will be ints and thus allocate an >> > int[], should that assumption turnout to be wrong, in later adds you >> > allocate an Object[]. >> > >> >> This would create some extra effort - instanceof, typecast and branches - >> in >> many critical methods like add(), get() etc. For collections of >> primitives, >> this would be much better than the cost of boxing; but for collections of >> reference types, we would make existing code (close to 100% of current >> code) >> slower, even if just a bit... which is not acceptable, breaks the "first, >> do >> no harm" principle. (Collections are critical enough that any 1% slowdown >> in >> the general/dominant usage will only pass with some big fight.) >> >> >> > I can see how this could be problematic for uninitialized values, >> > return null or 0? Maybe this could be solved by adding a generic >> > "zero"-value that is valid for all types? >> > >> >> This is not a problem. If the first value added is int and later we >> transition int[]->Object, any unused position (slack due to the >> exponential >> growing strategy) becomes nulls, but these values are not really contained >> by the collection (they are in positions >= size()). If we ever transition >> Object[]->int[], slack elements are initialized with 0 but once again >> they're not contained. If the collection (in Object[] state) does contains >> nulls, then that collection simply cannot transition to the optimized >> primitive-array form. >> >> A+ >> Osvaldo >> >> >> > >> > I'm not saying that this is particularily elegent though... >> > >> > BR, >> > John >> > >> > >> >> > > From Kieron.Wilkinson at paretopartners.com Thu Jul 15 08:40:17 2010 From: Kieron.Wilkinson at paretopartners.com (Kieron.Wilkinson at paretopartners.com) Date: Thu, 15 Jul 2010 16:40:17 +0100 Subject: Type inference of parameters In-Reply-To: <4C34C4E6.6040609@oracle.com> References: <4C34C4E6.6040609@oracle.com> Message-ID: Brian Goetz wrote on 07/07/2010 19:18:14: > Yes. I, for one, am in favor of type inference :) > > Type inference is a key underlying motivator for a number of JDK 7 language > changes, including lambda and diamond. We are all about static typing, but > that doesn't mean that the user has to laboriously write out every > single type > declaration when it is obvious from the context. This is a really exciting (and unexpected) development. I definitely vote for this! I'd be interested to know how far this goes (or how far it is intended to go). Will it be able to infer the types from generic parameters like Scala does? For example, given a "map" method defined in a List class: public List map(Function function); Could I write something like the following? List animals = ... List foods = animals.map({ a -> a.getFood() }); Rather than: List foods = animals.map({ Animal a -> a.getFood() }); ? We use our own collection library that has Function-like classes used for fold/map/etc, and it would be great if we didn't have to put in type annotations where they are obvious (especially helpful with multiple parameters or long class names). Thanks, Kieron This message may contain confidential and privileged information and is intended solely for the use of the named addressee. Access, copying or re-use of the e-mail or any information contained therein by any other person is not authorised. If you are not the intended recipient please notify us immediately by returning the e-mail to the originator and then immediately delete this message. Although we attempt to sweep e-mail and attachments for viruses, we do not guarantee that either are virus-free and accept no liability for any damage sustained as a result of viruses. Please refer to http://www.bnymellon.com/disclaimer/piml.html for certain disclosures. From maurizio.cimadamore at oracle.com Thu Jul 15 10:42:54 2010 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Thu, 15 Jul 2010 18:42:54 +0100 Subject: Type inference of parameters In-Reply-To: References: <4C34C4E6.6040609@oracle.com> Message-ID: <4C3F489E.9030005@oracle.com> On 15/07/10 16:40, Kieron.Wilkinson at paretopartners.com wrote: > Brian Goetz wrote on 07/07/2010 19:18:14: > >> Yes. I, for one, am in favor of type inference :) >> >> Type inference is a key underlying motivator for a number of JDK 7 >> > language > >> changes, including lambda and diamond. We are all about static typing, >> > but > >> that doesn't mean that the user has to laboriously write out every >> single type >> declaration when it is obvious from the context. >> > This is a really exciting (and unexpected) development. I definitely vote > for this! > > I'd be interested to know how far this goes (or how far it is intended to > go). Will it be able to infer the types from generic parameters like Scala > does? For example, given a "map" method defined in a List class: > > public List map(Function function); > > Could I write something like the following? > > List animals = ... > List foods = animals.map({ a -> a.getFood() }); > > Rather than: > > List foods = animals.map({ Animal a -> a.getFood() }); > > ? > > We use our own collection library that has Function-like classes used for > fold/map/etc, and it would be great if we didn't have to put in type > annotations where they are obvious (especially helpful with multiple > parameters or long class names). > > Thanks, > Kieron > > Hi Kieron I'm glad to know that you like the way things are shaping out; I'm also glad to tell you that your example should work with the current inference scheme and that removing 'obvious' type annotation is one of the goals of the new proposal! More specifically, in your example, the compiler instantiates the unknown lambda argument type to the element type of the list upon which the 'map' method is being called (hence the type of 'a' is Animal); the type-variable 'B' will be inferred from the assignment context (hence B == Food). Maurizio > This message may contain confidential and privileged information and is intended solely for the use of the named addressee. Access, copying or re-use of the e-mail or any information contained therein by any other person is not authorised. If you are not the intended recipient please notify us immediately by returning the e-mail to the originator and then immediately delete this message. Although we attempt to sweep e-mail and attachments for viruses, we do not guarantee that either are virus-free and accept no liability for any damage sustained as a result of viruses. > > Please refer to http://www.bnymellon.com/disclaimer/piml.html for certain disclosures. > > From reinier at zwitserloot.com Thu Jul 15 10:47:43 2010 From: reinier at zwitserloot.com (Reinier Zwitserloot) Date: Thu, 15 Jul 2010 19:47:43 +0200 Subject: Transparency In-Reply-To: References: <7FDA6630E1822F448C97A48D5D733094010A91FB@EXVMSTOR302.intra.rakuten.co.jp> <4C3CCEEA.3070203@optrak.co.uk> <7FDA6630E1822F448C97A48D5D733094010A9261@EXVMSTOR302.intra.rakuten.co.jp> Message-ID: What happens if some class has multiple type parameters? Should we create a Map$Short$Byte? That's going to explode into a very very large number of types as you add on more type parameters. What seems to be required for your scheme is a mix of complete reification combined with a lot of magic bytecode rewriting, which you haven't fully elaborated on. A few questions that I'm not sure your proposal answers: - When should the JVM stop specializing new classes? With for example 4 type parameters, there would be 8*8*8*8 = 4096 different classes required, which seems quite infeasible. - Exactly how does Magic.reify() work? The backing array field in the ArrayList implementation is not of type T[]; it's of type Object[]. How could any library know that it should rewrite this Object[] to int[] in the generated specialization without hardcoding a ruleset for all known classes? - Even if ArrayList is rewritten to use T[] instead of Object[], that's no help for third party classes that used Object[] - Oracle can't very well change every library ever written for java. In a class where T[] cannot be rewritten to int[], exactly what does ArrayList$Int even do? Presumably it would have to have an (I)Z method auto-generated by the classloader as calling code will expect it to exist. - What happens when heap corruption occurs? i.e: List list = new ArrayList() /* Raw type! */; list.add(10) /* Generates call to non-existing (I)Z add method */; - Normally this results in a MethodNotFoundError, which is rarely caught as errors aren't usually supposed to be caught. That seems entirely inappropriate for heap corruption. - If a dummy reify method is allowable, then how is a call to add(I)Z ever going to work? - If the syntax is to remain ArrayList and not ArrayList, then what happens if one attempts to add null? - How does the auto-generator know that isEmpty() is not dependent on reification, but e.g. add() is? NB: At this point in time, the Oracle JVM does *NOT* eliminate autoboxing and unboxing, though the JRockit VM does. --Reinier Zwitserloot On Wed, Jul 14, 2010 at 6:16 PM, Osvaldo Doederlein wrote: > A concrete suggestion: we could support this via bytecode transformation, > e.g. > > ArrayList list = Magic.reify(ArrayList.class, int.class); > > ...where reify() will basically change the type of ArrayList's internal > Object[] backing store to int[]. This would probably benefit some help from > the original collection, e.g. annotations that list which > fields/parameters/returns/locals must be changed (or not); but this mapping > could be provided on a configuration-by-exception manner. The bytecode > transformation is mostly simple, it just needs also to translate some API > calls (e.g. to Arrays.copyOf(T[], int, int)) to equivalent calls for the > required primitive-array type (or fail the whole thing, if it hits some call > that is unknown to the transformer or has no primitive-friendly overloads). > > Several reifications of the same origin class could be arranged in a > hierarchy that's designed to both reflect the original hierarchy and to > share methods that don't directly depend on the element type (e.g. isEmpty() > { return size()==0; }). So we get synthetic classes like: > > AbstractCollection$ extends AbstractCollection > AbstractCollection$Int extends AbstractCollection$ > AbstractList$Int extends AbstractCollection$Int > ArrayList$Int extends AbstractList$Int > > The new classes can even inherit the top-level class from the original > hierarchy, as long as that class has no fields or at least no "container" > Object[] fields. > > Returning a type like ArrayList means no changes in syntax, method > signatures, classfile format, reflection etc. to support ArrayList. So > the proposal is a simple library without any exceptional demands - which is > the only kind of proposal that we could hope adding to JDK 7, at this late > time. The tradeoff is that we're stuck with signatures like add(I)Z, so we > depend on HotSpot to optimize out the boxing/unboxing that happens in each > method call. At least for inlined calls I guess this optimization is already > done; the remaining work should be feasible, and it could wait for some 7uXX > maintenance update if there's no time do do that until FCS. > > There is no Object[] field to worry about... but still, some Object[]s in > APIs like toArray(). In that case there is no significant extra conversion > cost (toArray() already does a defensive copy so we just do a different copy > from int[] to Object[]). The reified class could implement some extra, > type-specific helper interface that adds methods like int[] toArrayInt(); > this would probably make the transformer more tightly-bound to the Java SE > Collections API, or require extra annotations for custom additions; not > pretty, but not hard to implement. And while we're at it, I would love to > add equivalent methods that DON'T do a defensive copy, just return the > internal array, or assign an external array to the collection's field. > Perf-critical code often benefits from that; I have personally had this > problem. When your code is already optimized enough, redundant data copies > always bubble up to the top of the performance profile. This is why > estimating the initial size is everybody's favorite collections > optimization, as it avoids lots of allocation and copy from growing. > > Finally, it's also easy to implement as the bytecode transformation, full > reification for any non-public method. So we can't touch add(Object), but we > certainly can do that for all private and even protected methods, and also > entire non-public helper classes like AbstractList's Itr, ListItr, SubList, > and RandomAccessSubList. > > I don't like ideas like new IntList(), or some Lists.of() that's just a > factory to hide types like IntList. This would add enormous bloat to the > core, and/or would be very limited - there's a hefty number of collection > classes in the core (not to mention third-party code) and I may need > primitive-optimized versions of ANY of these classes. If I need a List of > int, I may also need the specific features or O(N) traits of particular > implementations like ArrayList, LinkedList, or even CopyOnWriteArrayList. > Don't make me choose between optimal element storage (primitive reification) > and optimal API/structure (compact array, linked nodes, concurrency etc.). A > partial solution is sometimes worse than no solution at all - it avoids a > much bigger mess, e.g. using a core collection for ArrayList-of-int, and GNU > Trove for LinkedList-of-int or whatever. > > As a final remark, this implementation would be valid: > > T reify (T collection, Class elementType) { > return collection; > } > > ..it just wouldn't gain any performance. But it's good enough, for example, > to add (as a compatibility feature) to platforms where code footprint is a > higher concern, like the lowest Java ME profiles. Or even to the first > release of JDK 7, if there is no time for a full implementation of this API. > > A+ > Osvaldo > > > 2010/7/14 Reinier Zwitserloot > > Indeed, having: new ArrayList(); be as fast as an int array requires >> reification; once ArrayList creates a backing array of type Object[], the >> performance game is lost. >> >> Nothing short of reification can address this issue. To be specific, >> specialization won't solve it any more than primitives-in-generics does. >> However, this: >> >> List list = Lists.of(int.class); >> list.add(10); >> >> or: >> >> List list = new IntList(); >> list.add(10); >> >> >> *CAN* be made fast. Specialization, interestingly enough, cannot do this*, >> but my primitives in generics proposal can. Which partly makes me wonder >> why >> we're talking about the far more complicated specialization concept in the >> first place. I know scala uses this, but they either introduced their >> collections API with specialization from day one, or they created a >> backwards incompatible retrofit later. >> >> *) list.add(10), is, as far as the compiler knows, a call to add(T) on a >> List. If this is to be fast under specialization, it would therefore >> have to generate a call to an add method with signature (I)Z and not with >> signature (Ljava/lang/Object;)Z like it would today (and autobox to make >> the >> int fit as an object). However, if it is to generate such a call, then >> *ANY* >> list, no matter how it was obtained, must have an add (I)Z method, but >> lists >> written today do not because that's not part of the List.java interface. >> We >> can't add methods to interfaces or we'd break backwards compatibility, and >> breaking backwards compatibility is not acceptable. Therefore, in list >> implementations that lack add(I)Z, the JVM or the classloader has to >> generate these. There's precedence of course: Brian Goetz' proposal for >> defender methods does something quite similar. Still, this is far from >> trivial, and I'd like to see a proposal on how one could update List.java >> so >> that the JVM / ClassLoader considers the specialization methods as >> defenderish, but the object-based one itself not. Primitives-in-generics >> keeps things simple by letting the call to (Ljava/lang/Object;)Z stand, >> and >> letting the JVM hotspot compiler eliminate the inefficiency instead. >> >> --Reinier Zwitserloot >> >> >> >> On Tue, Jul 13, 2010 at 11:08 PM, Nathan Bryant < >> nathan.bryant at linkshare.com >> > wrote: >> >> > John, it's just array creation >> > >> > class Foo { >> > T[] arr = new T[SIZE]; // Can't write this without reification >> > } >> > >> > class Foo { >> > Object[] arr = new Object[SIZE]; >> > >> > void set(int i, T t) { arr[i] = t; } // Either causes autoboxing >> > unbeknownst to the sorry fool who wrote new Foo, or becomes illegal >> > } >> > >> > >> > John Nilsson wrote: >> > >> > > What exactly is it that requires reification to work? >> > >> > BR, >> > John >> > >> > >> >> > From scolebourne at joda.org Thu Jul 15 16:34:17 2010 From: scolebourne at joda.org (Stephen Colebourne) Date: Fri, 16 Jul 2010 00:34:17 +0100 Subject: Multiple SAMs for same concept Message-ID: One problem noted with the removal of function types is that multiple SAMs with the same signature are not compatible. For example: http://gee.cs.oswego.edu/dl/jsr166/dist/extra166ydocs/extra166y/Ops.Generator.html http://guava-libraries.googlecode.com/svn/trunk/javadoc/com/google/common/base/Supplier.html http://google-guice.googlecode.com/svn/trunk/javadoc/com/google/inject/Provider.html Having these different incompatible definitions is problematic, for example when the developer has created a library of reusable implementations implementing one SAM, and then wants to use them using another (eg. a newly added JDK SAM). Given the method: private void process(Generator gen) { ... } the most recent draft SotL would allow the following two choices for conversion: Supplier supplier = ... process( {-> supplier.supply()} ); // manual conversion to a Generator process( supplier#supply() ); // using method ref to manually convert to Generator While the two solutions are not terrible, they are less than ideal. I would argue that the ideal would be: process(supplier); // auto-boxed from Supplier to Generator Auto-boxing is of course very tricky, as it is in method resolution. I'm not going to start to write rules to define how it should work. One compromise option I came up with was: process( supplier# ); Here, the caller syntax means that the compiler should infer the method name of the method reference on the basis that it is a SAM. This reduces verbosity, is a simple rule, and allows for easier conversion between the myriad of different SAMs that represent the same thing. Alhough P.Lambda could just duck this issue, I fear that doing so would be a mistake. There are lots of SAMs out there already, and just adding a standard set to JDK 7 doesn't solve this particular "backwards compatibility" issue. Stephen From scolebourne at joda.org Thu Jul 15 16:37:30 2010 From: scolebourne at joda.org (Stephen Colebourne) Date: Fri, 16 Jul 2010 00:37:30 +0100 Subject: Constructor references Message-ID: The latest SotL document does not mention constructor references. A constructor is no more than a special kind of method. Omitting it from JDK 7 would, I suspect, be an annoyance to developers. The obvious syntaxes are: Person#() Person#(String, int) or Person#() Person#(String, int) Has any consideration been given to constructor references? Stephen From opinali at gmail.com Thu Jul 15 16:45:57 2010 From: opinali at gmail.com (Osvaldo Doederlein) Date: Thu, 15 Jul 2010 20:45:57 -0300 Subject: Transparency In-Reply-To: References: <7FDA6630E1822F448C97A48D5D733094010A91FB@EXVMSTOR302.intra.rakuten.co.jp> <4C3CCEEA.3070203@optrak.co.uk> <7FDA6630E1822F448C97A48D5D733094010A9261@EXVMSTOR302.intra.rakuten.co.jp> Message-ID: 2010/7/15 Reinier Zwitserloot > What happens if some class has multiple type parameters? Should we create a > Map$Short$Byte? That's going to explode into a very very large number of > types as you add on more type parameters. What seems to be required for your > scheme is a mix of complete reification combined with a lot of magic > bytecode rewriting, which you haven't fully elaborated on. If you want a Map, yes, we create a Map$Short$Byte. This is the reason why you need dynamic code expansion, that happens on demand at reify() calls. You only pay for what you are actually using. The Cartesian product of all your generic parameters in collections and other interesting classes X all primitive types would indeed result in thousands of possible expansions; but in practice, any given application is not likely to use more than a handful of different expansions. The reifier would hold the produced classes in a weak cache, so when some class is not used for a long time, it's class-GC'd. (Same technique already used by synthetic classes produced by reflection, and other dynamic frameworks.) > A few questions that I'm not sure your proposal answers: > - When should the JVM stop specializing new classes? With for example 4 > type parameters, there would be 8*8*8*8 = 4096 different classes required, > which seems quite infeasible. > As explained above, this only happens if the process actually instantiates all these 4096 combinations... not very likely. > - Exactly how does Magic.reify() work? The backing array field in the > ArrayList implementation is not of type T[]; it's of type Object[]. How > could any library know that it should rewrite this Object[] to int[] in the > generated specialization without hardcoding a ruleset for all known classes? > I have actually proposed this, a configuration-by-exception mechanism - you can have annotations to hint what to change, but the reifier could also have default rules and heuristics to use in classes that lack such metadata. > - Even if ArrayList is rewritten to use T[] instead of Object[], that's no > help for third party classes that used Object[] - Oracle can't very well > change every library ever written for java. In a class where T[] cannot be > rewritten to int[], exactly what does ArrayList$Int even do? Presumably it > would have to have an (I)Z method auto-generated by the classloader as > calling code will expect it to exist. > Non an issue, see previous item. BTW the reifier _must_ only rely on the erased types like Object[], because it's a runtime transformation on bytecode, not a compile-time tool that could see source code. > - What happens when heap corruption occurs? i.e: List list = new > ArrayList() /* Raw type! */; list.add(10) /* Generates call to non-existing > (I)Z add method */; - Normally this results in a MethodNotFoundError, which > is rarely caught as errors aren't usually supposed to be caught. That seems > entirely inappropriate for heap corruption. > See http://mail.openjdk.java.net/pipermail/lambda-dev/2010-July/001925.html, we won't have any (I)Z method, we just call the standard add(Object) and rely on VM optimizations to get rid of the overhead of box/CALL/unbox sequences. > - If the syntax is to remain ArrayList and not ArrayList, > then what happens if one attempts to add null? > The first thing that the reified version of add(Object) does is unboxing - ((Integer)e).intValue() - so the result is a NullPointerException - that is consistent with the existing autoboxing semantics. Notice that when the VM can optimize those box/CALL/unbox patterns, the issue is non-existent as the original value is guaranteed not-null. > - How does the auto-generator know that isEmpty() is not dependent on > reification, but e.g. add() is? > NB: At this point in time, the Oracle JVM does *NOT* eliminate autoboxing > and unboxing, though the JRockit VM does. > I know that HotSpot doesn't provide special optimizations for autoboxing, but I was just guessing that it can remove this overhead at least for trivial cases as effect of EA & scalar replacement... if not, hopefully this could be added with little effort, I guess it's just a matter of intrinsifying the valueOf()/xxxValue() methods. A+ Osvaldo > > > > > > --Reinier Zwitserloot > > > > > On Wed, Jul 14, 2010 at 6:16 PM, Osvaldo Doederlein wrote: > >> A concrete suggestion: we could support this via bytecode transformation, >> e.g. >> >> ArrayList list = Magic.reify(ArrayList.class, int.class); >> >> ...where reify() will basically change the type of ArrayList's internal >> Object[] backing store to int[]. This would probably benefit some help from >> the original collection, e.g. annotations that list which >> fields/parameters/returns/locals must be changed (or not); but this mapping >> could be provided on a configuration-by-exception manner. The bytecode >> transformation is mostly simple, it just needs also to translate some API >> calls (e.g. to Arrays.copyOf(T[], int, int)) to equivalent calls for the >> required primitive-array type (or fail the whole thing, if it hits some call >> that is unknown to the transformer or has no primitive-friendly overloads). >> >> Several reifications of the same origin class could be arranged in a >> hierarchy that's designed to both reflect the original hierarchy and to >> share methods that don't directly depend on the element type (e.g. isEmpty() >> { return size()==0; }). So we get synthetic classes like: >> >> AbstractCollection$ extends AbstractCollection >> AbstractCollection$Int extends AbstractCollection$ >> AbstractList$Int extends AbstractCollection$Int >> ArrayList$Int extends AbstractList$Int >> >> The new classes can even inherit the top-level class from the original >> hierarchy, as long as that class has no fields or at least no "container" >> Object[] fields. >> >> Returning a type like ArrayList means no changes in syntax, >> method signatures, classfile format, reflection etc. to support >> ArrayList. So the proposal is a simple library without any exceptional >> demands - which is the only kind of proposal that we could hope adding to >> JDK 7, at this late time. The tradeoff is that we're stuck with signatures >> like add(I)Z, so we depend on HotSpot to optimize out the boxing/unboxing >> that happens in each method call. At least for inlined calls I guess this >> optimization is already done; the remaining work should be feasible, and it >> could wait for some 7uXX maintenance update if there's no time do do that >> until FCS. >> >> There is no Object[] field to worry about... but still, some Object[]s in >> APIs like toArray(). In that case there is no significant extra conversion >> cost (toArray() already does a defensive copy so we just do a different copy >> from int[] to Object[]). The reified class could implement some extra, >> type-specific helper interface that adds methods like int[] toArrayInt(); >> this would probably make the transformer more tightly-bound to the Java SE >> Collections API, or require extra annotations for custom additions; not >> pretty, but not hard to implement. And while we're at it, I would love to >> add equivalent methods that DON'T do a defensive copy, just return the >> internal array, or assign an external array to the collection's field. >> Perf-critical code often benefits from that; I have personally had this >> problem. When your code is already optimized enough, redundant data copies >> always bubble up to the top of the performance profile. This is why >> estimating the initial size is everybody's favorite collections >> optimization, as it avoids lots of allocation and copy from growing. >> >> Finally, it's also easy to implement as the bytecode transformation, full >> reification for any non-public method. So we can't touch add(Object), but we >> certainly can do that for all private and even protected methods, and also >> entire non-public helper classes like AbstractList's Itr, ListItr, SubList, >> and RandomAccessSubList. >> >> I don't like ideas like new IntList(), or some Lists.of() that's just a >> factory to hide types like IntList. This would add enormous bloat to the >> core, and/or would be very limited - there's a hefty number of collection >> classes in the core (not to mention third-party code) and I may need >> primitive-optimized versions of ANY of these classes. If I need a List of >> int, I may also need the specific features or O(N) traits of particular >> implementations like ArrayList, LinkedList, or even CopyOnWriteArrayList. >> Don't make me choose between optimal element storage (primitive reification) >> and optimal API/structure (compact array, linked nodes, concurrency etc.). A >> partial solution is sometimes worse than no solution at all - it avoids a >> much bigger mess, e.g. using a core collection for ArrayList-of-int, and GNU >> Trove for LinkedList-of-int or whatever. >> >> As a final remark, this implementation would be valid: >> >> T reify (T collection, Class elementType) { >> return collection; >> } >> >> ..it just wouldn't gain any performance. But it's good enough, for >> example, to add (as a compatibility feature) to platforms where code >> footprint is a higher concern, like the lowest Java ME profiles. Or even to >> the first release of JDK 7, if there is no time for a full implementation of >> this API. >> >> A+ >> Osvaldo >> >> >> 2010/7/14 Reinier Zwitserloot >> >> Indeed, having: new ArrayList(); be as fast as an int array requires >>> reification; once ArrayList creates a backing array of type Object[], the >>> performance game is lost. >>> >>> Nothing short of reification can address this issue. To be specific, >>> specialization won't solve it any more than primitives-in-generics does. >>> However, this: >>> >>> List list = Lists.of(int.class); >>> list.add(10); >>> >>> or: >>> >>> List list = new IntList(); >>> list.add(10); >>> >>> >>> *CAN* be made fast. Specialization, interestingly enough, cannot do >>> this*, >>> but my primitives in generics proposal can. Which partly makes me wonder >>> why >>> we're talking about the far more complicated specialization concept in >>> the >>> first place. I know scala uses this, but they either introduced their >>> collections API with specialization from day one, or they created a >>> backwards incompatible retrofit later. >>> >>> *) list.add(10), is, as far as the compiler knows, a call to add(T) on a >>> List. If this is to be fast under specialization, it would therefore >>> have to generate a call to an add method with signature (I)Z and not with >>> signature (Ljava/lang/Object;)Z like it would today (and autobox to make >>> the >>> int fit as an object). However, if it is to generate such a call, then >>> *ANY* >>> list, no matter how it was obtained, must have an add (I)Z method, but >>> lists >>> written today do not because that's not part of the List.java interface. >>> We >>> can't add methods to interfaces or we'd break backwards compatibility, >>> and >>> breaking backwards compatibility is not acceptable. Therefore, in list >>> implementations that lack add(I)Z, the JVM or the classloader has to >>> generate these. There's precedence of course: Brian Goetz' proposal for >>> defender methods does something quite similar. Still, this is far from >>> trivial, and I'd like to see a proposal on how one could update List.java >>> so >>> that the JVM / ClassLoader considers the specialization methods as >>> defenderish, but the object-based one itself not. Primitives-in-generics >>> keeps things simple by letting the call to (Ljava/lang/Object;)Z stand, >>> and >>> letting the JVM hotspot compiler eliminate the inefficiency instead. >>> >>> --Reinier Zwitserloot >>> >>> >>> >>> On Tue, Jul 13, 2010 at 11:08 PM, Nathan Bryant < >>> nathan.bryant at linkshare.com >>> > wrote: >>> >>> > John, it's just array creation >>> > >>> > class Foo { >>> > T[] arr = new T[SIZE]; // Can't write this without reification >>> > } >>> > >>> > class Foo { >>> > Object[] arr = new Object[SIZE]; >>> > >>> > void set(int i, T t) { arr[i] = t; } // Either causes autoboxing >>> > unbeknownst to the sorry fool who wrote new Foo, or becomes >>> illegal >>> > } >>> > >>> > >>> > John Nilsson wrote: >>> > >>> > > What exactly is it that requires reification to work? >>> > >>> > BR, >>> > John >>> > >>> > >>> >>> >> > From int19h at gmail.com Thu Jul 15 16:49:30 2010 From: int19h at gmail.com (Pavel Minaev) Date: Thu, 15 Jul 2010 16:49:30 -0700 Subject: Multiple SAMs for same concept In-Reply-To: References: Message-ID: On Thu, Jul 15, 2010 at 4:34 PM, Stephen Colebourne wrote: > One problem noted with the removal of function types is that multiple > SAMs with the same signature are not compatible. For example: > > http://gee.cs.oswego.edu/dl/jsr166/dist/extra166ydocs/extra166y/Ops.Generator.html > http://guava-libraries.googlecode.com/svn/trunk/javadoc/com/google/common/base/Supplier.html > http://google-guice.googlecode.com/svn/trunk/javadoc/com/google/inject/Provider.html > > Having these different incompatible definitions is problematic, for > example when the developer has created a library of reusable > implementations implementing one SAM, and then wants to use them using > another (eg. a newly added JDK SAM). > > Given the method: > ?private void process(Generator gen) { ... } > > the most recent draft SotL would allow the following two choices for conversion: > ?Supplier supplier = ... > ?process( {-> supplier.supply()} ); ?// manual conversion to a Generator > ?process( supplier#supply() ); ?// using method ref to manually > convert to Generator > > While the two solutions are not terrible, they are less than ideal. I > would argue that the ideal would be: > ?process(supplier); ?// auto-boxed from Supplier to Generator Such "autoboxing" (since no boxing happens here, I'd rather call it "autowrapping") wouldn't preserve object identity. I think it would be very confusing to pass a reference to object, only to receive a reference to a completely different (as in, r1 != r2) object on the other side - especially without any explicit casts or other indication that something unusual is going on. This wasn't a problem with primitives because they don't have their own object identity > One compromise option I came up with was: > process( supplier# ); This looks more reasonable. From collin.fagan at gmail.com Thu Jul 15 16:59:03 2010 From: collin.fagan at gmail.com (Collin Fagan) Date: Thu, 15 Jul 2010 18:59:03 -0500 Subject: Constructor references In-Reply-To: References: Message-ID: +1, Constructor references can turn a constructor into a factory which is a very handy tool to have. It could also remove the need in may cases to use reflection. Collin On Thu, Jul 15, 2010 at 6:37 PM, Stephen Colebourne wrote: > The latest SotL document does not mention constructor references. A > constructor is no more than a special kind of method. Omitting it from > JDK 7 would, I suspect, be an annoyance to developers. > > The obvious syntaxes are: > > Person#() > Person#(String, int) > > or > > Person#() > Person#(String, int) > > Has any consideration been given to constructor references? > > Stephen > > From lk at teamten.com Thu Jul 15 17:00:15 2010 From: lk at teamten.com (Lawrence Kesteloot) Date: Thu, 15 Jul 2010 17:00:15 -0700 Subject: Multiple SAMs for same concept In-Reply-To: References: Message-ID: In addition, just because the signatures are identical doesn't mean the contracts are identical. Provider.get(), for example, must never return null, but who knows what Generator.op() might return? Same with unchecked exceptions, allowed values of parameters, etc. The programmer should be forced to explicitly convert from one to the other (as in your "less than ideal" examples) to show that he has consciously decided to treat the SAMs as compatible. Lawrence On Thu, Jul 15, 2010 at 4:49 PM, Pavel Minaev wrote: > On Thu, Jul 15, 2010 at 4:34 PM, Stephen Colebourne > wrote: >> One problem noted with the removal of function types is that multiple >> SAMs with the same signature are not compatible. For example: >> >> http://gee.cs.oswego.edu/dl/jsr166/dist/extra166ydocs/extra166y/Ops.Generator.html >> http://guava-libraries.googlecode.com/svn/trunk/javadoc/com/google/common/base/Supplier.html >> http://google-guice.googlecode.com/svn/trunk/javadoc/com/google/inject/Provider.html >> >> Having these different incompatible definitions is problematic, for >> example when the developer has created a library of reusable >> implementations implementing one SAM, and then wants to use them using >> another (eg. a newly added JDK SAM). >> >> Given the method: >> ?private void process(Generator gen) { ... } >> >> the most recent draft SotL would allow the following two choices for conversion: >> ?Supplier supplier = ... >> ?process( {-> supplier.supply()} ); ?// manual conversion to a Generator >> ?process( supplier#supply() ); ?// using method ref to manually >> convert to Generator >> >> While the two solutions are not terrible, they are less than ideal. I >> would argue that the ideal would be: >> ?process(supplier); ?// auto-boxed from Supplier to Generator > > Such "autoboxing" (since no boxing happens here, I'd rather call it > "autowrapping") wouldn't preserve object identity. I think it would be > very confusing to pass a reference to object, only to receive a > reference to a completely different (as in, r1 != r2) object on the > other side - especially without any explicit casts or other indication > that something unusual is going on. > > This wasn't a problem with primitives because they don't have their > own object identity > >> One compromise option I came up with was: >> process( supplier# ); > > This looks more reasonable. > > From collin.fagan at gmail.com Thu Jul 15 17:11:41 2010 From: collin.fagan at gmail.com (Collin Fagan) Date: Thu, 15 Jul 2010 19:11:41 -0500 Subject: Multiple SAMs for same concept In-Reply-To: References: Message-ID: *> just because the signatures are identical doesn't mean the contracts are identical* This is true for any interface. Look at the implementations of Map. They are all different enough that many times they are not interchangeable. Yet Map appears in signatures of methods and constructors and everyone just hopes that they get something compatible. Most of the time the differences don't show themselves and that's good enough. I think in this case it's also good enough. I think the biggest obstacle here, as others have pointed out, is object identity. If this was implimented using a MethodHandle and not a wrapper class could one work around identity? Collin On Thu, Jul 15, 2010 at 7:00 PM, Lawrence Kesteloot wrote: > In addition, just because the signatures are identical doesn't mean > the contracts are identical. Provider.get(), for example, must never > return null, but who knows what Generator.op() might return? Same with > unchecked exceptions, allowed values of parameters, etc. The > programmer should be forced to explicitly convert from one to the > other (as in your "less than ideal" examples) to show that he has > consciously decided to treat the SAMs as compatible. > > Lawrence > > > On Thu, Jul 15, 2010 at 4:49 PM, Pavel Minaev wrote: > > On Thu, Jul 15, 2010 at 4:34 PM, Stephen Colebourne > > wrote: > >> One problem noted with the removal of function types is that multiple > >> SAMs with the same signature are not compatible. For example: > >> > >> > http://gee.cs.oswego.edu/dl/jsr166/dist/extra166ydocs/extra166y/Ops.Generator.html > >> > http://guava-libraries.googlecode.com/svn/trunk/javadoc/com/google/common/base/Supplier.html > >> > http://google-guice.googlecode.com/svn/trunk/javadoc/com/google/inject/Provider.html > >> > >> Having these different incompatible definitions is problematic, for > >> example when the developer has created a library of reusable > >> implementations implementing one SAM, and then wants to use them using > >> another (eg. a newly added JDK SAM). > >> > >> Given the method: > >> private void process(Generator gen) { ... } > >> > >> the most recent draft SotL would allow the following two choices for > conversion: > >> Supplier supplier = ... > >> process( {-> supplier.supply()} ); // manual conversion to a Generator > >> process( supplier#supply() ); // using method ref to manually > >> convert to Generator > >> > >> While the two solutions are not terrible, they are less than ideal. I > >> would argue that the ideal would be: > >> process(supplier); // auto-boxed from Supplier to Generator > > > > Such "autoboxing" (since no boxing happens here, I'd rather call it > > "autowrapping") wouldn't preserve object identity. I think it would be > > very confusing to pass a reference to object, only to receive a > > reference to a completely different (as in, r1 != r2) object on the > > other side - especially without any explicit casts or other indication > > that something unusual is going on. > > > > This wasn't a problem with primitives because they don't have their > > own object identity > > > >> One compromise option I came up with was: > >> process( supplier# ); > > > > This looks more reasonable. > > > > > > From lk at teamten.com Thu Jul 15 17:29:57 2010 From: lk at teamten.com (Lawrence Kesteloot) Date: Thu, 15 Jul 2010 17:29:57 -0700 Subject: Multiple SAMs for same concept In-Reply-To: References: Message-ID: On Thu, Jul 15, 2010 at 5:11 PM, Collin Fagan wrote: > This is true for any interface. I disagree. An interface like Map defines the responsibilities of the client and of the implementing class. If this is ambiguous, then it's a bug in the interface. If these are violated by the client or implementing class, then it's a bug there. But in the case of autowrapping, interfaces A and B may both be specified well, both the client and implementing classes may properly implement the contracts, but the application code thoughtlessly hooks up the implementation of A with a client of B. Then A returns null (which it's allowed to do) and the client of B doesn't handle it properly. This isn't something that happens with interfaces as they work today. The application programmers shouldn't be allowed to too easily and thoughtlessly mix and match interfaces this way. Lawrence From reinier at zwitserloot.com Thu Jul 15 17:56:16 2010 From: reinier at zwitserloot.com (Reinier Zwitserloot) Date: Fri, 16 Jul 2010 02:56:16 +0200 Subject: Multiple SAMs for same concept In-Reply-To: References: Message-ID: True, most lists of 'these SAMs are distinct but really mean the same thing' aren't 100% compatible with each other, but this in practice occurs all the time. For example, Map itself does not mention whether or not one can use null as either key or value. With HashMap, doing this is no problem, but with Hashtable, or for example ConcurrentHashMap, do *NOT* allow null as either key or value. When we receive an arbitrary Map object, and we decide to transfer the contents to a ConcurrentHashMap, problems can occur (specifically: NPEs). Nevertheless both the incoming map and the map we're trying to put them in follow the same contract. HashMap and Hashtable both implement the same interface, both do so entirely correctly, and yet they aren't compatible with each other in some regard - in this case, null keys/values. In effect, then, something like "supplier#" is not just a compiler crutch, it's conveying a useful construct: It's saying: I vouch for the notion that these things will end up being compatible. The compiler can't check every aspect of the contract, unfortunately. NB: For what its worth, I don't really see the problem with { x -> supplier(x)}, that's barely longer than supplier#, and is instantly obvious to anyone that knows about closures, whereas supplier# is a new concept. There's a (small) argument in favour of supplier# if the arglist grows longer, but as the param list grows, the number of existing duplicate SAM types shrinks rapidly, so that factor can be effectively ignored. We should embrace the benefit of the SotL strategy that lets us omit the parameter type. --Reinier Zwitserloot On Fri, Jul 16, 2010 at 2:29 AM, Lawrence Kesteloot wrote: > On Thu, Jul 15, 2010 at 5:11 PM, Collin Fagan > wrote: > > This is true for any interface. > > I disagree. An interface like Map defines the responsibilities of the > client and of the implementing class. If this is ambiguous, then it's > a bug in the interface. If these are violated by the client or > implementing class, then it's a bug there. > > But in the case of autowrapping, interfaces A and B may both be > specified well, both the client and implementing classes may properly > implement the contracts, but the application code thoughtlessly hooks > up the implementation of A with a client of B. Then A returns null > (which it's allowed to do) and the client of B doesn't handle it > properly. > > This isn't something that happens with interfaces as they work today. > The application programmers shouldn't be allowed to too easily and > thoughtlessly mix and match interfaces this way. > > Lawrence > > From reinier at zwitserloot.com Thu Jul 15 18:02:20 2010 From: reinier at zwitserloot.com (Reinier Zwitserloot) Date: Fri, 16 Jul 2010 03:02:20 +0200 Subject: Transparency In-Reply-To: References: <7FDA6630E1822F448C97A48D5D733094010A91FB@EXVMSTOR302.intra.rakuten.co.jp> <4C3CCEEA.3070203@optrak.co.uk> <7FDA6630E1822F448C97A48D5D733094010A9261@EXVMSTOR302.intra.rakuten.co.jp> Message-ID: Throwing a NullPointerException in response to this code: List list = new ArrayList(); list.add(null); is not backwards compatible. --Reinier Zwitserloot On Fri, Jul 16, 2010 at 1:45 AM, Osvaldo Doederlein wrote: > 2010/7/15 Reinier Zwitserloot > > What happens if some class has multiple type parameters? Should we create a >> Map$Short$Byte? That's going to explode into a very very large number of >> types as you add on more type parameters. What seems to be required for your >> scheme is a mix of complete reification combined with a lot of magic >> bytecode rewriting, which you haven't fully elaborated on. > > > If you want a Map, yes, we create a Map$Short$Byte. This is > the reason why you need dynamic code expansion, that happens on demand at > reify() calls. You only pay for what you are actually using. The Cartesian > product of all your generic parameters in collections and other interesting > classes X all primitive types would indeed result in thousands of possible > expansions; but in practice, any given application is not likely to use more > than a handful of different expansions. The reifier would hold the produced > classes in a weak cache, so when some class is not used for a long time, > it's class-GC'd. (Same technique already used by synthetic classes produced > by reflection, and other dynamic frameworks.) > > >> A few questions that I'm not sure your proposal answers: >> - When should the JVM stop specializing new classes? With for example 4 >> type parameters, there would be 8*8*8*8 = 4096 different classes required, >> which seems quite infeasible. >> > > As explained above, this only happens if the process actually instantiates > all these 4096 combinations... not very likely. > > >> - Exactly how does Magic.reify() work? The backing array field in the >> ArrayList implementation is not of type T[]; it's of type Object[]. How >> could any library know that it should rewrite this Object[] to int[] in the >> generated specialization without hardcoding a ruleset for all known classes? >> > > I have actually proposed this, a configuration-by-exception mechanism - you > can have annotations to hint what to change, but the reifier could also have > default rules and heuristics to use in classes that lack such metadata. > > >> - Even if ArrayList is rewritten to use T[] instead of Object[], that's >> no help for third party classes that used Object[] - Oracle can't very well >> change every library ever written for java. In a class where T[] cannot be >> rewritten to int[], exactly what does ArrayList$Int even do? Presumably it >> would have to have an (I)Z method auto-generated by the classloader as >> calling code will expect it to exist. >> > > Non an issue, see previous item. BTW the reifier _must_ only rely on the > erased types like Object[], because it's a runtime transformation on > bytecode, not a compile-time tool that could see source code. > > >> - What happens when heap corruption occurs? i.e: List list = new >> ArrayList() /* Raw type! */; list.add(10) /* Generates call to non-existing >> (I)Z add method */; - Normally this results in a MethodNotFoundError, which >> is rarely caught as errors aren't usually supposed to be caught. That seems >> entirely inappropriate for heap corruption. >> > > See > http://mail.openjdk.java.net/pipermail/lambda-dev/2010-July/001925.html, > we won't have any (I)Z method, we just call the standard add(Object) and > rely on VM optimizations to get rid of the overhead of box/CALL/unbox > sequences. > > >> - If the syntax is to remain ArrayList and not ArrayList, >> then what happens if one attempts to add null? >> > > The first thing that the reified version of add(Object) does is unboxing - > ((Integer)e).intValue() - so the result is a NullPointerException - that is > consistent with the existing autoboxing semantics. Notice that when the VM > can optimize those box/CALL/unbox patterns, the issue is non-existent as the > original value is guaranteed not-null. > > >> - How does the auto-generator know that isEmpty() is not dependent on >> reification, but e.g. add() is? >> NB: At this point in time, the Oracle JVM does *NOT* eliminate autoboxing >> and unboxing, though the JRockit VM does. >> > > I know that HotSpot doesn't provide special optimizations for autoboxing, > but I was just guessing that it can remove this overhead at least for > trivial cases as effect of EA & scalar replacement... if not, hopefully this > could be added with little effort, I guess it's just a matter of > intrinsifying the valueOf()/xxxValue() methods. > > A+ > Osvaldo > > >> >> >> >> >> >> --Reinier Zwitserloot >> >> >> >> >> On Wed, Jul 14, 2010 at 6:16 PM, Osvaldo Doederlein wrote: >> >>> A concrete suggestion: we could support this via bytecode transformation, >>> e.g. >>> >>> ArrayList list = Magic.reify(ArrayList.class, int.class); >>> >>> ...where reify() will basically change the type of ArrayList's internal >>> Object[] backing store to int[]. This would probably benefit some help from >>> the original collection, e.g. annotations that list which >>> fields/parameters/returns/locals must be changed (or not); but this mapping >>> could be provided on a configuration-by-exception manner. The bytecode >>> transformation is mostly simple, it just needs also to translate some API >>> calls (e.g. to Arrays.copyOf(T[], int, int)) to equivalent calls for the >>> required primitive-array type (or fail the whole thing, if it hits some call >>> that is unknown to the transformer or has no primitive-friendly overloads). >>> >>> Several reifications of the same origin class could be arranged in a >>> hierarchy that's designed to both reflect the original hierarchy and to >>> share methods that don't directly depend on the element type (e.g. isEmpty() >>> { return size()==0; }). So we get synthetic classes like: >>> >>> AbstractCollection$ extends AbstractCollection >>> AbstractCollection$Int extends AbstractCollection$ >>> AbstractList$Int extends AbstractCollection$Int >>> ArrayList$Int extends AbstractList$Int >>> >>> The new classes can even inherit the top-level class from the original >>> hierarchy, as long as that class has no fields or at least no "container" >>> Object[] fields. >>> >>> Returning a type like ArrayList means no changes in syntax, >>> method signatures, classfile format, reflection etc. to support >>> ArrayList. So the proposal is a simple library without any exceptional >>> demands - which is the only kind of proposal that we could hope adding to >>> JDK 7, at this late time. The tradeoff is that we're stuck with signatures >>> like add(I)Z, so we depend on HotSpot to optimize out the boxing/unboxing >>> that happens in each method call. At least for inlined calls I guess this >>> optimization is already done; the remaining work should be feasible, and it >>> could wait for some 7uXX maintenance update if there's no time do do that >>> until FCS. >>> >>> There is no Object[] field to worry about... but still, some Object[]s in >>> APIs like toArray(). In that case there is no significant extra conversion >>> cost (toArray() already does a defensive copy so we just do a different copy >>> from int[] to Object[]). The reified class could implement some extra, >>> type-specific helper interface that adds methods like int[] toArrayInt(); >>> this would probably make the transformer more tightly-bound to the Java SE >>> Collections API, or require extra annotations for custom additions; not >>> pretty, but not hard to implement. And while we're at it, I would love to >>> add equivalent methods that DON'T do a defensive copy, just return the >>> internal array, or assign an external array to the collection's field. >>> Perf-critical code often benefits from that; I have personally had this >>> problem. When your code is already optimized enough, redundant data copies >>> always bubble up to the top of the performance profile. This is why >>> estimating the initial size is everybody's favorite collections >>> optimization, as it avoids lots of allocation and copy from growing. >>> >>> Finally, it's also easy to implement as the bytecode transformation, full >>> reification for any non-public method. So we can't touch add(Object), but we >>> certainly can do that for all private and even protected methods, and also >>> entire non-public helper classes like AbstractList's Itr, ListItr, SubList, >>> and RandomAccessSubList. >>> >>> I don't like ideas like new IntList(), or some Lists.of() that's just a >>> factory to hide types like IntList. This would add enormous bloat to the >>> core, and/or would be very limited - there's a hefty number of collection >>> classes in the core (not to mention third-party code) and I may need >>> primitive-optimized versions of ANY of these classes. If I need a List of >>> int, I may also need the specific features or O(N) traits of particular >>> implementations like ArrayList, LinkedList, or even CopyOnWriteArrayList. >>> Don't make me choose between optimal element storage (primitive reification) >>> and optimal API/structure (compact array, linked nodes, concurrency etc.). A >>> partial solution is sometimes worse than no solution at all - it avoids a >>> much bigger mess, e.g. using a core collection for ArrayList-of-int, and GNU >>> Trove for LinkedList-of-int or whatever. >>> >>> As a final remark, this implementation would be valid: >>> >>> T reify (T collection, Class elementType) { >>> return collection; >>> } >>> >>> ..it just wouldn't gain any performance. But it's good enough, for >>> example, to add (as a compatibility feature) to platforms where code >>> footprint is a higher concern, like the lowest Java ME profiles. Or even to >>> the first release of JDK 7, if there is no time for a full implementation of >>> this API. >>> >>> A+ >>> Osvaldo >>> >>> >>> 2010/7/14 Reinier Zwitserloot >>> >>> Indeed, having: new ArrayList(); be as fast as an int array requires >>>> reification; once ArrayList creates a backing array of type Object[], >>>> the >>>> performance game is lost. >>>> >>>> Nothing short of reification can address this issue. To be specific, >>>> specialization won't solve it any more than primitives-in-generics does. >>>> However, this: >>>> >>>> List list = Lists.of(int.class); >>>> list.add(10); >>>> >>>> or: >>>> >>>> List list = new IntList(); >>>> list.add(10); >>>> >>>> >>>> *CAN* be made fast. Specialization, interestingly enough, cannot do >>>> this*, >>>> but my primitives in generics proposal can. Which partly makes me wonder >>>> why >>>> we're talking about the far more complicated specialization concept in >>>> the >>>> first place. I know scala uses this, but they either introduced their >>>> collections API with specialization from day one, or they created a >>>> backwards incompatible retrofit later. >>>> >>>> *) list.add(10), is, as far as the compiler knows, a call to add(T) on a >>>> List. If this is to be fast under specialization, it would >>>> therefore >>>> have to generate a call to an add method with signature (I)Z and not >>>> with >>>> signature (Ljava/lang/Object;)Z like it would today (and autobox to make >>>> the >>>> int fit as an object). However, if it is to generate such a call, then >>>> *ANY* >>>> list, no matter how it was obtained, must have an add (I)Z method, but >>>> lists >>>> written today do not because that's not part of the List.java interface. >>>> We >>>> can't add methods to interfaces or we'd break backwards compatibility, >>>> and >>>> breaking backwards compatibility is not acceptable. Therefore, in list >>>> implementations that lack add(I)Z, the JVM or the classloader has to >>>> generate these. There's precedence of course: Brian Goetz' proposal for >>>> defender methods does something quite similar. Still, this is far from >>>> trivial, and I'd like to see a proposal on how one could update >>>> List.java so >>>> that the JVM / ClassLoader considers the specialization methods as >>>> defenderish, but the object-based one itself not. Primitives-in-generics >>>> keeps things simple by letting the call to (Ljava/lang/Object;)Z stand, >>>> and >>>> letting the JVM hotspot compiler eliminate the inefficiency instead. >>>> >>>> --Reinier Zwitserloot >>>> >>>> >>>> >>>> On Tue, Jul 13, 2010 at 11:08 PM, Nathan Bryant < >>>> nathan.bryant at linkshare.com >>>> > wrote: >>>> >>>> > John, it's just array creation >>>> > >>>> > class Foo { >>>> > T[] arr = new T[SIZE]; // Can't write this without reification >>>> > } >>>> > >>>> > class Foo { >>>> > Object[] arr = new Object[SIZE]; >>>> > >>>> > void set(int i, T t) { arr[i] = t; } // Either causes >>>> autoboxing >>>> > unbeknownst to the sorry fool who wrote new Foo, or becomes >>>> illegal >>>> > } >>>> > >>>> > >>>> > John Nilsson wrote: >>>> > >>>> > > What exactly is it that requires reification to work? >>>> > >>>> > BR, >>>> > John >>>> > >>>> > >>>> >>>> >>> >> > From opinali at gmail.com Thu Jul 15 19:09:42 2010 From: opinali at gmail.com (Osvaldo Doederlein) Date: Thu, 15 Jul 2010 23:09:42 -0300 Subject: Transparency In-Reply-To: References: <7FDA6630E1822F448C97A48D5D733094010A91FB@EXVMSTOR302.intra.rakuten.co.jp> <4C3CCEEA.3070203@optrak.co.uk> <7FDA6630E1822F448C97A48D5D733094010A9261@EXVMSTOR302.intra.rakuten.co.jp> Message-ID: 2010/7/15 Reinier Zwitserloot > > Throwing a NullPointerException in response to this code: > List list = new ArrayList(); > list.add(null); > is not backwards compatible. I agree. But the proposed reification will not allocate a list with "new ArrayList()". It will use a special, explicit factory method like reify(ArrayList.class, int.class); and the return value has type List. The Collections spec allows specific implementations to not tolerate null values. For a general-purpose reifier that handles any generic class, not just java.util's collections, we would just document that the returned class will NPE at any attempt of using null values. I know that this isn't the most elegant design, but the practical impact is basically zero. If you need to pass some List to a legacy method that will pass nulls to some methods in that list, you just avoid using a reified List. There is no better solution if we desire compatibility with traditional interfaces, with method signatures like add(Object). A+ Osvaldo > > ?--Reinier Zwitserloot > > > > On Fri, Jul 16, 2010 at 1:45 AM, Osvaldo Doederlein wrote: >> >> 2010/7/15 Reinier Zwitserloot >>> >>> What happens if some class has multiple type parameters? Should we create a Map$Short$Byte? That's going to explode into a very very large number of types as you add on more type parameters. What seems to be required for your scheme is a mix of complete reification combined with a lot of magic bytecode rewriting, which you haven't fully elaborated on. >> >> If you want a Map, yes, we create a Map$Short$Byte. This is the reason why you need dynamic code expansion, that happens on demand at reify() calls. You only pay for what you are actually using. The Cartesian product of all your generic parameters in collections and other interesting classes X all primitive types would indeed result in thousands of possible expansions; but in practice, any given application is not likely to use more than a handful of different expansions. The reifier would hold the produced classes in a weak cache, so when some class is not used for a long time, it's class-GC'd. (Same technique already used by synthetic classes produced by reflection, and other dynamic frameworks.) >> >>> >>> A few questions that I'm not sure your proposal answers: >>> ?- When should the JVM stop specializing new classes? With for example 4 type parameters, there would be 8*8*8*8 = 4096 different classes required, which seems quite infeasible. >> >> As explained above, this only happens if the process actually instantiates all these 4096 combinations... not very likely. >> >>> >>> ?- Exactly how does Magic.reify() work? The backing array field in the ArrayList implementation is not of type T[]; it's of type Object[]. How could any library know that it should rewrite this Object[] to int[] in the generated specialization without hardcoding a ruleset for all known classes? >> >> I have actually proposed this, a configuration-by-exception mechanism - you can have annotations to hint what to change, but the reifier could also have default rules and heuristics to use in classes that lack such metadata. >> >>> >>> ?- Even if ArrayList is rewritten to use T[] instead of Object[], that's no help for third party classes that used Object[] - Oracle can't very well change every library ever written for java. In a class where T[] cannot be rewritten to int[], exactly what does ArrayList$Int even do? Presumably it would have to have an (I)Z method auto-generated by the classloader as calling code will expect it to exist. >> >> Non an issue, see previous item. BTW the reifier _must_ only rely on the erased types like Object[], because it's a runtime transformation on bytecode, not a compile-time tool that could see source code. >> >>> >>> ?- What happens when heap corruption occurs? i.e: List list = new ArrayList() /* Raw type! */; list.add(10) /* Generates call to non-existing (I)Z add method */; - Normally this results in a MethodNotFoundError, which is rarely caught as errors aren't usually supposed to be caught. That seems entirely inappropriate for heap corruption. >> >> See http://mail.openjdk.java.net/pipermail/lambda-dev/2010-July/001925.html, we won't have any (I)Z method, we just call the standard add(Object) and rely on VM optimizations to get rid of the overhead of box/CALL/unbox sequences. >> >>> >>> ?- If the syntax is to remain ArrayList and not ArrayList, then what happens if one attempts to add null? >> >> The first thing that the reified version of add(Object) does is unboxing - ((Integer)e).intValue() - so the result is a NullPointerException - that is consistent with the existing autoboxing semantics. Notice that when the VM can optimize those box/CALL/unbox patterns, the issue is non-existent as the original value is guaranteed not-null. >> >>> >>> ?- How does the auto-generator know that isEmpty() is not dependent on reification, but e.g. add() is? >>> NB: At this point in time, the Oracle JVM does *NOT* eliminate autoboxing and unboxing, though the JRockit VM does. >> >> I know that HotSpot doesn't provide special optimizations for autoboxing, but I was just guessing that it can remove this overhead at least for trivial cases as effect of EA & scalar replacement... if not, hopefully this could be added with little effort, I guess it's just a matter of intrinsifying the valueOf()/xxxValue() methods. >> >> A+ >> Osvaldo >> >>> >>> >>> >>> >>> ?--Reinier Zwitserloot >>> >>> >>> >>> On Wed, Jul 14, 2010 at 6:16 PM, Osvaldo Doederlein wrote: >>>> >>>> A concrete suggestion: we could support this via bytecode transformation, e.g. >>>> >>>> ArrayList list = Magic.reify(ArrayList.class, int.class); >>>> >>>> ...where reify() will basically change the type of ArrayList's internal Object[] backing store to int[]. This would probably benefit some help from the original collection, e.g. annotations that list which fields/parameters/returns/locals must be changed (or not); but this mapping could be provided on a configuration-by-exception manner. The bytecode transformation is mostly simple, it just needs also to translate some API calls (e.g. to Arrays.copyOf(T[], int, int)) to equivalent calls for the required primitive-array type (or fail the whole thing, if it hits some call that is unknown to the transformer or has no primitive-friendly overloads). >>>> >>>> Several reifications of the same origin class could be arranged in a hierarchy that's designed to both reflect the original hierarchy and to share methods that don't directly depend on the element type (e.g. isEmpty() { return size()==0; }). So we get synthetic classes like: >>>> >>>> AbstractCollection$ extends AbstractCollection >>>> AbstractCollection$Int extends AbstractCollection$ >>>> AbstractList$Int extends AbstractCollection$Int >>>> ArrayList$Int extends AbstractList$Int >>>> >>>> The new classes can even inherit the top-level class from the original hierarchy, as long as that class has no fields or at least no "container" Object[] fields. >>>> >>>> Returning a type like ArrayList means no changes in syntax, method signatures, classfile format, reflection etc. to support ArrayList. So the proposal is a simple library without any exceptional demands - which is the only kind of proposal that we could hope adding to JDK 7, at this late time. The tradeoff is that we're stuck with signatures like add(I)Z, so we depend on HotSpot to optimize out the boxing/unboxing that happens in each method call. At least for inlined calls I guess this optimization is already done; the remaining work should be feasible, and it could wait for some 7uXX maintenance update if there's no time do do that until FCS. >>>> >>>> There is no Object[] field to worry about... but still, some Object[]s in APIs like toArray(). In that case there is no significant extra conversion cost (toArray() already does a defensive copy so we just do a different copy from int[] to Object[]). The reified class could implement some extra, type-specific helper interface that adds methods like int[] toArrayInt(); this would probably make the transformer more tightly-bound to the Java SE Collections API, or require extra annotations for custom additions; not pretty, but not hard to implement. And while we're at it, I would love to add equivalent methods that DON'T do a defensive copy, just return the internal array, or assign an external array to the collection's field. Perf-critical code often benefits from that; I have personally had this problem. When your code is already optimized enough, redundant data copies always bubble up to the top of the performance profile. This is why estimating the initial size is everybody's favorite collections optimization, as it avoids lots of allocation and copy from growing. >>>> >>>> Finally, it's also easy to implement as the bytecode transformation, full reification for any non-public method. So we can't touch add(Object), but we certainly can do that for all private and even protected methods, and also entire non-public helper classes like AbstractList's Itr, ListItr, SubList, and RandomAccessSubList. >>>> >>>> I don't like ideas like new IntList(), or some Lists.of() that's just a factory to hide types like IntList. This would add enormous bloat to the core, and/or would be very limited - there's a hefty number of collection classes in the core (not to mention third-party code) and I may need primitive-optimized versions of ANY of these classes. If I need a List of int, I may also need the specific features or O(N) traits of particular implementations like ArrayList, LinkedList, or even CopyOnWriteArrayList. Don't make me choose between optimal element storage (primitive reification) and optimal API/structure (compact array, linked nodes, concurrency etc.). A partial solution is sometimes worse than no solution at all - it avoids a much bigger mess, e.g. using a core collection for ArrayList-of-int, and GNU Trove for LinkedList-of-int or whatever. >>>> >>>> As a final remark, this implementation would be valid: >>>> >>>> T reify (T collection, Class elementType) { >>>> ? return collection; >>>> } >>>> >>>> ..it just wouldn't gain any performance. But it's good enough, for example, to add (as a compatibility feature) to platforms where code footprint is a higher concern, like the lowest Java ME profiles. Or even to the first release of JDK 7, if there is no time for a full implementation of this API. >>>> >>>> A+ >>>> Osvaldo >>>> >>>> >>>> 2010/7/14 Reinier Zwitserloot >>>>> >>>>> Indeed, having: new ArrayList(); be as fast as an int array requires >>>>> reification; once ArrayList creates a backing array of type Object[], the >>>>> performance game is lost. >>>>> >>>>> Nothing short of reification can address this issue. To be specific, >>>>> specialization won't solve it any more than primitives-in-generics does. >>>>> However, this: >>>>> >>>>> List list = Lists.of(int.class); >>>>> list.add(10); >>>>> >>>>> or: >>>>> >>>>> List list = new IntList(); >>>>> list.add(10); >>>>> >>>>> >>>>> *CAN* be made fast. Specialization, interestingly enough, cannot do this*, >>>>> but my primitives in generics proposal can. Which partly makes me wonder why >>>>> we're talking about the far more complicated specialization concept in the >>>>> first place. I know scala uses this, but they either introduced their >>>>> collections API with specialization from day one, or they created a >>>>> backwards incompatible retrofit later. >>>>> >>>>> *) list.add(10), is, as far as the compiler knows, a call to add(T) on a >>>>> List. If this is to be fast under specialization, it would therefore >>>>> have to generate a call to an add method with signature (I)Z and not with >>>>> signature (Ljava/lang/Object;)Z like it would today (and autobox to make the >>>>> int fit as an object). However, if it is to generate such a call, then *ANY* >>>>> list, no matter how it was obtained, must have an add (I)Z method, but lists >>>>> written today do not because that's not part of the List.java interface. We >>>>> can't add methods to interfaces or we'd break backwards compatibility, and >>>>> breaking backwards compatibility is not acceptable. Therefore, in list >>>>> implementations that lack add(I)Z, the JVM or the classloader has to >>>>> generate these. There's precedence of course: Brian Goetz' proposal for >>>>> defender methods does something quite similar. Still, this is far from >>>>> trivial, and I'd like to see a proposal on how one could update List.java so >>>>> that the JVM / ClassLoader considers the specialization methods as >>>>> defenderish, but the object-based one itself not. Primitives-in-generics >>>>> keeps things simple by letting the call to (Ljava/lang/Object;)Z stand, and >>>>> letting the JVM hotspot compiler eliminate the inefficiency instead. >>>>> >>>>> ?--Reinier Zwitserloot >>>>> >>>>> >>>>> >>>>> On Tue, Jul 13, 2010 at 11:08 PM, Nathan Bryant >>>> > wrote: >>>>> >>>>> > John, it's just array creation >>>>> > >>>>> > class Foo { >>>>> > ? ? ? ?T[] arr = new T[SIZE]; // Can't write this without reification >>>>> > } >>>>> > >>>>> > class Foo { >>>>> > ? ? ? ?Object[] arr = new Object[SIZE]; >>>>> > >>>>> > ? ? ? ?void set(int i, T t) { arr[i] = t; } // Either causes autoboxing >>>>> > unbeknownst to the sorry fool who wrote new Foo, or becomes illegal >>>>> > } >>>>> > >>>>> > >>>>> > John Nilsson wrote: >>>>> > >>>>> > > What exactly is it that requires reification to work? >>>>> > >>>>> > BR, >>>>> > John >>>>> > >>>>> > >>>>> >>>> >>> >> > From int19h at gmail.com Thu Jul 15 20:28:23 2010 From: int19h at gmail.com (Pavel Minaev) Date: Thu, 15 Jul 2010 20:28:23 -0700 Subject: Multiple SAMs for same concept In-Reply-To: References: Message-ID: On Thu, Jul 15, 2010 at 5:56 PM, Reinier Zwitserloot wrote: > True, most lists of 'these SAMs are distinct but really mean the same thing' > aren't 100% compatible with each other, but this in practice occurs all the > time. For example, Map itself does not mention whether or not one can use > null as either key or value. Actually, Map does mention that either case is valid. In other words, the fact that null keys _may_ be invalid is part of the contract, so client knows that he must not try to add entries with such keys, and also knows that he must be prepared to deal with such entries while iterating etc. A more apt comparison would be some form of structural typing for interfaces - the ability to take any interface, and "magic-cast" it to another unrelated interface which just happens to have the same methods. This proposal is, in effect, the same thing, but restricted to SAMs. From lk at teamten.com Thu Jul 15 20:34:26 2010 From: lk at teamten.com (Lawrence Kesteloot) Date: Thu, 15 Jul 2010 20:34:26 -0700 Subject: Multiple SAMs for same concept In-Reply-To: References: Message-ID: On Thu, Jul 15, 2010 at 5:56 PM, Reinier Zwitserloot wrote: > HashMap and Hashtable both > implement the same interface, both do so entirely correctly, and yet they > aren't compatible with each other in some regard - in this case, null > keys/values. Exactly, and if a function took a Map object it would know to defend itself against null keys and values (for example, by not blindly calling toString() on either). But imagine a signature-compatible NonNullMap interface and some syntactic sugar that allowed developers to trivially autowrap objects from one to the other. It'd be too easy to send an object into a method that violated the contract of the parameter. In any case, I think we agree that it shouldn't be completely transparent, which is what I was objecting to. The variant with only # appended is Perl-like magic noise to me, but I always prefer explicitness at the expense of terseness. (We need a variant of Godwin's Law: "As an online programming language discussion grows longer, the probability of a comparison involving Perl approaches 1, and the person who brings it up has lost.") Lawrence From oehrstroem at gmail.com Fri Jul 16 00:31:04 2010 From: oehrstroem at gmail.com (Fredrik Ohrstrom) Date: Fri, 16 Jul 2010 09:31:04 +0200 Subject: Constructor references In-Reply-To: References: Message-ID: Yes, however it is not quite a normal method reference because the constructor does not allocate so you want the constructor prefixed with an alloc. To do that new code has to be created, so why not go with a lambda? { name, age -> new Person(name, age) } 2010/7/16 Stephen Colebourne : > The latest SotL document does not mention constructor references. A > constructor is no more than a special kind of method. Omitting it from > JDK 7 would, I suspect, be an annoyance to developers. > > The obvious syntaxes are: > > ?Person#() > ?Person#(String, int) > > or > > ?Person#() > ?Person#(String, int) > > Has any consideration been given to constructor references? > > Stephen > > From scolebourne at joda.org Fri Jul 16 02:46:00 2010 From: scolebourne at joda.org (Stephen Colebourne) Date: Fri, 16 Jul 2010 10:46:00 +0100 Subject: Constructor references In-Reply-To: References: Message-ID: On 16 July 2010 08:31, Fredrik Ohrstrom wrote: > Yes, however it is not quite a normal method reference because the > constructor does not allocate so you want the constructor prefixed > with an alloc. Well, I'd simply make the constructor reference do the allocation. Thats the most expected behaviour. > To do that new code has to be created, so why not > go with a lambda? > { name, age -> new Person(name, age) } Same argument applies to method references. They are being added because they are a common and useful convenience. Stephen From forax at univ-mlv.fr Fri Jul 16 03:08:55 2010 From: forax at univ-mlv.fr (=?ISO-8859-1?Q?R=E9mi_Forax?=) Date: Fri, 16 Jul 2010 12:08:55 +0200 Subject: Constructor references In-Reply-To: References: Message-ID: <4C402FB7.6020600@univ-mlv.fr> Le 16/07/2010 11:46, Stephen Colebourne a ?crit : > On 16 July 2010 08:31, Fredrik Ohrstrom wrote: > >> Yes, however it is not quite a normal method reference because the >> constructor does not allocate so you want the constructor prefixed >> with an alloc. >> > Well, I'd simply make the constructor reference do the allocation. > Thats the most expected behaviour. > Agree, that why the syntax should contains 'new'. The fact you need 3 instruction to do an allocation is a VM language quirk not a part of Java the language story. By the way, John (and I agree with him) already propose a syntax for constructor reference. see method testConstructors() in http://hg.openjdk.java.net/mlvm/mlvm/langtools/file/5fd2ac635231/meth-ldc-6939203.patch the proposed syntax is #new String(String) or outer#new Inner() for a non static inner class. > >> To do that new code has to be created, so why not >> go with a lambda? >> { name, age -> new Person(name, age) } >> > Same argument applies to method references. They are being added > because they are a common and useful convenience. > > Stephen > > R?mi From peter.levart at marand.si Fri Jul 16 08:28:46 2010 From: peter.levart at marand.si (Peter Levart) Date: Fri, 16 Jul 2010 17:28:46 +0200 Subject: Primitives in Generics (+ the meaning of this) In-Reply-To: <4C377AB5.7050205@oracle.com> References: <4C3778EB.60405@cs.oswego.edu> <4C377AB5.7050205@oracle.com> Message-ID: <201007161728.46392.peter.levart@marand.si> On 07/09/10, Brian Goetz wrote: > >> ... some mad professor cooked up :) :) :) > > > > My take is that you have three choices: > > A) Support function types covering primitives > > B) Support primitives in generics > > C) Do something that doesn't restrict your options to reconsider > > (A) or (B) in Java8 in response to all the complaints about > > sucky parallel performance. > > Indeed so. We believe we have chosen (C). If we have not, please say > something! > Hello Brian, I just have a question regarding C -> A path. It concerns the meaning of explicit and implicit "this" inside lambda body. As proposed in latest lambda -> SAM conversion, "this" refers to the target SAM instance, so does "implicit this" - lambda body "sees" target SAM instance unqualified members first and then other lexicaly surrounding stuff. What would be the meaning of "this" in the (A) future when lambda expressions are "converted" to a function type? Would it be the same or different as when converted to a SAM type? Regardless of that, do you aggree that the statistical evidence, prepared by some members of this list, showed that usage of explicit and implicit "this" inside anonymous inner SAM subclass creation expressions is practically non-existent and is actualy "in the way" of using outer.this and/or outer instance members (even a cause of bugs in JDK itself)? So why did you choose to treat "this" inside lambda body the way it is specified? Regards, Peter From peter.levart at marand.si Fri Jul 16 08:58:46 2010 From: peter.levart at marand.si (Peter Levart) Date: Fri, 16 Jul 2010 17:58:46 +0200 Subject: Removal of function types In-Reply-To: <4C34C460.800@oracle.com> References: <4C34C460.800@oracle.com> Message-ID: <201007161758.46836.peter.levart@marand.si> On 07/07/10, Brian Goetz wrote: > > The latest (6 July) lambda document removes function types. I support > > this (even though they were in FCM). > > Indeed it does. People will ask why, so let me share my thinking here. Among > other reasons: > > 1. There are two basic approaches to typing: nominal and structural. The > identity of a nominal is based on its name; the identity of a structural type > is based on what it is composed of (such as "tuple of int, int" or "function > from int to float".) > > Most languages pick mostly nominal or mostly structural; there are not a lot > of languages that successfully mix nominal and structural typing except > "around the edges." Java is almost entirely nominal (with a few exceptions: > arrays are a structural type, but at the bottom there is always a nominal > element type; generics have a mix of nominal and structural too, and this is > in fact part of the source of many of people's complaints about generics.) > > Grafting a structural type system (function types) onto Java's nominal type > system means new complexity and edge cases. Is the benefit of function types > worth this? Brian, I think it is and for that reason I urge you to not do anything that would prevent adding function types later (maybe in JDK8). The best way to make sure this doesn't happen is to specify them now, but leave them out of implementation for JDK7 (maybe even put them into the prototype but so that they can be "disabled" with a boolean flag). I think that current specification that removes function types could be accompanied with an optional addendum called "function types". BGGA specification did something like that. This way any conflicts found between "main lambda specification" and "function types addendum" could be polished out early. The "function types" addendum could be carried out as a community effort if Oracle has limited resources. I'm sure there are volunteers on this list (me included) if Oracle is prepared to consider this as a viable way to affect the "main specification" in areas where conflicts with "function types addendum" are identified. What do you think? Regards, Peter > > 2. We've been using SAM types for years, and Java developers are comfortable > with them. If we added function types, it would immediately bifurcate the > libraries (both JDK and external) into "old style" libraries (those that use > SAM types) and "new style" libraries (those that use function types.) That's > a big ugly seam in the middle of the platform. > > 3. No reification. There was a long thread about how useful it would be for > function types to be reified. Without reification, function types are hobbled. > > For these reasons and others, we would prefer to leave out function types now. > > > - Removing them reduces the scope, thus increasing the delivery likelihood > > - Function types look pig ugly with exception transparancy > > - Function types are a very different concept to the rest of Java, > > which is very name based > > - Function type don't visually look much like types (which are names > > everywhere else) > > - SAMs centralise the definition of Javadoc and give a meaningful name > > - function types don't > > - Function types were causing trouble when in a list/array due to > > generic erasure > > - Removing them removes some of the most contentious syntax debates > > - Function types can be added later if the right approach is taken (as > > in the doc) of declaring lambda expressions to have no expressible > > type. > > > > I suggest using this thread to comment on the removal of function types :-) > > > > Stephen > > > > From jjb at google.com Fri Jul 16 09:27:26 2010 From: jjb at google.com (Joshua Bloch) Date: Fri, 16 Jul 2010 09:27:26 -0700 Subject: Primitives in Generics (+ the meaning of this) In-Reply-To: <201007161728.46392.peter.levart@marand.si> References: <4C3778EB.60405@cs.oswego.edu> <4C377AB5.7050205@oracle.com> <201007161728.46392.peter.levart@marand.si> Message-ID: Peter, On Fri, Jul 16, 2010 at 8:28 AM, Peter Levart wrote: > > > Regardless of that, do you aggree that the statistical evidence, prepared > by some members of this list, showed that usage of explicit and implicit > "this" inside anonymous inner SAM subclass creation expressions is > practically non-existent and is actualy "in the way" of using outer.this > and/or outer instance members (even a cause of bugs in JDK itself)? > I don't remember the details of the statistical evidence, but it seems suspect. It's not uncommon for periodic TimerTasks to cancel themselves or (as illustrated in the TimerTask documentation) to call their own scheduledExecutionTime()method from their single abstract method (run). It may be the case that it's much more common to want to access enclosing instances, but I don't believe that intentional access to the function object from within its body is "practically non-existent." Josh P.S. It took me ten seconds of searching to find this example: www.google.com/codesearch/p?hl=en#xJfZyRTTR4E/trunk/src/com/bwepow/wwj/awt/KeyPollTimer.java&q=TimerTask%20scheduledExecutionTime&sa=N&cd=27&ct=rc&l=118 From hlship at gmail.com Fri Jul 16 09:40:47 2010 From: hlship at gmail.com (Howard Lewis Ship) Date: Fri, 16 Jul 2010 09:40:47 -0700 Subject: Searchable archives? Message-ID: I'm trying to find searchable archives for the list ... I've checked around the project home page, markmail.org, etc. and haven't found anything searchable. Thanks! -- Howard M. Lewis Ship Creator of Apache Tapestry The source for Tapestry training, mentoring and support. Contact me to learn how I can get you up and productive in Tapestry fast! (971) 678-5210 http://howardlewisship.com From int19h at gmail.com Fri Jul 16 09:48:23 2010 From: int19h at gmail.com (Pavel Minaev) Date: Fri, 16 Jul 2010 09:48:23 -0700 Subject: Searchable archives? In-Reply-To: References: Message-ID: Google for "site:http://mail.openjdk.java.net/pipermail/lambda-dev/ " On Fri, Jul 16, 2010 at 9:40 AM, Howard Lewis Ship wrote: > I'm trying to find searchable archives for the list ... ?I've checked > around the project home page, markmail.org, etc. and haven't found > anything searchable. > > Thanks! > > -- > Howard M. Lewis Ship > > Creator of Apache Tapestry > > The source for Tapestry training, mentoring and support. Contact me to > learn how I can get you up and productive in Tapestry fast! > > (971) 678-5210 > http://howardlewisship.com > > From brian.goetz at oracle.com Fri Jul 16 10:07:15 2010 From: brian.goetz at oracle.com (Brian Goetz) Date: Fri, 16 Jul 2010 13:07:15 -0400 Subject: Primitives in Generics (+ the meaning of this) In-Reply-To: <201007161728.46392.peter.levart@marand.si> References: <4C3778EB.60405@cs.oswego.edu> <4C377AB5.7050205@oracle.com> <201007161728.46392.peter.levart@marand.si> Message-ID: <4C4091C3.8060800@oracle.com> > Regardless of that, do you aggree that the statistical evidence, prepared by some members of this list, showed that usage of explicit and implicit "this" inside anonymous inner SAM subclass creation expressions is practically non-existent and is actualy "in the way" of using outer.this and/or outer instance members (even a cause of bugs in JDK itself)? No, I don't agree with this. What we have is, basically, arguments over which use cases are more important and which error modes are worse, which are almost always just opinion contests, just like syntax discussions about which alternative "looks more like Java". Issues such as the one Josh raised (where TimerTask wants to call cancel()) come up with some frequency; so do accidental shadowing of methods like toString() in inner classes. This generally just comes down to a "which error would you rather have" question. And, given that inner classes *already* suffer from the accidental shadowing problem, creating a mechanism that goes the other way (even if the other way is absolutely right!) is even more confusing for Java developers, who now have to keep track of two separate and inconsistent bits of puzzlerhood. That said, the questions you raise here (and in your next e-mail) are fair and we've already planned to devote some thought to it -- but not now, because we've got other things that demand our attention more urgently, such as refining and implementing the features that are already in the must-have column. From neal at gafter.com Fri Jul 16 10:25:41 2010 From: neal at gafter.com (Neal Gafter) Date: Fri, 16 Jul 2010 10:25:41 -0700 Subject: Primitives in Generics (+ the meaning of this) In-Reply-To: <4C4091C3.8060800@oracle.com> References: <4C3778EB.60405@cs.oswego.edu> <4C377AB5.7050205@oracle.com> <201007161728.46392.peter.levart@marand.si> <4C4091C3.8060800@oracle.com> Message-ID: On Fri, Jul 16, 2010 at 10:07 AM, Brian Goetz wrote: > And, given that inner classes *already* suffer from the accidental > shadowing > problem, creating a mechanism that goes the other way (even if the other > way > is absolutely right!) is even more confusing for Java developers, who now > have > to keep track of two separate and inconsistent bits of puzzlerhood. > Brian- I don't believe it would be more confusing for Java developers (except those completely unfamiliar with the new language feature). But setting that aside, this leads to the natural question: If we're wanting to recreate all the problems of anonymous inner classes, what, if any, problems ARE addressed by project lambda, and what is newly possible that was not possible previously? -Neal From hlship at gmail.com Fri Jul 16 10:28:41 2010 From: hlship at gmail.com (Howard Lewis Ship) Date: Fri, 16 Jul 2010 10:28:41 -0700 Subject: Do inner classes cause a memory leak? Message-ID: I use a huge number of inner classes for closure-like operations in Tapestry. One thing I've noticed (often while using the debugger) is that non-static inner classes appear to keep a reference to the containing object, even when not absolutely necessary. Consider the following: package com.example; public class LeakExample { public Runnable create() { return new Runnable() { public void run() { System.out.println("Inner class."); } }; } } Usine JDK 1.6 on my Mac: $ jad -a -p target/test-classes/com/example/LeakExample*.class // Decompiled by Jad v1.5.8g. Copyright 2001 Pavel Kouznetsov. // Jad home page: http://www.kpdus.com/jad.html // Decompiler options: packimports(3) annotate // Source File Name: LeakExample.java package com.example; import java.io.PrintStream; public class LeakExample { public LeakExample() { // 0 0:aload_0 // 1 1:invokespecial #1 // 2 4:return } public Runnable create() { return new Runnable() { public void run() { System.out.println("Inner class."); // 0 0:getstatic #3 // 1 3:ldc1 #4 // 2 5:invokevirtual #5 // 3 8:return } final LeakExample this$0; { this$0 = LeakExample.this; // 0 0:aload_0 // 1 1:aload_1 // 2 2:putfield #1 super(); // 3 5:aload_0 // 4 6:invokespecial #2 // 5 9:return } } ; // 0 0:new #2 // 1 3:dup // 2 4:aload_0 // 3 5:invokespecial #3 // 4 8:areturn } } ~/work/t5-project/tapestry-ioc $ ~/work/t5-project/tapestry-ioc $ javac -version javac 1.6.0_20 So looking at this, or the raw bytecode, it seems a non-static inner class always includes a reference to the containing class, even when (as in this example) the inner class has no dependencies on (private) members of the enclosing class. Perhaps I'm misreading the dissembled bytecode, but I think not. It seems to me that basic escape analysis should make it possible to avoid this case, both for inner classes, and for the lambda syntax being proposed. In fact, as long as the inner class only refers to final instance variables (that could be passed to the constructed inner class via its constructor) and not to methods of the outer class, it should always be possible to avoid the leaked this reference. This is important to me; Tapestry often follows a pattern wherein large builder objects (full of mutable state) execute to produce final runtime objects. Often inner classes (poor man's closures) are part of this process. I want the builder objects to be GC'ed once the runtime objects are created. It is all too easy for an inner class to hold a reference to a builder object and prevent it from being GC'ed. Currently, I frequently resort to a static method to construct inner classes so as to ensure that this does not escape (and with current inner class syntax, that's often preferable from a code readability point of view). Further, when an inner class is used inline (with the leaked this), my code only does so to gain access to a final field (typically an injected dependency). -- Howard M. Lewis Ship Creator of Apache Tapestry The source for Tapestry training, mentoring and support. Contact me to learn how I can get you up and productive in Tapestry fast! (971) 678-5210 http://howardlewisship.com From forax at univ-mlv.fr Fri Jul 16 10:54:56 2010 From: forax at univ-mlv.fr (=?ISO-8859-1?Q?R=E9mi_Forax?=) Date: Fri, 16 Jul 2010 19:54:56 +0200 Subject: Primitives in Generics (+ the meaning of this) In-Reply-To: References: <4C3778EB.60405@cs.oswego.edu> <4C377AB5.7050205@oracle.com> <201007161728.46392.peter.levart@marand.si> <4C4091C3.8060800@oracle.com> Message-ID: <4C409CF0.3040009@univ-mlv.fr> Le 16/07/2010 19:25, Neal Gafter a ?crit : > On Fri, Jul 16, 2010 at 10:07 AM, Brian Goetzwrote: > > >> And, given that inner classes *already* suffer from the accidental >> shadowing >> problem, creating a mechanism that goes the other way (even if the other >> way >> is absolutely right!) is even more confusing for Java developers, who now >> have >> to keep track of two separate and inconsistent bits of puzzlerhood. >> >> > Brian- > > I don't believe it would be more confusing for Java developers (except those > completely unfamiliar with the new language feature). But setting that > aside, this leads to the natural question: If we're wanting to recreate all > the problems of anonymous inner classes, what, if any, problems ARE > addressed by project lambda, and what is newly possible that was not > possible previously? > > -Neal > Good question. and I haven't no good answer. Runnable r = { -> foo(this); } Callable c = { -> foo(this); } what is the signature of foo in those cases ? BTW, there is at least something that inner-class does and lambda doesn't. You can send argument to the constructor of an inner-class and you can't do that with lambda. R?mi From peter.levart at marand.si Fri Jul 16 11:17:00 2010 From: peter.levart at marand.si (Peter Levart) Date: Fri, 16 Jul 2010 20:17:00 +0200 Subject: Primitives in Generics (+ the meaning of this) In-Reply-To: References: <201007161728.46392.peter.levart@marand.si> Message-ID: <201007162017.00566.peter.levart@marand.si> On 07/16/10, Joshua Bloch wrote: > Peter, > > On Fri, Jul 16, 2010 at 8:28 AM, Peter Levart wrote: > > > > > > > Regardless of that, do you aggree that the statistical evidence, prepared > > by some members of this list, showed that usage of explicit and implicit > > "this" inside anonymous inner SAM subclass creation expressions is > > practically non-existent and is actualy "in the way" of using outer.this > > and/or outer instance members (even a cause of bugs in JDK itself)? > > > > I don't remember the details of the statistical evidence, but it seems > suspect. I aggree that the sample of sources should be much greater to have any statistical meaning. I remember that not much more than 4 or so projects were scanned (OpenJDK and IntelliJ IDEA among the bigger ones) and not all of them used consistent methodology. It was only an illustrative attempt but one which indicated that the usages of outer instance members and/or outer.this might be much more frequent than usages of inner this and/or inner instance members. So what is stopping us from doing a more complete analysis as the next step? > It's not uncommon for periodic TimerTasks to cancel themselves or > (as illustrated in the TimerTask documentation) to call their own > scheduledExecutionTime()method from their single abstract method (run). It > may be the case that it's much more common to want to access enclosing > instances, but I don't believe that intentional access to the function > object from within its body is "practically non-existent." There absolutely should be an acceptable, elegant and consistent way to intentionally access the members of the target SAM instance from within the lambda body. There were already long discussions about alternatives to "this" on this list. None of them seems to be liked by deciding factors so the latest specification uses "this" for that purpose. But do you aggree that if non-disputable statistics showed that usages of "this" referring to target SAM instance are greatly outnumbered by usages of outer.this, wouldn't it be right to consider a more awkward alternative for accessing target SAM instance and use "this" to refer to lexically surrounding instance (and do the same with unqualified members)? Does that depend on how awkward the alternative might be? Ok, here's another attempt: > > Josh > > P.S. It took me ten seconds of searching to find this example: > www.google.com/codesearch/p?hl=en#xJfZyRTTR4E/trunk/src/com/bwepow/wwj/awt/KeyPollTimer.java&q=TimerTask%20scheduledExecutionTime&sa=N&cd=27&ct=rc&l=118 > Taking the above example and pasting relevant lines here: public synchronized void start() { if (this.timerTask == null) { this.timerTask = new java.util.TimerTask() { public void run() { long time = System.currentTimeMillis(); if (time - this.scheduledExecutionTime() >= 2 * KeyPollTimer.this.period) return; KeyPollTimer.this.updateAndNotify(KeyPollTimer.this.listener); } }; if (this.timer == null) this.timer = new java.util.Timer(); this.timer.schedule(timerTask, 0, this.period); } } would be rewritten to use lambda expression like this: public synchronized void start() { if (this.timerTask == null) { this.timerTask = { -> long time = System.currentTimeMillis(); if (time - this.scheduledExecutionTime() >= 2 * KeyPollTimer.this.period) return; KeyPollTimer.this.updateAndNotify(KeyPollTimer.this.listener); }; if (this.timer == null) this.timer = new java.util.Timer(); this.timer.schedule(timerTask, 0, this.period); } } alternative way to refer to target SAM instance might be: public synchronized void start() { if (this.timerTask == null) { this.timerTask = {task: -> long time = System.currentTimeMillis(); if (time - task.scheduledExecutionTime() >= 2 * KeyPollTimer.this.period) return; KeyPollTimer.this.updateAndNotify(KeyPollTimer.this.listener); }; if (this.timer == null) this.timer = new java.util.Timer(); this.timer.schedule(timerTask, 0, this.period); } } The optional "self" parameter is the first parameter in the list and delimited with ':' from other function parameters. With target typing it doesn't take to much space. Is this too awkward and unintuitive? This alternative has one advantage to "this". If you want to refer to an "outer" lambda instance from within a nested lambda expression body, currently you could do that only by assigning "this" to a local final variable: final Executor executor = ...; Runnable r = {-> final Runnable outer = this; executor.execute({-> executor.execute(outer); }); }; With alternative approach this is a one-liner: Runnable r = {outer: -> executor.execute({-> executor.execute(outer); }); }; I know that such usages might be even less frequent than referrals to inner SAM instance from the lambda body, so this is not an issue - just illustration. But in the future, such optional syntax addition could also be used as the target name of the long return for example: final List strings = ...; Callable fistLambda = {outer: -> strings.forEach({s -> if (s.startsWith("lambda")) return outer : s; }); return null; }; Regards, Peter From reinier at zwitserloot.com Fri Jul 16 11:45:33 2010 From: reinier at zwitserloot.com (Reinier Zwitserloot) Date: Fri, 16 Jul 2010 20:45:33 +0200 Subject: Primitives in Generics (+ the meaning of this) In-Reply-To: <4C4091C3.8060800@oracle.com> References: <4C3778EB.60405@cs.oswego.edu> <4C377AB5.7050205@oracle.com> <201007161728.46392.peter.levart@marand.si> <4C4091C3.8060800@oracle.com> Message-ID: The rather obvious answer would seem to be that implicit this refers to the inner object, but explicit this defaults to the outer object. You can still get an explicit this referencing the inner object by using the SAM type as a qualifier. Thus: public class Example { public void cancel() { System.out.println("OUTER"); } public void runMe() { TimerTask t = { -> cancel(); //LINE 1 this.cancel(); //LINE 2 Example.this.cancel(); //LINE 3 TimerTask.this.cancel(); //LINE 4 }; } } In the above example, LINE 1 and LINE 4 cancel the timertask, whereas LINE 2 and LINE 3 just print "OUTER" to sysout. As has been said, everything is going to either be hard to understand or not the thing one would expect in certain situations, and this is of course no different - now there's a difference between unqualified invocation and this-qualified invocation for method calls, which doesn't normally happen in java land. In my purely personal opinion on what's going to do the right thing and be the least surprising the most often, this one nevertheless wins. NB: As per the usual lexical scoping rules, if this was say a Runnable and not a TimerTask, then LINE 1 would also print 'OUTER'. --Reinier Zwitserloot On Fri, Jul 16, 2010 at 7:07 PM, Brian Goetz wrote: > > Regardless of that, do you aggree that the statistical evidence, prepared > by some members of this list, showed that usage of explicit and implicit > "this" inside anonymous inner SAM subclass creation expressions is > practically non-existent and is actualy "in the way" of using outer.this > and/or outer instance members (even a cause of bugs in JDK itself)? > > No, I don't agree with this. What we have is, basically, arguments over > which > use cases are more important and which error modes are worse, which are > almost > always just opinion contests, just like syntax discussions about which > alternative "looks more like Java". Issues such as the one Josh raised > (where > TimerTask wants to call cancel()) come up with some frequency; so do > accidental shadowing of methods like toString() in inner classes. This > generally just comes down to a "which error would you rather have" > question. > And, given that inner classes *already* suffer from the accidental > shadowing > problem, creating a mechanism that goes the other way (even if the other > way > is absolutely right!) is even more confusing for Java developers, who now > have > to keep track of two separate and inconsistent bits of puzzlerhood. > > That said, the questions you raise here (and in your next e-mail) are fair > and > we've already planned to devote some thought to it -- but not now, because > we've got other things that demand our attention more urgently, such as > refining and implementing the features that are already in the must-have > column. > > > From forax at univ-mlv.fr Fri Jul 16 11:57:48 2010 From: forax at univ-mlv.fr (=?ISO-8859-1?Q?R=E9mi_Forax?=) Date: Fri, 16 Jul 2010 20:57:48 +0200 Subject: Primitives in Generics (+ the meaning of this) In-Reply-To: References: <4C3778EB.60405@cs.oswego.edu> <4C377AB5.7050205@oracle.com> <201007161728.46392.peter.levart@marand.si> <4C4091C3.8060800@oracle.com> Message-ID: <4C40ABAC.7030503@univ-mlv.fr> Le 16/07/2010 20:45, Reinier Zwitserloot a ?crit : > The rather obvious answer would seem to be that implicit this refers to the > inner object, but explicit this defaults to the outer object. You can still > get an explicit this referencing the inner object by using the SAM type as a > qualifier. > > Thus: > > public class Example { > public void cancel() { System.out.println("OUTER"); } > > public void runMe() { > TimerTask t = { -> > cancel(); //LINE 1 > this.cancel(); //LINE 2 > Example.this.cancel(); //LINE 3 > TimerTask.this.cancel(); //LINE 4 > }; > } > } > > In the above example, LINE 1 and LINE 4 cancel the timertask, whereas LINE 2 > and LINE 3 just print "OUTER" to sysout. > > As has been said, everything is going to either be hard to understand or not > the thing one would expect in certain situations, and this is of course no > different - now there's a difference between unqualified invocation and > this-qualified invocation for method calls, which doesn't normally happen in > java land. In my purely personal opinion on what's going to do the right > thing and be the least surprising the most often, this one nevertheless > wins. > > NB: As per the usual lexical scoping rules, if this was say a Runnable and > not a TimerTask, then LINE 1 would also print 'OUTER'. > > --Reinier Zwitserloot > And what about: class A { public static void main(String[] args) { A a = { -> A.this; }; } } R?mi > > > On Fri, Jul 16, 2010 at 7:07 PM, Brian Goetz wrote: > > >>> Regardless of that, do you aggree that the statistical evidence, prepared >>> >> by some members of this list, showed that usage of explicit and implicit >> "this" inside anonymous inner SAM subclass creation expressions is >> practically non-existent and is actualy "in the way" of using outer.this >> and/or outer instance members (even a cause of bugs in JDK itself)? >> >> No, I don't agree with this. What we have is, basically, arguments over >> which >> use cases are more important and which error modes are worse, which are >> almost >> always just opinion contests, just like syntax discussions about which >> alternative "looks more like Java". Issues such as the one Josh raised >> (where >> TimerTask wants to call cancel()) come up with some frequency; so do >> accidental shadowing of methods like toString() in inner classes. This >> generally just comes down to a "which error would you rather have" >> question. >> And, given that inner classes *already* suffer from the accidental >> shadowing >> problem, creating a mechanism that goes the other way (even if the other >> way >> is absolutely right!) is even more confusing for Java developers, who now >> have >> to keep track of two separate and inconsistent bits of puzzlerhood. >> >> That said, the questions you raise here (and in your next e-mail) are fair >> and >> we've already planned to devote some thought to it -- but not now, because >> we've got other things that demand our attention more urgently, such as >> refining and implementing the features that are already in the must-have >> column. >> >> >> >> > From reinier at zwitserloot.com Fri Jul 16 11:56:31 2010 From: reinier at zwitserloot.com (Reinier Zwitserloot) Date: Fri, 16 Jul 2010 20:56:31 +0200 Subject: Primitives in Generics (+ the meaning of this) In-Reply-To: <4C409CF0.3040009@univ-mlv.fr> References: <4C3778EB.60405@cs.oswego.edu> <4C377AB5.7050205@oracle.com> <201007161728.46392.peter.levart@marand.si> <4C4091C3.8060800@oracle.com> <4C409CF0.3040009@univ-mlv.fr> Message-ID: In both cases, the way brian puts it, the first call will resolve to the same method as this call: foo(new Runnable() {public void run(){}}); and the second: Callable c = null; foo(c); After all, the 'this' refers to the closures themselves, and the type of the closure is determined BEFORE the code in the closure is resolved. In both of these cases the (so far hypothetical) SotL closure type resolver would trivially resolve these by looking at the target types of these variable declarations, which are Runnable and Callable respectively. In an alternate proposal where 'this' refers to outer, then both would resolve to whatever this call to foo resolves to: WhateverTypeTheseClosuresAreIn x = null; foo(x); i.e. 'foo(this)' in a closure would be no different from the same thing outside of it. Or so I understand SotL and the discussed alternative, at any rate. --Reinier Zwitserloot On Fri, Jul 16, 2010 at 7:54 PM, R?mi Forax wrote: > Le 16/07/2010 19:25, Neal Gafter a ?crit : > > On Fri, Jul 16, 2010 at 10:07 AM, Brian Goetz >wrote: > > > > > >> And, given that inner classes *already* suffer from the accidental > >> shadowing > >> problem, creating a mechanism that goes the other way (even if the other > >> way > >> is absolutely right!) is even more confusing for Java developers, who > now > >> have > >> to keep track of two separate and inconsistent bits of puzzlerhood. > >> > >> > > Brian- > > > > I don't believe it would be more confusing for Java developers (except > those > > completely unfamiliar with the new language feature). But setting that > > aside, this leads to the natural question: If we're wanting to recreate > all > > the problems of anonymous inner classes, what, if any, problems ARE > > addressed by project lambda, and what is newly possible that was not > > possible previously? > > > > -Neal > > > > Good question. and I haven't no good answer. > > Runnable r = { -> foo(this); } > Callable c = { -> foo(this); } > > what is the signature of foo in those cases ? > > BTW, there is at least something that inner-class does and lambda doesn't. > You can send argument to the constructor of an inner-class > and you can't do that with lambda. > > R?mi > > From forax at univ-mlv.fr Fri Jul 16 12:02:15 2010 From: forax at univ-mlv.fr (=?UTF-8?B?UsOpbWkgRm9yYXg=?=) Date: Fri, 16 Jul 2010 21:02:15 +0200 Subject: Primitives in Generics (+ the meaning of this) In-Reply-To: References: <4C3778EB.60405@cs.oswego.edu> <4C377AB5.7050205@oracle.com> <201007161728.46392.peter.levart@marand.si> <4C4091C3.8060800@oracle.com> <4C409CF0.3040009@univ-mlv.fr> Message-ID: <4C40ACB7.9050101@univ-mlv.fr> And now explain me how you want to introduce function type ? R?mi Le 16/07/2010 20:56, Reinier Zwitserloot a ?crit : > In both cases, the way brian puts it, the first call will resolve to > the same method as this call: > > foo(new Runnable() {public void run(){}}); > > and the second: > > Callable c = null; > foo(c); > > > After all, the 'this' refers to the closures themselves, and the type > of the closure is determined BEFORE the code in the closure is > resolved. In both of these cases the (so far hypothetical) SotL > closure type resolver would trivially resolve these by looking at the > target types of these variable declarations, which are Runnable and > Callable respectively. > > In an alternate proposal where 'this' refers to outer, then both would > resolve to whatever this call to foo resolves to: > > WhateverTypeTheseClosuresAreIn x = null; > foo(x); > > i.e. 'foo(this)' in a closure would be no different from the same > thing outside of it. > > Or so I understand SotL and the discussed alternative, at any rate. > > --Reinier Zwitserloot > > > > On Fri, Jul 16, 2010 at 7:54 PM, R?mi Forax > wrote: > > Le 16/07/2010 19:25, Neal Gafter a ?crit : > > On Fri, Jul 16, 2010 at 10:07 AM, Brian > Goetz>wrote: > > > > > >> And, given that inner classes *already* suffer from the accidental > >> shadowing > >> problem, creating a mechanism that goes the other way (even if > the other > >> way > >> is absolutely right!) is even more confusing for Java > developers, who now > >> have > >> to keep track of two separate and inconsistent bits of puzzlerhood. > >> > >> > > Brian- > > > > I don't believe it would be more confusing for Java developers > (except those > > completely unfamiliar with the new language feature). But > setting that > > aside, this leads to the natural question: If we're wanting to > recreate all > > the problems of anonymous inner classes, what, if any, problems ARE > > addressed by project lambda, and what is newly possible that was not > > possible previously? > > > > -Neal > > > > Good question. and I haven't no good answer. > > Runnable r = { -> foo(this); } > Callable c = { -> foo(this); } > > what is the signature of foo in those cases ? > > BTW, there is at least something that inner-class does and lambda > doesn't. > You can send argument to the constructor of an inner-class > and you can't do that with lambda. > > R?mi > > From reinier at zwitserloot.com Fri Jul 16 11:59:26 2010 From: reinier at zwitserloot.com (Reinier Zwitserloot) Date: Fri, 16 Jul 2010 20:59:26 +0200 Subject: Primitives in Generics (+ the meaning of this) In-Reply-To: <4C40ABAC.7030503@univ-mlv.fr> References: <4C3778EB.60405@cs.oswego.edu> <4C377AB5.7050205@oracle.com> <201007161728.46392.peter.levart@marand.si> <4C4091C3.8060800@oracle.com> <4C40ABAC.7030503@univ-mlv.fr> Message-ID: Yes, the somewhat academic Aha! case. I think we can declare that as sufficiently rare that any arbitrary choice will work. Most likely that arbitrary choice can be: The meaning of "A.this" is itself resolved lexically, with the first matching scope winning the battle. Thus, In that example, A.this refers to the inner this. After all, the outer this can already be accessed via just 'this', so that works out nicely, and is (arguably!) the least surprising result. --Reinier Zwitserloot On Fri, Jul 16, 2010 at 8:57 PM, R?mi Forax wrote: > Le 16/07/2010 20:45, Reinier Zwitserloot a ?crit : > > The rather obvious answer would seem to be that implicit this refers to > the > > inner object, but explicit this defaults to the outer object. You can > still > > get an explicit this referencing the inner object by using the SAM type > as a > > qualifier. > > > > Thus: > > > > public class Example { > > public void cancel() { System.out.println("OUTER"); } > > > > public void runMe() { > > TimerTask t = { -> > > cancel(); //LINE 1 > > this.cancel(); //LINE 2 > > Example.this.cancel(); //LINE 3 > > TimerTask.this.cancel(); //LINE 4 > > }; > > } > > } > > > > In the above example, LINE 1 and LINE 4 cancel the timertask, whereas > LINE 2 > > and LINE 3 just print "OUTER" to sysout. > > > > As has been said, everything is going to either be hard to understand or > not > > the thing one would expect in certain situations, and this is of course > no > > different - now there's a difference between unqualified invocation and > > this-qualified invocation for method calls, which doesn't normally happen > in > > java land. In my purely personal opinion on what's going to do the right > > thing and be the least surprising the most often, this one nevertheless > > wins. > > > > NB: As per the usual lexical scoping rules, if this was say a Runnable > and > > not a TimerTask, then LINE 1 would also print 'OUTER'. > > > > --Reinier Zwitserloot > > > > And what about: > class A { > public static void main(String[] args) { > A a = { -> > A.this; > }; > } > } > > R?mi > > > > > > > On Fri, Jul 16, 2010 at 7:07 PM, Brian Goetz > wrote: > > > > > >>> Regardless of that, do you aggree that the statistical evidence, > prepared > >>> > >> by some members of this list, showed that usage of explicit and implicit > >> "this" inside anonymous inner SAM subclass creation expressions is > >> practically non-existent and is actualy "in the way" of using outer.this > >> and/or outer instance members (even a cause of bugs in JDK itself)? > >> > >> No, I don't agree with this. What we have is, basically, arguments over > >> which > >> use cases are more important and which error modes are worse, which are > >> almost > >> always just opinion contests, just like syntax discussions about which > >> alternative "looks more like Java". Issues such as the one Josh raised > >> (where > >> TimerTask wants to call cancel()) come up with some frequency; so do > >> accidental shadowing of methods like toString() in inner classes. This > >> generally just comes down to a "which error would you rather have" > >> question. > >> And, given that inner classes *already* suffer from the accidental > >> shadowing > >> problem, creating a mechanism that goes the other way (even if the other > >> way > >> is absolutely right!) is even more confusing for Java developers, who > now > >> have > >> to keep track of two separate and inconsistent bits of puzzlerhood. > >> > >> That said, the questions you raise here (and in your next e-mail) are > fair > >> and > >> we've already planned to devote some thought to it -- but not now, > because > >> we've got other things that demand our attention more urgently, such as > >> refining and implementing the features that are already in the must-have > >> column. > >> > >> > >> > >> > > > > > From reinier at zwitserloot.com Fri Jul 16 12:10:55 2010 From: reinier at zwitserloot.com (Reinier Zwitserloot) Date: Fri, 16 Jul 2010 21:10:55 +0200 Subject: Revisiting primitives in generics: This can *NOT* be delayed for later. Message-ID: I realized this rather troublesome problem of going with SotL and not considering primitives in generics: There's no way back from that. If ParallelArrays is released with the full contingent of 132 SAMs in Ops, then we'll not only be stuck with them forever in the traditional sense of not eliminating classes to preserve backwards compatibility, but there's also no practical way forward, either. Currently, PA's DoubleParallelArray has a binarySearch method with the following signature: binarySearch(double target, Ops.DoubleComparator c). DoubleComparator itself is a SAM type, with signature #int(double, double). Therefore, a call to binarySearch with lambda would look like: binarySearch(12345678.9, {double a, double b -> Math.abs(a)-Math.abs(b)}); But if in a later JDK there's a way to deprecate Ops.DoubleComparator in favour of either a function type #int(double, double), -OR- my preferred option of allowing java.util.Comparator, then there would be a second binarySearch method with signature: binarySearch(double target, Comparator c). Calling this one would look like: binarySearch(12345678.9, {double a, double b -> Math.abs(a)-Math.abs(b)}); which is indistinguishable from the Ops.DoubleComparator case. The compiler can therefore not compile it, complaining about ambiguity. Users would have to explicitly specify either Ops.DoubleComparator or Comparator, breaking backwards compatibility and in general uglying up the API. I don't see a nice solution for this dilemma; one could use 'binarySearch2' but that feels like a copout. One could attempt a scheme whereby overridden methods can use some extra keyword to dictate to the compiler which of a set is to be preferred in case there's ambiguity, but that's adding extra complexity, which may not be palatable at that point. --Reinier Zwitserloot From reinier at zwitserloot.com Fri Jul 16 12:18:03 2010 From: reinier at zwitserloot.com (Reinier Zwitserloot) Date: Fri, 16 Jul 2010 21:18:03 +0200 Subject: Do inner classes cause a memory leak? In-Reply-To: References: Message-ID: Correct; the way the JLS currently specifies how anonymous inner classes work means they can NOT be static, and that a reference to the outer instance is passed into the inner instance even if it isn't actually necessary. The gc in turn cannot use hotspot tech to realize a certain inner field is blocking a gc but isn't actually useful for anything, as one can always get this value back out using for example reflection. However, the draft plan for translating lambdas to bytecode uses an entirely different mechanism, and your problem will not occur if this draft plan doesn't change too drastically. The full plan can be read here: http://mail.openjdk.java.net/pipermail/lambda-dev/2010-May/001355.html Check chapter 5 named 'Type 1 lambdas - "stateless" functions'. --Reinier Zwitserloot On Fri, Jul 16, 2010 at 7:28 PM, Howard Lewis Ship wrote: > I use a huge number of inner classes for closure-like operations in > Tapestry. One thing I've noticed (often while using the debugger) is > that non-static inner classes appear to keep a reference to the > containing object, even when not absolutely necessary. > > Consider the following: > > package com.example; > > public class LeakExample > { > public Runnable create() > { > return new Runnable() > { > public void run() > { > System.out.println("Inner class."); > } > }; > } > } > > > Usine JDK 1.6 on my Mac: > > $ jad -a -p target/test-classes/com/example/LeakExample*.class > // Decompiled by Jad v1.5.8g. Copyright 2001 Pavel Kouznetsov. > // Jad home page: http://www.kpdus.com/jad.html > // Decompiler options: packimports(3) annotate > // Source File Name: LeakExample.java > > package com.example; > > import java.io.PrintStream; > > public class LeakExample > { > > public LeakExample() > { > // 0 0:aload_0 > // 1 1:invokespecial #1 > // 2 4:return > } > > public Runnable create() > { > return new Runnable() { > > public void run() > { > System.out.println("Inner class."); > // 0 0:getstatic #3 > // 1 3:ldc1 #4 > // 2 5:invokevirtual #5 PrintStream.println(String)> > // 3 8:return > } > > final LeakExample this$0; > > > { > this$0 = LeakExample.this; > // 0 0:aload_0 > // 1 1:aload_1 > // 2 2:putfield #1 > super(); > // 3 5:aload_0 > // 4 6:invokespecial #2 > // 5 9:return > } > } > ; > // 0 0:new #2 > // 1 3:dup > // 2 4:aload_0 > // 3 5:invokespecial #3 LeakExample$1(LeakExample)> > // 4 8:areturn > } > } > ~/work/t5-project/tapestry-ioc > $ > > > > > ~/work/t5-project/tapestry-ioc > $ javac -version > javac 1.6.0_20 > > > So looking at this, or the raw bytecode, it seems a non-static inner > class always includes a reference to the containing class, even when > (as in this example) the inner class has no dependencies on (private) > members of the enclosing class. Perhaps I'm misreading the dissembled > bytecode, but I think not. > > It seems to me that basic escape analysis should make it possible to > avoid this case, both for inner classes, and for the lambda syntax > being proposed. In fact, as long as the inner class only refers to > final instance variables (that could be passed to the constructed > inner class via its constructor) and not to methods of the outer > class, it should always be possible to avoid the leaked this > reference. > > This is important to me; Tapestry often follows a pattern wherein > large builder objects (full of mutable state) execute to produce final > runtime objects. Often inner classes (poor man's closures) are part of > this process. I want the builder objects to be GC'ed once the runtime > objects are created. It is all too easy for an inner class to hold a > reference to a builder object and prevent it from being GC'ed. > > Currently, I frequently resort to a static method to construct inner > classes so as to ensure that this does not escape (and with current > inner class syntax, that's often preferable from a code readability > point of view). Further, when an inner class is used inline (with the > leaked this), my code only does so to gain access to a final field > (typically an injected dependency). > > -- > Howard M. Lewis Ship > > Creator of Apache Tapestry > > The source for Tapestry training, mentoring and support. Contact me to > learn how I can get you up and productive in Tapestry fast! > > (971) 678-5210 > http://howardlewisship.com > > From lk at teamten.com Fri Jul 16 12:29:43 2010 From: lk at teamten.com (Lawrence Kesteloot) Date: Fri, 16 Jul 2010 12:29:43 -0700 Subject: Revisiting primitives in generics: This can *NOT* be delayed for later. In-Reply-To: References: Message-ID: Ops.DoubleComparator could be retrofitted to be an empty sub-interface of Comparator and the version of binarySearch that takes a DoubleComparator could be removed. The version that takes a Comparator would work with old binary classes, old source classes, and new source classes. Lawrence On Fri, Jul 16, 2010 at 12:10 PM, Reinier Zwitserloot wrote: > I realized this rather troublesome problem of going with SotL and not > considering primitives in generics: > > There's no way back from that. > > If ParallelArrays is released with the full contingent of 132 SAMs in Ops, > then we'll not only be stuck with them forever in the traditional sense of > not eliminating classes to preserve backwards compatibility, but there's > also no practical way forward, either. > > Currently, PA's DoubleParallelArray has a binarySearch method with the > following signature: > > binarySearch(double target, Ops.DoubleComparator c). > > DoubleComparator itself is a SAM type, with signature #int(double, double). > > Therefore, a call to binarySearch with lambda would look like: > > binarySearch(12345678.9, {double a, double b -> Math.abs(a)-Math.abs(b)}); > > > But if in a later JDK there's a way to deprecate Ops.DoubleComparator in > favour of either a function type #int(double, double), -OR- my preferred > option of allowing java.util.Comparator, then there would be a > second binarySearch method with signature: > > binarySearch(double target, Comparator c). > > Calling this one would look like: > > binarySearch(12345678.9, {double a, double b -> Math.abs(a)-Math.abs(b)}); > > which is indistinguishable from the Ops.DoubleComparator case. The compiler > can therefore not compile it, complaining about ambiguity. Users would have > to explicitly specify either Ops.DoubleComparator or Comparator, > breaking backwards compatibility and in general uglying up the API. > > I don't see a nice solution for this dilemma; one could use 'binarySearch2' > but that feels like a copout. One could attempt a scheme whereby overridden > methods can use some extra keyword to dictate to the compiler which of a set > is to be preferred in case there's ambiguity, but that's adding extra > complexity, which may not be palatable at that point. > > ?--Reinier Zwitserloot > > From mthornton at optrak.co.uk Fri Jul 16 13:23:18 2010 From: mthornton at optrak.co.uk (Mark Thornton) Date: Fri, 16 Jul 2010 21:23:18 +0100 Subject: Revisiting primitives in generics: This can *NOT* be delayed for later. In-Reply-To: References: Message-ID: <4C40BFB6.3050004@optrak.co.uk> Reinier Zwitserloot wrote: > I realized this rather troublesome problem of going with SotL and not > considering primitives in generics: > > There's no way back from that. > > If ParallelArrays is released with the full contingent of 132 SAMs in Ops, > then we'll not only be stuck with them forever in the traditional sense of > not eliminating classes to preserve backwards compatibility, but there's > also no practical way forward, either. > Without a solution to the primitive problem I would expect ParallelArray's to be left out of JDK7 or at least the primitive forms to be omitted. Mark From reinier at zwitserloot.com Fri Jul 16 16:36:53 2010 From: reinier at zwitserloot.com (Reinier Zwitserloot) Date: Sat, 17 Jul 2010 01:36:53 +0200 Subject: Revisiting primitives in generics: This can *NOT* be delayed for later. In-Reply-To: References: Message-ID: No, that doesn't actually work, because method calls are stored as (class, name, descriptor) tuples. Thus a call to the 'old' Ops based version would be encoded as: java.util.concurrent.pa.DoubleParallelArray binarySearch (DLj/u/c/p/Ops$DoubleComparator;)V and by removing this method in favour of one based on e.g. j/u/Comparator, that method descriptor would find nothing, and thus produce a MethodNotFoundError at runtime. This is incidentally also the explanation for why j.u.Vector is still used by name in various swing classes. Can't remove the method. --Reinier Zwitserloot On Fri, Jul 16, 2010 at 9:29 PM, Lawrence Kesteloot wrote: > Ops.DoubleComparator could be retrofitted to be an empty sub-interface > of Comparator and the version of binarySearch that takes a > DoubleComparator could be removed. The version that takes a Comparator > would work with old binary classes, old source classes, and new source > classes. > > Lawrence > > > On Fri, Jul 16, 2010 at 12:10 PM, Reinier Zwitserloot > wrote: > > I realized this rather troublesome problem of going with SotL and not > > considering primitives in generics: > > > > There's no way back from that. > > > > If ParallelArrays is released with the full contingent of 132 SAMs in > Ops, > > then we'll not only be stuck with them forever in the traditional sense > of > > not eliminating classes to preserve backwards compatibility, but there's > > also no practical way forward, either. > > > > Currently, PA's DoubleParallelArray has a binarySearch method with the > > following signature: > > > > binarySearch(double target, Ops.DoubleComparator c). > > > > DoubleComparator itself is a SAM type, with signature #int(double, > double). > > > > Therefore, a call to binarySearch with lambda would look like: > > > > binarySearch(12345678.9, {double a, double b -> > Math.abs(a)-Math.abs(b)}); > > > > > > But if in a later JDK there's a way to deprecate Ops.DoubleComparator in > > favour of either a function type #int(double, double), -OR- my preferred > > option of allowing java.util.Comparator, then there would be a > > second binarySearch method with signature: > > > > binarySearch(double target, Comparator c). > > > > Calling this one would look like: > > > > binarySearch(12345678.9, {double a, double b -> > Math.abs(a)-Math.abs(b)}); > > > > which is indistinguishable from the Ops.DoubleComparator case. The > compiler > > can therefore not compile it, complaining about ambiguity. Users would > have > > to explicitly specify either Ops.DoubleComparator or Comparator, > > breaking backwards compatibility and in general uglying up the API. > > > > I don't see a nice solution for this dilemma; one could use > 'binarySearch2' > > but that feels like a copout. One could attempt a scheme whereby > overridden > > methods can use some extra keyword to dictate to the compiler which of a > set > > is to be preferred in case there's ambiguity, but that's adding extra > > complexity, which may not be palatable at that point. > > > > --Reinier Zwitserloot > > > > > > From hlship at gmail.com Fri Jul 16 17:30:56 2010 From: hlship at gmail.com (Howard Lewis Ship) Date: Fri, 16 Jul 2010 17:30:56 -0700 Subject: Do inner classes cause a memory leak? In-Reply-To: <4C40F44A.3020703@oracle.com> References: <4C40F44A.3020703@oracle.com> Message-ID: On Fri, Jul 16, 2010 at 5:07 PM, David Holmes wrote: > Howard, > > By definition a non-static inner class (nearly) always have a reference to > an enclosing instance (modulo the bizarre edge-case of anonymous classes in > static blocks). > See, that sounds to me like a language developer telling an application developer how the world works. I'm often guilty on the other side (as a framework developer telling application developers what to do), and I can tell you from personal experience that the adapting the framework to their needs is often the right thing to do, even if it means more work for the framework developer, but it ultimately results in a better tool for everyone. >From your point of view, there's no reason to define the inner class inline, inside a instance method, if it doesn't need access to private members of the containing instance. As an application developer (and a framework author) there are times when I want to keep related concepts close together in the code. Thus, despite the bulky inner class syntax, I will often have a method create and use or return an inner class. I don't like having to think about this "memory leak" issue (I believe it's referred to elsewhere as "shadowing"). It sounds like this is a recognized problem to be addressed as stateless lambdas ... and that's fine. However, making the compiler smarter about inner classes (which are, after all, primarily syntactic sugar for defining top-level classes, with some extra invisible methods on the containing class to address visibility concerns) seems like a win as well ... and in my estimation, it is possible to make the compiler do the right thing without changing the semantics of the generated code. Again, via escape analysis the compiler should be able to see if the inner class really does need access to the surrounding instance and generate different code that does not pass the containing object's this into the inner class' constructor. > If there is truly no need to access an enclosing instance then don't use a > non-static inner class. Likewise, if I could say something like: new Thread(new static Runnable() { public void run() { ... }; }); ... or some other notation, or even an annotation on the inner class, that would be fine. However simply saying "well that's the way the compiler works and there's a workaround using a static method" is a bit dismissive. I think the whole spirit of lambda is to improve on what Java is and does today, both in terms of syntax and semantics, to keep it a vibrant and competitive language. To be honest, I wasn't tracking lambda very much until quite recently and I'm surprised at how nicely the latest take on the syntax reads. > > David Holmes > > Howard Lewis Ship said the following on 07/17/10 03:28: >> >> I use a huge number of inner classes for closure-like operations in >> Tapestry. ?One thing I've noticed (often while using the debugger) is >> that non-static inner classes appear to keep a reference to the >> containing object, even when not absolutely necessary. >> >> Consider the following: >> >> package com.example; >> >> public class LeakExample >> { >> ? ?public Runnable create() >> ? ?{ >> ? ? ? ?return new Runnable() >> ? ? ? ?{ >> ? ? ? ? ? ?public void run() >> ? ? ? ? ? ?{ >> ? ? ? ? ? ? ? ?System.out.println("Inner class."); >> ? ? ? ? ? ?} >> ? ? ? ?}; >> ? ?} >> } >> >> >> Usine JDK 1.6 on my Mac: >> >> $ jad -a -p ?target/test-classes/com/example/LeakExample*.class >> // Decompiled by Jad v1.5.8g. Copyright 2001 Pavel Kouznetsov. >> // Jad home page: http://www.kpdus.com/jad.html >> // Decompiler options: packimports(3) annotate >> // Source File Name: ? LeakExample.java >> >> package com.example; >> >> import java.io.PrintStream; >> >> public class LeakExample >> { >> >> ? ?public LeakExample() >> ? ?{ >> ? ?// ? ?0 ? ?0:aload_0 >> ? ?// ? ?1 ? ?1:invokespecial ? #1 ? >> ? ?// ? ?2 ? ?4:return >> ? ?} >> >> ? ?public Runnable create() >> ? ?{ >> ? ? ? ?return new Runnable() { >> >> ? ? ? ? ? ?public void run() >> ? ? ? ? ? ?{ >> ? ? ? ? ? ? ? ?System.out.println("Inner class."); >> ? ? ? ? ? ?// ? ?0 ? ?0:getstatic ? ? ? #3 ? > System.out> >> ? ? ? ? ? ?// ? ?1 ? ?3:ldc1 ? ? ? ? ? ?#4 ? >> ? ? ? ? ? ?// ? ?2 ? ?5:invokevirtual ? #5 ? > PrintStream.println(String)> >> ? ? ? ? ? ?// ? ?3 ? ?8:return >> ? ? ? ? ? ?} >> >> ? ? ? ? ? ?final LeakExample this$0; >> >> >> ? ? ? ? ? ?{ >> ? ? ? ? ? ? ? ?this$0 = LeakExample.this; >> ? ? ? ? ? ?// ? ?0 ? ?0:aload_0 >> ? ? ? ? ? ?// ? ?1 ? ?1:aload_1 >> ? ? ? ? ? ?// ? ?2 ? ?2:putfield ? ? ? ?#1 ? >> ? ? ? ? ? ? ? ?super(); >> ? ? ? ? ? ?// ? ?3 ? ?5:aload_0 >> ? ? ? ? ? ?// ? ?4 ? ?6:invokespecial ? #2 ? >> ? ? ? ? ? ?// ? ?5 ? ?9:return >> ? ? ? ? ? ?} >> ? ? ? ?} >> ; >> ? ?// ? ?0 ? ?0:new ? ? ? ? ? ? #2 ? >> ? ?// ? ?1 ? ?3:dup >> ? ?// ? ?2 ? ?4:aload_0 >> ? ?// ? ?3 ? ?5:invokespecial ? #3 ? > LeakExample$1(LeakExample)> >> ? ?// ? ?4 ? ?8:areturn >> ? ?} >> } >> ~/work/t5-project/tapestry-ioc >> $ >> >> >> >> >> ~/work/t5-project/tapestry-ioc >> $ javac -version >> javac 1.6.0_20 >> >> >> So looking at this, or the raw bytecode, it seems a non-static inner >> class always includes a reference to the containing class, even when >> (as in this example) the inner class has no dependencies on (private) >> members of the enclosing class. Perhaps I'm misreading the dissembled >> bytecode, but I think not. >> >> It seems to me that basic escape analysis should make it possible to >> avoid this case, both for inner classes, and for the lambda syntax >> being proposed. In fact, as long as the inner class only refers to >> final instance variables (that could be passed to the constructed >> inner class via its constructor) and not to methods of the outer >> class, it should always be possible to avoid the leaked this >> reference. >> >> This is important to me; Tapestry often follows a pattern wherein >> large builder objects (full of mutable state) execute to produce final >> runtime objects. Often inner classes (poor man's closures) are part of >> this process. I want the builder objects to be GC'ed once the runtime >> objects are created. It is all too easy for an inner class to hold a >> reference to a builder object and prevent it from being GC'ed. >> >> Currently, I frequently resort to a static method to construct inner >> classes so as to ensure that this does not escape (and with current >> inner class syntax, that's often preferable from a code readability >> point of view). ?Further, when an inner class is used inline (with the >> leaked this), my code only does so to gain access to a final field >> (typically an injected dependency). >> > -- Howard M. Lewis Ship Creator of Apache Tapestry The source for Tapestry training, mentoring and support. Contact me to learn how I can get you up and productive in Tapestry fast! (971) 678-5210 http://howardlewisship.com From neal at gafter.com Fri Jul 16 17:38:47 2010 From: neal at gafter.com (Neal Gafter) Date: Fri, 16 Jul 2010 17:38:47 -0700 Subject: Do inner classes cause a memory leak? In-Reply-To: References: Message-ID: On Fri, Jul 16, 2010 at 12:18 PM, Reinier Zwitserloot < reinier at zwitserloot.com> wrote: > Correct; the way the JLS currently specifies how anonymous inner classes > work means they can NOT be static, and that a reference to the outer > instance is passed into the inner instance even if it isn't actually > necessary. > So far so good. The problem arises because javac generates unnecessary code to store that parameter in a field of the anonymous class, and that's where the memory leak occurs. Nothing in the language specification nor in any compatibility issues requires the compiler to do that. It can and should be fixed in javac. From john.r.rose at oracle.com Fri Jul 16 18:39:40 2010 From: john.r.rose at oracle.com (John Rose) Date: Fri, 16 Jul 2010 18:39:40 -0700 Subject: Do inner classes cause a memory leak? In-Reply-To: References: Message-ID: On Jul 16, 2010, at 12:18 PM, Reinier Zwitserloot wrote: > the way the JLS currently specifies how anonymous inner classes > work means they can NOT be static, and that a reference to the outer > instance is passed into the inner instance even if it isn't actually > necessary. The JLS says that the reference to the enclosing instance is passed to the constructor. I don't think it implies anywhere that this value has to be *stored* in a *field* (i.e, "this$N"). If the field is not used, it should not be generated, though I suppose compilers don't always do this optimization. Any storage leaks due to needless up-level links is an issue of implementation quality (of javac), not an issue of architecture. That was the original intention. Closures (in any GC language) are vulnerable to the same sorts of quality issues. -- John From collin.fagan at gmail.com Fri Jul 16 20:02:02 2010 From: collin.fagan at gmail.com (Collin Fagan) Date: Fri, 16 Jul 2010 22:02:02 -0500 Subject: Primitives in Generics (+ the meaning of this) In-Reply-To: References: <4C3778EB.60405@cs.oswego.edu> <4C377AB5.7050205@oracle.com> <201007161728.46392.peter.levart@marand.si> <4C4091C3.8060800@oracle.com> <4C40ABAC.7030503@univ-mlv.fr> Message-ID: There were many discussions previously about what 'this' should refer to. Many times the lambdas/closures/whatever were based on interfaces or function types. If I'm understanding things correctly it is now a goal to allow abstract classes which have a SAM to be expressed as a lambda. If this is the case then I would think changing what 'this' refers to is a bad idea. A SAM interface expressed as lambda shouldn't (can't?) contain state on it's own. In these use cases making 'this' refer to the lambda doesn't make a lot of sense. If you want access to *any *state you need to go to the enclosing scope to get it. This direction brought people to MethodHandel as a way of implementing a lambda and the idea that lambdas are *not* objects. While it's possible to have a SAM abstract class that does not have state, I would not put money on it being the most common case. SAM abstract classes really *are *objects. I'm not sure there is a way around that one. If it is an object then we have to worry about identity, equality, new (implicit new?) and all the other things that go along with being an object. Unfortunately that also probably includes the meaning of 'this'. This brings me back to what I consider the fundamental divide among the participants on this list. Hopefully I've understood enough of the messages here to separate the argument into two groups. Group 1 prefers: function types lexical scoping exception transparency lexical break, return, continue MethodHandle and most importantly lambdas are *not* objects Group 2 prefers: interface SAM conversion abstract class SAM conversion local return, break, continue 'this' refers to lambda identity is preserved Method,Constructor and Field references. more like anonymous inner classes lambdas *are* objects There is a word in java for types that aren't objects, primitives. I believe group 1 wants *lambda primitives* and group 2 wants* *(or is willing to tolerate)* lambda objects*. I'm not going to list who belongs to what group but you know who you are. From the proposals so far I get the feeling that the current Oracle work is focused more in the *lambda object* direction. There could be some middle ground but obviously we haven't found it yet. Can we have both? Is there a syntax for *lambda objects* that will leave the door open later for* lambda primitives*? Collin On Fri, Jul 16, 2010 at 1:59 PM, Reinier Zwitserloot < reinier at zwitserloot.com> wrote: > Yes, the somewhat academic Aha! case. > > I think we can declare that as sufficiently rare that any arbitrary choice > will work. Most likely that arbitrary choice can be: The meaning of > "A.this" > is itself resolved lexically, with the first matching scope winning the > battle. Thus, In that example, A.this refers to the inner this. After all, > the outer this can already be accessed via just 'this', so that works out > nicely, and is (arguably!) the least surprising result. > > --Reinier Zwitserloot > > > > On Fri, Jul 16, 2010 at 8:57 PM, R?mi Forax wrote: > > > Le 16/07/2010 20:45, Reinier Zwitserloot a ?crit : > > > The rather obvious answer would seem to be that implicit this refers to > > the > > > inner object, but explicit this defaults to the outer object. You can > > still > > > get an explicit this referencing the inner object by using the SAM type > > as a > > > qualifier. > > > > > > Thus: > > > > > > public class Example { > > > public void cancel() { System.out.println("OUTER"); } > > > > > > public void runMe() { > > > TimerTask t = { -> > > > cancel(); //LINE 1 > > > this.cancel(); //LINE 2 > > > Example.this.cancel(); //LINE 3 > > > TimerTask.this.cancel(); //LINE 4 > > > }; > > > } > > > } > > > > > > In the above example, LINE 1 and LINE 4 cancel the timertask, whereas > > LINE 2 > > > and LINE 3 just print "OUTER" to sysout. > > > > > > As has been said, everything is going to either be hard to understand > or > > not > > > the thing one would expect in certain situations, and this is of course > > no > > > different - now there's a difference between unqualified invocation and > > > this-qualified invocation for method calls, which doesn't normally > happen > > in > > > java land. In my purely personal opinion on what's going to do the > right > > > thing and be the least surprising the most often, this one nevertheless > > > wins. > > > > > > NB: As per the usual lexical scoping rules, if this was say a Runnable > > and > > > not a TimerTask, then LINE 1 would also print 'OUTER'. > > > > > > --Reinier Zwitserloot > > > > > > > And what about: > > class A { > > public static void main(String[] args) { > > A a = { -> > > A.this; > > }; > > } > > } > > > > R?mi > > > > > > > > > > > On Fri, Jul 16, 2010 at 7:07 PM, Brian Goetz > > wrote: > > > > > > > > >>> Regardless of that, do you aggree that the statistical evidence, > > prepared > > >>> > > >> by some members of this list, showed that usage of explicit and > implicit > > >> "this" inside anonymous inner SAM subclass creation expressions is > > >> practically non-existent and is actualy "in the way" of using > outer.this > > >> and/or outer instance members (even a cause of bugs in JDK itself)? > > >> > > >> No, I don't agree with this. What we have is, basically, arguments > over > > >> which > > >> use cases are more important and which error modes are worse, which > are > > >> almost > > >> always just opinion contests, just like syntax discussions about which > > >> alternative "looks more like Java". Issues such as the one Josh > raised > > >> (where > > >> TimerTask wants to call cancel()) come up with some frequency; so do > > >> accidental shadowing of methods like toString() in inner classes. > This > > >> generally just comes down to a "which error would you rather have" > > >> question. > > >> And, given that inner classes *already* suffer from the accidental > > >> shadowing > > >> problem, creating a mechanism that goes the other way (even if the > other > > >> way > > >> is absolutely right!) is even more confusing for Java developers, who > > now > > >> have > > >> to keep track of two separate and inconsistent bits of puzzlerhood. > > >> > > >> That said, the questions you raise here (and in your next e-mail) are > > fair > > >> and > > >> we've already planned to devote some thought to it -- but not now, > > because > > >> we've got other things that demand our attention more urgently, such > as > > >> refining and implementing the features that are already in the > must-have > > >> column. > > >> > > >> > > >> > > >> > > > > > > > > > > > From brian.goetz at oracle.com Fri Jul 16 20:13:47 2010 From: brian.goetz at oracle.com (Brian Goetz) Date: Fri, 16 Jul 2010 23:13:47 -0400 Subject: Primitives in Generics (+ the meaning of this) In-Reply-To: References: <4C3778EB.60405@cs.oswego.edu> <4C377AB5.7050205@oracle.com> <201007161728.46392.peter.levart@marand.si> <4C4091C3.8060800@oracle.com> <4C40ABAC.7030503@univ-mlv.fr> Message-ID: <4C411FEB.7080105@oracle.com> > There were many discussions previously about what 'this' should refer to. > Many times the lambdas/closures/whatever were based on interfaces or > function types. If I'm understanding things correctly it is now a goal to > allow abstract classes which have a SAM to be expressed as a lambda. This is correct. In my survey through several code bases, many useful SAM types are abstract classes. My experience bears this out; it is common for SAM types to start life as interfaces but then evolve to abstract classes with a few infrequently used methods which have default (usually empty) bodies. It would be nice to say "only interfaces" but my code survey tells me this is not likely to fly. It remains to be seen whether the restriction of "only no-arg constructors for abstract classes" will hold as the next defensible boundary. From alex.blewitt at gmail.com Sat Jul 17 01:22:23 2010 From: alex.blewitt at gmail.com (Alex Blewitt) Date: Sat, 17 Jul 2010 09:22:23 +0100 Subject: Primitives in Generics (+ the meaning of this) In-Reply-To: References: <4C3778EB.60405@cs.oswego.edu> <4C377AB5.7050205@oracle.com> <201007161728.46392.peter.levart@marand.si> <4C4091C3.8060800@oracle.com> <4C40ABAC.7030503@univ-mlv.fr> Message-ID: On Sat, Jul 17, 2010 at 4:02 AM, Collin Fagan wrote: > Group 1 prefers: > > function types > lexical scoping > exception transparency > lexical break, return, continue > MethodHandle > and most importantly lambdas are *not* objects > > Group 2 prefers: > > interface SAM conversion > abstract class SAM conversion > local return, break, continue > 'this' refers to lambda > identity is preserved > Method,Constructor and Field references. > more like anonymous inner classes > lambdas *are* objects I believe this is a good summary. What I don't think we should do is have a mix-and-match between the two; at the moment, we seem to have 'return' based on Group 1 and everything else on Group 2, with a new (overloaded) keyword 'yield' to mean local return. As Stephen pointed out in a different thread, it would make sense for return/break/continue to mean their current semantics and keywords, and base the decision of whether something is in Group 1 or Group 2 by the enclosing construct. Alex From alessiostalla at gmail.com Sat Jul 17 01:31:28 2010 From: alessiostalla at gmail.com (Alessio Stalla) Date: Sat, 17 Jul 2010 10:31:28 +0200 Subject: Primitives in Generics (+ the meaning of this) In-Reply-To: References: <4C3778EB.60405@cs.oswego.edu> <4C377AB5.7050205@oracle.com> <201007161728.46392.peter.levart@marand.si> <4C4091C3.8060800@oracle.com> <4C40ABAC.7030503@univ-mlv.fr> Message-ID: On Sat, Jul 17, 2010 at 5:02 AM, Collin Fagan wrote: > There were many discussions previously about what 'this' should refer to. > Many times the lambdas/closures/whatever were based on interfaces or > function types. If I'm understanding things correctly it is now a goal to > allow abstract classes which have a SAM to be expressed as a lambda. If this > is the case then I would think changing what 'this' refers to is a bad idea. > A SAM interface expressed as lambda shouldn't (can't?) contain state on it's > own. In these use cases making 'this' refer to the lambda doesn't make a lot > of sense. If you want access to *any *state you need to go to the enclosing > scope to get it. This direction brought people to MethodHandel as a way of > implementing a lambda and the idea that lambdas are *not* objects. While > it's possible to have a SAM abstract class that does not have state, I would > not put money on it being the most common case. SAM abstract classes really > *are *objects. I'm not sure there is a way around that one. If it is an > object then we have to worry about identity, equality, new (implicit new?) > and all the other things that go along with being an object. Unfortunately > that also probably includes the meaning of 'this'. > > This brings me back to what I consider the fundamental divide among the > participants on this list. Hopefully I've understood enough of the messages > here to separate the argument into two groups. > > Group 1 prefers: > > function types > lexical scoping > exception transparency > lexical break, return, continue > MethodHandle > and most importantly lambdas are *not* objects > > Group 2 prefers: > > interface SAM conversion > abstract class SAM conversion > local return, break, continue > 'this' refers to lambda > identity is preserved > Method,Constructor and Field references. > more like anonymous inner classes > lambdas *are* objects > > There is a word in java for types that aren't objects, primitives. I believe > group 1 wants *lambda primitives* and group 2 wants* *(or is willing to > tolerate)* lambda objects*. I'm not going to list who belongs to what group > but you know who you are. From the proposals so far I get the feeling that > the current Oracle work is focused more in the *lambda object* direction. > There could be some middle ground but obviously we haven't found it yet. Can > we have both? Is there a syntax for *lambda objects* that will leave the > door open later for* lambda primitives*? I wouldn't make this kind of distinction. Java already has too many primitive types :) Function types can still be objects (as in extends java.lang.Object). And function types do not imply transparency. In fact my ideal middle ground would be: function types lexical closures (full capture of local variables, lifting the 'final' restriction) interface SAM conversion abstract class SAM conversion local return (local break and continue make no sense, better to say no long break/continue) i.e. functions exist as first-class objects of a given type, but some magic makes them convertible to SAM types. In this scenario it makes no sense for this to refer to the function (except maybe for self-recursion) because SAM conversion only happens some time after the function is created. The compiler can optimize the conversion away and directly create the SAM type when it can, but this is just a possible optimization. Alessio From forax at univ-mlv.fr Sat Jul 17 03:40:45 2010 From: forax at univ-mlv.fr (=?ISO-8859-1?Q?R=E9mi_Forax?=) Date: Sat, 17 Jul 2010 12:40:45 +0200 Subject: Primitives in Generics (+ the meaning of this) In-Reply-To: References: <4C3778EB.60405@cs.oswego.edu> <4C377AB5.7050205@oracle.com> <201007161728.46392.peter.levart@marand.si> <4C4091C3.8060800@oracle.com> <4C40ABAC.7030503@univ-mlv.fr> Message-ID: <4C4188AD.6060308@univ-mlv.fr> Le 17/07/2010 10:31, Alessio Stalla a ?crit : > On Sat, Jul 17, 2010 at 5:02 AM, Collin Fagan wrote: > >> There were many discussions previously about what 'this' should refer to. >> Many times the lambdas/closures/whatever were based on interfaces or >> function types. If I'm understanding things correctly it is now a goal to >> allow abstract classes which have a SAM to be expressed as a lambda. If this >> is the case then I would think changing what 'this' refers to is a bad idea. >> A SAM interface expressed as lambda shouldn't (can't?) contain state on it's >> own. In these use cases making 'this' refer to the lambda doesn't make a lot >> of sense. If you want access to *any *state you need to go to the enclosing >> scope to get it. This direction brought people to MethodHandel as a way of >> implementing a lambda and the idea that lambdas are *not* objects. While >> it's possible to have a SAM abstract class that does not have state, I would >> not put money on it being the most common case. SAM abstract classes really >> *are *objects. I'm not sure there is a way around that one. If it is an >> object then we have to worry about identity, equality, new (implicit new?) >> and all the other things that go along with being an object. Unfortunately >> that also probably includes the meaning of 'this'. >> >> This brings me back to what I consider the fundamental divide among the >> participants on this list. Hopefully I've understood enough of the messages >> here to separate the argument into two groups. >> >> Group 1 prefers: >> >> function types >> lexical scoping >> exception transparency >> lexical break, return, continue >> MethodHandle >> and most importantly lambdas are *not* objects >> >> Group 2 prefers: >> >> interface SAM conversion >> abstract class SAM conversion >> local return, break, continue >> 'this' refers to lambda >> identity is preserved >> Method,Constructor and Field references. >> more like anonymous inner classes >> lambdas *are* objects >> >> There is a word in java for types that aren't objects, primitives. I believe >> group 1 wants *lambda primitives* and group 2 wants* *(or is willing to >> tolerate)* lambda objects*. I'm not going to list who belongs to what group >> but you know who you are. From the proposals so far I get the feeling that >> the current Oracle work is focused more in the *lambda object* direction. >> There could be some middle ground but obviously we haven't found it yet. Can >> we have both? Is there a syntax for *lambda objects* that will leave the >> door open later for* lambda primitives*? >> > I wouldn't make this kind of distinction. Java already has too many > primitive types :) > Function types can still be objects (as in extends java.lang.Object). > And function types do not imply transparency. In fact my ideal middle > ground would be: > > function types > lexical closures (full capture of local variables, lifting the 'final' > restriction) > interface SAM conversion > abstract class SAM conversion > local return (local break and continue make no sense, better to say no > long break/continue) > > i.e. functions exist as first-class objects of a given type, but some > magic makes them convertible to SAM types. In this scenario it makes > no sense for this to refer to the function (except maybe for > self-recursion) because SAM conversion only happens some time after > the function is created. The compiler can optimize the conversion away > and directly create the SAM type when it can, but this is just a > possible optimization. > > Alessio > > +1 R?mi From howard.lovatt at gmail.com Sat Jul 17 06:46:46 2010 From: howard.lovatt at gmail.com (Howard Lovatt) Date: Sat, 17 Jul 2010 21:46:46 +0800 Subject: Return Isn't Transparent Message-ID: A return statement isn't transparent. This is well articulated in Principles of Programming Languages R. D. Tennent (I don't have a copy handy - but I think page 56 is one such place). Consider: int m() { return 42; // To be wrapped in a lambda } Then imagine that you could create a lambda that encapsulated the long return and you called it: int m() { { -> return 42; }.(); // Oops error - must return value from m } Therefore arguments about transparency are moot in any language that has a return statement. The solution to this problem Tennent proposed was the normal Pascal solution of a dummy local for return values that has a default value, in a Javanese like language with no return following Tennent the original would be: int m() { m = 42; } The encapsulated form would be: int m() { { -> m = 42; }.(); // No error since m has a default value, 0 } ? -- Howard. From abies at adres.pl Sat Jul 17 08:35:06 2010 From: abies at adres.pl (Artur Biesiadowski) Date: Sat, 17 Jul 2010 17:35:06 +0200 Subject: Return Isn't Transparent In-Reply-To: References: Message-ID: <4C41CDAA.8020304@adres.pl> On 17/07/2010 15:46, Howard Lovatt wrote: > Then imagine that you could create a lambda that encapsulated the long > return and you called it: > > int m() { > { -> return 42; }.(); > // Oops error - must return value from m > } Yes, java is not perfect language. Forgetting about lambda int m() { if ( true ) { return 42; } // Oops error - must return value from m } Still, we have to live with it and accept the fact that sometimes compiler is bit too strict about requiring returns. Some people throw exceptions/errors in such places... I do not see your example as a particular point pro or against 'transparent' returns. Regards, Artur Biesiadowski From john at milsson.nu Sat Jul 17 10:00:16 2010 From: john at milsson.nu (John Nilsson) Date: Sat, 17 Jul 2010 19:00:16 +0200 Subject: Revisiting primitives in generics: This can *NOT* be delayed for later. In-Reply-To: References: Message-ID: I'm note sure I understand why this is a problem. But if it is, why not fix it? So I understand it correct that there is no runtime lookup for a compatible method (supertype of DoubleComparator)? If so, why not add this to the runtime? BR, John On Sat, Jul 17, 2010 at 1:36 AM, Reinier Zwitserloot wrote: > No, that doesn't actually work, because method calls are stored as (class, > name, descriptor) tuples. Thus a call to the 'old' Ops based version would > be encoded as: > > java.util.concurrent.pa.DoubleParallelArray binarySearch > (DLj/u/c/p/Ops$DoubleComparator;)V > > and by removing this method in favour of one based on e.g. j/u/Comparator, > that method descriptor would find nothing, and thus produce a > MethodNotFoundError at runtime. > > This is incidentally also the explanation for why j.u.Vector is still used > by name in various swing classes. Can't remove the method. > > ?--Reinier Zwitserloot > > > > On Fri, Jul 16, 2010 at 9:29 PM, Lawrence Kesteloot wrote: > >> Ops.DoubleComparator could be retrofitted to be an empty sub-interface >> of Comparator and the version of binarySearch that takes a >> DoubleComparator could be removed. The version that takes a Comparator >> would work with old binary classes, old source classes, and new source >> classes. >> >> Lawrence >> >> >> On Fri, Jul 16, 2010 at 12:10 PM, Reinier Zwitserloot >> wrote: >> > I realized this rather troublesome problem of going with SotL and not >> > considering primitives in generics: >> > >> > There's no way back from that. >> > >> > If ParallelArrays is released with the full contingent of 132 SAMs in >> Ops, >> > then we'll not only be stuck with them forever in the traditional sense >> of >> > not eliminating classes to preserve backwards compatibility, but there's >> > also no practical way forward, either. >> > >> > Currently, PA's DoubleParallelArray has a binarySearch method with the >> > following signature: >> > >> > binarySearch(double target, Ops.DoubleComparator c). >> > >> > DoubleComparator itself is a SAM type, with signature #int(double, >> double). >> > >> > Therefore, a call to binarySearch with lambda would look like: >> > >> > binarySearch(12345678.9, {double a, double b -> >> Math.abs(a)-Math.abs(b)}); >> > >> > >> > But if in a later JDK there's a way to deprecate Ops.DoubleComparator in >> > favour of either a function type #int(double, double), -OR- my preferred >> > option of allowing java.util.Comparator, then there would be a >> > second binarySearch method with signature: >> > >> > binarySearch(double target, Comparator c). >> > >> > Calling this one would look like: >> > >> > binarySearch(12345678.9, {double a, double b -> >> Math.abs(a)-Math.abs(b)}); >> > >> > which is indistinguishable from the Ops.DoubleComparator case. The >> compiler >> > can therefore not compile it, complaining about ambiguity. Users would >> have >> > to explicitly specify either Ops.DoubleComparator or Comparator, >> > breaking backwards compatibility and in general uglying up the API. >> > >> > I don't see a nice solution for this dilemma; one could use >> 'binarySearch2' >> > but that feels like a copout. One could attempt a scheme whereby >> overridden >> > methods can use some extra keyword to dictate to the compiler which of a >> set >> > is to be preferred in case there's ambiguity, but that's adding extra >> > complexity, which may not be palatable at that point. >> > >> > ?--Reinier Zwitserloot >> > >> > >> >> > > From neal at gafter.com Sat Jul 17 10:15:07 2010 From: neal at gafter.com (Neal Gafter) Date: Sat, 17 Jul 2010 10:15:07 -0700 Subject: Return Isn't Transparent In-Reply-To: References: Message-ID: On Sat, Jul 17, 2010 at 6:46 AM, Howard Lovatt wrote: > A return statement isn't transparent. This is well articulated in > Principles of Programming Languages R. D. Tennent (I don't have a copy > handy - but I think page 56 is one such place). Consider: > > int m() { > return 42; // To be wrapped in a lambda > } > > Then imagine that you could create a lambda that encapsulated the long > return and you called it: > > int m() { > { -> return 42; }.(); > // Oops error - must return value from m > } > This works just fine in BGGA. The language can indeed be designed so that return is transparent. From howard.lovatt at gmail.com Sat Jul 17 18:38:36 2010 From: howard.lovatt at gmail.com (Howard Lovatt) Date: Sun, 18 Jul 2010 09:38:36 +0800 Subject: Return Isn't Transparent In-Reply-To: References: Message-ID: Hi Neal, Are you saying that there are *no* cases where BGGA cannot reason about the control flow? This is a slightly different question than saying that a new language could not be designed with transparency in mind. ? -- Howard. On 18 July 2010 01:15, Neal Gafter wrote: > On Sat, Jul 17, 2010 at 6:46 AM, Howard Lovatt > wrote: >> >> A return statement isn't transparent. This is well articulated in >> Principles of Programming Languages R. D. Tennent (I don't have a copy >> handy - but I think page 56 is one such place). Consider: >> >> int m() { >> ?return 42; // To be wrapped in a lambda >> } >> >> Then imagine that you could create a lambda that encapsulated the long >> return and you called it: >> >> int m() { >> ?{ -> return 42; }.(); >> ?// Oops error - must return value from m >> } > > This works just fine in BGGA.? The language can indeed be designed so that > return is transparent. From reinier at zwitserloot.com Sat Jul 17 19:11:44 2010 From: reinier at zwitserloot.com (Reinier Zwitserloot) Date: Sun, 18 Jul 2010 04:11:44 +0200 Subject: Revisiting primitives in generics: This can *NOT* be delayed for later. In-Reply-To: References: Message-ID: Changing method resolution is unlikely to be backwards compatible. It would also have rather severe performance impacts, I bet. At any rate, more research would be required; a lot more, for such a change. --Reinier Zwitserloot On Sat, Jul 17, 2010 at 7:00 PM, John Nilsson wrote: > I'm note sure I understand why this is a problem. But if it is, why not fix > it? > > So I understand it correct that there is no runtime lookup for a > compatible method (supertype of DoubleComparator)? If so, why not add > this to the runtime? > > BR, > John > > On Sat, Jul 17, 2010 at 1:36 AM, Reinier Zwitserloot > wrote: > > No, that doesn't actually work, because method calls are stored as > (class, > > name, descriptor) tuples. Thus a call to the 'old' Ops based version > would > > be encoded as: > > > > java.util.concurrent.pa.DoubleParallelArray binarySearch > > (DLj/u/c/p/Ops$DoubleComparator;)V > > > > and by removing this method in favour of one based on e.g. > j/u/Comparator, > > that method descriptor would find nothing, and thus produce a > > MethodNotFoundError at runtime. > > > > This is incidentally also the explanation for why j.u.Vector is still > used > > by name in various swing classes. Can't remove the method. > > > > --Reinier Zwitserloot > > > > > > > > On Fri, Jul 16, 2010 at 9:29 PM, Lawrence Kesteloot > wrote: > > > >> Ops.DoubleComparator could be retrofitted to be an empty sub-interface > >> of Comparator and the version of binarySearch that takes a > >> DoubleComparator could be removed. The version that takes a Comparator > >> would work with old binary classes, old source classes, and new source > >> classes. > >> > >> Lawrence > >> > >> > >> On Fri, Jul 16, 2010 at 12:10 PM, Reinier Zwitserloot > >> wrote: > >> > I realized this rather troublesome problem of going with SotL and not > >> > considering primitives in generics: > >> > > >> > There's no way back from that. > >> > > >> > If ParallelArrays is released with the full contingent of 132 SAMs in > >> Ops, > >> > then we'll not only be stuck with them forever in the traditional > sense > >> of > >> > not eliminating classes to preserve backwards compatibility, but > there's > >> > also no practical way forward, either. > >> > > >> > Currently, PA's DoubleParallelArray has a binarySearch method with the > >> > following signature: > >> > > >> > binarySearch(double target, Ops.DoubleComparator c). > >> > > >> > DoubleComparator itself is a SAM type, with signature #int(double, > >> double). > >> > > >> > Therefore, a call to binarySearch with lambda would look like: > >> > > >> > binarySearch(12345678.9, {double a, double b -> > >> Math.abs(a)-Math.abs(b)}); > >> > > >> > > >> > But if in a later JDK there's a way to deprecate Ops.DoubleComparator > in > >> > favour of either a function type #int(double, double), -OR- my > preferred > >> > option of allowing java.util.Comparator, then there would be a > >> > second binarySearch method with signature: > >> > > >> > binarySearch(double target, Comparator c). > >> > > >> > Calling this one would look like: > >> > > >> > binarySearch(12345678.9, {double a, double b -> > >> Math.abs(a)-Math.abs(b)}); > >> > > >> > which is indistinguishable from the Ops.DoubleComparator case. The > >> compiler > >> > can therefore not compile it, complaining about ambiguity. Users would > >> have > >> > to explicitly specify either Ops.DoubleComparator or > Comparator, > >> > breaking backwards compatibility and in general uglying up the API. > >> > > >> > I don't see a nice solution for this dilemma; one could use > >> 'binarySearch2' > >> > but that feels like a copout. One could attempt a scheme whereby > >> overridden > >> > methods can use some extra keyword to dictate to the compiler which of > a > >> set > >> > is to be preferred in case there's ambiguity, but that's adding extra > >> > complexity, which may not be palatable at that point. > >> > > >> > --Reinier Zwitserloot > >> > > >> > > >> > >> > > > > > From neal at gafter.com Sun Jul 18 00:27:44 2010 From: neal at gafter.com (Neal Gafter) Date: Sun, 18 Jul 2010 00:27:44 -0700 Subject: Return Isn't Transparent In-Reply-To: References: Message-ID: On Sat, Jul 17, 2010 at 6:38 PM, Howard Lovatt wrote: > Are you saying that there are *no* cases where BGGA cannot reason > about the control flow? This is a slightly different question than > saying that a new language could not be designed with transparency in > mind. > Control-flow and reachability analysis is completely transparent in BGGA. From john at milsson.nu Sun Jul 18 10:48:11 2010 From: john at milsson.nu (John Nilsson) Date: Sun, 18 Jul 2010 19:48:11 +0200 Subject: Revisiting primitives in generics: This can *NOT* be delayed for later. In-Reply-To: References: Message-ID: Maybe you are right. Intuitively, though, I'm thinking since a simple re-compile of the source would make it work, what could possibly break by doing the "re-compile" at runtime (or when loading the class). BR, John On Sun, Jul 18, 2010 at 4:11 AM, Reinier Zwitserloot wrote: > Changing method resolution is unlikely to be backwards compatible. It would > also have rather severe performance impacts, I bet. At any rate, more > research would be required; a lot more, for such a change. > ?--Reinier Zwitserloot > > > > On Sat, Jul 17, 2010 at 7:00 PM, John Nilsson wrote: >> >> I'm note sure I understand why this is a problem. But if it is, why not >> fix it? >> >> So I understand it correct that there is no runtime lookup for a >> compatible method (supertype of DoubleComparator)? If so, why not add >> this to the runtime? >> >> BR, >> John >> >> On Sat, Jul 17, 2010 at 1:36 AM, Reinier Zwitserloot >> wrote: >> > No, that doesn't actually work, because method calls are stored as >> > (class, >> > name, descriptor) tuples. Thus a call to the 'old' Ops based version >> > would >> > be encoded as: >> > >> > java.util.concurrent.pa.DoubleParallelArray binarySearch >> > (DLj/u/c/p/Ops$DoubleComparator;)V >> > >> > and by removing this method in favour of one based on e.g. >> > j/u/Comparator, >> > that method descriptor would find nothing, and thus produce a >> > MethodNotFoundError at runtime. >> > >> > This is incidentally also the explanation for why j.u.Vector is still >> > used >> > by name in various swing classes. Can't remove the method. >> > >> > ?--Reinier Zwitserloot >> > >> > >> > >> > On Fri, Jul 16, 2010 at 9:29 PM, Lawrence Kesteloot >> > wrote: >> > >> >> Ops.DoubleComparator could be retrofitted to be an empty sub-interface >> >> of Comparator and the version of binarySearch that takes a >> >> DoubleComparator could be removed. The version that takes a Comparator >> >> would work with old binary classes, old source classes, and new source >> >> classes. >> >> >> >> Lawrence >> >> >> >> >> >> On Fri, Jul 16, 2010 at 12:10 PM, Reinier Zwitserloot >> >> wrote: >> >> > I realized this rather troublesome problem of going with SotL and not >> >> > considering primitives in generics: >> >> > >> >> > There's no way back from that. >> >> > >> >> > If ParallelArrays is released with the full contingent of 132 SAMs in >> >> Ops, >> >> > then we'll not only be stuck with them forever in the traditional >> >> > sense >> >> of >> >> > not eliminating classes to preserve backwards compatibility, but >> >> > there's >> >> > also no practical way forward, either. >> >> > >> >> > Currently, PA's DoubleParallelArray has a binarySearch method with >> >> > the >> >> > following signature: >> >> > >> >> > binarySearch(double target, Ops.DoubleComparator c). >> >> > >> >> > DoubleComparator itself is a SAM type, with signature #int(double, >> >> double). >> >> > >> >> > Therefore, a call to binarySearch with lambda would look like: >> >> > >> >> > binarySearch(12345678.9, {double a, double b -> >> >> Math.abs(a)-Math.abs(b)}); >> >> > >> >> > >> >> > But if in a later JDK there's a way to deprecate Ops.DoubleComparator >> >> > in >> >> > favour of either a function type #int(double, double), -OR- my >> >> > preferred >> >> > option of allowing java.util.Comparator, then there would be >> >> > a >> >> > second binarySearch method with signature: >> >> > >> >> > binarySearch(double target, Comparator c). >> >> > >> >> > Calling this one would look like: >> >> > >> >> > binarySearch(12345678.9, {double a, double b -> >> >> Math.abs(a)-Math.abs(b)}); >> >> > >> >> > which is indistinguishable from the Ops.DoubleComparator case. The >> >> compiler >> >> > can therefore not compile it, complaining about ambiguity. Users >> >> > would >> >> have >> >> > to explicitly specify either Ops.DoubleComparator or >> >> > Comparator, >> >> > breaking backwards compatibility and in general uglying up the API. >> >> > >> >> > I don't see a nice solution for this dilemma; one could use >> >> 'binarySearch2' >> >> > but that feels like a copout. One could attempt a scheme whereby >> >> overridden >> >> > methods can use some extra keyword to dictate to the compiler which >> >> > of a >> >> set >> >> > is to be preferred in case there's ambiguity, but that's adding extra >> >> > complexity, which may not be palatable at that point. >> >> > >> >> > ?--Reinier Zwitserloot >> >> > >> >> > >> >> >> >> >> > >> > > > From howard.lovatt at gmail.com Mon Jul 19 00:07:45 2010 From: howard.lovatt at gmail.com (Howard Lovatt) Date: Mon, 19 Jul 2010 17:07:45 +1000 Subject: Return Isn't Transparent In-Reply-To: References: Message-ID: @Neal, > Control-flow and reachability analysis is completely transparent in BGGA. That doesn't really answer the question directly. What about a minor variation in BGGA'ish syntax: int m() { final { -> Nothing } return42 = ?{ -> return 42; }.(); return42.(); ?// Oops error - must return value from m } ? -- Howard. On 18 July 2010 11:38, Howard Lovatt wrote: > Hi Neal, > > Are you saying that there are *no* cases where BGGA cannot reason > about the control flow? This is a slightly different question than > saying that a new language could not be designed with transparency in > mind. > > ? -- Howard. > > On 18 July 2010 01:15, Neal Gafter wrote: >> On Sat, Jul 17, 2010 at 6:46 AM, Howard Lovatt >> wrote: >>> >>> A return statement isn't transparent. This is well articulated in >>> Principles of Programming Languages R. D. Tennent (I don't have a copy >>> handy - but I think page 56 is one such place). Consider: >>> >>> int m() { >>> ?return 42; // To be wrapped in a lambda >>> } >>> >>> Then imagine that you could create a lambda that encapsulated the long >>> return and you called it: >>> >>> int m() { >>> ?{ -> return 42; }.(); >>> ?// Oops error - must return value from m >>> } >> >> This works just fine in BGGA.? The language can indeed be designed so that >> return is transparent. > From fweimer at bfk.de Mon Jul 19 06:58:37 2010 From: fweimer at bfk.de (Florian Weimer) Date: Mon, 19 Jul 2010 13:58:37 +0000 Subject: Revisiting primitives in generics: This can *NOT* be delayed for later. In-Reply-To: (John Nilsson's message of "Sat\, 17 Jul 2010 19\:00\:16 +0200") References: Message-ID: <82iq4b60aa.fsf@mid.bfk.de> * John Nilsson: > I'm note sure I understand why this is a problem. But if it is, why > not fix it? > > So I understand it correct that there is no runtime lookup for a > compatible method (supertype of DoubleComparator)? If so, why not add > this to the runtime? This would work. However, you still break backwards compatibility (at the source level, not the binary level) when a user has defined methods like this: void perform(Ops.DoubleComparator); void perform(Comparator); , assuming that Comparator and Comparator have the same erasure (at least at the Java level). Retrofitting a class to implement an interface is only safe if the interface is new. -- Florian Weimer BFK edv-consulting GmbH http://www.bfk.de/ Kriegsstra?e 100 tel: +49-721-96201-1 D-76133 Karlsruhe fax: +49-721-96201-99 From neal at gafter.com Mon Jul 19 07:58:52 2010 From: neal at gafter.com (Neal Gafter) Date: Mon, 19 Jul 2010 07:58:52 -0700 Subject: Return Isn't Transparent In-Reply-To: References: Message-ID: On Mon, Jul 19, 2010 at 12:07 AM, Howard Lovatt wrote: > > Control-flow and reachability analysis is completely transparent in > BGGA. > > That doesn't really answer the question directly. > Then perhaps I don't understand the question. > What about a minor variation in BGGA'ish syntax: > > int m() { > final { -> Nothing } return42 = { -> return 42; }.(); > return42.(); > // Oops error - must return value from m > } > One could certainly imagine modifying BGGA so it gets it wrong as your comment suggests, but that's not an error in BGGA. The end of m's body isn't reachable (because the last statement in m() doesn't complete normally). This method returns 42. You can download BGGA and try these yourself. From howard.lovatt at gmail.com Tue Jul 20 04:02:18 2010 From: howard.lovatt at gmail.com (Howard Lovatt) Date: Tue, 20 Jul 2010 21:02:18 +1000 Subject: Return Isn't Transparent In-Reply-To: References: Message-ID: @ Neal, I took up your suggestion of downloading BGGA and tried: 24: int m() { 25: final { ==> Nothing } return42 = { ==> return 42; }; 26: return42.invoke(); 27: } and got the error message: Return42.java:27: missing return statement } ^ So it would seem to me that BGGA only has return transparency for "simple cases". This was the point I was trying to clarify, is return transparency a general mechanism or does it have limitations? -- Howard. On 20 July 2010 00:58, Neal Gafter wrote: > On Mon, Jul 19, 2010 at 12:07 AM, Howard Lovatt > wrote: >> >> > Control-flow and reachability analysis is completely transparent in >> > BGGA. >> >> That doesn't really answer the question directly. > > Then perhaps I don't understand the question. > >> >> What about a minor variation in BGGA'ish syntax: >> >> ?int m() { >> ? final { -> Nothing } return42 = ?{ -> return 42; }.(); >> ? return42.(); >> ??// Oops error - must return value from m >> ?} > > One could certainly imagine modifying BGGA so it gets it wrong as your > comment suggests, but that's not an error in BGGA.? The end of m's body > isn't reachable (because the last statement in m() doesn't complete > normally).? This method returns 42. > > You can download BGGA and try these yourself. From neal at gafter.com Tue Jul 20 08:03:37 2010 From: neal at gafter.com (Neal Gafter) Date: Tue, 20 Jul 2010 08:03:37 -0700 Subject: Return Isn't Transparent In-Reply-To: References: Message-ID: It looks like you've found a bug. An invocation expression whose result type is "Nothing" is defined in the BGGA spec as not completing normally. On Tue, Jul 20, 2010 at 4:02 AM, Howard Lovatt wrote: > @ Neal, > > I took up your suggestion of downloading BGGA and tried: > > 24: int m() { > 25: final { ==> Nothing } return42 = { ==> return 42; }; > 26: return42.invoke(); > 27: } > > and got the error message: > > Return42.java:27: missing return statement > } > ^ > > So it would seem to me that BGGA only has return transparency for > "simple cases". This was the point I was trying to clarify, is return > transparency a general mechanism or does it have limitations? > > -- Howard. > > On 20 July 2010 00:58, Neal Gafter wrote: > > On Mon, Jul 19, 2010 at 12:07 AM, Howard Lovatt > > > wrote: > >> > >> > Control-flow and reachability analysis is completely transparent in > >> > BGGA. > >> > >> That doesn't really answer the question directly. > > > > Then perhaps I don't understand the question. > > > >> > >> What about a minor variation in BGGA'ish syntax: > >> > >> int m() { > >> final { -> Nothing } return42 = { -> return 42; }.(); > >> return42.(); > >> // Oops error - must return value from m > >> } > > > > One could certainly imagine modifying BGGA so it gets it wrong as your > > comment suggests, but that's not an error in BGGA. The end of m's body > > isn't reachable (because the last statement in m() doesn't complete > > normally). This method returns 42. > > > > You can download BGGA and try these yourself. > From howard.lovatt at gmail.com Tue Jul 20 17:58:13 2010 From: howard.lovatt at gmail.com (Howard Lovatt) Date: Wed, 21 Jul 2010 10:58:13 +1000 Subject: Return Isn't Transparent In-Reply-To: References: Message-ID: @Neal, Building on Reinier suggestion: private static int m6() { final { boolean ==> void } return42 = { boolean b ==> if ( b ) { return 42; } }; final { boolean ==> void } returnNot41 = { boolean b ==> if ( !b ) { return 41; } }; return42.invoke( true ); returnNot41.invoke( true ); } Presumably for difficult examples like this you would have no expectation of them working, which was my point that in general you can't have return transparency because analyzing the expressions is too difficult for the compiler. -- Howrad. On 21 July 2010 02:06, Neal Gafter wrote: > On Tue, Jul 20, 2010 at 8:55 AM, Reinier Zwitserloot > wrote: >> >> What happens when the closure isn't ===> Nothing, but ===> Void, e.g. to >> fit the parameter type of a bggaBasedForEach() method? > > Invoking something that is declared to return Void is considered by the > compiler to be capable of completing normally.? That's what you want for a > for-each loop: a loop can complete normally even if its iterations can't, > because it may execute zero iterations. > > -Neal -- ? -- Howard. From neal at gafter.com Tue Jul 20 19:54:44 2010 From: neal at gafter.com (Neal Gafter) Date: Tue, 20 Jul 2010 19:54:44 -0700 Subject: Return Isn't Transparent In-Reply-To: References: Message-ID: Howard- These are not difficult cases at all - they are not supposed to be "transparent". In these cases the programmer has explicitly selected a type to store the lambda that discards information about the lambda's completion status. That is both legal and allowed, and sometimes useful. Transparency does not and should not prevent that. Transparency makes it possible to select a type (or have a type selected by type inference) that does not discard that information. Cheers, Neal On Tue, Jul 20, 2010 at 5:58 PM, Howard Lovatt wrote: > @Neal, > > Building on Reinier suggestion: > > private static int m6() { > final { boolean ==> void } return42 = { boolean b ==> if ( b > ) { > return 42; } }; > final { boolean ==> void } returnNot41 = { boolean b ==> if > ( !b ) { > return 41; } }; > return42.invoke( true ); > returnNot41.invoke( true ); > } > > Presumably for difficult examples like this you would have no > expectation of them working, which was my point that in general you > can't have return transparency because analyzing the expressions is > too difficult for the compiler. > > -- Howrad. > > On 21 July 2010 02:06, Neal Gafter wrote: > > On Tue, Jul 20, 2010 at 8:55 AM, Reinier Zwitserloot > > wrote: > >> > >> What happens when the closure isn't ===> Nothing, but ===> Void, e.g. to > >> fit the parameter type of a bggaBasedForEach() method? > > > > Invoking something that is declared to return Void is considered by the > > compiler to be capable of completing normally. That's what you want for > a > > for-each loop: a loop can complete normally even if its iterations can't, > > because it may execute zero iterations. > > > > -Neal > > > > -- > -- Howard. > From neal at gafter.com Tue Jul 20 20:00:14 2010 From: neal at gafter.com (Neal Gafter) Date: Tue, 20 Jul 2010 20:00:14 -0700 Subject: Return Isn't Transparent In-Reply-To: References: Message-ID: One more thing. This example is indeed "transparent" because it behaves the same as the non-lambda version: private static int m6() { if ( true ) { return 42; } if ( !true ) { return 41; } } Cheers, Neal On Tue, Jul 20, 2010 at 5:58 PM, Howard Lovatt wrote: > @Neal, > > Building on Reinier suggestion: > > private static int m6() { > final { boolean ==> void } return42 = { boolean b ==> if ( b > ) { > return 42; } }; > final { boolean ==> void } returnNot41 = { boolean b ==> if > ( !b ) { > return 41; } }; > return42.invoke( true ); > returnNot41.invoke( true ); > } > > Presumably for difficult examples like this you would have no > expectation of them working, which was my point that in general you > can't have return transparency because analyzing the expressions is > too difficult for the compiler. > > -- Howrad. > > On 21 July 2010 02:06, Neal Gafter wrote: > > On Tue, Jul 20, 2010 at 8:55 AM, Reinier Zwitserloot > > wrote: > >> > >> What happens when the closure isn't ===> Nothing, but ===> Void, e.g. to > >> fit the parameter type of a bggaBasedForEach() method? > > > > Invoking something that is declared to return Void is considered by the > > compiler to be capable of completing normally. That's what you want for > a > > for-each loop: a loop can complete normally even if its iterations can't, > > because it may execute zero iterations. > > > > -Neal > > > > -- > -- Howard. > From howard.lovatt at gmail.com Tue Jul 20 22:01:01 2010 From: howard.lovatt at gmail.com (Howard Lovatt) Date: Wed, 21 Jul 2010 15:01:01 +1000 Subject: Return Isn't Transparent In-Reply-To: References: Message-ID: I would argue that if you can't do any re-factoring - what's the point? Starting with something along the lines of: private static int m6( final boolean b ) { if ( b ) { return 42; } else { return 41; } } and re-factoring to: private static int m6( final boolean b ) { final { boolean ==> void } return42 = { boolean b ==> if ( b ) { return 42; } }; final { boolean ==> void } returnNot41 = { boolean b ==> if ( !b ) { return 41; } }; return42.invoke( b ); returnNot41.invoke( b ); } might be quite a reasonable thing to do because you want to split a complicated function into stages and compose the stages. If the limitation is that you pretty much have to wrap the whole expression to get transparency, then the use cases are reduced to simple serial execution like the existing loops and if structure. This doesn't sound that useful, since these structures already exist. ? -- Howard. On 21 July 2010 13:00, Neal Gafter wrote: > One more thing.? This example is indeed "transparent" because it behaves the > same as the non-lambda version: > > ? ? ?? private static int m6() { > ? ? ? ? ? ? ?? if ( true ) { return 42; } > ? ? ? ? ? ? ?? if ( !true ) { return 41; } > ? ? ?? } > > Cheers, > Neal > > On Tue, Jul 20, 2010 at 5:58 PM, Howard Lovatt > wrote: >> >> @Neal, >> >> Building on Reinier suggestion: >> >> ? ? ? ?private static int m6() { >> ? ? ? ? ? ? ? ?final { boolean ==> void } return42 = { boolean b ==> if ( >> b ) { >> return 42; } }; >> ? ? ? ? ? ? ? ?final { boolean ==> void } returnNot41 = { boolean b ==> if >> ( !b ) { >> return 41; } }; >> ? ? ? ? ? ? ? ?return42.invoke( true ); >> ? ? ? ? ? ? ? ?returnNot41.invoke( true ); >> ? ? ? ?} >> >> Presumably for difficult examples like this you would have no >> expectation of them working, which was my point that in general you >> can't have return transparency because analyzing the expressions is >> too difficult for the compiler. >> >> ?-- Howrad. >> >> On 21 July 2010 02:06, Neal Gafter wrote: >> > On Tue, Jul 20, 2010 at 8:55 AM, Reinier Zwitserloot >> > wrote: >> >> >> >> What happens when the closure isn't ===> Nothing, but ===> Void, e.g. >> >> to >> >> fit the parameter type of a bggaBasedForEach() method? >> > >> > Invoking something that is declared to return Void is considered by the >> > compiler to be capable of completing normally.? That's what you want for >> > a >> > for-each loop: a loop can complete normally even if its iterations >> > can't, >> > because it may execute zero iterations. >> > >> > -Neal From int19h at gmail.com Tue Jul 20 23:16:27 2010 From: int19h at gmail.com (Pavel Minaev) Date: Tue, 20 Jul 2010 23:16:27 -0700 Subject: Return Isn't Transparent In-Reply-To: References: Message-ID: On Tue, Jul 20, 2010 at 10:01 PM, Howard Lovatt wrote: > I would argue that if you can't do any re-factoring - what's the > point? Starting with something along the lines of: > > ? ? ? private static int m6( final boolean b ) { > ? ? ? ? ? ? ? if ( b ) { return 42; } > ? ? ? ? ? ? ? else { return 41; } > ? ? ? } > > and re-factoring to: > ? ? ?private static int m6( final boolean b ) { > ? ? ? ? ? ? ? final { boolean ==> void } return42 = { boolean b ==> > if ( b ) { return 42; } }; > ? ? ? ? ? ? ? final { boolean ==> void } returnNot41 = { boolean b > ==> if ( !b ) { return 41; } }; > ? ? ? ? ? ? ? return42.invoke( b ); > ? ? ? ? ? ? ? returnNot41.invoke( b ); > ? ? ? } There is no "refactoring" in the definition of transparency. In any case, your code is still not equivalent according to Java reachability rules: "if (x) return; if (!x) return;" is not treated the same as "if (x) return; else return;" outside of lambdas as it is, so why it should be treated any differently inside them? That's what transparency is all about - that things just keep working the same way. > If the limitation is that you pretty much have to wrap the whole > expression to get transparency, then the use cases are reduced to > simple serial execution like the existing loops and if structure. This > doesn't sound that useful, since these structures already exist. Only some primitive forms of loops exist in Java on language level. The whole point of transparency as implemented by e.g. BGGA is to be able to implement more advanced control structures as libraries - including, yes, more advanced loop forms (e.g. a loop over several collections in parallel). From ss at comp.lancs.ac.uk Thu Jul 22 16:28:51 2010 From: ss at comp.lancs.ac.uk (Steven Simpson) Date: Fri, 23 Jul 2010 00:28:51 +0100 Subject: Yield considered harmful In-Reply-To: References: <2A48108C-0A72-4EC7-B4F6-1A4542F1DC2B@gmail.com> Message-ID: <4C48D433.8060408@comp.lancs.ac.uk> Hi, On 13/07/10 12:41, Stephen Colebourne wrote: > The option FCM-JCA outlined followed the strategy of using syntax to > enable two uses of "return" > > Syntax A example: > forEach(list, #(str) { // looks more like an inner class > if (str.length()> 3) return; // local-return > System.out.println(str); > }); > > Syntax B example: > list.each() for (String str) { // looks like a code block > if (str.length()> 3) return; // lexical return > System.out.println(str); > } While I like that, something else occurred to me. forEach(), along with everything else that provides control abstraction, must accept a block, so you need a type (or family thereof) that captures the abstraction of a block. What does a block do? Stuff, resulting in a return, a throw, a break, a continue or a fallthrough. So define forEach in terms of a function type returning a BlockExit type to represent those choices: static BlockExit forEach(Iterable coll, #BlockExit(T) block) { for (Iterator iter = coll.iterator(); iter.hasNext(); ) { BlockExit exit = block.invoke(iter.next()); if (exit.breaksLocal()) break; if (exit.continuesLocal()) continue; if (exit.returns()) // Go here for return, break/continue