From peter.levart at marand.si Mon Feb 1 00:16:44 2010 From: peter.levart at marand.si (Peter Levart) Date: Mon, 1 Feb 2010 09:16:44 +0100 Subject: What is the meaning of this? In-Reply-To: <997cab101001311417rd0c4f45lc18ac4b98769d4ec@mail.gmail.com> References: <17b2302a1001281119t4ea1c34cya8a05e59864b407@mail.gmail.com> <201001311039.55711.peter.levart@gmail.com> <997cab101001311417rd0c4f45lc18ac4b98769d4ec@mail.gmail.com> Message-ID: <201002010916.44955.peter.levart@marand.si> On Sunday 31 January 2010 23:17:07 Lawrence Kesteloot wrote: > On Sun, Jan 31, 2010 at 1:39 AM, Peter Levart wrote: > > #int(int) factorial = #fact(int i)(i == 0 ? 1 : i * (int) (fact.(i - 1))); > > That's where you'd want the optional return type to go. Better to just > allow a reference to the "factorial" variable. > > For example, if the compiler can't deduce the return type above, then > you'd write: > > #int(int) factorial = #int(int i)(i == 0 ? 1 : i * factorial.(i - 1)); > I thought about that too. It would have to be a special case of local variable declarator, otherwise it could get complicated. For example, would you allow this: #int(int) test(#int(int) func) { func.(10); return func; } ... final #int(int) factorial = test( #int(int i)(i == 0 ? 1 : i * factorial.(i - 1)) ); Besides, that forces you to declare a variable beforehand. One would want to specify a recursive lambda inline: doSomethingWith( #fact(int i)(i == 0 ? 1 : i * (int)fact.(i - 1)) ); Explicit return type in lambda expressions is hopefully not needed or it can be specified elsewhere. The place between '#' and '(' for a Type has the same drawbacks in function type syntax as in lambda expression syntax - it is not perfect for composing 2nd order functions. As the following syntax is more readable for function types in that case: FunctionType: '#' '(' ParameterTypes_opt -> ReturnType Throws_opt ')' The same could be made for lambda expressions: LambdaExpression: '#' '(' ParameterList_opt -> ReturnType_opt ')' Block Regards, Peter From neal at gafter.com Mon Feb 1 00:58:09 2010 From: neal at gafter.com (Neal Gafter) Date: Mon, 1 Feb 2010 00:58:09 -0800 Subject: Preparing for the 0.2 draft In-Reply-To: <4B62A2CE.7080300@sun.com> References: <4B62A2CE.7080300@sun.com> Message-ID: <15e8b9d21002010058w49c686e3i325627d6362614d8@mail.gmail.com> I ran the stats for openjdk6 sources, but not quite as you asked. I started answering questions that filter out those anons that couldn't possibly be lambdas for one reason or another, and then answered the rest of the interesting questions from among those. It isn't interesting to know which anonymous inner classes use "this" if they couldn't be lambdas because there are, for example, constructor arguments. So here are the stats: Total anonymous classes: 1155 But with no constructor args: 1068 And defining only one method: 883 And where the type is a SAM type: 811 Those that are recursive: 0 Those that reference 'this': 2 (see below) Those that reference an enclosing 'this': 48 Those that are interfaces: 752 Those that are classes: 59 Total distinct interfaces: 50 (see below) Total distinct classes: 13 (see below) A couple of interesting things about the use cases exposed by this analysis: There is virtually no utility to making "this" refer to the function itself. In the two cases where "this" appears, one does not actually use the value (null would work as well), and the other could be trivially rewritten using the proposed definite-assignment rule for a lambda appearing in a variable initializer. None of the anonymous SAMs are recursive. On the other hand, there are a fair number of explicit references to the instance of the enclosing class, which would benefit from lexical scoping. I also note that the use cases involving classes do not suffer from lexically scoped lambdas because they can continue to be written exactly as they are today. Even the proposed lambda spec does not provide typesafe access to the underlying SAM type, so I can't see anywhere this code base would benefit from lambdas in which "this" references the function object rather than the lexically enclosing instance. Since Josh hadn't seen any use cases that benefit from lexical scoping of "this", I've also enclosed pointers to 48 locations in the openjdk6 source base. ==================================== Below are shown the two places where "this" appears. The first would be easily solved by a rule treating a variable as definitely assigned in the right-hand-side of its declaration if the initializing expression is a delegate. In the second, "null" would work just as well as "this", because the value is discarded by the caller (on the next line). in src/share/classes/sun/java2d/StateTrackableDelegate.java: public synchronized StateTracker getStateTracker() { StateTracker st = theTracker; if (st == null) { switch (theState) { case IMMUTABLE: st = StateTracker.ALWAYS_CURRENT; break; case STABLE: st = new StateTracker() { public boolean isCurrent() { return (theTracker == this); } }; break; .... in src/share/classes/javax/imageio/spi/IIORegistry.java PrivilegedAction doRegistration = new PrivilegedAction() { public Object run() { Iterator categories = getCategories(); while (categories.hasNext()) { Class c = (Class)categories.next(); for (IIOServiceProvider p : ServiceLoader.loadInstalled(c)) { registerServiceProvider(p); } } return this; } }; AccessController.doPrivileged(doRegistration); ==================================== Below are all the distinct classes com.sun.tools.example.debug.tty.Commands.AsyncExecution com.sun.tools.hat.internal.model.AbstractJavaHeapObjectVisitor com.sun.tools.hat.internal.util.Comparer java.awt.font.TextLine.Function java.io.InputStream java.io.OutputStream java.lang.Thread java.nio.charset.CoderResult.Cache java.util.TimerTask java.util.regex.Pattern.CharProperty java.util.regex.Pattern.CharPropertyNames.CharPropertyFactory java.util.regex.Pattern.CharPropertyNames.CloneableProperty javax.swing.AbstractAction ==================================== Below are all the distinct interfaces com.sun.java.util.jar.pack.Histogram.BitMetric com.sun.jdi.connect.Transport com.sun.jmx.remote.internal.NotificationBufferFilter com.sun.media.sound.ModelTransform com.sun.net.ssl.HostnameVerifier com.sun.security.auth.callback.DialogCallbackHandler.Action com.sun.tools.example.debug.bdi.InputListener com.sun.tools.example.debug.bdi.OutputListener com.sun.tools.example.debug.expr.ExpressionParser.GetFrame com.sun.tools.hat.internal.oql.ObjectVisitor com.sun.tools.jdi.CommandSender com.sun.tools.script.shell.Main.Command java.awt.Conditional java.awt.KeyEventPostProcessor java.awt.event.ActionListener java.awt.event.HierarchyListener java.beans.ExceptionListener java.beans.PropertyChangeListener java.beans.VetoableChangeListener java.io.FilenameFilter java.io.ObjectInputValidation java.lang.Runnable java.net.CookiePolicy java.net.HttpCookie.CookieAttributeAssignor java.security.PrivilegedAction java.security.PrivilegedExceptionAction java.util.Comparator java.util.Enumeration java.util.concurrent.Callable java.util.concurrent.Executor java.util.concurrent.ThreadFactory javax.imageio.event.IIOReadWarningListener javax.imageio.event.IIOWriteWarningListener javax.management.NotificationListener javax.swing.UIDefaults.ActiveValue javax.swing.UIDefaults.LazyValue javax.swing.event.CaretListener javax.swing.event.ChangeListener javax.swing.event.HyperlinkListener javax.xml.crypto.KeySelectorResult javax.xml.crypto.NodeSetData sun.awt.RequestFocusController sun.java2d.StateTracker sun.java2d.cmm.ProfileActivator sun.java2d.loops.ProcessPath.EndSubPathHandler sun.misc.JavaIODeleteOnExitAccess sun.misc.JavaLangAccess sun.misc.JavaNetAccess sun.nio.ch.FileChannelImpl.FileLockTable.Releaser sun.nio.ch.Interruptible ==================================== Locations in openjdk6 where lexically-scoped "this" would be helpful: src/share/classes/com/sun/java/util/jar/pack/Histogram.java:187 src/share/classes/com/sun/media/sound/SoftJitterCorrector.java:126 src/share/classes/com/sun/security/auth/PolicyFile.java:849 src/share/classes/com/sun/tools/example/debug/bdi/JDIEventSource.java:80 src/share/classes/com/sun/tools/jdi/ClassTypeImpl.java:167 src/share/classes/com/sun/tools/jdi/ClassTypeImpl.java:189 src/share/classes/com/sun/tools/jdi/ObjectReferenceImpl.java:341 src/share/classes/java/awt/Container.java:2744 src/share/classes/java/awt/Container.java:4268 src/share/classes/java/awt/Container.java:4283 src/share/classes/java/awt/EventQueue.java:823 src/share/classes/java/awt/SequencedEvent.java:92 src/share/classes/java/awt/datatransfer/Clipboard.java:128 src/share/classes/java/awt/datatransfer/Clipboard.java:326 src/share/classes/java/beans/beancontext/BeanContextSupport.java:1296 src/share/classes/java/beans/beancontext/BeanContextSupport.java:1310 src/share/classes/java/lang/Class.java:1306 src/share/classes/java/nio/channels/spi/AbstractSelector.java:208 src/share/classes/java/security/ProtectionDomain.java:317 src/share/classes/java/util/regex/Pattern.java:3352 src/share/classes/javax/management/monitor/Monitor.java:1609 src/share/classes/javax/swing/BufferStrategyPaintManager.java:213 src/share/classes/javax/swing/JComponent.java:4804 src/share/classes/javax/swing/JOptionPane.java:1011 src/share/classes/javax/swing/JOptionPane.java:1524 src/share/classes/javax/swing/ProgressMonitor.java:214 src/share/classes/javax/swing/SwingWorker.java:902 src/share/classes/javax/swing/TimerQueue.java:95 src/share/classes/javax/swing/plaf/basic/BasicLookAndFeel.java:2123 src/share/classes/javax/swing/plaf/basic/BasicPopupMenuUI.java:745 src/share/classes/javax/swing/plaf/basic/BasicPopupMenuUI.java:778 src/share/classes/javax/swing/text/JTextComponent.java:4932 src/share/classes/sun/applet/AppletClassLoader.java:631 src/share/classes/sun/applet/AppletViewer.java:641 src/share/classes/sun/awt/datatransfer/SunClipboard.java:113 src/share/classes/sun/awt/datatransfer/SunClipboard.java:287 src/share/classes/sun/awt/im/InputContext.java:648 src/share/classes/sun/security/jca/ProviderConfig.java:244 src/share/classes/sun/security/provider/SeedGenerator.java:255 src/share/classes/sun/tools/jconsole/inspector/XSheet.java:181 src/share/classes/sun/tools/jconsole/inspector/XSheet.java:216 src/share/classes/sun/tools/jconsole/inspector/XSheet.java:329 src/share/classes/sun/tools/jconsole/inspector/XSheet.java:371 src/share/classes/sun/tools/jconsole/inspector/XSheet.java:411 src/share/classes/sun/tools/jconsole/inspector/XSheet.java:668 src/share/classes/sun/tools/jconsole/inspector/XSheet.java:686 src/share/classes/sun/tools/jconsole/inspector/XTree.java:159 src/share/classes/sun/tools/jconsole/inspector/XTree.java:261 From cdr at intellij.com Mon Feb 1 04:17:38 2010 From: cdr at intellij.com (Alexey Kudravtsev) Date: Mon, 1 Feb 2010 15:17:38 +0300 Subject: SAM interface/class figures [Re: Preparing for the 0.2 draft] Message-ID: <96324399B8D14BCBBFF33FC79060BFF7@Labs.IntelliJ.Net> These are figures for the Intellij IDEA (http://www.jetbrains.com/idea/) source base. Total files inspected: 34794 Total java classes: 69703 All SAM classes: 1920 (2,8%) All Anonymous classes: 21908 (31,4%) anonymous classes which Extend class: 7474 (34,1%) anonymous classes which implement interface: 14434 (65,9%) anonymous classes which extend SAM: 12597 (57,5%) anon classes with single method: 18431 (84,1%) anon classes with fields: 149 (0,7%) anon classes with 'this' references: 116 (0,5%) anon classes with qualified 'this' references: 77 (0,4%) anon classes which recurse: 15 (0,1%) Alexey Kudravtsev Intellij IDEA developer JetBrains, Inc http://www.jetbrains.com "Develop with pleasure!" > Message: 1 > Date: Fri, 29 Jan 2010 00:56:46 -0800 > From: Alex Buckley > Subject: Preparing for the 0.2 draft > To: "lambda-dev at openjdk.java.net" > Message-ID: <4B62A2CE.7080300 at sun.com> > Content-Type: text/plain; CHARSET=US-ASCII; format=flowed > > Many thanks for everyone's comments on the 0.1 draft. The "foo.()" > invocation syntax looks like a winner, and I am seriously considering > rearranging the function type syntax to have argument types first. > Requiring a lambda expression to denote its return type is also > increasingly appealing. SAM classes are staying for now. > > The discussion of transparent 'this' v. non-transparent 'this' comes > down to the question of what a lambda in Java is for. (Zdenek Tronicek > ably made this point earlier.) Part of the answer lies in understanding > when to use an anonymous inner class and when to use a lambda > expression. A step towards that understanding would be knowing how > anonymous inner classes are used today. > > It would be very interesting if list members could provide the following > statistics for codebases with which they are familiar: > > - What %age of anonymous inner classes declare a single non-Object method? > - What %age of single-method anonymous inner classes declare fields? > - What %age of single methods in anonymous inner classes use 'this'? > - What %age of single methods in anonymous inner classes recurse? > - Are the %ages different for anonymous inner classes that implement an > interface v. that extend a class ? > > (As well as anonymous inner classes, any utility class that > implements/extends a SAM type is of interest, such as this nested class > from the extra166y.ParallelArray Javadoc: > static final class IsSenior implements Predicate { > public boolean op(Student s) { return s.credits > 90; } > } > ) > > Alex > From Alex.Buckley at Sun.COM Mon Feb 1 13:07:09 2010 From: Alex.Buckley at Sun.COM (Alex Buckley) Date: Mon, 01 Feb 2010 13:07:09 -0800 Subject: Preparing for the 0.2 draft In-Reply-To: References: <4B62A2CE.7080300@sun.com> Message-ID: <4B67427D.9080608@sun.com> Alex Blewitt wrote: > Lastly, say we allow SAM on (abstract) classes iff they have no (non- > final) fields. We then end up with an evolution problem in that L can > extend A and be compiled, but then a field is added to A giving A'. So > it won't compile any more but now we have a problem with the compiled > L, which may be in a different file. The same holds for removing the > (implicit) default constructor. Sorry, terseness factor too high. I think L is a lambda expression and A is an abstract class and you mean to apply a lambda conversion from typeof(L) to A? Binary incompatibility is always possible with separate compilation. If lambda conversion could only target interfaces, not abstract classes, then removing a method from a SAM interface will cause problems for lambdas that were converted to the earlier definition of the interface. Source incompatibility is more interesting: adding a method to a SAM interface (binary *compatible* but source *incompatible*) will cause problems for lambdas that were converted to the earlier definition of the interface. (Of course, a good module system will provide a version scheme to document binary/source incompatible evolution, and a static mechanism to identify client modules affected by such evolution ;-) Alex From dl at cs.oswego.edu Mon Feb 1 17:18:05 2010 From: dl at cs.oswego.edu (Doug Lea) Date: Mon, 01 Feb 2010 20:18:05 -0500 Subject: Preparing for the 0.2 draft In-Reply-To: <4B67427D.9080608@sun.com> References: <4B62A2CE.7080300@sun.com> <4B67427D.9080608@sun.com> Message-ID: <4B677D4D.4030408@cs.oswego.edu> Sorry again that I've been swamped lately and haven't had time to offer much constructive feedback. But here's a reminder about the some of the needs/wants we have of lambdas wrt parallel execution. Among the main goals is to make it easier to correctly and efficiently perform operations such as apply, map, reduce to all elements in an aggregate -- usually an array or collection. The "correctly" part (increasingly known by the crummy term "deterministic parallelism") means that you do this while avoiding unintentional races/interference. But without more heavy language mechanics than is possible (or desirable) in Java, you cannot preclude all racy constructions. So we are left with language and API choices that make non-racy usages easy and natural, and racy ones less so. (Aside: this has nothing to do with pure functional programming. We equally like "apply(addOne, myArray)" and "map(plusOne, myArray)"). Anything that makes racy code appear to be plain and innocent makes me nervous: Ideally, for the sake of parallel execution, we'd disallow automatic sharing of locals and automatic elevation of "this", "return" or "break" to enclosing scopes. Because of course, when run in parallel, the lexically enclosing scopes have little to do with actual execution context. For data, these are possible races, and for control they are possible crazinesses (returns to nowhere, etc). So if any of these things are made possible (which I'm sympathetic to, or at least tolerant of exploring), they must be made bug-avoiding when used in parallel contexts. Not sure how. -Doug From neal at gafter.com Mon Feb 1 20:56:50 2010 From: neal at gafter.com (Neal Gafter) Date: Mon, 1 Feb 2010 20:56:50 -0800 Subject: Preparing for the 0.2 draft In-Reply-To: <4B677D4D.4030408@cs.oswego.edu> References: <4B62A2CE.7080300@sun.com> <4B67427D.9080608@sun.com> <4B677D4D.4030408@cs.oswego.edu> Message-ID: <15e8b9d21002012056t73d41a56we72d0d7de4f013c4@mail.gmail.com> On Mon, Feb 1, 2010 at 5:18 PM, Doug Lea
wrote: > Anything that makes racy code appear to be plain > and innocent makes me nervous: Ideally, for > the sake of parallel execution, we'd disallow automatic > sharing of locals and automatic elevation of "this", "return" > or "break" to enclosing scopes. Because of course, > when run in parallel, the lexically enclosing scopes > have little to do with actual execution context. For > data, these are possible races, Parallelism is orthogonal to whether or not lexically enclosing scopes "have to do with" the execution context of a lambda. Racy code arises from the intersection of mutating data and concurrency without synchronization. That, too, has nothing to do with whether or not the concurrency is expressed using higher-order code (i.e. lambdas) or imperative code. Conflating these concepts in the language design threatens to undermine their utility. > and for control they are possible crazinesses (returns to nowhere, etc). We've already shown that the ability to express nonlocal control (return, for example) can lead to very natural and expressive concurrency APIs, and the evidence from the use of such APIs (e.g. in Scala) is that the kind of errors you're concerned about don't arise in practice. From John.Rose at Sun.COM Mon Feb 1 23:28:02 2010 From: John.Rose at Sun.COM (John Rose) Date: Mon, 01 Feb 2010 23:28:02 -0800 Subject: Preparing for the 0.2 draft In-Reply-To: <4B67427D.9080608@sun.com> References: <4B62A2CE.7080300@sun.com> <4B67427D.9080608@sun.com> Message-ID: <96862318-7DE2-46C1-81F6-6CB5B4B026ED@sun.com> Playing with syntax dial settings between "terse" and "verbose" (aka "magic" and "tragic"), I just remembered a possible compromise for dealing with complicated SAM types. In case it hasn't been proposed yet in the context of closures: Given: class MySAM { MySAM(int arg1, int arg2) {...} abstract int justOneMethod(int x); } Then maybe: MySAM bar = new MySAM(arg1, arg2) #(int x) 1+x; As sugar for: MySAM bar = new MySAM(arg1, arg2) { int justOneMethod(int x) { return 1+x; } }; Making the 'new' explicit defuses the tricky arguments about when the object is created, whether the SAM has fields, has a nullary constructor, has construction side effects, etc. -- John P.S. For interfaces at least, +1 to keep the implicit assignment conversion: interface iSAM { int justOneMethod(int x); } Object oof = (iSAM) #(int x) 1+x; // cast conversion implied by implicit conversion And, please be loose about object initialization and identity, when 'new' is absent, so implementations can do hoisting & caching without breaking spec.: for (int i = 0; i < 2; i++) { iSAM foo = #(int x) 22+x; // initialization => implicit conversion if (i == 0) System.out.println(foo == oof); // must be false else System.out.println(foo == oof); // implementation defined result oof = foo; } From peter.levart at marand.si Tue Feb 2 01:32:37 2010 From: peter.levart at marand.si (Peter Levart) Date: Tue, 2 Feb 2010 10:32:37 +0100 Subject: Preparing for the 0.2 draft In-Reply-To: <4B677D4D.4030408@cs.oswego.edu> References: <4B62A2CE.7080300@sun.com> <4B67427D.9080608@sun.com> <4B677D4D.4030408@cs.oswego.edu> Message-ID: <201002021032.37544.peter.levart@marand.si> On Tuesday 02 February 2010 02:18:05 Doug Lea wrote: > Anything that makes racy code appear to be plain > and innocent makes me nervous: Ideally, for > the sake of parallel execution, we'd disallow automatic > sharing of locals and automatic elevation of "this", "return" > or "break" to enclosing scopes. Because of course, > when run in parallel, the lexically enclosing scopes > have little to do with actual execution context. For > data, these are possible races, and for control they > are possible crazinesses (returns to nowhere, etc). > > So if any of these things are made possible (which > I'm sympathetic to, or at least tolerant of exploring), > they must be made bug-avoiding when used in parallel > contexts. Not sure how. I'm not sure to what extent a language should provide tools that prevent a user to shoot himself in the foot. Surely parallel execution (concurrency) is one area where such tools would be useful. What you're asking for are some kind of semi-pure functions - in terms that the only state they have mutable access to is passed via formal parameters. BGGA's RestrictedFunction is an attempt in this direction (from http://www.javac.info/closures-v05.html): "If the target of the closure conversion extends the marker interface java.lang.RestrictedFunction then it is a compile-time error if the function being converted contains a break, continue or return statement whose target is outside the closure literal; and it is a compile-time error if the function being converted refers to a non-final local variable declared in an enclosing scope." But maybe the following restrictions would be more appropriate: it is a compile-time error if the function being converted refers to any variable declared in an enclosing scope except to final (local variables, instance fields or static fields) that are either of primitive type or of reference type which is "known to be immutable" (primitive wrappers, String, BigDecimal, etc...); and it is a compile-time error if the function is being converted to a subclass of a SAM class I would not restrict non-local transfers though, since they are proven useful even in parallel execution contexts and if they inadvertently provoke "crazinesses" (returns to nowhere) it is not hard to spot them at run-time. More importantly: non-local transfers don't affect "deterministic parallelism" - when your code is logically incorrect you always get an exception. There would have to be a distinct syntax for function types that extend "RestrictedFunction". Maybe the following: RestrictedFunctionType: '#' ReturnType '((' ParameterTypes_opt '))' Throws_opt Or alternatively (I like that more): RestrictedFunctionType: '#((' ParameterTypes_opt -> ReturnType Throws_opt '))' Double parens indicate that a function of that type is "isolated" from it's lexicaly enclosing mutable or lambda instance state. One question that remains is how to categorize reference types which are "known to be immutable". Some of the possibilities are: - only allow predefined Java platform types (as for example attributes of annotations) - only allow types which implement/extend a marker interface (Immutable) and retrofit platform types - this variant is more flexible but assumes that other implementors know what they're doing. Regards, Peter From dl at cs.oswego.edu Tue Feb 2 03:55:08 2010 From: dl at cs.oswego.edu (Doug Lea) Date: Tue, 02 Feb 2010 06:55:08 -0500 Subject: Preparing for the 0.2 draft In-Reply-To: <15e8b9d21002012056t73d41a56we72d0d7de4f013c4@mail.gmail.com> References: <4B62A2CE.7080300@sun.com> <4B67427D.9080608@sun.com> <4B677D4D.4030408@cs.oswego.edu> <15e8b9d21002012056t73d41a56we72d0d7de4f013c4@mail.gmail.com> Message-ID: <4B68129C.5000008@cs.oswego.edu> Neal Gafter wrote: > That, too, has nothing to do with whether or not the > concurrency is expressed using higher-order code (i.e. lambdas) or > imperative code. Conflating these concepts in the language design > threatens to undermine their utility. > Not so sure. My main point is that in the realm of parallelism, there is an increasing trend to enforce non-raciness by construction (for a Java-based effort, see "DPJ" http://dpj.cs.uiuc.edu/DPJ/Home.html). And again, just because you cannot fully preclude raciness in Java does not meant you should ignore the issue. > that the kind of errors you're concerned about don't arise > in practice. I don't think that we know very much about this. There is very little experience out there with extensive parallel usages of such constructs, and those few parallel languages that include some form of lambda/closure (including X10 and Fortress) do not hoist scopes. -Doug From peter.levart at marand.si Tue Feb 2 04:39:22 2010 From: peter.levart at marand.si (Peter Levart) Date: Tue, 2 Feb 2010 13:39:22 +0100 Subject: Preparing for the 0.2 draft In-Reply-To: <201002021032.37544.peter.levart@marand.si> References: <4B62A2CE.7080300@sun.com> <4B677D4D.4030408@cs.oswego.edu> <201002021032.37544.peter.levart@marand.si> Message-ID: <201002021339.22989.peter.levart@marand.si> On Tuesday 02 February 2010 10:32:37 Peter Levart wrote: > More importantly: non-local transfers don't affect "deterministic parallelism" - when your code is logically incorrect you always get an exception. That was not precise enough. I meant to say that if you view the system as composed of two parts: - a parallel execution framework written by experts that provides an API which accepts "restricted" functions; and - restricted functions writen by unexperienced users In this case non-local transfers triggered by restricted functions can be handled by expert framework in a way that makes the whole system deterministic. Likewise the mutability of objects passed to restricted functions as formal parameters can be allowed because they are controlled by expert framework. Regards, Peter From reinier at zwitserloot.com Tue Feb 2 05:53:22 2010 From: reinier at zwitserloot.com (Reinier Zwitserloot) Date: Tue, 2 Feb 2010 14:53:22 +0100 Subject: Joint types inconsistency: Suggesting we use & instead of | for exceptions. Message-ID: <560fb5ed1002020553x191d064aw1a7239338674a821@mail.gmail.com> In generics, there's a concept that's fairly close to a joint type, which involves the ampersand (&) character: public class Foo {} However, in the proposed (draft v0.1) closure syntax, | is used to make a "joint type" for exceptions: #int(int, int)(throws IOException|SQLException) x; While the exception list there isn't really a single (joint) type, there are some conceptual similarities, and it makes a lot more sense to say that a method throws both IOException and SQLException, than to say it throws either IOException or SQLException (checked exceptions are interesting on the type level to track what you need to catch, and if a method can throw either, then you need to catch both in order to satisfy javac, hence, it makes more sense to consider this a joint type instead of a disjoint type). I therefore suggest the syntax is changed to: #int(int, int)(throws IOException & SQLException) x; the side benefit of a bar being hard to differentiate from an uppercase i or a lowercase L in many fonts whereas the ampersand is always distinctive is extra gravy. &, of course, invokes thoughts of 'and', whereas the bar invokes thoughts of 'or', given that they are used in the and or op operators in java. NB: Is there any reason why it can't mirror method declarations and use commas? --Reinier Zwitserloot From peter.levart at marand.si Tue Feb 2 06:27:28 2010 From: peter.levart at marand.si (Peter Levart) Date: Tue, 2 Feb 2010 15:27:28 +0100 Subject: Joint types inconsistency: Suggesting we use & instead of | for exceptions. In-Reply-To: <560fb5ed1002020553x191d064aw1a7239338674a821@mail.gmail.com> References: <560fb5ed1002020553x191d064aw1a7239338674a821@mail.gmail.com> Message-ID: <201002021527.28618.peter.levart@marand.si> On Tuesday 02 February 2010 14:53:22 Reinier Zwitserloot wrote: > In generics, there's a concept that's fairly close to a joint type, which > involves the ampersand (&) character: > > public class Foo {} > > However, in the proposed (draft v0.1) closure syntax, | is used to make a > "joint type" for exceptions: > > #int(int, int)(throws IOException|SQLException) x; > > While the exception list there isn't really a single (joint) type, there are > some conceptual similarities, and it makes a lot more sense to say that a > method throws both IOException and SQLException, than to say it throws > either IOException or SQLException (checked exceptions are interesting on > the type level to track what you need to catch, and if a method can throw > either, then you need to catch both in order to satisfy javac, hence, it > makes more sense to consider this a joint type instead of a disjoint type). If the method throws an exception and you catch it and assign to a variable. What type does that variable have to be? If variable is of type "IOException|SQLException" that means it can hold a reference to an object that is either an IOException or a SQLException (or both, but such object is impossible in java). This is different from variable of type Exception which can hold objects of other types too. If variable is of type "Number&Comparable" that means it can hold an object which is a Number and a Comparable at the same time. It can not hold an object which is a Number but isn't a Comparable and it can not hold an object that is a Comparable but isn't a Number. > NB: Is there any reason why it can't mirror method declarations and use > commas? If throws declaration in function type is not enclosed in parens it is ambiguous: FunctionType: '#' ReturnType '(' ParameterTypes_opt ')' Throws_opt Throws: throws ExceptionTypes ExceptionTypes: ExceptionType ExceptionType ',' ExceptionTypes For example: #int( #void() throws IOException, SQLException ) Is this a function type returning int and taking 2 parameters of type "#void() throws IOException" and "SQLException" or is this a function type returning int ant taking 1 parameter of type "#void() throws IOException, SQLException" Regards, Peter From Maurizio.Cimadamore at Sun.COM Tue Feb 2 06:34:13 2010 From: Maurizio.Cimadamore at Sun.COM (Maurizio Cimadamore) Date: Tue, 02 Feb 2010 14:34:13 +0000 Subject: Joint types inconsistency: Suggesting we use & instead of | for exceptions. In-Reply-To: <560fb5ed1002020553x191d064aw1a7239338674a821@mail.gmail.com> References: <560fb5ed1002020553x191d064aw1a7239338674a821@mail.gmail.com> Message-ID: <4B6837E5.1030702@sun.com> Reinier Zwitserloot wrote: > In generics, there's a concept that's fairly close to a joint type, which > involves the ampersand (&) character: > > public class Foo {} > > However, in the proposed (draft v0.1) closure syntax, | is used to make a > "joint type" for exceptions: > > #int(int, int)(throws IOException|SQLException) x; > > While the exception list there isn't really a single (joint) type, there are > some conceptual similarities, and it makes a lot more sense to say that a > method throws both IOException and SQLException, than to say it throws > either IOException or SQLException (checked exceptions are interesting on > the type level to track what you need to catch, and if a method can throw > either, then you need to catch both in order to satisfy javac, hence, it > makes more sense to consider this a joint type instead of a disjoint type). > > I think the semantics of T1 & T2 is rather different from the one of T1 | T2: *) T1 & T2 is a type T which is a subtype of both T1 and T2 (note that this type does not even exists in the case both T1 and T2 are classtypes, as in your example) *) T1 | T2 is some type T which is *either* T1 or T2 The only similarity between the two types is that, from a type-checking perspective, T1 | T2 can be viewed as a type which has the same members as T1 & T2. But this is, an implementation aspect and I don't think we should base our syntax on how we are going to *implement* a given feature. Maurizio > I therefore suggest the syntax is changed to: > > #int(int, int)(throws IOException & SQLException) x; > > the side benefit of a bar being hard to differentiate from an uppercase i or > a lowercase L in many fonts whereas the ampersand is always distinctive is > extra gravy. &, of course, invokes thoughts of 'and', whereas the bar > invokes thoughts of 'or', given that they are used in the and or op > operators in java. > > NB: Is there any reason why it can't mirror method declarations and use > commas? > > --Reinier Zwitserloot > > From reinier at zwitserloot.com Tue Feb 2 06:45:15 2010 From: reinier at zwitserloot.com (Reinier Zwitserloot) Date: Tue, 2 Feb 2010 15:45:15 +0100 Subject: Joint types inconsistency: Suggesting we use & instead of | for exceptions. In-Reply-To: <201002021527.28618.peter.levart@marand.si> References: <560fb5ed1002020553x191d064aw1a7239338674a821@mail.gmail.com> <201002021527.28618.peter.levart@marand.si> Message-ID: <560fb5ed1002020645j6598db54y28d52e1cec6b06e3@mail.gmail.com> Well, without parens it would indeed be ambiguous, but as far as I can tell in the proposal, they are mandatory. Interesting dilemma - making parens mandatory is somewhat odd as they certainly aren't in method declarations and it makes the closure syntax rather paren heavy, but on the other hand without parens you're forced to use | or & instead of the commas that throws clauses on method declarations use. So, the idea, then, is that parens are optional, hence the bar instead of a comma? Point taken that from the perspective of those _catching_ the exception, exception types feel more 'or' than 'and', but I stick to the notion that exception types feels more 'and' like to me when thinking about what your method can throw (I declare at this point that I may throw both type A and type B in addition to RuntimeException and Error), and pipe is still somewhat unreadable. But, my original concerns about consistency have mostly been assuaged. My personal preference is leaning towards mandatory parens and commas. The reason for this: If a user misunderstands and gets this wrong*, then with the mandatory parens all the common mistakes will be trivially caught by the parser, and thus result in a clear and immediate error message - throws is a keyword and is trivially illegal in most places, except in a parenthesized throws clause on a function type. The commas can then be kept which is consistent with how 'throws' clauses have to be written where all java programmers know them from: method declarations. *) I rate this as likely; throwing multiple exception types is not something you type every day, and I can easily imagine hordes of java programmers will not be using closures all that often either, meaning they may only write a multi-throws type once a quarter or less, easily rare enough to forget the exact syntax. --Reinier Zwitserloot Need to receive donations via the web? Check https://tipit.to/ On Tue, Feb 2, 2010 at 3:27 PM, Peter Levart wrote: > On Tuesday 02 February 2010 14:53:22 Reinier Zwitserloot wrote: > > In generics, there's a concept that's fairly close to a joint type, which > > involves the ampersand (&) character: > > > > public class Foo {} > > > > However, in the proposed (draft v0.1) closure syntax, | is used to make a > > "joint type" for exceptions: > > > > #int(int, int)(throws IOException|SQLException) x; > > > > While the exception list there isn't really a single (joint) type, there > are > > some conceptual similarities, and it makes a lot more sense to say that a > > method throws both IOException and SQLException, than to say it throws > > either IOException or SQLException (checked exceptions are interesting on > > the type level to track what you need to catch, and if a method can throw > > either, then you need to catch both in order to satisfy javac, hence, it > > makes more sense to consider this a joint type instead of a disjoint > type). > > If the method throws an exception and you catch it and assign to a > variable. What type does that variable have to be? > > If variable is of type "IOException|SQLException" that means it can hold a > reference to an object that is either an IOException or a SQLException (or > both, but such object is impossible in java). This is different from > variable of type Exception which can hold objects of other types too. > > If variable is of type "Number&Comparable" that means it can hold an object > which is a Number and a Comparable at the same time. It can not hold an > object which is a Number but isn't a Comparable and it can not hold an > object that is a Comparable but isn't a Number. > > > NB: Is there any reason why it can't mirror method declarations and use > > commas? > > If throws declaration in function type is not enclosed in parens it is > ambiguous: > > FunctionType: > '#' ReturnType '(' ParameterTypes_opt ')' Throws_opt > > Throws: > throws ExceptionTypes > > ExceptionTypes: > ExceptionType > ExceptionType ',' ExceptionTypes > > > For example: > > #int( #void() throws IOException, SQLException ) > > Is this a function type returning int and taking 2 parameters of type > "#void() throws IOException" and "SQLException" or > is this a function type returning int ant taking 1 parameter of type > "#void() throws IOException, SQLException" > > > Regards, Peter > From schulz at the-loom.de Tue Feb 2 07:10:46 2010 From: schulz at the-loom.de (Stefan Schulz) Date: Tue, 02 Feb 2010 16:10:46 +0100 Subject: Joint types inconsistency: Suggesting we use & instead of | for exceptions. In-Reply-To: <560fb5ed1002020645j6598db54y28d52e1cec6b06e3@mail.gmail.com> References: <560fb5ed1002020553x191d064aw1a7239338674a821@mail.gmail.com> <201002021527.28618.peter.levart@marand.si> <560fb5ed1002020645j6598db54y28d52e1cec6b06e3@mail.gmail.com> Message-ID: <4B684076.1070309@the-loom.de> Am 02.02.2010 15:45, schrieb Reinier Zwitserloot: > Well, without parens it would indeed be ambiguous, but as far as I can tell > in the proposal, they are mandatory. Interesting dilemma - making parens > mandatory is somewhat odd as they certainly aren't in method declarations > and it makes the closure syntax rather paren heavy, but on the other hand > without parens you're forced to use | or& instead of the commas that throws > clauses on method declarations use. That's why I suggested thinking about a different syntax for function types. Another reason is the difficulty with higher-order functions, which might result in a hard to spot cascade of # and parens. To repeat my (naive) suggestion, using brackets to identify a function type: FunctionType: '[' ResultType '(' TypeList_opt ')' Throws_opt ']' e.g.: [[int(int,int)]() throws NoSuchOperation] operations = ...; Hence, the part inside the brackets looks like a method signature but is in a closed format. As brackets are in use already and planned for collection literals, they might need context-sensitive parsing by a compiler. Stefan From jjb at google.com Tue Feb 2 10:17:03 2010 From: jjb at google.com (Joshua Bloch) Date: Tue, 2 Feb 2010 10:17:03 -0800 Subject: What is the meaning of this? In-Reply-To: References: <17b2302a1001281119t4ea1c34cya8a05e59864b407@mail.gmail.com> <201001282246.44590.peter.levart@gmail.com> <17b2302a1001281619i72676ed1p954b010b1a23a614@mail.gmail.com> Message-ID: <17b2302a1002021017l76bcad7cw21cd864d8dd810a5@mail.gmail.com> Zdenek, Hi. I'm back from vacation and catching up. While I believe there's something to what you say, it's not so much that I view closures as "a concise notation of anonymous classes"; I view them as a concise notation for function objects. But they *are* objects, with identities and classes. There *are *primitives, which lack an identity distinct from their value, but the objects returned by evaluating lambda expressions are not primitive values. I believe the notion of "transparency" that some are arguing for conflicts with basic design principles of Java in a way that's likely to cause trouble. It glosses over the identity of a lambda without eliminating it entirely. Josh On Thu, Jan 28, 2010 at 10:56 PM, Zdenek Tronicek wrote: > Joshua Bloch napsal(a): > > More importantly, evaluating this in the enclosing context is > > perverse in Java. In every context where this is legal today, it refers > to > > the innermost instance. It would be inconsistent to do otherwise for > > lambdas, and we would need a very good reason to knowingly create this > > inconsistency. I haven't heard any such reason. There is, however, a real > > disadvantage to making this refer to the enclosing instance: people will > > streamline existing code that uses anonymous classes, and could easily > end > > up with code that doesn't compile or (worse) compiles but misbehaves > > because > > the meaning of this would be different in the new (lambda) and old > > (anonymous class) code. > > There is a substantial difference between your view and view of > "transparency guys". You view closures as a concise notation of anonymous > classes and then you, naturally, expect that 'this' refers to the instance > of such class. "Transparency guys" view closures as blocks of code or > functions. Then, 'this' should refer to the enclosing instance. > So, the discussion is not about 'this', but about what the closures will > be. > > Z. > -- > Zdenek Tronicek > FIT CTU in Prague > > From John.Rose at Sun.COM Tue Feb 2 10:25:20 2010 From: John.Rose at Sun.COM (John Rose) Date: Tue, 02 Feb 2010 10:25:20 -0800 Subject: Preparing for the 0.2 draft In-Reply-To: <4B68129C.5000008@cs.oswego.edu> References: <4B62A2CE.7080300@sun.com> <4B67427D.9080608@sun.com> <4B677D4D.4030408@cs.oswego.edu> <15e8b9d21002012056t73d41a56we72d0d7de4f013c4@mail.gmail.com> <4B68129C.5000008@cs.oswego.edu> Message-ID: <2EE62E6C-30B8-4A36-8B51-34994033F4F0@sun.com> On Feb 2, 2010, at 3:55 AM, Doug Lea wrote: > those few parallel languages that include some form > of lambda/closure (including X10 and Fortress) do not > hoist scopes. And, at present, Java hoists scopes (into inner classes) in a non-racy manner. This is a direct and intentional result of the 'must be final' rule. Alex is not yet proposing to weaken that rule, except to add 'as if' to the finality requirement. If there's a problem, it is with the basic Java object model, which does not externally distinguish "pure" Java objects from normal ones. Sure, you can define pure ones by using blank finals, etc. But you cannot make an API that only accepts pure objects. I think this problem should be solved, but it's not specifically a lambda-dev problem. Are you suggesting that lambdas should wait for a solution to the problem of checking object purity? That seems impractical. Maybe there is some compromise measure to be taken, to "discourage" use of impure objects? Perhaps we can have an advisory annotation that "certifies" objects as safe for parallel processing, and do some checks to warn when passing uncertified references (directly or indirectly) to a parallel loop? In any case, scope capture per se is a red herring. Any bulk processing or FJ API is going to have to pass request parameters into the little bits of code that actually run in parallel. If *any* of those parameters are general Java references, impure objects can and will introduce raciness. But (until the 'must be final' rule is relaxed) scope capture per se will *never* introduce raciness. -- John From scolebourne at joda.org Tue Feb 2 10:38:33 2010 From: scolebourne at joda.org (Stephen Colebourne) Date: Tue, 2 Feb 2010 18:38:33 +0000 Subject: Preparing for the 0.2 draft In-Reply-To: <2EE62E6C-30B8-4A36-8B51-34994033F4F0@sun.com> References: <4B62A2CE.7080300@sun.com> <4B67427D.9080608@sun.com> <4B677D4D.4030408@cs.oswego.edu> <15e8b9d21002012056t73d41a56we72d0d7de4f013c4@mail.gmail.com> <4B68129C.5000008@cs.oswego.edu> <2EE62E6C-30B8-4A36-8B51-34994033F4F0@sun.com> Message-ID: <4b4f45e01002021038s2e08b137y1a5118a223045415@mail.gmail.com> On 2 February 2010 18:25, John Rose wrote: > And, at present, Java hoists scopes (into inner classes) in a non-racy manner. ?This is a direct and intentional result of the 'must be final' rule. I disagree that this is the case from the developers POV. A developer can take a mutable object, declare it as a final variable, and use it freely within the inner class. And if a developer needs to get around the 'final' restriction, they will simply create a single entry array to do so. In other words, these problems exist today, and we're not making them worse here. Stephen From neal at gafter.com Tue Feb 2 10:40:39 2010 From: neal at gafter.com (Neal Gafter) Date: Tue, 2 Feb 2010 10:40:39 -0800 Subject: What is the meaning of this? In-Reply-To: <17b2302a1002021017l76bcad7cw21cd864d8dd810a5@mail.gmail.com> References: <17b2302a1001281119t4ea1c34cya8a05e59864b407@mail.gmail.com> <201001282246.44590.peter.levart@gmail.com> <17b2302a1001281619i72676ed1p954b010b1a23a614@mail.gmail.com> <17b2302a1002021017l76bcad7cw21cd864d8dd810a5@mail.gmail.com> Message-ID: <15e8b9d21002021040s7afefa46id70110e3ba944c14@mail.gmail.com> I think Zdenek had it exactly right. A lambda expression is not an object creation expression. The function is not declared within the scope of some object type. A lambda is written as an independent block of code, and should be scoped accordingly. Trying to make a lambda act the same as an anonymous class creation expression smacks of cargo-cult language design. Cheers, Neal On Tue, Feb 2, 2010 at 10:17 AM, Joshua Bloch wrote: > Zdenek, > > Hi. I'm back from vacation and catching up. While I believe there's > something to what you say, it's not so much that I view closures as "a > concise notation of anonymous classes"; I view them as a concise notation > for function objects. But they *are* objects, with identities and classes. > There *are *primitives, which lack an identity distinct from their value, > but the objects returned by evaluating lambda expressions are not primitive > values. I believe the notion of "transparency" that some are arguing for > conflicts with basic design principles of Java in a way that's likely to > cause trouble. It glosses over the identity of a lambda without eliminating > it entirely. > > Josh > > On Thu, Jan 28, 2010 at 10:56 PM, Zdenek Tronicek >wrote: > > > Joshua Bloch napsal(a): > > > More importantly, evaluating this in the enclosing context is > > > perverse in Java. In every context where this is legal today, it refers > > to > > > the innermost instance. It would be inconsistent to do otherwise for > > > lambdas, and we would need a very good reason to knowingly create this > > > inconsistency. I haven't heard any such reason. There is, however, a > real > > > disadvantage to making this refer to the enclosing instance: people > will > > > streamline existing code that uses anonymous classes, and could easily > > end > > > up with code that doesn't compile or (worse) compiles but misbehaves > > > because > > > the meaning of this would be different in the new (lambda) and old > > > (anonymous class) code. > > > > There is a substantial difference between your view and view of > > "transparency guys". You view closures as a concise notation of anonymous > > classes and then you, naturally, expect that 'this' refers to the > instance > > of such class. "Transparency guys" view closures as blocks of code or > > functions. Then, 'this' should refer to the enclosing instance. > > So, the discussion is not about 'this', but about what the closures will > > be. > > > > Z. > > -- > > Zdenek Tronicek > > FIT CTU in Prague > > > > > > From John.Rose at Sun.COM Tue Feb 2 10:41:46 2010 From: John.Rose at Sun.COM (John Rose) Date: Tue, 02 Feb 2010 10:41:46 -0800 Subject: Preparing for the 0.2 draft In-Reply-To: <4b4f45e01002021038s2e08b137y1a5118a223045415@mail.gmail.com> References: <4B62A2CE.7080300@sun.com> <4B67427D.9080608@sun.com> <4B677D4D.4030408@cs.oswego.edu> <15e8b9d21002012056t73d41a56we72d0d7de4f013c4@mail.gmail.com> <4B68129C.5000008@cs.oswego.edu> <2EE62E6C-30B8-4A36-8B51-34994033F4F0@sun.com> <4b4f45e01002021038s2e08b137y1a5118a223045415@mail.gmail.com> Message-ID: On Feb 2, 2010, at 10:38 AM, Stephen Colebourne wrote: > A developer can take a mutable object, declare it as a final variable, > and use it freely within the inner class. That's what the latter part of my message tried to convey (plus a straw-man suggestion for addressing it). -- John From forax at univ-mlv.fr Tue Feb 2 10:46:03 2010 From: forax at univ-mlv.fr (=?ISO-8859-1?Q?R=E9mi_Forax?=) Date: Tue, 02 Feb 2010 19:46:03 +0100 Subject: Preparing for the 0.2 draft In-Reply-To: <2EE62E6C-30B8-4A36-8B51-34994033F4F0@sun.com> References: <4B62A2CE.7080300@sun.com> <4B67427D.9080608@sun.com> <4B677D4D.4030408@cs.oswego.edu> <15e8b9d21002012056t73d41a56we72d0d7de4f013c4@mail.gmail.com> <4B68129C.5000008@cs.oswego.edu> <2EE62E6C-30B8-4A36-8B51-34994033F4F0@sun.com> Message-ID: <4B6872EB.7000703@univ-mlv.fr> Le 02/02/2010 19:25, John Rose a ?crit : > On Feb 2, 2010, at 3:55 AM, Doug Lea wrote: > > >> those few parallel languages that include some form >> of lambda/closure (including X10 and Fortress) do not >> hoist scopes. >> > And, at present, Java hoists scopes (into inner classes) in a non-racy manner. This is a direct and intentional result of the 'must be final' rule. Alex is not yet proposing to weaken that rule, except to add 'as if' to the finality requirement. > > If there's a problem, it is with the basic Java object model, which does not externally distinguish "pure" Java objects from normal ones. Sure, you can define pure ones by using blank finals, etc. But you cannot make an API that only accepts pure objects. I think this problem should be solved, but it's not specifically a lambda-dev problem. > > Are you suggesting that lambdas should wait for a solution to the problem of checking object purity? That seems impractical. Maybe there is some compromise measure to be taken, to "discourage" use of impure objects? Perhaps we can have an advisory annotation that "certifies" objects as safe for parallel processing, and do some checks to warn when passing uncertified references (directly or indirectly) to a parallel loop? > > In any case, scope capture per se is a red herring. Any bulk processing or FJ API is going to have to pass request parameters into the little bits of code that actually run in parallel. If *any* of those parameters are general Java references, impure objects can and will introduce raciness. But (until the 'must be final' rule is relaxed) scope capture per se will *never* introduce raciness. > Not *never*. It make racy code less frequent because it's more verbose to write. But remember that you can always pass an array of one element which is the frequent workaround used. > -- John > > R?mi From forax at univ-mlv.fr Tue Feb 2 10:48:15 2010 From: forax at univ-mlv.fr (=?ISO-8859-1?Q?R=E9mi_Forax?=) Date: Tue, 02 Feb 2010 19:48:15 +0100 Subject: What is the meaning of this? In-Reply-To: <15e8b9d21002021040s7afefa46id70110e3ba944c14@mail.gmail.com> References: <17b2302a1001281119t4ea1c34cya8a05e59864b407@mail.gmail.com> <201001282246.44590.peter.levart@gmail.com> <17b2302a1001281619i72676ed1p954b010b1a23a614@mail.gmail.com> <17b2302a1002021017l76bcad7cw21cd864d8dd810a5@mail.gmail.com> <15e8b9d21002021040s7afefa46id70110e3ba944c14@mail.gmail.com> Message-ID: <4B68736F.7030000@univ-mlv.fr> Le 02/02/2010 19:40, Neal Gafter a ?crit : > I think Zdenek had it exactly right. > > A lambda expression is not an object creation expression. The function is > not declared within the scope of some object type. A lambda is written as > an independent block of code, and should be scoped accordingly. Trying to > make a lambda act the same as an anonymous class creation expression smacks > of cargo-cult language design. > I agree. Lambda is an anonymous function. > Cheers, > Neal > regards, R?mi From grev at miginfocom.com Tue Feb 2 10:55:56 2010 From: grev at miginfocom.com (Mikael Grev) Date: Tue, 2 Feb 2010 19:55:56 +0100 Subject: What is the meaning of this? In-Reply-To: <4B68736F.7030000@univ-mlv.fr> References: <17b2302a1001281119t4ea1c34cya8a05e59864b407@mail.gmail.com> <201001282246.44590.peter.levart@gmail.com> <17b2302a1001281619i72676ed1p954b010b1a23a614@mail.gmail.com> <17b2302a1002021017l76bcad7cw21cd864d8dd810a5@mail.gmail.com> <15e8b9d21002021040s7afefa46id70110e3ba944c14@mail.gmail.com> <4B68736F.7030000@univ-mlv.fr> Message-ID: <08B2E113-34CF-4882-9C34-C157D8B2CEA7@miginfocom.com> I must, slightly reluctantly due to historic reasons, agree as well. No matter how things are turned lambdas is something new and the object implementation should not shine through if not needed or shown advantageous. Also, I have always thought declaring a variable as 'volatile' would've been a better solution than forcing it to be final since so many go around that anyway. I know I do sometimes, and I just feel dirty when I do, for no good reason since I know what I'm doing (most of the times anyway. Can't think of a bug introduced that way). Cheers, Mikael On Feb 2, 2010, at 19:48 PM, R?mi Forax wrote: > Le 02/02/2010 19:40, Neal Gafter a ?crit : >> I think Zdenek had it exactly right. >> >> A lambda expression is not an object creation expression. The function is >> not declared within the scope of some object type. A lambda is written as >> an independent block of code, and should be scoped accordingly. Trying to >> make a lambda act the same as an anonymous class creation expression smacks >> of cargo-cult language design. >> > > I agree. Lambda is an anonymous function. > >> Cheers, >> Neal >> > > regards, > R?mi From mthornton at optrak.co.uk Tue Feb 2 11:38:09 2010 From: mthornton at optrak.co.uk (Mark Thornton) Date: Tue, 02 Feb 2010 19:38:09 +0000 Subject: Preparing for the 0.2 draft In-Reply-To: <4B6872EB.7000703@univ-mlv.fr> References: <4B62A2CE.7080300@sun.com> <4B67427D.9080608@sun.com> <4B677D4D.4030408@cs.oswego.edu> <15e8b9d21002012056t73d41a56we72d0d7de4f013c4@mail.gmail.com> <4B68129C.5000008@cs.oswego.edu> <2EE62E6C-30B8-4A36-8B51-34994033F4F0@sun.com> <4B6872EB.7000703@univ-mlv.fr> Message-ID: <4B687F21.1080803@optrak.co.uk> R?mi Forax wrote: > Not *never*. It make racy code less frequent because it's more verbose > to write. > But remember that you can always pass an array of one element which is > the frequent workaround used. > > But at least that workaround looks evil. Mark Thornton From tronicek at fit.cvut.cz Tue Feb 2 12:18:15 2010 From: tronicek at fit.cvut.cz (Zdenek Tronicek) Date: Tue, 2 Feb 2010 21:18:15 +0100 Subject: What is the meaning of this? In-Reply-To: <17b2302a1002021017l76bcad7cw21cd864d8dd810a5@mail.gmail.com> References: <17b2302a1001281119t4ea1c34cya8a05e59864b407@mail.gmail.com> <201001282246.44590.peter.levart@gmail.com> <17b2302a1001281619i72676ed1p954b010b1a23a614@mail.gmail.com> <17b2302a1002021017l76bcad7cw21cd864d8dd810a5@mail.gmail.com> Message-ID: <38918b71bf767c4b08727a50d562b6d8.squirrel@imap.fit.cvut.cz> Joshua Bloch napsal(a): > I believe the notion of "transparency" that some are arguing for > conflicts with basic design principles of Java in a way that's likely to > cause trouble. It glosses over the identity of a lambda without > eliminating it entirely. This interests me. Could you eventually elaborate on it? Z. -- Zdenek Tronicek FIT CTU in Prague > > On Thu, Jan 28, 2010 at 10:56 PM, Zdenek Tronicek > wrote: > >> Joshua Bloch napsal(a): >> > More importantly, evaluating this in the enclosing context is >> > perverse in Java. In every context where this is legal today, it >> refers >> to >> > the innermost instance. It would be inconsistent to do otherwise for >> > lambdas, and we would need a very good reason to knowingly create this >> > inconsistency. I haven't heard any such reason. There is, however, a >> real >> > disadvantage to making this refer to the enclosing instance: people >> will >> > streamline existing code that uses anonymous classes, and could easily >> end >> > up with code that doesn't compile or (worse) compiles but misbehaves >> > because >> > the meaning of this would be different in the new (lambda) and old >> > (anonymous class) code. >> >> There is a substantial difference between your view and view of >> "transparency guys". You view closures as a concise notation of >> anonymous >> classes and then you, naturally, expect that 'this' refers to the >> instance >> of such class. "Transparency guys" view closures as blocks of code or >> functions. Then, 'this' should refer to the enclosing instance. >> So, the discussion is not about 'this', but about what the closures will >> be. >> >> Z. >> -- >> Zdenek Tronicek >> FIT CTU in Prague >> >> > From opinali at gmail.com Tue Feb 2 12:24:16 2010 From: opinali at gmail.com (Osvaldo Doederlein) Date: Tue, 2 Feb 2010 18:24:16 -0200 Subject: What is the meaning of this? In-Reply-To: <08B2E113-34CF-4882-9C34-C157D8B2CEA7@miginfocom.com> References: <17b2302a1001281119t4ea1c34cya8a05e59864b407@mail.gmail.com> <201001282246.44590.peter.levart@gmail.com> <17b2302a1001281619i72676ed1p954b010b1a23a614@mail.gmail.com> <17b2302a1002021017l76bcad7cw21cd864d8dd810a5@mail.gmail.com> <15e8b9d21002021040s7afefa46id70110e3ba944c14@mail.gmail.com> <4B68736F.7030000@univ-mlv.fr> <08B2E113-34CF-4882-9C34-C157D8B2CEA7@miginfocom.com> Message-ID: I agree too (with Mikael/R?mi/Neal). And I add that, it worries me this (apparent) thinking of "we cannot add new concepts to the language" - if this is not Josh's position, it boils down to that in practice: i.e., in Java there are only methods-inside-objects (or classes for statics), even inner classes are very explicitly methods-inside-objects, so any new construct to contain code must be a method-inside-object... we can't have a fundamentally new construct, like a function/closure/lambda. We can only have something that _looks like_ a function/closure/lambda but is firmly rooted (both syntactically and semantically) in the method-of-object concept, down to backwards-compatible behavior of 'this', identity / memory allocation, etc. With such restrictions, Java is doomed to evolve only in the most cumbersome way possible, every new feature always paying enormous tradeoffs not only to backwards compatibility (which is already often a big hit - and I agree that cannot be avoided), but also to the backwards thinking - let's not invent any really new concepts because most developers are idiots who can't learn that a function is something different and will certainly write all sorts of evil code that will fail or not be portable... I say, let's design functions as well as we can (ok as well ss you can - I 2010/2/2 Mikael Grev > I must, slightly reluctantly due to historic reasons, agree as well. No > matter how things are turned lambdas is something new and the object > implementation should not shine through if not needed or shown advantageous. > > Also, I have always thought declaring a variable as 'volatile' would've > been a better solution than forcing it to be final since so many go around > that anyway. I know I do sometimes, and I just feel dirty when I do, for no > good reason since I know what I'm doing (most of the times anyway. Can't > think of a bug introduced that way). > > Cheers, > Mikael > > On Feb 2, 2010, at 19:48 PM, R?mi Forax wrote: > > > Le 02/02/2010 19:40, Neal Gafter a ?crit : > >> I think Zdenek had it exactly right. > >> > >> A lambda expression is not an object creation expression. The function > is > >> not declared within the scope of some object type. A lambda is written > as > >> an independent block of code, and should be scoped accordingly. Trying > to > >> make a lambda act the same as an anonymous class creation expression > smacks > >> of cargo-cult language design. > >> > > > > I agree. Lambda is an anonymous function. > > > >> Cheers, > >> Neal > >> > > > > regards, > > R?mi > > > From opinali at gmail.com Tue Feb 2 12:28:55 2010 From: opinali at gmail.com (Osvaldo Doederlein) Date: Tue, 2 Feb 2010 18:28:55 -0200 Subject: What is the meaning of this? In-Reply-To: References: <17b2302a1001281119t4ea1c34cya8a05e59864b407@mail.gmail.com> <201001282246.44590.peter.levart@gmail.com> <17b2302a1001281619i72676ed1p954b010b1a23a614@mail.gmail.com> <17b2302a1002021017l76bcad7cw21cd864d8dd810a5@mail.gmail.com> <15e8b9d21002021040s7afefa46id70110e3ba944c14@mail.gmail.com> <4B68736F.7030000@univ-mlv.fr> <08B2E113-34CF-4882-9C34-C157D8B2CEA7@miginfocom.com> Message-ID: Oops, bad finger. To shortly continue the last email: design functions as well as you can, only with the strictly necessary tradeoffs for compatibility and legacy access (e.g. conversion / SAM types), but not any other compromise. A+ Osvaldo 2010/2/2 Osvaldo Doederlein > I agree too (with Mikael/R?mi/Neal). And I add that, it worries me this > (apparent) thinking of "we cannot add new concepts to the language" - if > this is not Josh's position, it boils down to that in practice: i.e., in > Java there are only methods-inside-objects (or classes for statics), even > inner classes are very explicitly methods-inside-objects, so any new > construct to contain code must be a method-inside-object... we can't have a > fundamentally new construct, like a function/closure/lambda. We can only > have something that _looks like_ a function/closure/lambda but is firmly > rooted (both syntactically and semantically) in the method-of-object > concept, down to backwards-compatible behavior of 'this', identity / memory > allocation, etc. > > With such restrictions, Java is doomed to evolve only in the most > cumbersome way possible, every new feature always paying enormous tradeoffs > not only to backwards compatibility (which is already often a big hit - and > I agree that cannot be avoided), but also to the backwards thinking - let's > not invent any really new concepts because most developers are idiots who > can't learn that a function is something different and will certainly write > all sorts of evil code that will fail or not be portable... > > I say, let's design functions as well as we can (ok as well ss you can - I > > > 2010/2/2 Mikael Grev > > I must, slightly reluctantly due to historic reasons, agree as well. No >> matter how things are turned lambdas is something new and the object >> implementation should not shine through if not needed or shown advantageous. >> >> Also, I have always thought declaring a variable as 'volatile' would've >> been a better solution than forcing it to be final since so many go around >> that anyway. I know I do sometimes, and I just feel dirty when I do, for no >> good reason since I know what I'm doing (most of the times anyway. Can't >> think of a bug introduced that way). >> >> Cheers, >> Mikael >> >> On Feb 2, 2010, at 19:48 PM, R?mi Forax wrote: >> >> > Le 02/02/2010 19:40, Neal Gafter a ?crit : >> >> I think Zdenek had it exactly right. >> >> >> >> A lambda expression is not an object creation expression. The function >> is >> >> not declared within the scope of some object type. A lambda is written >> as >> >> an independent block of code, and should be scoped accordingly. Trying >> to >> >> make a lambda act the same as an anonymous class creation expression >> smacks >> >> of cargo-cult language design. >> >> >> > >> > I agree. Lambda is an anonymous function. >> > >> >> Cheers, >> >> Neal >> >> >> > >> > regards, >> > R?mi >> >> >> > From jjb at google.com Tue Feb 2 14:08:40 2010 From: jjb at google.com (Joshua Bloch) Date: Tue, 2 Feb 2010 14:08:40 -0800 Subject: What is the meaning of this? In-Reply-To: <15e8b9d21002021040s7afefa46id70110e3ba944c14@mail.gmail.com> References: <17b2302a1001281119t4ea1c34cya8a05e59864b407@mail.gmail.com> <201001282246.44590.peter.levart@gmail.com> <17b2302a1001281619i72676ed1p954b010b1a23a614@mail.gmail.com> <17b2302a1002021017l76bcad7cw21cd864d8dd810a5@mail.gmail.com> <15e8b9d21002021040s7afefa46id70110e3ba944c14@mail.gmail.com> Message-ID: <17b2302a1002021408g5a4805buacc2904bac297876@mail.gmail.com> On Tue, Feb 2, 2010 at 10:40 AM, Neal Gafter wrote: > I think Zdenek had it exactly right. > > A lambda expression is not an object creation expression. This claims is ludicrous on the face of it. Is a lamda an expression? Yes. Does it "create" (i.e., furnish) an object (which may or may not be new)? Yes. So it's an object creation expression. As far as I'm concernect, that's all there is to it. The function is not declared within the scope of some object type. A > lambda is written as an independent block of code, and should be scoped > accordingly. You're creating an instance of an object. It has a type. Not scoping it accordingly is asking for trouble. > Trying to make a lambda act the same as an anonymous class creation > expression smacks of cargo-cult language design. > Doing something because its the way a bunch of other languages did it is cargo cult language-design (in the absence of other good arguments). Trying to match the syntax and semantics of existing constructs in the language you're proposing to extend is a best practice. Unlike cargo-cult design, it's very easy to argue why it's a good idea: it keeps the conceptual surface area of the language from growing--this one's already too big--and it avoids programmer astonishment and errors. It allows people to reason about the new construct using knowledge they acquired from using existing constructs. Josh From jjb at google.com Tue Feb 2 14:19:32 2010 From: jjb at google.com (Joshua Bloch) Date: Tue, 2 Feb 2010 14:19:32 -0800 Subject: What is the meaning of this? In-Reply-To: References: <17b2302a1001281119t4ea1c34cya8a05e59864b407@mail.gmail.com> <201001282246.44590.peter.levart@gmail.com> <17b2302a1001281619i72676ed1p954b010b1a23a614@mail.gmail.com> <17b2302a1002021017l76bcad7cw21cd864d8dd810a5@mail.gmail.com> <15e8b9d21002021040s7afefa46id70110e3ba944c14@mail.gmail.com> <4B68736F.7030000@univ-mlv.fr> <08B2E113-34CF-4882-9C34-C157D8B2CEA7@miginfocom.com> Message-ID: <17b2302a1002021419g7bf08315gd425637b9e7640f4@mail.gmail.com> Osvaldo, Of course we all believe that we should "design functions [lambdas] as well as we can." But we disagree about what constitutes quality. I'm deeply concerned about consistency with existing constructs, and conceptual surface area. Java currently has two main families of types: primitives and object reference types. Object reference types are further subdivided into interface types and class types. Arrays are class types. When we added enums and annotations, they fit conveniently into class types and interface types (respectively). As we add function types and expressions for producing instances of these types (lambdas), we should try as hard as we can to make them fit into the existing type framework of the language. That does not mean that we can't invent any new concepts. Enum types were a new concept, but they meshed well into the existing type framework, and most Java programmers found their use to be comfortable and intuitive. They could pretty much guess what code using enums did without reading a spec or tutorial. That's what we're aiming for. Josh On Tue, Feb 2, 2010 at 12:24 PM, Osvaldo Doederlein wrote: > I agree too (with Mikael/R?mi/Neal). And I add that, it worries me this > (apparent) thinking of "we cannot add new concepts to the language" - if > this is not Josh's position, it boils down to that in practice: i.e., in > Java there are only methods-inside-objects (or classes for statics), even > inner classes are very explicitly methods-inside-objects, so any new > construct to contain code must be a method-inside-object... we can't have a > fundamentally new construct, like a function/closure/lambda. We can only > have something that _looks like_ a function/closure/lambda but is firmly > rooted (both syntactically and semantically) in the method-of-object > concept, down to backwards-compatible behavior of 'this', identity / memory > allocation, etc. > > With such restrictions, Java is doomed to evolve only in the most > cumbersome > way possible, every new feature always paying enormous tradeoffs not only > to > backwards compatibility (which is already often a big hit - and I agree > that > cannot be avoided), but also to the backwards thinking - let's not invent > any really new concepts because most developers are idiots who can't learn > that a function is something different and will certainly write all sorts > of > evil code that will fail or not be portable... > > I say, let's design functions as well as we can (ok as well ss you can - I > > > 2010/2/2 Mikael Grev > > > I must, slightly reluctantly due to historic reasons, agree as well. No > > matter how things are turned lambdas is something new and the object > > implementation should not shine through if not needed or shown > advantageous. > > > > Also, I have always thought declaring a variable as 'volatile' would've > > been a better solution than forcing it to be final since so many go > around > > that anyway. I know I do sometimes, and I just feel dirty when I do, for > no > > good reason since I know what I'm doing (most of the times anyway. Can't > > think of a bug introduced that way). > > > > Cheers, > > Mikael > > > > On Feb 2, 2010, at 19:48 PM, R?mi Forax wrote: > > > > > Le 02/02/2010 19:40, Neal Gafter a ?crit : > > >> I think Zdenek had it exactly right. > > >> > > >> A lambda expression is not an object creation expression. The > function > > is > > >> not declared within the scope of some object type. A lambda is > written > > as > > >> an independent block of code, and should be scoped accordingly. > Trying > > to > > >> make a lambda act the same as an anonymous class creation expression > > smacks > > >> of cargo-cult language design. > > >> > > > > > > I agree. Lambda is an anonymous function. > > > > > >> Cheers, > > >> Neal > > >> > > > > > > regards, > > > R?mi > > > > > > > > From jjb at google.com Tue Feb 2 14:35:01 2010 From: jjb at google.com (Joshua Bloch) Date: Tue, 2 Feb 2010 14:35:01 -0800 Subject: What is the meaning of this? In-Reply-To: <201001311039.55711.peter.levart@gmail.com> References: <17b2302a1001281119t4ea1c34cya8a05e59864b407@mail.gmail.com> <201001311039.55711.peter.levart@gmail.com> Message-ID: <17b2302a1002021435i685ffa00u6184fc26c9c1cbd4@mail.gmail.com> Peter, This is an interesting idea. It certainly provides a way to access the function object from within its definition without polluting the surrounding scope, which is a good thing. Of course the ability to do non-local returns scares the crap out of me. Heck, I feel unclean when I do a labeled break in the language as it currently exists. Josh P.S. Your idea still leaves unanswered the question of what "this" *should *represent in the body of a lambda. On Sun, Jan 31, 2010 at 1:39 AM, Peter Levart wrote: > On Thursday 28 January 2010 20:19:05 Joshua Bloch wrote: > > Entirely banning the use of this in lambada expressions is a very bad > idea. > > It amounts to throwing out the baby with the bath water. If you need to > get > > your hands on the function object created by a lambda expression, but > > this doesn't > > provide it, you're out of luck.There is no general-purpose workaround. > And > > people will be tempted to use workarounds that aren't thread-safe, such > as > > stashing the lambda in a field. > > > > If we leave the debate about what lambda expression represents (an instance > of a class with > anonymous method or a referenceable block of code with formal parameters) > in the background for > a moment and just focus on "the baby" as you describe, then if you need to > get your hands on the > function object created by a lambda expression, the following syntax could > help: > > LambdaExpression: > '#' Identifier_opt '(' FormalParameters_opt ')' Block > '#' Identifier_opt '(' FormalParameters_opt ')' '(' Expression ')' > > Where "Identifier" represents a final local variable valid in the scope of > "Block" or "Expression" > with the function (or SAM) type of a lambda expression. The factorial > example would then read: > > #int(int) factorial = #fact(int i)(i == 0 ? 1 : i * (int) (fact.(i - 1))); > > This would enable referencing outer instances of lambda expressions from > within nested lambda > expressions, a feature not possible with anonymous inner classes today. > > The optional "Identifier" could be used for other purposes: > > break Identifier; // non-value-bearing break from the statement lambda > break Identifier : Expression; // value-bearing break from the > statement lambda > > In addition, if we make lambdas Serializable, the identifier could help > determine the name of the > class (or a method if java.dyn.MethodReference is used for implementation) > generated by the > compiler so that moving lambda expression around within the code doesn't > break serialization. > > Regards, Peter > From neal at gafter.com Tue Feb 2 14:38:17 2010 From: neal at gafter.com (Neal Gafter) Date: Tue, 2 Feb 2010 14:38:17 -0800 Subject: What is the meaning of this? In-Reply-To: <17b2302a1002021408g5a4805buacc2904bac297876@mail.gmail.com> References: <17b2302a1001281119t4ea1c34cya8a05e59864b407@mail.gmail.com> <201001282246.44590.peter.levart@gmail.com> <17b2302a1001281619i72676ed1p954b010b1a23a614@mail.gmail.com> <17b2302a1002021017l76bcad7cw21cd864d8dd810a5@mail.gmail.com> <15e8b9d21002021040s7afefa46id70110e3ba944c14@mail.gmail.com> <17b2302a1002021408g5a4805buacc2904bac297876@mail.gmail.com> Message-ID: <15e8b9d21002021438i7916ffe4q53f983a6f905b4b2@mail.gmail.com> On Tue, Feb 2, 2010 at 2:08 PM, Joshua Bloch wrote: > On Tue, Feb 2, 2010 at 10:40 AM, Neal Gafter wrote: > >> I think Zdenek had it exactly right. >> >> A lambda expression is not an object creation expression. > > > This claims is ludicrous on the face of it. Is a lamda an expression? > Yes. Does it "create" (i.e., furnish) an object (which may or may not be > new)? Yes. So it's an object creation expression. As far as I'm concernect, > that's all there is to it. > By that logic, an expression that is boxed to java.lang.Integer is an object creation expression too, and therefore the expression that computes the value to be boxed (i.e. the internal field's initializing expression) should be scoped in java.lang.Integer (by analogy with field initializer expressions). From Alex.Buckley at Sun.COM Tue Feb 2 14:39:09 2010 From: Alex.Buckley at Sun.COM (Alex Buckley) Date: Tue, 02 Feb 2010 14:39:09 -0800 Subject: What is the meaning of this? In-Reply-To: References: <17b2302a1001281119t4ea1c34cya8a05e59864b407@mail.gmail.com> <201001282246.44590.peter.levart@gmail.com> <17b2302a1001281619i72676ed1p954b010b1a23a614@mail.gmail.com> <17b2302a1002021017l76bcad7cw21cd864d8dd810a5@mail.gmail.com> <15e8b9d21002021040s7afefa46id70110e3ba944c14@mail.gmail.com> <4B68736F.7030000@univ-mlv.fr> <08B2E113-34CF-4882-9C34-C157D8B2CEA7@miginfocom.com> Message-ID: <4B68A98D.1080303@sun.com> -1 for unnecessary hyperbole. ("doomed", "most cumbersome", "idiots") Lambdas introduce plenty of new concepts: function types, lambda conversion, "effectively final" variables, shared variables (possibly), method references (possibly), extension methods (probably). To say nothing of API changes to actually consume and produce lambdas. I am pleased that this list has exposed different opinions and (more importantly) their underlying beliefs. Doubtless the draft spec will take a position that reflects some beliefs and not others, leading to much wailing and gnashing of teeth. That is inevitable. But I will say this: with nine million Java developers, designing something that looks familiar is a feature, not a bug. Alex Osvaldo Doederlein wrote: > I agree too (with Mikael/R?mi/Neal). And I add that, it worries me this > (apparent) thinking of "we cannot add new concepts to the language" - if > this is not Josh's position, it boils down to that in practice: i.e., in > Java there are only methods-inside-objects (or classes for statics), even > inner classes are very explicitly methods-inside-objects, so any new > construct to contain code must be a method-inside-object... we can't have a > fundamentally new construct, like a function/closure/lambda. We can only > have something that _looks like_ a function/closure/lambda but is firmly > rooted (both syntactically and semantically) in the method-of-object > concept, down to backwards-compatible behavior of 'this', identity / memory > allocation, etc. > > With such restrictions, Java is doomed to evolve only in the most cumbersome > way possible, every new feature always paying enormous tradeoffs not only to > backwards compatibility (which is already often a big hit - and I agree that > cannot be avoided), but also to the backwards thinking - let's not invent > any really new concepts because most developers are idiots who can't learn > that a function is something different and will certainly write all sorts of > evil code that will fail or not be portable... > > I say, let's design functions as well as we can (ok as well ss you can - I > > > 2010/2/2 Mikael Grev > >> I must, slightly reluctantly due to historic reasons, agree as well. No >> matter how things are turned lambdas is something new and the object >> implementation should not shine through if not needed or shown advantageous. >> >> Also, I have always thought declaring a variable as 'volatile' would've >> been a better solution than forcing it to be final since so many go around >> that anyway. I know I do sometimes, and I just feel dirty when I do, for no >> good reason since I know what I'm doing (most of the times anyway. Can't >> think of a bug introduced that way). >> >> Cheers, >> Mikael >> >> On Feb 2, 2010, at 19:48 PM, R?mi Forax wrote: >> >>> Le 02/02/2010 19:40, Neal Gafter a ?crit : >>>> I think Zdenek had it exactly right. >>>> >>>> A lambda expression is not an object creation expression. The function >> is >>>> not declared within the scope of some object type. A lambda is written >> as >>>> an independent block of code, and should be scoped accordingly. Trying >> to >>>> make a lambda act the same as an anonymous class creation expression >> smacks >>>> of cargo-cult language design. >>>> >>> I agree. Lambda is an anonymous function. >>> >>>> Cheers, >>>> Neal >>>> >>> regards, >>> R?mi >> >> > From jjb at google.com Tue Feb 2 14:47:17 2010 From: jjb at google.com (Joshua Bloch) Date: Tue, 2 Feb 2010 14:47:17 -0800 Subject: What is the meaning of this? In-Reply-To: <15e8b9d21002021438i7916ffe4q53f983a6f905b4b2@mail.gmail.com> References: <17b2302a1001281119t4ea1c34cya8a05e59864b407@mail.gmail.com> <201001282246.44590.peter.levart@gmail.com> <17b2302a1001281619i72676ed1p954b010b1a23a614@mail.gmail.com> <17b2302a1002021017l76bcad7cw21cd864d8dd810a5@mail.gmail.com> <15e8b9d21002021040s7afefa46id70110e3ba944c14@mail.gmail.com> <17b2302a1002021408g5a4805buacc2904bac297876@mail.gmail.com> <15e8b9d21002021438i7916ffe4q53f983a6f905b4b2@mail.gmail.com> Message-ID: <17b2302a1002021447l4a77836bp5caa5e7e1ccb1348@mail.gmail.com> Neal, This smacks of sophistry. Autoboxing isn't a construct to produce Integer instances; it's a conversion that is invisibly applied to an int-valued expression to produce an Integer. Josh On Tue, Feb 2, 2010 at 2:38 PM, Neal Gafter wrote: > On Tue, Feb 2, 2010 at 2:08 PM, Joshua Bloch wrote: > >> On Tue, Feb 2, 2010 at 10:40 AM, Neal Gafter wrote: >> >>> I think Zdenek had it exactly right. >>> >>> A lambda expression is not an object creation expression. >> >> >> This claims is ludicrous on the face of it. Is a lamda an expression? >> Yes. Does it "create" (i.e., furnish) an object (which may or may not be >> new)? Yes. So it's an object creation expression. As far as I'm concernect, >> that's all there is to it. >> > > By that logic, an expression that is boxed to java.lang.Integer is an > object creation expression too, and therefore the expression that computes > the value to be boxed (i.e. the internal field's initializing expression) > should be scoped in java.lang.Integer (by analogy with field initializer > expressions). > From fatih at coskuns-castle.de Tue Feb 2 15:11:13 2010 From: fatih at coskuns-castle.de (Fatih Coskun) Date: Wed, 03 Feb 2010 00:11:13 +0100 Subject: 2cents from an ordinary Java developer In-Reply-To: References: Message-ID: <4B68B111.9040305@coskuns-castle.de> Hi all, maybe I am not exactly the ordinary Java developer. The ordinary Java developer would not follow these lists, I assume. I am following this list just because some of the most known and popular individuals in the Java world are posting in it regularly. You don't get to read that much from them that often. Anyway, I consider myself very close to an ordinary Java developer. I do code in Java as my everyday work, and I am very close to all the problems and issues an ordinary Java developer encounters every day. I think it is important to hear the opinion of the ordinary developer, because he is member of the largest fraction of Java developers that will have to deal with a new language feature. Regarding the question of whether to introduce lambdas as syntactic sugar or as transparent closure constructs, I am at the side of the transparency-guys. I can't give any academic reason for my opinion. I do know the advantages and disadvantages of both approaches. In my opinion fully transparent closures increase the expressiveness of the Java language, whereas the other approach is nothing more than syntactic sugar. Furthermore, I have never understood how two of the most popular individuals in Java world can have so fundamentally different opinions and argue about this language feature for so many years. I have the greatest respect for both of them, but they should know that the ordinary Java developer has noticed the severeness of their dispute. With my biggest respect, Fatih Coskun From scolebourne at joda.org Tue Feb 2 15:42:03 2010 From: scolebourne at joda.org (Stephen Colebourne) Date: Tue, 2 Feb 2010 23:42:03 +0000 Subject: What is the meaning of this? In-Reply-To: <4B68A98D.1080303@sun.com> References: <17b2302a1001281119t4ea1c34cya8a05e59864b407@mail.gmail.com> <201001282246.44590.peter.levart@gmail.com> <17b2302a1001281619i72676ed1p954b010b1a23a614@mail.gmail.com> <17b2302a1002021017l76bcad7cw21cd864d8dd810a5@mail.gmail.com> <15e8b9d21002021040s7afefa46id70110e3ba944c14@mail.gmail.com> <4B68736F.7030000@univ-mlv.fr> <08B2E113-34CF-4882-9C34-C157D8B2CEA7@miginfocom.com> <4B68A98D.1080303@sun.com> Message-ID: <4b4f45e01002021542g576d430bvd84a2821d5aa6834@mail.gmail.com> On 2 February 2010 22:39, Alex Buckley wrote: > I am pleased that this list has exposed different opinions and (more > importantly) their underlying beliefs. Doubtless the draft spec will > take a position that reflects some beliefs and not others, leading to > much wailing and gnashing of teeth. That is inevitable Exactly. In the end someone (probably you) will take a decision and we will live with it. However, I would appeal to you to explain your reasoning wrt some of the key design choices, notably wrt "this". While I can see both sides of the argument, I came to the conclusion fairly easily that "this" should be lexical (most useful to developers, most expected in actual use) while "return" should be scoped (most useful to developers, most expected in actual use, least buggy, least scary). But what I really want to see are the decision makers motivations ;-) Stephen From jjb at google.com Tue Feb 2 17:42:18 2010 From: jjb at google.com (Joshua Bloch) Date: Tue, 2 Feb 2010 17:42:18 -0800 Subject: Preparing for the 0.2 draft In-Reply-To: <96862318-7DE2-46C1-81F6-6CB5B4B026ED@sun.com> References: <4B62A2CE.7080300@sun.com> <4B67427D.9080608@sun.com> <96862318-7DE2-46C1-81F6-6CB5B4B026ED@sun.com> Message-ID: <17b2302a1002021742j1288f5afu53fb6968193420e9@mail.gmail.com> John, I realize that this is orthogonal to your main point, but I see you've taken to dropping the parens on "expression lambdas", e.g., #(int x) 22+x. I find this terribly hard to read in context. I think it was with good reason that Alex made the parens mandatory. I agree that lambda expressions should not be forced to create new instances. I believe it was a mistake to require (Integer)(128) to create a new instance, and I don't think we should make that mistake twice. As for your main point, I'd like it if the syntax for creating an instance of a Sam type was just as nice as for creating an instance of an ordinary function type. I realize that this may not be easy, but I think it's worth trying. Josh On Mon, Feb 1, 2010 at 11:28 PM, John Rose wrote: > Playing with syntax dial settings between "terse" and "verbose" (aka > "magic" and "tragic"), I just remembered a possible compromise for dealing > with complicated SAM types. In case it hasn't been proposed yet in the > context of closures: > > Given: > class MySAM { MySAM(int arg1, int arg2) {...} abstract int > justOneMethod(int x); } > > Then maybe: > MySAM bar = new MySAM(arg1, arg2) #(int x) 1+x; > > As sugar for: > MySAM bar = new MySAM(arg1, arg2) { int justOneMethod(int x) { return > 1+x; } }; > > Making the 'new' explicit defuses the tricky arguments about when the > object is created, whether the SAM has fields, has a nullary constructor, > has construction side effects, etc. > > -- John > > P.S. For interfaces at least, +1 to keep the implicit assignment > conversion: > > interface iSAM { int justOneMethod(int x); } > Object oof = (iSAM) #(int x) 1+x; // cast conversion implied by implicit > conversion > > And, please be loose about object initialization and identity, when 'new' > is absent, so implementations can do hoisting & caching without breaking > spec.: > > for (int i = 0; i < 2; i++) { > iSAM foo = #(int x) 22+x; // initialization => implicit conversion > if (i == 0) > System.out.println(foo == oof); // must be false > else > System.out.println(foo == oof); // implementation defined result > oof = foo; > } > > > From jjb at google.com Tue Feb 2 17:49:45 2010 From: jjb at google.com (Joshua Bloch) Date: Tue, 2 Feb 2010 17:49:45 -0800 Subject: Preparing for the 0.2 draft In-Reply-To: <4B636050.6010801@univ-mlv.fr> References: <4B62A2CE.7080300@sun.com> <17b2302a1001291205n3b2436a6w3d45952ca1c6d0d@mail.gmail.com> <4B636050.6010801@univ-mlv.fr> Message-ID: <17b2302a1002021749j1e1cc5c1s23122c4a1eee528c@mail.gmail.com> R?mi, On Fri, Jan 29, 2010 at 2:25 PM, R?mi Forax wrote: > > > True, but I haven't heard *any *arguments in favor of the "transparent > this" > > that are in any way applicable to Java. Maybe I'm being dense, but I > believe > > it's cargo cult language design. In Java, I believe it would be a > > gratuitous inconsistency, pure and simple. And I believe it would bite > us > > sooner rather than later. If I'm wrong, I'd love to see some compelling, > > practical examples where having this refer to the enclosing instance is > > useful (in Java). > > > > public class AuthGroup { > private final List userList; > private final int priority; > ... > public List getAuthorizedUsers() { > //filter works like filter in Python > return userList.filter(#(User user) (user.getPriority() > >= this.priority)); > } > } > Unless I'm missing something, that code doesn't require a "this" at all. Just leave out the "this.", and the unqualified named priority will refer to the field that you want. I still haven't seen a compelling example (though I still have some catching up to do on lambda-dev email). Josh Josh From John.Rose at Sun.COM Tue Feb 2 17:53:52 2010 From: John.Rose at Sun.COM (John Rose) Date: Tue, 02 Feb 2010 17:53:52 -0800 Subject: Preparing for the 0.2 draft In-Reply-To: <17b2302a1002021742j1288f5afu53fb6968193420e9@mail.gmail.com> References: <4B62A2CE.7080300@sun.com> <4B67427D.9080608@sun.com> <96862318-7DE2-46C1-81F6-6CB5B4B026ED@sun.com> <17b2302a1002021742j1288f5afu53fb6968193420e9@mail.gmail.com> Message-ID: On Feb 2, 2010, at 5:42 PM, Joshua Bloch wrote: > I think it was with good reason that > Alex made the parens mandatory. Thanks; my bad. I wasn't trying to subvert that part of the spec. > I believe it was a mistake to require (Integer)(128) to create a > new instance, and I don't think we should make that mistake twice. It doesn't! The spec. requires caching in the range [-128,127], but does not disallow caching elsewhere. That's the practical basis for proposing fixnums as a future JVM optimization. (Or maybe I'm missing your point?) > I'd like it if the syntax for creating an instance > of a Sam type was just as nice as for creating an instance of an ordinary > function type. The good old syntax dial has many gradations. The dial setting I picked made sam instantiation "almost as nice but not quite, yet strangely familiar" (compared to a simple lambda) to take explicit control over constructor invocation and instance identity. The most implicit option (assignment conversion) has lots of corner cases with need sorting out. That option should (as I think we agree) allow (though not require) implementations to hoist and/or cache, just as with Integer. Best wishes, -- John From jjb at google.com Tue Feb 2 17:57:49 2010 From: jjb at google.com (Joshua Bloch) Date: Tue, 2 Feb 2010 17:57:49 -0800 Subject: Preparing for the 0.2 draft In-Reply-To: References: <4B62A2CE.7080300@sun.com> <4B67427D.9080608@sun.com> <96862318-7DE2-46C1-81F6-6CB5B4B026ED@sun.com> <17b2302a1002021742j1288f5afu53fb6968193420e9@mail.gmail.com> Message-ID: <17b2302a1002021757l99847eet62363286efd9aee0@mail.gmail.com> John, On Tue, Feb 2, 2010 at 5:53 PM, John Rose wrote: > On Feb 2, 2010, at 5:42 PM, Joshua Bloch wrote: > > > I think it was with good reason that > > Alex made the parens mandatory. > > Thanks; my bad. I wasn't trying to subvert that part of the spec. > > > I believe it was a mistake to require (Integer)(128) to create a > > new instance, and I don't think we should make that mistake twice. > > It doesn't! The spec. requires caching in the range [-128,127], but does > not disallow caching elsewhere. That's the practical basis for proposing > fixnums as a future JVM optimization. (Or maybe I'm missing your point?) > No, my bad. You're right, but still this piece of the spec sucks. It shouldn't have made any caching mandatory. Any code that depends on the caching of [-128,127] is evil. The spec should simply have said that it up to the implementation to decide what to cache. > > > I'd like it if the syntax for creating an instance > > of a Sam type was just as nice as for creating an instance of an ordinary > > function type. > > The good old syntax dial has many gradations. The dial setting I picked > made sam instantiation "almost as nice but not quite, yet strangely > familiar" (compared to a simple lambda) to take explicit control over > constructor invocation and instance identity. > > The most implicit option (assignment conversion) has lots of corner cases > with need sorting out. That option should (as I think we agree) allow > (though not require) implementations to hoist and/or cache, just as with > Integer. Agreed on both counts. But my hope is that we'll figure out a way to do this while retaining the same semantics for SAM types as functions. I haven't figured out the details (or even the rough outlines). Josh From lk at teamten.com Tue Feb 2 21:04:15 2010 From: lk at teamten.com (Lawrence Kesteloot) Date: Tue, 2 Feb 2010 21:04:15 -0800 Subject: 2cents from an ordinary Java developer In-Reply-To: <4B68B111.9040305@coskuns-castle.de> References: <4B68B111.9040305@coskuns-castle.de> Message-ID: <997cab101002022104g5d962201i504d97856bd556a4@mail.gmail.com> Hi Fatih, > In my opinion fully transparent > closures increase the expressiveness of the Java language I think few people would disagree with you. The question is whether this increased expressiveness would be a net win or lose for "ordinary" programmers. Since none of us is in this set, we have to simulate what an ordinary programmer would think when reading such code or writing it, and what existing mental model the programmer would approach the problem with. I wish it were as easy as just asking ordinary programmers, but if I may make an analogy to UI design again, you can do no worse than asking a user to design a user interface. That's because users are not good at simulating themselves or others like them. In fact, even experienced designers are pretty bad at this. That's why they have hallway usability tests. I wish we could do such tests with the various options we're throwing around on this list, but I don't know how to do that in a way that all sides will consider illuminating. > Furthermore, I have never understood how two of the most popular > individuals in Java world can have so fundamentally different opinions Because they both run simulations of ordinary programmers and their simulations end up with different results. (By the way, I think it's great that they have different opinions. I believe the result will be better than if they were in agreement.) I might here channel Neal and say that no simulation is necessary, we already know from Scala that transparent closures are not confusing. To which I would reply that by the time someone is using Scala for serious code involving closures, they are no longer in the set of ordinary programmers. (Apologies to Neal for any incorrect channeling. :-) Lawrence From neal at gafter.com Tue Feb 2 22:35:24 2010 From: neal at gafter.com (Neal Gafter) Date: Tue, 2 Feb 2010 22:35:24 -0800 Subject: 2cents from an ordinary Java developer In-Reply-To: <997cab101002022104g5d962201i504d97856bd556a4@mail.gmail.com> References: <4B68B111.9040305@coskuns-castle.de> <997cab101002022104g5d962201i504d97856bd556a4@mail.gmail.com> Message-ID: <15e8b9d21002022235w2579ef1fgc3d2d9cb57c21a6b@mail.gmail.com> On Tue, Feb 2, 2010 at 9:04 PM, Lawrence Kesteloot wrote: > I might here channel Neal and say that no simulation is necessary, we > already know from Scala that transparent closures are not confusing. > To which I would reply that by the time someone is using Scala for > serious code involving closures, they are no longer in the set of > ordinary programmers. (Apologies to Neal for any incorrect channeling. > :-) The ordinary Java programmers of today are not ordinary programmers from a point of view 5-10 years ago. As our languages change, so does our sense of common and ordinary. From markmahieu at googlemail.com Tue Feb 2 23:36:39 2010 From: markmahieu at googlemail.com (Mark Mahieu) Date: Wed, 3 Feb 2010 07:36:39 +0000 Subject: 2cents from an ordinary Java developer In-Reply-To: <15e8b9d21002022235w2579ef1fgc3d2d9cb57c21a6b@mail.gmail.com> References: <4B68B111.9040305@coskuns-castle.de> <997cab101002022104g5d962201i504d97856bd556a4@mail.gmail.com> <15e8b9d21002022235w2579ef1fgc3d2d9cb57c21a6b@mail.gmail.com> Message-ID: <282D0BDE-AD5A-44EB-9C44-6F177E81A9BC@googlemail.com> On 3 Feb 2010, at 06:35, Neal Gafter wrote: > > The ordinary Java programmers of today are not ordinary programmers > from a point of view 5-10 years ago. Neither is the code they work on. My, how the boilerplate hath grown... > As our languages change, so does > our sense of common and ordinary. > That's very true, and in more ways than one. 5 years ago the team I was working within contained very few programmers with any real experience using a language other than Java. About 2 years ago many of them were spending some of their time in Ruby-land *. As of about a year ago, we have Ruby programmers on the team who have never worked with Java (or they won't admit to having done so, I'm never quite sure). Mark * often their first exposure to lambdas, with "why on earth can't I just write it like that in Java?" being [a polite version of] the typical reaction. From opinali at gmail.com Wed Feb 3 02:11:52 2010 From: opinali at gmail.com (Osvaldo Pinali Doederlein) Date: Wed, 03 Feb 2010 08:11:52 -0200 Subject: What is the meaning of this? In-Reply-To: <4B68A98D.1080303@sun.com> References: <17b2302a1001281119t4ea1c34cya8a05e59864b407@mail.gmail.com> <201001282246.44590.peter.levart@gmail.com> <17b2302a1001281619i72676ed1p954b010b1a23a614@mail.gmail.com> <17b2302a1002021017l76bcad7cw21cd864d8dd810a5@mail.gmail.com> <15e8b9d21002021040s7afefa46id70110e3ba944c14@mail.gmail.com> <4B68736F.7030000@univ-mlv.fr> <08B2E113-34CF-4882-9C34-C157D8B2CEA7@miginfocom.com> <4B68A98D.1080303@sun.com> Message-ID: <4B694BE8.8090907@gmail.com> Sorry if the hyperbole did sound provocative or anything. In the future I will use... only cult, even biblical hyperbole, like "gnashing of teeth". Or I will wait until I have the status for that (I vividly remember my professor Theo D'Hondt in 1999 saying that "inner classes are 'ooorrible'" [fake, subtle Dutch accent for extra effect]). ;-) Ok, maybe I was just a little angry but that's natural after digesting a couple hundred emails and proposals around certain issues, like blocking transparent closure calls due to Java's field/method visibility scoping and the (purism of?) not wanting extra scoping cases or method resolution rules, or this new issue of 'this'. My gut feeling is that the problems caused by such designs would be vanishingly rare in the real world, and the benefit of transparent calls would be a very positive tradeoff. My point is that "common developers" are NOT idiots (oops!). But we believe/fear too much that they are, so we are going to select some syntax, like x.(y), that half the planet will hate like you aptly put. Programmers are flocking in droves to alternative languages, some of them less well designed than Java and full of pitfalls; or much more complex, like the monstrosity of C++ that's still used by millions. We don't have to do SO much baby-sitting. If a powerful, clean lambda proposal will have a few corner cases that allow Josh&Neal write an entire "Java Puzzlers - Special Lambda Edition" volume, so be it . A+ Osvaldo Em 02/02/2010 20:39, Alex Buckley escreveu: > -1 for unnecessary hyperbole. ("doomed", "most cumbersome", "idiots") > > Lambdas introduce plenty of new concepts: function types, lambda > conversion, "effectively final" variables, shared variables (possibly), > method references (possibly), extension methods (probably). To say > nothing of API changes to actually consume and produce lambdas. > > I am pleased that this list has exposed different opinions and (more > importantly) their underlying beliefs. Doubtless the draft spec will > take a position that reflects some beliefs and not others, leading to > much wailing and gnashing of teeth. That is inevitable. But I will say > this: with nine million Java developers, designing something that looks > familiar is a feature, not a bug. > > Alex > > Osvaldo Doederlein wrote: > >> I agree too (with Mikael/R?mi/Neal). And I add that, it worries me this >> (apparent) thinking of "we cannot add new concepts to the language" - if >> this is not Josh's position, it boils down to that in practice: i.e., in >> Java there are only methods-inside-objects (or classes for statics), even >> inner classes are very explicitly methods-inside-objects, so any new >> construct to contain code must be a method-inside-object... we can't have a >> fundamentally new construct, like a function/closure/lambda. We can only >> have something that _looks like_ a function/closure/lambda but is firmly >> rooted (both syntactically and semantically) in the method-of-object >> concept, down to backwards-compatible behavior of 'this', identity / memory >> allocation, etc. >> >> With such restrictions, Java is doomed to evolve only in the most cumbersome >> way possible, every new feature always paying enormous tradeoffs not only to >> backwards compatibility (which is already often a big hit - and I agree that >> cannot be avoided), but also to the backwards thinking - let's not invent >> any really new concepts because most developers are idiots who can't learn >> that a function is something different and will certainly write all sorts of >> evil code that will fail or not be portable... >> >> I say, let's design functions as well as we can (ok as well ss you can - I >> >> >> 2010/2/2 Mikael Grev >> >> >>> I must, slightly reluctantly due to historic reasons, agree as well. No >>> matter how things are turned lambdas is something new and the object >>> implementation should not shine through if not needed or shown advantageous. >>> >>> Also, I have always thought declaring a variable as 'volatile' would've >>> been a better solution than forcing it to be final since so many go around >>> that anyway. I know I do sometimes, and I just feel dirty when I do, for no >>> good reason since I know what I'm doing (most of the times anyway. Can't >>> think of a bug introduced that way). >>> >>> Cheers, >>> Mikael >>> >>> On Feb 2, 2010, at 19:48 PM, R?mi Forax wrote: >>> >>> >>>> Le 02/02/2010 19:40, Neal Gafter a ?crit : >>>> >>>>> I think Zdenek had it exactly right. >>>>> >>>>> A lambda expression is not an object creation expression. The function >>>>> >>> is >>> >>>>> not declared within the scope of some object type. A lambda is written >>>>> >>> as >>> >>>>> an independent block of code, and should be scoped accordingly. Trying >>>>> >>> to >>> >>>>> make a lambda act the same as an anonymous class creation expression >>>>> >>> smacks >>> >>>>> of cargo-cult language design. >>>>> >>>>> >>>> I agree. Lambda is an anonymous function. >>>> >>>> >>>>> Cheers, >>>>> Neal >>>>> >>>>> >>>> regards, >>>> R?mi >>>> >>> >> > From opinali at gmail.com Wed Feb 3 05:57:22 2010 From: opinali at gmail.com (Osvaldo Doederlein) Date: Wed, 3 Feb 2010 11:57:22 -0200 Subject: What is the meaning of this? In-Reply-To: <17b2302a1002021419g7bf08315gd425637b9e7640f4@mail.gmail.com> References: <17b2302a1001281119t4ea1c34cya8a05e59864b407@mail.gmail.com> <201001282246.44590.peter.levart@gmail.com> <17b2302a1001281619i72676ed1p954b010b1a23a614@mail.gmail.com> <17b2302a1002021017l76bcad7cw21cd864d8dd810a5@mail.gmail.com> <15e8b9d21002021040s7afefa46id70110e3ba944c14@mail.gmail.com> <4B68736F.7030000@univ-mlv.fr> <08B2E113-34CF-4882-9C34-C157D8B2CEA7@miginfocom.com> <17b2302a1002021419g7bf08315gd425637b9e7640f4@mail.gmail.com> Message-ID: 2010/2/2 Joshua Bloch > Osvaldo, > > Of course we all believe that we should "design functions [lambdas] as well > as we can." But we disagree about what constitutes quality. I'm deeply > concerned about consistency with existing constructs, and conceptual surface > area. Java currently has two main families of types: primitives and object > reference types. Object reference types are further subdivided into > interface types and class types. Arrays are class types. When we added enums > and annotations, they fit conveniently into class types and interface types > (respectively). As we add function types and expressions for producing > instances of these types (lambdas), we should try as hard as we can to make > them fit into the existing type framework of the language. > I agree, these principles are not even debatable - we need to fit new pegs in existing holes, as neatly as possible. A perfect fit is seldom possible when the core language is not really extensible. Even for something as simple as enums, Java5's solution had important tradeoffs (no compatibility for the enormous legacy of sets-of-int-constants used as enums; no switch() over enums except in the enum class). The debate happens where there are many possible ways to obtain the desired consistency, because there are many existing constructs and concepts to be consistent with and you often ought to pick priorities. A function type should certainly be a reference type, but... should they have some standard root type (like Enum and Annotation)? Should lambda conversion support any SAM type, or just SAM interfaces? Should every lambda expression produce an object with unique identity? Could a lambda be a simple MethodHandle, without any wrapper around that when not necessary? To summarize my POV, I think the lambda concept should not be tied to its implementation (whatever code that javac produces), and it should not be tied to the needs of SAM types (this is strictly a backwards-compatibility feature: the tail shall not wag the dog; if we need some ugly exceptional cases for SAM conversion in exchange for cleaner lambdas everywhere else, that's the right tradeoff IMO). A function type is a reference type, but lambdas are not always bound to variables. Lambdas should not be forced to look like methods-of-objects, complete with a self-referent 'this'. Lambdas that must be represented as instances but don't capture enclosing state should not be forced to be unique instances. And when one asks the question "What if the programmer writes some code that expects a lambda to be a [unique] object, or compiled to a certain pattern code, and uses typecasts/reflection/whatever to write code that's hardwired to that", well, my answer (educated version) is that this user won't be using the language correctly, just let his code fail or be non-portable or suffer incompatible linkage changes. Some bad things can be prevented by the language or javac (e.g. make synchronized(), and other unreasonable usages, illegal - or at least issue a warning). The current language and APIs are already shock-full of ways to shoot yourself in the foot, especially when you blatantly disrespect or ignore the language or API specs. In another thread, we're rehashing the debate of autoboxing - yeah, people can rely on identity guarantees that are not valid for cached Integers, they can write dumb code that uses equality comparison over Integers outside the guaranteed-cached range (or mixing in some Integer that was not produced by the caching factory API), or maybe synchronize on its monitor, and that code will behave differently in another JVM, and may bomb with severe and random / dificult-to-reproduce bugs like deadlocks. When Java5 was designed, we just said: just screw the programmer who does this, because autoboxing is very good and caching is critical so we won't make unreasonable compromises. A+ Osvaldo > > That does not mean that we can't invent any new concepts. Enum types were a > new concept, but they meshed well into the existing type framework, and most > Java programmers found their use to be comfortable and intuitive. They could > pretty much guess what code using enums did without reading a spec or > tutorial. That's what we're aiming for. > > Josh > > > On Tue, Feb 2, 2010 at 12:24 PM, Osvaldo Doederlein wrote: > >> I agree too (with Mikael/R?mi/Neal). And I add that, it worries me this >> (apparent) thinking of "we cannot add new concepts to the language" - if >> this is not Josh's position, it boils down to that in practice: i.e., in >> Java there are only methods-inside-objects (or classes for statics), even >> inner classes are very explicitly methods-inside-objects, so any new >> construct to contain code must be a method-inside-object... we can't have >> a >> fundamentally new construct, like a function/closure/lambda. We can only >> have something that _looks like_ a function/closure/lambda but is firmly >> rooted (both syntactically and semantically) in the method-of-object >> concept, down to backwards-compatible behavior of 'this', identity / >> memory >> allocation, etc. >> >> With such restrictions, Java is doomed to evolve only in the most >> cumbersome >> way possible, every new feature always paying enormous tradeoffs not only >> to >> backwards compatibility (which is already often a big hit - and I agree >> that >> cannot be avoided), but also to the backwards thinking - let's not invent >> any really new concepts because most developers are idiots who can't learn >> that a function is something different and will certainly write all sorts >> of >> evil code that will fail or not be portable... >> >> I say, let's design functions as well as we can (ok as well ss you can - I >> >> >> 2010/2/2 Mikael Grev >> >> > I must, slightly reluctantly due to historic reasons, agree as well. No >> > matter how things are turned lambdas is something new and the object >> > implementation should not shine through if not needed or shown >> advantageous. >> > >> > Also, I have always thought declaring a variable as 'volatile' would've >> > been a better solution than forcing it to be final since so many go >> around >> > that anyway. I know I do sometimes, and I just feel dirty when I do, for >> no >> > good reason since I know what I'm doing (most of the times anyway. Can't >> > think of a bug introduced that way). >> > >> > Cheers, >> > Mikael >> > >> > On Feb 2, 2010, at 19:48 PM, R?mi Forax wrote: >> > >> > > Le 02/02/2010 19:40, Neal Gafter a ?crit : >> > >> I think Zdenek had it exactly right. >> > >> >> > >> A lambda expression is not an object creation expression. The >> function >> > is >> > >> not declared within the scope of some object type. A lambda is >> written >> > as >> > >> an independent block of code, and should be scoped accordingly. >> Trying >> > to >> > >> make a lambda act the same as an anonymous class creation expression >> > smacks >> > >> of cargo-cult language design. >> > >> >> > > >> > > I agree. Lambda is an anonymous function. >> > > >> > >> Cheers, >> > >> Neal >> > >> >> > > >> > > regards, >> > > R?mi >> > >> > >> > >> >> > From jkuhnert at gmail.com Wed Feb 3 06:23:31 2010 From: jkuhnert at gmail.com (Jesse Kuhnert) Date: Wed, 3 Feb 2010 09:23:31 -0500 Subject: 2cents from an ordinary Java developer In-Reply-To: <282D0BDE-AD5A-44EB-9C44-6F177E81A9BC@googlemail.com> References: <4B68B111.9040305@coskuns-castle.de> <997cab101002022104g5d962201i504d97856bd556a4@mail.gmail.com> <15e8b9d21002022235w2579ef1fgc3d2d9cb57c21a6b@mail.gmail.com> <282D0BDE-AD5A-44EB-9C44-6F177E81A9BC@googlemail.com> Message-ID: <7926817e1002030623k4b981585rf0d0725ca334025c@mail.gmail.com> If it all comes down to guessing/thinking that it's possible ordinary developers will have issues why doesn't someone(s) just do a small test sampling of devs? Pick a comparable enough language (guess Scala) and grab a few engineers you think would have trouble and record results. There certainly would seem to be enough resources / developers floating around at the corporations people on this list work for to make it worth trying? Could be that everyone is right and closures will not be understood easily enough, but without any kind of semi-scientific method for determining this you're still just guessing.. On Wed, Feb 3, 2010 at 2:36 AM, Mark Mahieu wrote: > > On 3 Feb 2010, at 06:35, Neal Gafter wrote: >> >> The ordinary Java programmers of today are not ordinary programmers >> from a point of view 5-10 years ago. > > Neither is the code they work on. ?My, how the boilerplate hath grown... > > >> As our languages change, so does >> our sense of common and ordinary. >> > > That's very true, and in more ways than one. > > 5 years ago the team I was working within contained very few programmers with any real experience using a language other than Java. ?About 2 years ago many of them were spending some of their time in Ruby-land *. ?As of about a year ago, we have Ruby programmers on the team who have never worked with Java (or they won't admit to having done so, I'm never quite sure). > > > Mark > > > * often their first exposure to lambdas, with "why on earth can't I just write it like that in Java?" being [a polite version of] the typical reaction. > > > From markmahieu at googlemail.com Wed Feb 3 06:54:41 2010 From: markmahieu at googlemail.com (Mark Mahieu) Date: Wed, 3 Feb 2010 14:54:41 +0000 Subject: 2cents from an ordinary Java developer In-Reply-To: <7926817e1002030623k4b981585rf0d0725ca334025c@mail.gmail.com> References: <4B68B111.9040305@coskuns-castle.de> <997cab101002022104g5d962201i504d97856bd556a4@mail.gmail.com> <15e8b9d21002022235w2579ef1fgc3d2d9cb57c21a6b@mail.gmail.com> <282D0BDE-AD5A-44EB-9C44-6F177E81A9BC@googlemail.com> <7926817e1002030623k4b981585rf0d0725ca334025c@mail.gmail.com> Message-ID: I suspect it would be terribly difficult to draw useful conclusions from any such exercise. There are so many variables, and the chance of bias (however unintentional) is huge. Alex Buckley's request for statistics on existing codebases approaches the question from a different angle, but seems more promising. Personally, I think it would be good to see a few more responses. Regards, Mark On 3 Feb 2010, at 14:23, Jesse Kuhnert wrote: > If it all comes down to guessing/thinking that it's possible ordinary > developers will have issues why doesn't someone(s) just do a small > test sampling of devs? > > Pick a comparable enough language (guess Scala) and grab a few > engineers you think would have trouble and record results. There > certainly would seem to be enough resources / developers floating > around at the corporations people on this list work for to make it > worth trying? Could be that everyone is right and closures will not > be understood easily enough, but without any kind of semi-scientific > method for determining this you're still just guessing.. > > On Wed, Feb 3, 2010 at 2:36 AM, Mark Mahieu wrote: >> >> On 3 Feb 2010, at 06:35, Neal Gafter wrote: >>> >>> The ordinary Java programmers of today are not ordinary programmers >>> from a point of view 5-10 years ago. >> >> Neither is the code they work on. My, how the boilerplate hath grown... >> >> >>> As our languages change, so does >>> our sense of common and ordinary. >>> >> >> That's very true, and in more ways than one. >> >> 5 years ago the team I was working within contained very few programmers with any real experience using a language other than Java. About 2 years ago many of them were spending some of their time in Ruby-land *. As of about a year ago, we have Ruby programmers on the team who have never worked with Java (or they won't admit to having done so, I'm never quite sure). >> >> >> Mark >> >> >> * often their first exposure to lambdas, with "why on earth can't I just write it like that in Java?" being [a polite version of] the typical reaction. >> >> >> From jkuhnert at gmail.com Wed Feb 3 07:15:37 2010 From: jkuhnert at gmail.com (Jesse Kuhnert) Date: Wed, 3 Feb 2010 10:15:37 -0500 Subject: 2cents from an ordinary Java developer In-Reply-To: References: <4B68B111.9040305@coskuns-castle.de> <997cab101002022104g5d962201i504d97856bd556a4@mail.gmail.com> <15e8b9d21002022235w2579ef1fgc3d2d9cb57c21a6b@mail.gmail.com> <282D0BDE-AD5A-44EB-9C44-6F177E81A9BC@googlemail.com> <7926817e1002030623k4b981585rf0d0725ca334025c@mail.gmail.com> Message-ID: <7926817e1002030715w36d346ccxa9fb0ae009b3cfb4@mail.gmail.com> I think that's all we want. Some way of determining this that seems fair and logical and doesn't feel like an arbitrary decision being made. That would be much easier to swallow by the dev community as a whole than the arbitrary decision path, I'm guessing. On Wed, Feb 3, 2010 at 9:54 AM, Mark Mahieu wrote: > I suspect it would be terribly difficult to draw useful conclusions from any such exercise. ?There are so many variables, and the chance of bias (however unintentional) is huge. > > Alex Buckley's request for statistics on existing codebases approaches the question from a different angle, but seems more promising. ?Personally, I think it would be good to see a few more responses. > > Regards, > > Mark > > > On 3 Feb 2010, at 14:23, Jesse Kuhnert wrote: > >> If it all comes down to guessing/thinking that it's possible ordinary >> developers will have issues why doesn't someone(s) just do a small >> test sampling of devs? >> >> Pick a comparable enough language (guess Scala) and grab a few >> engineers you think would have trouble and record results. There >> certainly would seem to be enough resources / developers floating >> around at the corporations people on this list work for to make it >> worth trying? ?Could be that everyone is right and closures will not >> be understood easily enough, ?but without any kind of semi-scientific >> method for determining this you're still just guessing.. >> >> On Wed, Feb 3, 2010 at 2:36 AM, Mark Mahieu wrote: >>> >>> On 3 Feb 2010, at 06:35, Neal Gafter wrote: >>>> >>>> The ordinary Java programmers of today are not ordinary programmers >>>> from a point of view 5-10 years ago. >>> >>> Neither is the code they work on. ?My, how the boilerplate hath grown... >>> >>> >>>> As our languages change, so does >>>> our sense of common and ordinary. >>>> >>> >>> That's very true, and in more ways than one. >>> >>> 5 years ago the team I was working within contained very few programmers with any real experience using a language other than Java. ?About 2 years ago many of them were spending some of their time in Ruby-land *. ?As of about a year ago, we have Ruby programmers on the team who have never worked with Java (or they won't admit to having done so, I'm never quite sure). >>> >>> >>> Mark >>> >>> >>> * often their first exposure to lambdas, with "why on earth can't I just write it like that in Java?" being [a polite version of] the typical reaction. >>> >>> >>> > > From oallouch at free.fr Wed Feb 3 07:38:30 2010 From: oallouch at free.fr (Olivier Allouch) Date: Wed, 03 Feb 2010 16:38:30 +0100 Subject: 2cents from an ordinary Java developer In-Reply-To: <7926817e1002030623k4b981585rf0d0725ca334025c@mail.gmail.com> References: <4B68B111.9040305@coskuns-castle.de> <997cab101002022104g5d962201i504d97856bd556a4@mail.gmail.com> <15e8b9d21002022235w2579ef1fgc3d2d9cb57c21a6b@mail.gmail.com> <282D0BDE-AD5A-44EB-9C44-6F177E81A9BC@googlemail.com> <7926817e1002030623k4b981585rf0d0725ca334025c@mail.gmail.com> Message-ID: <4B699876.10005@free.fr> Hi, 2 more cents from another ordinary (whatever that means) developer. We always compare the proposal to Scala, but wouldn't it be interesting to compare it to other languages a lot of people use: scripting languages. If we want examples of ease of use, that's a good start. For instance, I remember coding in ActionScript 2 (in Flash of course) and having to bind every closure (objects of type Function and all) to make it use the enclosing Object as 'this'. 'this' was the Object calling the anonymous Function (or the first Object if the closure was called by another closure). That seemed logical...but every coder had to manually bind 'this' and everybody complained. So Adobe changed it in ActionScript 3 and 'this' got automatically bound to the enclosing Object. I don't see parallel programming as the main use case, but I see UI coding (closures for event handling) I don't see Scala as the contender, but I see JavaFX Script, or ActionScript. Please respect the 'Principle of Least Astonishment' for us, average coders. Btw, that also implies a 'return' that just exits from the closure :) Olivier Allouch Le 03/02/2010 15:23, Jesse Kuhnert a ?crit : > If it all comes down to guessing/thinking that it's possible ordinary > developers will have issues why doesn't someone(s) just do a small > test sampling of devs? > > Pick a comparable enough language (guess Scala) and grab a few > engineers you think would have trouble and record results. There > certainly would seem to be enough resources / developers floating > around at the corporations people on this list work for to make it > worth trying? Could be that everyone is right and closures will not > be understood easily enough, but without any kind of semi-scientific > method for determining this you're still just guessing.. > > On Wed, Feb 3, 2010 at 2:36 AM, Mark Mahieu wrote: > >> On 3 Feb 2010, at 06:35, Neal Gafter wrote: >> >>> The ordinary Java programmers of today are not ordinary programmers >>> from a point of view 5-10 years ago. >>> >> Neither is the code they work on. My, how the boilerplate hath grown... >> >> >> >>> As our languages change, so does >>> our sense of common and ordinary. >>> >>> >> That's very true, and in more ways than one. >> >> 5 years ago the team I was working within contained very few programmers with any real experience using a language other than Java. About 2 years ago many of them were spending some of their time in Ruby-land *. As of about a year ago, we have Ruby programmers on the team who have never worked with Java (or they won't admit to having done so, I'm never quite sure). >> >> >> Mark >> >> >> * often their first exposure to lambdas, with "why on earth can't I just write it like that in Java?" being [a polite version of] the typical reaction. >> >> >> >> > From mthornton at optrak.co.uk Wed Feb 3 08:06:03 2010 From: mthornton at optrak.co.uk (Mark Thornton) Date: Wed, 03 Feb 2010 16:06:03 +0000 Subject: 2cents from an ordinary Java developer In-Reply-To: <4B699876.10005@free.fr> References: <4B68B111.9040305@coskuns-castle.de> <997cab101002022104g5d962201i504d97856bd556a4@mail.gmail.com> <15e8b9d21002022235w2579ef1fgc3d2d9cb57c21a6b@mail.gmail.com> <282D0BDE-AD5A-44EB-9C44-6F177E81A9BC@googlemail.com> <7926817e1002030623k4b981585rf0d0725ca334025c@mail.gmail.com> <4B699876.10005@free.fr> Message-ID: <4B699EEB.10101@optrak.co.uk> Olivier Allouch wrote: > Please respect the 'Principle of Least Astonishment' for us, average > coders. Btw, that also implies a 'return' that just exits from the > closure :) > > Unfortunately what is least astonishing is likely depend on ones other experience. Amongst our developers we have experience with Groovy, Haskell, JavaScript, Scala* and others in addition to Java. Each will probably have subtlely different expectations of the behaviour of a lambda in Java. Mark Thornton * listed in alphabetical order From jjb at google.com Wed Feb 3 08:14:21 2010 From: jjb at google.com (Joshua Bloch) Date: Wed, 3 Feb 2010 08:14:21 -0800 Subject: What is the meaning of this? In-Reply-To: References: <17b2302a1001281119t4ea1c34cya8a05e59864b407@mail.gmail.com> <17b2302a1001281619i72676ed1p954b010b1a23a614@mail.gmail.com> <17b2302a1002021017l76bcad7cw21cd864d8dd810a5@mail.gmail.com> <15e8b9d21002021040s7afefa46id70110e3ba944c14@mail.gmail.com> <4B68736F.7030000@univ-mlv.fr> <08B2E113-34CF-4882-9C34-C157D8B2CEA7@miginfocom.com> <17b2302a1002021419g7bf08315gd425637b9e7640f4@mail.gmail.com> Message-ID: <17b2302a1002030814i330523fbsd1246f71609ab4cf@mail.gmail.com> Osvaldo, On Wed, Feb 3, 2010 at 5:57 AM, Osvaldo Doederlein wrote: > 2010/2/2 Joshua Bloch > > Osvaldo, >> >> Of course we all believe that we should "design functions [lambdas] as >> well as we can." But we disagree about what constitutes quality. I'm deeply >> concerned about consistency with existing constructs, and conceptual surface >> area. Java currently has two main families of types: primitives and object >> reference types. Object reference types are further subdivided into >> interface types and class types. Arrays are class types. When we added enums >> and annotations, they fit conveniently into class types and interface types >> (respectively). As we add function types and expressions for producing >> instances of these types (lambdas), we should try as hard as we can to make >> them fit into the existing type framework of the language. >> > > I agree, these principles are not even debatable - we need to fit new pegs > in existing holes, as neatly as possible. A perfect fit is seldom possible > when the core language is not really extensible. Even for something as > simple as enums, Java5's solution had important tradeoffs (no compatibility > for the enormous legacy of sets-of-int-constants used as enums; no switch() > over enums except in the enum class). > A minor point, but you can switch over enums wherever you like. In my view, it's rarely correct to do it *inside *the enum. But even this is debatable. See Effective Java, Pages 154-157 for my take on all this. > > The debate happens where there are many possible ways to obtain the desired > consistency, because there are many existing constructs and concepts to be > consistent with and you often ought to pick priorities. A function type > should certainly be a reference type, but... should they have some standard > root type (like Enum and Annotation)? Should lambda conversion support any > SAM type, or just SAM interfaces? Should every lambda expression produce an > object with unique identity? Could a lambda be a simple MethodHandle, > without any wrapper around that when not necessary? > > To summarize my POV, I think the lambda concept should not be tied to its > implementation (whatever code that javac produces), > You'll get no argument there. We're discussing the desirable semantics, not implementation. > and it should not be tied to the needs of SAM types (this is strictly a > backwards-compatibility feature: the tail shall not wag the dog; > I'm not so sure about this. SAM *interface *types are a good thing! Unlike function types, they have names (e.g. Reducer), semantic constraints (e.g., Must be associative), they provide the opportunity for type-specific documentation (e.g., A specialized combiner that accepts to instaces of some type and returns an instance of the same type). SAM interfaces provide great advantages when used with IDEs (autocompletion, context-sensitive documentation, etc.). So I think we all agree that it's critical that the solution works beautifully with SAM *interfaces*. SAM *classes *are perhaps less critical, though even they will have non-legacy uses. There are several valid reasons to use a SAM class instead of an interface. One is performance (e.g., TimerTask). Another is Instance control (with a class, you can write the sole constructor, and keep track of every instance). But I would be willing to sacrifice good support for SAM *classes *if we decide that it provides large benefits. > Lambdas should not be forced to look like methods-of-objects, complete > with a self-referent 'this'. > This is the fundamental issue of this thread. I don't see this is being forced to do the wrong thing. I see it as the obvious correct thing. I believe we need a strong argument to do otherwise, because doing otherwise is an inconsistency. I see the "transparency" argument as weak. I'm not even quite sure what people mean by it, and I don't know what it buys me in the context of this effort. > Lambdas that must be represented as instances but don't capture enclosing > state should not be forced to be unique instances. > You'll get no argument here! And I don't see this having any bearing on the meaning of "this." I see a lambda expression as a sort of a static factory. If you evaluate it ten times, it's fine if all ten evaluations produce the same object, in which case I believe that this should return the same value in all ten. Josh From grev at miginfocom.com Wed Feb 3 09:33:11 2010 From: grev at miginfocom.com (Mikael Grev) Date: Wed, 3 Feb 2010 18:33:11 +0100 Subject: 2cents from an ordinary Java developer In-Reply-To: <4B699EEB.10101@optrak.co.uk> References: <4B68B111.9040305@coskuns-castle.de> <997cab101002022104g5d962201i504d97856bd556a4@mail.gmail.com> <15e8b9d21002022235w2579ef1fgc3d2d9cb57c21a6b@mail.gmail.com> <282D0BDE-AD5A-44EB-9C44-6F177E81A9BC@googlemail.com> <7926817e1002030623k4b981585rf0d0725ca334025c@mail.gmail.com> <4B699876.10005@free.fr> <4B699EEB.10101@optrak.co.uk> Message-ID: <8EF2D65E-BE68-4967-96B2-3F1FE70BF746@miginfocom.com> Percentage wise, how many Java vs all others do you think? On Feb 3, 2010, at 17:06 PM, Mark Thornton wrote: > Olivier Allouch wrote: >> Please respect the 'Principle of Least Astonishment' for us, average >> coders. Btw, that also implies a 'return' that just exits from the >> closure :) >> >> > Unfortunately what is least astonishing is likely depend on ones other > experience. Amongst our developers we have experience with Groovy, > Haskell, JavaScript, Scala* and others in addition to Java. Each will > probably have subtlely different expectations of the behaviour of a > lambda in Java. > > Mark Thornton > > * listed in alphabetical order From john at milsson.nu Wed Feb 3 10:54:35 2010 From: john at milsson.nu (John Nilsson) Date: Wed, 3 Feb 2010 19:54:35 +0100 Subject: What is the meaning of this? In-Reply-To: <17b2302a1002030814i330523fbsd1246f71609ab4cf@mail.gmail.com> References: <17b2302a1001281119t4ea1c34cya8a05e59864b407@mail.gmail.com> <17b2302a1002021017l76bcad7cw21cd864d8dd810a5@mail.gmail.com> <15e8b9d21002021040s7afefa46id70110e3ba944c14@mail.gmail.com> <4B68736F.7030000@univ-mlv.fr> <08B2E113-34CF-4882-9C34-C157D8B2CEA7@miginfocom.com> <17b2302a1002021419g7bf08315gd425637b9e7640f4@mail.gmail.com> <17b2302a1002030814i330523fbsd1246f71609ab4cf@mail.gmail.com> Message-ID: On Wed, Feb 3, 2010 at 5:14 PM, Joshua Bloch wrote: > > Lambdas should not be forced to look like methods-of-objects, complete > > with a self-referent 'this'. > > > This is the fundamental issue of this thread. I don't see this is being > forced to do the wrong thing. I see it as the obvious correct thing. I > believe we need a strong argument to do otherwise, because doing otherwise > is an inconsistency. I see the "transparency" argument as weak. I'm not > even > quite sure what people mean by it, and I don't know what it buys me in the > context of this effort. Maybe it is an implementation issue? If the semantics of lambdas force them to be objects doesn't this limit the runtime representation for no good reason? WRT to the argument that they should fit inside the current language framework: Why can't they just be seen as anonymous methods? In my world the step from public int method() { return 1; } to public final #int() method = #int(){ return 1; } is quite natural it you think of the former as syntactic sugar for the latter. BR, John From opinali at gmail.com Wed Feb 3 11:11:42 2010 From: opinali at gmail.com (Osvaldo Doederlein) Date: Wed, 3 Feb 2010 17:11:42 -0200 Subject: What is the meaning of this? In-Reply-To: <17b2302a1002030814i330523fbsd1246f71609ab4cf@mail.gmail.com> References: <17b2302a1001281119t4ea1c34cya8a05e59864b407@mail.gmail.com> <17b2302a1002021017l76bcad7cw21cd864d8dd810a5@mail.gmail.com> <15e8b9d21002021040s7afefa46id70110e3ba944c14@mail.gmail.com> <4B68736F.7030000@univ-mlv.fr> <08B2E113-34CF-4882-9C34-C157D8B2CEA7@miginfocom.com> <17b2302a1002021419g7bf08315gd425637b9e7640f4@mail.gmail.com> <17b2302a1002030814i330523fbsd1246f71609ab4cf@mail.gmail.com> Message-ID: Joshua, 2010/2/3 Joshua Bloch > Osvaldo, > >> and it should not be tied to the needs of SAM types (this is strictly a >> backwards-compatibility feature: the tail shall not wag the dog; >> > I'm not so sure about this. SAM *interface *types are a good thing! > Unlike function types, they have names (e.g. Reducer), semantic constraints > (e.g., Must be associative), they provide the opportunity for type-specific > documentation (e.g., A specialized combiner that accepts to instaces of some > type and returns an instance of the same type). SAM interfaces provide great > advantages when used with IDEs (autocompletion, context-sensitive > documentation, etc.). So I think we all agree that it's critical that the > solution works beautifully with SAM *interfaces*. SAM *classes *are > perhaps less critical, though even they will have non-legacy uses. There > are several valid reasons to use a SAM class instead of an interface. One > is performance (e.g., TimerTask). Another is Instance control (with a class, > you can write the sole constructor, and keep track of every instance). But I > would be willing to sacrifice good support for SAM *classes *if we decide > that it provides large benefits. > Agreed, SAM interfaces are great, considering that we don't have a typedef construct. We keep wishing for typedefs, they would become even more useful now (for both generics and lambdas, as already discussed in this list) but this is a big RFE and not in the viable scope for Java7, so a nice support for SAM interfaces is really the way to go, even not considering legacy APIs. I'm also willing to drop SAM classes, remarkably if they impose *any*further tradeoff to lambdas at large. ;-) Although SAM classes are tempting (usage is not insignificant), it's not bad to favor usage of interfaces in general - people will often write a SAM class (or an abstract class with a handful, unrelated abstract methods) when they could have used a SAM interface (or a handful), perhaps just with the cost of some delegation, often not even that cost (just laziness to create a pair of interface + abstract base class). > > >> Lambdas should not be forced to look like methods-of-objects, complete >> with a self-referent 'this'. >> > This is the fundamental issue of this thread. I don't see this is being > forced to do the wrong thing. I see it as the obvious correct thing. I > believe we need a strong argument to do otherwise, because doing otherwise > is an inconsistency. I see the "transparency" argument as weak. I'm not even > quite sure what people mean by it, and I don't know what it buys me in the > context of this effort. > I think the core argument is discouraging the use to see the lambda as a method inside an object, and write code that depends on this assumption and perhaps will depend on implementation details, and make certain implementation strategies dangerous. Well, we might just force javac's translation strategy to some standard/"safe" option (synthetic class with certain visible properties, or MethodHandle if this can be used by arbitrary lambdas), for lambdas that use 'this' (and also, obviously lambdas that are assigned to variables or SAM-converted). Avoiding those will become just a best-practice: if you don't do that, javac and/or HotSpot may generate much simpler/faster code and perform aggressive optimizations like full inlining of uses of select control abstractions. If you don't, these opts cannot be used. > Lambdas that must be represented as instances but don't capture enclosing >> state should not be forced to be unique instances. >> > You'll get no argument here! And I don't see this having any bearing on the > meaning of "this." I see a lambda expression as a sort of a static factory. > If you evaluate it ten times, it's fine if all ten evaluations produce the > same object, in which case I believe that this should return the same value > in all ten. > This agreement makes me very happy. This was subject to debate in some emails. A+ Osvaldo From Alex.Buckley at Sun.COM Wed Feb 3 11:25:36 2010 From: Alex.Buckley at Sun.COM (Alex Buckley) Date: Wed, 03 Feb 2010 11:25:36 -0800 Subject: 2cents from an ordinary Java developer In-Reply-To: <7926817e1002030623k4b981585rf0d0725ca334025c@mail.gmail.com> References: <4B68B111.9040305@coskuns-castle.de> <997cab101002022104g5d962201i504d97856bd556a4@mail.gmail.com> <15e8b9d21002022235w2579ef1fgc3d2d9cb57c21a6b@mail.gmail.com> <282D0BDE-AD5A-44EB-9C44-6F177E81A9BC@googlemail.com> <7926817e1002030623k4b981585rf0d0725ca334025c@mail.gmail.com> Message-ID: <4B69CDB0.6000903@sun.com> Jesse Kuhnert wrote: > If it all comes down to guessing/thinking that it's possible ordinary > developers will have issues why doesn't someone(s) just do a small > test sampling of devs? Of course it all comes down to guessing. Unlike when James could sample all Java code in the world to prove that 'goto' was basically never used, all I can do now is propose a feature that doesn't come as too much of a surprise to existing developers and that I hope will best serve the language in its intended roles over the next, say, 20 years. What other languages have done is largely irrelevant. People's experience with those languages is interesting, and I thank Mark Mahieu and Olivier Allouch for their comments re: Ruby and ActionScript. > Pick a comparable enough language (guess Scala) and grab a few > engineers you think would have trouble and record results. There > certainly would seem to be enough resources / developers floating > around at the corporations people on this list work for to make it > worth trying? Could be that everyone is right and closures will not > be understood easily enough, but without any kind of semi-scientific > method for determining this you're still just guessing.. I am going to squash this right now. The external perception of corporations' resources are always, without fail, orders of magnitude larger than the reality. What you propose is impractical. This is why I requested statistics from the community and thank Neal, Stephen, Mark, and Alexey for taking the trouble to provide them. Alex From neal at gafter.com Wed Feb 3 11:52:06 2010 From: neal at gafter.com (Neal Gafter) Date: Wed, 3 Feb 2010 11:52:06 -0800 Subject: What is the meaning of this? In-Reply-To: References: <17b2302a1001281119t4ea1c34cya8a05e59864b407@mail.gmail.com> <17b2302a1002021017l76bcad7cw21cd864d8dd810a5@mail.gmail.com> <15e8b9d21002021040s7afefa46id70110e3ba944c14@mail.gmail.com> <4B68736F.7030000@univ-mlv.fr> <08B2E113-34CF-4882-9C34-C157D8B2CEA7@miginfocom.com> <17b2302a1002021419g7bf08315gd425637b9e7640f4@mail.gmail.com> <17b2302a1002030814i330523fbsd1246f71609ab4cf@mail.gmail.com> Message-ID: <15e8b9d21002031152o563355d7t5931f11c231d35e6@mail.gmail.com> On Wed, Feb 3, 2010 at 11:11 AM, Osvaldo Doederlein wrote: >> ?Lambdas that must be represented as instances but don't capture enclosing >>> state should not be forced to be unique instances. > 2010/2/3 Joshua Bloch >> You'll get no argument here! And I don't see this having any bearing on the >> meaning of "this." I see a lambda expression as a sort of a static factory. >> ?If you evaluate it ten times, it's fine if all ten evaluations produce the >> same object, in which case I believe that this should return the same value >> in all ten. > This agreement makes me very happy. This was subject to debate in some > emails. There are a few relationships between this issue and the meaning of "this". To take one, this agreed-upon position conflicts with SAM classes, because the execution of the constructor is visible. The specification must clearly detail the handling of exceptions thrown by the constructor in the compile-time and run-time semantics of the lambda conversion, and I don't see any alternative but to specify them occurring at the point of the conversion. SAM interfaces don't have this issue because they don't have constructors. But for SAM interfaces, while there is utility to lexically scoped "this", the proposed benefit of "this" referring to the function type (it can be cast to the SAM type) is de minimis. From Alex.Buckley at Sun.COM Wed Feb 3 12:04:55 2010 From: Alex.Buckley at Sun.COM (Alex Buckley) Date: Wed, 03 Feb 2010 12:04:55 -0800 Subject: SAM interface/class figures [Re: Preparing for the 0.2 draft] In-Reply-To: <96324399B8D14BCBBFF33FC79060BFF7@Labs.IntelliJ.Net> References: <96324399B8D14BCBBFF33FC79060BFF7@Labs.IntelliJ.Net> Message-ID: <4B69D6E7.7040608@sun.com> Thanks Alexey and Stephen. How are you producing these statistics? Alexey, do you really mean 1920 SAM *classes*, or 1920 SAM types? Because if 7474 anon.classes extend a class, and 12597 anon.classes extend a "SAM", then some of those "SAM" things must be interfaces. What is really interesting is how many of those 12597 anon.classes extend a SAM class v. implement a SAM interface. Alex Alexey Kudravtsev wrote: > These are figures for the Intellij IDEA (http://www.jetbrains.com/idea/) > source base. > > Total files inspected: 34794 > Total java classes: 69703 > All SAM classes: 1920 (2,8%) > All Anonymous classes: 21908 (31,4%) > anonymous classes which Extend class: 7474 (34,1%) > anonymous classes which implement interface: 14434 (65,9%) > > anonymous classes which extend SAM: 12597 (57,5%) > > anon classes with single method: 18431 (84,1%) > anon classes with fields: 149 (0,7%) > anon classes with 'this' references: 116 (0,5%) > anon classes with qualified 'this' references: 77 (0,4%) > anon classes which recurse: 15 (0,1%) > > Alexey Kudravtsev > Intellij IDEA developer > JetBrains, Inc > http://www.jetbrains.com > "Develop with pleasure!" > >> Message: 1 >> Date: Fri, 29 Jan 2010 00:56:46 -0800 >> From: Alex Buckley >> Subject: Preparing for the 0.2 draft >> To: "lambda-dev at openjdk.java.net" >> Message-ID: <4B62A2CE.7080300 at sun.com> >> Content-Type: text/plain; CHARSET=US-ASCII; format=flowed >> >> Many thanks for everyone's comments on the 0.1 draft. The "foo.()" >> invocation syntax looks like a winner, and I am seriously considering >> rearranging the function type syntax to have argument types first. >> Requiring a lambda expression to denote its return type is also >> increasingly appealing. SAM classes are staying for now. >> >> The discussion of transparent 'this' v. non-transparent 'this' comes >> down to the question of what a lambda in Java is for. (Zdenek Tronicek >> ably made this point earlier.) Part of the answer lies in understanding >> when to use an anonymous inner class and when to use a lambda >> expression. A step towards that understanding would be knowing how >> anonymous inner classes are used today. >> >> It would be very interesting if list members could provide the following >> statistics for codebases with which they are familiar: >> >> - What %age of anonymous inner classes declare a single non-Object method? >> - What %age of single-method anonymous inner classes declare fields? >> - What %age of single methods in anonymous inner classes use 'this'? >> - What %age of single methods in anonymous inner classes recurse? >> - Are the %ages different for anonymous inner classes that implement an >> interface v. that extend a class ? >> >> (As well as anonymous inner classes, any utility class that >> implements/extends a SAM type is of interest, such as this nested class >> from the extra166y.ParallelArray Javadoc: >> static final class IsSenior implements Predicate { >> public boolean op(Student s) { return s.credits > 90; } >> } >> ) >> >> Alex >> > > From neal at gafter.com Wed Feb 3 12:25:42 2010 From: neal at gafter.com (Neal Gafter) Date: Wed, 3 Feb 2010 12:25:42 -0800 Subject: SAM interface/class figures [Re: Preparing for the 0.2 draft] In-Reply-To: <4B69D6E7.7040608@sun.com> References: <96324399B8D14BCBBFF33FC79060BFF7@Labs.IntelliJ.Net> <4B69D6E7.7040608@sun.com> Message-ID: <15e8b9d21002031225hd5b8e82le3de15429742887d@mail.gmail.com> On Wed, Feb 3, 2010 at 12:04 PM, Alex Buckley wrote: > What is really interesting is how many of those 12597 anon.classes extend a > SAM class v. implement a SAM interface. I don't think that is precise enough. Those statistics are only interesting if restricted to anonymous inner classes that implement a SAM type (as you say), AND define a single method, AND have no fields, AND pass no constructor arguments, AND are not generic methods, etc. The rest of them wouldn't be candidates for lambda expressions anyway. That's why I generated the statistics I did and also provided the code for others to do the same. From neal at gafter.com Wed Feb 3 12:55:32 2010 From: neal at gafter.com (Neal Gafter) Date: Wed, 3 Feb 2010 12:55:32 -0800 Subject: Code to analyze a source base for project lambda Message-ID: <15e8b9d21002031255y45704467i32475a2c9b64803d@mail.gmail.com> Apparently it was rejected by the moderator last time. Here it is again. The enclosed code (Main.java) was thrown together hastily, but it might be useful to analyze a source base for project lambda. Compile and run it with tools.jar on the classpath (but on a mac, tools.jar is always on the classpath). It acts just like javac, but printing statistics before exiting. I used it to process most of the openjdk6 sources. From Alex.Buckley at Sun.COM Wed Feb 3 12:55:49 2010 From: Alex.Buckley at Sun.COM (Alex Buckley) Date: Wed, 03 Feb 2010 12:55:49 -0800 Subject: SAM interface/class figures [Re: Preparing for the 0.2 draft] In-Reply-To: <15e8b9d21002031225hd5b8e82le3de15429742887d@mail.gmail.com> References: <96324399B8D14BCBBFF33FC79060BFF7@Labs.IntelliJ.Net> <4B69D6E7.7040608@sun.com> <15e8b9d21002031225hd5b8e82le3de15429742887d@mail.gmail.com> Message-ID: <4B69E2D5.6050407@sun.com> Neal Gafter wrote: > On Wed, Feb 3, 2010 at 12:04 PM, Alex Buckley wrote: >> What is really interesting is how many of those 12597 anon.classes extend a >> SAM class v. implement a SAM interface. > > I don't think that is precise enough. Those statistics are only > interesting if restricted to anonymous inner classes that implement a > SAM type (as you say), AND define a single method, AND have no fields, > AND pass no constructor arguments, AND are not generic methods, etc. > The rest of them wouldn't be candidates for lambda expressions anyway. > That's why I generated the statistics I did and also provided the > code for others to do the same. I don't disagree about the restrictions, but I prefer to ask simple questions up front and get more detail about the results if they look interesting. What code are you talking about? There was nothing in your mail about OpenJDK 6 stats. Alex From scolebourne at joda.org Wed Feb 3 12:58:29 2010 From: scolebourne at joda.org (Stephen Colebourne) Date: Wed, 3 Feb 2010 20:58:29 +0000 Subject: SAM interface/class figures [Re: Preparing for the 0.2 draft] In-Reply-To: <4B69D6E7.7040608@sun.com> References: <96324399B8D14BCBBFF33FC79060BFF7@Labs.IntelliJ.Net> <4B69D6E7.7040608@sun.com> Message-ID: <4b4f45e01002031258m3cac63a6kfa77a005c432df7b@mail.gmail.com> Alex, I searched for all class files with the name $1 and analysed by hand from there (by looking at source files). Stephen On 3 February 2010 20:04, Alex Buckley wrote: > Thanks Alexey and Stephen. How are you producing these statistics? > > Alexey, do you really mean 1920 SAM *classes*, or 1920 SAM types? > Because if 7474 anon.classes extend a class, and 12597 anon.classes > extend a "SAM", then some of those "SAM" things must be interfaces. What > is really interesting is how many of those 12597 anon.classes extend a > SAM class v. implement a SAM interface. > > Alex > > Alexey Kudravtsev wrote: >> These are figures for the Intellij IDEA (http://www.jetbrains.com/idea/) >> source base. >> >> Total files inspected: 34794 >> Total java classes: 69703 >> All SAM classes: 1920 (2,8%) >> All Anonymous classes: 21908 (31,4%) >> ? anonymous classes which Extend class: 7474 (34,1%) >> ? anonymous classes which implement interface: 14434 (65,9%) >> >> ? anonymous classes which extend SAM: 12597 (57,5%) >> >> ? anon classes with single method: 18431 (84,1%) >> ? anon classes with fields: 149 (0,7%) >> ? anon classes with 'this' references: 116 (0,5%) >> ? anon classes with qualified 'this' references: 77 (0,4%) >> ? anon classes which recurse: 15 (0,1%) >> >> Alexey Kudravtsev >> Intellij IDEA developer >> JetBrains, Inc >> http://www.jetbrains.com >> "Develop with pleasure!" >> >>> Message: 1 >>> Date: Fri, 29 Jan 2010 00:56:46 -0800 >>> From: Alex Buckley >>> Subject: Preparing for the 0.2 draft >>> To: "lambda-dev at openjdk.java.net" >>> Message-ID: <4B62A2CE.7080300 at sun.com> >>> Content-Type: text/plain; CHARSET=US-ASCII; format=flowed >>> >>> Many thanks for everyone's comments on the 0.1 draft. The "foo.()" >>> invocation syntax looks like a winner, and I am seriously considering >>> rearranging the function type syntax to have argument types first. >>> Requiring a lambda expression to denote its return type is also >>> increasingly appealing. SAM classes are staying for now. >>> >>> The discussion of transparent 'this' v. non-transparent 'this' comes >>> down to the question of what a lambda in Java is for. (Zdenek Tronicek >>> ably made this point earlier.) Part of the answer lies in understanding >>> when to use an anonymous inner class and when to use a lambda >>> expression. A step towards that understanding would be knowing how >>> anonymous inner classes are used today. >>> >>> It would be very interesting if list members could provide the following >>> statistics for codebases with which they are familiar: >>> >>> - What %age of anonymous inner classes declare a single non-Object method? >>> - What %age of single-method anonymous inner classes declare fields? >>> - What %age of single methods in anonymous inner classes use 'this'? >>> - What %age of single methods in anonymous inner classes recurse? >>> - Are the %ages different for anonymous inner classes that implement an >>> interface v. that extend a class ? >>> >>> (As well as anonymous inner classes, any utility class that >>> implements/extends a SAM type is of interest, such as this nested class >>> from the extra166y.ParallelArray Javadoc: >>> ? static final class IsSenior implements Predicate { >>> ? ? ?public boolean op(Student s) { return s.credits > 90; } >>> ? ?} >>> ) >>> >>> Alex >>> >> >> > > From neal at gafter.com Wed Feb 3 13:04:17 2010 From: neal at gafter.com (Neal Gafter) Date: Wed, 3 Feb 2010 13:04:17 -0800 Subject: SAM interface/class figures [Re: Preparing for the 0.2 draft] In-Reply-To: <4B69E2D5.6050407@sun.com> References: <96324399B8D14BCBBFF33FC79060BFF7@Labs.IntelliJ.Net> <4B69D6E7.7040608@sun.com> <15e8b9d21002031225hd5b8e82le3de15429742887d@mail.gmail.com> <4B69E2D5.6050407@sun.com> Message-ID: <15e8b9d21002031304j499db014j10e1f9af6b822ab2@mail.gmail.com> On Wed, Feb 3, 2010 at 12:55 PM, Alex Buckley wrote: > What code are you talking about? There was nothing in your mail about > OpenJDK 6 stats. Perhaps I'm having moderation problems. Enclosed are the stats I thought I had sent. ---------- Forwarded message ---------- From: Neal Gafter Date: Mon, Feb 1, 2010 at 12:58 AM Subject: Re: Preparing for the 0.2 draft To: Alex Buckley Cc: "lambda-dev at openjdk.java.net" I ran the stats for openjdk6 sources, but not quite as you asked. I started answering questions that filter out those anons that couldn't possibly be lambdas for one reason or another, and then answered the rest of the interesting questions from among those. It isn't interesting to know which anonymous inner classes use "this" if they couldn't be lambdas because there are, for example, constructor arguments. So here are the stats: Total anonymous classes: 1155 But with no constructor args: 1068 And defining only one method: 883 And where the type is a SAM type: 811 Those that are recursive: 0 Those that reference 'this': 2 (see below) Those that reference an enclosing 'this': 48 Those that are interfaces: 752 Those that are classes: 59 Total distinct interfaces: 50 (see below) Total distinct classes: 13 (see below) A couple of interesting things about the use cases exposed by this analysis: There is virtually no utility to making "this" refer to the function itself. In the two cases where "this" appears, one does not actually use the value (null would work as well), and the other could be trivially rewritten using the proposed definite-assignment rule for a lambda appearing in a variable initializer. None of the anonymous SAMs are recursive. On the other hand, there are a fair number of explicit references to the instance of the enclosing class, which would benefit from lexical scoping. I also note that the use cases involving classes do not suffer from lexically scoped lambdas because they can continue to be written exactly as they are today. Even the proposed lambda spec does not provide typesafe access to the underlying SAM type, so I can't see anywhere this code base would benefit from lambdas in which "this" references the function object rather than the lexically enclosing instance. Since Josh hadn't seen any use cases that benefit from lexical scoping of "this", I've also enclosed pointers to 48 locations in the openjdk6 source base. ==================================== Below are shown the two places where "this" appears. The first would be easily solved by a rule treating a variable as definitely assigned in the right-hand-side of its declaration if the initializing expression is a delegate. In the second, "null" would work just as well as "this", because the value is discarded by the caller (on the next line). in src/share/classes/sun/java2d/StateTrackableDelegate.java: public synchronized StateTracker getStateTracker() { StateTracker st = theTracker; if (st == null) { switch (theState) { case IMMUTABLE: st = StateTracker.ALWAYS_CURRENT; break; case STABLE: st = new StateTracker() { public boolean isCurrent() { return (theTracker == this); } }; break; .... in src/share/classes/javax/imageio/spi/IIORegistry.java PrivilegedAction doRegistration = new PrivilegedAction() { public Object run() { Iterator categories = getCategories(); while (categories.hasNext()) { Class c = (Class)categories.next(); for (IIOServiceProvider p : ServiceLoader.loadInstalled(c)) { registerServiceProvider(p); } } return this; } }; AccessController.doPrivileged(doRegistration); ==================================== Below are all the distinct classes com.sun.tools.example.debug.tty.Commands.AsyncExecution com.sun.tools.hat.internal.model.AbstractJavaHeapObjectVisitor com.sun.tools.hat.internal.util.Comparer java.awt.font.TextLine.Function java.io.InputStream java.io.OutputStream java.lang.Thread java.nio.charset.CoderResult.Cache java.util.TimerTask java.util.regex.Pattern.CharProperty java.util.regex.Pattern.CharPropertyNames.CharPropertyFactory java.util.regex.Pattern.CharPropertyNames.CloneableProperty javax.swing.AbstractAction ==================================== Below are all the distinct interfaces com.sun.java.util.jar.pack.Histogram.BitMetric com.sun.jdi.connect.Transport com.sun.jmx.remote.internal.NotificationBufferFilter com.sun.media.sound.ModelTransform com.sun.net.ssl.HostnameVerifier com.sun.security.auth.callback.DialogCallbackHandler.Action com.sun.tools.example.debug.bdi.InputListener com.sun.tools.example.debug.bdi.OutputListener com.sun.tools.example.debug.expr.ExpressionParser.GetFrame com.sun.tools.hat.internal.oql.ObjectVisitor com.sun.tools.jdi.CommandSender com.sun.tools.script.shell.Main.Command java.awt.Conditional java.awt.KeyEventPostProcessor java.awt.event.ActionListener java.awt.event.HierarchyListener java.beans.ExceptionListener java.beans.PropertyChangeListener java.beans.VetoableChangeListener java.io.FilenameFilter java.io.ObjectInputValidation java.lang.Runnable java.net.CookiePolicy java.net.HttpCookie.CookieAttributeAssignor java.security.PrivilegedAction java.security.PrivilegedExceptionAction java.util.Comparator java.util.Enumeration java.util.concurrent.Callable java.util.concurrent.Executor java.util.concurrent.ThreadFactory javax.imageio.event.IIOReadWarningListener javax.imageio.event.IIOWriteWarningListener javax.management.NotificationListener javax.swing.UIDefaults.ActiveValue javax.swing.UIDefaults.LazyValue javax.swing.event.CaretListener javax.swing.event.ChangeListener javax.swing.event.HyperlinkListener javax.xml.crypto.KeySelectorResult javax.xml.crypto.NodeSetData sun.awt.RequestFocusController sun.java2d.StateTracker sun.java2d.cmm.ProfileActivator sun.java2d.loops.ProcessPath.EndSubPathHandler sun.misc.JavaIODeleteOnExitAccess sun.misc.JavaLangAccess sun.misc.JavaNetAccess sun.nio.ch.FileChannelImpl.FileLockTable.Releaser sun.nio.ch.Interruptible ==================================== Locations in openjdk6 where lexically-scoped "this" would be helpful: src/share/classes/com/sun/java/util/jar/pack/Histogram.java:187 src/share/classes/com/sun/media/sound/SoftJitterCorrector.java:126 src/share/classes/com/sun/security/auth/PolicyFile.java:849 src/share/classes/com/sun/tools/example/debug/bdi/JDIEventSource.java:80 src/share/classes/com/sun/tools/jdi/ClassTypeImpl.java:167 src/share/classes/com/sun/tools/jdi/ClassTypeImpl.java:189 src/share/classes/com/sun/tools/jdi/ObjectReferenceImpl.java:341 src/share/classes/java/awt/Container.java:2744 src/share/classes/java/awt/Container.java:4268 src/share/classes/java/awt/Container.java:4283 src/share/classes/java/awt/EventQueue.java:823 src/share/classes/java/awt/SequencedEvent.java:92 src/share/classes/java/awt/datatransfer/Clipboard.java:128 src/share/classes/java/awt/datatransfer/Clipboard.java:326 src/share/classes/java/beans/beancontext/BeanContextSupport.java:1296 src/share/classes/java/beans/beancontext/BeanContextSupport.java:1310 src/share/classes/java/lang/Class.java:1306 src/share/classes/java/nio/channels/spi/AbstractSelector.java:208 src/share/classes/java/security/ProtectionDomain.java:317 src/share/classes/java/util/regex/Pattern.java:3352 src/share/classes/javax/management/monitor/Monitor.java:1609 src/share/classes/javax/swing/BufferStrategyPaintManager.java:213 src/share/classes/javax/swing/JComponent.java:4804 src/share/classes/javax/swing/JOptionPane.java:1011 src/share/classes/javax/swing/JOptionPane.java:1524 src/share/classes/javax/swing/ProgressMonitor.java:214 src/share/classes/javax/swing/SwingWorker.java:902 src/share/classes/javax/swing/TimerQueue.java:95 src/share/classes/javax/swing/plaf/basic/BasicLookAndFeel.java:2123 src/share/classes/javax/swing/plaf/basic/BasicPopupMenuUI.java:745 src/share/classes/javax/swing/plaf/basic/BasicPopupMenuUI.java:778 src/share/classes/javax/swing/text/JTextComponent.java:4932 src/share/classes/sun/applet/AppletClassLoader.java:631 src/share/classes/sun/applet/AppletViewer.java:641 src/share/classes/sun/awt/datatransfer/SunClipboard.java:113 src/share/classes/sun/awt/datatransfer/SunClipboard.java:287 src/share/classes/sun/awt/im/InputContext.java:648 src/share/classes/sun/security/jca/ProviderConfig.java:244 src/share/classes/sun/security/provider/SeedGenerator.java:255 src/share/classes/sun/tools/jconsole/inspector/XSheet.java:181 src/share/classes/sun/tools/jconsole/inspector/XSheet.java:216 src/share/classes/sun/tools/jconsole/inspector/XSheet.java:329 src/share/classes/sun/tools/jconsole/inspector/XSheet.java:371 src/share/classes/sun/tools/jconsole/inspector/XSheet.java:411 src/share/classes/sun/tools/jconsole/inspector/XSheet.java:668 src/share/classes/sun/tools/jconsole/inspector/XSheet.java:686 src/share/classes/sun/tools/jconsole/inspector/XTree.java:159 src/share/classes/sun/tools/jconsole/inspector/XTree.java:261 From Alex.Buckley at Sun.COM Wed Feb 3 13:06:36 2010 From: Alex.Buckley at Sun.COM (Alex Buckley) Date: Wed, 03 Feb 2010 13:06:36 -0800 Subject: SAM interface/class figures [Re: Preparing for the 0.2 draft] In-Reply-To: <15e8b9d21002031304j499db014j10e1f9af6b822ab2@mail.gmail.com> References: <96324399B8D14BCBBFF33FC79060BFF7@Labs.IntelliJ.Net> <4B69D6E7.7040608@sun.com> <15e8b9d21002031225hd5b8e82le3de15429742887d@mail.gmail.com> <4B69E2D5.6050407@sun.com> <15e8b9d21002031304j499db014j10e1f9af6b822ab2@mail.gmail.com> Message-ID: <4B69E55C.9040007@sun.com> Sorry, unclear wording. In your mail about OpenJDK 6 stats, there was no mention of a utility to gather stats. The mail about OpenJDK 6 stats, which you repeated below, got through fine the first time. Neal Gafter wrote: > On Wed, Feb 3, 2010 at 12:55 PM, Alex Buckley wrote: >> What code are you talking about? There was nothing in your mail about >> OpenJDK 6 stats. > > Perhaps I'm having moderation problems. Enclosed are the stats I > thought I had sent. > > ---------- Forwarded message ---------- > From: Neal Gafter > Date: Mon, Feb 1, 2010 at 12:58 AM > Subject: Re: Preparing for the 0.2 draft > To: Alex Buckley > Cc: "lambda-dev at openjdk.java.net" > > > I ran the stats for openjdk6 sources, but not quite as you asked. I > started answering questions that filter out those anons that couldn't > possibly be lambdas for one reason or another, and then answered the > rest of the interesting questions from among those. It isn't > interesting to know which anonymous inner classes use "this" if they > couldn't be lambdas because there are, for example, constructor > arguments. > > So here are the stats: > > Total anonymous classes: 1155 > But with no constructor args: 1068 > And defining only one method: 883 > And where the type is a SAM type: 811 > Those that are recursive: 0 > Those that reference 'this': 2 (see below) > Those that reference an enclosing 'this': 48 > Those that are interfaces: 752 > Those that are classes: 59 > Total distinct interfaces: 50 (see below) > Total distinct classes: 13 (see below) > > A couple of interesting things about the use cases exposed by this analysis: > > There is virtually no utility to making "this" refer to the function > itself. In the two cases where "this" appears, one does not actually > use the value (null would work as well), and the other could be > trivially rewritten using the proposed definite-assignment rule for a > lambda appearing in a variable initializer. None of the anonymous > SAMs are recursive. On the other hand, there are a fair number of > explicit references to the instance of the enclosing class, which > would benefit from lexical scoping. > > I also note that the use cases involving classes do not suffer from > lexically scoped lambdas because they can continue to be written > exactly as they are today. Even the proposed lambda spec does not > provide typesafe access to the underlying SAM type, so I can't see > anywhere this code base would benefit from lambdas in which "this" > references the function object rather than the lexically enclosing > instance. > > Since Josh hadn't seen any use cases that benefit from lexical scoping > of "this", I've also enclosed pointers to 48 locations in the openjdk6 > source base. > > ==================================== > Below are shown the two places where "this" appears. > > The first would be easily solved by a rule treating a variable as > definitely assigned in the right-hand-side of its declaration if the > initializing expression is a delegate. > > In the second, "null" would work just as well as "this", because the > value is discarded by the caller (on the next line). > > in src/share/classes/sun/java2d/StateTrackableDelegate.java: > public synchronized StateTracker getStateTracker() { > StateTracker st = theTracker; > if (st == null) { > switch (theState) { > case IMMUTABLE: > st = StateTracker.ALWAYS_CURRENT; > break; > case STABLE: > st = new StateTracker() { > public boolean isCurrent() { > return (theTracker == this); > } > }; > break; > .... > > in src/share/classes/javax/imageio/spi/IIORegistry.java > PrivilegedAction doRegistration = > new PrivilegedAction() { > public Object run() { > Iterator categories = getCategories(); > while (categories.hasNext()) { > Class c = (Class)categories.next(); > for (IIOServiceProvider p : > ServiceLoader.loadInstalled(c)) { > registerServiceProvider(p); > } > } > return this; > } > }; > AccessController.doPrivileged(doRegistration); > > ==================================== > Below are all the distinct classes > > com.sun.tools.example.debug.tty.Commands.AsyncExecution > com.sun.tools.hat.internal.model.AbstractJavaHeapObjectVisitor > com.sun.tools.hat.internal.util.Comparer > java.awt.font.TextLine.Function > java.io.InputStream > java.io.OutputStream > java.lang.Thread > java.nio.charset.CoderResult.Cache > java.util.TimerTask > java.util.regex.Pattern.CharProperty > java.util.regex.Pattern.CharPropertyNames.CharPropertyFactory > java.util.regex.Pattern.CharPropertyNames.CloneableProperty > javax.swing.AbstractAction > > > ==================================== > Below are all the distinct interfaces > > com.sun.java.util.jar.pack.Histogram.BitMetric > com.sun.jdi.connect.Transport > com.sun.jmx.remote.internal.NotificationBufferFilter > com.sun.media.sound.ModelTransform > com.sun.net.ssl.HostnameVerifier > com.sun.security.auth.callback.DialogCallbackHandler.Action > com.sun.tools.example.debug.bdi.InputListener > com.sun.tools.example.debug.bdi.OutputListener > com.sun.tools.example.debug.expr.ExpressionParser.GetFrame > com.sun.tools.hat.internal.oql.ObjectVisitor > com.sun.tools.jdi.CommandSender > com.sun.tools.script.shell.Main.Command > java.awt.Conditional > java.awt.KeyEventPostProcessor > java.awt.event.ActionListener > java.awt.event.HierarchyListener > java.beans.ExceptionListener > java.beans.PropertyChangeListener > java.beans.VetoableChangeListener > java.io.FilenameFilter > java.io.ObjectInputValidation > java.lang.Runnable > java.net.CookiePolicy > java.net.HttpCookie.CookieAttributeAssignor > java.security.PrivilegedAction > java.security.PrivilegedExceptionAction > java.util.Comparator > java.util.Enumeration > java.util.concurrent.Callable > java.util.concurrent.Executor > java.util.concurrent.ThreadFactory > javax.imageio.event.IIOReadWarningListener > javax.imageio.event.IIOWriteWarningListener > javax.management.NotificationListener > javax.swing.UIDefaults.ActiveValue > javax.swing.UIDefaults.LazyValue > javax.swing.event.CaretListener > javax.swing.event.ChangeListener > javax.swing.event.HyperlinkListener > javax.xml.crypto.KeySelectorResult > javax.xml.crypto.NodeSetData > sun.awt.RequestFocusController > sun.java2d.StateTracker > sun.java2d.cmm.ProfileActivator > sun.java2d.loops.ProcessPath.EndSubPathHandler > sun.misc.JavaIODeleteOnExitAccess > sun.misc.JavaLangAccess > sun.misc.JavaNetAccess > sun.nio.ch.FileChannelImpl.FileLockTable.Releaser > sun.nio.ch.Interruptible > > > ==================================== > > Locations in openjdk6 where lexically-scoped "this" would be helpful: > > src/share/classes/com/sun/java/util/jar/pack/Histogram.java:187 > src/share/classes/com/sun/media/sound/SoftJitterCorrector.java:126 > src/share/classes/com/sun/security/auth/PolicyFile.java:849 > src/share/classes/com/sun/tools/example/debug/bdi/JDIEventSource.java:80 > src/share/classes/com/sun/tools/jdi/ClassTypeImpl.java:167 > src/share/classes/com/sun/tools/jdi/ClassTypeImpl.java:189 > src/share/classes/com/sun/tools/jdi/ObjectReferenceImpl.java:341 > src/share/classes/java/awt/Container.java:2744 > src/share/classes/java/awt/Container.java:4268 > src/share/classes/java/awt/Container.java:4283 > src/share/classes/java/awt/EventQueue.java:823 > src/share/classes/java/awt/SequencedEvent.java:92 > src/share/classes/java/awt/datatransfer/Clipboard.java:128 > src/share/classes/java/awt/datatransfer/Clipboard.java:326 > src/share/classes/java/beans/beancontext/BeanContextSupport.java:1296 > src/share/classes/java/beans/beancontext/BeanContextSupport.java:1310 > src/share/classes/java/lang/Class.java:1306 > src/share/classes/java/nio/channels/spi/AbstractSelector.java:208 > src/share/classes/java/security/ProtectionDomain.java:317 > src/share/classes/java/util/regex/Pattern.java:3352 > src/share/classes/javax/management/monitor/Monitor.java:1609 > src/share/classes/javax/swing/BufferStrategyPaintManager.java:213 > src/share/classes/javax/swing/JComponent.java:4804 > src/share/classes/javax/swing/JOptionPane.java:1011 > src/share/classes/javax/swing/JOptionPane.java:1524 > src/share/classes/javax/swing/ProgressMonitor.java:214 > src/share/classes/javax/swing/SwingWorker.java:902 > src/share/classes/javax/swing/TimerQueue.java:95 > src/share/classes/javax/swing/plaf/basic/BasicLookAndFeel.java:2123 > src/share/classes/javax/swing/plaf/basic/BasicPopupMenuUI.java:745 > src/share/classes/javax/swing/plaf/basic/BasicPopupMenuUI.java:778 > src/share/classes/javax/swing/text/JTextComponent.java:4932 > src/share/classes/sun/applet/AppletClassLoader.java:631 > src/share/classes/sun/applet/AppletViewer.java:641 > src/share/classes/sun/awt/datatransfer/SunClipboard.java:113 > src/share/classes/sun/awt/datatransfer/SunClipboard.java:287 > src/share/classes/sun/awt/im/InputContext.java:648 > src/share/classes/sun/security/jca/ProviderConfig.java:244 > src/share/classes/sun/security/provider/SeedGenerator.java:255 > src/share/classes/sun/tools/jconsole/inspector/XSheet.java:181 > src/share/classes/sun/tools/jconsole/inspector/XSheet.java:216 > src/share/classes/sun/tools/jconsole/inspector/XSheet.java:329 > src/share/classes/sun/tools/jconsole/inspector/XSheet.java:371 > src/share/classes/sun/tools/jconsole/inspector/XSheet.java:411 > src/share/classes/sun/tools/jconsole/inspector/XSheet.java:668 > src/share/classes/sun/tools/jconsole/inspector/XSheet.java:686 > src/share/classes/sun/tools/jconsole/inspector/XTree.java:159 > src/share/classes/sun/tools/jconsole/inspector/XTree.java:261 > From neal at gafter.com Wed Feb 3 13:10:36 2010 From: neal at gafter.com (Neal Gafter) Date: Wed, 3 Feb 2010 13:10:36 -0800 Subject: Code to analyze a source base for project lambda In-Reply-To: <15e8b9d21002031255y45704467i32475a2c9b64803d@mail.gmail.com> References: <15e8b9d21002031255y45704467i32475a2c9b64803d@mail.gmail.com> Message-ID: <15e8b9d21002031310n7c7f4c4eqcc0844ad450b5efb@mail.gmail.com> The code can be downloaded from http://www.javac.info/ijavac-Main.java On Wed, Feb 3, 2010 at 12:55 PM, Neal Gafter wrote: > Apparently it was rejected by the moderator last time. ?Here it is again. > > The enclosed code (Main.java) was thrown together hastily, but it > might be useful to analyze a source base for project lambda. ?Compile > and run it with tools.jar on the classpath (but on a mac, tools.jar is > always on the classpath). ?It acts just like javac, but printing > statistics before exiting. ?I used it to process most of the openjdk6 > sources. > From markmahieu at googlemail.com Wed Feb 3 13:21:35 2010 From: markmahieu at googlemail.com (Mark Mahieu) Date: Wed, 3 Feb 2010 21:21:35 +0000 Subject: SAM interface/class figures [Re: Preparing for the 0.2 draft] In-Reply-To: <4B69E55C.9040007@sun.com> References: <96324399B8D14BCBBFF33FC79060BFF7@Labs.IntelliJ.Net> <4B69D6E7.7040608@sun.com> <15e8b9d21002031225hd5b8e82le3de15429742887d@mail.gmail.com> <4B69E2D5.6050407@sun.com> <15e8b9d21002031304j499db014j10e1f9af6b822ab2@mail.gmail.com> <4B69E55C.9040007@sun.com> Message-ID: I may be wrong, but I seem to recall that openjdk mailing lists strip attachments by default. Might be worthwhile requesting that's changed for lambda-dev if so. Mark On 3 Feb 2010, at 21:06, Alex Buckley wrote: > Sorry, unclear wording. In your mail about OpenJDK 6 stats, there was no > mention of a utility to gather stats. The mail about OpenJDK 6 stats, > which you repeated below, got through fine the first time. > > Neal Gafter wrote: >> On Wed, Feb 3, 2010 at 12:55 PM, Alex Buckley wrote: >>> What code are you talking about? There was nothing in your mail about >>> OpenJDK 6 stats. >> >> Perhaps I'm having moderation problems. Enclosed are the stats I >> thought I had sent. >> >> ---------- Forwarded message ---------- >> From: Neal Gafter >> Date: Mon, Feb 1, 2010 at 12:58 AM >> Subject: Re: Preparing for the 0.2 draft >> To: Alex Buckley >> Cc: "lambda-dev at openjdk.java.net" >> >> >> I ran the stats for openjdk6 sources, but not quite as you asked. I >> started answering questions that filter out those anons that couldn't >> possibly be lambdas for one reason or another, and then answered the >> rest of the interesting questions from among those. It isn't >> interesting to know which anonymous inner classes use "this" if they >> couldn't be lambdas because there are, for example, constructor >> arguments. >> >> So here are the stats: >> >> Total anonymous classes: 1155 >> But with no constructor args: 1068 >> And defining only one method: 883 >> And where the type is a SAM type: 811 >> Those that are recursive: 0 >> Those that reference 'this': 2 (see below) >> Those that reference an enclosing 'this': 48 >> Those that are interfaces: 752 >> Those that are classes: 59 >> Total distinct interfaces: 50 (see below) >> Total distinct classes: 13 (see below) >> >> A couple of interesting things about the use cases exposed by this analysis: >> >> There is virtually no utility to making "this" refer to the function >> itself. In the two cases where "this" appears, one does not actually >> use the value (null would work as well), and the other could be >> trivially rewritten using the proposed definite-assignment rule for a >> lambda appearing in a variable initializer. None of the anonymous >> SAMs are recursive. On the other hand, there are a fair number of >> explicit references to the instance of the enclosing class, which >> would benefit from lexical scoping. >> >> I also note that the use cases involving classes do not suffer from >> lexically scoped lambdas because they can continue to be written >> exactly as they are today. Even the proposed lambda spec does not >> provide typesafe access to the underlying SAM type, so I can't see >> anywhere this code base would benefit from lambdas in which "this" >> references the function object rather than the lexically enclosing >> instance. >> >> Since Josh hadn't seen any use cases that benefit from lexical scoping >> of "this", I've also enclosed pointers to 48 locations in the openjdk6 >> source base. >> >> ==================================== >> Below are shown the two places where "this" appears. >> >> The first would be easily solved by a rule treating a variable as >> definitely assigned in the right-hand-side of its declaration if the >> initializing expression is a delegate. >> >> In the second, "null" would work just as well as "this", because the >> value is discarded by the caller (on the next line). >> >> in src/share/classes/sun/java2d/StateTrackableDelegate.java: >> public synchronized StateTracker getStateTracker() { >> StateTracker st = theTracker; >> if (st == null) { >> switch (theState) { >> case IMMUTABLE: >> st = StateTracker.ALWAYS_CURRENT; >> break; >> case STABLE: >> st = new StateTracker() { >> public boolean isCurrent() { >> return (theTracker == this); >> } >> }; >> break; >> .... >> >> in src/share/classes/javax/imageio/spi/IIORegistry.java >> PrivilegedAction doRegistration = >> new PrivilegedAction() { >> public Object run() { >> Iterator categories = getCategories(); >> while (categories.hasNext()) { >> Class c = (Class)categories.next(); >> for (IIOServiceProvider p : >> ServiceLoader.loadInstalled(c)) { >> registerServiceProvider(p); >> } >> } >> return this; >> } >> }; >> AccessController.doPrivileged(doRegistration); >> >> ==================================== >> Below are all the distinct classes >> >> com.sun.tools.example.debug.tty.Commands.AsyncExecution >> com.sun.tools.hat.internal.model.AbstractJavaHeapObjectVisitor >> com.sun.tools.hat.internal.util.Comparer >> java.awt.font.TextLine.Function >> java.io.InputStream >> java.io.OutputStream >> java.lang.Thread >> java.nio.charset.CoderResult.Cache >> java.util.TimerTask >> java.util.regex.Pattern.CharProperty >> java.util.regex.Pattern.CharPropertyNames.CharPropertyFactory >> java.util.regex.Pattern.CharPropertyNames.CloneableProperty >> javax.swing.AbstractAction >> >> >> ==================================== >> Below are all the distinct interfaces >> >> com.sun.java.util.jar.pack.Histogram.BitMetric >> com.sun.jdi.connect.Transport >> com.sun.jmx.remote.internal.NotificationBufferFilter >> com.sun.media.sound.ModelTransform >> com.sun.net.ssl.HostnameVerifier >> com.sun.security.auth.callback.DialogCallbackHandler.Action >> com.sun.tools.example.debug.bdi.InputListener >> com.sun.tools.example.debug.bdi.OutputListener >> com.sun.tools.example.debug.expr.ExpressionParser.GetFrame >> com.sun.tools.hat.internal.oql.ObjectVisitor >> com.sun.tools.jdi.CommandSender >> com.sun.tools.script.shell.Main.Command >> java.awt.Conditional >> java.awt.KeyEventPostProcessor >> java.awt.event.ActionListener >> java.awt.event.HierarchyListener >> java.beans.ExceptionListener >> java.beans.PropertyChangeListener >> java.beans.VetoableChangeListener >> java.io.FilenameFilter >> java.io.ObjectInputValidation >> java.lang.Runnable >> java.net.CookiePolicy >> java.net.HttpCookie.CookieAttributeAssignor >> java.security.PrivilegedAction >> java.security.PrivilegedExceptionAction >> java.util.Comparator >> java.util.Enumeration >> java.util.concurrent.Callable >> java.util.concurrent.Executor >> java.util.concurrent.ThreadFactory >> javax.imageio.event.IIOReadWarningListener >> javax.imageio.event.IIOWriteWarningListener >> javax.management.NotificationListener >> javax.swing.UIDefaults.ActiveValue >> javax.swing.UIDefaults.LazyValue >> javax.swing.event.CaretListener >> javax.swing.event.ChangeListener >> javax.swing.event.HyperlinkListener >> javax.xml.crypto.KeySelectorResult >> javax.xml.crypto.NodeSetData >> sun.awt.RequestFocusController >> sun.java2d.StateTracker >> sun.java2d.cmm.ProfileActivator >> sun.java2d.loops.ProcessPath.EndSubPathHandler >> sun.misc.JavaIODeleteOnExitAccess >> sun.misc.JavaLangAccess >> sun.misc.JavaNetAccess >> sun.nio.ch.FileChannelImpl.FileLockTable.Releaser >> sun.nio.ch.Interruptible >> >> >> ==================================== >> >> Locations in openjdk6 where lexically-scoped "this" would be helpful: >> >> src/share/classes/com/sun/java/util/jar/pack/Histogram.java:187 >> src/share/classes/com/sun/media/sound/SoftJitterCorrector.java:126 >> src/share/classes/com/sun/security/auth/PolicyFile.java:849 >> src/share/classes/com/sun/tools/example/debug/bdi/JDIEventSource.java:80 >> src/share/classes/com/sun/tools/jdi/ClassTypeImpl.java:167 >> src/share/classes/com/sun/tools/jdi/ClassTypeImpl.java:189 >> src/share/classes/com/sun/tools/jdi/ObjectReferenceImpl.java:341 >> src/share/classes/java/awt/Container.java:2744 >> src/share/classes/java/awt/Container.java:4268 >> src/share/classes/java/awt/Container.java:4283 >> src/share/classes/java/awt/EventQueue.java:823 >> src/share/classes/java/awt/SequencedEvent.java:92 >> src/share/classes/java/awt/datatransfer/Clipboard.java:128 >> src/share/classes/java/awt/datatransfer/Clipboard.java:326 >> src/share/classes/java/beans/beancontext/BeanContextSupport.java:1296 >> src/share/classes/java/beans/beancontext/BeanContextSupport.java:1310 >> src/share/classes/java/lang/Class.java:1306 >> src/share/classes/java/nio/channels/spi/AbstractSelector.java:208 >> src/share/classes/java/security/ProtectionDomain.java:317 >> src/share/classes/java/util/regex/Pattern.java:3352 >> src/share/classes/javax/management/monitor/Monitor.java:1609 >> src/share/classes/javax/swing/BufferStrategyPaintManager.java:213 >> src/share/classes/javax/swing/JComponent.java:4804 >> src/share/classes/javax/swing/JOptionPane.java:1011 >> src/share/classes/javax/swing/JOptionPane.java:1524 >> src/share/classes/javax/swing/ProgressMonitor.java:214 >> src/share/classes/javax/swing/SwingWorker.java:902 >> src/share/classes/javax/swing/TimerQueue.java:95 >> src/share/classes/javax/swing/plaf/basic/BasicLookAndFeel.java:2123 >> src/share/classes/javax/swing/plaf/basic/BasicPopupMenuUI.java:745 >> src/share/classes/javax/swing/plaf/basic/BasicPopupMenuUI.java:778 >> src/share/classes/javax/swing/text/JTextComponent.java:4932 >> src/share/classes/sun/applet/AppletClassLoader.java:631 >> src/share/classes/sun/applet/AppletViewer.java:641 >> src/share/classes/sun/awt/datatransfer/SunClipboard.java:113 >> src/share/classes/sun/awt/datatransfer/SunClipboard.java:287 >> src/share/classes/sun/awt/im/InputContext.java:648 >> src/share/classes/sun/security/jca/ProviderConfig.java:244 >> src/share/classes/sun/security/provider/SeedGenerator.java:255 >> src/share/classes/sun/tools/jconsole/inspector/XSheet.java:181 >> src/share/classes/sun/tools/jconsole/inspector/XSheet.java:216 >> src/share/classes/sun/tools/jconsole/inspector/XSheet.java:329 >> src/share/classes/sun/tools/jconsole/inspector/XSheet.java:371 >> src/share/classes/sun/tools/jconsole/inspector/XSheet.java:411 >> src/share/classes/sun/tools/jconsole/inspector/XSheet.java:668 >> src/share/classes/sun/tools/jconsole/inspector/XSheet.java:686 >> src/share/classes/sun/tools/jconsole/inspector/XTree.java:159 >> src/share/classes/sun/tools/jconsole/inspector/XTree.java:261 >> > From lk at teamten.com Wed Feb 3 15:27:22 2010 From: lk at teamten.com (Lawrence Kesteloot) Date: Wed, 3 Feb 2010 15:27:22 -0800 Subject: Code to analyze a source base for project lambda In-Reply-To: <15e8b9d21002031310n7c7f4c4eqcc0844ad450b5efb@mail.gmail.com> References: <15e8b9d21002031255y45704467i32475a2c9b64803d@mail.gmail.com> <15e8b9d21002031310n7c7f4c4eqcc0844ad450b5efb@mail.gmail.com> Message-ID: <997cab101002031527m53195e91te9ff51de57d7cfd1@mail.gmail.com> LOC: about 100K Total anonymous classes: 98 But with no constructor args: 79 And defining only one method: 66 And where the type is a SAM type: 57 Those that are recursive: 0 Those that reference 'this': 0 company.MonitoredDataSource.java:146: [Anon, OneMethodOnly, NeedsMethod, SuperThis] uses an enclosing 'this' company.MonitoredDataSource.java:157: [Anon, OneMethodOnly, NeedsMethod, SuperThis] uses an enclosing 'this' Those that reference an enclosing 'this': 2 Those that are interfaces: 41 Those that are classes: 16 Total distinct interfaces: 13 company.Function java.io.FilenameFilter java.lang.Iterable java.lang.Runnable java.util.Comparator java.util.concurrent.Callable java.util.concurrent.ThreadFactory org.springframework.jdbc.core.PreparedStatementCreator org.springframework.jdbc.core.ResultSetExtractor org.springframework.jdbc.core.RowCallbackHandler org.springframework.jdbc.core.RowMapper org.springframework.jdbc.core.StatementCallback org.springframework.jdbc.core.simple.ParameterizedRowMapper Total distinct classes: 4 company.Filter company.LazyLoadingMap java.util.TimerTask org.springframework.transaction.support.TransactionCallbackWithoutResult On Wed, Feb 3, 2010 at 1:10 PM, Neal Gafter wrote: > The code can be downloaded from http://www.javac.info/ijavac-Main.java > > On Wed, Feb 3, 2010 at 12:55 PM, Neal Gafter wrote: >> Apparently it was rejected by the moderator last time. ?Here it is again. >> >> The enclosed code (Main.java) was thrown together hastily, but it >> might be useful to analyze a source base for project lambda. ?Compile >> and run it with tools.jar on the classpath (but on a mac, tools.jar is >> always on the classpath). ?It acts just like javac, but printing >> statistics before exiting. ?I used it to process most of the openjdk6 >> sources. >> > > From peter.levart at gmail.com Wed Feb 3 15:55:03 2010 From: peter.levart at gmail.com (Peter Levart) Date: Thu, 4 Feb 2010 00:55:03 +0100 Subject: SAM interface/class figures [Re: Preparing for the 0.2 draft] In-Reply-To: <4B69E55C.9040007@sun.com> References: <96324399B8D14BCBBFF33FC79060BFF7@Labs.IntelliJ.Net> <15e8b9d21002031304j499db014j10e1f9af6b822ab2@mail.gmail.com> <4B69E55C.9040007@sun.com> Message-ID: <201002040055.03992.peter.levart@gmail.com> I ran Neal's analyzer over my humble little project: Total anonymous classes: 747 But with no constructor args: 617 And defining only one method: 456 And where the type is a SAM type: 437 Those that are recursive: 0 Those that reference 'this': 0 Those that reference an enclosing 'this': 6 Those that are interfaces: 365 Those that are classes: 72 Total distinct interfaces: 46 Total distinct classes: 22 Since the project is built with maven, I got one report per module. I used the following perl script to aggregate the results back into one report: #!/usr/bin/perl my $total_an_classes = 0; my $but_not_constr = 0; my $def_one_method = 0; my $type_is_sam = 0; my $sam_recursive = 0; my $sam_ref_this = 0; my $sam_ref_encl_this = 0; my $sam_interf = 0; my $sam_class = 0; my $interf = 0; my $class = 0; my %dist_interf = (); my %dist_class = (); while () { chomp; if (/^Total anonymous classes\:\s*(\d+)/) { $total_an_classes += $1; $interf = 0; $class = 0; } elsif (/^But with no constructor args\:\s*(\d+)/) { $but_not_constr += $1; } elsif (/^And defining only one method\:\s*(\d+)/) { $def_one_method += $1; } elsif (/^And where the type is a SAM type\:\s*(\d+)/) { $type_is_sam += $1; } elsif (/^ Those that are recursive\:\s*(\d+)/) { $sam_recursive += $1; } elsif (/^ Those that reference \'this\'\:\s*(\d+)/) { $sam_ref_this += $1; } elsif (/^ Those that reference an enclosing \'this\'\:\s*(\d+)/) { $sam_ref_encl_this += $1; } elsif (/^ Those that are interfaces\:\s*(\d+)/) { $sam_interf += $1; } elsif (/^ Those that are classes\:\s*(\d+)/) { $sam_class += $1; } elsif (/^Total distinct interfaces\:\s*(\d+)/) { $interf = 1; } elsif (/^Total distinct classes\:\s*(\d+)/) { $interf = 0; $class = 1; } elsif ($interf && /^ (\S+)/) { $dist_interf{$1}++; } elsif ($class && /^ (\S+)/) { $dist_class{$1}++; } else { $interf = 0; $class = 0; } } print < References: <4B62A2CE.7080300@sun.com> <15e8b9d21002010058w49c686e3i325627d6362614d8@mail.gmail.com> Message-ID: <17b2302a1002032156g7b6b7005l2d6c79c79c390f37@mail.gmail.com> Neal found 48 locations in OpenJDK 6 in which he claimed that evaluating "this" in the surrounding context would be helpful. I decided to take a closer look at them in hopes of learning something. Roughly speaking, here's what I found. Of the 48 locations, 32 appear to be "legitimate" (in that a qualified "this" is currently required, and could be replaced by an unqualified this if it were evaluated in the enclosing scope. Of these 32, 14 are calls to AccessController.doPrivileged, 12 are calls to EventQueue.invokeLater, and 5 add or remove PropertyChangeListeners. Some of these 32 instance creations also contain Of the remaining 16 locations, 4 are "superfluous" (the qualified this could be removed without harm to the code); 4 could not take advantage of the "enclosing scope semantics for this," because the anonymous class has a method that shadows the one in the enclosing class (see (1) below for an example); 2 require a "this" only because a local variable or parameter shadows a field in the enclosing class (the local variable or parameter should be renamed); and 6 aren't "legitimate" (as defined above) but don't fit into any of the above categories. For amusement, take a look at (22) below. Unless I'm deeply mistaken, it's an in-your-face Swing bug. I have no idea how it eluded detection. There is a more subtle Swing bug (race condition) in (23). I don't believe this constitutes a compelling case for interpreting this in the enclosing context, but I understands that others may differ on this point. Here, for masochists, are brief discussions of all 48 locations identified by Neal, including the code: (1) src/share/classes/com/sun/java/util/jar/pack/Histogram.java:187 class Histogram { public double getBitLength(int value) { double prob = (double) getFrequency(value) / getTotalWeight(); return - Math.log(prob) / log2; } private final BitMetric bitMetric = new BitMetric() { public double getBitLength(int value) { return Histogram.this.getBitLength(value); } }; } The qualified this is necessary only because the anonymous class gives its sole method the same name as the one that it wants to call in its enclosing instance. But you can't name the method in a lambda expression, so having "this" evaluated in the surrounding context would not be necessary (or helpful) here. (2) src/share/classes/com/sun/media/sound/SoftJitterCorrector.java:126 The relevant code looks like this: synchronized (JitterStream.this) { if (!active) break; } The only other code that synchronizes on JitterStream instances is this: synchronized (this) { active = false; } You could remove all of the synchronization and improve the code's beauty and performance by declaring active to be volatile. (3) src/share/classes/com/sun/security/auth/PolicyFile.java:849 The qualified this would be unnecessary if PolicyPermissions had been declared as a (nonstatic) nested class of PolicyFile. And the code would have been simpler and cleaner. Also note this class has been "entirely deprecated." (4) src/share/classes/com/sun/tools/example/debug/bdi/JDIEventSource.java:80 Here's the code: boolean wantInterrupt = JDIEventSource.this.wantInterrupt; The qualified this is necessary only because the local variable has the same name as the field in the enclosing class, hence the local variable shadows the field. If the local variable is given a different name, the resulting code requires no "this," qualified or otherwise. (5) src/share/classes/com/sun/tools/jdi/ClassTypeImpl.java:167 This appears to be a legitimate use: CommandSender sender = new CommandSender() { public PacketStream send() { return JDWP.ClassType.InvokeMethod.enqueueCommand( vm, ClassTypeImpl.this, thread, method.ref(), args, options); } }; In other words, you could eliminate the characters "ClassTypeImpl." if "this" were evaluated in the surrounding context. (Whether if would be more natural or not is a matter of debate.) (6) src/share/classes/com/sun/tools/jdi/ClassTypeImpl.java:189 This is a copy-and-paste of (5), and also legitimate: CommandSender sender = new CommandSender() { public PacketStream send() { return JDWP.ClassType.NewInstance.enqueueCommand( vm, ClassTypeImpl.this, thread, method.ref(), args, options); } }; (7) src/share/classes/com/sun/tools/jdi/ObjectReferenceImpl.java:341 Another copy-and-paste of (5): CommandSender sender = new CommandSender() { public PacketStream send() { return JDWP.ObjectReference.InvokeMethod.enqueueCommand( vm, ObjectReferenceImpl.this, thread, refType, method.ref(), args, options); } }; (8) src/share/classes/java/awt/Container.java:2744 Seems legitimate. Runnable pumpEventsForHierarchy = new Runnable() { public void run() { EventDispatchThread dispatchThread = (EventDispatchThread)Thread.currentThread(); dispatchThread.pumpEventsForHierarchy( new Conditional() { public boolean evaluate() { return ((windowClosingException == null) && (nativeContainer.modalComp != null)) ; } }, Container.this); } }; (9) src/share/classes/java/awt/Container.java:4268 Seems legitimate. java.security.AccessController.doPrivileged( new java.security.PrivilegedAction() { public Object run() { nativeContainer.getToolkit().addAWTEventListener( LightweightDispatcher.this, AWTEvent.MOUSE_EVENT_MASK | AWTEvent.MOUSE_MOTION_EVENT_MASK); return null; } } ); (10) src/share/classes/java/awt/Container.java:4283 Complementary to (9). Also legitimate. java.security.AccessController.doPrivileged( new java.security.PrivilegedAction() { public Object run() { nativeContainer.getToolkit().removeAWTEventListener(LightweightDispatcher.this); return null; } } ); (11) src/share/classes/java/awt/EventQueue.java:823 Seems legitimate: dispatchThread = (EventDispatchThread) AccessController.doPrivileged(new PrivilegedAction() { public Object run() { EventDispatchThread t = new EventDispatchThread(threadGroup, name, EventQueue.this); t.setContextClassLoader(classLoader); t.setPriority(Thread.NORM_PRIORITY + 1); t.setDaemon(false); return t; } }); (12) src/share/classes/java/awt/SequencedEvent.java:92 Superfluous. You could remove the characters "SequencedEvent.this." to make the code shorter and clearer: edt.pumpEvents(SentEvent.ID, new Conditional() { public boolean evaluate() { return !SequencedEvent.this.isFirstOrDisposed(); } }); (13) src/share/classes/java/awt/datatransfer/Clipboard.java:128 Appears legitimate: EventQueue.invokeLater(new Runnable() { public void run() { oldOwner.lostOwnership(Clipboard.this, oldContents); } }); (14) src/share/classes/java/awt/datatransfer/Clipboard.java:326 Appears legitimate: EventQueue.invokeLater(new Runnable() { public void run() { listener.flavorsChanged(new FlavorEvent(Clipboard.this)); } }); (15) src/share/classes/java/beans/beancontext/BeanContextSupport.java:1296 Like (1). Evaluating "this" in the surrounding context would not be helpful. public void propertyChange(PropertyChangeEvent pce) { BeanContextSupport.this.propertyChange(pce); } (16) src/share/classes/java/beans/beancontext/BeanContextSupport.java:1310 Like (1). public void vetoableChange(PropertyChangeEvent pce) throws PropertyVetoException { BeanContextSupport.this.vetoableChange(pce); } (17) src/share/classes/java/lang/Class.java:1306 Appears legitimate: Class[] result = (Class[]) java.security.AccessController.doPrivileged (new java.security.PrivilegedAction() { public Object run() { java.util.List list = new java.util.ArrayList(); Class currentClass = Class.this; while (currentClass != null) { Class[] members = currentClass.getDeclaredClasses(); for (int i = 0; i < members.length; i++) { if (Modifier.isPublic(members[i].getModifiers())) { list.add(members[i]); } } currentClass = currentClass.getSuperclass(); } Class[] empty = {}; return list.toArray(empty); } }); (18) src/share/classes/java/nio/channels/spi/AbstractSelector.java:208 Superfluous. Could (and should) remove "AbstractSelector.this." without harm to the code. interruptor = new Interruptible() { public void interrupt() { AbstractSelector.this.wakeup(); }}; (19) src/share/classes/java/security/ProtectionDomain.java:317 Appears legitimate: PermissionCollection perms = java.security.AccessController.doPrivileged (new java.security.PrivilegedAction() { public PermissionCollection run() { Policy p = Policy.getPolicyNoCheck(); return p.getPermissions(ProtectionDomain.this); } }); (20) src/share/classes/java/util/regex/Pattern.java:3352 Like (1). private static abstract class CharProperty extends Node { abstract boolean isSatisfiedBy(int ch); CharProperty complement() { return new CharProperty() { boolean isSatisfiedBy(int ch) { return ! CharProperty.this.isSatisfiedBy(ch);}}; } (21) src/share/classes/javax/management/monitor/Monitor.java:1609 Superfluous (and repeated 4 times!). PrivilegedAction action = new PrivilegedAction() { public Void run() { if (Monitor.this.isActive()) { final int an[] = alreadyNotifieds; int index = 0; for (ObservedObject o : Monitor.this.observedObjects) { if (Monitor.this.isActive()) { Monitor.this.monitor(o, index++, an); } } } return null; } }; (22) src/share/classes/javax/swing/BufferStrategyPaintManager.java:213 OH MY GOD! This code has four uses of this. Three are silly (necessitated by the fact that the the run method has a local variable with the same name as a field in the enclosing class, as in (4) above). One is horribly broken: SwingUtilities.invokeLater(new Runnable() { public void run() { java.util.List bufferInfos; synchronized(BufferStrategyPaintManager.this) { while (showing) { try { wait(); } catch (InterruptedException ie) { } } bufferInfos = BufferStrategyPaintManager.this.bufferInfos; BufferStrategyPaintManager.this.bufferInfos = null; } dispose(bufferInfos); } }); Note that the code is waiting on the anonymous instance, but synchronizing on the containing BufferStrategyPaintManager instance. This will throw an IllegalMonitorStateException as soon as it waits! This is definitely a bug. I wonder why it hasn't been reported. So what's the right solution? You could fix this by using BufferStrategyPaintManager.this twice, but this is ugly and error prone. Better would be to uses something higher-level than wait/notify. If you did that, you wouldn't need to access BufferStrategyPaintManager.this at all. (23) src/share/classes/javax/swing/JComponent.java:4804 This is also broken. The anonymous Runnable goes to great lengths to synchronize on its outer JComponent instance before calling setFlag, but there are plenty of public methods in the same file that call setFlag without synchronizing: Runnable callRevalidate = new Runnable() { public void run() { synchronized(JComponent.this) { setFlag(REVALIDATE_RUNNABLE_SCHEDULED, false); } revalidate(); } }; SwingUtilities.invokeLater(callRevalidate); (24) src/share/classes/javax/swing/JOptionPane.java:1011 Appears legitimate: addPropertyChangeListener(new PropertyChangeListener() { public void propertyChange(PropertyChangeEvent event) { // Let the defaultCloseOperation handle the closing // if the user closed the window without selecting a button // (newValue = null in that case). Otherwise, close the dialog. if (dialog.isVisible() && event.getSource() == JOptionPane.this && (event.getPropertyName().equals(VALUE_PROPERTY) || event.getPropertyName().equals(INPUT_VALUE_PROPERTY)) && event.getNewValue() != null && event.getNewValue() != JOptionPane.UNINITIALIZED_VALUE) { dialog.setVisible(false); } } }); (25) src/share/classes/javax/swing/JOptionPane.java:1524 Copy-and-paste from 24. if (iFrame.isVisible() && event.getSource() == JOptionPane.this && event.getPropertyName().equals(VALUE_PROPERTY)) { (26) src/share/classes/javax/swing/ProgressMonitor.java:214 Essentially a copy-and-paste from 24. addPropertyChangeListener(new PropertyChangeListener() { public void propertyChange(PropertyChangeEvent event) { if(dialog.isVisible() && event.getSource() == ProgressOptionPane.this && (event.getPropertyName().equals(VALUE_PROPERTY) || event.getPropertyName().equals(INPUT_VALUE_PROPERTY))){ dialog.setVisible(false); dialog.dispose(); } } }); (27) src/share/classes/javax/swing/SwingWorker.java:902 Near as I can tell, the "SwingWorkerPropertyChangeSupport.this." is superfluous: public void firePropertyChange(final PropertyChangeEvent evt) { if (SwingUtilities.isEventDispatchThread()) { super.firePropertyChange(evt); } else { doSubmit.add( new Runnable() { public void run() { SwingWorkerPropertyChangeSupport.this .firePropertyChange(evt); } }); } (28) src/share/classes/javax/swing/TimerQueue.java:95 Appears legitimate: java.security.AccessController.doPrivileged( new java.security.PrivilegedAction() { public Object run() { Thread timerThread = new Thread(threadGroup, TimerQueue.this, "TimerQueue"); timerThread.setDaemon(true); timerThread.setPriority(Thread.NORM_PRIORITY); timerThread.start(); return null; } }); running = true; } (29) src/share/classes/javax/swing/plaf/basic/BasicLookAndFeel.java:2123 Appears legitimate: byte[] buffer = (byte[])AccessController.doPrivileged( new PrivilegedAction() { public Object run() { try { InputStream resource = BasicLookAndFeel.this. getClass().getResourceAsStream(soundFile); (30) src/share/classes/javax/swing/plaf/basic/BasicPopupMenuUI.java:745 Appears legitimate: java.security.AccessController.doPrivileged( new java.security.PrivilegedAction() { public Object run() { tk.addAWTEventListener(MouseGrabber.this, AWTEvent.MOUSE_EVENT_MASK | AWTEvent.MOUSE_MOTION_EVENT_MASK | AWTEvent.MOUSE_WHEEL_EVENT_MASK | AWTEvent.WINDOW_EVENT_MASK | SunToolkit.GRAB_EVENT_MASK); return null; } } ); (31) src/share/classes/javax/swing/plaf/basic/BasicPopupMenuUI.java:778 Appears legitimate (complementary to (30)) java.security.AccessController.doPrivileged( new java.security.PrivilegedAction() { public Object run() { tk.removeAWTEventListener(MouseGrabber.this); return null; } } ); (32) src/share/classes/javax/swing/text/JTextComponent.java:4932 Looks legitimate (though ripe for a NullPointerException: isProcessInputMethodEventOverridden returns Boolean): Boolean ret = (Boolean)AccessController.doPrivileged(new PrivilegedAction() { public Object run() { return isProcessInputMethodEventOverridden( JTextComponent.this.getClass()); } }); (33) src/share/classes/sun/applet/AppletClassLoader.java:631 Looks legit (albeit obsolete in several ways): AccessController.doPrivileged(new PrivilegedAction() { public Object run() { threadGroup = new AppletThreadGroup(base + "-threadGroup"); AppContextCreator creatorThread = new AppContextCreator(threadGroup); creatorThread.setContextClassLoader(AppletClassLoader.this); ... } }); (34) src/share/classes/sun/applet/AppletViewer.java:641 Like (33), legit but obsolete: void appletSave() { AccessController.doPrivileged(new PrivilegedAction() { public Object run() { panel.sendEvent(AppletPanel.APPLET_STOP); FileDialog fd = new FileDialog(AppletViewer.this, amh.getMessage("appletsave.filedialogtitle"), FileDialog.SAVE); ... (35) src/share/classes/sun/awt/datatransfer/SunClipboard.java:113 Appears legitimate. Many uses of sunClipboard (= SunClipboard.this) are superfluous, but a couple are required (synchronization and callbacks). final Runnable runnable = new Runnable() { public void run() { final SunClipboard sunClipboard = SunClipboard.this; ClipboardOwner owner = null; Transferable contents = null; synchronized (sunClipboard) { final AppContext context = sunClipboard.contentsContext; if (context == null) { return; } if (disposedContext == null || context == disposedContext) { owner = sunClipboard.owner; contents = sunClipboard.contents; sunClipboard.contentsContext = null; sunClipboard.owner = null; sunClipboard.contents = null; sunClipboard.clearNativeContext(); context.removePropertyChangeListener (AppContext.DISPOSED_PROPERTY_NAME, sunClipboard); } else { return; } } if (owner != null) { owner.lostOwnership(sunClipboard, contents); } } }; (36) src/share/classes/sun/awt/datatransfer/SunClipboard.java:287 Copy-and-paste from 35. (37) src/share/classes/sun/awt/im/InputContext.java:648 Highly suspicious. The only reason for the qualified this is so that it can be cast to a subclass. This method belongs in the subclass. If it were their, the qualified this would be superfluous: EventQueue.invokeLater(new Runnable() { public void run() { ((InputMethodContext)InputContext.this).releaseCompositionArea(); } }); (38) src/share/classes/sun/security/jca/ProviderConfig.java:244 The debug println could be moved out of the runnable, eliminating the need for the "qualified this": return AccessController.doPrivileged(new PrivilegedAction() { public Provider run() { if (debug != null) { debug.println("Loading provider: " + ProviderConfig.this); } (39) src/share/classes/sun/security/provider/SeedGenerator.java:255 Appears legitimate: Thread t = java.security.AccessController.doPrivileged (new java.security.PrivilegedAction() { public Thread run() { ThreadGroup parent, group = Thread.currentThread().getThreadGroup(); while ((parent = group.getParent()) != null) group = parent; finalsg[0] = new ThreadGroup (group, "SeedGenerator ThreadGroup"); Thread newT = new Thread(finalsg[0], ThreadedSeedGenerator.this, "SeedGenerator Thread"); newT.setPriority(Thread.MIN_PRIORITY); newT.setDaemon(true); return newT; } }); (40) src/share/classes/sun/tools/jconsole/inspector/XSheet.java:181 Three uses of qualified this. The first is only needed because a parameter shadows a field. The second is purely extraneous. The third seems legitimate: private void displayMBeanNode(final DefaultMutableTreeNode node) { final XNodeInfo uo = (XNodeInfo) node.getUserObject(); if (!uo.getType().equals(Type.MBEAN)) { return; } mbeansTab.workerAdd(new Runnable() { public void run() { try { XSheet.this.node = node; // 1 XSheet.this.mbean = (XMBean) uo.getData(); // 2 mbeanInfo.addMBeanInfo(mbean, mbean.getMBeanInfo()); } catch (Throwable ex) { EventQueue.invokeLater(new ThreadDialog( XSheet.this, // 3 ex.getMessage(), Resources.getText("Problem displaying MBean"), JOptionPane.ERROR_MESSAGE)); return; } ... (41) src/share/classes/sun/tools/jconsole/inspector/XSheet.java:216 Copy-and-paste of 40. (42) src/share/classes/sun/tools/jconsole/inspector/XSheet.java:329 Copy-and-paste of 40. (43) src/share/classes/sun/tools/jconsole/inspector/XSheet.java:371 Copy-and-paste of 40. (44) src/share/classes/sun/tools/jconsole/inspector/XSheet.java:411 Copy-and-paste of 40. (45) src/share/classes/sun/tools/jconsole/inspector/XSheet.java:668 Like 40, but only the legit use. (46) src/share/classes/sun/tools/jconsole/inspector/XSheet.java:686 Like 40, but only the legit use. (47) src/share/classes/sun/tools/jconsole/inspector/XTree.java:159 The qualified this is used to get an object on which to synchronize. It may be the case that the only thread that ever synchronizes on this object is the event dispatch thread. If this is true, then the synchronization has no effect, but I'm not in a position to say for sure: public synchronized void addMBeanToView(final ObjectName mbean) { final XMBean xmbean; try { xmbean = new XMBean(mbean, mbeansTab); if (xmbean == null) { return; } } catch (Exception e) { // Got exception while trying to retrieve the // given MBean from the underlying MBeanServer // if (JConsole.isDebug()) { e.printStackTrace(); } return; } EventQueue.invokeLater(new Runnable() { public void run() { synchronized (XTree.this) { // Add the new nodes to the MBean tree from leaf to root ... (48) src/share/classes/sun/tools/jconsole/inspector/XTree.java:261 Like 47. From neal at gafter.com Wed Feb 3 22:25:11 2010 From: neal at gafter.com (Neal Gafter) Date: Wed, 3 Feb 2010 22:25:11 -0800 Subject: Preparing for the 0.2 draft In-Reply-To: <17b2302a1002032156g7b6b7005l2d6c79c79c390f37@mail.gmail.com> References: <4B62A2CE.7080300@sun.com> <15e8b9d21002010058w49c686e3i325627d6362614d8@mail.gmail.com> <17b2302a1002032156g7b6b7005l2d6c79c79c390f37@mail.gmail.com> Message-ID: <15e8b9d21002032225p4d33430cu5e679cddcb4e7f8f@mail.gmail.com> On Wed, Feb 3, 2010 at 9:56 PM, Joshua Bloch wrote: > Neal found 48 locations in OpenJDK 6 in which he claimed that evaluating > "this" in the surrounding context would be helpful. I decided to take a > closer look at them in hopes of learning something. > > Roughly speaking, here's what I found. Of the 48 locations, 32 appear to be > "legitimate" (in that a qualified "this" is currently required, and could be > replaced by an unqualified this if it were evaluated in the enclosing scope. Thanks, Josh. It's nice to see verification that the automatically-generated results bear some resemblance to the kind of deeper analysis it was a proxy for. Cheers, Neal From peter.levart at marand.si Wed Feb 3 23:54:35 2010 From: peter.levart at marand.si (Peter Levart) Date: Thu, 4 Feb 2010 08:54:35 +0100 Subject: Preparing for the 0.2 draft In-Reply-To: <17b2302a1002032156g7b6b7005l2d6c79c79c390f37@mail.gmail.com> References: <4B62A2CE.7080300@sun.com> <15e8b9d21002010058w49c686e3i325627d6362614d8@mail.gmail.com> <17b2302a1002032156g7b6b7005l2d6c79c79c390f37@mail.gmail.com> Message-ID: <201002040854.35273.peter.levart@marand.si> On Thursday 04 February 2010 06:56:35 Joshua Bloch wrote: > (22) src/share/classes/javax/swing/BufferStrategyPaintManager.java:213 > > OH MY GOD! This code has four uses of this. Three are silly (necessitated > by the fact that the the run method has a local variable with the same name > as a field in the enclosing class, as in (4) above). One is horribly > broken: > > SwingUtilities.invokeLater(new Runnable() { > public void run() { > java.util.List bufferInfos; > synchronized(BufferStrategyPaintManager.this) { > while (showing) { > try { > wait(); > } catch (InterruptedException ie) { > } > } > bufferInfos = > BufferStrategyPaintManager.this.bufferInfos; > BufferStrategyPaintManager.this.bufferInfos = null; > } > dispose(bufferInfos); > } > }); > > Note that the code is waiting on the anonymous instance, but synchronizing > on the containing BufferStrategyPaintManager instance. This will throw an > IllegalMonitorStateException as soon as it waits! This is definitely a bug. > I wonder why it hasn't been reported. > > So what's the right solution? You could fix this by using > BufferStrategyPaintManager.this twice, but this is ugly and error prone. > Better would be to uses something higher-level than wait/notify. If you did > that, you wouldn't need to access BufferStrategyPaintManager.this at all. > Was this code written before the JDK 5 days maybe? I think that what is interesting is that it shows an example where the block of code, posing as Runnable, has no identity of it's own - it goes to great lengths in an attempt to be executed in the surrounding context, not succeeding entirely as a call to wait() escaped the programmer's eye. This is the class of examples that would benefit from entirely transparent lambdas. Lambdas where even unqualified "equals(), hashCode(), wait(), notify(), getClass()" were executed in the surrounding context. Let's see how such lambdas would affect the rewrite of this example: /** * Cleans up any created BufferStrategies. */ protected void dispose() { // dipose can be invoked at any random time. To avoid // threading dependancies we do the actual diposing via an // invokeLater. SwingUtilities.invokeLater(#() { java.util.List bufferInfos; synchronized(this) { while (showing) { try { wait(); } catch (InterruptedException ie) { } } bufferInfos = this.bufferInfos; this.bufferInfos = null; } dispose(bufferInfos); }); } I don't know, but am I the only one that notices the beauty and simplicity of such solution? Regards, Peter From peter.levart at marand.si Thu Feb 4 01:00:57 2010 From: peter.levart at marand.si (Peter Levart) Date: Thu, 4 Feb 2010 10:00:57 +0100 Subject: What is the meaning of this? In-Reply-To: <17b2302a1002021435i685ffa00u6184fc26c9c1cbd4@mail.gmail.com> References: <17b2302a1001281119t4ea1c34cya8a05e59864b407@mail.gmail.com> <201001311039.55711.peter.levart@gmail.com> <17b2302a1002021435i685ffa00u6184fc26c9c1cbd4@mail.gmail.com> Message-ID: <201002041000.57864.peter.levart@marand.si> On Tuesday 02 February 2010 23:35:01 Joshua Bloch wrote: > Peter, > > This is an interesting idea. It certainly provides a way to access the > function object from within its definition without polluting the surrounding > scope, which is a good thing. Of course the ability to do non-local returns > scares the crap out of me. Heck, I feel unclean when I do a labeled break > in the language as it currently exists. > > Josh > > P.S. Your idea still leaves unanswered the question of what "this" *should > *represent in the body of a lambda. I'm a proponent of the following: - unqualified (this, equals(), hashCode(), wait(), notify(), notifyAll(), toString()) should be evaluated in the lambda's enclosing scope - any unqualified fields or methods should be evaluated in the lambda's enclosing scope even if lambda is converted to a SAM subclass with plenty of members. - the only way to obtain a reference to a function or SAM object is via a variable: final Runnable code = #() { ...; code.run(); ... } // or maybe (but not prefferably) #code() { ...; code.(); ... } I look at all that from the syntax perspective. I assert that when a die-hard-one-of-9-milion Java programmer sees an unqualified variable name or method invocation, she searches it's definition from the first enclosing class outwards (taking into account local variables that might shadow other vars). The syntax of a lambda expression does not look like a class declaration - it contains no CamelCasedName in the header and no method declaration inside like all top/nested/anonymous classes - in frequent cases it boils down to 3 characters: '#()' - it looks more like a simple block of code with minimum header that represents formal parameters. So when Alex says: "with nine million Java developers, designing something that looks familiar is a feature, not a bug.", he should ask himself what does the feature he is designing look familiar to. Regards, Peter > > On Sun, Jan 31, 2010 at 1:39 AM, Peter Levart wrote: > > > > LambdaExpression: > > '#' Identifier_opt '(' FormalParameters_opt ')' Block > > '#' Identifier_opt '(' FormalParameters_opt ')' '(' Expression ')' > > > > Where "Identifier" represents a final local variable valid in the scope of > > "Block" or "Expression" > > with the function (or SAM) type of a lambda expression. The factorial > > example would then read: > > > > #int(int) factorial = #fact(int i)(i == 0 ? 1 : i * (int) (fact.(i - 1))); > > > > This would enable referencing outer instances of lambda expressions from > > within nested lambda > > expressions, a feature not possible with anonymous inner classes today. > > > > The optional "Identifier" could be used for other purposes: > > > > break Identifier; // non-value-bearing break from the statement lambda > > break Identifier : Expression; // value-bearing break from the > > statement lambda > > > > In addition, if we make lambdas Serializable, the identifier could help > > determine the name of the > > class (or a method if java.dyn.MethodReference is used for implementation) > > generated by the > > compiler so that moving lambda expression around within the code doesn't > > break serialization. > > > > Regards, Peter > > > > From markmahieu at googlemail.com Thu Feb 4 12:27:41 2010 From: markmahieu at googlemail.com (Mark Mahieu) Date: Thu, 4 Feb 2010 20:27:41 +0000 Subject: Preparing for the 0.2 draft In-Reply-To: <17b2302a1002032156g7b6b7005l2d6c79c79c390f37@mail.gmail.com> References: <4B62A2CE.7080300@sun.com> <15e8b9d21002010058w49c686e3i325627d6362614d8@mail.gmail.com> <17b2302a1002032156g7b6b7005l2d6c79c79c390f37@mail.gmail.com> Message-ID: <0909768B-4A52-4F33-80E8-F4D3F10033B8@googlemail.com> On 4 Feb 2010, at 05:56, Joshua Bloch wrote: > Neal found 48 locations in OpenJDK 6 in which he claimed that evaluating > "this" in the surrounding context would be helpful. I decided to take a > closer look at them in hopes of learning something. > 4 could not take advantage of the > "enclosing scope semantics for this," because the anonymous class has a > method that shadows the one in the enclosing class (see (1) below for an > example) If "this" were evaluated in the context of the enclosing class, how could shadowing come into play here? Wouldn't the former preclude the latter? > (1) src/share/classes/com/sun/java/util/jar/pack/Histogram.java:187 > > class Histogram { > public double getBitLength(int value) { > double prob = (double) getFrequency(value) / getTotalWeight(); > return - Math.log(prob) / log2; > } > > private final BitMetric bitMetric = new BitMetric() { > public double getBitLength(int value) { > return Histogram.this.getBitLength(value); > } > }; > } > > The qualified this is necessary only because the anonymous class gives its > sole method the same name as the one that it wants to call in its enclosing > instance. But you can't name the method in a lambda expression, so having > "this" evaluated in the surrounding context would not be necessary (or > helpful) here. The method is named anyway, by the BitMetric interface. Surely it's whether or not that is visible in the body of an equivalent lambda which determines how helpful a given approach is? For this particular example, the "evaluated in surrounding context" approach works out the same as the approach in the v0.1 draft spec, I think. Or is that your point? Regards, Mark From jjb at google.com Thu Feb 4 13:08:39 2010 From: jjb at google.com (Joshua Bloch) Date: Thu, 4 Feb 2010 13:08:39 -0800 Subject: Preparing for the 0.2 draft In-Reply-To: <0909768B-4A52-4F33-80E8-F4D3F10033B8@googlemail.com> References: <4B62A2CE.7080300@sun.com> <15e8b9d21002010058w49c686e3i325627d6362614d8@mail.gmail.com> <17b2302a1002032156g7b6b7005l2d6c79c79c390f37@mail.gmail.com> <0909768B-4A52-4F33-80E8-F4D3F10033B8@googlemail.com> Message-ID: <17b2302a1002041308q21ea76a9w5a11aaa9fa8351b9@mail.gmail.com> Mark, On Thu, Feb 4, 2010 at 12:27 PM, Mark Mahieu wrote: > > On 4 Feb 2010, at 05:56, Joshua Bloch wrote: > > > Neal found 48 locations in OpenJDK 6 in which he claimed that evaluating > > "this" in the surrounding context would be helpful. I decided to take a > > closer look at them in hopes of learning something. > > > > > 4 could not take advantage of the > > "enclosing scope semantics for this," because the anonymous class has a > > method that shadows the one in the enclosing class (see (1) below for an > > example) > > If "this" were evaluated in the context of the enclosing class, how could > shadowing come into play here? Wouldn't the former preclude the latter? > This class of uses concern SAM types, exclusively. Technically you're right. But evaluating "this" in the enclosing context of a "SAM lambda" is a bit harder to fathom. If you're implementing the compare method of a Comparator instance and you invoke the compare method, you probably expect a recursive invocation. > > > (1) src/share/classes/com/sun/java/util/jar/pack/Histogram.java:187 > > > > class Histogram { > > public double getBitLength(int value) { > > double prob = (double) getFrequency(value) / getTotalWeight(); > > return - Math.log(prob) / log2; > > } > > > > private final BitMetric bitMetric = new BitMetric() { > > public double getBitLength(int value) { > > return Histogram.this.getBitLength(value); > > } > > }; > > } > > > > The qualified this is necessary only because the anonymous class gives > its > > sole method the same name as the one that it wants to call in its > enclosing > > instance. But you can't name the method in a lambda expression, so having > > "this" evaluated in the surrounding context would not be necessary (or > > helpful) here. > > The method is named anyway, by the BitMetric interface. Surely it's > whether or not that is visible in the body of an equivalent lambda which > determines how helpful a given approach is? For this particular example, > the "evaluated in surrounding context" approach works out the same as the > approach in the v0.1 draft spec, I think. Or is that your point? > I was purpose vague in discussing "class 4." I do believe the examples in this class don't benefit from evaluating this in the surrounding context, but it's hard to discuss them precisely until we firm up our story on SAM types. Josh P.S. Also I reviewed 48 (!) uses, which limited the amount of time I could spend thinking about each one. From Alex.Buckley at Sun.COM Thu Feb 4 13:26:37 2010 From: Alex.Buckley at Sun.COM (Alex Buckley) Date: Thu, 04 Feb 2010 13:26:37 -0800 Subject: What is the meaning of this? In-Reply-To: <942ADAA5-46AE-472A-A10F-A458681F02CE@googlemail.com> References: <17b2302a1001281119t4ea1c34cya8a05e59864b407@mail.gmail.com> <201001282246.44590.peter.levart@gmail.com> <17b2302a1001281619i72676ed1p954b010b1a23a614@mail.gmail.com> <15e8b9d21001290912t20f3a4ffkb57e29838652b7da@mail.gmail.com> <4B631C27.6000007@the-loom.de> <4B631D61.3010105@the-loom.de> <942ADAA5-46AE-472A-A10F-A458681F02CE@googlemail.com> Message-ID: <4B6B3B8D.4080905@sun.com> Mark Mahieu wrote: > The User Guide at http://groovy.codehaus.org/Closures says: > > "this : as in Java, this refers to the instance of the enclosing class where a Closure is defined" > > As in Java. Well, fancy that. For completeness on the Groovy front, note that technically 'this' is the closure instance, but that 'this.foo' refers to the lexically enclosing object ('owner') as if by magic: http://stackoverflow.com/questions/820084/groovy-meaning-of-this-inside-a-closure Alex From adam.hawthorne at gmail.com Thu Feb 4 14:16:00 2010 From: adam.hawthorne at gmail.com (Adam Hawthorne) Date: Thu, 4 Feb 2010 17:16:00 -0500 Subject: Preparing for the 0.2 draft In-Reply-To: <0909768B-4A52-4F33-80E8-F4D3F10033B8@googlemail.com> References: <4B62A2CE.7080300@sun.com> <15e8b9d21002010058w49c686e3i325627d6362614d8@mail.gmail.com> <17b2302a1002032156g7b6b7005l2d6c79c79c390f37@mail.gmail.com> <0909768B-4A52-4F33-80E8-F4D3F10033B8@googlemail.com> Message-ID: <96d1a17e1002041416m2ca348ecxc96d002dfcdd0781@mail.gmail.com> Hi, We have a project implementing a legacy language platform in Java. The project began in 1998 and our codebase is 1.6MLOC. Neal's program gives stats below. I've elided names of classes in our codebase, but I've left references to classes and interfaces from public libraries. I'd be happy to run a program that produces statistics requested by Alex also. I'll attempt to write something if no one has such a program, but I can't promise it will be soon. HTH, Adam Total anonymous classes: 1420 But with no constructor args: 1264 And defining only one method: 1092 And where the type is a SAM type: 943 Those that are recursive: 0 Those that reference 'this': 6 Those that reference an enclosing 'this': 133 Those that are interfaces: 829 Those that are classes: 114 Total distinct interfaces: 58 [snip] com.google.common.base.Function com.google.gwt.event.dom.client.ClickHandler com.google.gwt.event.dom.client.KeyDownHandler com.google.gwt.event.dom.client.KeyPressHandler com.google.gwt.event.dom.client.MouseDownHandler com.google.gwt.event.dom.client.MouseOutHandler com.google.gwt.event.dom.client.MouseOverHandler com.google.gwt.event.dom.client.MouseUpHandler com.google.gwt.event.logical.shared.CloseHandler com.google.gwt.event.logical.shared.ResizeHandler java.awt.event.AWTEventListener java.awt.event.ActionListener java.awt.event.HierarchyListener java.awt.event.ItemListener java.awt.image.ImageObserver java.awt.print.Printable java.beans.ExceptionListener java.beans.PropertyChangeListener java.io.FileFilter java.io.FilenameFilter java.lang.Runnable java.lang.reflect.InvocationHandler java.net.URLStreamHandlerFactory java.security.PrivilegedAction java.security.PrivilegedExceptionAction java.util.Comparator java.util.concurrent.Callable javax.swing.UIDefaults.ActiveValue javax.swing.UIDefaults.LazyValue javax.swing.event.ChangeListener javax.swing.event.HyperlinkListener javax.swing.event.ListSelectionListener javax.swing.event.TableModelListener javax.swing.event.TreeSelectionListener javax.swing.event.UndoableEditListener org.netbeans.editor.Acceptor org.netbeans.editor.TextBatchProcessor org.openide.filesystems.FileSystem.AtomicAction org.openide.util.actions.ActionPerformer Total distinct classes: 21 [snip] com.google.gwt.user.client.Timer java.awt.event.FocusAdapter java.awt.event.KeyAdapter java.io.InputStream java.util.HashMap java.util.TimerTask javax.swing.AbstractAction javax.swing.AbstractListModel org.openide.util.datatransfer.PasteType From markmahieu at googlemail.com Thu Feb 4 14:26:28 2010 From: markmahieu at googlemail.com (Mark Mahieu) Date: Thu, 4 Feb 2010 22:26:28 +0000 Subject: Preparing for the 0.2 draft In-Reply-To: <17b2302a1002041308q21ea76a9w5a11aaa9fa8351b9@mail.gmail.com> References: <4B62A2CE.7080300@sun.com> <15e8b9d21002010058w49c686e3i325627d6362614d8@mail.gmail.com> <17b2302a1002032156g7b6b7005l2d6c79c79c390f37@mail.gmail.com> <0909768B-4A52-4F33-80E8-F4D3F10033B8@googlemail.com> <17b2302a1002041308q21ea76a9w5a11aaa9fa8351b9@mail.gmail.com> Message-ID: On 4 Feb 2010, at 21:08, Joshua Bloch wrote: > Mark, > > On Thu, Feb 4, 2010 at 12:27 PM, Mark Mahieu wrote: > > On 4 Feb 2010, at 05:56, Joshua Bloch wrote: > > > Neal found 48 locations in OpenJDK 6 in which he claimed that evaluating > > "this" in the surrounding context would be helpful. I decided to take a > > closer look at them in hopes of learning something. > > > > > 4 could not take advantage of the > > "enclosing scope semantics for this," because the anonymous class has a > > method that shadows the one in the enclosing class (see (1) below for an > > example) > > If "this" were evaluated in the context of the enclosing class, how could shadowing come into play here? Wouldn't the former preclude the latter? > > This class of uses concern SAM types, exclusively. Technically you're right. But evaluating "this" in the enclosing context of a "SAM lambda" is a bit harder to fathom. If you're implementing the compare method of a Comparator instance and you invoke the compare method, you probably expect a recursive invocation. > > > > (1) src/share/classes/com/sun/java/util/jar/pack/Histogram.java:187 > > > > class Histogram { > > public double getBitLength(int value) { > > double prob = (double) getFrequency(value) / getTotalWeight(); > > return - Math.log(prob) / log2; > > } > > > > private final BitMetric bitMetric = new BitMetric() { > > public double getBitLength(int value) { > > return Histogram.this.getBitLength(value); > > } > > }; > > } > > > > The qualified this is necessary only because the anonymous class gives its > > sole method the same name as the one that it wants to call in its enclosing > > instance. But you can't name the method in a lambda expression, so having > > "this" evaluated in the surrounding context would not be necessary (or > > helpful) here. > > The method is named anyway, by the BitMetric interface. Surely it's whether or not that is visible in the body of an equivalent lambda which determines how helpful a given approach is? For this particular example, the "evaluated in surrounding context" approach works out the same as the approach in the v0.1 draft spec, I think. Or is that your point? > > I was purpose vague in discussing "class 4." I do believe the examples in this class don't benefit from evaluating this in the surrounding context, but it's hard to discuss them precisely until we firm up our story on SAM types. > > Josh > > P.S. Also I reviewed 48 (!) uses, which limited the amount of time I could spend thinking about each one. > Ok, no criticism was intended, I just thought I was having a reading comprehension failure or something. Thanks for taking the time to expand on your original comments. Actually, that particular example grabbed my eye because I've noticed a number of similar cases in the last few days, but where a class has been written almost as it would if it directly implemented some interface, but the programmer has wanted to keep that an implementation detail so used an intermediary anonymous inner class which forwards to a (normally private) method on the containing class: interface Listener { void onEvent(Event e); } class Foo { ... bar.addListener(new Listener() { public void onEvent(Event e) { Foo.this.onEvent(e); }}); ... private void onEvent(Event e) { // important code } } Just as is the case with your Comparator example (and the Histogram code), I certainly agree that it's interesting to consider what the author would expect if they were to write something like: bar.addListener(#(Event e) { onEvent(e); }); But equally interesting is what someone else would make of it when reading that code at a later point, especially since the only obvious onEvent method present is on the containing class. Regards, Mark From cdr at intellij.com Fri Feb 5 01:07:05 2010 From: cdr at intellij.com (Alexey Kudravtsev) Date: Fri, 5 Feb 2010 12:07:05 +0300 Subject: SAM interface/class figures [Re: Preparing for the 0.2 In-Reply-To: References: Message-ID: > > Thanks Alexey and Stephen. How are you producing these statistics? I've cobbled up plugin for Intellij IDEA which analyses current project and produces statistics. You can get the plugin jar and source files at the http://dl.dropbox.com/u/4469125/lambdaStat.jar Interestingly, the most difficult part was to realize what the SAM type is. I've used the definition from http://mail.openjdk.java.net/pipermail/lambda-dev/attachments/20100122/3764c21a/attachment.txt except the requirement for SAM to be top level class. And I am still not completely sure I got it right. In any case, in this post by Neal Gafter, where these classes are supposed to be SAM types, the majority of them are obviously not. ================== >Below are all the distinct classes > > com.sun.tools.example.debug.tty.Commands.AsyncExecution > com.sun.tools.hat.internal.model.AbstractJavaHeapObjectVisitor > com.sun.tools.hat.internal.util.Comparer > java.awt.font.TextLine.Function > java.io.InputStream > java.io.OutputStream > java.lang.Thread > java.nio.charset.CoderResult.Cache > java.util.TimerTask > java.util.regex.Pattern.CharProperty > java.util.regex.Pattern.CharPropertyNames.CharPropertyFactory > java.util.regex.Pattern.CharPropertyNames.CloneableProperty > javax.swing.AbstractAction ====================== > > Alexey, do you really mean 1920 SAM *classes*, or 1920 SAM types? > Because if 7474 anon.classes extend a class, and 12597 anon.classes > extend a "SAM", then some of those "SAM" things must be interfaces. What > is really interesting is how many of those 12597 anon.classes extend a > SAM class v. implement a SAM interface. Yes, I meant both SAM classes and SAM interfaces. Here is the revised and more elaborate statistics (Numbers are different a little since the codebase is live): Total files inspected: 34842 Total LOC: 3771256 Total java classes: 69822 All SAM types: 1249 (1,8%) SAM classes: 1124 (90,0%) SAM interfaces: 125 (10,0%) Anonymous classes: 21956 (31,4%) anonymous classes which extend class: 7492 (34,1%) anonymous classes which implement interface: 14464 (65,9%) anonymous classes which extend SAM type: 13013 (59,3%) anonymous classes which extend SAM class: 244 (1,9%) anonymous classes which implement SAM interface: 12769 (98,1%) anon classes with single method: 18470 (84,1%) anon classes with fields: 240 (1,1%) anon classes with 'this' references: 116 (0,5%) anon classes with qualified 'this' references: 77 (0,4%) anon classes which recurse: 15 (0,1%) anon classes extending SAM, with single method, no fields, no ctr parameters: 12811 (58,3%) and they extend 34 classes and implement 547 interfaces. --regards, Alexey Kudravtsev > > Alex > > Alexey Kudravtsev wrote: >> These are figures for the Intellij IDEA (http://www.jetbrains.com/idea/) >> source base. >> >> Total files inspected: 34794 >> Total java classes: 69703 >> All SAM classes: 1920 (2,8%) >> All Anonymous classes: 21908 (31,4%) >> anonymous classes which Extend class: 7474 (34,1%) >> anonymous classes which implement interface: 14434 (65,9%) >> >> anonymous classes which extend SAM: 12597 (57,5%) >> >> anon classes with single method: 18431 (84,1%) >> anon classes with fields: 149 (0,7%) >> anon classes with 'this' references: 116 (0,5%) >> anon classes with qualified 'this' references: 77 (0,4%) >> anon classes which recurse: 15 (0,1%) From markmahieu at googlemail.com Fri Feb 5 01:13:16 2010 From: markmahieu at googlemail.com (Mark Mahieu) Date: Fri, 5 Feb 2010 09:13:16 +0000 Subject: Code to analyze a source base for project lambda In-Reply-To: <15e8b9d21002031310n7c7f4c4eqcc0844ad450b5efb@mail.gmail.com> References: <15e8b9d21002031255y45704467i32475a2c9b64803d@mail.gmail.com> <15e8b9d21002031310n7c7f4c4eqcc0844ad450b5efb@mail.gmail.com> Message-ID: <67103117-46CB-4258-A544-814E1AD00D3A@googlemail.com> I think I've identified a smallish bug in Neal's analysis code, having cross referenced its output with that of my own. Specifically, it builds a TreeMap whose keys are the JCNewClass javac trees which represent the various anonymous classes found as the compiler trawls its way across the code. However, the Comparator used by this TreeMap deems two keys equal if the anonymous classes appear at the same position in their respective source files; ie. it doesn't account for the possibility that they are in different compilation units altogether. The clearest symptom is that obvious non-SAM types are listed in the final output, but more importantly the total stats are potentially inaccurate (although not necessarily by a large degree). However, I don't believe the code is currently making deliberate use of the sorted nature of the Map, and as far as I can tell it doesn't actually have to cope with multiple JCNewClass instances representing the same anonymous class (but please shout if you think I'm wrong). So, I think one very quick fix is to change the AnonStats to extend HashMap rather than TreeMap, and remove the super-constructor call a couple of lines further down. This change seems to result in more accurate statistics. (The same cross referencing process highlighted two more bugs in my code which also skew the results slightly, so I plan to re-run it with Neal's shortly) Mark On 3 Feb 2010, at 21:10, Neal Gafter wrote: > The code can be downloaded from http://www.javac.info/ijavac-Main.java > > On Wed, Feb 3, 2010 at 12:55 PM, Neal Gafter wrote: >> Apparently it was rejected by the moderator last time. Here it is again. >> >> The enclosed code (Main.java) was thrown together hastily, but it >> might be useful to analyze a source base for project lambda. Compile >> and run it with tools.jar on the classpath (but on a mac, tools.jar is >> always on the classpath). It acts just like javac, but printing >> statistics before exiting. I used it to process most of the openjdk6 >> sources. >> > From Alex.Buckley at Sun.COM Fri Feb 5 12:58:22 2010 From: Alex.Buckley at Sun.COM (Alex Buckley) Date: Fri, 05 Feb 2010 12:58:22 -0800 Subject: SAM interface/class figures [Re: Preparing for the 0.2 In-Reply-To: References: Message-ID: <4B6C866E.7000207@sun.com> Hi Alexey, Alexey Kudravtsev wrote: > In any case, in this post by Neal Gafter, where these classes are supposed > to be SAM types, the majority of them are obviously not. > ================== >> Below are all the distinct classes >> >> com.sun.tools.example.debug.tty.Commands.AsyncExecution >> com.sun.tools.hat.internal.model.AbstractJavaHeapObjectVisitor >> com.sun.tools.hat.internal.util.Comparer >> java.awt.font.TextLine.Function >> java.io.InputStream >> java.io.OutputStream >> java.lang.Thread >> java.nio.charset.CoderResult.Cache >> java.util.TimerTask >> java.util.regex.Pattern.CharProperty >> java.util.regex.Pattern.CharPropertyNames.CharPropertyFactory >> java.util.regex.Pattern.CharPropertyNames.CloneableProperty >> javax.swing.AbstractAction > ====================== 1) I did not interpret Neal as saying that these classes are SAMs. He said: Total anonymous classes: 1155 ... And where the type is a SAM type: 811 ... Those that are interfaces: 752 Those that are classes: 59 Total distinct interfaces: 50 (see below) Total distinct classes: 13 (see below) That is, 1155 anon.classes implement/extend 50 interfaces and 13 classes. Only 811 anon.classes implement/extend SAM types; Neal did not say how many of the 50 interfaces and 13 classes are SAM types. 2) As it happens, a majority of the classes are SAM types. Only com.sun.tools.example.debug.tty.Commands.AsyncExecution, java.lang.Thread, and javax.swing.AbstractAction are not. Probably you said a majority are not SAM types because a majority are not top-level classes. However, if a nested class is static, then it ought to qualify (other conditions permitting) as a SAM type. The key is that a SAM is non-inner, not that it's top-level. 3) It would be interesting to know how many distinct interfaces (of the 50) are implemented by the 752 anon.classes that implement a SAM type. Alex From markmahieu at googlemail.com Fri Feb 5 13:34:56 2010 From: markmahieu at googlemail.com (Mark Mahieu) Date: Fri, 5 Feb 2010 21:34:56 +0000 Subject: SAM interface/class figures [Re: Preparing for the 0.2 In-Reply-To: <4B6C866E.7000207@sun.com> References: <4B6C866E.7000207@sun.com> Message-ID: Hi Alex, On 5 Feb 2010, at 20:58, Alex Buckley wrote: > Hi Alexey, > > Alexey Kudravtsev wrote: >> In any case, in this post by Neal Gafter, where these classes are supposed >> to be SAM types, the majority of them are obviously not. >> ================== >>> Below are all the distinct classes >>> >>> com.sun.tools.example.debug.tty.Commands.AsyncExecution >>> com.sun.tools.hat.internal.model.AbstractJavaHeapObjectVisitor >>> com.sun.tools.hat.internal.util.Comparer >>> java.awt.font.TextLine.Function >>> java.io.InputStream >>> java.io.OutputStream >>> java.lang.Thread >>> java.nio.charset.CoderResult.Cache >>> java.util.TimerTask >>> java.util.regex.Pattern.CharProperty >>> java.util.regex.Pattern.CharPropertyNames.CharPropertyFactory >>> java.util.regex.Pattern.CharPropertyNames.CloneableProperty >>> javax.swing.AbstractAction >> ====================== > > 1) I did not interpret Neal as saying that these classes are SAMs. He said: > > Total anonymous classes: 1155 > ... > And where the type is a SAM type: 811 > ... > Those that are interfaces: 752 > Those that are classes: 59 > Total distinct interfaces: 50 (see below) > Total distinct classes: 13 (see below) > > That is, 1155 anon.classes implement/extend 50 interfaces and 13 > classes. Only 811 anon.classes implement/extend SAM types; Neal did not > say how many of the 50 interfaces and 13 classes are SAM types. I thought the same at first, but the code starts out with the set of all anonymous classes and then whittles that down by applying the 3 filters ("no constructor args", "only one method", "SAM type") until arriving at a refined set which satisfies all of those conditions. Any subsequent output, beginning with the first "Those that are..." line, is based on that refined set, so in the example above, I think the 50 interfaces and 13 classes are all supposed to be SAM types. The reason they aren't (if my interpretation above is correct) is only due to the bug I mentioned earlier today. Mark From neal at gafter.com Sat Feb 6 15:29:59 2010 From: neal at gafter.com (Neal Gafter) Date: Sat, 6 Feb 2010 15:29:59 -0800 Subject: Code to analyze a source base for project lambda In-Reply-To: <67103117-46CB-4258-A544-814E1AD00D3A@googlemail.com> References: <15e8b9d21002031255y45704467i32475a2c9b64803d@mail.gmail.com> <15e8b9d21002031310n7c7f4c4eqcc0844ad450b5efb@mail.gmail.com> <67103117-46CB-4258-A544-814E1AD00D3A@googlemail.com> Message-ID: <15e8b9d21002061529r3a962c61v7fd6996fee015777@mail.gmail.com> On Fri, Feb 5, 2010 at 1:13 AM, Mark Mahieu wrote: > I think I've identified a smallish bug in Neal's analysis code, having > cross referenced its output with that of my own. > Yes, that was a bug, and it did skew the results in a way that erroneously favored both supporting SAM classes and nonlexical "this". The corrected stats for openjdk6 are below. The corrected code can be downloaded from http://www.javac.info/ijavac-Main.java Cheers, Neal Total anonymous classes: 1177 But with no constructor args: 1089 And defining only one method: 900 And where the type is a SAM type: 828 Those that are recursive: 0 Those that reference 'this': 2 Those that reference an enclosing 'this': 49 Those that are interfaces: 771 Those that are classes: 57 Total distinct interfaces: 48 Total distinct classes: 11 The distinct interfaces were com.sun.java.util.jar.pack.Histogram.BitMetric com.sun.jdi.connect.Transport com.sun.jmx.remote.internal.NotificationBufferFilter com.sun.media.sound.ModelTransform com.sun.net.ssl.HostnameVerifier com.sun.security.auth.callback.DialogCallbackHandler.Action com.sun.tools.example.debug.bdi.InputListener com.sun.tools.example.debug.bdi.OutputListener com.sun.tools.example.debug.expr.ExpressionParser.GetFrame com.sun.tools.hat.internal.oql.ObjectVisitor com.sun.tools.jdi.CommandSender com.sun.tools.script.shell.Main.Command java.awt.Conditional java.awt.KeyEventPostProcessor java.awt.event.ActionListener java.awt.event.HierarchyListener java.beans.ExceptionListener java.beans.PropertyChangeListener java.beans.VetoableChangeListener java.io.FilenameFilter java.io.ObjectInputValidation java.lang.Runnable java.net.CookiePolicy java.net.HttpCookie.CookieAttributeAssignor java.security.PrivilegedAction java.security.PrivilegedExceptionAction java.util.Comparator java.util.concurrent.Callable java.util.concurrent.Executor java.util.concurrent.ThreadFactory javax.imageio.event.IIOReadWarningListener javax.imageio.event.IIOWriteWarningListener javax.management.NotificationListener javax.swing.UIDefaults.ActiveValue javax.swing.UIDefaults.LazyValue javax.swing.event.CaretListener javax.swing.event.ChangeListener javax.swing.event.HyperlinkListener javax.xml.crypto.KeySelectorResult javax.xml.crypto.NodeSetData sun.awt.RequestFocusController sun.java2d.StateTracker sun.java2d.cmm.ProfileActivator sun.java2d.loops.ProcessPath.EndSubPathHandler sun.misc.JavaIODeleteOnExitAccess sun.misc.JavaNetAccess sun.nio.ch.FileChannelImpl.FileLockTable.Releaser sun.nio.ch.Interruptible The distinct classes were: com.sun.tools.example.debug.tty.Commands.AsyncExecution com.sun.tools.hat.internal.model.AbstractJavaHeapObjectVisitor com.sun.tools.hat.internal.util.Comparer java.awt.font.TextLine.Function java.io.OutputStream java.nio.charset.CoderResult.Cache java.util.TimerTask java.util.regex.Pattern.CharProperty java.util.regex.Pattern.CharPropertyNames.CharPropertyFactory java.util.regex.Pattern.CharPropertyNames.CloneableProperty javax.swing.AbstractAction From jkuhnert at gmail.com Sun Feb 7 09:02:28 2010 From: jkuhnert at gmail.com (Jesse Kuhnert) Date: Sun, 7 Feb 2010 12:02:28 -0500 Subject: related link Message-ID: <7926817e1002070902h99c558s221685f1989b80d2@mail.gmail.com> With no intention of implying that either language is close enough to be compared directly, just that it's a lengthy analysis of the subject matter here which might provide more mental fuel. http://yehudakatz.com/2010/02/07/the-building-blocks-of-ruby/ From neal at gafter.com Sun Feb 7 09:10:47 2010 From: neal at gafter.com (Neal Gafter) Date: Sun, 7 Feb 2010 09:10:47 -0800 Subject: related link In-Reply-To: <7926817e1002070902h99c558s221685f1989b80d2@mail.gmail.com> References: <7926817e1002070902h99c558s221685f1989b80d2@mail.gmail.com> Message-ID: <15e8b9d21002070910x5326fd26o33ab126aabdd2d61@mail.gmail.com> Jesse- That article provides a good understanding of the goals of BGGA and CfJ 0.6a/b (and the openjdk closures project), as contrasted with the goals of the current effort (openjdk project lambda). Cheers, Neal On Sun, Feb 7, 2010 at 9:02 AM, Jesse Kuhnert wrote: > With no intention of implying that either language is close enough to > be compared directly, just that it's a lengthy analysis of the > subject matter here which might provide more mental fuel. > > http://yehudakatz.com/2010/02/07/the-building-blocks-of-ruby/ > > From neal at gafter.com Sun Feb 7 10:41:18 2010 From: neal at gafter.com (Neal Gafter) Date: Sun, 7 Feb 2010 10:41:18 -0800 Subject: Function types versus arrays Message-ID: <15e8b9d21002071041j57e926e9k8e7c36e01f80ee46@mail.gmail.com> I've been asked if arrays and function types will play nicely together.? Specifically, for example, whether or not something like the following will be allowed: #String(String)[] arrayOfFunction = new #String(String)[10]; The current draft spec, by the absence of any constraining rules, implies yes.? But if I have to guess, I would say the end result of project lambda will probably say no.? Although we haven't talked much about implementation techniques, I'm not aware of any proposed implementation technique that would allow this without opening a hole in the type system. I don't know whether we should consider this important or not (I don't; I'm perfectly happy using java.util.List), but if it is important then someone should be thinking about what needs to happen (VM support?) to make it work. Cheers, Neal From john at milsson.nu Sun Feb 7 14:50:55 2010 From: john at milsson.nu (John Nilsson) Date: Sun, 7 Feb 2010 23:50:55 +0100 Subject: related link In-Reply-To: <15e8b9d21002070910x5326fd26o33ab126aabdd2d61@mail.gmail.com> References: <7926817e1002070902h99c558s221685f1989b80d2@mail.gmail.com> <15e8b9d21002070910x5326fd26o33ab126aabdd2d61@mail.gmail.com> Message-ID: It seems to me that project lambda fits better with, well, lambdas. The other goals would probably be better addressed with some static construct, like macros. BR, John On Sun, Feb 7, 2010 at 6:10 PM, Neal Gafter wrote: > Jesse- > > That article provides a good understanding of the goals of BGGA and CfJ > 0.6a/b (and the openjdk closures project), as contrasted with the goals of > the current effort (openjdk project lambda). > > Cheers, > Neal > > On Sun, Feb 7, 2010 at 9:02 AM, Jesse Kuhnert wrote: > > > With no intention of implying that either language is close enough to > > be compared directly, just that it's a lengthy analysis of the > > subject matter here which might provide more mental fuel. > > > > http://yehudakatz.com/2010/02/07/the-building-blocks-of-ruby/ > > > > > > From neal at gafter.com Sun Feb 7 15:36:58 2010 From: neal at gafter.com (Neal Gafter) Date: Sun, 7 Feb 2010 15:36:58 -0800 Subject: related link In-Reply-To: References: <7926817e1002070902h99c558s221685f1989b80d2@mail.gmail.com> <15e8b9d21002070910x5326fd26o33ab126aabdd2d61@mail.gmail.com> Message-ID: <15e8b9d21002071536h9b1d58n29b5af89aa85dc61@mail.gmail.com> John- It hasn't been shown that a simple macro facility can support control constructs in a library in the absence of transparent closures (I'd welcome your attempt to do so). However, it has been shown that transparent closures are sufficient. Macros are nice in making the syntax of such control APIs more convenient; BGGA and CfJ 0.6b < http://www.javac.info/closures-v06b.html> address the convenience issue by supporting one particular but common special case (where a function is accepted by an API as its last parameter) rather than attempting to add a complete hygienic macro system. Cheers, Neal On Sun, Feb 7, 2010 at 2:50 PM, John Nilsson wrote: > It seems to me that project lambda fits better with, well, lambdas. > > The other goals would probably be better addressed with some static > construct, like macros. > > BR, > John > > On Sun, Feb 7, 2010 at 6:10 PM, Neal Gafter wrote: > >> Jesse- >> >> That article provides a good understanding of the goals of BGGA and CfJ >> 0.6a/b (and the openjdk closures project), as contrasted with the goals of >> the current effort (openjdk project lambda). >> >> Cheers, >> Neal >> >> On Sun, Feb 7, 2010 at 9:02 AM, Jesse Kuhnert wrote: >> >> > With no intention of implying that either language is close enough to >> > be compared directly, just that it's a lengthy analysis of the >> > subject matter here which might provide more mental fuel. >> > >> > http://yehudakatz.com/2010/02/07/the-building-blocks-of-ruby/ >> > >> > >> >> > From forax at univ-mlv.fr Mon Feb 8 02:17:56 2010 From: forax at univ-mlv.fr (=?ISO-8859-1?Q?R=E9mi_Forax?=) Date: Mon, 08 Feb 2010 11:17:56 +0100 Subject: Function types versus arrays In-Reply-To: <15e8b9d21002071041j57e926e9k8e7c36e01f80ee46@mail.gmail.com> References: <15e8b9d21002071041j57e926e9k8e7c36e01f80ee46@mail.gmail.com> Message-ID: <4B6FE4D4.2040208@univ-mlv.fr> Le 07/02/2010 19:41, Neal Gafter a ?crit : > I've been asked if arrays and function types will play nicely > together. Specifically, for example, whether or not something like > the following will be allowed: > > #String(String)[] arrayOfFunction = new #String(String)[10]; > > The current draft spec, by the absence of any constraining rules, > implies yes. But if I have to guess, I would say the end result of > project lambda will probably say no. Although we haven't talked much > about implementation techniques, I'm not aware of any proposed > implementation technique that would allow this without opening a hole > in the type system. > > I don't know whether we should consider this important or not (I > don't; I'm perfectly happy using java.util.List), but if it is > important then someone should be thinking about what needs to happen > (VM support?) to make it work. > For the record the problem is: #String(String)[] arrayOfFunction = new #String(String)[10]; Object[] array = arrayOfFunction; array[0] = #int() (2); The last line should raise an array store exception. So the VM should be aware of the precise type of the array of function. Currently, neither the anonymous class translation nor the method handle one provide reified function type so array of function types are unsafe. Because function type doesn't exist in the language, I propose to make the syntax for an array of function type (#String(String)[]) illegal. I'm also happy with List, perhaps we should introduce a new interface ReadOnlyList which is a super type of List and array of objects but that another story. > Cheers, > Neal > R?mi From neal at gafter.com Mon Feb 8 07:48:45 2010 From: neal at gafter.com (Neal Gafter) Date: Mon, 8 Feb 2010 07:48:45 -0800 Subject: Function types versus arrays In-Reply-To: <4B6FE4D4.2040208@univ-mlv.fr> References: <15e8b9d21002071041j57e926e9k8e7c36e01f80ee46@mail.gmail.com> <4B6FE4D4.2040208@univ-mlv.fr> Message-ID: <15e8b9d21002080748m4fe62717y5b21b18a68a66492@mail.gmail.com> If arrays of functions are illegal, then the function-type syntax #(Types Throws_opt)->Type (which was previously rejected because there was no way to write an array of function type) can be reconsidered. On Mon, Feb 8, 2010 at 2:17 AM, R?mi Forax wrote: > Le 07/02/2010 19:41, Neal Gafter a ?crit : >> I've been asked if arrays and function types will play nicely >> together. ?Specifically, for example, whether or not something like >> the following will be allowed: >> >> #String(String)[] arrayOfFunction = new #String(String)[10]; >> >> The current draft spec, by the absence of any constraining rules, >> implies yes. ?But if I have to guess, I would say the end result of >> project lambda will probably say no. ?Although we haven't talked much >> about implementation techniques, I'm not aware of any proposed >> implementation technique that would allow this without opening a hole >> in the type system. >> >> I don't know whether we should consider this important or not (I >> don't; I'm perfectly happy using java.util.List), but if it is >> important then someone should be thinking about what needs to happen >> (VM support?) to make it work. >> > > For the record the problem is: > > #String(String)[] arrayOfFunction = new #String(String)[10]; > Object[] array = arrayOfFunction; > array[0] = #int() (2); > > The last line should raise an array store exception. > So the VM should be aware of the precise type of the array of function. > > Currently, neither the anonymous class translation nor the method handle one > provide reified function type so array of function types are unsafe. > > Because function type doesn't exist in the language, I propose to make > the syntax > for an array of function type (#String(String)[]) illegal. > > I'm also happy with List, perhaps we should introduce a new interface > ReadOnlyList > which is a super type of List and array of objects but that another story. > >> Cheers, >> Neal >> > > R?mi > > From john at milsson.nu Mon Feb 8 09:21:19 2010 From: john at milsson.nu (John Nilsson) Date: Mon, 8 Feb 2010 18:21:19 +0100 Subject: related link In-Reply-To: <15e8b9d21002071536h9b1d58n29b5af89aa85dc61@mail.gmail.com> References: <7926817e1002070902h99c558s221685f1989b80d2@mail.gmail.com> <15e8b9d21002070910x5326fd26o33ab126aabdd2d61@mail.gmail.com> <15e8b9d21002071536h9b1d58n29b5af89aa85dc61@mail.gmail.com> Message-ID: In my, admittedly rather na?ve, mind such a static construct could be modelled as transparent closures with the added restriction that it is to in-lined at compile time, and as such would disallow any treatment of them as run-time constructs. F.ex. moving references around would be a static error. The rather narrow goal in my mind being to allow "non-local returns" without the scary stuff ;) When you said "has been shown" did you have any particular paper in mind? (I'm rather curious about PL-research) BR, John On Mon, Feb 8, 2010 at 12:36 AM, Neal Gafter wrote: > John- > > It hasn't been shown that a simple macro facility can support control > constructs in a library in the absence of transparent closures (I'd welcome > your attempt to do so). However, it has been shown that transparent > closures are sufficient. Macros are nice in making the syntax of such > control APIs more convenient; BGGA and CfJ 0.6b < > http://www.javac.info/closures-v06b.html> address the convenience issue by > supporting one particular but common special case (where a function is > accepted by an API as its last parameter) rather than attempting to add a > complete hygienic macro system. > > Cheers, > Neal > > > On Sun, Feb 7, 2010 at 2:50 PM, John Nilsson wrote: > >> It seems to me that project lambda fits better with, well, lambdas. >> >> The other goals would probably be better addressed with some static >> construct, like macros. >> >> BR, >> John >> >> On Sun, Feb 7, 2010 at 6:10 PM, Neal Gafter wrote: >> >>> Jesse- >>> >>> That article provides a good understanding of the goals of BGGA and CfJ >>> 0.6a/b (and the openjdk closures project), as contrasted with the goals >>> of >>> the current effort (openjdk project lambda). >>> >>> Cheers, >>> Neal >>> >>> On Sun, Feb 7, 2010 at 9:02 AM, Jesse Kuhnert >>> wrote: >>> >>> > With no intention of implying that either language is close enough to >>> > be compared directly, just that it's a lengthy analysis of the >>> > subject matter here which might provide more mental fuel. >>> > >>> > http://yehudakatz.com/2010/02/07/the-building-blocks-of-ruby/ >>> > >>> > >>> >>> >> > From Alex.Buckley at Sun.COM Mon Feb 8 11:02:08 2010 From: Alex.Buckley at Sun.COM (Alex Buckley) Date: Mon, 08 Feb 2010 11:02:08 -0800 Subject: related link In-Reply-To: References: <7926817e1002070902h99c558s221685f1989b80d2@mail.gmail.com> <15e8b9d21002070910x5326fd26o33ab126aabdd2d61@mail.gmail.com> <15e8b9d21002071536h9b1d58n29b5af89aa85dc61@mail.gmail.com> Message-ID: <4B705FB0.4030005@sun.com> Off-topic for lambda-dev. Please do not cross-post. John Nilsson wrote: > In my, admittedly rather na?ve, mind such a static construct could > be modelled as transparent closures with the added restriction that it is to > in-lined at compile time, and as such would disallow any treatment of them > as run-time constructs. F.ex. moving references around would be a static > error. > > The rather narrow goal in my mind being to allow "non-local returns" without > the scary stuff ;) > > When you said "has been shown" did you have any particular paper in mind? > (I'm rather curious about PL-research) > > BR, > John > > On Mon, Feb 8, 2010 at 12:36 AM, Neal Gafter wrote: > >> John- >> >> It hasn't been shown that a simple macro facility can support control >> constructs in a library in the absence of transparent closures (I'd welcome >> your attempt to do so). However, it has been shown that transparent >> closures are sufficient. Macros are nice in making the syntax of such >> control APIs more convenient; BGGA and CfJ 0.6b < >> http://www.javac.info/closures-v06b.html> address the convenience issue by >> supporting one particular but common special case (where a function is >> accepted by an API as its last parameter) rather than attempting to add a >> complete hygienic macro system. >> >> Cheers, >> Neal >> >> >> On Sun, Feb 7, 2010 at 2:50 PM, John Nilsson wrote: >> >>> It seems to me that project lambda fits better with, well, lambdas. >>> >>> The other goals would probably be better addressed with some static >>> construct, like macros. >>> >>> BR, >>> John >>> >>> On Sun, Feb 7, 2010 at 6:10 PM, Neal Gafter wrote: >>> >>>> Jesse- >>>> >>>> That article provides a good understanding of the goals of BGGA and CfJ >>>> 0.6a/b (and the openjdk closures project), as contrasted with the goals >>>> of >>>> the current effort (openjdk project lambda). >>>> >>>> Cheers, >>>> Neal >>>> >>>> On Sun, Feb 7, 2010 at 9:02 AM, Jesse Kuhnert >>>> wrote: >>>> >>>>> With no intention of implying that either language is close enough to >>>>> be compared directly, just that it's a lengthy analysis of the >>>>> subject matter here which might provide more mental fuel. >>>>> >>>>> http://yehudakatz.com/2010/02/07/the-building-blocks-of-ruby/ >>>>> >>>>> >>>> > From Alex.Buckley at Sun.COM Mon Feb 8 11:30:42 2010 From: Alex.Buckley at Sun.COM (Alex Buckley) Date: Mon, 08 Feb 2010 11:30:42 -0800 Subject: Function types versus arrays In-Reply-To: <4B6FE4D4.2040208@univ-mlv.fr> References: <15e8b9d21002071041j57e926e9k8e7c36e01f80ee46@mail.gmail.com> <4B6FE4D4.2040208@univ-mlv.fr> Message-ID: <4B706662.7050603@sun.com> Neal raises a fair point. I recognize that arrays of function types are as potentially unsafe at runtime as arrays of non-wildcard parameterized types. And this is the same problem one sees when passing variables of parameterized types as varargs parameters. The varargs method body can silently do unsafe things, thanks to Object being a universal supertype. Banning arrays of function types is an option. It somewhat conflicts with Project Coin's promotion of [] syntax to read/write collections. I am interested in people's thoughts on the limitations of arrays of parameterized types. (Reification was not and is not an option, so please no ranting about the stupid implications of erasure.) Does anyone care about arrays anymore? Alex R?mi Forax wrote: > Le 07/02/2010 19:41, Neal Gafter a ?crit : >> I've been asked if arrays and function types will play nicely >> together. Specifically, for example, whether or not something like >> the following will be allowed: >> >> #String(String)[] arrayOfFunction = new #String(String)[10]; >> >> The current draft spec, by the absence of any constraining rules, >> implies yes. But if I have to guess, I would say the end result of >> project lambda will probably say no. Although we haven't talked much >> about implementation techniques, I'm not aware of any proposed >> implementation technique that would allow this without opening a hole >> in the type system. >> >> I don't know whether we should consider this important or not (I >> don't; I'm perfectly happy using java.util.List), but if it is >> important then someone should be thinking about what needs to happen >> (VM support?) to make it work. >> > > For the record the problem is: > > #String(String)[] arrayOfFunction = new #String(String)[10]; > Object[] array = arrayOfFunction; > array[0] = #int() (2); > > The last line should raise an array store exception. > So the VM should be aware of the precise type of the array of function. > > Currently, neither the anonymous class translation nor the method handle one > provide reified function type so array of function types are unsafe. > > Because function type doesn't exist in the language, I propose to make > the syntax > for an array of function type (#String(String)[]) illegal. > > I'm also happy with List, perhaps we should introduce a new interface > ReadOnlyList > which is a super type of List and array of objects but that another story. > >> Cheers, >> Neal >> > > R?mi > From mthornton at optrak.co.uk Mon Feb 8 11:38:43 2010 From: mthornton at optrak.co.uk (Mark Thornton) Date: Mon, 08 Feb 2010 19:38:43 +0000 Subject: Function types versus arrays In-Reply-To: <4B706662.7050603@sun.com> References: <15e8b9d21002071041j57e926e9k8e7c36e01f80ee46@mail.gmail.com> <4B6FE4D4.2040208@univ-mlv.fr> <4B706662.7050603@sun.com> Message-ID: <4B706843.6030002@optrak.co.uk> Alex Buckley wrote: > I am interested in people's thoughts on the limitations of arrays of > parameterized types. (Reification was not and is not an option, so > please no ranting about the stupid implications of erasure.) Does anyone > care about arrays anymore? > > Alex > I use arrays quite frequently, including of parameterized types and suffer the warnings. Some of this is in the course of implementing specialized collections. However I am not sure that I would have any use for arrays of function types. Mark Thornton From grev at miginfocom.com Mon Feb 8 11:40:33 2010 From: grev at miginfocom.com (Mikael Grev) Date: Mon, 8 Feb 2010 20:40:33 +0100 Subject: Function types versus arrays In-Reply-To: <4B706662.7050603@sun.com> References: <15e8b9d21002071041j57e926e9k8e7c36e01f80ee46@mail.gmail.com> <4B6FE4D4.2040208@univ-mlv.fr> <4B706662.7050603@sun.com> Message-ID: I tend to use arrays a lot, but almost exclusively with primitives. I do it for the speed and lesser memory consumption. When I have a collection of references to objects, i.e. when I'm not storing primitives, I almost exclusively use the collection classes. I have never encountered an ArrayStoreException to this date, obviously. I do not think one would ever _need_ to store lambdas in arrays. I see no need since I estimate the relative performance and memory consumption winnings would be minimal. Cheers, Mikael On Feb 8, 2010, at 20:30 PM, Alex Buckley wrote: > Neal raises a fair point. I recognize that arrays of function types are > as potentially unsafe at runtime as arrays of non-wildcard parameterized > types. > > And this is the same problem one sees when passing variables of > parameterized types as varargs parameters. The varargs method body can > silently do unsafe things, thanks to Object being a universal supertype. > > Banning arrays of function types is an option. It somewhat conflicts > with Project Coin's promotion of [] syntax to read/write collections. > > I am interested in people's thoughts on the limitations of arrays of > parameterized types. (Reification was not and is not an option, so > please no ranting about the stupid implications of erasure.) Does anyone > care about arrays anymore? > > Alex > > R?mi Forax wrote: >> Le 07/02/2010 19:41, Neal Gafter a ?crit : >>> I've been asked if arrays and function types will play nicely >>> together. Specifically, for example, whether or not something like >>> the following will be allowed: >>> >>> #String(String)[] arrayOfFunction = new #String(String)[10]; >>> >>> The current draft spec, by the absence of any constraining rules, >>> implies yes. But if I have to guess, I would say the end result of >>> project lambda will probably say no. Although we haven't talked much >>> about implementation techniques, I'm not aware of any proposed >>> implementation technique that would allow this without opening a hole >>> in the type system. >>> >>> I don't know whether we should consider this important or not (I >>> don't; I'm perfectly happy using java.util.List), but if it is >>> important then someone should be thinking about what needs to happen >>> (VM support?) to make it work. >>> >> >> For the record the problem is: >> >> #String(String)[] arrayOfFunction = new #String(String)[10]; >> Object[] array = arrayOfFunction; >> array[0] = #int() (2); >> >> The last line should raise an array store exception. >> So the VM should be aware of the precise type of the array of function. >> >> Currently, neither the anonymous class translation nor the method handle one >> provide reified function type so array of function types are unsafe. >> >> Because function type doesn't exist in the language, I propose to make >> the syntax >> for an array of function type (#String(String)[]) illegal. >> >> I'm also happy with List, perhaps we should introduce a new interface >> ReadOnlyList >> which is a super type of List and array of objects but that another story. >> >>> Cheers, >>> Neal >>> >> >> R?mi >> From lk at teamten.com Mon Feb 8 11:43:26 2010 From: lk at teamten.com (Lawrence Kesteloot) Date: Mon, 8 Feb 2010 11:43:26 -0800 Subject: Function types versus arrays In-Reply-To: <4B706662.7050603@sun.com> References: <15e8b9d21002071041j57e926e9k8e7c36e01f80ee46@mail.gmail.com> <4B6FE4D4.2040208@univ-mlv.fr> <4B706662.7050603@sun.com> Message-ID: <997cab101002081143v63692adcp916e659d74f5cea@mail.gmail.com> On Mon, Feb 8, 2010 at 11:30 AM, Alex Buckley wrote: > I am interested in people's thoughts on the limitations of arrays of > parameterized types. Could we ban the assignment to Object[]? Specifically, an array of function types cannot be assigned to an array of any superclass of the function type (including contra/covariant modifications). It's inconsistent, but so is banning arrays of function types. Lawrence From neal at gafter.com Mon Feb 8 11:53:48 2010 From: neal at gafter.com (Neal Gafter) Date: Mon, 8 Feb 2010 11:53:48 -0800 Subject: Function types versus arrays In-Reply-To: <997cab101002081143v63692adcp916e659d74f5cea@mail.gmail.com> References: <15e8b9d21002071041j57e926e9k8e7c36e01f80ee46@mail.gmail.com> <4B6FE4D4.2040208@univ-mlv.fr> <4B706662.7050603@sun.com> <997cab101002081143v63692adcp916e659d74f5cea@mail.gmail.com> Message-ID: <15e8b9d21002081153t7418c45ah41facf3dfbc754f6@mail.gmail.com> On Mon, Feb 8, 2010 at 11:43 AM, Lawrence Kesteloot wrote: > On Mon, Feb 8, 2010 at 11:30 AM, Alex Buckley wrote: >> I am interested in people's thoughts on the limitations of arrays of >> parameterized types. > > Could we ban the assignment to Object[]? Specifically, an array of > function types cannot be assigned to an array of any superclass of the > function type (including contra/covariant modifications). It's > inconsistent, but so is banning arrays of function types. You can't ban an assignment that is a widening reference conversion. Even if you tried to ban it directly, there are lots of ways to work around it (within the type system) using generics, or by doing something like assigning it to Object and then casting to Object[]. From markmahieu at googlemail.com Mon Feb 8 12:22:55 2010 From: markmahieu at googlemail.com (Mark Mahieu) Date: Mon, 8 Feb 2010 20:22:55 +0000 Subject: Function types versus arrays In-Reply-To: <4B706662.7050603@sun.com> References: <15e8b9d21002071041j57e926e9k8e7c36e01f80ee46@mail.gmail.com> <4B6FE4D4.2040208@univ-mlv.fr> <4B706662.7050603@sun.com> Message-ID: <4D309AE5-FD1B-4C78-B8E8-7A595402B0C9@googlemail.com> On 8 Feb 2010, at 19:30, Alex Buckley wrote: > > I am interested in people's thoughts on the limitations of arrays of > parameterized types. (Reification was not and is not an option, so > please no ranting about the stupid implications of erasure.) Does anyone > care about arrays anymore? Not in this context. They occasionally see the light of day in APIs which 'serialize' data to a common-interchange format, but there the types rarely get more advanced than boolean, numbers, Strings and Dates - I can't see function types coming into play. Beyond that, they mainly occur deep in the implementation of certain kinds of API, and as Mark Thornton suggests, they're normally arrays of some type variable rather than of some specific type. Never say never I suppose, but there appear to be adequate workarounds anyway. > > Alex Mark From abies at adres.pl Mon Feb 8 12:37:01 2010 From: abies at adres.pl (Artur Biesiadowski) Date: Mon, 08 Feb 2010 21:37:01 +0100 Subject: Function types versus arrays In-Reply-To: <4B6FE4D4.2040208@univ-mlv.fr> References: <15e8b9d21002071041j57e926e9k8e7c36e01f80ee46@mail.gmail.com> <4B6FE4D4.2040208@univ-mlv.fr> Message-ID: <4B7075ED.4040400@adres.pl> R?mi Forax wrote: > For the record the problem is: > > #String(String)[] arrayOfFunction = new #String(String)[10]; > Object[] array = arrayOfFunction; > array[0] = #int() (2); > > The last line should raise an array store exception. > So the VM should be aware of the precise type of the array of function. > > Currently, neither the anonymous class translation nor the method handle one > provide reified function type so array of function types are unsafe. > What about having auto-generated interfaces (abstract classes?) representing lambdas in well known place? Something like java.gen.*, with names being equal to signature of the lambda (escaped/encoded as needed). It would require considerably trickery to allocate them in most parent possible classloader (so all java.* +primitives would go to root classloader, anything referencing application classes would have to stay there) - but not too high, as multiple classloaders can load distinct classes with same names. Could be probably done purely on classloader code, even without jvm support (for the interface lookup/generation at least). I think it would solve arrays issues. Problem is polluting the top-level classloader with loads of primitive/core lambda types, but I don't think it is a showstopper. Regards, Artur Biesiadowski From neal at gafter.com Mon Feb 8 13:11:53 2010 From: neal at gafter.com (Neal Gafter) Date: Mon, 8 Feb 2010 13:11:53 -0800 Subject: Function types versus arrays In-Reply-To: <4B7075ED.4040400@adres.pl> References: <15e8b9d21002071041j57e926e9k8e7c36e01f80ee46@mail.gmail.com> <4B6FE4D4.2040208@univ-mlv.fr> <4B7075ED.4040400@adres.pl> Message-ID: <15e8b9d21002081311i1f770521m229ae95864ac6350@mail.gmail.com> On Mon, Feb 8, 2010 at 12:37 PM, Artur Biesiadowski wrote: > What about having auto-generated interfaces (abstract classes?) > representing lambdas in well known place? Something like java.gen.*, > with names being equal to signature of the lambda (escaped/encoded as > needed). The obvious way of doing this doesn't properly handle the subtype relationships among function types. Cheers, Neal From jjb at google.com Mon Feb 8 14:47:23 2010 From: jjb at google.com (Joshua Bloch) Date: Mon, 8 Feb 2010 17:47:23 -0500 Subject: Function types versus arrays In-Reply-To: <4B706662.7050603@sun.com> References: <15e8b9d21002071041j57e926e9k8e7c36e01f80ee46@mail.gmail.com> <4B6FE4D4.2040208@univ-mlv.fr> <4B706662.7050603@sun.com> Message-ID: <17b2302a1002081447x65e0ce85of0d6352cc56e395e@mail.gmail.com> I just assumed that we'd do whatever it takes to make this works, up to and including VM changes. At least we don't have the migration compatibility issues that we had with generics. If arrays of functions don't work, functions are a second-class type family. That arrays and generics don't play nicely together is a major pain in the butt. I really would prefer not to see this repeated. I understand the difficulties in making this work but (as I keep saying) we shouldn't give up without a fight. Much as some of us consider arrays to be denigrated in favor of List, they are still *widely *used, even in new code. If arrays of function types aren't supported, people will complain. And they may rightly accuse us of doing what was easy to implement rather than what is easy to use. Regards from Cape Canaveral, Josh On Mon, Feb 8, 2010 at 2:30 PM, Alex Buckley wrote: > Neal raises a fair point. I recognize that arrays of function types are > as potentially unsafe at runtime as arrays of non-wildcard parameterized > types. > > And this is the same problem one sees when passing variables of > parameterized types as varargs parameters. The varargs method body can > silently do unsafe things, thanks to Object being a universal supertype. > > Banning arrays of function types is an option. It somewhat conflicts > with Project Coin's promotion of [] syntax to read/write collections. > > I am interested in people's thoughts on the limitations of arrays of > parameterized types. (Reification was not and is not an option, so > please no ranting about the stupid implications of erasure.) Does anyone > care about arrays anymore? > > Alex > > R?mi Forax wrote: > > Le 07/02/2010 19:41, Neal Gafter a ?crit : > >> I've been asked if arrays and function types will play nicely > >> together. Specifically, for example, whether or not something like > >> the following will be allowed: > >> > >> #String(String)[] arrayOfFunction = new #String(String)[10]; > >> > >> The current draft spec, by the absence of any constraining rules, > >> implies yes. But if I have to guess, I would say the end result of > >> project lambda will probably say no. Although we haven't talked much > >> about implementation techniques, I'm not aware of any proposed > >> implementation technique that would allow this without opening a hole > >> in the type system. > >> > >> I don't know whether we should consider this important or not (I > >> don't; I'm perfectly happy using java.util.List), but if it is > >> important then someone should be thinking about what needs to happen > >> (VM support?) to make it work. > >> > > > > For the record the problem is: > > > > #String(String)[] arrayOfFunction = new #String(String)[10]; > > Object[] array = arrayOfFunction; > > array[0] = #int() (2); > > > > The last line should raise an array store exception. > > So the VM should be aware of the precise type of the array of function. > > > > Currently, neither the anonymous class translation nor the method handle > one > > provide reified function type so array of function types are unsafe. > > > > Because function type doesn't exist in the language, I propose to make > > the syntax > > for an array of function type (#String(String)[]) illegal. > > > > I'm also happy with List, perhaps we should introduce a new interface > > ReadOnlyList > > which is a super type of List and array of objects but that another > story. > > > >> Cheers, > >> Neal > >> > > > > R?mi > > > > From howard.lovatt at gmail.com Mon Feb 8 14:52:02 2010 From: howard.lovatt at gmail.com (Howard Lovatt) Date: Tue, 9 Feb 2010 09:52:02 +1100 Subject: Function types versus arrays Message-ID: <3dd3f56a1002081452o36d2b695kc766842ec2e8348b@mail.gmail.com> It would be very odd if arrays of function types were not supported, just not Java. The worry about ArrayStoreExceptions is in my experience overstated, it is exceptionally rare and caught at runtime anyway. > I am interested in people's thoughts on the limitations of arrays of > parameterized types. (Reification was not and is not an option, so > please no ranting about the stupid implications of erasure.) Does anyone > care about arrays anymore? > > Alex From peter.levart at gmail.com Mon Feb 8 15:05:48 2010 From: peter.levart at gmail.com (Peter Levart) Date: Tue, 9 Feb 2010 00:05:48 +0100 Subject: Function types versus arrays (reification using interface injection) In-Reply-To: <4B706662.7050603@sun.com> References: <15e8b9d21002071041j57e926e9k8e7c36e01f80ee46@mail.gmail.com> <4B6FE4D4.2040208@univ-mlv.fr> <4B706662.7050603@sun.com> Message-ID: <201002090005.48443.peter.levart@gmail.com> On Monday 08 February 2010 20:30:42 Alex Buckley wrote: > Neal raises a fair point. I recognize that arrays of function types are > as potentially unsafe at runtime as arrays of non-wildcard parameterized > types. > > And this is the same problem one sees when passing variables of > parameterized types as varargs parameters. The varargs method body can > silently do unsafe things, thanks to Object being a universal supertype. > > Banning arrays of function types is an option. It somewhat conflicts > with Project Coin's promotion of [] syntax to read/write collections. > > I am interested in people's thoughts on the limitations of arrays of > parameterized types. (Reification was not and is not an option, so > please no ranting about the stupid implications of erasure.) Does anyone > care about arrays anymore? Hello Alex, Did you mean "Reification was not and is not an option" for general case of generic types or did you mean it is not an option "just for function types" too? I don't know, maybe this idea will sound stupid, but I'll try anyway... If you implement function types as interfaces, then function types that take just primitive parameters and have primitive return type are already reified aren't they? When a function type takes a reference type parameter and/or has a reference return type, then some proposals represented it as a generic interface which is represented in run-time as an interface with erased parameter/return types - such function type is not reified. Why not treat function types with reference type parameters and/or return type the same as function types with primitive only parameters and return type? Because of sub-type relationships among function types? Read on... What we're trying to achieve is that following would be possible: #String(Object) func1 = #(Object o)(o == null ? "null" : o.toString()); #Object(String) func2 = func1; If the two function types above were represented as the following two interfaces (respectively): public interface FuncObjectToString { String invoke(Object param); } public interface FuncStringToObject { Object invoke(String param); } Then the javac compiler could, for the lambda expression above, generate a class that would implement interface FuncObjectToString: public class LambdaImpl implements FuncObjectToString { ... } JVM's classloader would have to maintain a registry of all function type interfaces and implementations (lambda classes). Each time a new function type or lambda implementation class is loaded, JVM would consolidate implementation classes by "injecting" function interfaces into them and generating "bridge" methods (http://blogs.sun.com/jrose/entry/interface_injection_in_the_vm). In the above example, if FuncObjectToString and LambdaImpl were already loaded after the following line was executed: #String(Object) func1 = #(Object o)(o == null ? "null" : o.toString()); which is just a translation of: FuncObjectToString func1 = new LambdaImpl(); and the next line is about to be executed: #Object(String) func2 = func1; which is just a translation of: FuncStringToObject func2 = (FuncStringToObject) func1; Then JVM would proceed by "lazy" loading of interface FuncStringToObject which would trigger injecting this interface into all registered lambda classes that are "by definition" subtypes of FuncStringToObject. It's invoke method would be "implemented" as a "bridge" method invoking lambda's main invoke method. So after FuncStringToObject is loaded LambdaImpl would be implementing an additional interface "FuncStringToObject" so a cast occuring after that point would succeed. This would "reify" function types as much as possible. Not totally, since the following two types for example: #List() and #List() would still be represented by the same interface at runtime. Also this would by default return incorrect result: #Object(String).class.isAssignableFrom(#String(Object).class) But it could be made to return correct result. Is this to lunatic? Am I missing a bunch of problems possible with this approach? Regards, Peter > > Alex > > R?mi Forax wrote: > > Le 07/02/2010 19:41, Neal Gafter a ?crit : > >> I've been asked if arrays and function types will play nicely > >> together. Specifically, for example, whether or not something like > >> the following will be allowed: > >> > >> #String(String)[] arrayOfFunction = new #String(String)[10]; > >> > >> The current draft spec, by the absence of any constraining rules, > >> implies yes. But if I have to guess, I would say the end result of > >> project lambda will probably say no. Although we haven't talked much > >> about implementation techniques, I'm not aware of any proposed > >> implementation technique that would allow this without opening a hole > >> in the type system. > >> > >> I don't know whether we should consider this important or not (I > >> don't; I'm perfectly happy using java.util.List), but if it is > >> important then someone should be thinking about what needs to happen > >> (VM support?) to make it work. > > > > For the record the problem is: > > > > #String(String)[] arrayOfFunction = new #String(String)[10]; > > Object[] array = arrayOfFunction; > > array[0] = #int() (2); > > > > The last line should raise an array store exception. > > So the VM should be aware of the precise type of the array of function. > > > > Currently, neither the anonymous class translation nor the method handle > > one provide reified function type so array of function types are unsafe. > > > > Because function type doesn't exist in the language, I propose to make > > the syntax > > for an array of function type (#String(String)[]) illegal. > > > > I'm also happy with List, perhaps we should introduce a new interface > > ReadOnlyList > > which is a super type of List and array of objects but that another > > story. > > > >> Cheers, > >> Neal > > > > R?mi > From neal at gafter.com Mon Feb 8 15:26:04 2010 From: neal at gafter.com (Neal Gafter) Date: Mon, 8 Feb 2010 15:26:04 -0800 Subject: Function types versus arrays (reification using interface injection) In-Reply-To: <201002090005.48443.peter.levart@gmail.com> References: <15e8b9d21002071041j57e926e9k8e7c36e01f80ee46@mail.gmail.com> <4B6FE4D4.2040208@univ-mlv.fr> <4B706662.7050603@sun.com> <201002090005.48443.peter.levart@gmail.com> Message-ID: <15e8b9d21002081526g3ce4d632m895b8d0b3c845927@mail.gmail.com> On Mon, Feb 8, 2010 at 3:05 PM, Peter Levart wrote: > If you implement function types as interfaces, then function types that take just primitive > parameters and have primitive return type are already reified aren't they? Not necessarily, because function types also have exception signatures, and must obey subtype relations that respect them. > When a function type takes a reference type parameter and/or has a reference return type, then > some proposals represented it as a generic interface which is represented in run-time as an > interface with erased parameter/return types - such function type is not reified. > > Why not treat function types with reference type parameters and/or return type the same as > function types with primitive only parameters and return type? Because of sub-type relationships > among function types? No, because it doesn't work for code in generic methods, which must be compiled to use particular classes. > Then the javac compiler could, for the lambda expression above, generate a class that would > implement interface FuncObjectToString: You can't have code that is required to execute when a subtype relation is involved, because that doesn't interact properly with subtype relations involving wildcards. Specifically, a value of type X should be a subtype of X whenever A is a subtype of B. Your translation scheme doesn't provide any way to make that happen when A and B are related (but not the same) function types. > Is this to lunatic? Am I missing a bunch of problems possible with this approach? Howard Lovatt has proposed something essentially similar, and patched his proposal every time I asked him how some particular example would work. Nothing like this could possibly be considered unless the scheme has been proven to result in no holes in the type system. I haven't yet seen any attempt at a proof for a scheme like this. From neal at gafter.com Mon Feb 8 15:37:23 2010 From: neal at gafter.com (Neal Gafter) Date: Mon, 8 Feb 2010 15:37:23 -0800 Subject: Function types versus arrays In-Reply-To: <3dd3f56a1002081452o36d2b695kc766842ec2e8348b@mail.gmail.com> References: <3dd3f56a1002081452o36d2b695kc766842ec2e8348b@mail.gmail.com> Message-ID: <15e8b9d21002081537p5ed0b3ebs21c06a8decdced74@mail.gmail.com> On Mon, Feb 8, 2010 at 2:52 PM, Howard Lovatt wrote: > It would be very odd if arrays of function types were not supported, > just not Java. The worry about ArrayStoreExceptions is in my > experience overstated, it is exceptionally rare and caught at runtime > anyway. The worry is the *absence* of ArrayStoreException when a lambda with the wrong type is placed into an array. If we could make it safe at runtime by throwing ArrayStoreException, then I wouldn't be worried about allowing arrays of function type. From neal at gafter.com Mon Feb 8 15:40:23 2010 From: neal at gafter.com (Neal Gafter) Date: Mon, 8 Feb 2010 15:40:23 -0800 Subject: Function types versus arrays In-Reply-To: <17b2302a1002081447x65e0ce85of0d6352cc56e395e@mail.gmail.com> References: <15e8b9d21002071041j57e926e9k8e7c36e01f80ee46@mail.gmail.com> <4B6FE4D4.2040208@univ-mlv.fr> <4B706662.7050603@sun.com> <17b2302a1002081447x65e0ce85of0d6352cc56e395e@mail.gmail.com> Message-ID: <15e8b9d21002081540q5bb1148x9b6e2d8aa4982d41@mail.gmail.com> On Mon, Feb 8, 2010 at 2:47 PM, Joshua Bloch wrote: > I just assumed that we'd do whatever it takes to make this works, up to and > including VM changes. I'd love to hear ideas about how to make it work. The only way I know to make it work would be to reify generics. Other than that, and absent any well founded ideas, there isn't anything to do. From forax at univ-mlv.fr Mon Feb 8 16:05:10 2010 From: forax at univ-mlv.fr (=?ISO-8859-1?Q?R=E9mi_Forax?=) Date: Tue, 09 Feb 2010 01:05:10 +0100 Subject: Function types versus arrays In-Reply-To: <15e8b9d21002081540q5bb1148x9b6e2d8aa4982d41@mail.gmail.com> References: <15e8b9d21002071041j57e926e9k8e7c36e01f80ee46@mail.gmail.com> <4B6FE4D4.2040208@univ-mlv.fr> <4B706662.7050603@sun.com> <17b2302a1002081447x65e0ce85of0d6352cc56e395e@mail.gmail.com> <15e8b9d21002081540q5bb1148x9b6e2d8aa4982d41@mail.gmail.com> Message-ID: <4B70A6B6.6050106@univ-mlv.fr> Le 09/02/2010 00:40, Neal Gafter a ?crit : > On Mon, Feb 8, 2010 at 2:47 PM, Joshua Bloch wrote: > >> I just assumed that we'd do whatever it takes to make this works, up to and >> including VM changes. >> > I'd love to hear ideas about how to make it work. The only way I know > to make it work would be to reify generics. Other than that, and > absent any well founded ideas, there isn't anything to do. > > Or choose to use method handles :) In that case, the rules are the same as the ones for generics. The common supertype of all function types is java.dyn.MethodHandle - you can't create an array of function types - you can create an array of MethodHandle - casting an array of MethodHandle to an array of function types is an unsafe conversion. In that case, the following code is legal: #int(int)[] array = (#int(int)[])new MethodHandle[10]; but the cast will raise a warning. Because function type are translated at compile time to MethodHandle, no ClassCastException at runtime. BTW, I see two use cases for array of function types, the first one is if someone want to simulate something like a vtable. The second one is if you want an array of getter or an array of setters. R?mi From neal at gafter.com Mon Feb 8 16:09:41 2010 From: neal at gafter.com (Neal Gafter) Date: Mon, 8 Feb 2010 16:09:41 -0800 Subject: Function types versus arrays In-Reply-To: <4B70A6B6.6050106@univ-mlv.fr> References: <15e8b9d21002071041j57e926e9k8e7c36e01f80ee46@mail.gmail.com> <4B6FE4D4.2040208@univ-mlv.fr> <4B706662.7050603@sun.com> <17b2302a1002081447x65e0ce85of0d6352cc56e395e@mail.gmail.com> <15e8b9d21002081540q5bb1148x9b6e2d8aa4982d41@mail.gmail.com> <4B70A6B6.6050106@univ-mlv.fr> Message-ID: <15e8b9d21002081609o7d3a098tc53f89f3e546c1cc@mail.gmail.com> On Mon, Feb 8, 2010 at 4:05 PM, R?mi Forax wrote: > Le 09/02/2010 00:40, Neal Gafter a ?crit : >> On Mon, Feb 8, 2010 at 2:47 PM, Joshua Bloch ?wrote: >>> I just assumed that we'd do whatever it takes to make this works, up to and >>> including VM changes. >>> >> I'd love to hear ideas about how to make it work. ?The only way I know >> to make it work would be to reify generics. ?Other than that, and >> absent any well founded ideas, there isn't anything to do. > > Or choose to use method handles :) > > In that case, the rules are the same as the ones for generics. > The common supertype of all function types is java.dyn.MethodHandle > - you can't create an array of function types In other words, make it work by not making it work? I don't get it. You're joking? From peter.levart at gmail.com Mon Feb 8 16:21:41 2010 From: peter.levart at gmail.com (Peter Levart) Date: Tue, 9 Feb 2010 01:21:41 +0100 Subject: Function types versus arrays (reification using interface injection) In-Reply-To: <15e8b9d21002081526g3ce4d632m895b8d0b3c845927@mail.gmail.com> References: <15e8b9d21002071041j57e926e9k8e7c36e01f80ee46@mail.gmail.com> <201002090005.48443.peter.levart@gmail.com> <15e8b9d21002081526g3ce4d632m895b8d0b3c845927@mail.gmail.com> Message-ID: <201002090121.41316.peter.levart@gmail.com> > On Mon, Feb 8, 2010 at 3:05 PM, Peter Levart wrote: > > Is this to lunatic? Am I missing a bunch of problems possible with this > > approach? > And the worst thing is that the following would not be possible: #Object(String)[] array = new #String(Object)[10]; ...which defies the purpose of reifying function types in order to enable arrays of function types. I new it was a stupid idea ;-) Regards, Peter From forax at univ-mlv.fr Mon Feb 8 16:30:48 2010 From: forax at univ-mlv.fr (=?ISO-8859-1?Q?R=E9mi_Forax?=) Date: Tue, 09 Feb 2010 01:30:48 +0100 Subject: Function types versus arrays In-Reply-To: <15e8b9d21002081609o7d3a098tc53f89f3e546c1cc@mail.gmail.com> References: <15e8b9d21002071041j57e926e9k8e7c36e01f80ee46@mail.gmail.com> <4B6FE4D4.2040208@univ-mlv.fr> <4B706662.7050603@sun.com> <17b2302a1002081447x65e0ce85of0d6352cc56e395e@mail.gmail.com> <15e8b9d21002081540q5bb1148x9b6e2d8aa4982d41@mail.gmail.com> <4B70A6B6.6050106@univ-mlv.fr> <15e8b9d21002081609o7d3a098tc53f89f3e546c1cc@mail.gmail.com> Message-ID: <4B70ACB8.6010104@univ-mlv.fr> Le 09/02/2010 01:09, Neal Gafter a ?crit : > On Mon, Feb 8, 2010 at 4:05 PM, R?mi Forax wrote: > >> Le 09/02/2010 00:40, Neal Gafter a ?crit : >> >>> On Mon, Feb 8, 2010 at 2:47 PM, Joshua Bloch wrote: >>> >>>> I just assumed that we'd do whatever it takes to make this works, up to and >>>> including VM changes. >>>> >>>> >>> I'd love to hear ideas about how to make it work. The only way I know >>> to make it work would be to reify generics. Other than that, and >>> absent any well founded ideas, there isn't anything to do. >>> >> Or choose to use method handles :) >> >> In that case, the rules are the same as the ones for generics. >> The common supertype of all function types is java.dyn.MethodHandle >> - you can't create an array of function types >> > In other words, make it work by not making it work? I don't get it. > You're joking? > Yes and No :) I propose to allow array of function type but not to be able to create it, exactly like array of non reified parameterized type. R?mi From lk at teamten.com Mon Feb 8 16:42:43 2010 From: lk at teamten.com (Lawrence Kesteloot) Date: Mon, 8 Feb 2010 16:42:43 -0800 Subject: Function types versus arrays In-Reply-To: <4B70A6B6.6050106@univ-mlv.fr> References: <15e8b9d21002071041j57e926e9k8e7c36e01f80ee46@mail.gmail.com> <4B6FE4D4.2040208@univ-mlv.fr> <4B706662.7050603@sun.com> <17b2302a1002081447x65e0ce85of0d6352cc56e395e@mail.gmail.com> <15e8b9d21002081540q5bb1148x9b6e2d8aa4982d41@mail.gmail.com> <4B70A6B6.6050106@univ-mlv.fr> Message-ID: <997cab101002081642u1dd3e750g68bf7be9755b912b@mail.gmail.com> On Mon, Feb 8, 2010 at 4:05 PM, R?mi Forax wrote: > BTW, I see two use cases for array of function types, > the first one is if someone want to simulate something like a vtable. > The second one is if you want an array of getter or an array of setters. Why wouldn't ArrayList work for these? Lawrence From forax at univ-mlv.fr Mon Feb 8 17:14:57 2010 From: forax at univ-mlv.fr (=?ISO-8859-1?Q?R=E9mi_Forax?=) Date: Tue, 09 Feb 2010 02:14:57 +0100 Subject: Function type -postfix Message-ID: <4B70B711.5050305@univ-mlv.fr> I wonder if postfix syntax id not better than the prefix one. Let's try some examples: a function that takes an int and returns an int: int->int a function that takes two ints and returns an int: int,int->int I think this syntax will not generate conflicts despite the fact that ',' can be used in expressions if int->int.class and implementing a function type are not allowed. Here parenthesis are mandatory, the rule is if you want to have a function type in a function type, parens are required: int, (int->int)->int this means that int -> int -> int is illegal and should be written: int -> (int -> int) or (int -> int) -> int The syntax for array of function type, again parenthesis are mandatory: (int->int)[] The syntax with an exception: throws is before the arrow: int,int throws Exception->int Here is the syntax of a function that takes an int and a function that takes an int and return an int. The former function throws an Exception and returns an int. int, (int -> int) throws Exception->int Here the syntax is can be confused with int, (int throws Exception -> int) -> int but the rule is simple throws is before the arrow. Here is the proposed grammars: BasicType = ClassType | PrimitiveType | ArrayType ; Type = BasicType | FunctionType ; ArrayType = BasicType '[' ']' | '(' FunctionType ')' '[' ']' ; FunctionType = ElementTypes ThrowsOpt '->' ReturnElementType ; ElementTypes = ElementType | ElementTypes ',' ElementType ; ElementType = BasicType | '(' FunctionType ')' ; ReturnElementType = ElementType | void ; '#' can be added at the starts of the function type, but I think it is not a requirement. R?mi From forax at univ-mlv.fr Mon Feb 8 17:16:20 2010 From: forax at univ-mlv.fr (=?ISO-8859-1?Q?R=E9mi_Forax?=) Date: Tue, 09 Feb 2010 02:16:20 +0100 Subject: Function types versus arrays In-Reply-To: <997cab101002081642u1dd3e750g68bf7be9755b912b@mail.gmail.com> References: <15e8b9d21002071041j57e926e9k8e7c36e01f80ee46@mail.gmail.com> <4B6FE4D4.2040208@univ-mlv.fr> <4B706662.7050603@sun.com> <17b2302a1002081447x65e0ce85of0d6352cc56e395e@mail.gmail.com> <15e8b9d21002081540q5bb1148x9b6e2d8aa4982d41@mail.gmail.com> <4B70A6B6.6050106@univ-mlv.fr> <997cab101002081642u1dd3e750g68bf7be9755b912b@mail.gmail.com> Message-ID: <4B70B764.8010701@univ-mlv.fr> Le 09/02/2010 01:42, Lawrence Kesteloot a ?crit : > On Mon, Feb 8, 2010 at 4:05 PM, R?mi Forax wrote: > >> BTW, I see two use cases for array of function types, >> the first one is if someone want to simulate something like a vtable. >> The second one is if you want an array of getter or an array of setters. >> > Why wouldn't ArrayList work for these? > > Lawrence > It's just less efficient. R?mi From howard.lovatt at iee.org Mon Feb 8 17:33:01 2010 From: howard.lovatt at iee.org (Howard Lovatt) Date: Tue, 9 Feb 2010 01:33:01 +0000 Subject: Function types versus arrays In-Reply-To: <15e8b9d21002081537p5ed0b3ebs21c06a8decdced74@mail.gmail.com> References: <3dd3f56a1002081452o36d2b695kc766842ec2e8348b@mail.gmail.com> <15e8b9d21002081537p5ed0b3ebs21c06a8decdced74@mail.gmail.com> Message-ID: <3dd3f56a1002081733x786e329at5d2f273dcf3cec69@mail.gmail.com> If you have reified lambdas then you don't have a problem, if you have erased lambdas then it is the same as generics. I would favor reified lambdas so that the following happens: #int()[] ils = new #int()[1]; ils[0] = #double() (1); // error But if erased lambdas are used then you need a notation for the raw type of a lambda, just like generics. Perhaps # on its own, e.g.: #[] ls = new #[1]; ls = #double() (1); // ok #int()[] ils = (#int()) new #[1]; // issues warning ils = #double() (1); // error On 8 February 2010 23:37, Neal Gafter wrote: > On Mon, Feb 8, 2010 at 2:52 PM, Howard Lovatt > wrote: > > It would be very odd if arrays of function types were not supported, > > just not Java. The worry about ArrayStoreExceptions is in my > > experience overstated, it is exceptionally rare and caught at runtime > > anyway. > > The worry is the *absence* of ArrayStoreException when a lambda with > the wrong type is placed into an array. If we could make it safe at > runtime by throwing ArrayStoreException, then I wouldn't be worried > about allowing arrays of function type. > -- -- Howard. From neal at gafter.com Mon Feb 8 17:54:45 2010 From: neal at gafter.com (Neal Gafter) Date: Mon, 8 Feb 2010 17:54:45 -0800 Subject: Function types versus arrays In-Reply-To: <3dd3f56a1002081733x786e329at5d2f273dcf3cec69@mail.gmail.com> References: <3dd3f56a1002081452o36d2b695kc766842ec2e8348b@mail.gmail.com> <15e8b9d21002081537p5ed0b3ebs21c06a8decdced74@mail.gmail.com> <3dd3f56a1002081733x786e329at5d2f273dcf3cec69@mail.gmail.com> Message-ID: <15e8b9d21002081754p428832e5mff96cc9586c6a47b@mail.gmail.com> On Mon, Feb 8, 2010 at 5:33 PM, Howard Lovatt wrote: > If you have reified lambdas then you don't have a problem, if you have > erased lambdas then it is the same as generics. Right. And so far, we do not have any proposal that demonstrates how to do one without the other. Cheers, Neal From howard.lovatt at iee.org Mon Feb 8 19:44:34 2010 From: howard.lovatt at iee.org (Howard Lovatt) Date: Tue, 9 Feb 2010 03:44:34 +0000 Subject: Function types versus arrays In-Reply-To: <15e8b9d21002081754p428832e5mff96cc9586c6a47b@mail.gmail.com> References: <3dd3f56a1002081452o36d2b695kc766842ec2e8348b@mail.gmail.com> <15e8b9d21002081537p5ed0b3ebs21c06a8decdced74@mail.gmail.com> <3dd3f56a1002081733x786e329at5d2f273dcf3cec69@mail.gmail.com> <15e8b9d21002081754p428832e5mff96cc9586c6a47b@mail.gmail.com> Message-ID: <3dd3f56a1002081944p6d6e0119g181fb968d7da747a@mail.gmail.com> There is: http://www.artima.com/weblogs/viewpost.jsp?thread=278567 That does indeed allow: #int()[] ils = new #int()[1]; and catches: ils[0] = #double() (1); // error ils[0] = (#int()) #double() (1); // error and would allow: #T()[] tls = (#T()[]) new Object[1]; // unchecked warning and would behave like generics: tls[0] = #double() (1); // error tls[0] = (#T()) #double() (1); // unchecked warning but would catch at runtime: tls[0] = (#T()) #double() (1); // unchecked warning ils = (#int()) tls; // runtime error -- Howard. On 9 February 2010 01:54, Neal Gafter wrote: > On Mon, Feb 8, 2010 at 5:33 PM, Howard Lovatt > wrote: > > If you have reified lambdas then you don't have a problem, if you have > > erased lambdas then it is the same as generics. > > Right. And so far, we do not have any proposal that demonstrates how > to do one without the other. > > Cheers, > Neal > > ______________________________________________________________________ > This email has been scanned by the MessageLabs Email Security System. > For more information please visit http://www.messagelabs.com/email > ______________________________________________________________________ > -- -- Howard. From neal at gafter.com Mon Feb 8 21:29:36 2010 From: neal at gafter.com (Neal Gafter) Date: Mon, 8 Feb 2010 21:29:36 -0800 Subject: Function types versus arrays In-Reply-To: <3dd3f56a1002081944p6d6e0119g181fb968d7da747a@mail.gmail.com> References: <3dd3f56a1002081452o36d2b695kc766842ec2e8348b@mail.gmail.com> <15e8b9d21002081537p5ed0b3ebs21c06a8decdced74@mail.gmail.com> <3dd3f56a1002081733x786e329at5d2f273dcf3cec69@mail.gmail.com> <15e8b9d21002081754p428832e5mff96cc9586c6a47b@mail.gmail.com> <3dd3f56a1002081944p6d6e0119g181fb968d7da747a@mail.gmail.com> Message-ID: <15e8b9d21002082129i21a209cel925ad8b6ba3313cd@mail.gmail.com> On Mon, Feb 8, 2010 at 7:44 PM, Howard Lovatt wrote: > There is: > http://www.artima.com/weblogs/viewpost.jsp?thread=278567 > That does indeed allow: > #T()[] tls = (#T()[]) new Object[1]; // unchecked warning Perhaps I should have been more specific: there is no *sound* proposal for how to do it. From howard.lovatt at iee.org Mon Feb 8 22:01:17 2010 From: howard.lovatt at iee.org (Howard Lovatt) Date: Tue, 9 Feb 2010 06:01:17 +0000 Subject: Function types versus arrays In-Reply-To: <15e8b9d21002082129i21a209cel925ad8b6ba3313cd@mail.gmail.com> References: <3dd3f56a1002081452o36d2b695kc766842ec2e8348b@mail.gmail.com> <15e8b9d21002081537p5ed0b3ebs21c06a8decdced74@mail.gmail.com> <3dd3f56a1002081733x786e329at5d2f273dcf3cec69@mail.gmail.com> <15e8b9d21002081754p428832e5mff96cc9586c6a47b@mail.gmail.com> <3dd3f56a1002081944p6d6e0119g181fb968d7da747a@mail.gmail.com> <15e8b9d21002082129i21a209cel925ad8b6ba3313cd@mail.gmail.com> Message-ID: <3dd3f56a1002082201l304a5473ge2859fcc52032913@mail.gmail.com> Well you *speculate* that reified lambdas are type unsound, you haven't and nor has anyone else demonstrated that - in fact quite the opposite. I have asked for examples of problem areas and your response has been that you want to see a formal proof. However, I would say "what area isn't already proven?" The technique translates into current Java, it doesn't use raw types, and it doesn't do type inference. To the extent that current Java is type sound; the reified lambdas are type sound, i.e. no change to the type safety. I could prove that contravariant overriding is type sound; but so what, I assume that everyone accepts that. Perhaps you could be specific about which area you think there is a problem? On 9 February 2010 05:29, Neal Gafter wrote: > On Mon, Feb 8, 2010 at 7:44 PM, Howard Lovatt > wrote: > > There is: > > http://www.artima.com/weblogs/viewpost.jsp?thread=278567 > > That does indeed allow: > > #T()[] tls = (#T()[]) new Object[1]; // unchecked warning > > Perhaps I should have been more specific: there is no *sound* proposal > for how to do it. > > ______________________________________________________________________ > This email has been scanned by the MessageLabs Email Security System. > For more information please visit http://www.messagelabs.com/email > ______________________________________________________________________ > -- -- Howard. From neal at gafter.com Mon Feb 8 22:13:57 2010 From: neal at gafter.com (Neal Gafter) Date: Mon, 8 Feb 2010 22:13:57 -0800 Subject: Function types versus arrays In-Reply-To: <3dd3f56a1002082201l304a5473ge2859fcc52032913@mail.gmail.com> References: <3dd3f56a1002081452o36d2b695kc766842ec2e8348b@mail.gmail.com> <15e8b9d21002081537p5ed0b3ebs21c06a8decdced74@mail.gmail.com> <3dd3f56a1002081733x786e329at5d2f273dcf3cec69@mail.gmail.com> <15e8b9d21002081754p428832e5mff96cc9586c6a47b@mail.gmail.com> <3dd3f56a1002081944p6d6e0119g181fb968d7da747a@mail.gmail.com> <15e8b9d21002082129i21a209cel925ad8b6ba3313cd@mail.gmail.com> <3dd3f56a1002082201l304a5473ge2859fcc52032913@mail.gmail.com> Message-ID: <15e8b9d21002082213xbf17872v34593d712410e4e6@mail.gmail.com> On Mon, Feb 8, 2010 at 10:01 PM, Howard Lovatt wrote: > Well you *speculate* that reified lambdas are type unsound, you haven't and > nor has anyone else demonstrated that - in fact quite the opposite. I've demonstrated your proposal unsound three times, and each time you modified it and asked again. I gave up looking for loopholes, as that game can go on forever. The burden of proof is yours, not mine. The absence of counterexamples is not evidence of soundness. > I have asked for examples of problem areas and your response has been that > you want to see a formal proof. However, I would say "what area isn't > already proven?" Soundness isn't a local property, it is a global property that accrues to the system as a whole. The system as a whole hasn't been proven sound. From schulz at the-loom.de Mon Feb 8 23:29:49 2010 From: schulz at the-loom.de (Stefan Schulz) Date: Tue, 09 Feb 2010 08:29:49 +0100 Subject: Function type -postfix In-Reply-To: <4B70B711.5050305@univ-mlv.fr> References: <4B70B711.5050305@univ-mlv.fr> Message-ID: <4B710EED.6080006@the-loom.de> And I still wonder, why not using a clear, lambda-resembling syntax with mandatory brackets instead of a leading #. It would clearly separate nested function types and not require extra parens for throws declaration. Having optional parens that are mandatory in some cases does not sound convincing to me, despite that postfix syntax "reads the wrong direction" for the common Java developer's eye. Examples within for comparison. On 09.02.2010 02:14, R?mi Forax wrote: > int->int #int(int) // draft [int(int)] // brackets > int,int->int #int(int,int) [int(int,int)] > I think this syntax will not generate conflicts despite the fact > that ',' can be used in expressions if int->int.class and > implementing a function type are not allowed. #int(int).class // not sure [int(int)].class > Here parenthesis are mandatory, the rule is if you want to have > a function type in a function type, parens are required: > int, (int->int)->int #int(int, #int(int)) [int(int, [int(int)])] > this means that int -> int -> int is illegal and > should be written: > int -> (int -> int) or (int -> int) -> int ##int(int)(int) or #int(#int(int)) [[int(int)](int)] or [int([int(int)])] > (int->int)[] #int(int)[] [int(int)][] > int,int throws Exception->int #int(int,int) (throws Exception) [int(int,int) throws Exception] > int, (int -> int) throws Exception->int #int(int, #int(int) (throws Exception)) [int(int, [int(int) throws Exception])] > int, (int throws Exception -> int) -> int #int(int, #int(int) (throws Exception)) [int(int, [int(int) throws Exception])] "brackets" add one character per function type compared to "draft" but parenthesises a type, not only introduces one. I only can think of one place of "visual" conflict, where an array is defined giving its size by a method: Object[size(param)] But maybe I overlooked something. Stefan From peter.levart at marand.si Tue Feb 9 00:14:35 2010 From: peter.levart at marand.si (Peter Levart) Date: Tue, 9 Feb 2010 09:14:35 +0100 Subject: Function type -postfix In-Reply-To: <4B710EED.6080006@the-loom.de> References: <4B70B711.5050305@univ-mlv.fr> <4B710EED.6080006@the-loom.de> Message-ID: <201002090914.35153.peter.levart@marand.si> On Tuesday 09 February 2010 08:29:49 Stefan Schulz wrote: > > ##int(int)(int) or #int(#int(int)) > [[int(int)](int)] or [int([int(int)])] I have one objection to this syntax - it's too noisy on parens/brackets. The above example shows that even prefix notation is easier on the eye than your proposed bracketed syntax. It's nice if a pair of brackets or parens enclose the whole construct, but then it should not require a second pair of parens/brackets inside. Regards, Peter From peter.levart at marand.si Tue Feb 9 01:45:33 2010 From: peter.levart at marand.si (Peter Levart) Date: Tue, 9 Feb 2010 10:45:33 +0100 Subject: yet another function types syntax variation Message-ID: <201002091045.33689.peter.levart@marand.si> This time a variation of the proposed (prefix) notation which goes like: FunctionType: '#' ReturnType '(' ParameterTypes_opt ')' Throws_opt What would happen if the '#' and 'ReturnType' changed places? FunctionType: ReturnType '#' '(' ParameterTypes_opt ')' Throws_opt Nested return types might look better: ##int(byte[],int,int)(throws IOException)(File)(throws IOException) vs. int #(byte[],int,int) throws IOException #(File) throws IOException Here the '#' is in the place where a method name is expected, so everything on the left of '#' is a return type. One might prefer puting an optional space between ReturnType and '#' to make it visualy lighter. Using '|' instead of ',' between throws types eliminates the need for a second pair of parens. What do you think? Regards, Peter From peter.levart at marand.si Tue Feb 9 03:04:32 2010 From: peter.levart at marand.si (Peter Levart) Date: Tue, 9 Feb 2010 12:04:32 +0100 Subject: yet another function types syntax variation In-Reply-To: <201002091045.33689.peter.levart@marand.si> References: <201002091045.33689.peter.levart@marand.si> Message-ID: <201002091204.32952.peter.levart@marand.si> On Tuesday 09 February 2010 10:45:33 Peter Levart wrote: > int #(byte[],int,int) throws IOException #(File) throws IOException ...when such (nested) type is the result type of a method: public static int #(byte[],int,int) throws IOException #(File) throws IOException filer(int mode) throws IllegalArgumentException { ... } the sequence continues with similar pattern where '#' is finally replaced with method name. IDEs might reformat such cases like this: public static int #(byte[],int,int) throws IOException #(File) throws IOException filer(int mode) throws IllegalArgumentException { ... } to improve visual clearness. Peter From forax at univ-mlv.fr Tue Feb 9 03:24:40 2010 From: forax at univ-mlv.fr (=?ISO-8859-1?Q?R=E9mi_Forax?=) Date: Tue, 09 Feb 2010 12:24:40 +0100 Subject: Function type -postfix In-Reply-To: <4B710EED.6080006@the-loom.de> References: <4B70B711.5050305@univ-mlv.fr> <4B710EED.6080006@the-loom.de> Message-ID: <4B7145F8.2070108@univ-mlv.fr> Le 09/02/2010 08:29, Stefan Schulz a ?crit : > And I still wonder, why not using a clear, lambda-resembling syntax > with mandatory brackets instead of a leading #. It would clearly > separate nested function types and not require extra parens for throws > declaration. Having optional parens that are mandatory in some cases > does not sound convincing to me, despite that postfix syntax "reads > the wrong direction" for the common Java developer's eye. > > Examples within for comparison. > > On 09.02.2010 02:14, R?mi Forax wrote: >> int->int > > #int(int) // draft > [int(int)] // brackets > >> int,int->int > > #int(int,int) > [int(int,int)] > >> I think this syntax will not generate conflicts despite the fact >> that ',' can be used in expressions if int->int.class and >> implementing a function type are not allowed. > > #int(int).class // not sure > [int(int)].class > >> Here parenthesis are mandatory, the rule is if you want to have >> a function type in a function type, parens are required: >> int, (int->int)->int > > #int(int, #int(int)) > [int(int, [int(int)])] > >> this means that int -> int -> int is illegal and >> should be written: >> int -> (int -> int) or (int -> int) -> int > > ##int(int)(int) or #int(#int(int)) > [[int(int)](int)] or [int([int(int)])] > >> (int->int)[] > > #int(int)[] > [int(int)][] > >> int,int throws Exception->int > > #int(int,int) (throws Exception) > [int(int,int) throws Exception] > >> int, (int -> int) throws Exception->int > > #int(int, #int(int) (throws Exception)) > [int(int, [int(int) throws Exception])] > >> int, (int throws Exception -> int) -> int > > #int(int, #int(int) (throws Exception)) > [int(int, [int(int) throws Exception])] > > "brackets" add one character per function type compared to "draft" but > parenthesises a type, not only introduces one. > > I only can think of one place of "visual" conflict, where an array is > defined giving its size by a method: > > Object[size(param)] > > But maybe I overlooked something. > > Stefan > Stefan, I've tried to explain prefix and postfix notation last week-end at FOSDEM. At the end of the talk and the day after, I've talked to 'real' people about that. Here are the questions often asked about the function type syntax: - I prefer prefix syntax: #int(int) is more Java-like than int -> int - why do you need parenthesis between the throws [in the prefix syntax] ? - I don't understand ##int()int()int could you explain it ? Now you know as much as me :) About your syntax, I wonder if there is not a conflict with upcoming coin syntax for collection litteral. As Peter says mixing parenthesis and brackets make the syntax is barely unreadable. In my opinion, if you have function type that contains a function type, having parenthesis is mandatory because it's more readable. Having encompassing parenthesis doesn't play well with prefix syntax that also required parenthesis to separate return type from parameter types. R?mi From tronicek at fit.cvut.cz Tue Feb 9 04:44:42 2010 From: tronicek at fit.cvut.cz (Zdenek Tronicek) Date: Tue, 9 Feb 2010 13:44:42 +0100 Subject: Function types versus arrays In-Reply-To: <17b2302a1002081447x65e0ce85of0d6352cc56e395e@mail.gmail.com> References: <15e8b9d21002071041j57e926e9k8e7c36e01f80ee46@mail.gmail.com> <4B6FE4D4.2040208@univ-mlv.fr> <4B706662.7050603@sun.com> <17b2302a1002081447x65e0ce85of0d6352cc56e395e@mail.gmail.com> Message-ID: <86b9ddba80036d42184a24a750a42e00.squirrel@imap.fit.cvut.cz> Another argument why to strive after arrays of function types is that language should be intuitive. And if programmers can create arrays of any type now (except generic arrays), they will, naturally, expect arrays of function types. I think that exceptions in type system are evil. To anticipate questions: I do not have any proposal how to do it and I understand we are limited by the current type system (non-reified generics) and JVM. Z. -- Zdenek Tronicek FIT CTU in Prague Joshua Bloch napsal(a): > I just assumed that we'd do whatever it takes to make this works, up to > and > including VM changes. At least we don't have the migration compatibility > issues that we had with generics. If arrays of functions don't work, > functions are a second-class type family. That arrays and generics don't > play nicely together is a major pain in the butt. I really would prefer > not > to see this repeated. I understand the difficulties in making this work > but > (as I keep saying) we shouldn't give up without a fight. Much as some of > us > consider arrays to be denigrated in favor of List, they are still > *widely > *used, even in new code. If arrays of function types aren't supported, > people will complain. And they may rightly accuse us of doing what was > easy > to implement rather than what is easy to use. > > Regards from Cape Canaveral, > > Josh > > On Mon, Feb 8, 2010 at 2:30 PM, Alex Buckley wrote: > >> Neal raises a fair point. I recognize that arrays of function types are >> as potentially unsafe at runtime as arrays of non-wildcard parameterized >> types. >> >> And this is the same problem one sees when passing variables of >> parameterized types as varargs parameters. The varargs method body can >> silently do unsafe things, thanks to Object being a universal supertype. >> >> Banning arrays of function types is an option. It somewhat conflicts >> with Project Coin's promotion of [] syntax to read/write collections. >> >> I am interested in people's thoughts on the limitations of arrays of >> parameterized types. (Reification was not and is not an option, so >> please no ranting about the stupid implications of erasure.) Does anyone >> care about arrays anymore? >> >> Alex >> >> R?mi Forax wrote: >> > Le 07/02/2010 19:41, Neal Gafter a ?crit : >> >> I've been asked if arrays and function types will play nicely >> >> together. Specifically, for example, whether or not something like >> >> the following will be allowed: >> >> >> >> #String(String)[] arrayOfFunction = new #String(String)[10]; >> >> >> >> The current draft spec, by the absence of any constraining rules, >> >> implies yes. But if I have to guess, I would say the end result of >> >> project lambda will probably say no. Although we haven't talked much >> >> about implementation techniques, I'm not aware of any proposed >> >> implementation technique that would allow this without opening a hole >> >> in the type system. >> >> >> >> I don't know whether we should consider this important or not (I >> >> don't; I'm perfectly happy using java.util.List), but if it is >> >> important then someone should be thinking about what needs to happen >> >> (VM support?) to make it work. >> >> >> > >> > For the record the problem is: >> > >> > #String(String)[] arrayOfFunction = new #String(String)[10]; >> > Object[] array = arrayOfFunction; >> > array[0] = #int() (2); >> > >> > The last line should raise an array store exception. >> > So the VM should be aware of the precise type of the array of >> function. >> > >> > Currently, neither the anonymous class translation nor the method >> handle >> one >> > provide reified function type so array of function types are unsafe. >> > >> > Because function type doesn't exist in the language, I propose to make >> > the syntax >> > for an array of function type (#String(String)[]) illegal. >> > >> > I'm also happy with List, perhaps we should introduce a new interface >> > ReadOnlyList >> > which is a super type of List and array of objects but that another >> story. >> > >> >> Cheers, >> >> Neal >> >> >> > >> > R?mi >> > >> >> > > From tronicek at fit.cvut.cz Tue Feb 9 05:05:58 2010 From: tronicek at fit.cvut.cz (Zdenek Tronicek) Date: Tue, 9 Feb 2010 14:05:58 +0100 Subject: yet another function types syntax variation In-Reply-To: <201002091045.33689.peter.levart@marand.si> References: <201002091045.33689.peter.levart@marand.si> Message-ID: <68bfa7a5a6c5a424f46c92eb0b9d4df0.squirrel@imap.fit.cvut.cz> Peter Levart napsal(a): > What would happen if the '#' and 'ReturnType' changed places? > > FunctionType: > ReturnType '#' '(' ParameterTypes_opt ')' Throws_opt > > > Nested return types might look better: > > ##int(byte[],int,int)(throws IOException)(File)(throws IOException) > > vs. > > int #(byte[],int,int) throws IOException #(File) throws IOException My subjective opinion is: it does not look better. Why? I am reading the line from left to right. If I see ## I know that the return type is function. In your proposal, I have to read all "int #(byte[],int,int) throws IOException" until I get to know that it is return type. Z. -- Zdenek Tronicek FIT CTU in Prague From peter.levart at marand.si Tue Feb 9 08:07:04 2010 From: peter.levart at marand.si (Peter Levart) Date: Tue, 9 Feb 2010 17:07:04 +0100 Subject: yet another function types syntax variation In-Reply-To: <68bfa7a5a6c5a424f46c92eb0b9d4df0.squirrel@imap.fit.cvut.cz> References: <201002091045.33689.peter.levart@marand.si> <68bfa7a5a6c5a424f46c92eb0b9d4df0.squirrel@imap.fit.cvut.cz> Message-ID: <201002091707.04943.peter.levart@marand.si> On Tuesday 09 February 2010 14:05:58 Zdenek Tronicek wrote: > Peter Levart napsal(a): > > What would happen if the '#' and 'ReturnType' changed places? > > > > FunctionType: > > ReturnType '#' '(' ParameterTypes_opt ')' Throws_opt > > > > > > Nested return types might look better: > > > > ##int(byte[],int,int)(throws IOException)(File)(throws IOException) > > > > vs. > > > > int #(byte[],int,int) throws IOException #(File) throws IOException > > My subjective opinion is: it does not look better. Why? I am reading the > line from left to right. If I see ## I know that the return type is > function. In your proposal, I have to read all "int #(byte[],int,int) > throws IOException" until I get to know that it is return type. > If you see ## you know that return type is function, but you still don't know the return type. You have to find a matching ')' to know what constitutes it, isn't that right? This is worsened by the maybe-yes-maybe-not presence of second pair of parens enclosing optional throws declaration. I think that finding '#' is easier since it sticks out from the crowd. Regards, Peter From dl at cs.oswego.edu Tue Feb 9 09:02:29 2010 From: dl at cs.oswego.edu (Doug Lea) Date: Tue, 09 Feb 2010 12:02:29 -0500 Subject: Function types versus arrays In-Reply-To: <4B706662.7050603@sun.com> References: <15e8b9d21002071041j57e926e9k8e7c36e01f80ee46@mail.gmail.com> <4B6FE4D4.2040208@univ-mlv.fr> <4B706662.7050603@sun.com> Message-ID: <4B719525.9040908@cs.oswego.edu> Alex Buckley wrote: > Neal raises a fair point. I recognize that arrays of function types are > as potentially unsafe at runtime as arrays of non-wildcard parameterized > types. > Alternatively, the array botch itself could be addressed by creating an efficient Array class tightly coupled to the compiler and VM. (I confess that every time I use an array that ought to be generic (which is quite often), I think unkind thoughts about Java, and myself for letting it happen.) As a special bonus, class java.lang.Array could * Allow "[]" syntax, just like Project Coin maps * Allow "long" indices, which are needed anyway. A reasonably simple implementation is available via Unsafe (which is a defacto standard so unlikely to be controversial), although it would need tighter integration to be good enough to use without having to manually validate decent performance in practice. Probably the main challenge is for Garbage Collectors, that don't know what to do with giant arrays. -Doug From opinali at gmail.com Tue Feb 9 10:21:56 2010 From: opinali at gmail.com (Osvaldo Doederlein) Date: Tue, 9 Feb 2010 16:21:56 -0200 Subject: Code to analyze a source base for project lambda In-Reply-To: <15e8b9d21002061529r3a962c61v7fd6996fee015777@mail.gmail.com> References: <15e8b9d21002031255y45704467i32475a2c9b64803d@mail.gmail.com> <15e8b9d21002031310n7c7f4c4eqcc0844ad450b5efb@mail.gmail.com> <67103117-46CB-4258-A544-814E1AD00D3A@googlemail.com> <15e8b9d21002061529r3a962c61v7fd6996fee015777@mail.gmail.com> Message-ID: FWIW, follows results for my largest project here, with the fixed tool: Total anonymous classes: 312 But with no constructor args: 312 And defining only one method: 294 And where the type is a SAM type: 60 Those that are recursive: 0 Those that reference 'this': 0 C:\Workspaces\nb\Switch\src\java\br\com\tecban\switchtb\core\translation\TranslationConfig.java:172: [Anon, OneMethodOnly, NeedsMethod, SuperThis] uses an enclosing 'this' C:\Workspaces\nb\Switch\src\java\br\com\tecban\switchtb\core\translation\TranslationConfig.java:108: [Anon, OneMethodOnly, NeedsMethod, SuperThis] uses an enclosing 'this' Those that reference an enclosing 'this': 2 Those that are interfaces: 52 Those that are classes: 8 Total distinct interfaces: 7 br.com.tecban.switchtb.core.Lifecycle.Fire br.com.tecban.switchtb.core.config.ConfigurationFilter br.com.tecban.switchtb.core.config.ConfigurationValidator br.com.tecban.switchtb.core.crypto.CryptoMethod br.com.tecban.switchtb.core.decision.RuleDef.Filter java.io.FilenameFilter java.util.Comparator Total distinct classes: 3 br.com.tecban.switchtb.core.util.ejb.AbstractSessionBean.Finder br.com.tecban.switchtb.core.util.ejb.AbstractSessionBean.FinderN java.util.TimerTask My only uses of enclosing 'this' were due to naming conflict: the inner class defines a method 'validate', which implementation must invoke a helper method from the enclosing class that's also called 'validate', like: public void validate (final ConfigurationEntity entity) { TranslationConfig.this.validate((LayoutDef)entity, parsing); } A+ Osvaldo 2010/2/6 Neal Gafter > On Fri, Feb 5, 2010 at 1:13 AM, Mark Mahieu >wrote: > > > I think I've identified a smallish bug in Neal's analysis code, having > > cross referenced its output with that of my own. > > > > Yes, that was a bug, and it did skew the results in a way that erroneously > favored both supporting SAM classes and nonlexical "this". The corrected > stats for openjdk6 are below. The corrected code can be downloaded from > http://www.javac.info/ijavac-Main.java > > Cheers, > Neal > > Total anonymous classes: 1177 > But with no constructor args: 1089 > And defining only one method: 900 > And where the type is a SAM type: 828 > Those that are recursive: 0 > Those that reference 'this': 2 > Those that reference an enclosing 'this': 49 > Those that are interfaces: 771 > Those that are classes: 57 > Total distinct interfaces: 48 > Total distinct classes: 11 > > The distinct interfaces were > com.sun.java.util.jar.pack.Histogram.BitMetric > com.sun.jdi.connect.Transport > com.sun.jmx.remote.internal.NotificationBufferFilter > com.sun.media.sound.ModelTransform > com.sun.net.ssl.HostnameVerifier > com.sun.security.auth.callback.DialogCallbackHandler.Action > com.sun.tools.example.debug.bdi.InputListener > com.sun.tools.example.debug.bdi.OutputListener > com.sun.tools.example.debug.expr.ExpressionParser.GetFrame > com.sun.tools.hat.internal.oql.ObjectVisitor > com.sun.tools.jdi.CommandSender > com.sun.tools.script.shell.Main.Command > java.awt.Conditional > java.awt.KeyEventPostProcessor > java.awt.event.ActionListener > java.awt.event.HierarchyListener > java.beans.ExceptionListener > java.beans.PropertyChangeListener > java.beans.VetoableChangeListener > java.io.FilenameFilter > java.io.ObjectInputValidation > java.lang.Runnable > java.net.CookiePolicy > java.net.HttpCookie.CookieAttributeAssignor > java.security.PrivilegedAction > java.security.PrivilegedExceptionAction > java.util.Comparator > java.util.concurrent.Callable > java.util.concurrent.Executor > java.util.concurrent.ThreadFactory > javax.imageio.event.IIOReadWarningListener > javax.imageio.event.IIOWriteWarningListener > javax.management.NotificationListener > javax.swing.UIDefaults.ActiveValue > javax.swing.UIDefaults.LazyValue > javax.swing.event.CaretListener > javax.swing.event.ChangeListener > javax.swing.event.HyperlinkListener > javax.xml.crypto.KeySelectorResult > javax.xml.crypto.NodeSetData > sun.awt.RequestFocusController > sun.java2d.StateTracker > sun.java2d.cmm.ProfileActivator > sun.java2d.loops.ProcessPath.EndSubPathHandler > sun.misc.JavaIODeleteOnExitAccess > sun.misc.JavaNetAccess > sun.nio.ch.FileChannelImpl.FileLockTable.Releaser > sun.nio.ch.Interruptible > > The distinct classes were: > com.sun.tools.example.debug.tty.Commands.AsyncExecution > com.sun.tools.hat.internal.model.AbstractJavaHeapObjectVisitor > com.sun.tools.hat.internal.util.Comparer > java.awt.font.TextLine.Function > java.io.OutputStream > java.nio.charset.CoderResult.Cache > java.util.TimerTask > java.util.regex.Pattern.CharProperty > java.util.regex.Pattern.CharPropertyNames.CharPropertyFactory > java.util.regex.Pattern.CharPropertyNames.CloneableProperty > javax.swing.AbstractAction > > From opinali at gmail.com Tue Feb 9 10:35:36 2010 From: opinali at gmail.com (Osvaldo Doederlein) Date: Tue, 9 Feb 2010 16:35:36 -0200 Subject: Function types versus arrays In-Reply-To: <4B719525.9040908@cs.oswego.edu> References: <15e8b9d21002071041j57e926e9k8e7c36e01f80ee46@mail.gmail.com> <4B6FE4D4.2040208@univ-mlv.fr> <4B706662.7050603@sun.com> <4B719525.9040908@cs.oswego.edu> Message-ID: 2010/2/9 Doug Lea
> Alex Buckley wrote: > > Neal raises a fair point. I recognize that arrays of function types are > > as potentially unsafe at runtime as arrays of non-wildcard parameterized > > types. > > > > Alternatively, the array botch itself could be addressed > by creating an efficient Array class tightly coupled > to the compiler and VM. > Any chance JSR-83 could be resuscitated? IBM's "Ninja" project seems to have mostly solved this and other issues, from my (now very vague) recollection of their papers. In a more modern setting, the MLVM community is pondering about enhancements like inline allocation of arrays in tail position (not a solution for mutable collections, but pretty nice boost for immutable ones - which is now a strong trend). > Probably the main challenge is for Garbage Collectors, > that don't know what to do with giant arrays. > Metronome's "arraylets" seem to be a good solution, e.g. not support arrays at all, with the tradeoff of extra cost for indexing and bulk operations (but I guess both costs can be greately reduced/amortized for large arrays by optimizations with good integration to unrolling etc; and not paid for smaller arrays that fit in a single chunk). Huge arrays will always screw up with attempts to deliver increasingly better incremental GCs, like G1. A+ Osvaldo From john at milsson.nu Tue Feb 9 11:12:03 2010 From: john at milsson.nu (John Nilsson) Date: Tue, 9 Feb 2010 20:12:03 +0100 Subject: yet another function types syntax variation In-Reply-To: <201002091045.33689.peter.levart@marand.si> References: <201002091045.33689.peter.levart@marand.si> Message-ID: On Tue, Feb 9, 2010 at 10:45 AM, Peter Levart wrote: > What do you think? I think it is very much an improvement. Still find it hard to read long sequences of nested return types though. With some training and good indentation convention I guess it will do though. What about generics though? public static < T extends Number #(byte[],Integer,Integer) throws IOException, U extends T #(File) throws IOException > U filer(int mode) throws IllegalArgumentException { ... } BR, John From neal at gafter.com Tue Feb 9 12:17:36 2010 From: neal at gafter.com (Neal Gafter) Date: Tue, 9 Feb 2010 12:17:36 -0800 Subject: yet another function types syntax variation In-Reply-To: References: <201002091045.33689.peter.levart@marand.si> Message-ID: <15e8b9d21002091217x7f8dd8a4id75cecdf29d76e9a@mail.gmail.com> As long as we're throwing our proposals into the ring, here's mine: *Type*: *FunctionType* *FunctionType*: *(** Types*opt *Throws*opt *)* *->* *Type* *Types*: *Type* *Types **,* *Type* *Throws*: *throws **QualifiedIdentifierList* The grammar for ArrayType would be written in such a way that it does not allow a FunctionType to be the array's element type. The following syntax for lambdas would not make the result type explicit *Primary*: *StatementLambda* *Expression*: *ExpressionLambda* *StatementLambda*: *FormalParameters **->* *Block* *ExpressionLambda* *FormalParameters **->* *Expression* For example, here is a currying method: *static (T)->(U)->V curry((T,U)->V toCurry) {* * return (T t)->(U u)->toCurry.(t, u);* *}* That corresponds to the following in the current draft *static ##V(U)(T) curry(#V(T,U) toCurry) {* * return #(T t)(#(U u)(toCurry.(t, u)));* *}* I find the arrow notation far easier to read when composed. It does interact a bit strangely with generics, however: *List<(String)->int> list;* because the *->* looks, at first glance, as if it is closing the type argument. I don't think *#* works as well as the arrow, however *static (T)#(U)#V curry((T,U)#V toCurry) {* * return (T t)#(U u)#toCurry(t, u);* *}* * *Cheers, Neal * * From john at milsson.nu Tue Feb 9 13:05:51 2010 From: john at milsson.nu (John Nilsson) Date: Tue, 9 Feb 2010 22:05:51 +0100 Subject: yet another function types syntax variation In-Reply-To: <15e8b9d21002091217x7f8dd8a4id75cecdf29d76e9a@mail.gmail.com> References: <201002091045.33689.peter.levart@marand.si> <15e8b9d21002091217x7f8dd8a4id75cecdf29d76e9a@mail.gmail.com> Message-ID: On Tue, Feb 9, 2010 at 9:17 PM, Neal Gafter wrote: > I don't think *#* works as well as the arrow, however > > *static (T)#(U)#V curry((T,U)#V toCurry) {* > * return (T t)#(U u)#toCurry(t, u);* > *}* > Would : work (unambigously) ? > *static (T):(U):V curry((T,U):V toCurry) {* > * return (T t):(U u):toCurry(t, u);* > *}* > As long as it is always preceded with parentheses it shouldn't collide with labels. BR, John From neal at gafter.com Tue Feb 9 13:23:04 2010 From: neal at gafter.com (Neal Gafter) Date: Tue, 9 Feb 2010 13:23:04 -0800 Subject: yet another function types syntax variation In-Reply-To: References: <201002091045.33689.peter.levart@marand.si> <15e8b9d21002091217x7f8dd8a4id75cecdf29d76e9a@mail.gmail.com> Message-ID: <15e8b9d21002091323s47c767e1me3c30793811daf2b@mail.gmail.com> I think that would be confusing when combined with ?: expressions. On Tue, Feb 9, 2010 at 1:05 PM, John Nilsson wrote: > On Tue, Feb 9, 2010 at 9:17 PM, Neal Gafter wrote: > >> I don't think *#* works as well as the arrow, however >> >> *static (T)#(U)#V curry((T,U)#V toCurry) {* >> * return (T t)#(U u)#toCurry(t, u);* >> *}* >> > > Would : work (unambigously) ? > > >> *static (T):(U):V curry((T,U):V toCurry) {* >> * return (T t):(U u):toCurry(t, u);* >> *}* >> > > As long as it is always preceded with parentheses it shouldn't collide with > labels. > > BR, > John > From john at milsson.nu Tue Feb 9 13:58:42 2010 From: john at milsson.nu (John Nilsson) Date: Tue, 9 Feb 2010 22:58:42 +0100 Subject: yet another function types syntax variation In-Reply-To: <15e8b9d21002091323s47c767e1me3c30793811daf2b@mail.gmail.com> References: <201002091045.33689.peter.levart@marand.si> <15e8b9d21002091217x7f8dd8a4id75cecdf29d76e9a@mail.gmail.com> <15e8b9d21002091323s47c767e1me3c30793811daf2b@mail.gmail.com> Message-ID: On Tue, Feb 9, 2010 at 10:23 PM, Neal Gafter wrote: > I think that would be confusing when combined with ?: expressions. I would suggest ~ but it's nigh impossible to type on Swedish keyboards. (Altgr + a button that doesn't even seem to have a standard placement + space). BR, John From forax at univ-mlv.fr Tue Feb 9 14:37:02 2010 From: forax at univ-mlv.fr (=?ISO-8859-1?Q?R=E9mi_Forax?=) Date: Tue, 09 Feb 2010 23:37:02 +0100 Subject: yet another function types syntax variation In-Reply-To: References: <201002091045.33689.peter.levart@marand.si> <15e8b9d21002091217x7f8dd8a4id75cecdf29d76e9a@mail.gmail.com> <15e8b9d21002091323s47c767e1me3c30793811daf2b@mail.gmail.com> Message-ID: <4B71E38E.3090906@univ-mlv.fr> Le 09/02/2010 22:58, John Nilsson a ?crit : > On Tue, Feb 9, 2010 at 10:23 PM, Neal Gafter wrote: > > >> I think that would be confusing when combined with ?: expressions. >> > > I would suggest ~ but it's nigh impossible to type on Swedish keyboards. > (Altgr + a button that doesn't even seem to have a standard placement + > space). > > BR, > John > > It remind me a possible syntax for lambda expression/statement: ,\(int x) (x) comma + backslash is almost a lambda. R?mi From howard.lovatt at iee.org Tue Feb 9 15:17:39 2010 From: howard.lovatt at iee.org (Howard Lovatt) Date: Tue, 9 Feb 2010 23:17:39 +0000 Subject: peter Message-ID: <3dd3f56a1002091517x60934f5h99f8e90cb9a6b889@mail.gmail.com> This proposal: http://www.artima.com/weblogs/viewpost.jsp?thread=278567 Would allow: #Object(String)[] array = new #String(Object)[10]; It would be translated into: _Callable_Object_String[] array = _Array_From_String_Object__To_Object_String.instance( new _Callable_String_Object[10] ); A more interesting example is: final #String(Object)[] sols = new #String(Object)[1]; sols[0] = #String(final Object o)(o.toString()); final #Object(String)[] osls = sols; System.out.println("osls[0].(\"Hello\") = " + osls[0].("Hello")); Which is translated to: final _Callable_String_Object[] sols = new _Callable_String_Object[1]; sols[0] = new _Callable_String_Object() { @Override public String call(final Object o) { return o.toString(); } }; final _Callable_Object_String[] osls = _Array_From_String_Object__To_Object_String.instance( sols ); System.out.println("osls[0].(\"Hello\") = " + osls[0].call("Hello")); _Array_From...'s instance method does all the necessary translation - see reference above for details. To give an idea of the scheme, the relevant 'From' classes are: public final class _Array_From_String_Object__To_Object_String { public static _Callable_Object_String[] instance(final _Callable_String_Object[] from) { if (from == null) { return null; } final _Callable_Object_String[] to = new _Callable_Object_String[from.length]; for (int i = 0; i < from.length; i++) { to[i] = _From_String_Object__To_Object_String.instance(from[i]); } return to; } } public final class _From_String_Object__To_Object_String extends _Callable_Object_String implements Wrapper { private final _Callable_String_Object f; private _From_String_Object__To_Object_String(final _Callable_String_Object f) { this.f = f; } public static _From_String_Object__To_Object_String instance(final _Callable_String_Object from) { if (from == null) { return null; } final _Callable_String_Object f = (_Callable_String_Object)Wrappers.unWrap(from); return new _From_String_Object__To_Object_String(f); } @Override public Object call(final String a1) { return f.call(a1); } @Override public Object getWrappee() { return f; } @Override public int hashCode() { return f.hashCode(); } } Also see reference for actual type names, _Callable_... are a simplification to allow easy testing of the idea, and the relevant classes, _Callable... and 'From' are dynamically generated by the class loader. ?-- Howard. From neal at gafter.com Tue Feb 9 16:09:13 2010 From: neal at gafter.com (Neal Gafter) Date: Tue, 9 Feb 2010 16:09:13 -0800 Subject: peter In-Reply-To: <3dd3f56a1002091517x60934f5h99f8e90cb9a6b889@mail.gmail.com> References: <3dd3f56a1002091517x60934f5h99f8e90cb9a6b889@mail.gmail.com> Message-ID: <15e8b9d21002091609g691aa19ard8debbcc7705ff24@mail.gmail.com> Like your previous proposals, this isn't sound. I can't tell which way it fails: either your arrays can't store lambdas that were created inside generic methods (you get an array store exception). Or, if that does work, then your arrays can be assigned lambdas of the wrong type inside a generic method. The reason I can't tell which way it fails is that the specification isn't coherent. I tried reading the specification, (section 5), but immediately ran into problems. For example, starting with section 5.1, you don't say what you mean for a type to be "generic", and the JLS doesn't provide a definition that appears to match your meaning (in the JLS, the word is an adjective used to describe declarations, not types). In describing the expansion (still section 5.1), when both the return and argument types are "generic" types you have the names of the types of the function arguments and return type encoded into the generated class name. (You specify the generated class *name *but not what the compiler ought to generate for it) But if these "generic" types are type parameters, those names are not unique, nor are they types that can be used outside the declaration of which they are type parameters. The translation in 5.1 only works when there is precisely one lambda parameter (not zero or two). Similarly, section 5.2 falls apart in the presence of primitives. I didn't bother reading further. A more coherent analysis would require a coherent specification. On Tue, Feb 9, 2010 at 3:17 PM, Howard Lovatt wrote: > This proposal: > > http://www.artima.com/weblogs/viewpost.jsp?thread=278567 > > Would allow: > > #Object(String)[] array = new #String(Object)[10]; > > It would be translated into: > > _Callable_Object_String[] array = > _Array_From_String_Object__To_Object_String.instance( new > _Callable_String_Object[10] ); > > A more interesting example is: > > final #String(Object)[] sols = new #String(Object)[1]; > sols[0] = #String(final Object o)(o.toString()); > final #Object(String)[] osls = sols; > System.out.println("osls[0].(\"Hello\") = " + osls[0].("Hello")); > > Which is translated to: > > final _Callable_String_Object[] sols = new _Callable_String_Object[1]; > sols[0] = new _Callable_String_Object() { > @Override public String call(final Object o) { return o.toString(); } > }; > final _Callable_Object_String[] osls = > _Array_From_String_Object__To_Object_String.instance( sols ); > System.out.println("osls[0].(\"Hello\") = " + osls[0].call("Hello")); > > _Array_From...'s instance method does all the necessary translation - > see reference above for details. To give an idea of the scheme, the > relevant 'From' classes are: > > public final class _Array_From_String_Object__To_Object_String { > public static _Callable_Object_String[] instance(final > _Callable_String_Object[] from) { > if (from == null) { return null; } > final _Callable_Object_String[] to = new > _Callable_Object_String[from.length]; > for (int i = 0; i < from.length; i++) { to[i] = > _From_String_Object__To_Object_String.instance(from[i]); } > return to; > } > } > > public final class _From_String_Object__To_Object_String extends > _Callable_Object_String implements Wrapper { > private final _Callable_String_Object f; > private _From_String_Object__To_Object_String(final > _Callable_String_Object f) { this.f = f; } > public static _From_String_Object__To_Object_String instance(final > _Callable_String_Object from) { > if (from == null) { return null; } > final _Callable_String_Object f = > (_Callable_String_Object)Wrappers.unWrap(from); > return new _From_String_Object__To_Object_String(f); > } > @Override public Object call(final String a1) { return f.call(a1); } > @Override public Object getWrappee() { return f; } > @Override public int hashCode() { return f.hashCode(); } > } > > Also see reference for actual type names, _Callable_... are a > simplification to allow easy testing of the idea, and the relevant > classes, _Callable... and 'From' are dynamically generated by the > class loader. > > -- Howard. > > From neal at gafter.com Tue Feb 9 17:12:41 2010 From: neal at gafter.com (Neal Gafter) Date: Tue, 9 Feb 2010 17:12:41 -0800 Subject: Local functions Message-ID: <15e8b9d21002091712j5d3a0488y48459bb6d183fa09@mail.gmail.com> The Lambda project page (and various other references to the project) describe it as being about "First-class functions, function types, and lambda expressions".? So far we've been talking exclusively about the latter two issues, and not "first-class functions". Functions would provide a way of writing recursive computations in a local scope.? Unlike lambda expressions (which are anonymous functions), function declarations introduce a name for the function being declared. For example, if we add functions, you could write (in a local scope) either something like int factorial(int x) ??? (x <= 1) ? 1 : x*factorial.(x-1); or int factorial(int x) { ??? if (x <= 1) return 1; ??? return x * factorial.(x-1); } or both (syntax TBD). For consistency with function types and lambda expressions, I would expect the introduced name to be in the variable namespace, and describe a final variable of function type.? In short, the two declarations above would be defined as meaning final #int(int) factorial = #(int x) ??? ((x <= 1) ? 1 : x*factorial.(x-1)); and final #int(int) factorial = #(int x) { ??? if (x <= 1) return 1; ??? return x * factorial.(x-1); }; But without any worry about the variable being usable (recursively) inside the function because of definite assignment issues. These would be usable as a value of function type #int(int) fac2 = factorial; The traditional distinction between functions and methods is that methods are defined in an object scope, and therefore the object's members are in scope.? Functions, on the other hand, are defined in whatever scope they are declared in, and therefore inherit their lexical scope (including, e.g., "this") from their point of declaration. The main downside of the syntax I've selected, above, is that it provides no way to distinguish a method declaration from a member function declaration.? The reason that the distinction is meaningful is that these two concepts imply different namespaces for the name that is defined, and also because methods are by default virtual. (Interestingly, functions and methods declared in class scope would have the same treatment of "this" - the enclosing instance - because both are declared in class scope.) Adding support for "method references" eases distinction. Local functions can be convenient in writing code that would otherwise require writing a complex lambda (e.g. too complex to write inline in the middle of some expression, but you still want to keep it local to its use site). I don't know if Sun had anything like this in mind when they described project Lambda.? Local functions appeared in version 00 of BGGA but were dropped in later revisions. From Alex.Buckley at Sun.COM Tue Feb 9 17:34:35 2010 From: Alex.Buckley at Sun.COM (Alex Buckley) Date: Tue, 09 Feb 2010 17:34:35 -0800 Subject: Local functions In-Reply-To: <15e8b9d21002091712j5d3a0488y48459bb6d183fa09@mail.gmail.com> References: <15e8b9d21002091712j5d3a0488y48459bb6d183fa09@mail.gmail.com> Message-ID: <4B720D2B.3070704@sun.com> Initial reaction: local functions are a step too far, even if they do have a straightforward translation to lambdas. Neal Gafter wrote: > The traditional distinction between functions and methods is that > methods are defined in an object scope, and therefore the object's > members are in scope. Functions, on the other hand, are defined in > whatever scope they are declared in, and therefore inherit their > lexical scope (including, e.g., "this") from their point of > declaration. And with that, how could anyone argue against lexical 'this'? :-) My concern about lambdas duplicating the functionality of anon.inner classes has led me to think about dropping statement lambdas, keeping expression lambdas, and then adding block expressions. (I am avoiding saying anything about 'this' in such circumstances.) Alex From neal at gafter.com Tue Feb 9 18:19:09 2010 From: neal at gafter.com (Neal Gafter) Date: Tue, 9 Feb 2010 18:19:09 -0800 Subject: Local functions In-Reply-To: <4B720D2B.3070704@sun.com> References: <15e8b9d21002091712j5d3a0488y48459bb6d183fa09@mail.gmail.com> <4B720D2B.3070704@sun.com> Message-ID: <15e8b9d21002091819k2674ed9ai256f7221ed57570@mail.gmail.com> On Tue, Feb 9, 2010 at 5:34 PM, Alex Buckley wrote: > Initial reaction: local functions are a step too far, even if they do have a > straightforward translation to lambdas. Which is why we removed them after v00. I was just wondering if there was some deeper meaning in "first class functions". > My concern about lambdas duplicating the functionality of anon.inner classes > has led me to think about dropping statement lambdas, keeping expression > lambdas, and then adding block expressions. (I am avoiding saying anything > about 'this' in such circumstances.) Wow. Sounds great to me. Then you don't have to specify the return type when you write a lambda (it is just the expression's type). You probably need a way to write a lambda with a "void" result (e.g. introduce a meaning for "void" in an expression context). If you go this way, you might consider adding a way to yield a result early from a block expression. Or, alternately, you could take the position that if the programmer's control flow is so complicated that they "need" to use a "return" statement, then they probably ought to use a method (perhaps in an anonymous inner class). That was where we ended up, but I continued to look (without success) for a good way to yield a result early. From John.Rose at Sun.COM Tue Feb 9 19:10:36 2010 From: John.Rose at Sun.COM (John Rose) Date: Tue, 09 Feb 2010 19:10:36 -0800 Subject: Local functions In-Reply-To: <15e8b9d21002091819k2674ed9ai256f7221ed57570@mail.gmail.com> References: <15e8b9d21002091712j5d3a0488y48459bb6d183fa09@mail.gmail.com> <4B720D2B.3070704@sun.com> <15e8b9d21002091819k2674ed9ai256f7221ed57570@mail.gmail.com> Message-ID: On Feb 9, 2010, at 6:19 PM, Neal Gafter wrote: > On Tue, Feb 9, 2010 at 5:34 PM, Alex Buckley wrote: >> My concern about lambdas duplicating the functionality of anon.inner classes >> has led me to think about dropping statement lambdas, keeping expression >> lambdas, and then adding block expressions. (I am avoiding saying anything >> about 'this' in such circumstances.) > > Wow. Sounds great to me. Then you don't have to specify the return > type when you write a lambda (it is just the expression's type). > > You probably need a way to write a lambda with a "void" result (e.g. > introduce a meaning for "void" in an expression context). static void java.lang.Void.value() { } ! Alex will come up with something better. :-) > If you go this way, you might consider adding a way to yield a result > early from a block expression. Or, alternately, you could take the > position that if the programmer's control flow is so complicated that > they "need" to use a "return" statement, then they probably ought to > use a method (perhaps in an anonymous inner class). That was where we > ended up, but I continued to look (without success) for a good way to > yield a result early. By factoring block expression syntax from lambda syntax, and making the two independent, they can be independently useful. For starters, a block expression within a lambda will yield the functionality of the 0.1 statement lambda, while (conversely) a lambda inside a block expression could express a named self-recursive lambda (of the kind Josh said was sometimes necessary but not previously possible). final (int n)->int factorial = #(int x)( (x <= 1) ? 1 : x*factorial.(x-1) ) /*picking random proposed syntaxes*/ === final (int n)->int factorial = ({ (int n)->int f = #(int x)( (x <= 1) ? 1 : x*f.(x-1) ); f }) -- John P.S. As has been observed, the DA rules will have to be adjusted, but there is an opportunity to do this at the boundary between a lambda and its enclosing construct. The hard case from a DA point of view is this one: final (int)->int fn = DoSomething.toSomeFn( (int)->int (someExpression(fn)) ); The value fn can clearly be self-recursive if 'DoSomething.toSomeFn' is omitted, since the lambda will certainly not be called until 'fn' is initialized. The same is true if DoSomething.toSomeFn is some combinatory wrapper (or identity function, such as a logger) that does not actually invoke its argument. But if it does invoke its argument, a null value of 'fn' will be visible, in much the same way as a final class or instance variable can sometimes be observed to have its default initial value. This would be a "low road" type of design, but it would be consistent enough (IMO) with other quirks in Java's initialization semantics. I think the disadvantage of potentially observing nulls is outweighed by the usefulness of recursion. Another way to avoid the problem (taken by the JavaScript standard, section 13) is to mandate a separate pre-pass for initializing function-valued variables. This only makes sense when there is a distinct syntax for functions as opposed to lambdas (as you mentioned, Neal). The special syntax would not allow any use of a combinatory wrapper. From scolebourne at joda.org Wed Feb 10 00:15:53 2010 From: scolebourne at joda.org (Stephen Colebourne) Date: Wed, 10 Feb 2010 08:15:53 +0000 Subject: Local functions In-Reply-To: <4B720D2B.3070704@sun.com> References: <15e8b9d21002091712j5d3a0488y48459bb6d183fa09@mail.gmail.com> <4B720D2B.3070704@sun.com> Message-ID: <4b4f45e01002100015y421b2a15x7bc8deae122d76f4@mail.gmail.com> On 10 February 2010 01:34, Alex Buckley wrote: > My concern about lambdas duplicating the functionality of anon.inner > classes has led me to think about dropping statement lambdas, keeping > expression lambdas, and then adding block expressions. (I am avoiding > saying anything about 'this' in such circumstances.) What concern about "duplicating the functionality of anon.inner classes" ? Please explain! This appears to be a sudden jump from a relatively small conceptual change (from a developers POV) to a much larger one (which I suspect many may feel to be alien to the Java they know). Stephen From howard.lovatt at gmail.com Wed Feb 10 02:53:51 2010 From: howard.lovatt at gmail.com (Howard Lovatt) Date: Wed, 10 Feb 2010 21:53:51 +1100 Subject: Function types versus arrays Message-ID: This is what: http://www.artima.com/weblogs/viewpost.jsp?thread=278567 Does. -- Howard Lovatt +61 419 971 263 (sent from iPhone) From markmahieu at googlemail.com Wed Feb 10 03:40:48 2010 From: markmahieu at googlemail.com (Mark Mahieu) Date: Wed, 10 Feb 2010 11:40:48 +0000 Subject: Local functions In-Reply-To: <4B720D2B.3070704@sun.com> References: <15e8b9d21002091712j5d3a0488y48459bb6d183fa09@mail.gmail.com> <4B720D2B.3070704@sun.com> Message-ID: <5C8E718F-2AA5-4EA3-9796-D1A46FCA6EB8@googlemail.com> On 10 Feb 2010, at 01:34, Alex Buckley wrote: > > My concern about lambdas duplicating the functionality of anon.inner > classes has led me to think about dropping statement lambdas, keeping > expression lambdas, and then adding block expressions. (I am avoiding > saying anything about 'this' in such circumstances.) Very encouraging that you consider this an option worth exploring. There are many possible ways to slice and dice lambdas, but extremely limited scope for doing so - the current distinction between statement and expression forms doesn't strike me as one which really carries its weight. If there is justification for having two distinct forms, I think it's more likely to lie in answering the kind of concerns Doug Lea raised recently [1] (although I take a slightly different view on some of them). Regards, Mark [1] http://mail.openjdk.java.net/pipermail/lambda-dev/2010-February/000486.html From forax at univ-mlv.fr Wed Feb 10 05:09:03 2010 From: forax at univ-mlv.fr (=?ISO-8859-1?Q?R=E9mi_Forax?=) Date: Wed, 10 Feb 2010 14:09:03 +0100 Subject: Local functions In-Reply-To: <4B720D2B.3070704@sun.com> References: <15e8b9d21002091712j5d3a0488y48459bb6d183fa09@mail.gmail.com> <4B720D2B.3070704@sun.com> Message-ID: <4B72AFEF.9010807@univ-mlv.fr> Le 10/02/2010 02:34, Alex Buckley a ?crit : > Initial reaction: local functions are a step too far, even if they do > have a straightforward translation to lambdas. > > Neal Gafter wrote: > >> The traditional distinction between functions and methods is that >> methods are defined in an object scope, and therefore the object's >> members are in scope. Functions, on the other hand, are defined in >> whatever scope they are declared in, and therefore inherit their >> lexical scope (including, e.g., "this") from their point of >> declaration. >> > And with that, how could anyone argue against lexical 'this'? :-) > > My concern about lambdas duplicating the functionality of anon.inner > classes has led me to think about dropping statement lambdas, keeping > expression lambdas, and then adding block expressions. (I am avoiding > saying anything about 'this' in such circumstances.) > > Alex > Block expressions aren't used elsewhere in Java, do you plan to overhaul the language to introduce block expressions with if, for, try/catch, etc ? There is a simple solution, if you want to write recursive function, create a static method and reference it as a lambda: class A { static int factorial(int x) { return (x<= 1) ? 1 : x*factorial.(x-1); } public static void main(String[] args) { #int(int) lambda = A#factorial; } } I like the fact a lambda is just a simple expression/block of code and if you want something more, you have to use an inner class or a side static method. R?mi From markmahieu at googlemail.com Wed Feb 10 05:52:21 2010 From: markmahieu at googlemail.com (Mark Mahieu) Date: Wed, 10 Feb 2010 13:52:21 +0000 Subject: Local functions In-Reply-To: <4B72AFEF.9010807@univ-mlv.fr> References: <15e8b9d21002091712j5d3a0488y48459bb6d183fa09@mail.gmail.com> <4B720D2B.3070704@sun.com> <4B72AFEF.9010807@univ-mlv.fr> Message-ID: <1D97026B-D743-44F5-8903-23776D11689C@googlemail.com> On 10 Feb 2010, at 13:09, R?mi Forax wrote: > > There is a simple solution, if you want to write recursive function, > create a static method > and reference it as a lambda: > > class A { > static int factorial(int x) { > return (x<= 1) ? 1 : x*factorial.(x-1); > } > > public static void main(String[] args) { > #int(int) lambda = A#factorial; > } > } > I don't think that approach is quite so simple if your recursive function needs to use locals in the method which 'creates' it (eg. it references 'args' in 'main', in your example). Regards, Mark From forax at univ-mlv.fr Wed Feb 10 07:05:41 2010 From: forax at univ-mlv.fr (=?ISO-8859-1?Q?R=E9mi_Forax?=) Date: Wed, 10 Feb 2010 16:05:41 +0100 Subject: Local functions In-Reply-To: <1D97026B-D743-44F5-8903-23776D11689C@googlemail.com> References: <15e8b9d21002091712j5d3a0488y48459bb6d183fa09@mail.gmail.com> <4B720D2B.3070704@sun.com> <4B72AFEF.9010807@univ-mlv.fr> <1D97026B-D743-44F5-8903-23776D11689C@googlemail.com> Message-ID: <4B72CB45.4010701@univ-mlv.fr> Le 10/02/2010 14:52, Mark Mahieu a ?crit : > On 10 Feb 2010, at 13:09, R?mi Forax wrote: > >> There is a simple solution, if you want to write recursive function, >> create a static method >> and reference it as a lambda: >> >> class A { >> static int factorial(int x) { >> return (x<= 1) ? 1 : x*factorial.(x-1); >> } >> >> public static void main(String[] args) { >> #int(int) lambda = A#factorial; >> } >> } >> >> > I don't think that approach is quite so simple if your recursive function needs to use locals in the method which 'creates' it (eg. it references 'args' in 'main', in your example). > > Regards, > > Mark > In that case, the anonymous class syntax is your friend. Or if lambdas are translated to method handles, you can write: class A { static int foo(String[] args, int x) { return (x<= 1) ? args.length : x*foo.(x-1); } public static void main(String[] args) { #int(int) lambda = A#foo.bindTo(args); } } cheers, R?mi From markmahieu at googlemail.com Wed Feb 10 07:36:45 2010 From: markmahieu at googlemail.com (Mark Mahieu) Date: Wed, 10 Feb 2010 15:36:45 +0000 Subject: Local functions In-Reply-To: <4B72CB45.4010701@univ-mlv.fr> References: <15e8b9d21002091712j5d3a0488y48459bb6d183fa09@mail.gmail.com> <4B720D2B.3070704@sun.com> <4B72AFEF.9010807@univ-mlv.fr> <1D97026B-D743-44F5-8903-23776D11689C@googlemail.com> <4B72CB45.4010701@univ-mlv.fr> Message-ID: <4023DE02-B32D-4A04-B70F-380A2DFB63B1@googlemail.com> On 10 Feb 2010, at 15:05, R?mi Forax wrote: > Le 10/02/2010 14:52, Mark Mahieu a ?crit : >> On 10 Feb 2010, at 13:09, R?mi Forax wrote: >> >>> There is a simple solution, if you want to write recursive function, >>> create a static method >>> and reference it as a lambda: >>> >>> class A { >>> static int factorial(int x) { >>> return (x<= 1) ? 1 : x*factorial.(x-1); >>> } >>> >>> public static void main(String[] args) { >>> #int(int) lambda = A#factorial; >>> } >>> } >>> >>> >> I don't think that approach is quite so simple if your recursive function needs to use locals in the method which 'creates' it (eg. it references 'args' in 'main', in your example). >> >> Regards, >> >> Mark >> > > In that case, the anonymous class syntax is your friend. > > Or if lambdas are translated to method handles, you can write: > > class A { > static int foo(String[] args, int x) { > return (x<= 1) ? args.length : x*foo.(x-1); > } > > public static void main(String[] args) { > #int(int) lambda = A#foo.bindTo(args); > } > } > > cheers, > R?mi > > You *can* do it without method references, method handles, or partial application too. But you mentioned a simple solution, which to me suggests an ideal of being able to 'create' the lambda in approximately the same manner regardless of whether it's recursive or references local variables. Anyway, we could do with finding some additional recursive examples - factorial is getting worn out ;-) One which I frequently write involves walking a tree and doing something with (eg. collecting) all nodes which match some criteria. I'm slightly surprised it didn't crop up in the recent codebase stats I submitted though... I might look into why that is. Cheers, Mark From forax at univ-mlv.fr Wed Feb 10 07:53:55 2010 From: forax at univ-mlv.fr (=?ISO-8859-1?Q?R=E9mi_Forax?=) Date: Wed, 10 Feb 2010 16:53:55 +0100 Subject: Local functions In-Reply-To: <4023DE02-B32D-4A04-B70F-380A2DFB63B1@googlemail.com> References: <15e8b9d21002091712j5d3a0488y48459bb6d183fa09@mail.gmail.com> <4B720D2B.3070704@sun.com> <4B72AFEF.9010807@univ-mlv.fr> <1D97026B-D743-44F5-8903-23776D11689C@googlemail.com> <4B72CB45.4010701@univ-mlv.fr> <4023DE02-B32D-4A04-B70F-380A2DFB63B1@googlemail.com> Message-ID: <4B72D693.2000209@univ-mlv.fr> Le 10/02/2010 16:36, Mark Mahieu a ?crit : > On 10 Feb 2010, at 15:05, R?mi Forax wrote: > > >> Le 10/02/2010 14:52, Mark Mahieu a ?crit : >> >>> On 10 Feb 2010, at 13:09, R?mi Forax wrote: >>> >>> >>>> There is a simple solution, if you want to write recursive function, >>>> create a static method >>>> and reference it as a lambda: >>>> >>>> class A { >>>> static int factorial(int x) { >>>> return (x<= 1) ? 1 : x*factorial.(x-1); >>>> } >>>> >>>> public static void main(String[] args) { >>>> #int(int) lambda = A#factorial; >>>> } >>>> } >>>> >>>> >>>> >>> I don't think that approach is quite so simple if your recursive function needs to use locals in the method which 'creates' it (eg. it references 'args' in 'main', in your example). >>> >>> Regards, >>> >>> Mark >>> >>> >> In that case, the anonymous class syntax is your friend. >> >> Or if lambdas are translated to method handles, you can write: >> >> class A { >> static int foo(String[] args, int x) { >> return (x<= 1) ? args.length : x*foo.(x-1); >> } >> >> public static void main(String[] args) { >> #int(int) lambda = A#foo.bindTo(args); >> } >> } >> >> cheers, >> R?mi >> >> >> > > You *can* do it without method references, method handles, or partial application too. But you mentioned a simple solution, which to me suggests an ideal of being able to 'create' the lambda in approximately the same manner regardless of whether it's recursive or references local variables. > > Anyway, we could do with finding some additional recursive examples - factorial is getting worn out ;-) One which I frequently write involves walking a tree and doing something with (eg. collecting) all nodes which match some criteria. I'm slightly surprised it didn't crop up in the recent codebase stats I submitted though... I might look into why that is. > > > Cheers, > > Mark > Mark, if you walk a tree, the method that walk is recursive, not the lambda taken as argument. simple solution => if its a complex lambda, write it as a method or as an inner class then construct a lambda on it. class A { static int foo(String[] args, int x) { return (x<= 1) ? args.length : x*foo.(x-1); } public static void main(final String[] args) { #int(int) lambda = #(int x) (foo(args, x)); } } R?mi From Alex.Buckley at Sun.COM Wed Feb 10 11:26:22 2010 From: Alex.Buckley at Sun.COM (Alex Buckley) Date: Wed, 10 Feb 2010 11:26:22 -0800 Subject: Local functions In-Reply-To: <4B72D693.2000209@univ-mlv.fr> References: <15e8b9d21002091712j5d3a0488y48459bb6d183fa09@mail.gmail.com> <4B720D2B.3070704@sun.com> <4B72AFEF.9010807@univ-mlv.fr> <1D97026B-D743-44F5-8903-23776D11689C@googlemail.com> <4B72CB45.4010701@univ-mlv.fr> <4023DE02-B32D-4A04-B70F-380A2DFB63B1@googlemail.com> <4B72D693.2000209@univ-mlv.fr> Message-ID: <4B73085E.7080101@sun.com> R?mi Forax wrote: > if you walk a tree, the method that walk is recursive, not the lambda > taken as argument. > > simple solution => if its a complex lambda, write it as a method or as > an inner class then construct a lambda on it. > > class A { > static int foo(String[] args, int x) { > return (x<= 1) ? args.length : x*foo.(x-1); > } > > public static void main(final String[] args) { > #int(int) lambda = #(int x) (foo(args, x)); > } > } I'm glad you are also thinking about the limits of lambdas. I do not relish developers continually having to choose between putting code in a nearby method, a static nested SAM class, an anon.inner SAM class, or a lambda. Things used to be so simple... Setting up a lambda expression as above is characterized as the "Closure/Stored message invokability principle" in http://blogs.oracle.com/ohrstrom/2009/08/using_methodhandles_to_reconci.html: "A method reference like: MyApp#saveState() will be transformed into the closure { MyApp m ==> m.saveState() } this is identical to the stored message (aka MethodHandle) #MyApp.saveState. " Method references are in scope for this project, though I tend to think that relying on ordinary methods ignores Doug's plea for clear demarcation of (and restrictions on) code that experiences parallel execution. Alex From neal at gafter.com Wed Feb 10 11:47:08 2010 From: neal at gafter.com (Neal Gafter) Date: Wed, 10 Feb 2010 11:47:08 -0800 Subject: Local functions In-Reply-To: <4B73085E.7080101@sun.com> References: <15e8b9d21002091712j5d3a0488y48459bb6d183fa09@mail.gmail.com> <4B720D2B.3070704@sun.com> <4B72AFEF.9010807@univ-mlv.fr> <1D97026B-D743-44F5-8903-23776D11689C@googlemail.com> <4B72CB45.4010701@univ-mlv.fr> <4023DE02-B32D-4A04-B70F-380A2DFB63B1@googlemail.com> <4B72D693.2000209@univ-mlv.fr> <4B73085E.7080101@sun.com> Message-ID: <15e8b9d21002101147u64c8bb9dhc5be3f2a3c65feb9@mail.gmail.com> On Wed, Feb 10, 2010 at 11:26 AM, Alex Buckley wrote: > Setting up a lambda expression as above is characterized as the > "Closure/Stored message invokability principle" in > http://blogs.oracle.com/ohrstrom/2009/08/using_methodhandles_to_reconci.html: > > "A method reference like: MyApp#saveState() will be transformed into the > closure { MyApp m ==> m.saveState() } ?this is identical to the stored > message (aka MethodHandle) #MyApp.saveState. " I wasn't aware of that article before. How interesting! > Method references are in scope for this project, though I tend to think > that relying on ordinary methods ignores Doug's plea for clear > demarcation of (and restrictions on) code that experiences parallel > execution. Sounds great, but it seems to me that's orthogonal to lambdas and function types. Are you considering how to distinguish "parallel" closures from other closures? I don't think restricting them to the parallel use cases is wise. From forax at univ-mlv.fr Wed Feb 10 13:28:23 2010 From: forax at univ-mlv.fr (=?ISO-8859-1?Q?R=E9mi_Forax?=) Date: Wed, 10 Feb 2010 22:28:23 +0100 Subject: Local functions In-Reply-To: <4B73085E.7080101@sun.com> References: <15e8b9d21002091712j5d3a0488y48459bb6d183fa09@mail.gmail.com> <4B720D2B.3070704@sun.com> <4B72AFEF.9010807@univ-mlv.fr> <1D97026B-D743-44F5-8903-23776D11689C@googlemail.com> <4B72CB45.4010701@univ-mlv.fr> <4023DE02-B32D-4A04-B70F-380A2DFB63B1@googlemail.com> <4B72D693.2000209@univ-mlv.fr> <4B73085E.7080101@sun.com> Message-ID: <4B7324F7.8030903@univ-mlv.fr> Le 10/02/2010 20:26, Alex Buckley a ?crit : > I'm glad you are also thinking about the limits of lambdas. I do not > relish developers continually having to choose between putting code in a > nearby method, a static nested SAM class, an anon.inner SAM class, or a > lambda. Things used to be so simple... > Recursive lambda is a corner case, I even think that a lot developers will never write a recursive lambda. So writing a method for this peculiar case will not prevent me to sleep. > Setting up a lambda expression as above is characterized as the > "Closure/Stored message invokability principle" in > http://blogs.oracle.com/ohrstrom/2009/08/using_methodhandles_to_reconci.html: > > "A method reference like: MyApp#saveState() will be transformed into the > closure { MyApp m ==> m.saveState() } this is identical to the stored > message (aka MethodHandle) #MyApp.saveState. " > > Method references are in scope for this project, though I tend to think > that relying on ordinary methods ignores Doug's plea for clear > demarcation of (and restrictions on) code that experiences parallel > execution. > > Alex > R?mi From Alex.Buckley at Sun.COM Wed Feb 10 14:35:50 2010 From: Alex.Buckley at Sun.COM (Alex Buckley) Date: Wed, 10 Feb 2010 14:35:50 -0800 Subject: Local functions In-Reply-To: <15e8b9d21002101147u64c8bb9dhc5be3f2a3c65feb9@mail.gmail.com> References: <15e8b9d21002091712j5d3a0488y48459bb6d183fa09@mail.gmail.com> <4B720D2B.3070704@sun.com> <4B72AFEF.9010807@univ-mlv.fr> <1D97026B-D743-44F5-8903-23776D11689C@googlemail.com> <4B72CB45.4010701@univ-mlv.fr> <4023DE02-B32D-4A04-B70F-380A2DFB63B1@googlemail.com> <4B72D693.2000209@univ-mlv.fr> <4B73085E.7080101@sun.com> <15e8b9d21002101147u64c8bb9dhc5be3f2a3c65feb9@mail.gmail.com> Message-ID: <4B7334C6.2030506@sun.com> Neal Gafter wrote: > On Wed, Feb 10, 2010 at 11:26 AM, Alex Buckley wrote: >> Method references are in scope for this project, though I tend to think >> that relying on ordinary methods ignores Doug's plea for clear >> demarcation of (and restrictions on) code that experiences parallel >> execution. > > Sounds great, but it seems to me that's orthogonal to lambdas and > function types. Are you considering how to distinguish "parallel" > closures from other closures? I don't think restricting them to the > parallel use cases is wise. I agree demarcation of/restrictions on parallel code are orthogonal to lambdas and function types. A glance at the FindBugs bugs for multithreaded correctness shows little the language could internalize to make the body of a lambda expression correct by construction. (Short of horrendous type system "enhancements".) That said, I am happy to prioritize parallel use cases over non-parallel use cases. Sun is not aiming to add lambdas to allow control abstraction or DSL building, but rather to support FJ ParallelArray and APIs like it. So I'm fine if lambda bodies are restricted along the lines Doug suggested. ("Ideally, for the sake of parallel execution, we'd disallow automatic sharing of locals and automatic elevation of "this", "return" or "break" to enclosing scopes.") The Lambda strawman is at odds with some of those restrictions; resolution is forthcoming. Alex From scolebourne at joda.org Wed Feb 10 16:11:49 2010 From: scolebourne at joda.org (Stephen Colebourne) Date: Thu, 11 Feb 2010 00:11:49 +0000 Subject: Local functions In-Reply-To: <4B7334C6.2030506@sun.com> References: <15e8b9d21002091712j5d3a0488y48459bb6d183fa09@mail.gmail.com> <4B720D2B.3070704@sun.com> <4B72AFEF.9010807@univ-mlv.fr> <1D97026B-D743-44F5-8903-23776D11689C@googlemail.com> <4B72CB45.4010701@univ-mlv.fr> <4023DE02-B32D-4A04-B70F-380A2DFB63B1@googlemail.com> <4B72D693.2000209@univ-mlv.fr> <4B73085E.7080101@sun.com> <15e8b9d21002101147u64c8bb9dhc5be3f2a3c65feb9@mail.gmail.com> <4B7334C6.2030506@sun.com> Message-ID: <4b4f45e01002101611i6a46310oc267f66545ffeb98@mail.gmail.com> On 10 February 2010 22:35, Alex Buckley wrote: > I agree demarcation of/restrictions on parallel code are orthogonal to > lambdas and function types. A glance at the FindBugs bugs for > multithreaded correctness shows little the language could internalize to > make the body of a lambda expression correct by construction. (Short of > horrendous type system "enhancements".) +1 > That said, I am happy to prioritize parallel use cases over non-parallel > use cases. Sun is not aiming to add lambdas to allow control abstraction > or DSL building, but rather to support FJ ParallelArray and APIs like > it. And the whole use case category of functors and predicates operating on regular collections? Abandoned? Only accessible via a second class syntax? I didn't think thats what lambda was about. But then we don't have any REQUIREMENTS so we can't judge. > So I'm fine if lambda bodies are restricted along the lines Doug > suggested. ("Ideally, for the sake of parallel execution, we'd disallow > automatic sharing of locals and automatic elevation of "this", "return" > or "break" to enclosing scopes.") It sounds like the tail is wagging the dog. And this could wreck the whole lambda change. I posit that Parallel Arrays and their ilk will *not* be the major usage of a suitably designed language change. Most developers simply don't need massively parallel type operations. Writing a typical web application, e-commerce app or desktop app doesn't benefit from Parallel Arrays - they are a specialist feature. While I understand the problem (Sun wants to add parallel arrays to the JDK but can't due to interface explosion and ugly inner class syntax), this needs to be treated as just one use case, that is relatively minor. Again, we still need some REQUIREMENTS for lambda to understand why on earth we're making this language change!!!! > The Lambda strawman is at odds with > some of those restrictions; resolution is forthcoming. I'm looking forward to it with trepidation. Stephen From neal at gafter.com Wed Feb 10 16:21:04 2010 From: neal at gafter.com (Neal Gafter) Date: Wed, 10 Feb 2010 16:21:04 -0800 Subject: Local functions In-Reply-To: <4B7334C6.2030506@sun.com> References: <15e8b9d21002091712j5d3a0488y48459bb6d183fa09@mail.gmail.com> <4B720D2B.3070704@sun.com> <4B72AFEF.9010807@univ-mlv.fr> <1D97026B-D743-44F5-8903-23776D11689C@googlemail.com> <4B72CB45.4010701@univ-mlv.fr> <4023DE02-B32D-4A04-B70F-380A2DFB63B1@googlemail.com> <4B72D693.2000209@univ-mlv.fr> <4B73085E.7080101@sun.com> <15e8b9d21002101147u64c8bb9dhc5be3f2a3c65feb9@mail.gmail.com> <4B7334C6.2030506@sun.com> Message-ID: <15e8b9d21002101621o32357120n22472fb43d5f397f@mail.gmail.com> On Wed, Feb 10, 2010 at 2:35 PM, Alex Buckley wrote: > That said, I am happy to prioritize parallel use cases over non-parallel > use cases. Sun is not aiming to add lambdas to allow control abstraction > or DSL building, but rather to support FJ ParallelArray and APIs like > it. So I'm fine if lambda bodies are restricted along the lines Doug > suggested. ("Ideally, for the sake of parallel execution, we'd disallow > automatic sharing of locals and automatic elevation of "this", "return" > or "break" to enclosing scopes.") The Lambda strawman is at odds with > some of those restrictions; resolution is forthcoming. Interesting. The point of disallowing the use of "this" would be, presumably, that the enclosing object might be mutable. By that same precise logic, we should disallow access to members of the enclosing class as well (after all, the "this" variable is itself final, while its members might not be). From Alex.Buckley at Sun.COM Wed Feb 10 16:33:08 2010 From: Alex.Buckley at Sun.COM (Alex Buckley) Date: Wed, 10 Feb 2010 16:33:08 -0800 Subject: Local functions In-Reply-To: <15e8b9d21002101621o32357120n22472fb43d5f397f@mail.gmail.com> References: <15e8b9d21002091712j5d3a0488y48459bb6d183fa09@mail.gmail.com> <4B720D2B.3070704@sun.com> <4B72AFEF.9010807@univ-mlv.fr> <1D97026B-D743-44F5-8903-23776D11689C@googlemail.com> <4B72CB45.4010701@univ-mlv.fr> <4023DE02-B32D-4A04-B70F-380A2DFB63B1@googlemail.com> <4B72D693.2000209@univ-mlv.fr> <4B73085E.7080101@sun.com> <15e8b9d21002101147u64c8bb9dhc5be3f2a3c65feb9@mail.gmail.com> <4B7334C6.2030506@sun.com> <15e8b9d21002101621o32357120n22472fb43d5f397f@mail.gmail.com> Message-ID: <4B735044.9000404@sun.com> Neal Gafter wrote: > On Wed, Feb 10, 2010 at 2:35 PM, Alex Buckley wrote: >> That said, I am happy to prioritize parallel use cases over non-parallel >> use cases. Sun is not aiming to add lambdas to allow control abstraction >> or DSL building, but rather to support FJ ParallelArray and APIs like >> it. So I'm fine if lambda bodies are restricted along the lines Doug >> suggested. ("Ideally, for the sake of parallel execution, we'd disallow >> automatic sharing of locals and automatic elevation of "this", "return" >> or "break" to enclosing scopes.") The Lambda strawman is at odds with >> some of those restrictions; resolution is forthcoming. > > Interesting. The point of disallowing the use of "this" would be, > presumably, that the enclosing object might be mutable. By that same > precise logic, we should disallow access to members of the enclosing > class as well (after all, the "this" variable is itself final, while > its members might not be). No. Doug's point was that many things may be accessed, but defaults must be cautious. Alex From Alex.Buckley at Sun.COM Wed Feb 10 16:38:03 2010 From: Alex.Buckley at Sun.COM (Alex Buckley) Date: Wed, 10 Feb 2010 16:38:03 -0800 Subject: Local functions In-Reply-To: <4b4f45e01002101611i6a46310oc267f66545ffeb98@mail.gmail.com> References: <15e8b9d21002091712j5d3a0488y48459bb6d183fa09@mail.gmail.com> <4B720D2B.3070704@sun.com> <4B72AFEF.9010807@univ-mlv.fr> <1D97026B-D743-44F5-8903-23776D11689C@googlemail.com> <4B72CB45.4010701@univ-mlv.fr> <4023DE02-B32D-4A04-B70F-380A2DFB63B1@googlemail.com> <4B72D693.2000209@univ-mlv.fr> <4B73085E.7080101@sun.com> <15e8b9d21002101147u64c8bb9dhc5be3f2a3c65feb9@mail.gmail.com> <4B7334C6.2030506@sun.com> <4b4f45e01002101611i6a46310oc267f66545ffeb98@mail.gmail.com> Message-ID: <4B73516B.2000100@sun.com> Stephen Colebourne wrote: >> That said, I am happy to prioritize parallel use cases over non-parallel >> use cases. Sun is not aiming to add lambdas to allow control abstraction >> or DSL building, but rather to support FJ ParallelArray and APIs like >> it. > > And the whole use case category of functors and predicates operating > on regular collections? Abandoned? Only accessible via a second class > syntax? "APIs like it" is a comprehensive term. You mention predicates; any filter operation, be it in ParallelArray or the most sequential collections API, is an obvious candidate for lambdas. > I didn't think thats what lambda was about. But then we don't have any > REQUIREMENTS so we can't judge. Goals at http://blogs.sun.com/mr/entry/closures. I am not interested in a philosophical debate, but the kind of requirements you mean are for frameworks, not languages. (Except for dull things like "Must support Unicode 5.0".) For example, I don't recall requirements for BGGA, on which you based JCA. >> So I'm fine if lambda bodies are restricted along the lines Doug >> suggested. ("Ideally, for the sake of parallel execution, we'd disallow >> automatic sharing of locals and automatic elevation of "this", "return" >> or "break" to enclosing scopes.") > > It sounds like the tail is wagging the dog. And this could wreck the > whole lambda change. I posit that Parallel Arrays and their ilk will > *not* be the major usage of a suitably designed language change. Most > developers simply don't need massively parallel type operations. > Writing a typical web application, e-commerce app or desktop app > doesn't benefit from Parallel Arrays - they are a specialist feature. Thanks for your view. The Java ecosystem is a big place. This project is interested primarily in supporting parallel execution. If you are interested primarily in something else, there are other projects for you. Alex From neal at gafter.com Wed Feb 10 16:57:04 2010 From: neal at gafter.com (Neal Gafter) Date: Wed, 10 Feb 2010 16:57:04 -0800 Subject: Local functions In-Reply-To: <4B73516B.2000100@sun.com> References: <15e8b9d21002091712j5d3a0488y48459bb6d183fa09@mail.gmail.com> <1D97026B-D743-44F5-8903-23776D11689C@googlemail.com> <4B72CB45.4010701@univ-mlv.fr> <4023DE02-B32D-4A04-B70F-380A2DFB63B1@googlemail.com> <4B72D693.2000209@univ-mlv.fr> <4B73085E.7080101@sun.com> <15e8b9d21002101147u64c8bb9dhc5be3f2a3c65feb9@mail.gmail.com> <4B7334C6.2030506@sun.com> <4b4f45e01002101611i6a46310oc267f66545ffeb98@mail.gmail.com> <4B73516B.2000100@sun.com> Message-ID: <15e8b9d21002101657v1f7a2f1aob8e379459306da76@mail.gmail.com> On Wed, Feb 10, 2010 at 4:38 PM, Alex Buckley wrote: > I don't recall requirements for BGGA, on which you based JCA. For BGGA, first paragraph. See also the JSR proposal. In short (eliding definitions) "Add support so programs can operate on an arbitrary block of code with parameters, and simplify the use of methods that receive such blocks". Cheers, Neal From scolebourne at joda.org Wed Feb 10 17:05:03 2010 From: scolebourne at joda.org (Stephen Colebourne) Date: Thu, 11 Feb 2010 01:05:03 +0000 Subject: Local functions In-Reply-To: <4B73516B.2000100@sun.com> References: <15e8b9d21002091712j5d3a0488y48459bb6d183fa09@mail.gmail.com> <1D97026B-D743-44F5-8903-23776D11689C@googlemail.com> <4B72CB45.4010701@univ-mlv.fr> <4023DE02-B32D-4A04-B70F-380A2DFB63B1@googlemail.com> <4B72D693.2000209@univ-mlv.fr> <4B73085E.7080101@sun.com> <15e8b9d21002101147u64c8bb9dhc5be3f2a3c65feb9@mail.gmail.com> <4B7334C6.2030506@sun.com> <4b4f45e01002101611i6a46310oc267f66545ffeb98@mail.gmail.com> <4B73516B.2000100@sun.com> Message-ID: <4b4f45e01002101705n4796199cp310f4533b66711b0@mail.gmail.com> On 11 February 2010 00:38, Alex Buckley wrote: > "APIs like it" is a comprehensive term. You mention predicates; any > filter operation, be it in ParallelArray or the most sequential > collections API, is an obvious candidate for lambdas. [then later] > Thanks for your view. The Java ecosystem is a big place. This project is > interested primarily in supporting parallel execution. These two statements appear to be in contradiction. Or do you believe that you can primarily support parallel execution without compromising the more common case of regular inline non-parallel functors? > Goals at http://blogs.sun.com/mr/entry/closures. I'm looking for something clearer now we are in the formal project (not a kick off blog entry). If you'll forgive me, I seem to recall lack of requirements being a common complaint about Project Jigsaw too. ie. I'm suggesting you might find it something useful to do from a community engagement POV, to set expectations in the wider community as well as to clarify the debate here. IMO, doing so on Jigsaw would have helped the community debate there too. I certainly don't think the need for requirements is philosophical! (ie. I'm trying to encourage you to do things that will help everyone outside Sun understand what is going on here and make your lives a lot easier down the road) > If you are > interested primarily in something else, there are other projects for you. There are no other projects with any chance of altering the Java language. My interest here is solely in ensuring that the design chosen will appeal to and be usable by to the wider community of Java developers (non alpha-geeks) - and yes, I know that will be my interpretation of their needs. I remain open to reading any proposal you may put forward and judging it on its merits. Stephen From jkuhnert at gmail.com Wed Feb 10 17:09:36 2010 From: jkuhnert at gmail.com (Jesse Kuhnert) Date: Wed, 10 Feb 2010 20:09:36 -0500 Subject: Local functions In-Reply-To: <4b4f45e01002101705n4796199cp310f4533b66711b0@mail.gmail.com> References: <15e8b9d21002091712j5d3a0488y48459bb6d183fa09@mail.gmail.com> <4B72CB45.4010701@univ-mlv.fr> <4023DE02-B32D-4A04-B70F-380A2DFB63B1@googlemail.com> <4B72D693.2000209@univ-mlv.fr> <4B73085E.7080101@sun.com> <15e8b9d21002101147u64c8bb9dhc5be3f2a3c65feb9@mail.gmail.com> <4B7334C6.2030506@sun.com> <4b4f45e01002101611i6a46310oc267f66545ffeb98@mail.gmail.com> <4B73516B.2000100@sun.com> <4b4f45e01002101705n4796199cp310f4533b66711b0@mail.gmail.com> Message-ID: <7926817e1002101709i58b8fd07ifb0c85c7a02d23d0@mail.gmail.com> By asking for requirements you seem to be ultimately asking for the design and implementation as well, since the two kind of go hand in hand in this instance don't they? Seems like something that shouldn't be created with your standard "requirements defined first" sort of structure. Free flowing thoughts and random changes will likely produce better results but who knows ...I don't really know what I'm talking about anyways. On Wed, Feb 10, 2010 at 8:05 PM, Stephen Colebourne wrote: > On 11 February 2010 00:38, Alex Buckley wrote: >> "APIs like it" is a comprehensive term. You mention predicates; any >> filter operation, be it in ParallelArray or the most sequential >> collections API, is an obvious candidate for lambdas. > [then later] >> Thanks for your view. The Java ecosystem is a big place. This project is >> interested primarily in supporting parallel execution. > > These two statements appear to be in contradiction. Or do you believe > that you can primarily support parallel execution without compromising > the more common case of regular inline non-parallel functors? > >> Goals at http://blogs.sun.com/mr/entry/closures. > > I'm looking for something clearer now we are in the formal project > (not a kick off blog entry). > > If you'll forgive me, I seem to recall lack of requirements being a > common complaint about Project Jigsaw too. ie. I'm suggesting you > might find it something useful to do from a community engagement POV, > to set expectations in the wider community as well as to clarify the > debate here. IMO, doing so on Jigsaw would have helped the community > debate there too. > > I certainly don't think the need for requirements is philosophical! > (ie. I'm trying to encourage you to do things that will help everyone > outside Sun understand what is going on here and make your lives a lot > easier down the road) > >> ?If you are >> interested primarily in something else, there are other projects for you. > > There are no other projects with any chance of altering the Java > language. My interest here is solely in ensuring that the design > chosen will appeal to and be usable by to the wider community of Java > developers (non alpha-geeks) - and yes, I know that will be my > interpretation of their needs. I remain open to reading any proposal > you may put forward and judging it on its merits. > > Stephen > > From Alex.Buckley at Sun.COM Wed Feb 10 17:34:18 2010 From: Alex.Buckley at Sun.COM (Alex Buckley) Date: Wed, 10 Feb 2010 17:34:18 -0800 Subject: Local functions In-Reply-To: <15e8b9d21002101657v1f7a2f1aob8e379459306da76@mail.gmail.com> References: <15e8b9d21002091712j5d3a0488y48459bb6d183fa09@mail.gmail.com> <1D97026B-D743-44F5-8903-23776D11689C@googlemail.com> <4B72CB45.4010701@univ-mlv.fr> <4023DE02-B32D-4A04-B70F-380A2DFB63B1@googlemail.com> <4B72D693.2000209@univ-mlv.fr> <4B73085E.7080101@sun.com> <15e8b9d21002101147u64c8bb9dhc5be3f2a3c65feb9@mail.gmail.com> <4B7334C6.2030506@sun.com> <4b4f45e01002101611i6a46310oc267f66545ffeb98@mail.gmail.com> <4B73516B.2000100@sun.com> <15e8b9d21002101657v1f7a2f1aob8e379459306da76@mail.gmail.com> Message-ID: <4B735E9A.2050906@sun.com> Neal Gafter wrote: > On Wed, Feb 10, 2010 at 4:38 PM, Alex Buckley wrote: >> I don't recall requirements for BGGA, on which you based JCA. > > For BGGA, first paragraph. See also the JSR proposal. In short > (eliding definitions) "Add support so programs can operate on an > arbitrary block of code with parameters, and simplify the use of > methods that receive such blocks". Lambda has similar high-level wording about boilerplate for parallel execution. I know what Stephen meant by "requirements", and a couple of high-level sentences was not it. Alex From neal at gafter.com Wed Feb 10 17:50:45 2010 From: neal at gafter.com (Neal Gafter) Date: Wed, 10 Feb 2010 17:50:45 -0800 Subject: Local functions In-Reply-To: <4B735E9A.2050906@sun.com> References: <15e8b9d21002091712j5d3a0488y48459bb6d183fa09@mail.gmail.com> <4023DE02-B32D-4A04-B70F-380A2DFB63B1@googlemail.com> <4B72D693.2000209@univ-mlv.fr> <4B73085E.7080101@sun.com> <15e8b9d21002101147u64c8bb9dhc5be3f2a3c65feb9@mail.gmail.com> <4B7334C6.2030506@sun.com> <4b4f45e01002101611i6a46310oc267f66545ffeb98@mail.gmail.com> <4B73516B.2000100@sun.com> <15e8b9d21002101657v1f7a2f1aob8e379459306da76@mail.gmail.com> <4B735E9A.2050906@sun.com> Message-ID: <15e8b9d21002101750g63043f51jb73c39ada9c327c6@mail.gmail.com> On Wed, Feb 10, 2010 at 5:34 PM, Alex Buckley wrote: > Lambda has similar high-level wording about boilerplate for parallel > execution. I know what Stephen meant by "requirements", and a couple of > high-level sentences was not it. I read "The goal of this Project is to formulate a proposal to add first-class functions, function types, and lambda expressions (informally, "closures") to Java, and to implement a prototype suitable for inclusion in JDK 7 so as to enable broad experimentation." And "Working with parallel arrays in Java, unfortunately, requires lots of boilerplate code to solve even simple problems. Closures can eliminate that boilerplate." I cannot follow the thread of reasoning that leads from these goals to the kinds of restrictions you're considering (e.g. denying access to variables from the enclosing scope). Each additional restriction becomes a source of boilerplate for programmers who must work around those restrictions, so it would appear you should be removing restrictions rather than adding them. Cheers, Neal From Alex.Buckley at Sun.COM Wed Feb 10 18:04:24 2010 From: Alex.Buckley at Sun.COM (Alex Buckley) Date: Wed, 10 Feb 2010 18:04:24 -0800 Subject: Local functions In-Reply-To: <15e8b9d21002101750g63043f51jb73c39ada9c327c6@mail.gmail.com> References: <15e8b9d21002091712j5d3a0488y48459bb6d183fa09@mail.gmail.com> <4023DE02-B32D-4A04-B70F-380A2DFB63B1@googlemail.com> <4B72D693.2000209@univ-mlv.fr> <4B73085E.7080101@sun.com> <15e8b9d21002101147u64c8bb9dhc5be3f2a3c65feb9@mail.gmail.com> <4B7334C6.2030506@sun.com> <4b4f45e01002101611i6a46310oc267f66545ffeb98@mail.gmail.com> <4B73516B.2000100@sun.com> <15e8b9d21002101657v1f7a2f1aob8e379459306da76@mail.gmail.com> <4B735E9A.2050906@sun.com> <15e8b9d21002101750g63043f51jb73c39ada9c327c6@mail.gmail.com> Message-ID: <4B7365A8.2040102@sun.com> Neal Gafter wrote: > I cannot follow the thread of reasoning that leads from these goals to > the kinds of restrictions you're considering (e.g. denying access to > variables from the enclosing scope). I don't know where this business of denying access has come from. Alex From Alex.Buckley at Sun.COM Wed Feb 10 18:06:34 2010 From: Alex.Buckley at Sun.COM (Alex Buckley) Date: Wed, 10 Feb 2010 18:06:34 -0800 Subject: Local functions In-Reply-To: <4b4f45e01002101705n4796199cp310f4533b66711b0@mail.gmail.com> References: <15e8b9d21002091712j5d3a0488y48459bb6d183fa09@mail.gmail.com> <1D97026B-D743-44F5-8903-23776D11689C@googlemail.com> <4B72CB45.4010701@univ-mlv.fr> <4023DE02-B32D-4A04-B70F-380A2DFB63B1@googlemail.com> <4B72D693.2000209@univ-mlv.fr> <4B73085E.7080101@sun.com> <15e8b9d21002101147u64c8bb9dhc5be3f2a3c65feb9@mail.gmail.com> <4B7334C6.2030506@sun.com> <4b4f45e01002101611i6a46310oc267f66545ffeb98@mail.gmail.com> <4B73516B.2000100@sun.com> <4b4f45e01002101705n4796199cp310f4533b66711b0@mail.gmail.com> Message-ID: <4B73662A.1000008@sun.com> Stephen Colebourne wrote: > On 11 February 2010 00:38, Alex Buckley wrote: >> "APIs like it" is a comprehensive term. You mention predicates; any >> filter operation, be it in ParallelArray or the most sequential >> collections API, is an obvious candidate for lambdas. > [then later] >> Thanks for your view. The Java ecosystem is a big place. This project is >> interested primarily in supporting parallel execution. > > These two statements appear to be in contradiction. Or do you believe > that you can primarily support parallel execution without compromising > the more common case of regular inline non-parallel functors? Since nothing is being removed from the language, and since lambdas for parallel execution are likely to be quite restricted, it seems to me that nothing is being compromised. If as an Apache member you are concerned about the impact of lambdas on Commons Collections and its functors package, then you should say so, giving as much detail as possible. >> Goals at http://blogs.sun.com/mr/entry/closures. > > I'm looking for something clearer now we are in the formal project > (not a kick off blog entry). > > If you'll forgive me, I seem to recall lack of requirements being a > common complaint about Project Jigsaw too. ie. Your recall is faulty. A handful of people asked for requirements on the Jigsaw module system. Everyone else was rather excited by the progress that had already been made. The modular JDK is a clear driver for the Jigsaw module system; see my StrangeLoop presentation for more. > (ie. I'm trying to encourage you to do things that will help everyone > outside Sun understand what is going on here and make your lives a lot > easier down the road) What is going on here is a discussion among a small set of people with varying levels of expertise and interest. It is very early stage. As I said, I read all mails and - with the parallel execution goal mainly but not exclusively in mind - try to find the most acceptable features we could explain to a broad developer base. > There are no other projects with any chance of altering the Java > language. Project Lambda is not a JSR and Sun is not the only member of the JCP Executive Committee. Perhaps you should solicit their views on these topics. Here they are: http://jcp.org/en/participation/committee Alex From Alex.Buckley at Sun.COM Wed Feb 10 18:11:47 2010 From: Alex.Buckley at Sun.COM (Alex Buckley) Date: Wed, 10 Feb 2010 18:11:47 -0800 Subject: Local functions In-Reply-To: <4B73662A.1000008@sun.com> References: <15e8b9d21002091712j5d3a0488y48459bb6d183fa09@mail.gmail.com> <1D97026B-D743-44F5-8903-23776D11689C@googlemail.com> <4B72CB45.4010701@univ-mlv.fr> <4023DE02-B32D-4A04-B70F-380A2DFB63B1@googlemail.com> <4B72D693.2000209@univ-mlv.fr> <4B73085E.7080101@sun.com> <15e8b9d21002101147u64c8bb9dhc5be3f2a3c65feb9@mail.gmail.com> <4B7334C6.2030506@sun.com> <4b4f45e01002101611i6a46310oc267f66545ffeb98@mail.gmail.com> <4B73516B.2000100@sun.com> <4b4f45e01002101705n4796199cp310f4533b66711b0@mail.gmail.com> <4B73662A.1000008@sun.com> Message-ID: <4B736763.6020502@sun.com> Alex Buckley wrote: > Since nothing is being removed from the language, and since lambdas for > parallel execution are likely to be quite restricted, it seems to me > that nothing is being compromised. Before anyone twists my words, "quite restricted" in the above means in comparison to the full wonder of BGGA and FCM+JCA. It does not mean "lambdas can do nothing except access their formal parameters". Alex From scolebourne at joda.org Wed Feb 10 18:23:19 2010 From: scolebourne at joda.org (Stephen Colebourne) Date: Thu, 11 Feb 2010 02:23:19 +0000 Subject: Parallel-safe lambdas Message-ID: <4b4f45e01002101823i2a1eeb06l6fc3067b7cbd5926@mail.gmail.com> Recent threads have suggested that Project Lambda is primarily (ie very) interested in the parallel execution use case (specifically Parallel Array). The problem has arisen that exposing "this" and local variables can cause race conditions. As such, I thought I'd outline a way to avoid that: Example, filter a large list in parallel based on a predicate that uses a local (or instance) variable. It filters strings of length 0, 2, 3 and 5 (yes, its a stupid example): boolean[] lengthAcceptable = new boolean[] {true, false, true, true, false, true}; ParallelArrays.filter(list, #(String str : boolean[] lengths = lengthsAcceptable) { if (str.length() > lengths.length) { return true; } return lengths[str.length()]; }); Rules (not formal JLS): - "this" is scoped to the lambda and provides no access to the lexical scope (or anything other than the lambda object) - variables must be 'imported' into the lambda scope via manual capture - a colon followed by variable declaration style statements - the ": boolean[] lengths = lengthsAcceptable" part in the example - variables imported in this manner are deep-cloned or serialized to create a fully independent copy to be used by the lambda - variables imported are copied at lambda creation time - there is no 'capture' of the variable - attempts to use the lexical scope ("this", instance variables, local variables) do not compile - if access to the lexical "this" is required, then it must be manually captured, which will cause it to be deep cloned With these rules, I believe (its late here) that a lambda cannot suffer a data race. Therefore, I claim that this solution correctly meets the "primary goal" of parallel execution and parallel arrays. Of course, this is actually an ugly, excessively boilerplate, overly protective approach that is extremely use-case driven and would be a terrible addition to the Java language. But it would seem a reasonably natural conclusion of the "primary goal" of parallel execution argument. Sometimes its instructive to see what the full-blown solution might look like before arriving at a suitable middle ground. As I hope is plain, I believe the middle ground involves lexically-scoped "this", amongst other things. Stephen From Alex.Buckley at Sun.COM Wed Feb 10 19:29:19 2010 From: Alex.Buckley at Sun.COM (Alex Buckley) Date: Wed, 10 Feb 2010 19:29:19 -0800 Subject: Parallel-safe lambdas In-Reply-To: <4b4f45e01002101823i2a1eeb06l6fc3067b7cbd5926@mail.gmail.com> References: <4b4f45e01002101823i2a1eeb06l6fc3067b7cbd5926@mail.gmail.com> Message-ID: <4B73798F.6000003@sun.com> You've set up quite a strawman. Since the draft Lambda spec doesn't require import of variables, doesn't ban use of variables in the lexical scopem and doesn't say anything about copying locals, I don't know why you're wasting time with fake rules that do. Your experience with Apache Commons' functors would be valuable to this list. But if all you're going to do is argue with the goal of the project, unsubscribe. Alex From neal at gafter.com Wed Feb 10 19:32:27 2010 From: neal at gafter.com (Neal Gafter) Date: Wed, 10 Feb 2010 19:32:27 -0800 Subject: Local functions In-Reply-To: <4B7365A8.2040102@sun.com> References: <15e8b9d21002091712j5d3a0488y48459bb6d183fa09@mail.gmail.com> <4B73085E.7080101@sun.com> <15e8b9d21002101147u64c8bb9dhc5be3f2a3c65feb9@mail.gmail.com> <4B7334C6.2030506@sun.com> <4b4f45e01002101611i6a46310oc267f66545ffeb98@mail.gmail.com> <4B73516B.2000100@sun.com> <15e8b9d21002101657v1f7a2f1aob8e379459306da76@mail.gmail.com> <4B735E9A.2050906@sun.com> <15e8b9d21002101750g63043f51jb73c39ada9c327c6@mail.gmail.com> <4B7365A8.2040102@sun.com> Message-ID: <15e8b9d21002101932i44691260j1b2d858a39cad378@mail.gmail.com> On Wed, Feb 10, 2010 at 6:04 PM, Alex Buckley wrote: > I don't know where this business of denying access has come from. Sorry, I leapt before I looked. Let's see what you have in mind before commenting further on this. From oallouch at free.fr Thu Feb 11 01:57:36 2010 From: oallouch at free.fr (Olivier Allouch) Date: Thu, 11 Feb 2010 10:57:36 +0100 Subject: Parallel-safe lambdas In-Reply-To: <4B73798F.6000003@sun.com> References: <4b4f45e01002101823i2a1eeb06l6fc3067b7cbd5926@mail.gmail.com> <4B73798F.6000003@sun.com> Message-ID: <4B73D490.7010105@free.fr> Ouch ! I, like Stephen Colebourne, am surprised. I thought the goal was to add closures to the language, not to a single library. I can tell you all the articles and tutorials that will come after the inclusion of closures will be about predicates and forEach/map/filter-like examples. I understand more why GUI coding is never mentioned here. The real issue Java has isn't parallel execution (I'll probably never use it), it's the lack of closures in the everyday coding. Olivier Allouch Le 11/02/2010 04:29, Alex Buckley a ?crit : > You've set up quite a strawman. Since the draft Lambda spec doesn't > require import of variables, doesn't ban use of variables in the lexical > scopem and doesn't say anything about copying locals, I don't know why > you're wasting time with fake rules that do. > > Your experience with Apache Commons' functors would be valuable to this > list. But if all you're going to do is argue with the goal of the > project, unsubscribe. > > Alex > > From mthornton at optrak.co.uk Thu Feb 11 02:12:10 2010 From: mthornton at optrak.co.uk (Mark Thornton) Date: Thu, 11 Feb 2010 10:12:10 +0000 Subject: Parallel-safe lambdas In-Reply-To: <4B73D490.7010105@free.fr> References: <4b4f45e01002101823i2a1eeb06l6fc3067b7cbd5926@mail.gmail.com> <4B73798F.6000003@sun.com> <4B73D490.7010105@free.fr> Message-ID: <4B73D7FA.5090405@optrak.co.uk> Olivier Allouch wrote: > Ouch ! > > I, like Stephen Colebourne, am surprised. I thought the goal was to add > closures to the language, not to a single library. > I'm not sure why you are surprised given that the closures exercise was restarted with this: http://blogs.sun.com/mr/entry/closures The motivation is clearly to aid use of parallelism. Now this doesn't mean that many ordinary programmers will be writing parallel code, but that more API may use parallel implementations. So if someone uses Collections.sort or Arrays.sort, the implementation may be parallel if the size warrants it. For API taking lambda parameters it would be desireable to be able to mark such parameters as 'restricted' in some way. Or perhaps have lambdas that satisfy appropriate concurrency rules implement some marker interface. The API could then use a parallel implementation for such marked lambdas. The user can then be largely oblivious to the concurrency issues, while leaving the likes of Doug Lea to provide high performance implementations. Mark Thornton From forax at univ-mlv.fr Thu Feb 11 02:17:13 2010 From: forax at univ-mlv.fr (=?ISO-8859-1?Q?R=E9mi_Forax?=) Date: Thu, 11 Feb 2010 11:17:13 +0100 Subject: Parallel-safe lambdas In-Reply-To: <4B73D490.7010105@free.fr> References: <4b4f45e01002101823i2a1eeb06l6fc3067b7cbd5926@mail.gmail.com> <4B73798F.6000003@sun.com> <4B73D490.7010105@free.fr> Message-ID: <4B73D929.1030906@univ-mlv.fr> Olivier and Stephen, ParallelArray is full of predicates, filters and al. see http://www.ibm.com/developerworks/java/library/j-jtp03048.html I don't understand why targeting ParallelArray API will make lambdas unusable elsewhere ? R?mi Le 11/02/2010 10:57, Olivier Allouch a ?crit : > Ouch ! > > I, like Stephen Colebourne, am surprised. I thought the goal was to add > closures to the language, not to a single library. > I can tell you all the articles and tutorials that will come after the > inclusion of closures will be about predicates and > forEach/map/filter-like examples. > I understand more why GUI coding is never mentioned here. > The real issue Java has isn't parallel execution (I'll probably never > use it), it's the lack of closures in the everyday coding. > > Olivier Allouch > > Le 11/02/2010 04:29, Alex Buckley a ?crit : > >> You've set up quite a strawman. Since the draft Lambda spec doesn't >> require import of variables, doesn't ban use of variables in the lexical >> scopem and doesn't say anything about copying locals, I don't know why >> you're wasting time with fake rules that do. >> >> Your experience with Apache Commons' functors would be valuable to this >> list. But if all you're going to do is argue with the goal of the >> project, unsubscribe. >> >> Alex >> >> >> > > From pdoubleya at gmail.com Thu Feb 11 02:50:25 2010 From: pdoubleya at gmail.com (Patrick Wright) Date: Thu, 11 Feb 2010 11:50:25 +0100 Subject: Parallel-safe lambdas In-Reply-To: <4B73D490.7010105@free.fr> References: <4b4f45e01002101823i2a1eeb06l6fc3067b7cbd5926@mail.gmail.com> <4B73798F.6000003@sun.com> <4B73D490.7010105@free.fr> Message-ID: <64efa1ba1002110250n50b92bbbj3be2a38774e3c58e@mail.gmail.com> > The real issue Java has isn't parallel execution (I'll probably never > use it), it's the lack of closures in the everyday coding. I think an important driver for this whole exercise is that without some way to make the parallel array API more manageable, it will be left out of JDK 7--which means it may not show up for another 2 or probably 3 years, till JDK 8. That's a lifetime in technology innovation, these days. Personally, I'm all in favor of having a highly targeted usage scenario driving this project. 0.02 Patrick From forax at univ-mlv.fr Thu Feb 11 03:04:15 2010 From: forax at univ-mlv.fr (=?ISO-8859-1?Q?R=E9mi_Forax?=) Date: Thu, 11 Feb 2010 12:04:15 +0100 Subject: Parallel-safe lambdas In-Reply-To: <4B73D7FA.5090405@optrak.co.uk> References: <4b4f45e01002101823i2a1eeb06l6fc3067b7cbd5926@mail.gmail.com> <4B73798F.6000003@sun.com> <4B73D490.7010105@free.fr> <4B73D7FA.5090405@optrak.co.uk> Message-ID: <4B73E42F.4080402@univ-mlv.fr> Le 11/02/2010 11:12, Mark Thornton a ?crit : > Olivier Allouch wrote: > >> Ouch ! >> >> I, like Stephen Colebourne, am surprised. I thought the goal was to add >> closures to the language, not to a single library. >> >> > I'm not sure why you are surprised given that the closures exercise was > restarted with this: > http://blogs.sun.com/mr/entry/closures > The motivation is clearly to aid use of parallelism. Now this doesn't > mean that many ordinary programmers will be writing parallel code, but > that more API may use parallel implementations. So if someone uses > Collections.sort or Arrays.sort, the implementation may be parallel if > the size warrants it. > > For API taking lambda parameters it would be desireable to be able to > mark such parameters as 'restricted' in some way. Or perhaps have > lambdas that satisfy appropriate concurrency rules implement some marker > interface. The API could then use a parallel implementation for such > marked lambdas. The user can then be largely oblivious to the > concurrency issues, while leaving the likes of Doug Lea to provide high > performance implementations. > > Mark Thornton > Mark, I don't think we need to distinguish between restricted and unrestricted closure. I prefer the idea to avoid shared states to let the implementation choose if the code has to be run parallel or not. By example, using the Apache collections, you can write: shared int i=0; List list = CollectionsUtils.forAllDo(list, #(String s) { if (s.startsWith("foo")) i++; }): But this code can't run in parallel. It's better to use a kind of map/reduce: int i = Collections.mapReduce(list, #(String s) (s.startsWith("foo")?1:0), #(int value, int value2) ( value + value2)); In my opinion, lambdas should be restricted to only access final local variables and final fields. But perhaps have seen too much talks of Erik Meijer :) R?mi From mthornton at optrak.co.uk Thu Feb 11 03:11:25 2010 From: mthornton at optrak.co.uk (Mark Thornton) Date: Thu, 11 Feb 2010 11:11:25 +0000 Subject: Parallel-safe lambdas In-Reply-To: <4B73E42F.4080402@univ-mlv.fr> References: <4b4f45e01002101823i2a1eeb06l6fc3067b7cbd5926@mail.gmail.com> <4B73798F.6000003@sun.com> <4B73D490.7010105@free.fr> <4B73D7FA.5090405@optrak.co.uk> <4B73E42F.4080402@univ-mlv.fr> Message-ID: <4B73E5DD.8040008@optrak.co.uk> R?mi Forax wrote: > Mark, I don't think we need to distinguish between restricted > and unrestricted closure. > I prefer the idea to avoid shared states to let the implementation > choose if the code has to be run parallel or not. > I think that is the restriction (no shared state) to which Stephen and Olivier are objecting. In fact I think they are saying that shared state is so important to their view of lambdas that they would prefer to discard the parallel use of lambdas. Mark From forax at univ-mlv.fr Thu Feb 11 03:44:50 2010 From: forax at univ-mlv.fr (=?ISO-8859-1?Q?R=E9mi_Forax?=) Date: Thu, 11 Feb 2010 12:44:50 +0100 Subject: Parallel-safe lambdas In-Reply-To: <4B73E5DD.8040008@optrak.co.uk> References: <4b4f45e01002101823i2a1eeb06l6fc3067b7cbd5926@mail.gmail.com> <4B73798F.6000003@sun.com> <4B73D490.7010105@free.fr> <4B73D7FA.5090405@optrak.co.uk> <4B73E42F.4080402@univ-mlv.fr> <4B73E5DD.8040008@optrak.co.uk> Message-ID: <4B73EDB2.9040106@univ-mlv.fr> Le 11/02/2010 12:11, Mark Thornton a ?crit : > R?mi Forax wrote: >> Mark, I don't think we need to distinguish between restricted >> and unrestricted closure. >> I prefer the idea to avoid shared states to let the implementation >> choose if the code has to be run parallel or not. > > I think that is the restriction (no shared state) to which Stephen and > Olivier are objecting. In fact I think they are saying that shared > state is so important to their view of lambdas that they would prefer > to discard the parallel use of lambdas. > > Mark > I don't think we can come with a set of rules that doesn't allow to mutate something. By example, because there is no way in Java to know is a method is pure or not and because you can't disable all method calls, you can always mutate an objects using a method call. // no mutation sort(array, #(String s1, String s2) (s1.compareIgnoreCase(s2))); // mutations final Collection c = ... forEach(array, #(String s) { c.add(s); }); Perhaps the best option is to maintain the current status quo, local variable should be final, fields can be mutated. lambda is already a restricted form of anonymous class. A lambda doesn't define a class, unlike anonymous class, so mutating something is less easy. R?mi From scolebourne at joda.org Thu Feb 11 06:21:29 2010 From: scolebourne at joda.org (Stephen Colebourne) Date: Thu, 11 Feb 2010 14:21:29 +0000 Subject: Parallel-safe lambdas In-Reply-To: <4B73798F.6000003@sun.com> References: <4b4f45e01002101823i2a1eeb06l6fc3067b7cbd5926@mail.gmail.com> <4B73798F.6000003@sun.com> Message-ID: <4b4f45e01002110621l4ebd7d7coa1d5879aa0fb6481@mail.gmail.com> On 11 February 2010 03:29, Alex Buckley wrote: >I don't know why you're wasting time with fake rules that do. > But if all you're going to do is argue with the goal of the project, > unsubscribe. Actually, if you read the other responses in this thread you'll see quite divergent views on how important or not the parallel use case is, and whether it should or shouldn't constrain the solution - as such, I feel the exercise was extremely valuable. You have indicated that the parallel array use case is "primary", but what is still unclear is to what extent that statement means compromise in other use cases. If it is the primary one, what is the "secondary" one? What relative importance attaches to the "secondary" use case versus the "primary" one? These seem to be important questions. If the "primary" parallel use case is 99%, then as a whole, this mailing list should be designing for safety and against data races. And then the "strawman" above doesn't seem that unreasonable. Stephen From scolebourne at joda.org Thu Feb 11 06:30:20 2010 From: scolebourne at joda.org (Stephen Colebourne) Date: Thu, 11 Feb 2010 14:30:20 +0000 Subject: Parallel-safe lambdas In-Reply-To: <4B73E5DD.8040008@optrak.co.uk> References: <4b4f45e01002101823i2a1eeb06l6fc3067b7cbd5926@mail.gmail.com> <4B73798F.6000003@sun.com> <4B73D490.7010105@free.fr> <4B73D7FA.5090405@optrak.co.uk> <4B73E42F.4080402@univ-mlv.fr> <4B73E5DD.8040008@optrak.co.uk> Message-ID: <4b4f45e01002110630y79cc2c1fw222358c2039adf64@mail.gmail.com> On 11 February 2010 11:11, Mark Thornton wrote: > R?mi Forax wrote: >> Mark, I don't think we need to distinguish between restricted >> and unrestricted closure. >> I prefer the idea to avoid shared states to let the implementation >> choose if the code has to be run parallel or not. > > I think that is the restriction (no shared state) to which Stephen and > Olivier are objecting. In fact I think they are saying that shared state > is so important to their view of lambdas that they would prefer to > discard the parallel use of lambdas. Nope. Right now I'm trying to elide to what degree we are meant to be designing for parallel vs inline. I certainly don't want to "discard the parallel use of lambdas". I believe that both parallel and inline are fair and reasonable use cases for a well-designed lambda implementation. Some others have indicated that the inline use case should be effectively discarded. I'd also note that shared state is already common in inner classes that are the current working implementation of something-like-lambda in Java. These parallel data races already exist in code today, so extra restrictions feel heavy handed. Stephen From opinali at gmail.com Thu Feb 11 09:21:06 2010 From: opinali at gmail.com (Osvaldo Doederlein) Date: Thu, 11 Feb 2010 15:21:06 -0200 Subject: Local functions In-Reply-To: <4b4f45e01002101611i6a46310oc267f66545ffeb98@mail.gmail.com> References: <15e8b9d21002091712j5d3a0488y48459bb6d183fa09@mail.gmail.com> <4B72AFEF.9010807@univ-mlv.fr> <1D97026B-D743-44F5-8903-23776D11689C@googlemail.com> <4B72CB45.4010701@univ-mlv.fr> <4023DE02-B32D-4A04-B70F-380A2DFB63B1@googlemail.com> <4B72D693.2000209@univ-mlv.fr> <4B73085E.7080101@sun.com> <15e8b9d21002101147u64c8bb9dhc5be3f2a3c65feb9@mail.gmail.com> <4B7334C6.2030506@sun.com> <4b4f45e01002101611i6a46310oc267f66545ffeb98@mail.gmail.com> Message-ID: 2010/2/10 Stephen Colebourne > > So I'm fine if lambda bodies are restricted along the lines Doug > > suggested. ("Ideally, for the sake of parallel execution, we'd disallow > > automatic sharing of locals and automatic elevation of "this", "return" > > or "break" to enclosing scopes.") > > It sounds like the tail is wagging the dog. And this could wreck the > whole lambda change. I posit that Parallel Arrays and their ilk will > *not* be the major usage of a suitably designed language change. Most > developers simply don't need massively parallel type operations. > Writing a typical web application, e-commerce app or desktop app > doesn't benefit from Parallel Arrays - they are a specialist feature. > > While I understand the problem (Sun wants to add parallel arrays to > the JDK but can't due to interface explosion and ugly inner class > syntax), this needs to be treated as just one use case, that is > relatively minor. > #{+1}, #{+1}, #{+1}, #{+1} // Lambda agreements, in parallel on a quad-core Even if Parallel Arrays _are_ a major use case, I don't think the lambda spec should provide any level of baby-sitting for developers, and especially not anything that creates restriction or even harder usage (requiring extra modifiers) for other use cases. If a programmer creates a parallel computation with lambdas that share some mutable variable and race on it, just let that program explode in the programmer's face. Next release FindBugs will certainly have a new rule that emits a warning for such code, and that's all. (If we are kind enough, maybe javac itself will have such [optional] warning.) Parallel Array will be used mostly by expert programmers, and mostly for algorithms with critical performance requirements. Unless we consider further language sugar, e.g. auto-parallelization (accum = 0; par-for (x: elems) {accum += x.something()} => emits both ParallelArray code and a lambda) but this is not in the roadmap for SE7 and if we do that in the future, it's a different design setting, javac would have much more control (will typically generate the lambdas) so it will just generate the "right" code. In fact, I expect this to be a realistic roadmap, if not for Java 8+ then certainly (and much sooner anyway) for other languages like Groovy, Scala, Clojure or Fortress. So, in the long term I see Parallel Arrays as a "raw" low-level API that will be used directly mostly by experts / library writers / language designers... even with lambdas. Most programmers won't need Parallel Array within the SE7 lifecycle; and a few years down the road, when SE8+ is available (not to mention other languages), most programmers will need it (unless we already have graphene-based 100GHz CPUs, he he) but then they will jump straight to higher-level syntax. A+ Osvaldo From jkuhnert at gmail.com Thu Feb 11 09:44:19 2010 From: jkuhnert at gmail.com (Jesse Kuhnert) Date: Thu, 11 Feb 2010 12:44:19 -0500 Subject: Local functions In-Reply-To: References: <15e8b9d21002091712j5d3a0488y48459bb6d183fa09@mail.gmail.com> <1D97026B-D743-44F5-8903-23776D11689C@googlemail.com> <4B72CB45.4010701@univ-mlv.fr> <4023DE02-B32D-4A04-B70F-380A2DFB63B1@googlemail.com> <4B72D693.2000209@univ-mlv.fr> <4B73085E.7080101@sun.com> <15e8b9d21002101147u64c8bb9dhc5be3f2a3c65feb9@mail.gmail.com> <4B7334C6.2030506@sun.com> <4b4f45e01002101611i6a46310oc267f66545ffeb98@mail.gmail.com> Message-ID: <7926817e1002110944k18234bd0j157d143506a3f315@mail.gmail.com> That sounds completely reasonable to me .. (esp after recently reading http://prog21.dadgum.com/61.html - which supports the idea that parallel sorting algorithms shouldn't be done automatically) On Thu, Feb 11, 2010 at 12:21 PM, Osvaldo Doederlein wrote: > 2010/2/10 Stephen Colebourne > >> > So I'm fine if lambda bodies are restricted along the lines Doug >> > suggested. ("Ideally, for the sake of parallel execution, we'd disallow >> > automatic sharing of locals and automatic elevation of "this", "return" >> > or "break" to enclosing scopes.") >> >> It sounds like the tail is wagging the dog. And this could wreck the >> whole lambda change. I posit that Parallel Arrays and their ilk will >> *not* be the major usage of a suitably designed language change. Most >> developers simply don't need massively parallel type operations. >> Writing a typical web application, e-commerce app or desktop app >> doesn't benefit from Parallel Arrays - they are a specialist feature. >> >> While I understand the problem (Sun wants to add parallel arrays to >> the JDK but can't due to interface explosion and ugly inner class >> syntax), this needs to be treated as just one use case, that is >> relatively minor. >> > > #{+1}, #{+1}, #{+1}, #{+1} ? ? // Lambda agreements, in parallel on a > quad-core > > Even if Parallel Arrays _are_ a major use case, I don't think the lambda > spec should provide any level of baby-sitting for developers, and especially > not anything that creates restriction or even harder usage (requiring extra > modifiers) for other use cases. > > If a programmer creates a parallel computation with lambdas that share some > mutable variable and race on it, just let that program explode in the > programmer's face. Next release FindBugs will certainly have a new rule that > emits a warning for such code, and that's all. (If we are kind enough, maybe > javac itself will have such [optional] warning.) > > Parallel Array will be used mostly by expert programmers, and mostly for > algorithms with critical performance requirements. Unless we consider > further language sugar, e.g. auto-parallelization (accum = 0; par-for (x: > elems) {accum += x.something()} => emits both ParallelArray code and a > lambda) but this is not in the roadmap for SE7 and if we do that in the > future, it's a different design setting, javac would have much more control > (will typically generate the lambdas) so it will just generate the "right" > code. In fact, I expect this to be a realistic roadmap, if not for Java 8+ > then certainly (and much sooner anyway) for other languages like Groovy, > Scala, Clojure or Fortress. So, in the long term I see Parallel Arrays as a > "raw" low-level API that will be used directly mostly by experts / library > writers / language designers... even with lambdas. Most programmers won't > need Parallel Array within the SE7 lifecycle; and a few years down the road, > when SE8+ is available (not to mention other languages), most programmers > will need it (unless we already have graphene-based 100GHz CPUs, he he) but > then they will jump straight to higher-level syntax. > > A+ > Osvaldo > > From patrick.viry at ateji.com Thu Feb 11 10:02:34 2010 From: patrick.viry at ateji.com (Patrick Viry) Date: Thu, 11 Feb 2010 19:02:34 +0100 Subject: Parallel-safe lambdas (Stephen Colebourne) Message-ID: > > These seem to be important questions. If the "primary" parallel use > case is 99%, then as a whole, this mailing list should be designing > for safety and against data races. And then the "strawman" above > doesn't seem that unreasonable. > > +1 This is a major question indeed. *If* the major use case is to express data parallelism, then what you probably want is constructs for parallelism, not closures. *If* parallelism is an important use case, then it is important to keep in mind that parallel also often means distributed, and the last thing you'd want when distributing code is to capture and "carry away" variables rather than values. -Patrick www.ateji.com From mthornton at optrak.co.uk Thu Feb 11 10:57:28 2010 From: mthornton at optrak.co.uk (Mark Thornton) Date: Thu, 11 Feb 2010 18:57:28 +0000 Subject: Local functions In-Reply-To: <7926817e1002110944k18234bd0j157d143506a3f315@mail.gmail.com> References: <15e8b9d21002091712j5d3a0488y48459bb6d183fa09@mail.gmail.com> <1D97026B-D743-44F5-8903-23776D11689C@googlemail.com> <4B72CB45.4010701@univ-mlv.fr> <4023DE02-B32D-4A04-B70F-380A2DFB63B1@googlemail.com> <4B72D693.2000209@univ-mlv.fr> <4B73085E.7080101@sun.com> <15e8b9d21002101147u64c8bb9dhc5be3f2a3c65feb9@mail.gmail.com> <4B7334C6.2030506@sun.com> <4b4f45e01002101611i6a46310oc267f66545ffeb98@mail.gmail.com> <7926817e1002110944k18234bd0j157d143506a3f315@mail.gmail.com> Message-ID: <4B745318.8010004@optrak.co.uk> Jesse Kuhnert wrote: > That sounds completely reasonable to me .. (esp after recently reading > http://prog21.dadgum.com/61.html - which supports the idea that > parallel sorting algorithms shouldn't be done automatically) > No it supports the idea that the parallel sorting should take place somewhere else. In my work, our customers will usually tolerate a computation taking up to about 15 minutes. Beyond that they start to complain, and at an hour they complain a lot. I concede that there probably aren't too many on this list writing applications that take that long to execute. Mark From John.Rose at Sun.COM Thu Feb 11 10:57:37 2010 From: John.Rose at Sun.COM (John Rose) Date: Thu, 11 Feb 2010 10:57:37 -0800 Subject: Parallel-safe lambdas In-Reply-To: <4B73EDB2.9040106@univ-mlv.fr> References: <4b4f45e01002101823i2a1eeb06l6fc3067b7cbd5926@mail.gmail.com> <4B73798F.6000003@sun.com> <4B73D490.7010105@free.fr> <4B73D7FA.5090405@optrak.co.uk> <4B73E42F.4080402@univ-mlv.fr> <4B73E5DD.8040008@optrak.co.uk> <4B73EDB2.9040106@univ-mlv.fr> Message-ID: On Feb 11, 2010, at 3:44 AM, R?mi Forax wrote: > Perhaps the best option is to maintain the current status quo, > local variable should be final, fields can be mutated. Yes, that could work, especially with an "as if final" rule to make coding easier. The trade-off is (and always has been) between transparency hazards and MP hazards. A high-risk construct (opinions differ here!) should require an explicit programmer declaration before it becomes transparent into a nested scope (inner class and/or closure). Unrestricted, implicit sharing of mutable local variables has *always been* (IMO) a bad idea, and it certainly gets worse every year as core counts grow. So there's even less reason to make these transparent today, than there was in Java 1.1. But, an explicit @Shared annotation, for when the user is willing to promise that shared state is OK, is (IMO) a good compromise, adding a transparency hazard to defend against an MP hazard. (Maybe Doug will disagree about how to manage this trade-off?) > lambda is already a restricted form of anonymous class. > A lambda doesn't define a class, unlike anonymous class, > so mutating something is less easy. This thought made be realize that there's another design point here which needs discussion. (Skip ahead to points 0..4 below for the quick proposal.) Simple, transparent access to immutable names (fields, locals) in uplevel scopes should be a no-brainer, even given the current emphasis on MP safety. Access to uplevel mutable names could be treated as another matter. (In theory, methods needs special treatment too, since they can present MP hazards. I wish we had pure methods! But that would entail defining a pure sublanguage with its own subset type system and loopholes for interoperability. So the reality is we need to rely on "cultural" conventions to guide programmers to use methods safely when doing MP programming. The language as it stands can have only limited interaction with such conventions.) Inner classes share access to the mutable fields of their enclosing classes. It makes sense (IMO) that an inner object, since it is lexically a part of the outer object, has the same easy, unqualified access to outer fields as the outer object itself. Programmers who use objects know that mutable fields are a risk to parallelism. The programming culture deals adequately with mutable fields, for now. (Still, we should invent pure objects and add them to Java someday, since hardware is changing. Can't do that today.) Now, the unqualified sharing of fields, enjoyed by inner classes, does not necessarily need to be true for closures. Just as with @Shared, we could add an intentional transparency hazard whose resolution requires extra explicitness, for access to mutable object fields. We could (and this is just another tweak to the "syntax dial", really) require that references to mutable fields of enclosing objects must be qualified with a suitable 'this'. (That name, as many have pointed out, is not mutable.) A safe and still useful tradeoff regarding uplevel members (fields, methods) could be this: 0. no restrictions on qualified names 1. allow unqualified references to immutable variables of all sorts ('unqualified' = 'simple name, no dots') ('immutable' = 'final, or a local implicitly markable as final') 2. allow unqualified references to uplevel variables (all sorts) if they are marked @Shared 3. allow unqualified references to uplevel methods if they are marked @Shared 4. allow unqualified references to uplevel methods if they are "pure" (marked @Pure) Point 0 lets the user of an API get to it always, by using the dot '.' syntax. No change to selection expression semantics. Point 1 lets demonstrably "safe" names be fully transparent. Points 2,3,4 let the definer of an API (who can make general guarantees) contract with the user about MP safety, and grant easy access. When coding, put @Shared on things (locals, fields, methods) you really mean to share (potentially) across execution contexts. Point 4 may not be motivated by present-day use cases; I include it as a "taste" of what the pure sublanguage might feel like. Pure things are a subset of things which are meant to be shared. ...A most interesting subset. The theory could be as follows: Inner objects "inherit" complete access to the members of their enclosing objects. They can handle this because they are objects, and can easily incorporate synchronization patterns. (Although there is the well-known, and recently emphasized, problem of synchronizing on the "wrong this", a direct result of the decision to give every object, no matter how small, a monitor.) But closures are not (not necessarily, and should not be IMO) first-class objects, which assume the whole setup of private methods, fields, 'this', etc. So a closure does not necessarily have "full rights" to the fields of enclosing objects: We can grant such rights based on the hazard trade-off mentioned above. A lot of this is trying to guess Doug Lea's mind on what he perceives (based on deep experience) leads to racy code. Doug, does any of this help with those concerns? -- John From mthornton at optrak.co.uk Thu Feb 11 11:42:10 2010 From: mthornton at optrak.co.uk (Mark Thornton) Date: Thu, 11 Feb 2010 19:42:10 +0000 Subject: Parallel-safe lambdas In-Reply-To: References: <4b4f45e01002101823i2a1eeb06l6fc3067b7cbd5926@mail.gmail.com> <4B73798F.6000003@sun.com> <4B73D490.7010105@free.fr> <4B73D7FA.5090405@optrak.co.uk> <4B73E42F.4080402@univ-mlv.fr> <4B73E5DD.8040008@optrak.co.uk> <4B73EDB2.9040106@univ-mlv.fr> Message-ID: <4B745D92.9040608@optrak.co.uk> John Rose wrote: > 4. allow unqualified references to uplevel methods if they are "pure" (marked @Pure) > A colleague recently suggested that a parameter to one of my methods needed to be marked as @Pure. The parameter was a single method interface, where the method would be called an indeterminate number of times and must return the same value on each occasion without side effects. The usage in this case wasn't necessarily concurrent. Mark From Alex.Buckley at Sun.COM Thu Feb 11 12:01:49 2010 From: Alex.Buckley at Sun.COM (Alex Buckley) Date: Thu, 11 Feb 2010 12:01:49 -0800 Subject: Parallel-safe lambdas In-Reply-To: <4B745D92.9040608@optrak.co.uk> References: <4b4f45e01002101823i2a1eeb06l6fc3067b7cbd5926@mail.gmail.com> <4B73798F.6000003@sun.com> <4B73D490.7010105@free.fr> <4B73D7FA.5090405@optrak.co.uk> <4B73E42F.4080402@univ-mlv.fr> <4B73E5DD.8040008@optrak.co.uk> <4B73EDB2.9040106@univ-mlv.fr> <4B745D92.9040608@optrak.co.uk> Message-ID: <4B74622D.9060309@sun.com> Mark Thornton wrote: > John Rose wrote: >> 4. allow unqualified references to uplevel methods if they are "pure" (marked @Pure) >> > A colleague recently suggested that a parameter to one of my methods > needed to be marked as @Pure. The parameter was a single method > interface, where the method would be called an indeterminate number of > times and must return the same value on each occasion without side > effects. The usage in this case wasn't necessarily concurrent. You really want to mark it pure AND idempotent, so the runtime can aggressively inline it. You'd be set for parallel execution, for free. This is not the only dimension in which methods are under-characterized in Java, see "Fuzzy Access Modifiers" in http://wiki.apidesign.org/wiki/ClarityOfAccessModifiers (e.g. "public final" == "callable", "protected final" == "callback"). Alex From dl at cs.oswego.edu Thu Feb 11 12:06:27 2010 From: dl at cs.oswego.edu (Doug Lea) Date: Thu, 11 Feb 2010 15:06:27 -0500 Subject: Parallel-safe lambdas In-Reply-To: References: <4b4f45e01002101823i2a1eeb06l6fc3067b7cbd5926@mail.gmail.com> <4B73798F.6000003@sun.com> <4B73D490.7010105@free.fr> <4B73D7FA.5090405@optrak.co.uk> <4B73E42F.4080402@univ-mlv.fr> <4B73E5DD.8040008@optrak.co.uk> <4B73EDB2.9040106@univ-mlv.fr> Message-ID: <4B746343.8060509@cs.oswego.edu> First some meta-chat: As I stated a few weeks ago, one of the main thrusts in parallelism these days to make constructions that are correct and efficient easier to express. Which implies making incorrect and/or inefficient constructions impossible, or more difficult, or at least no easier to express than today. Unfortunately, correctness-enhancing language rules are harder to arrive at for parallel programs than sequential programs. It would be great if we had better analogs of static type-safety, definite assignment, etc that Java does so well to improve quality of sequential code. Lacking very many of these, we can focus on constructions that are conditionally OK. As in calling map, reduce etc with "well-behaved" lambdas, which allows focusing on what makes them well-behaved, and then making these cases the easiest to express. John's sketch of... > A safe and still useful tradeoff regarding uplevel members (fields, methods) could be this: > 0. no restrictions on qualified names > 1. allow unqualified references to immutable variables of all sorts > ('unqualified' = 'simple name, no dots') > ('immutable' = 'final, or a local implicitly markable as final') > 2. allow unqualified references to uplevel variables (all sorts) if they are marked @Shared > 3. allow unqualified references to uplevel methods if they are marked @Shared > 4. allow unqualified references to uplevel methods if they are "pure" (marked @Pure) > ... sounds like it is on the right track to me. > A lot of this is trying to guess Doug Lea's mind on what he perceives (based on deep experience) leads to racy code. Doug, does any of this help with those concerns? > (John Rose has been dealing with these issues a long time too. Several of them were first seen in "C*" -- http://en.wikipedia.org/wiki/C*) Last, more meta-chat: I honestly do not have an opinion about whether exploring lambdas with parallel application as use case #1 is the best plan for evolving Java or even the best plan for exploring closure-like constructions. It may well be the case that despite these efforts, other languages running on JVMs will be more sensible choices as everyday ubiquitous parallelism becomes common. But it does seem worthwhile to make the very best attempt possible. -Doug From Alex.Buckley at Sun.COM Thu Feb 11 15:06:22 2010 From: Alex.Buckley at Sun.COM (Alex Buckley) Date: Thu, 11 Feb 2010 15:06:22 -0800 Subject: Function types versus arrays In-Reply-To: <15e8b9d21002071041j57e926e9k8e7c36e01f80ee46@mail.gmail.com> References: <15e8b9d21002071041j57e926e9k8e7c36e01f80ee46@mail.gmail.com> Message-ID: <4B748D6E.6090605@sun.com> Neal Gafter wrote: > I've been asked if arrays and function types will play nicely > together. Specifically, for example, whether or not something like > the following will be allowed: > > #String(String)[] arrayOfFunction = new #String(String)[10]; > > The current draft spec, by the absence of any constraining rules, > implies yes. Finishing up here for now, the 0.1 draft spec doesn't fully imply "yes". - It proposes no changes to JLS3 15.10, so an array creation expression syntactically cannot have a function type as the element type. - It proposes no changes to JLS3 10.6, so an array initializer for a function-type array is possible iff function types are reifiable. For now, they are not reifiable, and I will tweak JLS3 4.7 to exclude them. - It implicitly allows, and will continue to allow, an array type whose element type is a function type. Alex From neal at gafter.com Thu Feb 11 15:26:59 2010 From: neal at gafter.com (Neal Gafter) Date: Thu, 11 Feb 2010 15:26:59 -0800 Subject: Function types versus arrays In-Reply-To: <4B748D6E.6090605@sun.com> References: <15e8b9d21002071041j57e926e9k8e7c36e01f80ee46@mail.gmail.com> <4B748D6E.6090605@sun.com> Message-ID: <15e8b9d21002111526k262c0f3fid13e7395d78fffab@mail.gmail.com> On Thu, Feb 11, 2010 at 3:06 PM, Alex Buckley wrote: >> The current draft spec, by the absence of any constraining rules, >> implies yes. > > Finishing up here for now, the 0.1 draft spec doesn't fully imply "yes". > > - It implicitly allows, and will continue to allow, an array type whose > element type is a function type. Given that there is no way to create a value of such a type (or a value of a subtype of such a type), other than null, what value is there in allowing it? Why would you commit yourself now to that position for future revisions of the spec? From peter.levart at marand.si Thu Feb 11 23:39:26 2010 From: peter.levart at marand.si (Peter Levart) Date: Fri, 12 Feb 2010 08:39:26 +0100 Subject: Function types versus arrays In-Reply-To: <15e8b9d21002111526k262c0f3fid13e7395d78fffab@mail.gmail.com> References: <15e8b9d21002071041j57e926e9k8e7c36e01f80ee46@mail.gmail.com> <4B748D6E.6090605@sun.com> <15e8b9d21002111526k262c0f3fid13e7395d78fffab@mail.gmail.com> Message-ID: <201002120839.26695.peter.levart@marand.si> On Friday 12 February 2010 00:26:59 Neal Gafter wrote: > On Thu, Feb 11, 2010 at 3:06 PM, Alex Buckley wrote: > >> The current draft spec, by the absence of any constraining rules, > >> implies yes. > > > > Finishing up here for now, the 0.1 draft spec doesn't fully imply "yes". > > > > - It implicitly allows, and will continue to allow, an array type whose > > element type is a function type. > > Given that there is no way to create a value of such a type (or a > value of a subtype of such a type), other than null, what value is > there in allowing it? Why would you commit yourself now to that > position for future revisions of the spec? > > In analogy with parameterized types I propose a notation for "raw" function types. For example: // produces unchecked assignment warning #String(String,String)[] a1 = new #?(?,?)[] { #(String s1, String s2)(s1+"_"+s2) }; Primitive returns/parameters can be mixed into "raw" function type to produce a "mixed raw/primitive" function type: // produces unchecked assignment warning #int(String)[] a2 = new #int(?)[] { #(String s)(s.length()) }; What to do with throws types? Treat them the same as a return type? // produces unchecked assignment warning #String(File)(throws IOException)[] a3 = new #?(?)(throws ?)[] { }; With this scheme, there is a straightforward mapping to parameterized interfaces. Regards, Peter From peter.levart at marand.si Fri Feb 12 00:45:09 2010 From: peter.levart at marand.si (Peter Levart) Date: Fri, 12 Feb 2010 09:45:09 +0100 Subject: Function types versus arrays In-Reply-To: <201002120839.26695.peter.levart@marand.si> References: <15e8b9d21002071041j57e926e9k8e7c36e01f80ee46@mail.gmail.com> <15e8b9d21002111526k262c0f3fid13e7395d78fffab@mail.gmail.com> <201002120839.26695.peter.levart@marand.si> Message-ID: <201002120945.09137.peter.levart@marand.si> On Friday 12 February 2010 08:39:26 Peter Levart wrote: > In analogy with parameterized types I propose a notation for "raw" function types. For example: > > // produces unchecked assignment warning > #String(String,String)[] a1 = new #?(?,?)[] { #(String s1, String s2)(s1+"_"+s2) }; > > Primitive returns/parameters can be mixed into "raw" function type to produce a "mixed raw/primitive" function type: > > // produces unchecked assignment warning > #int(String)[] a2 = new #int(?)[] { #(String s)(s.length()) }; > > What to do with throws types? Treat them the same as a return type? > > // produces unchecked assignment warning > #String(File)(throws IOException)[] a3 = new #?(?)(throws ?)[] { }; > Or better (since '?' already has a different meaning in parameterized types) the following: #String(String,String)[] a1 = new #*(*,*)[] { #(String s1, String s2)(s1+"_"+s2) }; #int(String)[] a2 = new #int(*)[] { #(String s)(s.length()) }; #String(File)(throws IOException)[] a3 = new #*(*)(throws *)[] { }; Regards, Peter From forax at univ-mlv.fr Fri Feb 12 01:00:01 2010 From: forax at univ-mlv.fr (=?ISO-8859-1?Q?R=E9mi_Forax?=) Date: Fri, 12 Feb 2010 10:00:01 +0100 Subject: Function types versus arrays In-Reply-To: <15e8b9d21002111526k262c0f3fid13e7395d78fffab@mail.gmail.com> References: <15e8b9d21002071041j57e926e9k8e7c36e01f80ee46@mail.gmail.com> <4B748D6E.6090605@sun.com> <15e8b9d21002111526k262c0f3fid13e7395d78fffab@mail.gmail.com> Message-ID: <4B751891.7050408@univ-mlv.fr> Le 12/02/2010 00:26, Neal Gafter a ?crit : > On Thu, Feb 11, 2010 at 3:06 PM, Alex Buckley wrote: > >>> The current draft spec, by the absence of any constraining rules, >>> implies yes. >>> >> Finishing up here for now, the 0.1 draft spec doesn't fully imply "yes". >> >> - It implicitly allows, and will continue to allow, an array type whose >> element type is a function type. >> > Given that there is no way to create a value of such a type (or a > value of a subtype of such a type), other than null, what value is > there in allowing it? Why would you commit yourself now to that > position for future revisions of the spec? > Not true. java.dyn.MethodHandle can be that type. R?mi From neal at gafter.com Fri Feb 12 08:32:16 2010 From: neal at gafter.com (Neal Gafter) Date: Fri, 12 Feb 2010 08:32:16 -0800 Subject: Function types versus arrays In-Reply-To: <201002120945.09137.peter.levart@marand.si> References: <15e8b9d21002071041j57e926e9k8e7c36e01f80ee46@mail.gmail.com> <15e8b9d21002111526k262c0f3fid13e7395d78fffab@mail.gmail.com> <201002120839.26695.peter.levart@marand.si> <201002120945.09137.peter.levart@marand.si> Message-ID: <15e8b9d21002120832y1e4bcc75xb9542a5384a10c5@mail.gmail.com> On Fri, Feb 12, 2010 at 12:45 AM, Peter Levart wrote: > Or better (since '?' already has a different meaning in parameterized types) the following: Actually, ? has the same meaning in parameterized types. Cheers, Neal From neal at gafter.com Fri Feb 12 08:33:19 2010 From: neal at gafter.com (Neal Gafter) Date: Fri, 12 Feb 2010 08:33:19 -0800 Subject: Function types versus arrays In-Reply-To: <4B751891.7050408@univ-mlv.fr> References: <15e8b9d21002071041j57e926e9k8e7c36e01f80ee46@mail.gmail.com> <4B748D6E.6090605@sun.com> <15e8b9d21002111526k262c0f3fid13e7395d78fffab@mail.gmail.com> <4B751891.7050408@univ-mlv.fr> Message-ID: <15e8b9d21002120833j645f79cbn599d51496cb98552@mail.gmail.com> On Fri, Feb 12, 2010 at 1:00 AM, R?mi Forax wrote: >> Given that there is no way to create a value of such a type (or a >> value of a subtype of such a type), other than null, what value is >> there in allowing it? ?Why would you commit yourself now to that >> position for future revisions of the spec? >> > > Not true. > java.dyn.MethodHandle can be that type. MethodHandle can be a subtype of every function type? I think you mean supertype, not subtype? From forax at univ-mlv.fr Fri Feb 12 08:52:45 2010 From: forax at univ-mlv.fr (=?ISO-8859-1?Q?R=E9mi_Forax?=) Date: Fri, 12 Feb 2010 17:52:45 +0100 Subject: Function types versus arrays In-Reply-To: <15e8b9d21002120833j645f79cbn599d51496cb98552@mail.gmail.com> References: <15e8b9d21002071041j57e926e9k8e7c36e01f80ee46@mail.gmail.com> <4B748D6E.6090605@sun.com> <15e8b9d21002111526k262c0f3fid13e7395d78fffab@mail.gmail.com> <4B751891.7050408@univ-mlv.fr> <15e8b9d21002120833j645f79cbn599d51496cb98552@mail.gmail.com> Message-ID: <4B75875D.9020204@univ-mlv.fr> Le 12/02/2010 17:33, Neal Gafter a ?crit : > On Fri, Feb 12, 2010 at 1:00 AM, R?mi Forax wrote: > >>> Given that there is no way to create a value of such a type (or a >>> value of a subtype of such a type), other than null, what value is >>> there in allowing it? Why would you commit yourself now to that >>> position for future revisions of the spec? >>> >>> >> Not true. >> java.dyn.MethodHandle can be that type. >> > MethodHandle can be a subtype of every function type? I think you > mean supertype, not subtype? > Yes, MethodHandle is a supertype of all function type. R?mi From Alex.Buckley at Sun.COM Fri Feb 12 14:20:43 2010 From: Alex.Buckley at Sun.COM (Alex Buckley) Date: Fri, 12 Feb 2010 14:20:43 -0800 Subject: Function types versus arrays In-Reply-To: <4B75875D.9020204@univ-mlv.fr> References: <15e8b9d21002071041j57e926e9k8e7c36e01f80ee46@mail.gmail.com> <4B748D6E.6090605@sun.com> <15e8b9d21002111526k262c0f3fid13e7395d78fffab@mail.gmail.com> <4B751891.7050408@univ-mlv.fr> <15e8b9d21002120833j645f79cbn599d51496cb98552@mail.gmail.com> <4B75875D.9020204@univ-mlv.fr> Message-ID: <4B75D43B.6050304@sun.com> R?mi Forax wrote: > Le 12/02/2010 17:33, Neal Gafter a ?crit : >> MethodHandle can be a subtype of every function type? I think you >> mean supertype, not subtype? > > Yes, MethodHandle is a supertype of all function type. OK ... and then, MethodHandle objects can be safely stored in a function-typed array because of your translation scheme: > In that case, the following code is legal: > #int(int)[] array = (#int(int)[])new MethodHandle[10]; > but the cast will raise a warning. > Because function type are translated at compile time > to MethodHandle, no ClassCastException at runtime. But can you show me how an array of MethodHandles performs dynamic typechecking at array store? Alex From neal at gafter.com Fri Feb 12 14:41:26 2010 From: neal at gafter.com (Neal Gafter) Date: Fri, 12 Feb 2010 14:41:26 -0800 Subject: Function types versus arrays In-Reply-To: <4B75D43B.6050304@sun.com> References: <15e8b9d21002071041j57e926e9k8e7c36e01f80ee46@mail.gmail.com> <4B748D6E.6090605@sun.com> <15e8b9d21002111526k262c0f3fid13e7395d78fffab@mail.gmail.com> <4B751891.7050408@univ-mlv.fr> <15e8b9d21002120833j645f79cbn599d51496cb98552@mail.gmail.com> <4B75875D.9020204@univ-mlv.fr> <4B75D43B.6050304@sun.com> Message-ID: <15e8b9d21002121441l78a2447fw6126eaa7b80e33f6@mail.gmail.com> On Fri, Feb 12, 2010 at 2:20 PM, Alex Buckley wrote: > But can you show me how an array of MethodHandles performs dynamic > typechecking at array store? I think the story is that the dynamic checking happens when a MethodHandle is invoked. From John.Rose at Sun.COM Fri Feb 12 14:46:17 2010 From: John.Rose at Sun.COM (John Rose) Date: Fri, 12 Feb 2010 14:46:17 -0800 Subject: Function types versus arrays In-Reply-To: <4B75D43B.6050304@sun.com> References: <15e8b9d21002071041j57e926e9k8e7c36e01f80ee46@mail.gmail.com> <4B748D6E.6090605@sun.com> <15e8b9d21002111526k262c0f3fid13e7395d78fffab@mail.gmail.com> <4B751891.7050408@univ-mlv.fr> <15e8b9d21002120833j645f79cbn599d51496cb98552@mail.gmail.com> <4B75875D.9020204@univ-mlv.fr> <4B75D43B.6050304@sun.com> Message-ID: On Feb 12, 2010, at 2:20 PM, Alex Buckley wrote: > But can you show me how an array of MethodHandles performs dynamic > typechecking at array store? Yes, that's the problem; there's no way at present to tag a MethodHandle[] array with the extra typing information (the static information which was erased). The root problem is that, while it is easy to reify and runtime-check the static function type (say, as a java.dyn.MethodType), it is an unsolved problem to put such extra type info on an array. (It can be done on a list with a wrapper, since lists are fully abstract. But arrays are not.) The best way I know of to tag function arrays is to define function arrays as pseudo-arrays which work at runtime like a Collections.checkedList but look at the source level like arrays. The most surprising wrinkle of such a design, IMO, is that they would fail to be subtypes of Object[], much like int[] fails today. -- John From Alex.Buckley at Sun.COM Fri Feb 12 15:06:15 2010 From: Alex.Buckley at Sun.COM (Alex Buckley) Date: Fri, 12 Feb 2010 15:06:15 -0800 Subject: Function types versus arrays In-Reply-To: References: <15e8b9d21002071041j57e926e9k8e7c36e01f80ee46@mail.gmail.com> <4B748D6E.6090605@sun.com> <15e8b9d21002111526k262c0f3fid13e7395d78fffab@mail.gmail.com> <4B751891.7050408@univ-mlv.fr> <15e8b9d21002120833j645f79cbn599d51496cb98552@mail.gmail.com> <4B75875D.9020204@univ-mlv.fr> <4B75D43B.6050304@sun.com> Message-ID: <4B75DEE7.1050209@sun.com> John Rose wrote: > The best way I know of to tag function arrays is to define function > arrays as pseudo-arrays which work at runtime like a > Collections.checkedList but look at the source level like arrays. > The most surprising wrinkle of such a design, IMO, is that they would > fail to be subtypes of Object[], much like int[] fails today. This compilation strategy would also enable arrays of parameterized types, and generally open the door to wonderful new type constructors that are useful in source and whose representation is known to the JIT. But unless we also change the compilation strategy for storing to an Object[], the universal supertype will bite us every time. I feel like we're peering over the edge of a very tall cliff. Alex From John.Rose at Sun.COM Fri Feb 12 15:18:33 2010 From: John.Rose at Sun.COM (John Rose) Date: Fri, 12 Feb 2010 15:18:33 -0800 Subject: Function types versus arrays In-Reply-To: <4B75DEE7.1050209@sun.com> References: <15e8b9d21002071041j57e926e9k8e7c36e01f80ee46@mail.gmail.com> <4B748D6E.6090605@sun.com> <15e8b9d21002111526k262c0f3fid13e7395d78fffab@mail.gmail.com> <4B751891.7050408@univ-mlv.fr> <15e8b9d21002120833j645f79cbn599d51496cb98552@mail.gmail.com> <4B75875D.9020204@univ-mlv.fr> <4B75D43B.6050304@sun.com> <4B75DEE7.1050209@sun.com> Message-ID: On Feb 12, 2010, at 3:06 PM, Alex Buckley wrote: > John Rose wrote: >> The best way I know of to tag function arrays is to define function >> arrays as pseudo-arrays which work at runtime like a >> Collections.checkedList but look at the source level like arrays. >> The most surprising wrinkle of such a design, IMO, is that they would >> fail to be subtypes of Object[], much like int[] fails today. > > This compilation strategy would also enable arrays of parameterized > types, and generally open the door to wonderful new type constructors > that are useful in source and whose representation is known to the JIT. > But unless we also change the compilation strategy for storing to an > Object[], the universal supertype will bite us every time. I feel like > we're peering over the edge of a very tall cliff. Maybe there is a way to back up one step from that cliff, though it might be off a different cliff (the erasure cliff). How about this: A function type is not a reference type. Types are comprised of 1. reference types, 2. primitive types, 3. function types. (In the bytecodes, function types can still be *represented* by references, of course.) Define one reference type FT (e.g., MethodHandle) as representing all function types in the reference type hierarchy, via conversion. FT will serve as a an erased or raw holder type for all function types. Add an implicit assignment conversion from any function type to FT. (Cf. List to List.) Add an explicit casting conversion from FT to any function type. It does a runtime check against some attribute query (e.g., MethodHandle.type). After the smoke clears, we find that arrays of function types are not related to the type FT[] or Object[]. Also, arrays of function types can be pseudo-arrays. We would still have a problem (which we did anyway) with representing function types within field and method signatures. In erased form, they would look simply like FT. -- John From Alex.Buckley at Sun.COM Fri Feb 12 17:21:38 2010 From: Alex.Buckley at Sun.COM (Alex Buckley) Date: Fri, 12 Feb 2010 17:21:38 -0800 Subject: Project Lambda: Java Language Specification draft 0.1.5 Message-ID: <4B75FEA2.9080904@sun.com> I was hoping to reach enough conclusions to justify sending a 0.2 draft, but the ongoing discussions on function-typed arrays and variable capture mean we are not there yet. For now, here's 0.1.5; a changelog is near the top. An increasing number of discussion points in the text refer to lambda-dev threads; this is a good thing. Alex -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: lambda-jls-0.1.5.txt Url: http://mail.openjdk.java.net/pipermail/lambda-dev/attachments/20100212/af8d2cc5/attachment-0001.txt From neal at gafter.com Fri Feb 12 19:04:46 2010 From: neal at gafter.com (Neal Gafter) Date: Fri, 12 Feb 2010 19:04:46 -0800 Subject: Project Lambda: Java Language Specification draft 0.1.5 In-Reply-To: <4B75FEA2.9080904@sun.com> References: <4B75FEA2.9080904@sun.com> Message-ID: <15e8b9d21002121904u55b47769w9449c113709dc54a@mail.gmail.com> A few flubs and omissions that I noticed: - Your grammar is ambiguous. Expression goes to both LambdaExpression and (indirectly) to Primary, both of which have a way of writing a lambda. - It is unclear what the type of "this" is in an expression lambda. Is it forbidden? - I wonder if the identity of a lambda can change spuriously. For example, if "this" refers to the lambda expression, can it be expected to have the same value on each invocation of the same lambda expression? In other words, #Object() self = #()this; assert self.() == self.(); // guaranteed?? - Similar question if the lambda has been converted to a SAM Object() self1 = #()this; Callable self2 = self1; assert self1.() == self2.call(); // guaranteed?? - What is the meaning of "toString()" inside a lambda? Does it refer to the method inherited by the lambda from Object, or from the enclosing scope? - It isn't clear why you would want special rules for recursive lambdas in this revision of the spec, given that one could just invoke "this" inside the lambda. Or, failing that, just use a method or an anonymous inner class. - In your example where there is an "unreachable return statement", the return statement is (formally) reachable. Any unreachable return statement would be a compile-time error because it is an error to have an unreachable statement in Java. - Your definition of effectively final mentions "target of an initialization", but that doesn't make sense. It can only be the target of an initialization in its declaration, and it is always definitely unassigned before then. (Yes, even local variables in switch statements) - The spec "It is a compile-time error to modify the value of an effectively-final variable in the body of a lambda expression." is meaningless. If there were an assignment to a variable from an enclosing scope in a lambda, the variable would not be definitely unassigned at that point, and so by definition the variable would not be effectively-final. So this rule could never be applied in the way you appear to intend. The only way the rule could be applied is definitely something you do not want to forbid: Object o = #(){ int x; // x is effectively final x = 3; /* error: It is a compile-time error to modify the value of an effectively-final variable in the body of a lambda expression */ }; - Your example "The constraint #int(String) >> #int(T) resolves to String>>T" in incorrect. It resolves to T>>String. - Your inference algorithm doesn't allow the use of the lambda conversion. Specifically, if you try to pass #()("Foo") to a method void f(Callable); there is no way to infer that T=String. - Your assignment conversion doesn't allow a subtype conversion followed by a lambda conversion. Consequently, you may force programmers to use temporary variables just to force the compiler to do a subtype conversion before the lambda conversion. In expression contexts (i.e. super() invocations) that is cumbersome indeed. - Your rules for lambda conversion appear to improperly erase some types. For example, the following code is ruled illegal interface Receiver { void f(T); } Receiver> rls = #(List ls)(); // error: "List" is not the same as "List". - The specification for a SAM type says that "If a SAM type is a class type, then it must be declared abstract and have an accessible no-args constructor.". That makes the following (currently legal code) illegal: abstract class SAM { private SAM(int arg){} abstract void f(); } // error: SAM type does not have no-args constructor - There are no accessibility constraints on the methods of a SAM. What if the abstract method is package-private to someone else's package? Or, what if there are two override-equivalent abstract methods that are package-private to different packages? Generating correct code for (a lambda conversion in) this case is extremely cumbersome, and may require generating three or more classes in different packages. - The following is a SAM (it has only one non-generic abstract method), but should not be considered a SAM interface SAM { void foo(); void bar(int x); } - Your specification [11.2.2] makes no sense. You have the exceptions from the body of a statement lambda being thrown where the lambda expression appears. A lambda expression should throw no checked exceptions. - You don't describe the cases in which a lambda conversion can throw a checked exception. For example, what if the SAM constructor declared an exception? - Your handling of exceptions in the lambda conversion is not correct. For example interface I1 { void f() throws X1; } interface I2 { void f() throws X2; } interface I3 : I1, I2 {} I3 x = #(){ throw new X1(); } // allowed, even though I2.f() doesn't throw X1. - You say "let M be the set of methods..." but you mean "let S be the set of abstract methods...". Cheers, Neal From reinier at zwitserloot.com Fri Feb 12 20:22:15 2010 From: reinier at zwitserloot.com (Reinier Zwitserloot) Date: Sat, 13 Feb 2010 05:22:15 +0100 Subject: Function types versus arrays In-Reply-To: <3dd3f56a1002082201l304a5473ge2859fcc52032913@mail.gmail.com> References: <3dd3f56a1002081452o36d2b695kc766842ec2e8348b@mail.gmail.com> <15e8b9d21002081537p5ed0b3ebs21c06a8decdced74@mail.gmail.com> <3dd3f56a1002081733x786e329at5d2f273dcf3cec69@mail.gmail.com> <15e8b9d21002081754p428832e5mff96cc9586c6a47b@mail.gmail.com> <3dd3f56a1002081944p6d6e0119g181fb968d7da747a@mail.gmail.com> <15e8b9d21002082129i21a209cel925ad8b6ba3313cd@mail.gmail.com> <3dd3f56a1002082201l304a5473ge2859fcc52032913@mail.gmail.com> Message-ID: <560fb5ed1002122022u27f0ed29x44d7f821ddb44ab5@mail.gmail.com> On Tue, Feb 9, 2010 at 7:01 AM, Howard Lovatt wrote: > Well you *speculate* that reified lambdas are type unsound, you haven't and > nor has anyone else demonstrated that - in fact quite the opposite. > Well, you *speculate* that disallowing arrays of functional types is 'just not java', but you haven't demonstrated that. Arrays, really? They're almost deprecated at this point. As long as function types are legal in generics, and any usage in arrays is flagged with a clear error, I doubt the greater java community is going to get out the tar and feathers. However, if function types aren't reified, then arrays are the least of my worries. Does that mean something like this: void print(#(Number)String formatter, Number number) { System.out.println(formatter.(number)); } void print(#(String)String formatter, String string) { System.out.println(formatter.(string)); } is not legal (if this is done like generics, the signatures are equal post-erasure, which is a compile error). --Reinier Zwitserloot From reinier at zwitserloot.com Fri Feb 12 20:43:49 2010 From: reinier at zwitserloot.com (Reinier Zwitserloot) Date: Sat, 13 Feb 2010 05:43:49 +0100 Subject: Function types versus arrays In-Reply-To: <4B6FE4D4.2040208@univ-mlv.fr> References: <15e8b9d21002071041j57e926e9k8e7c36e01f80ee46@mail.gmail.com> <4B6FE4D4.2040208@univ-mlv.fr> Message-ID: <560fb5ed1002122043i7a3a2b27m115d1acb49e49f5@mail.gmail.com> On Mon, Feb 8, 2010 at 11:17 AM, R?mi Forax wrote: > I'm also happy with List, perhaps we should introduce a new interface > ReadOnlyList > which is a super type of List and array of objects but that another story. > > Nope. Can't be done. It would mean that List (and therefore, ArrayList) is an 'instanceof' ReadOnlyList. Which sounds like a pretty bad plan. The only way that's halfway feasible and does not result in instanceos of "ReadOnlyList" not necessarily being readonly that I managed in thought experiments is to create a new supertype of List which implies neither read-onlyness nor mutability. This supertype then has 2 children, and as a convention one should never implement this supertype directly, only one of its two children (or subtypes thereof). One child is java.util.List (which will more or less imply mutability), the other is java.util.ImmutableList. The supertype, as it does not imply editability, should not include the write-only methods. Of course, java.util.Collection is also around, and has both add() and clear(), so it too needs splitting up like this. Naming it is very hard as almost all names have already been taken. The only usable word left in my thesaurus is "Roll", which isn't exactly the most logical choice. Another option is to retire java.util. for collections and move them to java.lang or create a new package for them. Then, java.util.List can be deprecated, and the inheritance model can become: java.lang.List (top-level, does not imply nor has the mutable methods, but it doesn't imply immutability either). java.lang.ImmutableList extends java.lang.List java.util.List extends java.lang.List (deprecated - don't use) java.lang.MutableList extends java.util.List (just wraps java.util.List. Use this one when making mutable lists). and ArrayList, Vector, etcetera are modified to implement MutableList instead. On the surface this seems to be backwards compatible, but I haven't delved very deep into the repercussions of introducing this, as now (specifically: the java7 release) does not seem to be the right time. The above does have the advantage of allowing one to define List classes that implement java.util.List but not java.lang.MutableList as being ambiguous in whether or not they are supposed to be mutable or not. That would fit the notion of practical backwards compatibility: There are tons of java.util.Lists out there today that are in fact immutable. > > Cheers, > > Neal > > > > R?mi > > From reinier at zwitserloot.com Fri Feb 12 20:51:20 2010 From: reinier at zwitserloot.com (Reinier Zwitserloot) Date: Sat, 13 Feb 2010 05:51:20 +0100 Subject: Function types versus arrays In-Reply-To: <4B7075ED.4040400@adres.pl> References: <15e8b9d21002071041j57e926e9k8e7c36e01f80ee46@mail.gmail.com> <4B6FE4D4.2040208@univ-mlv.fr> <4B7075ED.4040400@adres.pl> Message-ID: <560fb5ed1002122051wa4ba1b5hfc14bcf523328d99@mail.gmail.com> reply at end of message. On Mon, Feb 8, 2010 at 9:37 PM, Artur Biesiadowski wrote: > What about having auto-generated interfaces (abstract classes?) > representing lambdas in well known place? Something like java.gen.*, > > On Mon, Feb 8, 2010 at 11:47 PM, Joshua Bloch wrote: > I just assumed that we'd do whatever it takes to make this works, up to and > including VM changes ... And they may rightly accuse us of doing what was > easy > to implement rather than what is easy to use. > > Doesn't the very existence of generics prevent either of these approaches? Something like: public String format(#(T)String formatter, T object) { return formatter.(object); } has a type in it: #(T)String, which cannot be turned into a java.gen.* concept, nor can a more extensive VM-level integration give us reification of function types, without generics itself being reified, which isn't on the table for JDK7. I agree with Josh's sentiments, except for one detail: We can already give up on the idea, because it can't be done with reifying generics. From neal at gafter.com Fri Feb 12 22:39:30 2010 From: neal at gafter.com (Neal Gafter) Date: Fri, 12 Feb 2010 22:39:30 -0800 Subject: Function types versus arrays In-Reply-To: <560fb5ed1002122022u27f0ed29x44d7f821ddb44ab5@mail.gmail.com> References: <3dd3f56a1002081452o36d2b695kc766842ec2e8348b@mail.gmail.com> <15e8b9d21002081537p5ed0b3ebs21c06a8decdced74@mail.gmail.com> <3dd3f56a1002081733x786e329at5d2f273dcf3cec69@mail.gmail.com> <15e8b9d21002081754p428832e5mff96cc9586c6a47b@mail.gmail.com> <3dd3f56a1002081944p6d6e0119g181fb968d7da747a@mail.gmail.com> <15e8b9d21002082129i21a209cel925ad8b6ba3313cd@mail.gmail.com> <3dd3f56a1002082201l304a5473ge2859fcc52032913@mail.gmail.com> <560fb5ed1002122022u27f0ed29x44d7f821ddb44ab5@mail.gmail.com> Message-ID: <15e8b9d21002122239p158cf4d3vfc45faaa2b48dced@mail.gmail.com> On Fri, Feb 12, 2010 at 8:22 PM, Reinier Zwitserloot wrote: > However, if function types aren't reified, then arrays are the least of my > worries. Does that mean something like this: > void print(#(Number)String formatter, Number number) { > ?? ?System.out.println(formatter.(number)); > } > void print(#(String)String formatter, String string) { > ?? ?System.out.println(formatter.(string)); > } > is not legal (if this is done like generics, the signatures are equal > post-erasure, which is a compile error). These signatures are distinct post-erasure, because the erasure of their second parameters are distinct. From reinier at zwitserloot.com Sat Feb 13 00:03:01 2010 From: reinier at zwitserloot.com (Reinier Zwitserloot) Date: Sat, 13 Feb 2010 09:03:01 +0100 Subject: Function types versus arrays In-Reply-To: <15e8b9d21002122239p158cf4d3vfc45faaa2b48dced@mail.gmail.com> References: <3dd3f56a1002081452o36d2b695kc766842ec2e8348b@mail.gmail.com> <15e8b9d21002081537p5ed0b3ebs21c06a8decdced74@mail.gmail.com> <3dd3f56a1002081733x786e329at5d2f273dcf3cec69@mail.gmail.com> <15e8b9d21002081754p428832e5mff96cc9586c6a47b@mail.gmail.com> <3dd3f56a1002081944p6d6e0119g181fb968d7da747a@mail.gmail.com> <15e8b9d21002082129i21a209cel925ad8b6ba3313cd@mail.gmail.com> <3dd3f56a1002082201l304a5473ge2859fcc52032913@mail.gmail.com> <560fb5ed1002122022u27f0ed29x44d7f821ddb44ab5@mail.gmail.com> <15e8b9d21002122239p158cf4d3vfc45faaa2b48dced@mail.gmail.com> Message-ID: <560fb5ed1002130003r7779e414sc6ca566375ea3452@mail.gmail.com> Er, okay, same question, but with "Object" as type for both second parameters. In editing it to look a bit more like a real situation I added a distinct post-erasure type by accident. As if you didn't know what I was asking. --Reinier Zwitserloot Need to receive donations via the web? Check https://tipit.to/ On Sat, Feb 13, 2010 at 7:39 AM, Neal Gafter wrote: > On Fri, Feb 12, 2010 at 8:22 PM, Reinier Zwitserloot > wrote: > > However, if function types aren't reified, then arrays are the least of > my > > worries. Does that mean something like this: > > void print(#(Number)String formatter, Number number) { > > System.out.println(formatter.(number)); > > } > > void print(#(String)String formatter, String string) { > > System.out.println(formatter.(string)); > > } > > is not legal (if this is done like generics, the signatures are equal > > post-erasure, which is a compile error). > > These signatures are distinct post-erasure, because the erasure of > their second parameters are distinct. > From pbenedict at apache.org Sat Feb 13 06:25:43 2010 From: pbenedict at apache.org (Paul Benedict) Date: Sat, 13 Feb 2010 08:25:43 -0600 Subject: Project Lambda: Java Language Specification draft 0.1.5 In-Reply-To: <15e8b9d21002121904u55b47769w9449c113709dc54a@mail.gmail.com> References: <4B75FEA2.9080904@sun.com> <15e8b9d21002121904u55b47769w9449c113709dc54a@mail.gmail.com> Message-ID: What happened to the self.invoke() syntax? A barrage of people came out saying self.() looks like a syntax error (I agree); I don't recall anyone advocating the syntax after the first draft was released. On Fri, Feb 12, 2010 at 9:04 PM, Neal Gafter wrote: > A few flubs and omissions that I noticed: > > - Your grammar is ambiguous. ?Expression goes to both LambdaExpression > and (indirectly) to Primary, both of which have a way of writing a > lambda. > > - It is unclear what the type of "this" is in an expression lambda. Is > it forbidden? > > - I wonder if the identity of a lambda can change spuriously. ?For > example, if "this" refers to the lambda expression, can it be expected > to have the same value on each invocation of the same lambda > expression? ?In other words, > > #Object() self = #()this; > assert self.() == self.(); // guaranteed?? > > - Similar question if the lambda has been converted to a SAM > > Object() self1 = #()this; > Callable self2 = self1; > assert self1.() == self2.call(); // guaranteed?? > > - What is the meaning of "toString()" inside a lambda? ?Does it refer > to the method inherited by the lambda from Object, or from the > enclosing scope? > > - It isn't clear why you would want special rules for recursive > lambdas in this revision of the spec, given that one could just invoke > "this" inside the lambda. ?Or, failing that, just use a method or an > anonymous inner class. > > - In your example where there is an "unreachable return statement", > the return statement is (formally) reachable. ?Any unreachable return > statement would be a compile-time error because it is an error to have > an unreachable statement in Java. > > - Your definition of effectively final mentions "target of an > initialization", but that doesn't make sense. ?It can only be the > target of an initialization in its declaration, and it is always > definitely unassigned before then. ?(Yes, even local variables in > switch statements) > > - The spec "It is a compile-time error to modify the value of an > effectively-final variable in the body of a lambda expression." is > meaningless. ?If there were an assignment to a variable from an > enclosing scope in a lambda, the variable would not be definitely > unassigned at that point, and so by definition the variable would not > be effectively-final. ?So this rule could never be applied in the way > you appear to intend. ?The only way the rule could be applied is > definitely something you do not want to forbid: > > Object o = #(){ > ? ?int x; // x is effectively final > ? ?x = 3; /* error: It is a compile-time error to modify the value of > an effectively-final variable in the body of a lambda expression */ > }; > > - Your example "The constraint #int(String) >> #int(T) resolves to > String>>T" in incorrect. ?It resolves to T>>String. > > - Your inference algorithm doesn't allow the use of the lambda > conversion. ?Specifically, if you try to pass > ?#()("Foo") > to a method > ?void f(Callable); > there is no way to infer that T=String. > > - Your assignment conversion doesn't allow a subtype conversion > followed by a lambda conversion. ?Consequently, you may force > programmers to use temporary variables just to force the compiler to > do a subtype conversion before the lambda conversion. ?In expression > contexts (i.e. super() invocations) that is cumbersome indeed. > > - Your rules for lambda conversion appear to improperly erase some > types. ?For example, the following code is ruled illegal > > interface Receiver { void f(T); } > Receiver> rls = #(List ls)(); // error: > "List" is not the same as "List". > > - The specification for a SAM type says that "If a SAM type is a class > type, then it must be declared abstract and have an accessible no-args > constructor.". ?That makes the following (currently legal code) > illegal: > > abstract class SAM { private SAM(int arg){} abstract void f(); } // > error: SAM type does not have no-args constructor > > - There are no accessibility constraints on the methods of a SAM. > What if the abstract method is package-private to someone else's > package? ?Or, what if there are two override-equivalent abstract > methods that are package-private to different packages? ?Generating > correct code for (a lambda conversion in) this case is extremely > cumbersome, and may require generating three or more classes in > different packages. > > - The following is a SAM (it has only one non-generic abstract > method), but should not be considered a SAM > > interface SAM { void foo(); void bar(int x); } > > - Your specification [11.2.2] makes no sense. ?You have the exceptions > from the body of a statement lambda being thrown where the lambda > expression appears. ?A lambda expression should throw no checked > exceptions. > > - You don't describe the cases in which a lambda conversion can throw > a checked exception. For example, what if the SAM constructor declared > an exception? > > - Your handling of exceptions in the lambda conversion is not correct. > ?For example > > interface I1 { void f() throws X1; } > interface I2 { void f() throws X2; } > interface I3 : I1, I2 {} > > I3 x = #(){ throw new X1(); } // allowed, even though I2.f() doesn't throw X1. > > - You say "let M be the set of methods..." but you mean "let S be the > set of abstract methods...". > > Cheers, > Neal > > From neal at gafter.com Sat Feb 13 08:59:58 2010 From: neal at gafter.com (Neal Gafter) Date: Sat, 13 Feb 2010 08:59:58 -0800 Subject: Function types versus arrays In-Reply-To: <560fb5ed1002130003r7779e414sc6ca566375ea3452@mail.gmail.com> References: <3dd3f56a1002081452o36d2b695kc766842ec2e8348b@mail.gmail.com> <15e8b9d21002081537p5ed0b3ebs21c06a8decdced74@mail.gmail.com> <3dd3f56a1002081733x786e329at5d2f273dcf3cec69@mail.gmail.com> <15e8b9d21002081754p428832e5mff96cc9586c6a47b@mail.gmail.com> <3dd3f56a1002081944p6d6e0119g181fb968d7da747a@mail.gmail.com> <15e8b9d21002082129i21a209cel925ad8b6ba3313cd@mail.gmail.com> <3dd3f56a1002082201l304a5473ge2859fcc52032913@mail.gmail.com> <560fb5ed1002122022u27f0ed29x44d7f821ddb44ab5@mail.gmail.com> <15e8b9d21002122239p158cf4d3vfc45faaa2b48dced@mail.gmail.com> <560fb5ed1002130003r7779e414sc6ca566375ea3452@mail.gmail.com> Message-ID: <15e8b9d21002130859w7b1e3e6fsfd00103448eb7e0@mail.gmail.com> On Sat, Feb 13, 2010 at 12:03 AM, Reinier Zwitserloot wrote: > Er, okay, same question, but with "Object" as type for both second > parameters. In editing it to look a bit more like a real situation I added a > distinct post-erasure type by accident. As if you didn't know what I was > asking. As you've seen, these issues don't arise often in practice, and can easily be fixed by renaming one of the methods. From peter.levart at gmail.com Sun Feb 14 13:34:17 2010 From: peter.levart at gmail.com (Peter Levart) Date: Sun, 14 Feb 2010 22:34:17 +0100 Subject: Function types versus arrays In-Reply-To: References: <15e8b9d21002071041j57e926e9k8e7c36e01f80ee46@mail.gmail.com> <4B75DEE7.1050209@sun.com> Message-ID: <201002142234.17260.peter.levart@gmail.com> On Saturday 13 February 2010 00:18:33 John Rose wrote: > On Feb 12, 2010, at 3:06 PM, Alex Buckley wrote: > > John Rose wrote: > >> The best way I know of to tag function arrays is to define function > >> arrays as pseudo-arrays which work at runtime like a > >> Collections.checkedList but look at the source level like arrays. > >> The most surprising wrinkle of such a design, IMO, is that they would > >> fail to be subtypes of Object[], much like int[] fails today. > > > > This compilation strategy would also enable arrays of parameterized > > types, and generally open the door to wonderful new type constructors > > that are useful in source and whose representation is known to the JIT. > > But unless we also change the compilation strategy for storing to an > > Object[], the universal supertype will bite us every time. I feel like > > we're peering over the edge of a very tall cliff. > > Maybe there is a way to back up one step from that cliff, though it might > be off a different cliff (the erasure cliff). How about this: > > A function type is not a reference type. Types are comprised of 1. > reference types, 2. primitive types, 3. function types. (In the > bytecodes, function types can still be *represented* by references, of > course.) > > Define one reference type FT (e.g., MethodHandle) as representing all > function types in the reference type hierarchy, via conversion. FT will > serve as a an erased or raw holder type for all function types. > > Add an implicit assignment conversion from any function type to FT. (Cf. > List to List.) > > Add an explicit casting conversion from FT to any function type. It does a > runtime check against some attribute query (e.g., MethodHandle.type). > > After the smoke clears, we find that arrays of function types are not > related to the type FT[] or Object[]. Also, arrays of function types can > be pseudo-arrays. > > We would still have a problem (which we did anyway) with representing > function types within field and method signatures. In erased form, they > would look simply like FT. Isn't JavaMethodHandle a way to bring diversity among function types' runtime representation? At least some diversity. Let's say function types were represented as parameterized subclasses of JavaMethodHandle (just like BGGA did with interfaces). Runtime representation would be their erased (raw) conterparts - this means that all subtype-related function types would be represented by a single JavaMethodHandle subclass at runtime. This would bring at least some runtime safety. The question of course is whether half-safety is better then no safety at all. > > -- John > Regards, Peter From neal at gafter.com Sun Feb 14 13:59:17 2010 From: neal at gafter.com (Neal Gafter) Date: Sun, 14 Feb 2010 13:59:17 -0800 Subject: Function types versus arrays In-Reply-To: <201002142234.17260.peter.levart@gmail.com> References: <15e8b9d21002071041j57e926e9k8e7c36e01f80ee46@mail.gmail.com> <4B75DEE7.1050209@sun.com> <201002142234.17260.peter.levart@gmail.com> Message-ID: <15e8b9d21002141359k49c21358pf12dfbc5f26bf60a@mail.gmail.com> On Sun, Feb 14, 2010 at 1:34 PM, Peter Levart wrote: > Isn't JavaMethodHandle a way to bring diversity among function types' runtime representation? At > least some diversity. Let's say function types were represented as parameterized subclasses of > JavaMethodHandle (just like BGGA did with interfaces). Runtime representation would be their > erased (raw) conterparts - this means that all subtype-related function types would be > represented by a single JavaMethodHandle subclass at runtime. This would bring at least some > runtime safety. The question of course is whether half-safety is better then no safety at all. That is one of the implementation techniques I assume is being considered. It would not support arrays of function type, for exactly the same reason that in Java today you can't create an array of a parameterized type. From peter.levart at marand.si Mon Feb 15 00:12:26 2010 From: peter.levart at marand.si (Peter Levart) Date: Mon, 15 Feb 2010 09:12:26 +0100 Subject: Function types versus arrays In-Reply-To: References: <15e8b9d21002071041j57e926e9k8e7c36e01f80ee46@mail.gmail.com> <4B75DEE7.1050209@sun.com> Message-ID: <201002150912.26122.peter.levart@marand.si> On Saturday 13 February 2010 00:18:33 John Rose wrote: > On Feb 12, 2010, at 3:06 PM, Alex Buckley wrote: > > > John Rose wrote: > >> The best way I know of to tag function arrays is to define function > >> arrays as pseudo-arrays which work at runtime like a > >> Collections.checkedList but look at the source level like arrays. > >> The most surprising wrinkle of such a design, IMO, is that they would > >> fail to be subtypes of Object[], much like int[] fails today. > > > > This compilation strategy would also enable arrays of parameterized > > types, and generally open the door to wonderful new type constructors > > that are useful in source and whose representation is known to the JIT. > > But unless we also change the compilation strategy for storing to an > > Object[], the universal supertype will bite us every time. I feel like > > we're peering over the edge of a very tall cliff. > > Maybe there is a way to back up one step from that cliff, though it might be off a different cliff (the erasure cliff). How about this: > > A function type is not a reference type. Types are comprised of 1. reference types, 2. primitive types, 3. function types. (In the bytecodes, function types can still be *represented* by references, of course.) > > Define one reference type FT (e.g., MethodHandle) as representing all function types in the reference type hierarchy, via conversion. FT will serve as a an erased or raw holder type for all function types. > > Add an implicit assignment conversion from any function type to FT. (Cf. List to List.) > > Add an explicit casting conversion from FT to any function type. It does a runtime check against some attribute query (e.g., MethodHandle.type). > > After the smoke clears, we find that arrays of function types are not related to the type FT[] or Object[]. Also, arrays of function types can be pseudo-arrays. But could function types then be used as parameters or parameter bounds in parameterized types? There's no top function type in this scheme is it? Could implicit conversion relation be treated the same as sub-type relation in parameter bounds? Regards, Peter From neal at gafter.com Mon Feb 15 00:43:42 2010 From: neal at gafter.com (Neal Gafter) Date: Mon, 15 Feb 2010 00:43:42 -0800 Subject: Function types versus arrays In-Reply-To: References: <15e8b9d21002071041j57e926e9k8e7c36e01f80ee46@mail.gmail.com> <4B748D6E.6090605@sun.com> <15e8b9d21002111526k262c0f3fid13e7395d78fffab@mail.gmail.com> <4B751891.7050408@univ-mlv.fr> <15e8b9d21002120833j645f79cbn599d51496cb98552@mail.gmail.com> <4B75875D.9020204@univ-mlv.fr> <4B75D43B.6050304@sun.com> <4B75DEE7.1050209@sun.com> Message-ID: <15e8b9d21002150043h32791611sf58fde105f06c10e@mail.gmail.com> On Fri, Feb 12, 2010 at 3:18 PM, John Rose wrote: > Add an explicit casting conversion from FT to any function type. ?It does a runtime check against some attribute query (e.g., MethodHandle.type). Where will this attribute come from? If the lambda appears inside a generic method, the compiler cannot produce it statically. And unless generics are reified, it cannot be generated dynamically either. From John.Rose at Sun.COM Mon Feb 15 01:25:04 2010 From: John.Rose at Sun.COM (John Rose) Date: Mon, 15 Feb 2010 01:25:04 -0800 Subject: Function types versus arrays In-Reply-To: <201002150912.26122.peter.levart@marand.si> References: <15e8b9d21002071041j57e926e9k8e7c36e01f80ee46@mail.gmail.com> <4B75DEE7.1050209@sun.com> <201002150912.26122.peter.levart@marand.si> Message-ID: On Feb 15, 2010, at 12:12 AM, Peter Levart wrote: > But could function types then be used as parameters or parameter bounds in parameterized types? Oops; thanks for picking up that point. That's a serious flaw in the "tertium quid" version of function types I sketched. Without function types under Object, we don't have List<#(int)->int>. That's clearly a non-starter. Or, we'd have to allow function types to be type bounds, even though they are not Objects, and then have an erasure story that went all the way to FT. That sounds like plenty of trouble, too, although perhaps it is possible (in ways that primitive-typed type parameters are not possible; see below). > There's no top function type in this scheme is it? Nope, just FT which stands under Object but "to the side" (via conversion) of all the function types. Because of contravariance, any function type with at least one parameter has a potentially infinite regression of supertypes, so there's no natural top type unless you force a peculiar 'FT' type into the system. > Could implicit conversion relation be treated the same as sub-type relation in parameter bounds? No, because of the following distinctions between subtyping and other implicit conversions: All operations on a supertype work on a subtype; the subtype adds more operations or different behavior but does not subtract operations. In addition subtype/supertype conversions do not change object identity. These properties make generic code valid for all subtypes of type variable bounds (supertypes), even though the code operates (concretely) only on references of the bounds types. Note that implicit conversions (such as int->Integer, int->double) can change bit patterns and/or object identity. This makes it impractical to write reusable generic code that operates equally well on all implicitly-convertible relatives of a type. Some implicit conversions even lose information, such as int->float or a null Integer to an int. The stronger notion of reference subtyping is needed to build generic code. (Oddly, the JLS refers to int as a subtype of double, etc., so primitive subtyping is a distinct and weaker notion compared to reference subtyping. As far as generic code goes, there is no useful notion of primitive subtyping; primitives are too irregular. But maybe function types are regular enough to make type variables with non-reference bounds.) -- John From John.Rose at Sun.COM Mon Feb 15 01:45:09 2010 From: John.Rose at Sun.COM (John Rose) Date: Mon, 15 Feb 2010 01:45:09 -0800 Subject: Function types versus arrays In-Reply-To: <15e8b9d21002150043h32791611sf58fde105f06c10e@mail.gmail.com> References: <15e8b9d21002071041j57e926e9k8e7c36e01f80ee46@mail.gmail.com> <4B748D6E.6090605@sun.com> <15e8b9d21002111526k262c0f3fid13e7395d78fffab@mail.gmail.com> <4B751891.7050408@univ-mlv.fr> <15e8b9d21002120833j645f79cbn599d51496cb98552@mail.gmail.com> <4B75875D.9020204@univ-mlv.fr> <4B75D43B.6050304@sun.com> <4B75DEE7.1050209@sun.com> <15e8b9d21002150043h32791611sf58fde105f06c10e@mail.gmail.com> Message-ID: On Feb 15, 2010, at 12:43 AM, Neal Gafter wrote: > Where will this attribute come from? If the lambda appears inside a > generic method, the compiler cannot produce it statically. And unless > generics are reified, it cannot be generated dynamically either. Right. Such an attribute is intrinsic to the function object and therefore needs to be fully established when the object is created. If a lambda is created by non-generic code, then all types are known by the compiler and can be baked in. If a lambda is created by generic code, that code will be operating in terms of the bounds types of any type variables, so if the lambda is returned to code which "sees" only a certain instance of the generic type, the lambda will have a dynamic type which looks "unfinished" relative to the expectations of the calling code. It may be that generic code can only be expected to build tightly-typed lambdas if it takes Class arguments to go with relevant parameters, so that lambdas can be created with the desired dynamic type. For example, here are a couple of test cases: static #(A)->T identityFunction() // hard to infer A/T, must return concrete #(Object x)(x) { return #(A x)(x); } static #()->T constantFunction(T x) // must return concrete #()((Object)x) { return type.cast( #()(x) ); } static #(A)->T identityFunction(FunctionType<#(A)->T> type) // assuming a suitable FunctionType.cast, all is well { return type.cast( #(A x)(x) ); } static #()->T constantFunction(FunctionType<#()->T> type, T x) { return type.cast( #()(x) ); } The dynamic cast functions would be required to build occasionally necessary bridges between erased types and their explicit reifications, as we do today with java.lang.Class.cast, etc. Unlike Class.cast, FunctionType.cast would be specified to create a wrapper of the requested type. -- John From neal at gafter.com Mon Feb 15 07:43:23 2010 From: neal at gafter.com (Neal Gafter) Date: Mon, 15 Feb 2010 07:43:23 -0800 Subject: Function types versus arrays In-Reply-To: References: <15e8b9d21002071041j57e926e9k8e7c36e01f80ee46@mail.gmail.com> <4B751891.7050408@univ-mlv.fr> <15e8b9d21002120833j645f79cbn599d51496cb98552@mail.gmail.com> <4B75875D.9020204@univ-mlv.fr> <4B75D43B.6050304@sun.com> <4B75DEE7.1050209@sun.com> <15e8b9d21002150043h32791611sf58fde105f06c10e@mail.gmail.com> Message-ID: <15e8b9d21002150743s487c7d25t546d18d827bc5402@mail.gmail.com> On Mon, Feb 15, 2010 at 1:45 AM, John Rose wrote: > The dynamic cast functions would be required to build occasionally necessary > bridges between erased types and their explicit reifications, as we do today > with java.lang.Class.cast, etc. Class objects are always of the erased type. In other words, we don't do that today. >?Unlike Class.cast, FunctionType.cast would > be specified to create a wrapper of the requested type. > -- John Even if the requested type is different from the "original type" of the lambda? In other words, all this work to reify them but in a way that undermines the type system? In addition, these wrappers do not preserve object identity. From neal at gafter.com Mon Feb 15 08:19:07 2010 From: neal at gafter.com (Neal Gafter) Date: Mon, 15 Feb 2010 08:19:07 -0800 Subject: Project Lambda: Java Language Specification draft 0.1.5 In-Reply-To: References: <4B75FEA2.9080904@sun.com> <15e8b9d21002121904u55b47769w9449c113709dc54a@mail.gmail.com> Message-ID: <15e8b9d21002150819n538ebc80t22fdc412e293ce9d@mail.gmail.com> On Sat, Feb 13, 2010 at 6:25 AM, Paul Benedict wrote: > What happened to the self.invoke() syntax? A barrage of people came > out saying self.() looks like a syntax error (I agree); I don't recall > anyone advocating the syntax after the first draft was released. To refresh your memory, here was my advocacy: http://mail.openjdk.java.net/pipermail/lambda-dev/2010-January/000370.html See also http://mail.openjdk.java.net/pipermail/lambda-dev/2010-January/000365.html http://mail.openjdk.java.net/pipermail/lambda-dev/2010-January/000350.html http://mail.openjdk.java.net/pipermail/lambda-dev/2010-January/000445.html Cheers, Neal From howard.lovatt at iee.org Mon Feb 15 13:57:11 2010 From: howard.lovatt at iee.org (Howard Lovatt) Date: Tue, 16 Feb 2010 08:57:11 +1100 Subject: Function types versus arrays In-Reply-To: <15e8b9d21002122239p158cf4d3vfc45faaa2b48dced@mail.gmail.com> References: <3dd3f56a1002081452o36d2b695kc766842ec2e8348b@mail.gmail.com> <15e8b9d21002081537p5ed0b3ebs21c06a8decdced74@mail.gmail.com> <3dd3f56a1002081733x786e329at5d2f273dcf3cec69@mail.gmail.com> <15e8b9d21002081754p428832e5mff96cc9586c6a47b@mail.gmail.com> <3dd3f56a1002081944p6d6e0119g181fb968d7da747a@mail.gmail.com> <15e8b9d21002082129i21a209cel925ad8b6ba3313cd@mail.gmail.com> <3dd3f56a1002082201l304a5473ge2859fcc52032913@mail.gmail.com> <560fb5ed1002122022u27f0ed29x44d7f821ddb44ab5@mail.gmail.com> <15e8b9d21002122239p158cf4d3vfc45faaa2b48dced@mail.gmail.com> Message-ID: <3dd3f56a1002151357m2d5b13bs5e078c69d604e31b@mail.gmail.com> Neal Gafter gave an array example on his blog: http://gafter.blogspot.com/ Using: http://www.artima.com/weblogs/viewpost.jsp?thread=278567 It is translated to: // 1. #void()[] arrayOfFunction = new #void()[1]; _Callable_void[] arrayOfFunction = new _Callable_void[1]; // 2. Object[] objectArray = arrayOfFunction; Object[] objectArray = arrayOfFunction; // No translation! // 3. objectArray[0] = #(){ throw new IOException(); }; objectArray[0] = new _Callable_void_throws_IOException() { @Override public void callPrimitive() throws IOException { throw new IOException(); } }; // Throws ArrayStoreException // 4. arrayOfFunction[0].(); // invoke the function out of the array arrayOfFunction[0].callPrimitive(); Line 3 throws an array store exception, as you would expect, because the array is of type _Callable_void[] and you are attempting to put in a _Callable_void_throws_IOException. Therefore using Reified Lambdas the example works as expected. -- Howard. From howard.lovatt at iee.org Mon Feb 15 14:12:53 2010 From: howard.lovatt at iee.org (Howard Lovatt) Date: Tue, 16 Feb 2010 09:12:53 +1100 Subject: Function types versus arrays In-Reply-To: <560fb5ed1002122022u27f0ed29x44d7f821ddb44ab5@mail.gmail.com> References: <3dd3f56a1002081452o36d2b695kc766842ec2e8348b@mail.gmail.com> <15e8b9d21002081537p5ed0b3ebs21c06a8decdced74@mail.gmail.com> <3dd3f56a1002081733x786e329at5d2f273dcf3cec69@mail.gmail.com> <15e8b9d21002081754p428832e5mff96cc9586c6a47b@mail.gmail.com> <3dd3f56a1002081944p6d6e0119g181fb968d7da747a@mail.gmail.com> <15e8b9d21002082129i21a209cel925ad8b6ba3313cd@mail.gmail.com> <3dd3f56a1002082201l304a5473ge2859fcc52032913@mail.gmail.com> <560fb5ed1002122022u27f0ed29x44d7f821ddb44ab5@mail.gmail.com> Message-ID: <3dd3f56a1002151412g5edf9fa2v2e2a2aed7291a28b@mail.gmail.com> Hi Reinier, On 13 February 2010 15:22, Reinier Zwitserloot wrote: > On Tue, Feb 9, 2010 at 7:01 AM, Howard Lovatt??wrote: >> >> Well you *speculate* that reified lambdas are type unsound, you haven't >> and >> nor has anyone else demonstrated that - in fact quite the opposite. > > Well, you *speculate* that disallowing arrays of functional types is 'just > not java', but you haven't demonstrated that. > Arrays, really? They're almost deprecated at this point. As long as function > types are legal in generics, and any usage in arrays is flagged with a clear > error, I doubt the greater java community is going to get out the tar and > feathers. Internally in my classes I still use arrays quite a bit, particularly arrays of primitives. Alex was asking for what people think about arrays of lambdas and my feeling is that disallowing them will be yet another puzzler 101 that will confuse people. This to me is much like erased generics, sure they have their good points but overall the reaction is negative because they don't fit seamlessly into the rest of Java. > > However, if function types aren't reified, then arrays are the least of my > worries. Does that mean something like this: > void print(#(Number)String formatter, Number number) { > ?? ?System.out.println(formatter.(number)); > } > void print(#(String)String formatter, String string) { > ?? ?System.out.println(formatter.(string)); > } > is not legal (if this is done like generics, the signatures are equal > post-erasure, which is a compile error). An alternate example might be something like ParallelArray with specific (fast) reduce methods, i.e.: int reduce(#int(int, int)), double reduce(#double(double, double)), etc., as well as T reduce(#T(T, T)). This wouldn't be possible with erased lambdas. -- Howard. > ?--Reinier Zwitserloot > > > ______________________________________________________________________ > This email has been scanned by the MessageLabs Email Security System. > For more information please visit http://www.messagelabs.com/email > ______________________________________________________________________ > -- -- Howard. From Alex.Buckley at Sun.COM Mon Feb 15 14:31:02 2010 From: Alex.Buckley at Sun.COM (Alex Buckley) Date: Mon, 15 Feb 2010 14:31:02 -0800 Subject: Function types versus arrays In-Reply-To: References: <15e8b9d21002071041j57e926e9k8e7c36e01f80ee46@mail.gmail.com> <4B748D6E.6090605@sun.com> <15e8b9d21002111526k262c0f3fid13e7395d78fffab@mail.gmail.com> <4B751891.7050408@univ-mlv.fr> <15e8b9d21002120833j645f79cbn599d51496cb98552@mail.gmail.com> <4B75875D.9020204@univ-mlv.fr> <4B75D43B.6050304@sun.com> <4B75DEE7.1050209@sun.com> <15e8b9d21002150043h32791611sf58fde105f06c10e@mail.gmail.com> Message-ID: <4B79CB26.3090806@sun.com> John Rose wrote: > If a lambda is created by generic code, that code will be operating > in terms of the bounds types of any type variables, so if the lambda > is returned to code which "sees" only a certain instance of the > generic type, the lambda will have a dynamic type which looks > "unfinished" relative to the expectations of the calling code. It > may be that generic code can only be expected to build tightly-typed > lambdas if it takes Class arguments to go with relevant > parameters, so that lambdas can be created with the desired dynamic > type. For example, here are a couple of test cases: > > static > #(A)->T identityFunction() > // hard to infer A/T, must return concrete #(Object x)(x) > { return #(A x)(x); } > > static > #()->T constantFunction(T x) > // must return concrete #()((Object)x) > { return type.cast( #()(x) ); } > > static > #(A)->T identityFunction(FunctionType<#(A)->T> type) > // assuming a suitable FunctionType.cast, all is well > { return type.cast( #(A x)(x) ); } > > static > #()->T constantFunction(FunctionType<#()->T> type, T x) > { return type.cast( #()(x) ); } Requiring FunctionType arguments is a red herring in my view. The caller of a generic method is also in a position to perform the cast. Indeed, the caller of the first (and second) example must do *something* since otherwise, invoking foo.identityFunction() will return a function expected to return a String but which only returns Object. (If the caller's cast is really a wrap, Neal objections w.r.t. object identity stand. But wrapping doesn't seem like the only option.) Alex From neal at gafter.com Mon Feb 15 14:48:07 2010 From: neal at gafter.com (Neal Gafter) Date: Mon, 15 Feb 2010 14:48:07 -0800 Subject: Function types versus arrays In-Reply-To: <3dd3f56a1002151357m2d5b13bs5e078c69d604e31b@mail.gmail.com> References: <3dd3f56a1002081452o36d2b695kc766842ec2e8348b@mail.gmail.com> <15e8b9d21002081537p5ed0b3ebs21c06a8decdced74@mail.gmail.com> <3dd3f56a1002081733x786e329at5d2f273dcf3cec69@mail.gmail.com> <15e8b9d21002081754p428832e5mff96cc9586c6a47b@mail.gmail.com> <3dd3f56a1002081944p6d6e0119g181fb968d7da747a@mail.gmail.com> <15e8b9d21002082129i21a209cel925ad8b6ba3313cd@mail.gmail.com> <3dd3f56a1002082201l304a5473ge2859fcc52032913@mail.gmail.com> <560fb5ed1002122022u27f0ed29x44d7f821ddb44ab5@mail.gmail.com> <15e8b9d21002122239p158cf4d3vfc45faaa2b48dced@mail.gmail.com> <3dd3f56a1002151357m2d5b13bs5e078c69d604e31b@mail.gmail.com> Message-ID: <15e8b9d21002151448i2ca40f8fpbf40c70caffeab2c@mail.gmail.com> On Mon, Feb 15, 2010 at 1:57 PM, Howard Lovatt wrote: > Neal Gafter gave an array example on his blog: > > http://gafter.blogspot.com/ > > Using: > > http://www.artima.com/weblogs/viewpost.jsp?thread=278567 > > It is translated to: That's just not true. Your post does not give any hint as to how throw clauses or exceptions in function types are to affect the translation. The word "throws" doesn't even appear anywhere in your post at all. But I'm glad to see that you're now considering some of the areas where your specification was completely lacking before. How would your translation scheme handle the code below? Your post contains not a single hint. #void()(throws ChangedCharSetException, UnknownHostException, SAXParseException)[] array1 = new #void()(throws ChangedCharSetException, UnknownHostException)[1]; #void()(throws IOException, AWTException, SAXException)[] array2 = array1; Cheers, Neal From forax at univ-mlv.fr Mon Feb 15 16:40:22 2010 From: forax at univ-mlv.fr (=?ISO-8859-1?Q?R=E9mi_Forax?=) Date: Tue, 16 Feb 2010 01:40:22 +0100 Subject: Function types versus arrays In-Reply-To: <4B79CB26.3090806@sun.com> References: <15e8b9d21002071041j57e926e9k8e7c36e01f80ee46@mail.gmail.com> <4B748D6E.6090605@sun.com> <15e8b9d21002111526k262c0f3fid13e7395d78fffab@mail.gmail.com> <4B751891.7050408@univ-mlv.fr> <15e8b9d21002120833j645f79cbn599d51496cb98552@mail.gmail.com> <4B75875D.9020204@univ-mlv.fr> <4B75D43B.6050304@sun.com> <4B75DEE7.1050209@sun.com> <15e8b9d21002150043h32791611sf58fde105f06c10e@mail.gmail.com> <4B79CB26.3090806@sun.com> Message-ID: <4B79E976.6000900@univ-mlv.fr> Le 15/02/2010 23:31, Alex Buckley a ?crit : > John Rose wrote: > >> If a lambda is created by generic code, that code will be operating >> in terms of the bounds types of any type variables, so if the lambda >> is returned to code which "sees" only a certain instance of the >> generic type, the lambda will have a dynamic type which looks >> "unfinished" relative to the expectations of the calling code. It >> may be that generic code can only be expected to build tightly-typed >> lambdas if it takes Class arguments to go with relevant >> parameters, so that lambdas can be created with the desired dynamic >> type. For example, here are a couple of test cases: >> >> static >> #(A)->T identityFunction() >> // hard to infer A/T, must return concrete #(Object x)(x) >> { return #(A x)(x); } >> >> static >> #()->T constantFunction(T x) >> // must return concrete #()((Object)x) >> { return type.cast( #()(x) ); } >> >> static >> #(A)->T identityFunction(FunctionType<#(A)->T> type) >> // assuming a suitable FunctionType.cast, all is well >> { return type.cast( #(A x)(x) ); } >> >> static >> #()->T constantFunction(FunctionType<#()->T> type, T x) >> { return type.cast( #()(x) ); } >> > Requiring FunctionType arguments is a red herring in my view. The caller > of a generic method is also in a position to perform the cast. Indeed, > the caller of the first (and second) example must do *something* since > otherwise, invoking foo.identityFunction() will return a > function expected to return a String but which only returns Object. > I agree. Like with generics/erased code, the conversion must be done at call site. > (If the caller's cast is really a wrap, Neal objections w.r.t. object > identity stand. But wrapping doesn't seem like the only option.) > In my opinion, object identity is not important here, conversions like boxing/unboxing can already occur at call site of a generics instantiation. You only need to have object identity when you deal with widening/narrowing conversions. #T() func(T t) { return #() (t); } func("foo") can be translated to func("foo").asType(ldc #String()); > Alex > > R?mi From neal at gafter.com Mon Feb 15 17:14:10 2010 From: neal at gafter.com (Neal Gafter) Date: Mon, 15 Feb 2010 17:14:10 -0800 Subject: Function types versus arrays In-Reply-To: <4B79E976.6000900@univ-mlv.fr> References: <15e8b9d21002071041j57e926e9k8e7c36e01f80ee46@mail.gmail.com> <4B75875D.9020204@univ-mlv.fr> <4B75D43B.6050304@sun.com> <4B75DEE7.1050209@sun.com> <15e8b9d21002150043h32791611sf58fde105f06c10e@mail.gmail.com> <4B79CB26.3090806@sun.com> <4B79E976.6000900@univ-mlv.fr> Message-ID: <15e8b9d21002151714r7c9f07cdg8e1fcbe5f2c91052@mail.gmail.com> On Mon, Feb 15, 2010 at 4:40 PM, R?mi Forax wrote: > You only need to have object identity when you deal > with widening/narrowing conversions. > > #T() func(T t) { > ? return #() (t); > } > > func("foo") > can be translated to > func("foo").asType(ldc #String()); And what if the lambda was placed into a collection before being returned? From pbenedict at apache.org Mon Feb 15 22:40:26 2010 From: pbenedict at apache.org (Paul Benedict) Date: Tue, 16 Feb 2010 00:40:26 -0600 Subject: Project Lambda: Java Language Specification draft 0.1.5 In-Reply-To: <15e8b9d21002150819n538ebc80t22fdc412e293ce9d@mail.gmail.com> References: <4B75FEA2.9080904@sun.com> <15e8b9d21002121904u55b47769w9449c113709dc54a@mail.gmail.com> <15e8b9d21002150819n538ebc80t22fdc412e293ce9d@mail.gmail.com> Message-ID: Neal, Thanks for the links. To offer my perspective, those who support the syntax don't appear to have a typical background -- usually those who are admitted language designers or are serious hobbyists about languages. If that sounds funny for a "lamda dev" mailing list, there's definitely a spread subscribed here based on responses. Nothing wrong with that. I just think those who opposed better represented the casual Java population. Not being a language designer myself, I empathize with their position. Paul On Mon, Feb 15, 2010 at 10:19 AM, Neal Gafter wrote: > On Sat, Feb 13, 2010 at 6:25 AM, Paul Benedict wrote: >> What happened to the self.invoke() syntax? A barrage of people came >> out saying self.() looks like a syntax error (I agree); I don't recall >> anyone advocating the syntax after the first draft was released. > > To refresh your memory, here was my advocacy: > http://mail.openjdk.java.net/pipermail/lambda-dev/2010-January/000370.html > > See also http://mail.openjdk.java.net/pipermail/lambda-dev/2010-January/000365.html > http://mail.openjdk.java.net/pipermail/lambda-dev/2010-January/000350.html > http://mail.openjdk.java.net/pipermail/lambda-dev/2010-January/000445.html > > Cheers, > Neal > From neal at gafter.com Mon Feb 15 22:57:18 2010 From: neal at gafter.com (Neal Gafter) Date: Mon, 15 Feb 2010 22:57:18 -0800 Subject: Project Lambda: Java Language Specification draft 0.1.5 In-Reply-To: References: <4B75FEA2.9080904@sun.com> <15e8b9d21002121904u55b47769w9449c113709dc54a@mail.gmail.com> <15e8b9d21002150819n538ebc80t22fdc412e293ce9d@mail.gmail.com> Message-ID: <15e8b9d21002152257s72ee5183p11f458816170fc55@mail.gmail.com> On Mon, Feb 15, 2010 at 10:40 PM, Paul Benedict wrote: > Thanks for the links. To offer my perspective, those who support the > syntax don't appear to have a typical background -- usually those who > are admitted language designers or are serious hobbyists about > languages. If that sounds funny for a "lamda dev" mailing list, > there's definitely a spread subscribed here based on responses. > Nothing wrong with that. I just think those who opposed better > represented the casual Java population. Not being a language designer > myself, I empathize with their position. Yes, I definitely see that point. Experienced language developers are often better able to judge how difficult a job it will be for developers to internalize a language change based on experience with other languages, while programmers often fear things that appear to be different, even though that difference may improve their lot once they begin using the feature. In any case, this isn't a democracy. We get to peek over the shoulders of folks at Sun every now and then to see their progress. From forax at univ-mlv.fr Tue Feb 16 00:42:12 2010 From: forax at univ-mlv.fr (=?ISO-8859-1?Q?R=E9mi_Forax?=) Date: Tue, 16 Feb 2010 09:42:12 +0100 Subject: Function types versus arrays In-Reply-To: <15e8b9d21002151714r7c9f07cdg8e1fcbe5f2c91052@mail.gmail.com> References: <15e8b9d21002071041j57e926e9k8e7c36e01f80ee46@mail.gmail.com> <4B75875D.9020204@univ-mlv.fr> <4B75D43B.6050304@sun.com> <4B75DEE7.1050209@sun.com> <15e8b9d21002150043h32791611sf58fde105f06c10e@mail.gmail.com> <4B79CB26.3090806@sun.com> <4B79E976.6000900@univ-mlv.fr> <15e8b9d21002151714r7c9f07cdg8e1fcbe5f2c91052@mail.gmail.com> Message-ID: <4B7A5A64.9090804@univ-mlv.fr> Le 16/02/2010 02:14, Neal Gafter a ?crit : > On Mon, Feb 15, 2010 at 4:40 PM, R?mi Forax wrote: > >> You only need to have object identity when you deal >> with widening/narrowing conversions. >> >> #T() func(T t) { >> return #() (t); >> } >> >> func("foo") >> can be translated to >> func("foo").asType(ldc #String()); >> > And what if the lambda was placed into a collection before being returned? > You kill my dream :) R?mi From pbenedict at apache.org Tue Feb 16 00:45:20 2010 From: pbenedict at apache.org (Paul Benedict) Date: Tue, 16 Feb 2010 02:45:20 -0600 Subject: Project Lambda: Java Language Specification draft 0.1.5 In-Reply-To: <15e8b9d21002152257s72ee5183p11f458816170fc55@mail.gmail.com> References: <4B75FEA2.9080904@sun.com> <15e8b9d21002121904u55b47769w9449c113709dc54a@mail.gmail.com> <15e8b9d21002150819n538ebc80t22fdc412e293ce9d@mail.gmail.com> <15e8b9d21002152257s72ee5183p11f458816170fc55@mail.gmail.com> Message-ID: My only advice is to keep simplicity and the obvious in mind. I think the "object.()" syntax violates the principle of least surprise -- I was surprised to suddenly see method calls without a method name. The decision does make sense within context, but it is obscure and esoteric; I think it's too much of a break with the past. If you really believe that the "object.()" syntax is the best, I'll trust your decision (i.e., having a doctorate, working on C#, etc.) but I wanted to raise my objection still. On Tue, Feb 16, 2010 at 12:57 AM, Neal Gafter wrote: > On Mon, Feb 15, 2010 at 10:40 PM, Paul Benedict wrote: >> Thanks for the links. To offer my perspective, those who support the >> syntax don't appear to have a typical background -- usually those who >> are admitted language designers or are serious hobbyists about >> languages. If that sounds funny for a "lamda dev" mailing list, >> there's definitely a spread subscribed here based on responses. >> Nothing wrong with that. I just think those who opposed better >> represented the casual Java population. Not being a language designer >> myself, I empathize with their position. > > Yes, I definitely see that point. ?Experienced language developers are > often better able to judge how difficult a job it will be for > developers to internalize a language change based on experience with > other languages, while programmers often fear things that appear to be > different, even though that difference may improve their lot once they > begin using the feature. > > In any case, this isn't a democracy. ?We get to peek over the > shoulders of folks at Sun every now and then to see their progress. > From schulz at the-loom.de Tue Feb 16 01:22:05 2010 From: schulz at the-loom.de (Stefan Schulz) Date: Tue, 16 Feb 2010 10:22:05 +0100 Subject: Project Lambda: Java Language Specification draft 0.1.5 In-Reply-To: References: <4B75FEA2.9080904@sun.com> <15e8b9d21002121904u55b47769w9449c113709dc54a@mail.gmail.com> <15e8b9d21002150819n538ebc80t22fdc412e293ce9d@mail.gmail.com> <15e8b9d21002152257s72ee5183p11f458816170fc55@mail.gmail.com> Message-ID: <4B7A63BD.4090805@the-loom.de> On16.02.2010 09:45, Paul Benedict wrote: > My only advice is to keep simplicity and the obvious in mind. I think > the "object.()" syntax violates the principle of least surprise -- I > was surprised to suddenly see method calls without a method name. The > decision does make sense within context, but it is obscure and > esoteric; I think it's too much of a break with the past. If you > really believe that the "object.()" syntax is the best, I'll trust > your decision (i.e., having a doctorate, working on C#, etc.) but I > wanted to raise my objection still. I second that. The tiny dot gets lost in the noise of symbols and actually only saves from typing two or so characters. I don't think it's worth introducing this faulty looking syntax for a really marginal gain. If it cannot be made function-invocation syntax, I'd advocate for making it Java style rather than going Perl-like. Cheers, Stefan From daniel.latremoliere at gmail.com Tue Feb 16 02:08:53 2010 From: daniel.latremoliere at gmail.com (=?ISO-8859-1?Q?Daniel_Latr=E9moli=E8re?=) Date: Tue, 16 Feb 2010 11:08:53 +0100 Subject: A small question Message-ID: <4B7A6EB5.2000309@gmail.com> When reading these threads, I have a small question. It seems the key driver for lambda is parallelism (which is a very small part of their code for many programmers, hopefully for avoiding brain damaging) and avoiding explosion of number of interfaces (probably the interfaces of extra166y.Ops.*). Given I see a probable consensus in mailing list for implementing lambda as java.dyn.MethodHandle, I would like to know how not replacing all these interfaces of extra166y.Ops.* by a MethodHandle in code using these interfaces? (and not defining any lambda in Java language). --- Some motivations/explanations of my question: With this replacement type of lambda will be clearly erased, but you can introduce some annotations for helping error detection (like JSR 305): @Signature(returnType=@Type($ReturnType.class), arguments={@Type(ArgType1.class), ...}) [1]. When, and only when, experience will be here and usage of lambda sufficiently frequent, it will be time for changing Java language. Currently Java has many other typing features, more frequently used, not supported by VM, javac and language, like @NotNull (JSR 303) / @Nonnull (JSR 305). I don't think the shortening of source code is a good reason, given it will be much more generic and useful for all programmers to have something like type aliasing, which will save much more characters in all source code (particularly in code using generics, even if diamond operator has already given some improvements). If selection of MethodHandle by name, is really a problem, this will need method/field literals (currently without many talks on this list). Adding of method/field literals can be useful, even if I think more large input would be useful (possibly useful in some validation annotations, JPQL made by annotations, etc.). This feature seems to have been rejected for Project Coin. Not allowing lambda will reduce tendency to have many nested levels of context (class -> method -> lambda) with loops (for, while) or conditional paths (if-then-else) adding their own changing contexts. Simplicity is always needed for avoiding bugs (programmer need to understand code, immediately and perfectly!). I prefer having a slightly more verbose solution in current Java and adding some partial but really generic solutions to having a half-baked solution made only for a specific library hampering all Java programmers. I prefer some form of progressive enhancements to big breaking changes. We don't know lifetime of this library in Java platform. Personally, I'm not really considering possible uses of this library: if I need highly parallel computing framework, I would probably use Java to dynamically generate optimized OpenCL code. For me, Java is a language for complex logic programming (CPU) with random-access memory and big caches, not for small-tasks parallel programming (DSP/GPU) with streaming-access memory. These use-models of memory are intrinsics of the performances of these two silicon chips and I don't think personally they will disappear/merge, even by migrating on the same SoC (cf. embedded). When you manipulate methods for generating OpenCL code, having names (identifiers) and signature is very useful, then the java.lang.reflect model is probably correct; performance is not really needed, because you don't execute these methods: you create OpenCL code and send it to DSP for parallel execution. My two cents, Daniel Latr?moli?re. [1]: Strictly speaking, this solution need some convolutions to allow native types, given annotations does not support Integer.TYPE as valid class literal. public @interface Signature { Type returnType() default @Type; // void by default Type[] argumentTypes() default {}; } public @interface Type { Class value() default Void.class; Native nativeType() default Native.CLASS; } public enum Native { BYTE, SHORT, INT, LONG, FLOAT, DOUBLE, CHAR, CLASS; } Example with "Integer Integer.valueOf(int)": @Signature(returnType=@Type(Integer.class), argumentTypes={@Type(nativeType=Native.INT)}) MethodHandle valueOf; From David.Moss at ubs.com Tue Feb 16 02:51:05 2010 From: David.Moss at ubs.com (David.Moss at ubs.com) Date: Tue, 16 Feb 2010 11:51:05 +0100 Subject: Project Lambda: Java Language Specification draft 0.1.5 In-Reply-To: <4B7A63BD.4090805@the-loom.de> References: <4B75FEA2.9080904@sun.com> <15e8b9d21002121904u55b47769w9449c113709dc54a@mail.gmail.com> <15e8b9d21002150819n538ebc80t22fdc412e293ce9d@mail.gmail.com> <15e8b9d21002152257s72ee5183p11f458816170fc55@mail.gmail.com> <4B7A63BD.4090805@the-loom.de> Message-ID: <9FF0EECF0E58D849A9D537E324DDDC0D044A43F1@NZURC102PEX1.ubsw.net> > I don't think it's worth introducing this faulty looking syntax > for a really marginal gain. > If it cannot be made function-invocation syntax, I'd advocate > for making it Java style rather than going Perl-like. I disagree with this view. This is not "faulty looking syntax", it is "new syntax", much the same way the enhanced for loop or generics would be faulty syntax under java 2, but valid after being introduced in a later version. Furthermore, the var.() call is perfectly appropriate for an anonymous function, as it has no name. The alternative would be to make the named variable look like a method call, i.e. var(), which is no less "faulty looking syntax", but more confusing, or to add a 'default method' such as var.do() or var.invoke(), but then your anonymous function suddenly has a name. Kind Regards, David. -- EUR2 EG.1168 [1923 96757] SST: DL-SST-DEV "Where's the Kaboom? There was supposed to be an Earth-shattering Kaboom!" - Marvin the Martian "Don't worry about what anybody else is going to do. The best way to predict the future is to invent it." - Alan Kay Based on the present E-Mail exchange, and/or on the agreement reached with you, respectively, UBS is entitled to contact you via insecure E-Mail: (a) E-Mails contain substantial risks such as lack of confidentiality, manipulation of content and sender, misdirection, viruses etc. UBS does not accept any liability for damages arising from use of E-mail. Accordingly, UBS recommends to abstain from sending any sensitive information via E-Mail, from forwarding the text received when submitting reply E-Mails and recommends to manually capture the E-Mail address in every instance. If you should wish to verify the content of this message, please request a hard-copy version. (b) In principle, UBS does not accept any (purchase) orders, cancellation of orders or authorizations etc. via E-mail. If UBS receives such E-Mails, UBS is not obliged to expressly decline them. If you have received this E-Mail by mistake or do not wish to be contacted by E-Mail in the future, you are kindly asked to inform UBS accordingly. Any E-Mail received by mistake (including all its annexes) needs to be destroyed and the content may not be forwarded nor disclosed to any further persons. c) This message is provided for informational purposes and should not be construed as a solicitation or offer to buy or sell any securities or related financial instruments. UBS reserves the right to retain all messages. Messages are protected and accessed only in legally justified cases. From schulz at the-loom.de Tue Feb 16 03:33:53 2010 From: schulz at the-loom.de (Stefan Schulz) Date: Tue, 16 Feb 2010 12:33:53 +0100 Subject: Project Lambda: Java Language Specification draft 0.1.5 In-Reply-To: <9FF0EECF0E58D849A9D537E324DDDC0D044A43F1@NZURC102PEX1.ubsw.net> References: <4B75FEA2.9080904@sun.com> <15e8b9d21002121904u55b47769w9449c113709dc54a@mail.gmail.com> <15e8b9d21002150819n538ebc80t22fdc412e293ce9d@mail.gmail.com> <15e8b9d21002152257s72ee5183p11f458816170fc55@mail.gmail.com> <4B7A63BD.4090805@the-loom.de> <9FF0EECF0E58D849A9D537E324DDDC0D044A43F1@NZURC102PEX1.ubsw.net> Message-ID: <4B7A82A1.3000104@the-loom.de> On 16.02.2010 11:51, David.Moss at ubs.com wrote: >> I don't think it's worth introducing this faulty looking syntax >> for a really marginal gain. >> If it cannot be made function-invocation syntax, I'd advocate >> for making it Java style rather than going Perl-like. > > I disagree with this view. This is not "faulty looking syntax", it is > "new syntax", much the same way the enhanced for loop or generics would > be faulty syntax under java 2, but valid after being introduced in a > later version. I say it's faulty, because as a Java developer i expect a method name after a dot. The syntax Identifier '.' is not new, as you imply, but is a fragment of an already existing syntax. It cannot be compared to generics, as these had a completely new syntax not raising any expectations before being introduced. > Furthermore, the var.() call is perfectly appropriate for > an anonymous function, as it has no name. I think, you are mixing the function with the way to invoke it. The function you refer to is stored in "var" (maybe a reference to a block of code or whatever works). What we are looking at here is a means to execute the function "var" refers to. And to me, if we cannot have functions as first class elements in Java (see below), the syntax for executing a function should be similar to operating on an object. (Objects are anonymous as well, but a programmer knows about their API. I'd expect a lambda to conform to some function type API as well.) > The alternative would be to > make the named variable look like a method call, i.e. var(), which is no > less "faulty looking syntax", but more confusing, This syntax does not look faulty and actually would be the correct way of invoking a function (similar to a method), as most programmers would expect. The only reason it is out of discussion are the problems with name spaces and conflicts wrt. methods. Cheers, Stefan From markmahieu at googlemail.com Tue Feb 16 03:35:16 2010 From: markmahieu at googlemail.com (Mark Mahieu) Date: Tue, 16 Feb 2010 11:35:16 +0000 Subject: Project Lambda: Java Language Specification draft 0.1.5 In-Reply-To: References: <4B75FEA2.9080904@sun.com> <15e8b9d21002121904u55b47769w9449c113709dc54a@mail.gmail.com> <15e8b9d21002150819n538ebc80t22fdc412e293ce9d@mail.gmail.com> <15e8b9d21002152257s72ee5183p11f458816170fc55@mail.gmail.com> Message-ID: On 16 Feb 2010, at 08:45, Paul Benedict wrote: > My only advice is to keep simplicity and the obvious in mind. I think > the "object.()" syntax violates the principle of least surprise -- I > was surprised to suddenly see method calls without a method name. On 16 Feb 2010, at 10:51, wrote: > Furthermore, the var.() call is perfectly appropriate for > an anonymous function, as it has no name. That's an important difference. Arguably 'surprising' when thought of as a method call, yet 'perfectly appropriate' when seen as a function call. Regards, Mark From reinier at zwitserloot.com Tue Feb 16 04:55:08 2010 From: reinier at zwitserloot.com (Reinier Zwitserloot) Date: Tue, 16 Feb 2010 13:55:08 +0100 Subject: Project Lambda: Java Language Specification draft 0.1.5 In-Reply-To: References: <4B75FEA2.9080904@sun.com> <15e8b9d21002121904u55b47769w9449c113709dc54a@mail.gmail.com> <15e8b9d21002150819n538ebc80t22fdc412e293ce9d@mail.gmail.com> <15e8b9d21002152257s72ee5183p11f458816170fc55@mail.gmail.com> Message-ID: <560fb5ed1002160455l24a024bey4428d210c26368b1@mail.gmail.com> That's not the principle of least surprise. The PLS means: If a significant chunk of those people using it think a certain library call or construct does X, and believe this is sufficiently logic to not immediately dive for the documentation, but this call or construct actually does Y, that's a violation of PLS. That's _clearly_ not happening here. The only possible expectation here is that "foo.()" is simply invalid syntax, but this argument feels like weak tea to me: You can make that argument against just about every new language feature. The dot is easy to miss: No, it isn't. If you take the union of the groups: [ programs in a non-monospace font OR uses a style convention to add spaces right before the open paren in a method invocation ] you've covered everyone that could possibly miss it. I don't have research, but I very much doubt it's a big group. the dot is "obscure and esoteric": That's a pleonasm. It also needs backup - what's obscure about it? The dot is a widely known character that stands for "dereference". Parentheses following dots stand for method invocation. Seems about right. Sure, if you show this syntax to an average joe Java programmer on the street today it's likely he wont know what it means, which does not reflect well on this syntax proposal, _but_, that by itself cannot be a mandatory feature of a language change, for obvious reasons. "faulty looking syntax": That argument can be used against any language feature, and is hence meaningless. I don't think shortening ".invoke" to just "." is particularly important, but as Mark said, the way lambda-dev is heading, closures are being 'sold', so to speak, as function calls and not merely as inline SAM implementations similar to Anonymous Inner Class literals. Invoking a method with no name, while perhaps surprising to some, is the whole point - if we add closures at all with function types then this idea needs to become part of Joe Java's toolbox, there's no getting around it. --Reinier Zwitserloot On Tue, Feb 16, 2010 at 9:45 AM, Paul Benedict wrote: > My only advice is to keep simplicity and the obvious in mind. I think > the "object.()" syntax violates the principle of least surprise -- I > was surprised to suddenly see method calls without a method name. The > decision does make sense within context, but it is obscure and > esoteric; I think it's too much of a break with the past. If you > really believe that the "object.()" syntax is the best, I'll trust > your decision (i.e., having a doctorate, working on C#, etc.) but I > wanted to raise my objection still. > > On Tue, Feb 16, 2010 at 12:57 AM, Neal Gafter wrote: > > On Mon, Feb 15, 2010 at 10:40 PM, Paul Benedict > wrote: > >> Thanks for the links. To offer my perspective, those who support the > >> syntax don't appear to have a typical background -- usually those who > >> are admitted language designers or are serious hobbyists about > >> languages. If that sounds funny for a "lamda dev" mailing list, > >> there's definitely a spread subscribed here based on responses. > >> Nothing wrong with that. I just think those who opposed better > >> represented the casual Java population. Not being a language designer > >> myself, I empathize with their position. > > > > Yes, I definitely see that point. Experienced language developers are > > often better able to judge how difficult a job it will be for > > developers to internalize a language change based on experience with > > other languages, while programmers often fear things that appear to be > > different, even though that difference may improve their lot once they > > begin using the feature. > > > > In any case, this isn't a democracy. We get to peek over the > > shoulders of folks at Sun every now and then to see their progress. > > > > From reinier at zwitserloot.com Tue Feb 16 05:01:06 2010 From: reinier at zwitserloot.com (Reinier Zwitserloot) Date: Tue, 16 Feb 2010 14:01:06 +0100 Subject: A small question In-Reply-To: <4B7A6EB5.2000309@gmail.com> References: <4B7A6EB5.2000309@gmail.com> Message-ID: <560fb5ed1002160501q32380580x7a9bc784bacb4db3@mail.gmail.com> So, we avoid an explosion of actual types in jsr166's Ops class by.... writing the same types in an awkward new format based on annotations. I think your long argument can be boiled down to something much, much simpler: Is reducing jsr166's Ops class by itself reason enough to add closures to java? I'm guessing that Mark Reinhold thinking for years since the closure wars of '07 about the very topic means this question has been decided already. Whether jsr166 is the only reason or whether it was merely a catalyst, closures are going to happen. I would make a passionate argument in favour of dropping function types altogether (nicely sidestepping all that pain, such as arrays of closures, and negative feeling about the .() operation) in favour of a solid inference algorithm and relying only on SAMs, but it seems that this, too, has already been decided. --Reinier Zwitserloot 2010/2/16 Daniel Latr?moli?re > When reading these threads, I have a small question. > > It seems the key driver for lambda is parallelism (which is a very small > part of their code for many programmers, hopefully for avoiding brain > damaging) and avoiding explosion of number of interfaces (probably the > interfaces of extra166y.Ops.*). > > Given I see a probable consensus in mailing list for implementing lambda > as java.dyn.MethodHandle, I would like to know how not replacing all > these interfaces of extra166y.Ops.* by a MethodHandle in code using > these interfaces? (and not defining any lambda in Java language). > > > --- Some motivations/explanations of my question: > > With this replacement type of lambda will be clearly erased, but you can > introduce some annotations for helping error detection (like JSR 305): > @Signature(returnType=@Type($ReturnType.class), > arguments={@Type(ArgType1.class), ...}) [1]. When, and only when, > experience will be here and usage of lambda sufficiently frequent, it > will be time for changing Java language. > > Currently Java has many other typing features, more frequently used, not > supported by VM, javac and language, like @NotNull (JSR 303) / @Nonnull > (JSR 305). > > I don't think the shortening of source code is a good reason, given it > will be much more generic and useful for all programmers to have > something like type aliasing, which will save much more characters in > all source code (particularly in code using generics, even if diamond > operator has already given some improvements). > > If selection of MethodHandle by name, is really a problem, this will > need method/field literals (currently without many talks on this list). > Adding of method/field literals can be useful, even if I think more > large input would be useful (possibly useful in some validation > annotations, JPQL made by annotations, etc.). This feature seems to have > been rejected for Project Coin. > > Not allowing lambda will reduce tendency to have many nested levels of > context (class -> method -> lambda) with loops (for, while) or > conditional paths (if-then-else) adding their own changing contexts. > Simplicity is always needed for avoiding bugs (programmer need to > understand code, immediately and perfectly!). > > I prefer having a slightly more verbose solution in current Java and > adding some partial but really generic solutions to having a half-baked > solution made only for a specific library hampering all Java > programmers. I prefer some form of progressive enhancements to big > breaking changes. > > We don't know lifetime of this library in Java platform. Personally, I'm > not really considering possible uses of this library: if I need highly > parallel computing framework, I would probably use Java to dynamically > generate optimized OpenCL code. For me, Java is a language for complex > logic programming (CPU) with random-access memory and big caches, not > for small-tasks parallel programming (DSP/GPU) with streaming-access > memory. These use-models of memory are intrinsics of the performances of > these two silicon chips and I don't think personally they will > disappear/merge, even by migrating on the same SoC (cf. embedded). When > you manipulate methods for generating OpenCL code, having names > (identifiers) and signature is very useful, then the java.lang.reflect > model is probably correct; performance is not really needed, because you > don't execute these methods: you create OpenCL code and send it to DSP > for parallel execution. > > > My two cents, > Daniel Latr?moli?re. > > > [1]: Strictly speaking, this solution need some convolutions to allow > native types, given annotations does not support Integer.TYPE as valid > class literal. > > public @interface Signature { > Type returnType() default @Type; // void by default > Type[] argumentTypes() default {}; > } > > public @interface Type { > Class value() default Void.class; > Native nativeType() default Native.CLASS; > } > > public enum Native { > BYTE, SHORT, INT, LONG, FLOAT, DOUBLE, CHAR, CLASS; > } > > Example with "Integer Integer.valueOf(int)": > @Signature(returnType=@Type(Integer.class), > argumentTypes={@Type(nativeType=Native.INT)}) MethodHandle valueOf; > > > > From pbenedict at apache.org Tue Feb 16 07:32:49 2010 From: pbenedict at apache.org (Paul Benedict) Date: Tue, 16 Feb 2010 09:32:49 -0600 Subject: Project Lambda: Java Language Specification draft 0.1.5 In-Reply-To: <560fb5ed1002160455l24a024bey4428d210c26368b1@mail.gmail.com> References: <4B75FEA2.9080904@sun.com> <15e8b9d21002121904u55b47769w9449c113709dc54a@mail.gmail.com> <15e8b9d21002150819n538ebc80t22fdc412e293ce9d@mail.gmail.com> <15e8b9d21002152257s72ee5183p11f458816170fc55@mail.gmail.com> <560fb5ed1002160455l24a024bey4428d210c26368b1@mail.gmail.com> Message-ID: To your definition of the PLS, I don't think there is an official definition. However, exempting the native types of Java, everything is an object (even methods). To have over a decade of Java invoking methods on objects, and then suddenly introducing an unnamed method -- and not requiring a method name -- can't thwart objections under the tarp of "it's a new language feature". On Tue, Feb 16, 2010 at 6:55 AM, Reinier Zwitserloot wrote: > That's not the principle of least surprise. The PLS means: If a significant > chunk of those people using it think a certain library call or construct > does X, and believe this is sufficiently logic to not immediately dive for > the documentation, but this call or construct actually does Y, that's a > violation of PLS. That's _clearly_ not happening here. The only possible > expectation here is that "foo.()" is simply invalid syntax, but this > argument feels like weak tea to me: You can make that argument against just > about every new language feature. From jkuhnert at gmail.com Tue Feb 16 08:14:23 2010 From: jkuhnert at gmail.com (Jesse Kuhnert) Date: Tue, 16 Feb 2010 11:14:23 -0500 Subject: Project Lambda: Java Language Specification draft 0.1.5 In-Reply-To: References: <4B75FEA2.9080904@sun.com> <15e8b9d21002121904u55b47769w9449c113709dc54a@mail.gmail.com> <15e8b9d21002150819n538ebc80t22fdc412e293ce9d@mail.gmail.com> <15e8b9d21002152257s72ee5183p11f458816170fc55@mail.gmail.com> <560fb5ed1002160455l24a024bey4428d210c26368b1@mail.gmail.com> Message-ID: <7926817e1002160814j56903eb6ve79b78f029f157da@mail.gmail.com> Why not? It ~is~ a new feature. In this case it's probably even beneficial to make something look different because it is different. On Tue, Feb 16, 2010 at 10:32 AM, Paul Benedict wrote: > To your definition of the PLS, I don't think there is an official > definition. However, exempting the native types of Java, everything is > an object (even methods). To have over a decade of Java invoking > methods on objects, and then suddenly introducing an unnamed method -- > and not requiring a method name -- can't thwart objections under the > tarp of "it's a new language feature". > > On Tue, Feb 16, 2010 at 6:55 AM, Reinier Zwitserloot > wrote: >> That's not the principle of least surprise. The PLS means: If a significant >> chunk of those people using it think a certain library call or construct >> does X, and believe this is sufficiently logic to not immediately dive for >> the documentation, but this call or construct actually does Y, that's a >> violation of PLS. That's _clearly_ not happening here. The only possible >> expectation here is that "foo.()" is simply invalid syntax, but this >> argument feels like weak tea to me: You can make that argument against just >> about every new language feature. > > From reinier at zwitserloot.com Tue Feb 16 08:21:39 2010 From: reinier at zwitserloot.com (Reinier Zwitserloot) Date: Tue, 16 Feb 2010 17:21:39 +0100 Subject: Project Lambda: Java Language Specification draft 0.1.5 In-Reply-To: References: <4B75FEA2.9080904@sun.com> <15e8b9d21002121904u55b47769w9449c113709dc54a@mail.gmail.com> <15e8b9d21002150819n538ebc80t22fdc412e293ce9d@mail.gmail.com> <15e8b9d21002152257s72ee5183p11f458816170fc55@mail.gmail.com> <560fb5ed1002160455l24a024bey4428d210c26368b1@mail.gmail.com> Message-ID: <560fb5ed1002160821gf019dedy4957e4df24a5b249@mail.gmail.com> Why can't I thwart them? You're complaining about something being new and unfamiliar to java programmers, and I'm replying with: Yes. That's because it's new. Seems tautological when you put it that way, doesn't it? What's your opinion about the notion that the way lambda-dev is going, closures are a new and unique concept, and Joe Java is going to have to understand this concept, which very much includes the idea of having an 'unnamed method', in order to work with Java7? If you agree, it would seem that avoiding x.() syntax is not going to make any difference in the amount of understanding Java Joe needs to do, and thus makes that particular objection to the syntax irrelevant. In other words, I'm with Jesse Kuhnert on this one. It IS different, so making it look different is good. --Reinier Zwitserloot On Tue, Feb 16, 2010 at 4:32 PM, Paul Benedict wrote: > To your definition of the PLS, I don't think there is an official > definition. However, exempting the native types of Java, everything is > an object (even methods). To have over a decade of Java invoking > methods on objects, and then suddenly introducing an unnamed method -- > and not requiring a method name -- can't thwart objections under the > tarp of "it's a new language feature". > > On Tue, Feb 16, 2010 at 6:55 AM, Reinier Zwitserloot > wrote: > > That's not the principle of least surprise. The PLS means: If a > significant > > chunk of those people using it think a certain library call or construct > > does X, and believe this is sufficiently logic to not immediately dive > for > > the documentation, but this call or construct actually does Y, that's a > > violation of PLS. That's _clearly_ not happening here. The only possible > > expectation here is that "foo.()" is simply invalid syntax, but this > > argument feels like weak tea to me: You can make that argument against > just > > about every new language feature. > From oallouch at free.fr Tue Feb 16 08:26:58 2010 From: oallouch at free.fr (Olivier Allouch) Date: Tue, 16 Feb 2010 17:26:58 +0100 Subject: Project Lambda: Java Language Specification draft 0.1.5 In-Reply-To: References: <4B75FEA2.9080904@sun.com> <15e8b9d21002121904u55b47769w9449c113709dc54a@mail.gmail.com> <15e8b9d21002150819n538ebc80t22fdc412e293ce9d@mail.gmail.com> <15e8b9d21002152257s72ee5183p11f458816170fc55@mail.gmail.com> <560fb5ed1002160455l24a024bey4428d210c26368b1@mail.gmail.com> Message-ID: <4B7AC752.6020400@free.fr> Hi, I also think the foo.() notation is the worst of 3. I like the "function" notation: foo() and wouldn't mind the last one: foo.invoke() I like everything "function" actually. Le 16/02/2010 16:32, Paul Benedict a ?crit : > To your definition of the PLS, I don't think there is an official > definition. However, exempting the native types of Java, everything is > an object (even methods). To have over a decade of Java invoking > methods on objects, and then suddenly introducing an unnamed method -- > and not requiring a method name -- can't thwart objections under the > tarp of "it's a new language feature". > > On Tue, Feb 16, 2010 at 6:55 AM, Reinier Zwitserloot > wrote: > >> That's not the principle of least surprise. The PLS means: If a significant >> chunk of those people using it think a certain library call or construct >> does X, and believe this is sufficiently logic to not immediately dive for >> the documentation, but this call or construct actually does Y, that's a >> violation of PLS. That's _clearly_ not happening here. The only possible >> expectation here is that "foo.()" is simply invalid syntax, but this >> argument feels like weak tea to me: You can make that argument against just >> about every new language feature. >> > From pbenedict at apache.org Tue Feb 16 08:58:15 2010 From: pbenedict at apache.org (Paul Benedict) Date: Tue, 16 Feb 2010 10:58:15 -0600 Subject: Project Lambda: Java Language Specification draft 0.1.5 In-Reply-To: <560fb5ed1002160821gf019dedy4957e4df24a5b249@mail.gmail.com> References: <4B75FEA2.9080904@sun.com> <15e8b9d21002121904u55b47769w9449c113709dc54a@mail.gmail.com> <15e8b9d21002150819n538ebc80t22fdc412e293ce9d@mail.gmail.com> <15e8b9d21002152257s72ee5183p11f458816170fc55@mail.gmail.com> <560fb5ed1002160455l24a024bey4428d210c26368b1@mail.gmail.com> <560fb5ed1002160821gf019dedy4957e4df24a5b249@mail.gmail.com> Message-ID: On Tue, Feb 16, 2010 at 10:21 AM, Reinier Zwitserloot wrote: > Why can't I thwart them? > You're complaining about something being new and unfamiliar to java > programmers, and I'm replying with: Yes. That's because it's new. Seems > tautological when you put it that way, doesn't it? Invoking a method on an object is not new. It is not ontologically different. We haven't ventured into the unknown here. You still have an object and you still have a method. I grant you the situation is different -- Java hasn't had function objects before -- but it's not a radical departure: an object and a method still exist. The big difference is the anonymity of the method name, yet I am not convinced of the proposed solution. A general name will do just fine and keep consistency with historical Java syntax. The proposed syntax is radical, and what for? Because we can? It is unjustifiable because the same paradigm of invoking a method on an object still exist. Paul From peter.levart at marand.si Tue Feb 16 09:44:43 2010 From: peter.levart at marand.si (Peter Levart) Date: Tue, 16 Feb 2010 18:44:43 +0100 Subject: Project Lambda: Java Language Specification draft 0.1.5 In-Reply-To: References: <4B75FEA2.9080904@sun.com> <15e8b9d21002121904u55b47769w9449c113709dc54a@mail.gmail.com> Message-ID: <201002161844.43108.peter.levart@marand.si> On Saturday 13 February 2010 15:25:43 Paul Benedict wrote: > What happened to the self.invoke() syntax? A barrage of people came > out saying self.() looks like a syntax error (I agree); I don't recall > anyone advocating the syntax after the first draft was released. > What about the following combo: FunctionType: ReturnType #(ParameterTypes) Throws LambaExpression: #(Parameters) Block LambdaInvocationExpression: FnTypeExpression . #(ParameterExpressions) The magic '#' is allways where a method name is expected in established syntax. I think this wouldn't be hard to explain to newbies. Regards, Peter From neal at gafter.com Tue Feb 16 10:41:35 2010 From: neal at gafter.com (Neal Gafter) Date: Tue, 16 Feb 2010 10:41:35 -0800 Subject: Project Lambda: Java Language Specification draft 0.1.5 In-Reply-To: <201002161844.43108.peter.levart@marand.si> References: <4B75FEA2.9080904@sun.com> <15e8b9d21002121904u55b47769w9449c113709dc54a@mail.gmail.com> <201002161844.43108.peter.levart@marand.si> Message-ID: <15e8b9d21002161041y1d9b3643n20a762da294dc1a6@mail.gmail.com> Because throws clauses can have commas in them, this grammar is ambiguous. On Tue, Feb 16, 2010 at 9:44 AM, Peter Levart wrote: > On Saturday 13 February 2010 15:25:43 Paul Benedict wrote: >> What happened to the self.invoke() syntax? A barrage of people came >> out saying self.() looks like a syntax error (I agree); I don't recall >> anyone advocating the syntax after the first draft was released. >> > > What about the following combo: > > FunctionType: > ? ?ReturnType #(ParameterTypes) Throws > > LambaExpression: > ? ?#(Parameters) Block > > LambdaInvocationExpression: > ? ?FnTypeExpression . #(ParameterExpressions) > > > The magic '#' is allways where a method name is expected in established syntax. I think this wouldn't be hard to explain to newbies. > > > Regards, Peter > > From schulz at the-loom.de Tue Feb 16 10:52:52 2010 From: schulz at the-loom.de (Stefan Schulz) Date: Tue, 16 Feb 2010 19:52:52 +0100 Subject: Function type -postfix In-Reply-To: <201002090914.35153.peter.levart@marand.si> References: <4B70B711.5050305@univ-mlv.fr> <4B710EED.6080006@the-loom.de> <201002090914.35153.peter.levart@marand.si> Message-ID: <4B7AE984.5090204@the-loom.de> On 09.02.2010 09:14, Peter Levart wrote: > On Tuesday 09 February 2010 08:29:49 Stefan Schulz wrote: >> >> ##int(int)(int) or #int(#int(int)) [[int(int)](int)] or >> [int([int(int)])] > > I have one objection to this syntax - it's too noisy on > parens/brackets. The above example shows that even prefix notation is > easier on the eye than your proposed bracketed syntax. It's nice if a > pair of brackets or parens enclose the whole construct, but then it > should not require a second pair of parens/brackets inside. So I had a glimpse at Neal's blog, whose suggestion would result in: (int) -> (int) -> int and ((int) -> int) -> int for the above examples. To me, it's hard to spot the difference, which is in shifting braces a bit. I think that defining function types not necessarily has to resemble lambdas (although I thought this to be advantageous in the first). So having a syntax for function types that resembles the mathematical definition of a function could work quite well. I still advocate for some parentheses ;) Mathematics (sort of): f: int -> (f: int -> int) and f: (f: int -> int) -> int Java function types, for example: (:int -> (:int -> int)) and (:(:int -> int) -> int) While still the same amount of parens, this looks much clearer to me. But I might be wrong again. One could drop the colon, of course, but here's a smiley within ... actually it kind of resembles the f: if one squints. ;) Would make the following syntax: FunctionType: (: TypeList_opt -> ResultType Throws_opt ) This also is very easy to distinct from concrete lambdas. To return to Neal's blog and his example: static (:T->(:U->V throws X)) curry((:T,U->V throws X) function) { return #(T t)(#(U u)(function.(t,u))); } Cheers, Stefan From john at milsson.nu Tue Feb 16 11:26:15 2010 From: john at milsson.nu (John Nilsson) Date: Tue, 16 Feb 2010 20:26:15 +0100 Subject: Project Lambda: Java Language Specification draft 0.1.5 In-Reply-To: References: <4B75FEA2.9080904@sun.com> <15e8b9d21002150819n538ebc80t22fdc412e293ce9d@mail.gmail.com> <15e8b9d21002152257s72ee5183p11f458816170fc55@mail.gmail.com> <560fb5ed1002160455l24a024bey4428d210c26368b1@mail.gmail.com> <560fb5ed1002160821gf019dedy4957e4df24a5b249@mail.gmail.com> Message-ID: On Tue, Feb 16, 2010 at 5:58 PM, Paul Benedict wrote: > Invoking a method on an object is not new. It is not ontologically > different. We haven't ventured into the unknown here. You still have > an object and you still have a method. I grant you the situation is > different -- Java hasn't had function objects before -- but it's not a > radical departure: an object and a method still exist. Is it really an object with an anonymous method? Or is it just a method? Or maybe just a lambda? Or a closure? I can see how object with an anonymous method kind of fits with the current state of things. OTOH if there is no real need for a function-object duality, why force that semantics on it? In my mind one argument against functions as objects is that it implies a class implementing that function which could imply restrictions on the runtime representation of the code in question that might not be warranted. BR, John From howard.lovatt at iee.org Tue Feb 16 20:58:53 2010 From: howard.lovatt at iee.org (Howard Lovatt) Date: Wed, 17 Feb 2010 15:58:53 +1100 Subject: Function types versus arrays In-Reply-To: <15e8b9d21002151448i2ca40f8fpbf40c70caffeab2c@mail.gmail.com> References: <3dd3f56a1002081452o36d2b695kc766842ec2e8348b@mail.gmail.com> <3dd3f56a1002081733x786e329at5d2f273dcf3cec69@mail.gmail.com> <15e8b9d21002081754p428832e5mff96cc9586c6a47b@mail.gmail.com> <3dd3f56a1002081944p6d6e0119g181fb968d7da747a@mail.gmail.com> <15e8b9d21002082129i21a209cel925ad8b6ba3313cd@mail.gmail.com> <3dd3f56a1002082201l304a5473ge2859fcc52032913@mail.gmail.com> <560fb5ed1002122022u27f0ed29x44d7f821ddb44ab5@mail.gmail.com> <15e8b9d21002122239p158cf4d3vfc45faaa2b48dced@mail.gmail.com> <3dd3f56a1002151357m2d5b13bs5e078c69d604e31b@mail.gmail.com> <15e8b9d21002151448i2ca40f8fpbf40c70caffeab2c@mail.gmail.com> Message-ID: <3dd3f56a1002162058j7f6e248cj197705273c2054b3@mail.gmail.com> Hi Neal, On 16 February 2010 09:48, Neal Gafter wrote: > On Mon, Feb 15, 2010 at 1:57 PM, Howard Lovatt wrote: >> Neal Gafter gave an array example on his blog: >> >> http://gafter.blogspot.com/ >> >> Using: >> >> http://www.artima.com/weblogs/viewpost.jsp?thread=278567 >> >> It is translated to: > > That's just not true. ?Your post does not give any hint as to how > throw clauses or exceptions in function types are to affect the > translation. ?The word "throws" doesn't even appear anywhere in your > post at all. The post notes in the ToDo section that checked exceptions could be added (last point section 2.3). The reason for not detailing how was twofold: 1. Checked exceptions weren't in the original stawman, hence not in my proposal (which was based on the strawman). 2. As I am certain you will appreciate, you really want generics to be extended to allow a vararg like systax for checked exceptions and that the vararg list of exceptions could potentially be empty. I will outline below how the scheme might be extended, but first as an aside to comment that: you seem to equate an incomplete specification, in this case of checked exceptions noted as potentially incomplete, as a fundamental flaw. To me an incomplete specification does not mean it is flawed, it means it's incomplete. >?But I'm glad to see that you're now considering some of > the areas where your specification was completely lacking before. > > How would your translation scheme handle the code below? ?Your post > contains not a single hint. As mentioned above I would need extensions to generics, but then so would all schemes. For the moment lets assume that generics were extended so that you could have a vararg type construct e.g.: interface _Callable_0 { R call() throws E; } // E is a, potentially empty, set of Throwables (The above is an extension of section 4.1.) The set of Throwables, E, needs a total ordering (I haven't thought through a suitable sorting technique - I will *assume* that a total ordering is possible and also *assume* that a set like data structure is suitable). The hierarchy of your Exceptions are: Throwable -> Exception -> IOException -> ChangedCharSetException Throwable -> Exception -> IOException -> UnknownHostException Throwable -> Exception -> SAXException -> SAXParseException Throwable -> Exception -> AWTException *Assuming* that a suitable total ordering is depth from Object followed by alphanumerical sorting, then the total ordering for these exceptions is: Throwable, Exception, AWTException, IOException, SAXException, ChangedCharSetException, SAXParseException, UnkownHostException Using this ordering the type: #void()(throws ChangedCharSetException, UnknownHostException, > SAXParseException) is _Callable_void_throws_ChangedCharSetException_SAXParseException_UnkownHostException (extension of section 5.2.1), which is in turn: abstract class _Callable_void_throws_ChangedCharSetException_SAXParseException_UnkownHostException implements _Callable_0 { public Void call() throws ChangedCharSetException, SAXParseException, UnkownHostException { callPrimitive(); return null; } public abstract void callPrimitive() throws ChangedCharSetException, SAXParseException, UnkownHostException; } This is an extension of section 4.2.2 and: #void()(throws ChangedCharSetException, UnknownHostException) is _Callable_void_throws_ChangedCharSetException_UnkownHostException which is: abstract class _Callable_void_throws_ChangedCharSetException_UnkownHostException implements _Callable_0 { public Void call() throws ChangedCharSetException, UnkownHostException { callPrimitive(); return null; } public abstract void callPrimitive() throws ChangedCharSetException, UnkownHostException; } and: #void()(throws IOException, AWTException, SAXException) is _Callable_void_throws_AWTException_IOException_SAXException (note ordering), which is: abstract class _Callable_void_throws_AWTException_IOException_SAXException implements _Callable_0 { public Void call() throws AWTException, IOException, SAXException { callPrimitive(); return null; } public abstract void callPrimitive() throws AWTException, IOException, SAXException; } Then: > ? #void()(throws ChangedCharSetException, UnknownHostException, > SAXParseException)[] array1 = > ? ? ?new #void()(throws ChangedCharSetException, UnknownHostException)[1]; Becomes: _Callable_void_throws_ChangedCharSetException_SAXParseException_UnkownHostException[] array1 = _Array_From_void_throws_ChangedCharSetException_UnkownHostException__To_void_throws_ChangedCharSetException_SAXParseException_UnkownHostException.instance( new _Callable_void_throws_ChangedCharSetException_UnkownHostException[1] ); (the above translation is an extension of sections 5.2.2, 5.4, and 4.4) and: > ? #void()(throws IOException, AWTException, SAXException)[] array2 = array1; becomes: _Callable_void_throws_AWTException_IOException_SAXException[] array2 = _Array_From_void_throws_ChangedCharSetException_SAXParseException_UnkownHostException__To_void_throws_AWTException_IOException_SAXException.instance( array1 ); Therefore I think the technique, suitably extended for checked exceptions, will work as expected for the example. Cheers, -- Howard. > > Cheers, > Neal > > ______________________________________________________________________ > This email has been scanned by the MessageLabs Email Security System. > For more information please visit http://www.messagelabs.com/email > ______________________________________________________________________ > -- -- Howard. From neal at gafter.com Tue Feb 16 21:26:40 2010 From: neal at gafter.com (Neal Gafter) Date: Tue, 16 Feb 2010 21:26:40 -0800 Subject: Function types versus arrays In-Reply-To: <3dd3f56a1002162058j7f6e248cj197705273c2054b3@mail.gmail.com> References: <3dd3f56a1002081452o36d2b695kc766842ec2e8348b@mail.gmail.com> <15e8b9d21002081754p428832e5mff96cc9586c6a47b@mail.gmail.com> <3dd3f56a1002081944p6d6e0119g181fb968d7da747a@mail.gmail.com> <15e8b9d21002082129i21a209cel925ad8b6ba3313cd@mail.gmail.com> <3dd3f56a1002082201l304a5473ge2859fcc52032913@mail.gmail.com> <560fb5ed1002122022u27f0ed29x44d7f821ddb44ab5@mail.gmail.com> <15e8b9d21002122239p158cf4d3vfc45faaa2b48dced@mail.gmail.com> <3dd3f56a1002151357m2d5b13bs5e078c69d604e31b@mail.gmail.com> <15e8b9d21002151448i2ca40f8fpbf40c70caffeab2c@mail.gmail.com> <3dd3f56a1002162058j7f6e248cj197705273c2054b3@mail.gmail.com> Message-ID: <15e8b9d21002162126l26f89047q22d25e953d55be6@mail.gmail.com> Howard- Can you please show the implementation of _Array_From_void_throws_ChangedCharSetException_SAXParseException_UnkownHostException__To_void_throws_AWTException_IOException_SAXException.instance? Cheers, Neal On Tue, Feb 16, 2010 at 8:58 PM, Howard Lovatt wrote: > (the above translation is an extension of sections 5.2.2, 5.4, and 4.4) and: > >> ? #void()(throws IOException, AWTException, SAXException)[] array2 = array1; > > becomes: > > _Callable_void_throws_AWTException_IOException_SAXException[] array2 = > _Array_From_void_throws_ChangedCharSetException_SAXParseException_UnkownHostException__To_void_throws_AWTException_IOException_SAXException.instance( > array1 ); From peter.levart at marand.si Wed Feb 17 00:28:05 2010 From: peter.levart at marand.si (Peter Levart) Date: Wed, 17 Feb 2010 09:28:05 +0100 Subject: Project Lambda: Java Language Specification draft 0.1.5 In-Reply-To: <15e8b9d21002161041y1d9b3643n20a762da294dc1a6@mail.gmail.com> References: <4B75FEA2.9080904@sun.com> <201002161844.43108.peter.levart@marand.si> <15e8b9d21002161041y1d9b3643n20a762da294dc1a6@mail.gmail.com> Message-ID: <201002170928.05962.peter.levart@marand.si> Ok, I wasn't precise enough - I meant Throws to use '|' to delimit exception types. The point of combo I wanted to emphasize was "consistency". Like for example with arrays we have: Type[] // representing array type new Type[IntExpression] // representing array creation expression ArrayExpression[IntExpression] // representing array element dereferencing expression All three constructs are somehow similar and therefore easy to comprehend. Regards, Peter On Tuesday 16 February 2010 19:41:35 Neal Gafter wrote: > Because throws clauses can have commas in them, this grammar is ambiguous. > > On Tue, Feb 16, 2010 at 9:44 AM, Peter Levart wrote: > > On Saturday 13 February 2010 15:25:43 Paul Benedict wrote: > >> What happened to the self.invoke() syntax? A barrage of people came > >> out saying self.() looks like a syntax error (I agree); I don't recall > >> anyone advocating the syntax after the first draft was released. > >> > > > > What about the following combo: > > > > FunctionType: > > ReturnType #(ParameterTypes) Throws > > > > LambaExpression: > > #(Parameters) Block > > > > LambdaInvocationExpression: > > FnTypeExpression . #(ParameterExpressions) > > > > > > The magic '#' is allways where a method name is expected in established syntax. I think this wouldn't be hard to explain to newbies. > > > > > > Regards, Peter > > > > > From jjb at google.com Wed Feb 17 09:25:01 2010 From: jjb at google.com (Joshua Bloch) Date: Wed, 17 Feb 2010 09:25:01 -0800 Subject: Function types versus arrays In-Reply-To: <15e8b9d21002162126l26f89047q22d25e953d55be6@mail.gmail.com> References: <3dd3f56a1002081452o36d2b695kc766842ec2e8348b@mail.gmail.com> <3dd3f56a1002081944p6d6e0119g181fb968d7da747a@mail.gmail.com> <15e8b9d21002082129i21a209cel925ad8b6ba3313cd@mail.gmail.com> <3dd3f56a1002082201l304a5473ge2859fcc52032913@mail.gmail.com> <560fb5ed1002122022u27f0ed29x44d7f821ddb44ab5@mail.gmail.com> <15e8b9d21002122239p158cf4d3vfc45faaa2b48dced@mail.gmail.com> <3dd3f56a1002151357m2d5b13bs5e078c69d604e31b@mail.gmail.com> <15e8b9d21002151448i2ca40f8fpbf40c70caffeab2c@mail.gmail.com> <3dd3f56a1002162058j7f6e248cj197705273c2054b3@mail.gmail.com> <15e8b9d21002162126l26f89047q22d25e953d55be6@mail.gmail.com> Message-ID: <17b2302a1002170925i28e0e7ceu33c30f9c4172998a@mail.gmail.com> I'm not sure why you folks are worrying so much about implementation. At this point, I believe we should be searching for the best spec. Then we can worry about implementation. If we can't implement our desired spec, then we can talk about how to weaken it. Josh On Tue, Feb 16, 2010 at 9:26 PM, Neal Gafter wrote: > Howard- > > Can you please show the implementation of > > _Array_From_void_throws_ChangedCharSetException_SAXParseException_UnkownHostException__To_void_throws_AWTException_IOException_SAXException.instance? > > Cheers, > Neal > > On Tue, Feb 16, 2010 at 8:58 PM, Howard Lovatt > wrote: > > (the above translation is an extension of sections 5.2.2, 5.4, and 4.4) > and: > > > >> #void()(throws IOException, AWTException, SAXException)[] array2 = > array1; > > > > becomes: > > > > _Callable_void_throws_AWTException_IOException_SAXException[] array2 = > > > _Array_From_void_throws_ChangedCharSetException_SAXParseException_UnkownHostException__To_void_throws_AWTException_IOException_SAXException.instance( > > array1 ); > > From neal at gafter.com Wed Feb 17 09:39:25 2010 From: neal at gafter.com (Neal Gafter) Date: Wed, 17 Feb 2010 09:39:25 -0800 Subject: Function types versus arrays In-Reply-To: <17b2302a1002170925i28e0e7ceu33c30f9c4172998a@mail.gmail.com> References: <3dd3f56a1002081452o36d2b695kc766842ec2e8348b@mail.gmail.com> <15e8b9d21002082129i21a209cel925ad8b6ba3313cd@mail.gmail.com> <3dd3f56a1002082201l304a5473ge2859fcc52032913@mail.gmail.com> <560fb5ed1002122022u27f0ed29x44d7f821ddb44ab5@mail.gmail.com> <15e8b9d21002122239p158cf4d3vfc45faaa2b48dced@mail.gmail.com> <3dd3f56a1002151357m2d5b13bs5e078c69d604e31b@mail.gmail.com> <15e8b9d21002151448i2ca40f8fpbf40c70caffeab2c@mail.gmail.com> <3dd3f56a1002162058j7f6e248cj197705273c2054b3@mail.gmail.com> <15e8b9d21002162126l26f89047q22d25e953d55be6@mail.gmail.com> <17b2302a1002170925i28e0e7ceu33c30f9c4172998a@mail.gmail.com> Message-ID: <15e8b9d21002170939n43e51bfeie7fa2a01a5cab4c9@mail.gmail.com> On Wed, Feb 17, 2010 at 9:25 AM, Joshua Bloch wrote: > I'm not sure why you folks are worrying so much about implementation. ?At > this point, I believe we should be searching for the best spec. ?Then we can > worry about implementation. ?If we can't implement our desired spec, then we > can talk about how to weaken it. Josh- I desire a specification that is a consistent extension of the existing Java language. Interactions with the existing language specification (not implementation) necessarily implies certain limitations to the spec. Howard asserts otherwise, and I'm trying to help him see how his attempts to work around those limitations are fruitless. Moreover, understanding and accepting those limitations can help improve the form of the specification; see, for example, . Cheers, Neal From Alex.Buckley at Sun.COM Wed Feb 17 13:32:46 2010 From: Alex.Buckley at Sun.COM (Alex Buckley) Date: Wed, 17 Feb 2010 13:32:46 -0800 Subject: Function types versus arrays In-Reply-To: <15e8b9d21002170939n43e51bfeie7fa2a01a5cab4c9@mail.gmail.com> References: <3dd3f56a1002081452o36d2b695kc766842ec2e8348b@mail.gmail.com> <15e8b9d21002082129i21a209cel925ad8b6ba3313cd@mail.gmail.com> <3dd3f56a1002082201l304a5473ge2859fcc52032913@mail.gmail.com> <560fb5ed1002122022u27f0ed29x44d7f821ddb44ab5@mail.gmail.com> <15e8b9d21002122239p158cf4d3vfc45faaa2b48dced@mail.gmail.com> <3dd3f56a1002151357m2d5b13bs5e078c69d604e31b@mail.gmail.com> <15e8b9d21002151448i2ca40f8fpbf40c70caffeab2c@mail.gmail.com> <3dd3f56a1002162058j7f6e248cj197705273c2054b3@mail.gmail.com> <15e8b9d21002162126l26f89047q22d25e953d55be6@mail.gmail.com> <17b2302a1002170925i28e0e7ceu33c30f9c4172998a@mail.gmail.com> <15e8b9d21002170939n43e51bfeie7fa2a01a5cab4c9@mail.gmail.com> Message-ID: <4B7C607E.9090104@sun.com> Neal Gafter wrote: > I desire a specification that is a consistent extension of the > existing Java language. Interactions with the existing language > specification (not implementation) necessarily implies certain > limitations to the spec. Howard asserts otherwise, and I'm trying to > help him see how his attempts to work around those limitations are > fruitless. I find it very difficult to follow Howard's scheme, since some is at Artima and some is sprinkled around this mailing list. I am surprised at the assumptions that suddenly appear from time to time, e.g. "For the moment lets assume that generics were extended so that you could have a vararg type construct". I worry about the boot loader having to auto-generate classes, since loaders are already under heavy modification for module systems. And it all feels similar to NextGen's approach to reification, though no proper comparison has ever been made. For these reasons, I welcome Neal's continued analysis of the scheme on lambda-dev, even if it sometimes looks like the implementation focus is getting out of hand. Alex From jjb at google.com Wed Feb 17 14:39:51 2010 From: jjb at google.com (Joshua Bloch) Date: Wed, 17 Feb 2010 14:39:51 -0800 Subject: Function types versus arrays In-Reply-To: <15e8b9d21002170939n43e51bfeie7fa2a01a5cab4c9@mail.gmail.com> References: <3dd3f56a1002081452o36d2b695kc766842ec2e8348b@mail.gmail.com> <3dd3f56a1002082201l304a5473ge2859fcc52032913@mail.gmail.com> <560fb5ed1002122022u27f0ed29x44d7f821ddb44ab5@mail.gmail.com> <15e8b9d21002122239p158cf4d3vfc45faaa2b48dced@mail.gmail.com> <3dd3f56a1002151357m2d5b13bs5e078c69d604e31b@mail.gmail.com> <15e8b9d21002151448i2ca40f8fpbf40c70caffeab2c@mail.gmail.com> <3dd3f56a1002162058j7f6e248cj197705273c2054b3@mail.gmail.com> <15e8b9d21002162126l26f89047q22d25e953d55be6@mail.gmail.com> <17b2302a1002170925i28e0e7ceu33c30f9c4172998a@mail.gmail.com> <15e8b9d21002170939n43e51bfeie7fa2a01a5cab4c9@mail.gmail.com> Message-ID: <17b2302a1002171439o467b40f0lf46d96f004b2feca@mail.gmail.com> Neal, I'm having a bit of trouble understanding this. I'm not endorsing any implementation scheme, but you say that this program necessarily breaks the type system: *public void main(String[] args) { #void()[] arrayOfFunction = new #void()[1]; Object[] objectArray = arrayOfFunction; objectArray[0] = #(){ throw new IOException(); }; // ** arrayOfFunction[0].(); // invoke the function out of the array }* It would appear that the starred line should generate an ArrayStoreException at runtime, as #void()(throws IOException) isn't a subtype of #void(). I understand that it might take a sophisticated implementation to enforce this. What am I missing? Josh On Wed, Feb 17, 2010 at 9:39 AM, Neal Gafter wrote: > On Wed, Feb 17, 2010 at 9:25 AM, Joshua Bloch wrote: > > I'm not sure why you folks are worrying so much about implementation. At > > this point, I believe we should be searching for the best spec. Then we > can > > worry about implementation. If we can't implement our desired spec, then > we > > can talk about how to weaken it. > > Josh- > > I desire a specification that is a consistent extension of the > existing Java language. Interactions with the existing language > specification (not implementation) necessarily implies certain > limitations to the spec. Howard asserts otherwise, and I'm trying to > help him see how his attempts to work around those limitations are > fruitless. > > Moreover, understanding and accepting those limitations can help > improve the form of the specification; see, for example, > >. > > Cheers, > Neal > From neal at gafter.com Wed Feb 17 14:48:27 2010 From: neal at gafter.com (Neal Gafter) Date: Wed, 17 Feb 2010 14:48:27 -0800 Subject: Function types versus arrays In-Reply-To: <17b2302a1002171439o467b40f0lf46d96f004b2feca@mail.gmail.com> References: <3dd3f56a1002081452o36d2b695kc766842ec2e8348b@mail.gmail.com> <560fb5ed1002122022u27f0ed29x44d7f821ddb44ab5@mail.gmail.com> <15e8b9d21002122239p158cf4d3vfc45faaa2b48dced@mail.gmail.com> <3dd3f56a1002151357m2d5b13bs5e078c69d604e31b@mail.gmail.com> <15e8b9d21002151448i2ca40f8fpbf40c70caffeab2c@mail.gmail.com> <3dd3f56a1002162058j7f6e248cj197705273c2054b3@mail.gmail.com> <15e8b9d21002162126l26f89047q22d25e953d55be6@mail.gmail.com> <17b2302a1002170925i28e0e7ceu33c30f9c4172998a@mail.gmail.com> <15e8b9d21002170939n43e51bfeie7fa2a01a5cab4c9@mail.gmail.com> <17b2302a1002171439o467b40f0lf46d96f004b2feca@mail.gmail.com> Message-ID: <15e8b9d21002171448p7177fe3crfd64a2846307c13d@mail.gmail.com> On Wed, Feb 17, 2010 at 2:39 PM, Joshua Bloch wrote: > Neal, > I'm having a bit of trouble understanding this. ?I'm not endorsing any > implementation scheme, but you say that this program necessarily breaks the > type system: > > public void main(String[] args) { > #void()[] arrayOfFunction = new #void()[1]; > Object[] objectArray = arrayOfFunction; > objectArray[0] = #(){ throw new IOException(); }; // ** > arrayOfFunction[0].(); // invoke the function out of the array > } > > It would appear that the starred line should generate an ArrayStoreException > at runtime, as #void()(throws IOException) isn't a subtype of #void(). ?I > understand that it might take a sophisticated implementation to enforce > this. What am I missing? > ?? ? ? ? Josh Presuming the source were legal, that's exactly right. We're exploring how such a "sophisticated implementation" breaks compatibility with the existing language specification (thus the "implementation" focus of the discussion), with the consequence that the source must not be allowed. Cheers, Neal From jjb at google.com Wed Feb 17 14:59:32 2010 From: jjb at google.com (Joshua Bloch) Date: Wed, 17 Feb 2010 14:59:32 -0800 Subject: Function types versus arrays In-Reply-To: <15e8b9d21002171448p7177fe3crfd64a2846307c13d@mail.gmail.com> References: <3dd3f56a1002081452o36d2b695kc766842ec2e8348b@mail.gmail.com> <15e8b9d21002122239p158cf4d3vfc45faaa2b48dced@mail.gmail.com> <3dd3f56a1002151357m2d5b13bs5e078c69d604e31b@mail.gmail.com> <15e8b9d21002151448i2ca40f8fpbf40c70caffeab2c@mail.gmail.com> <3dd3f56a1002162058j7f6e248cj197705273c2054b3@mail.gmail.com> <15e8b9d21002162126l26f89047q22d25e953d55be6@mail.gmail.com> <17b2302a1002170925i28e0e7ceu33c30f9c4172998a@mail.gmail.com> <15e8b9d21002170939n43e51bfeie7fa2a01a5cab4c9@mail.gmail.com> <17b2302a1002171439o467b40f0lf46d96f004b2feca@mail.gmail.com> <15e8b9d21002171448p7177fe3crfd64a2846307c13d@mail.gmail.com> Message-ID: <17b2302a1002171459n6212bb2em458174add2f83afd@mail.gmail.com> On Wed, Feb 17, 2010 at 2:48 PM, Neal Gafter wrote: > On Wed, Feb 17, 2010 at 2:39 PM, Joshua Bloch wrote: > > Neal, > > I'm having a bit of trouble understanding this. I'm not endorsing any > > implementation scheme, but you say that this program necessarily breaks > the > > type system: > > > > public void main(String[] args) { > > #void()[] arrayOfFunction = new #void()[1]; > > Object[] objectArray = arrayOfFunction; > > objectArray[0] = #(){ throw new IOException(); }; // ** > > arrayOfFunction[0].(); // invoke the function out of the array > > } > > > > It would appear that the starred line should generate an > ArrayStoreException > > at runtime, as #void()(throws IOException) isn't a subtype of #void(). I > > understand that it might take a sophisticated implementation to enforce > > this. What am I missing? > > Josh > > Presuming the source were legal, that's exactly right. We're > exploring how such a "sophisticated implementation" breaks > compatibility with the existing language specification (thus the > "implementation" focus of the discussion), with the consequence that > the source must not be allowed. > I really don't understand this. Earlier on, Alex said that we should refrain from thinking too hard about the implementation at this point, and I still believe that to be true. If you could show that some interaction of the proposed syntax and semantics interacted poorly with the existing type system, that would be a big problem, but I haven't seen such an argument that didn't presuppose a particular implementation of function types. I really do think we should design the best spec (syntax and semantics) and then see if it's implementable. Earlier, I suggested that we should allow arrays of function types if we can. \Otherwise, we're damning function types to life as second-class citizens of the Java type system. We're also screwing up variable-arity function typed args, as varargs is a leaky abstraction built atop arrays. This isn't a theoretical concern; it impacts reasonable programs. For example, consider a method that takes a bunch of #int(int) functions and returns their composition: #int(int) compose(#int(int)... fns) { } This should definitely be legal, and shouldn't generate a warning (especially given that we're providing a means to eliminate this sort of warning on generic varargs parameters in Java 7). Josh From neal at gafter.com Wed Feb 17 15:10:59 2010 From: neal at gafter.com (Neal Gafter) Date: Wed, 17 Feb 2010 15:10:59 -0800 Subject: Function types versus arrays In-Reply-To: <17b2302a1002171459n6212bb2em458174add2f83afd@mail.gmail.com> References: <3dd3f56a1002081452o36d2b695kc766842ec2e8348b@mail.gmail.com> <3dd3f56a1002151357m2d5b13bs5e078c69d604e31b@mail.gmail.com> <15e8b9d21002151448i2ca40f8fpbf40c70caffeab2c@mail.gmail.com> <3dd3f56a1002162058j7f6e248cj197705273c2054b3@mail.gmail.com> <15e8b9d21002162126l26f89047q22d25e953d55be6@mail.gmail.com> <17b2302a1002170925i28e0e7ceu33c30f9c4172998a@mail.gmail.com> <15e8b9d21002170939n43e51bfeie7fa2a01a5cab4c9@mail.gmail.com> <17b2302a1002171439o467b40f0lf46d96f004b2feca@mail.gmail.com> <15e8b9d21002171448p7177fe3crfd64a2846307c13d@mail.gmail.com> <17b2302a1002171459n6212bb2em458174add2f83afd@mail.gmail.com> Message-ID: <15e8b9d21002171510ldb6ad0dh4e85f866cd96fbc2@mail.gmail.com> On Wed, Feb 17, 2010 at 2:59 PM, Joshua Bloch wrote: > I really don't understand this. ?Earlier on, Alex said that we should > refrain from thinking too hard about the implementation at this point, and I > still believe that to be true. ?If you could show that some interaction of > the proposed syntax and semantics interacted poorly with the existing type > system, that would be a big problem, but I haven't seen such an argument > that didn't presuppose a particular implementation of function types. I > really do think we should design the best spec (syntax and semantics) and > then see if it's implementable. I agree with the process you describe. I just think we're a couple of steps ahead of you. Setting aside the precise syntax, there is a specific problematic interaction of the currently proposed semantics with the existing type system, and we're exploring the fallout of that issue. We don't have to wait for the syntax to be settled to explore the fallout of the semantics issue, and as I pointed out resolving the semantic issue can help guide the syntax. > Earlier, I suggested that we should allow arrays of function types if we > can. ?\Otherwise, we're damning function types to life as second-class > citizens of the Java type system. We're also screwing up variable-arity > function typed args, as varargs is a leaky abstraction built atop arrays. > ?This isn't a theoretical concern; it impacts reasonable programs. For > example, consider a method that takes a bunch of #int(int) functions and > returns their composition: > ?? ?#int(int) compose(#int(int)... fns) { ? } > This should definitely be legal, and shouldn't generate a warning > (especially given that we're providing a means to eliminate this sort of > warning on generic varargs parameters in Java 7). This is all true, but the facts of the universe we live in have conspired against us. We should if we can, but we can't. All of the good reasons for the "should"s don't reduce the impact of those facts; we have to face them and design the best feature we can given the constraints of the real world. From neal at gafter.com Wed Feb 17 15:51:28 2010 From: neal at gafter.com (Neal Gafter) Date: Wed, 17 Feb 2010 15:51:28 -0800 Subject: Function types versus arrays In-Reply-To: <15e8b9d21002171510ldb6ad0dh4e85f866cd96fbc2@mail.gmail.com> References: <3dd3f56a1002081452o36d2b695kc766842ec2e8348b@mail.gmail.com> <15e8b9d21002151448i2ca40f8fpbf40c70caffeab2c@mail.gmail.com> <3dd3f56a1002162058j7f6e248cj197705273c2054b3@mail.gmail.com> <15e8b9d21002162126l26f89047q22d25e953d55be6@mail.gmail.com> <17b2302a1002170925i28e0e7ceu33c30f9c4172998a@mail.gmail.com> <15e8b9d21002170939n43e51bfeie7fa2a01a5cab4c9@mail.gmail.com> <17b2302a1002171439o467b40f0lf46d96f004b2feca@mail.gmail.com> <15e8b9d21002171448p7177fe3crfd64a2846307c13d@mail.gmail.com> <17b2302a1002171459n6212bb2em458174add2f83afd@mail.gmail.com> <15e8b9d21002171510ldb6ad0dh4e85f866cd96fbc2@mail.gmail.com> Message-ID: <15e8b9d21002171551v7343965u5d7e46304a65c155@mail.gmail.com> On Wed, Feb 17, 2010 at 3:10 PM, Neal Gafter wrote: > On Wed, Feb 17, 2010 at 2:59 PM, Joshua Bloch wrote: >> I really don't understand this. ?Earlier on, Alex said that we should >> refrain from thinking too hard about the implementation at this point, and I >> still believe that to be true. ?If you could show that some interaction of >> the proposed syntax and semantics interacted poorly with the existing type >> system, that would be a big problem, but I haven't seen such an argument >> that didn't presuppose a particular implementation of function types. I >> really do think we should design the best spec (syntax and semantics) and >> then see if it's implementable. > > I agree with the process you describe. ?I just think we're a couple of > steps ahead of you. ?Setting aside the precise syntax, there is a > specific problematic interaction of the currently proposed semantics > with the existing type system, and we're exploring the fallout of that > issue. ?We don't have to wait for the syntax to be settled to explore > the fallout of the semantics issue, and as I pointed out resolving the > semantic issue can help guide the syntax. I should add that Howard relaunched this thread with the idea of relaxing Java's usual assumption that a subtype conversion preserves the identity of the reference. I don't think that relaxation actually solves the problem, and it does break other things in the language. But it is worth spending some time determining if and how it breaks. From jjb at google.com Wed Feb 17 16:47:46 2010 From: jjb at google.com (Joshua Bloch) Date: Wed, 17 Feb 2010 16:47:46 -0800 Subject: Function types versus arrays In-Reply-To: <15e8b9d21002171510ldb6ad0dh4e85f866cd96fbc2@mail.gmail.com> References: <3dd3f56a1002081452o36d2b695kc766842ec2e8348b@mail.gmail.com> <15e8b9d21002151448i2ca40f8fpbf40c70caffeab2c@mail.gmail.com> <3dd3f56a1002162058j7f6e248cj197705273c2054b3@mail.gmail.com> <15e8b9d21002162126l26f89047q22d25e953d55be6@mail.gmail.com> <17b2302a1002170925i28e0e7ceu33c30f9c4172998a@mail.gmail.com> <15e8b9d21002170939n43e51bfeie7fa2a01a5cab4c9@mail.gmail.com> <17b2302a1002171439o467b40f0lf46d96f004b2feca@mail.gmail.com> <15e8b9d21002171448p7177fe3crfd64a2846307c13d@mail.gmail.com> <17b2302a1002171459n6212bb2em458174add2f83afd@mail.gmail.com> <15e8b9d21002171510ldb6ad0dh4e85f866cd96fbc2@mail.gmail.com> Message-ID: <17b2302a1002171647l4e4b62fep582ec4cd84a1c0e3@mail.gmail.com> Neal, On Wed, Feb 17, 2010 at 3:10 PM, Neal Gafter wrote: > On Wed, Feb 17, 2010 at 2:59 PM, Joshua Bloch wrote: > > I really don't understand this. Earlier on, Alex said that we should > > refrain from thinking too hard about the implementation at this point, > and I > > still believe that to be true. If you could show that some interaction > of > > the proposed syntax and semantics interacted poorly with the existing > type > > system, that would be a big problem, but I haven't seen such an argument > > that didn't presuppose a particular implementation of function types. I > > really do think we should design the best spec (syntax and semantics) and > > then see if it's implementable. > > I agree with the process you describe. I just think we're a couple of > steps ahead of you. Setting aside the precise syntax, there is a > specific problematic interaction of the currently proposed semantics > with the existing type system, and we're exploring the fallout of that > issue. We don't have to wait for the syntax to be settled to explore > the fallout of the semantics issue, and as I pointed out resolving the > semantic issue can help guide the syntax. > I don't buy this. All we know is that it requires some effort to implement reified function types. We always claimed that the reason that we did generics via erasure was migration compatibility, not ease of implementation. *We don't have that excuse this time.* > > > Earlier, I suggested that we should allow arrays of function types if we > > can. \Otherwise, we're damning function types to life as second-class > > citizens of the Java type system. We're also screwing up variable-arity > > function typed args, as varargs is a leaky abstraction built atop arrays. > > This isn't a theoretical concern; it impacts reasonable programs. For > > example, consider a method that takes a bunch of #int(int) functions and > > returns their composition: > > #int(int) compose(#int(int)... fns) { } > > This should definitely be legal, and shouldn't generate a warning > > (especially given that we're providing a means to eliminate this sort of > > warning on generic varargs parameters in Java 7). > > This is all true, but the facts of the universe we live in have > conspired against us. We should if we can, but we can't. Can't is a strong word. All we can say is that it would involve VM change. It may be worth that. I should add that Howard relaunched this thread with the idea of relaxing Java's usual assumption that a subtype conversion preserves the identity of the reference. I don't think that relaxation actually solves the problem, and it does break other things in the language. But it is worth spending some time determining if and how it breaks. I do not think this is a good idea. It violates the principle of least astonishment by sacrificing a fundamental truth of the language ("casting an object reference to an object reference type does not change its identity). Josh From neal at gafter.com Wed Feb 17 16:54:46 2010 From: neal at gafter.com (Neal Gafter) Date: Wed, 17 Feb 2010 16:54:46 -0800 Subject: Function types versus arrays In-Reply-To: <17b2302a1002171647l4e4b62fep582ec4cd84a1c0e3@mail.gmail.com> References: <3dd3f56a1002081452o36d2b695kc766842ec2e8348b@mail.gmail.com> <3dd3f56a1002162058j7f6e248cj197705273c2054b3@mail.gmail.com> <15e8b9d21002162126l26f89047q22d25e953d55be6@mail.gmail.com> <17b2302a1002170925i28e0e7ceu33c30f9c4172998a@mail.gmail.com> <15e8b9d21002170939n43e51bfeie7fa2a01a5cab4c9@mail.gmail.com> <17b2302a1002171439o467b40f0lf46d96f004b2feca@mail.gmail.com> <15e8b9d21002171448p7177fe3crfd64a2846307c13d@mail.gmail.com> <17b2302a1002171459n6212bb2em458174add2f83afd@mail.gmail.com> <15e8b9d21002171510ldb6ad0dh4e85f866cd96fbc2@mail.gmail.com> <17b2302a1002171647l4e4b62fep582ec4cd84a1c0e3@mail.gmail.com> Message-ID: <15e8b9d21002171654p7b4ec961u4b36f033577789aa@mail.gmail.com> On Wed, Feb 17, 2010 at 4:47 PM, Joshua Bloch wrote: >> This is all true, but the facts of the universe we live in have >> conspired against us. ?We should if we can, but we can't. > > Can't is a strong word. ?All we can say is that it would involve VM change. > ?It may be worth that. It would involve reifying generics uniformly (i.e. everywhere), which is not backward compatible. It may be worth that to you, but I think it would be a disaster. >> I should add that Howard relaunched this thread with the idea of >> relaxing Java's usual assumption that a subtype conversion preserves >> the identity of the reference. ?I don't think that relaxation actually >> solves the problem, and it does break other things in the language. >> But it is worth spending some time determining if and how it breaks. > > I do not think this is a good idea. It violates the principle of least > astonishment by sacrificing a fundamental truth of the language ("casting an > object reference to an object reference type does not change its identity). I don't think it is a good idea either, (nor do I think it works) but it was you who said that we should do whatever it takes to make arrays of functions work. Perhaps you want to rethink that position. From forax at univ-mlv.fr Wed Feb 17 17:00:47 2010 From: forax at univ-mlv.fr (=?ISO-8859-1?Q?R=E9mi_Forax?=) Date: Thu, 18 Feb 2010 02:00:47 +0100 Subject: Function types versus arrays In-Reply-To: <17b2302a1002171647l4e4b62fep582ec4cd84a1c0e3@mail.gmail.com> References: <3dd3f56a1002081452o36d2b695kc766842ec2e8348b@mail.gmail.com> <15e8b9d21002151448i2ca40f8fpbf40c70caffeab2c@mail.gmail.com> <3dd3f56a1002162058j7f6e248cj197705273c2054b3@mail.gmail.com> <15e8b9d21002162126l26f89047q22d25e953d55be6@mail.gmail.com> <17b2302a1002170925i28e0e7ceu33c30f9c4172998a@mail.gmail.com> <15e8b9d21002170939n43e51bfeie7fa2a01a5cab4c9@mail.gmail.com> <17b2302a1002171439o467b40f0lf46d96f004b2feca@mail.gmail.com> <15e8b9d21002171448p7177fe3crfd64a2846307c13d@mail.gmail.com> <17b2302a1002171459n6212bb2em458174add2f83afd@mail.gmail.com> <15e8b9d21002171510ldb6ad0dh4e85f866cd96fbc2@mail.gmail.com> <17b2302a1002171647l4e4b62fep582ec4cd84a1c0e3@mail.gmail.com> Message-ID: <4B7C913F.1040202@univ-mlv.fr> Le 18/02/2010 01:47, Joshua Bloch a ?crit : > Neal, > > On Wed, Feb 17, 2010 at 3:10 PM, Neal Gafter wrote: > > >> On Wed, Feb 17, 2010 at 2:59 PM, Joshua Bloch wrote: >> >>> I really don't understand this. Earlier on, Alex said that we should >>> refrain from thinking too hard about the implementation at this point, >>> >> and I >> >>> still believe that to be true. If you could show that some interaction >>> >> of >> >>> the proposed syntax and semantics interacted poorly with the existing >>> >> type >> >>> system, that would be a big problem, but I haven't seen such an argument >>> that didn't presuppose a particular implementation of function types. I >>> really do think we should design the best spec (syntax and semantics) and >>> then see if it's implementable. >>> >> I agree with the process you describe. I just think we're a couple of >> steps ahead of you. Setting aside the precise syntax, there is a >> specific problematic interaction of the currently proposed semantics >> with the existing type system, and we're exploring the fallout of that >> issue. We don't have to wait for the syntax to be settled to explore >> the fallout of the semantics issue, and as I pointed out resolving the >> semantic issue can help guide the syntax. >> >> > I don't buy this. All we know is that it requires some effort to implement > reified function types. We always claimed that the reason that we did > generics via erasure was migration compatibility, not ease of > implementation. *We don't have that excuse this time.* > > >> >>> Earlier, I suggested that we should allow arrays of function types if we >>> can. \Otherwise, we're damning function types to life as second-class >>> citizens of the Java type system. We're also screwing up variable-arity >>> function typed args, as varargs is a leaky abstraction built atop arrays. >>> This isn't a theoretical concern; it impacts reasonable programs. For >>> example, consider a method that takes a bunch of #int(int) functions and >>> returns their composition: >>> #int(int) compose(#int(int)... fns) { } >>> This should definitely be legal, and shouldn't generate a warning >>> (especially given that we're providing a means to eliminate this sort of >>> warning on generic varargs parameters in Java 7). >>> >> This is all true, but the facts of the universe we live in have >> conspired against us. We should if we can, but we can't. >> > > Can't is a strong word. All we can say is that it would involve VM change. > It may be worth that. > Generics are not reified but you can create a function type using a type variable or a parameterized type. So even if we have fully refied function type, we also need fully reified generics. So yes we can .... we "just" have to reified generics. [...] > Josh > R?mi From neal at gafter.com Wed Feb 17 17:00:22 2010 From: neal at gafter.com (Neal Gafter) Date: Wed, 17 Feb 2010 17:00:22 -0800 Subject: Function types versus arrays In-Reply-To: <17b2302a1002171647l4e4b62fep582ec4cd84a1c0e3@mail.gmail.com> References: <3dd3f56a1002081452o36d2b695kc766842ec2e8348b@mail.gmail.com> <3dd3f56a1002162058j7f6e248cj197705273c2054b3@mail.gmail.com> <15e8b9d21002162126l26f89047q22d25e953d55be6@mail.gmail.com> <17b2302a1002170925i28e0e7ceu33c30f9c4172998a@mail.gmail.com> <15e8b9d21002170939n43e51bfeie7fa2a01a5cab4c9@mail.gmail.com> <17b2302a1002171439o467b40f0lf46d96f004b2feca@mail.gmail.com> <15e8b9d21002171448p7177fe3crfd64a2846307c13d@mail.gmail.com> <17b2302a1002171459n6212bb2em458174add2f83afd@mail.gmail.com> <15e8b9d21002171510ldb6ad0dh4e85f866cd96fbc2@mail.gmail.com> <17b2302a1002171647l4e4b62fep582ec4cd84a1c0e3@mail.gmail.com> Message-ID: <15e8b9d21002171700n4da1c9ffy754c2698658597@mail.gmail.com> On Wed, Feb 17, 2010 at 4:47 PM, Joshua Bloch wrote: > I do not think this is a good idea. It violates the principle of least > astonishment by sacrificing a fundamental truth of the language ("casting an > object reference to an object reference type does not change its identity). By the way, do you feel that an assignment conversion from one reference type to another reference type should preserve identity too? From scolebourne at joda.org Wed Feb 17 17:06:46 2010 From: scolebourne at joda.org (Stephen Colebourne) Date: Thu, 18 Feb 2010 01:06:46 +0000 Subject: Are function types a requirement? Message-ID: <4B7C92A6.9020008@joda.org> Recent threads have indicated the difficulties of adding function types to the language. - difficult/impossible to integrate with arrays (an omission that would certainly be a surprise to many I suspect) - difficult to integrate with varargs - difficult to implement (various strategies suggested) - difficult syntax choices (difficult to find something that is easy to read, given Java's painful checked exceptions) - debated invocation syntax - the func.() syntax One alternative is to omit function types from JDK 7, and only include conversion to SAM interface types. It would be a compile time error to declare a lambda expression/block that did not convert to a SAM. Function types could then be included in JDK 8 if a valid approach was available (ie. it gives the appearance of needing much more in depth study than the timescale allows) Pros: - reduces risk of making the wrong decision in the short JDK 7 time-frame - avoids all the issues above (AFAIK) - doesn't restrict adding function types later (we'd need some proof of this) - reduces the initial learning curve Cons: - doesn't tackle the problem of fork-joins excess of interfaces, which is declared as a major project goal - results in multiple, incompatible SAM interfaces for the same concept - no automatic handling of co/contra variance I therefore ask (in a positive way!) as to whether function types are a fixed requirement of project lambda? Or whether the SAM-conversion-only strategy would be considered for JDK 7? (My concern is to avoid the experience of generic wildcards where it was rather rushed in at the last minute by all accounts) I'm really looking for a simple answer from Alex rather than a debate on this... Stephen From jjb at google.com Wed Feb 17 17:59:27 2010 From: jjb at google.com (Joshua Bloch) Date: Wed, 17 Feb 2010 17:59:27 -0800 Subject: Function types versus arrays In-Reply-To: <15e8b9d21002171654p7b4ec961u4b36f033577789aa@mail.gmail.com> References: <3dd3f56a1002081452o36d2b695kc766842ec2e8348b@mail.gmail.com> <15e8b9d21002162126l26f89047q22d25e953d55be6@mail.gmail.com> <17b2302a1002170925i28e0e7ceu33c30f9c4172998a@mail.gmail.com> <15e8b9d21002170939n43e51bfeie7fa2a01a5cab4c9@mail.gmail.com> <17b2302a1002171439o467b40f0lf46d96f004b2feca@mail.gmail.com> <15e8b9d21002171448p7177fe3crfd64a2846307c13d@mail.gmail.com> <17b2302a1002171459n6212bb2em458174add2f83afd@mail.gmail.com> <15e8b9d21002171510ldb6ad0dh4e85f866cd96fbc2@mail.gmail.com> <17b2302a1002171647l4e4b62fep582ec4cd84a1c0e3@mail.gmail.com> <15e8b9d21002171654p7b4ec961u4b36f033577789aa@mail.gmail.com> Message-ID: <17b2302a1002171759p41ac3359v7e74d9ccb98a612f@mail.gmail.com> Neal, On Wed, Feb 17, 2010 at 4:54 PM, Neal Gafter wrote: > On Wed, Feb 17, 2010 at 4:47 PM, Joshua Bloch wrote: > >> This is all true, but the facts of the universe we live in have > >> conspired against us. We should if we can, but we can't. > > > > Can't is a strong word. All we can say is that it would involve VM > change. > > It may be worth that. > > It would involve reifying generics uniformly (i.e. everywhere), which > is not backward compatible. It may be worth that to you, but I think > it would be a disaster. Can you provide a concise proof that it would require reifying generics? It isn't obvious (to me). Josh From Alex.Buckley at Sun.COM Wed Feb 17 18:13:50 2010 From: Alex.Buckley at Sun.COM (Alex Buckley) Date: Wed, 17 Feb 2010 18:13:50 -0800 Subject: Are function types a requirement? In-Reply-To: <4B7C92A6.9020008@joda.org> References: <4B7C92A6.9020008@joda.org> Message-ID: <4B7CA25E.80909@sun.com> You have summed up the situation perfectly. Anonymous inner classes would be much better loved without the 'final' gotcha. I would hate for lambdas to fall into the same "What were they thinking?" bucket over a technically minor detail like function types. Furthermore, function types in themselves still leave Doug needing duplicate ParallelArrays to get fast primitive array implementations. Counterpoint: one could say that function types + upgraded anonymous inner classes (without the 'final' restriction) + lambda conversion would be fine. No lambda expressions needed. No arguments over 'this'. Compatible with adding method references. Let's remember that this OpenJDK project is an experiment, with a small mailing list (~150 people), and without interaction from many valued JCP partners. So I don't see the need to throw out function types today, barely six weeks after the project started being active. Alex Stephen Colebourne wrote: > Recent threads have indicated the difficulties of adding function types > to the language. > > - difficult/impossible to integrate with arrays (an omission that would > certainly be a surprise to many I suspect) > > - difficult to integrate with varargs > > - difficult to implement (various strategies suggested) > > - difficult syntax choices (difficult to find something that is easy to > read, given Java's painful checked exceptions) > > - debated invocation syntax - the func.() syntax > > > One alternative is to omit function types from JDK 7, and only include > conversion to SAM interface types. It would be a compile time error to > declare a lambda expression/block that did not convert to a SAM. > > Function types could then be included in JDK 8 if a valid approach was > available (ie. it gives the appearance of needing much more in depth > study than the timescale allows) > > Pros: > - reduces risk of making the wrong decision in the short JDK 7 time-frame > > - avoids all the issues above (AFAIK) > > - doesn't restrict adding function types later (we'd need some proof of > this) > > - reduces the initial learning curve > > Cons: > - doesn't tackle the problem of fork-joins excess of interfaces, which > is declared as a major project goal > > - results in multiple, incompatible SAM interfaces for the same concept > > - no automatic handling of co/contra variance > > > I therefore ask (in a positive way!) as to whether function types are a > fixed requirement of project lambda? Or whether the SAM-conversion-only > strategy would be considered for JDK 7? (My concern is to avoid the > experience of generic wildcards where it was rather rushed in at the > last minute by all accounts) > > I'm really looking for a simple answer from Alex rather than a debate on > this... > > Stephen > > > > From neal at gafter.com Wed Feb 17 18:18:39 2010 From: neal at gafter.com (Neal Gafter) Date: Wed, 17 Feb 2010 18:18:39 -0800 Subject: Function types versus arrays In-Reply-To: <17b2302a1002171759p41ac3359v7e74d9ccb98a612f@mail.gmail.com> References: <3dd3f56a1002081452o36d2b695kc766842ec2e8348b@mail.gmail.com> <17b2302a1002170925i28e0e7ceu33c30f9c4172998a@mail.gmail.com> <15e8b9d21002170939n43e51bfeie7fa2a01a5cab4c9@mail.gmail.com> <17b2302a1002171439o467b40f0lf46d96f004b2feca@mail.gmail.com> <15e8b9d21002171448p7177fe3crfd64a2846307c13d@mail.gmail.com> <17b2302a1002171459n6212bb2em458174add2f83afd@mail.gmail.com> <15e8b9d21002171510ldb6ad0dh4e85f866cd96fbc2@mail.gmail.com> <17b2302a1002171647l4e4b62fep582ec4cd84a1c0e3@mail.gmail.com> <15e8b9d21002171654p7b4ec961u4b36f033577789aa@mail.gmail.com> <17b2302a1002171759p41ac3359v7e74d9ccb98a612f@mail.gmail.com> Message-ID: <15e8b9d21002171818j69b3f6a4s141b1af8fb75de78@mail.gmail.com> On Wed, Feb 17, 2010 at 5:59 PM, Joshua Bloch wrote: > Can you provide a concise proof that it would require reifying generics? ?It > isn't obvious (to me). Sure. If you need more formality, I can work on it this weekend. One form of proof goes something like this: Consider some an existing generic method. We assume that the existing callers of this method do not pass enough information for the body of the method to know, at runtime, the actual types of the type parameters. (If needed, I can prove this assumption.) Now add a lambda expression inside the body of the method, with the type parameter appearing in its signature. The JLS says that changes to the body of a method are binary compatible. If function types are reified, then the type of this lambda expression is available at runtime. But that contradicts our assumption that existing callers of the method do not pass enough information to reify the type parameter. You might imagine addressing this by disallowing lambdas with type parameters in their signature, but this would prevent lots of code (for example, the parallel arrays implementation) from using lambdas. Lambda expressions are the only way to create values of function type; anonymous inner classes are not an implementation alternative. Therefore, the restriction would prevent the parallel arrays API from using function types. -Neal From jjb at google.com Wed Feb 17 18:56:15 2010 From: jjb at google.com (Joshua Bloch) Date: Wed, 17 Feb 2010 18:56:15 -0800 Subject: Function types versus arrays In-Reply-To: <15e8b9d21002171818j69b3f6a4s141b1af8fb75de78@mail.gmail.com> References: <3dd3f56a1002081452o36d2b695kc766842ec2e8348b@mail.gmail.com> <15e8b9d21002170939n43e51bfeie7fa2a01a5cab4c9@mail.gmail.com> <17b2302a1002171439o467b40f0lf46d96f004b2feca@mail.gmail.com> <15e8b9d21002171448p7177fe3crfd64a2846307c13d@mail.gmail.com> <17b2302a1002171459n6212bb2em458174add2f83afd@mail.gmail.com> <15e8b9d21002171510ldb6ad0dh4e85f866cd96fbc2@mail.gmail.com> <17b2302a1002171647l4e4b62fep582ec4cd84a1c0e3@mail.gmail.com> <15e8b9d21002171654p7b4ec961u4b36f033577789aa@mail.gmail.com> <17b2302a1002171759p41ac3359v7e74d9ccb98a612f@mail.gmail.com> <15e8b9d21002171818j69b3f6a4s141b1af8fb75de78@mail.gmail.com> Message-ID: <17b2302a1002171856w5813a062g36cb7330b2610398@mail.gmail.com> Neal, Actually this is a bit too formal/abstract for my tastes. Show me some code that would break. I'm looking for something along the lines of the "Why generic array creation is illegal" example on page 120 of Effective Java (Second Ed.). (I believe it was you who first showed me this example.) Thanks, Josh On Wed, Feb 17, 2010 at 6:18 PM, Neal Gafter wrote: > On Wed, Feb 17, 2010 at 5:59 PM, Joshua Bloch wrote: > > Can you provide a concise proof that it would require reifying generics? > It > > isn't obvious (to me). > > Sure. If you need more formality, I can work on it this weekend. One > form of proof goes something like this: > > Consider some an existing generic method. We assume that the existing > callers of this method do not pass enough information for the body of > the method to know, at runtime, the actual types of the type > parameters. (If needed, I can prove this assumption.) Now add a > lambda expression inside the body of the method, with the type > parameter appearing in its signature. The JLS says that changes to > the body of a method are binary compatible. If function types are > reified, then the type of this lambda expression is available at > runtime. But that contradicts our assumption that existing callers of > the method do not pass enough information to reify the type parameter. > > You might imagine addressing this by disallowing lambdas with type > parameters in their signature, but this would prevent lots of code > (for example, the parallel arrays implementation) from using lambdas. > Lambda expressions are the only way to create values of function type; > anonymous inner classes are not an implementation alternative. > Therefore, the restriction would prevent the parallel arrays API from > using function types. > > -Neal > From gdennis at alum.mit.edu Wed Feb 17 19:22:28 2010 From: gdennis at alum.mit.edu (Greg Dennis) Date: Wed, 17 Feb 2010 22:22:28 -0500 Subject: Function types versus arrays Message-ID: <57c931191002171922v50fd567dnd82c2262209ee48a@mail.gmail.com> Ignorant question from a lurker ... Would it be possible to reify just those function types that don't have generics in their type signature? If this were the case, would that allow arrays of functions so long as the functions do not have generics in their signature? I would consider that consistent with the existing restriction against arrays of generics. > Date: Wed, 17 Feb 2010 18:18:39 -0800 > From: Neal Gafter > Subject: Re: Function types versus arrays > To: Joshua Bloch > Cc: lambda-dev > Message-ID: > ? ? ? ?<15e8b9d21002171818j69b3f6a4s141b1af8fb75de78 at mail.gmail.com> > Content-Type: text/plain; charset=ISO-8859-1 > > On Wed, Feb 17, 2010 at 5:59 PM, Joshua Bloch wrote: >> Can you provide a concise proof that it would require reifying generics? ?It >> isn't obvious (to me). > > Sure. ?If you need more formality, I can work on it this weekend. ?One > form of proof goes something like this: > > Consider some an existing generic method. ?We assume that the existing > callers of this method do not pass enough information for the body of > the method to know, at runtime, the actual types of the type > parameters. ?(If needed, I can prove this assumption.) ?Now add a > lambda expression inside the body of the method, with the type > parameter appearing in its signature. ?The JLS says that changes to > the body of a method are binary compatible. ?If function types are > reified, then the type of this lambda expression is available at > runtime. ?But that contradicts our assumption that existing callers of > the method do not pass enough information to reify the type parameter. > > You might imagine addressing this by disallowing lambdas with type > parameters in their signature, but this would prevent lots of code > (for example, the parallel arrays implementation) from using lambdas. > Lambda expressions are the only way to create values of function type; > anonymous inner classes are not an implementation alternative. > Therefore, the restriction would prevent the parallel arrays API from > using function types. > > -Neal From reinier at zwitserloot.com Wed Feb 17 20:06:38 2010 From: reinier at zwitserloot.com (Reinier Zwitserloot) Date: Thu, 18 Feb 2010 05:06:38 +0100 Subject: Are function types a requirement? In-Reply-To: <4B7C92A6.9020008@joda.org> References: <4B7C92A6.9020008@joda.org> Message-ID: <560fb5ed1002172006n40851460i3548077b3a74c1e6@mail.gmail.com> I've been cheerleading this idea (no function types) at Devoxx '09, but so far it's fallen on deaf ears. I've been playing with syntax and implementations based on this for quite a while, so I'll share some of my conclusions: Syntax basics are almost the same as the current lambda-dev proposals, with two addendums: You may optionally put a type name before the #, and a method name after it. Thus, you can actually write lambda-style closures for non-SAM types, which is good for working with existing java code: WindowListener listener = WindowAdapter#windowClosing(WindowEvent e) { ... }; It's not _that_ good, though, as only insane levels of inference could possibly guess that just "#windowClosing", without also putting "WindowAdapter" in the closure literal, is to be interpreted as being for WindowAdapter when trying to assign it to a variable of type WindowListener. Still, this part of it is certainly more convenient than other proposals. (In this example, omitting either the class name or the method name would result in an error; without a class name, inference would end up assuming you meant to write WindowListener#windowClosing, which won't work for obvious reasons, and without a method name the compiler would complain because WindowAdapter isn't a SAM). Inference is, of course, where it gets _really_ complicated, but on the plus side, it allows us to pour all the limited resources we have left before Java7 is out to work on inference. Once you've got inference perfected, the rest is trivial; a closure literal is EITHER immediately interpreted as simply an expression that produces a value of a type, or the compiler immediately generates an error. There's no intermediate function type phase. So, how would inference have to work? Well, if the Type name is mandatory, inference is again trivial; if the Type is a SAM, then the method name is inferred to be the one abstract method in that type. If not, then you must have a method name. However, things are only this trivial if the type also lists all generics information, and then we not only have the considerable burden of writing probably reptitive type information anytime we want to write a closure, it also won't solve the explosion of types in ParallelArray. Thus, clearly, inference needs to be a lot more complicated than this for it to be a passable closures implementation for java. With some work on the JVM, it should be relatively simple to detect the following situation: 1. Some method runs Integer.valueOf() whilst constructing a stack frame to be passed off to another method via one of the INVOKE opcodes (if this is difficult to detect, the compiler can perhaps emit a special call or opcode to 'box' an integer this way). This is an immutable property for any given parameter in any given INVOKEx opcode. 2. The invoked method's very first act when it first touches the object at this stack position is to call intValue() on it. This is an immutable property for any given parameter in any given method. That's, in other words, the situation where one method calls another with a primitive. Once this feature is in, which isn't useful just for closures but a boon to the JVM in general, then it becomes possible to eliminate a lot of types in ParallelArray's Ops; instead of having: Ops.DoubleMaxReducer Ops.IntMaxReducer Ops.LongMaxReducer Ops.MaxReducer You'd just have: Ops.MaxReducer. All methods in the ParallelArray library that currently take an Ops.DoubleMaxReducer will instead take an Ops.MaxReducer, and the JVM will make sure that performance is not adversely affected as long as the caller also works with a direct double. If the caller doesn't, this can only be described as a win, as in such a situation the caller either has to do a runtime if/elseif block or use MaxReducer without the benefit of JVM optimized autobox/unbox elimination. If that's done, then "all" that's left is inferring that something like this: Ops.MaxReducer x = #(double a, double b) { ... } or even harder, this: ParallelDoubleArray a = ...; a.cumulate(#(double a, double b) { ... }, 0); is legal, and in fact means that the closure is in fact implementing Ops.MaxReducer#invoke(double a, double b), that (in the second case), the intended target method signature is ParallelDoubleArray.cumulate(Ops.MaxReducer, double), and of course that using primitive types in generics is legal in the first place. This again has the property that its certainly not trivial to do so, but the benefit of doing it extends well beyond merely closures. For example, even with full blown BGGA closures, I don't see how you could ever write something like: new TreeSet({int a, int b => ...}); whereas with a scheme to allow primitive types as type arguments, you could not only write such a thing, it would be simpler: new TreeSet(#(int a, int b)( ... )); How to allow primitives in generics declarations has been researched, and abandoned, before, but if we can solve this problem today then function types can be dropped. The cost/value analysis of trying to make it work has perhaps changed now - simplying closures considerably is now added to the value. A very rough sketch on how to go about allowing primitive types in generics: For Type _Parameters_ nothing needs to change. Writing class X makes no sense. For Type _Arguments_, 'int' is legal anywhere Integer would be. As far as type bounds go, this means 'int' will take the place of either something like (including just ?/T, which is of course shorthand for ? extends Object), or something like . As an implementation, all locations where a type "int" is used to fit any parameterized type are by the compiler turned into Integer, and appropriate box/unbox code is inserted at an appropriate place (Per field access for fields, at the top of a method for parameters, and right before invoking / setting when invoking methods / setting fields). The generated conversion code for outgoing calls is simple (just wrap with Integer.valueOf), but for incoming changes (from Integer to int), it needs to do both .intValue() and a nullcheck. Sequential Integer.valueOf/.intValue calls, even across invokes, would be eliminated by the JVM, so the performance hit should be negligible in most cases. Apply the same reasoning for all the other primitives. If anyone remembers or has a link to the problems inherent in allowing primitives in generics, that would be useful for this discussion. Staggered release of these features (closures itself in JDK7, allowing primitives in generics in JDK8) is possibly but a great solution; changing an API to go from e.g. cumulate(DoubleMaxReducer r, double base) to cumulate(MaxReducer r, double base) is backwards compatible, but only if both methods continue to exist simultaneously, and the DoubleMaxReducer type can never be deleted. That's clearly not an optimal situation. Of course, if function types are an base requirement than forget I mentioned it. --Reinier Zwitserloot On Thu, Feb 18, 2010 at 2:06 AM, Stephen Colebourne wrote: > Recent threads have indicated the difficulties of adding function types > to the language. > > - difficult/impossible to integrate with arrays (an omission that would > certainly be a surprise to many I suspect) > > - difficult to integrate with varargs > > - difficult to implement (various strategies suggested) > > - difficult syntax choices (difficult to find something that is easy to > read, given Java's painful checked exceptions) > > - debated invocation syntax - the func.() syntax > > > One alternative is to omit function types from JDK 7, and only include > conversion to SAM interface types. It would be a compile time error to > declare a lambda expression/block that did not convert to a SAM. > > Function types could then be included in JDK 8 if a valid approach was > available (ie. it gives the appearance of needing much more in depth > study than the timescale allows) > > Pros: > - reduces risk of making the wrong decision in the short JDK 7 time-frame > > - avoids all the issues above (AFAIK) > > - doesn't restrict adding function types later (we'd need some proof of > this) > > - reduces the initial learning curve > > Cons: > - doesn't tackle the problem of fork-joins excess of interfaces, which > is declared as a major project goal > > - results in multiple, incompatible SAM interfaces for the same concept > > - no automatic handling of co/contra variance > > > I therefore ask (in a positive way!) as to whether function types are a > fixed requirement of project lambda? Or whether the SAM-conversion-only > strategy would be considered for JDK 7? (My concern is to avoid the > experience of generic wildcards where it was rather rushed in at the > last minute by all accounts) > > I'm really looking for a simple answer from Alex rather than a debate on > this... > > Stephen > > > > > From neal at gafter.com Wed Feb 17 20:16:07 2010 From: neal at gafter.com (Neal Gafter) Date: Wed, 17 Feb 2010 20:16:07 -0800 Subject: Are function types a requirement? In-Reply-To: <560fb5ed1002172006n40851460i3548077b3a74c1e6@mail.gmail.com> References: <4B7C92A6.9020008@joda.org> <560fb5ed1002172006n40851460i3548077b3a74c1e6@mail.gmail.com> Message-ID: <15e8b9d21002172016g3482dcaao3e94f47b6666efe3@mail.gmail.com> Reinier- All of this machinery does not fix the basic issue. It is true you avoid the conflict between function types and arrays by not having function types, but you can't create an array of type Function either so this is no better for the programmer. Cheers, Neal On Wed, Feb 17, 2010 at 8:06 PM, Reinier Zwitserloot wrote: > I've been cheerleading this idea (no function types) at Devoxx '09, but so > far it's fallen on deaf ears. I've been playing with syntax and > implementations based on this for quite a while, so I'll share some of my > conclusions: > > Syntax basics are almost the same as the current lambda-dev proposals, with > two addendums: You may optionally put a type name before the #, and a method > name after it. Thus, you can actually write lambda-style closures for > non-SAM types, which is good for working with existing java code: > > WindowListener listener = WindowAdapter#windowClosing(WindowEvent e) { ... > }; > > It's not _that_ good, though, as only insane levels of inference could > possibly guess that just "#windowClosing", without also putting > "WindowAdapter" in the closure literal, is to be interpreted as being for > WindowAdapter when trying to assign it to a variable of type WindowListener. > Still, this part of it is certainly more convenient than other proposals. > (In this example, omitting either the class name or the method name would > result in an error; without a class name, inference would end up assuming > you meant to write WindowListener#windowClosing, which won't work for > obvious reasons, and without a method name the compiler would complain > because WindowAdapter isn't a SAM). > > > Inference is, of course, where it gets _really_ complicated, but on the plus > side, it allows us to pour all the limited resources we have left before > Java7 is out to work on inference. Once you've got inference perfected, the > rest is trivial; a closure literal is EITHER immediately interpreted as > simply an expression that produces a value of a type, or the compiler > immediately generates an error. There's no intermediate function type phase. > > > So, how would inference have to work? Well, if the Type name is mandatory, > inference is again trivial; if the Type is a SAM, then the method name is > inferred to be the one abstract method in that type. If not, then you must > have a method name. However, things are only this trivial if the type also > lists all generics information, and then we not only have the considerable > burden of writing probably reptitive type information anytime we want to > write a closure, it also won't solve the explosion of types in > ParallelArray. Thus, clearly, inference needs to be a lot more complicated > than this for it to be a passable closures implementation for java. > > With some work on the JVM, it should be relatively simple to detect the > following situation: > > 1. Some method runs Integer.valueOf() whilst constructing a stack frame to > be passed off to another method via one of the INVOKE opcodes (if this is > difficult to detect, the compiler can perhaps emit a special call or opcode > to 'box' an integer this way). This is an immutable property for any given > parameter in any given INVOKEx opcode. > > 2. The invoked method's very first act when it first touches the object at > this stack position is to call intValue() on it. This is an immutable > property for any given parameter in any given method. > > That's, in other words, the situation where one method calls another with a > primitive. Once this feature is in, which isn't useful just for closures but > a boon to the JVM in general, then it becomes possible to eliminate a lot of > types in ParallelArray's Ops; instead of having: > > Ops.DoubleMaxReducer > Ops.IntMaxReducer > Ops.LongMaxReducer > Ops.MaxReducer > > You'd just have: > > Ops.MaxReducer. > > All methods in the ParallelArray library that currently take an > Ops.DoubleMaxReducer will instead take an Ops.MaxReducer, and the > JVM will make sure that performance is not adversely affected as long as the > caller also works with a direct double. If the caller doesn't, this can only > be described as a win, as in such a situation the caller either has to do a > runtime if/elseif block or use MaxReducer without the benefit of JVM > optimized autobox/unbox elimination. > > If that's done, then "all" that's left is inferring that something like > this: > > Ops.MaxReducer x = #(double a, double b) { ... } > > or even harder, this: > > ParallelDoubleArray a = ...; > a.cumulate(#(double a, double b) { ... }, 0); > > is legal, and in fact means that the closure is in fact implementing > Ops.MaxReducer#invoke(double a, double b), that (in the second > case), the intended target method signature is > ParallelDoubleArray.cumulate(Ops.MaxReducer, double), and of course > that using primitive types in generics is legal in the first place. > > This again has the property that its certainly not trivial to do so, but the > benefit of doing it extends well beyond merely closures. For example, even > with full blown BGGA closures, I don't see how you could ever write > something like: > > new TreeSet({int a, int b => ...}); > > whereas with a scheme to allow primitive types as type arguments, you could > not only write such a thing, it would be simpler: > > new TreeSet(#(int a, int b)( ... )); > > How to allow primitives in generics declarations has been researched, and > abandoned, before, but if we can solve this problem today then function > types can be dropped. The cost/value analysis of trying to make it work has > perhaps changed now - simplying closures considerably is now added to the > value. > > > A very rough sketch on how to go about allowing primitive types in generics: > > For Type _Parameters_ nothing needs to change. Writing class X int> makes no sense. > > For Type _Arguments_, 'int' is legal anywhere Integer would be. As far as > type bounds go, this means 'int' will take the place of either something > like (including just ?/T, which is of course shorthand > for ? extends Object), or something like . As an > implementation, all locations where a type "int" is used to fit any > parameterized type are by the compiler turned into Integer, and appropriate > box/unbox code is inserted at an appropriate place (Per field access for > fields, at the top of a method for parameters, and right before invoking / > setting when invoking methods / setting fields). The generated conversion > code for outgoing calls is simple (just wrap with Integer.valueOf), but for > incoming changes (from Integer to int), it needs to do both .intValue() and > a nullcheck. Sequential Integer.valueOf/.intValue calls, even across > invokes, would be eliminated by the JVM, so the performance hit should be > negligible in most cases. > > Apply the same reasoning for all the other primitives. > > > If anyone remembers or has a link to the problems inherent in allowing > primitives in generics, that would be useful for this discussion. > > > Staggered release of these features (closures itself in JDK7, allowing > primitives in generics in JDK8) is possibly but a great solution; changing > an API to go from e.g. cumulate(DoubleMaxReducer r, double base) to > cumulate(MaxReducer r, double base) is backwards compatible, but > only if both methods continue to exist simultaneously, and the > DoubleMaxReducer type can never be deleted. That's clearly not an optimal > situation. > > > Of course, if function types are an base requirement than forget I mentioned > it. > > --Reinier Zwitserloot > > > > On Thu, Feb 18, 2010 at 2:06 AM, Stephen Colebourne wrote: > >> Recent threads have indicated the difficulties of adding function types >> to the language. >> >> - difficult/impossible to integrate with arrays (an omission that would >> certainly be a surprise to many I suspect) >> >> - difficult to integrate with varargs >> >> - difficult to implement (various strategies suggested) >> >> - difficult syntax choices (difficult to find something that is easy to >> read, given Java's painful checked exceptions) >> >> - debated invocation syntax - the func.() syntax >> >> >> One alternative is to omit function types from JDK 7, and only include >> conversion to SAM interface types. It would be a compile time error to >> declare a lambda expression/block that did not convert to a SAM. >> >> Function types could then be included in JDK 8 if a valid approach was >> available (ie. it gives the appearance of needing much more in depth >> study than the timescale allows) >> >> Pros: >> - reduces risk of making the wrong decision in the short JDK 7 time-frame >> >> - avoids all the issues above (AFAIK) >> >> - doesn't restrict adding function types later (we'd need some proof of >> this) >> >> - reduces the initial learning curve >> >> Cons: >> - doesn't tackle the problem of fork-joins excess of interfaces, which >> is declared as a major project goal >> >> - results in multiple, incompatible SAM interfaces for the same concept >> >> - no automatic handling of co/contra variance >> >> >> I therefore ask (in a positive way!) as to whether function types are a >> fixed requirement of project lambda? Or whether the SAM-conversion-only >> strategy would be considered for JDK 7? (My concern is to avoid the >> experience of generic wildcards where it was rather rushed in at the >> last minute by all accounts) >> >> I'm really looking for a simple answer from Alex rather than a debate on >> this... >> >> Stephen >> >> >> >> >> > > From neal at gafter.com Wed Feb 17 21:56:26 2010 From: neal at gafter.com (Neal Gafter) Date: Wed, 17 Feb 2010 21:56:26 -0800 Subject: Function types versus arrays In-Reply-To: <17b2302a1002171856w5813a062g36cb7330b2610398@mail.gmail.com> References: <3dd3f56a1002081452o36d2b695kc766842ec2e8348b@mail.gmail.com> <17b2302a1002171439o467b40f0lf46d96f004b2feca@mail.gmail.com> <15e8b9d21002171448p7177fe3crfd64a2846307c13d@mail.gmail.com> <17b2302a1002171459n6212bb2em458174add2f83afd@mail.gmail.com> <15e8b9d21002171510ldb6ad0dh4e85f866cd96fbc2@mail.gmail.com> <17b2302a1002171647l4e4b62fep582ec4cd84a1c0e3@mail.gmail.com> <15e8b9d21002171654p7b4ec961u4b36f033577789aa@mail.gmail.com> <17b2302a1002171759p41ac3359v7e74d9ccb98a612f@mail.gmail.com> <15e8b9d21002171818j69b3f6a4s141b1af8fb75de78@mail.gmail.com> <17b2302a1002171856w5813a062g36cb7330b2610398@mail.gmail.com> Message-ID: <15e8b9d21002172156g37f98afbie1e9695bebb62a2@mail.gmail.com> On Wed, Feb 17, 2010 at 6:56 PM, Joshua Bloch wrote: > Actually this is a bit too formal/abstract for my tastes. ?Show me some code > that would break. I'm looking for something along the lines of the "Why > generic array creation is illegal" example on page 120 of Effective Java > (Second Ed.). ?(I believe it was you who first showed me this example.) Since reification of function types is visible by its effect in arrays (project lambda hasn't yet described the reflective APIs), we can illustrate the problem this way: class Ex { static #String()[] array = new #String()[1]; static Object[] objectArray = array; static void impossible() { final T t = null; objectArray[0] = #()(t); // assign a function of type #T() into objectArray } } Now, if we call Ex.impossible(), the code should run without problems (the lambda inside is of type #String()). If we call Ex.impossible(), the code must throw an array store exception (the lambda inside is of type #Integer()). Both calls must generate the exact same bytecode due to erasure of generics and the existing binary compatibility rules. Since the same code must behave differently at runtime due to information that is unavailable at runtime, we have a logical contradiction. From reinier at zwitserloot.com Wed Feb 17 22:01:05 2010 From: reinier at zwitserloot.com (Reinier Zwitserloot) Date: Thu, 18 Feb 2010 07:01:05 +0100 Subject: Are function types a requirement? In-Reply-To: <15e8b9d21002172016g3482dcaao3e94f47b6666efe3@mail.gmail.com> References: <4B7C92A6.9020008@joda.org> <560fb5ed1002172006n40851460i3548077b3a74c1e6@mail.gmail.com> <15e8b9d21002172016g3482dcaao3e94f47b6666efe3@mail.gmail.com> Message-ID: <560fb5ed1002172201s1c1eae5am4dd7d8f8627e7eae@mail.gmail.com> True, but it's not as bad as the arrays-of-function-types problem for two reasons: 1. Function won't be used as often as #void(String) would be - with SAMs as the de-facto type basis for closures, and better supported (via improved inference), they'd be a bit more popular. A SAM type without generics won't suffer from this problem. 2. The notion that you can't create a new array of a type that includes generics is already a known 'feature' of java. I believe that so far we're still stuck on the principle that any function types that themselves contain a parameterized type can't be reified without reifying all of generics, so that particular problem isn't going away. Moving away from function types softens the blow. On a different note, generified varargs are legal. You just get a warning. However, that warning is moving to the target method (it's one of project coin's features). I proposed at the time that this warning too can be eliminated if the following holds: The array variable is only read from, iterated over (foreach), or a method/field access (.length, .toString()) done on it. Or, to be more specific: 1. The array variable is not used in any expression, even as a primitive (Object o = varargsParam;), except for: 1A. LHS of a method invocation, so, varargsParam.toString() is acceptable (not much use in doing it, but it won't cause the warning). 1B. LHS of a field access, so, varargsParam.length is okay. 1C. Used as a primitive expression as the iterable in a foreach, so, for (T x : varargsParam) is okay. 2. array assignment to the varargsParam does not occur. So, varargsParam[expr] = whatever; means the warning appears. If the varargs method adheres to all these rules, then the warning is not shown. None of this seems prohibitive to scan for (none of it requires type resolution, for example), it's sufficient for type failures due to lack of reification to be impossible, and (guesstimating) it'll cover 99%+ of all uses of varargs out there. It wasn't considered at the time probably due to the complexity, but with closures also having the same issues when used in combination with varargs, it may warrant a revisit. If this concept is added to JDK7, then writing a compose method, and using it, generates 0 warnings: #int(int) compose(final #int(int)... functions) { return #(int in) { for (#int(int) f : functions) in = f.(in); return in; }; } void example() { System.out.println(compose(#(int a)(a+8), #(int a)(a*a)).(2)); } will print '100' and generate no warnings, and can work even if #int(int)... is erased to just 'FT'. (Of course, if you also make compose(final #double(double)) you're in trouble, due to erasure, but that's a different problem). --Reinier Zwitserloot On Thu, Feb 18, 2010 at 5:16 AM, Neal Gafter wrote: > Reinier- > > All of this machinery does not fix the basic issue. It is true you > avoid the conflict between function types and arrays by not having > function types, but you can't create an array of type Function > either so this is no better for the programmer. > > Cheers, > Neal > > On Wed, Feb 17, 2010 at 8:06 PM, Reinier Zwitserloot > wrote: > > I've been cheerleading this idea (no function types) at Devoxx '09, but > so > > far it's fallen on deaf ears. I've been playing with syntax and > > implementations based on this for quite a while, so I'll share some of my > > conclusions: > > > > Syntax basics are almost the same as the current lambda-dev proposals, > with > > two addendums: You may optionally put a type name before the #, and a > method > > name after it. Thus, you can actually write lambda-style closures for > > non-SAM types, which is good for working with existing java code: > > > > WindowListener listener = WindowAdapter#windowClosing(WindowEvent e) { > ... > > }; > > > > It's not _that_ good, though, as only insane levels of inference could > > possibly guess that just "#windowClosing", without also putting > > "WindowAdapter" in the closure literal, is to be interpreted as being for > > WindowAdapter when trying to assign it to a variable of type > WindowListener. > > Still, this part of it is certainly more convenient than other proposals. > > (In this example, omitting either the class name or the method name would > > result in an error; without a class name, inference would end up assuming > > you meant to write WindowListener#windowClosing, which won't work for > > obvious reasons, and without a method name the compiler would complain > > because WindowAdapter isn't a SAM). > > > > > > Inference is, of course, where it gets _really_ complicated, but on the > plus > > side, it allows us to pour all the limited resources we have left before > > Java7 is out to work on inference. Once you've got inference perfected, > the > > rest is trivial; a closure literal is EITHER immediately interpreted as > > simply an expression that produces a value of a type, or the compiler > > immediately generates an error. There's no intermediate function type > phase. > > > > > > So, how would inference have to work? Well, if the Type name is > mandatory, > > inference is again trivial; if the Type is a SAM, then the method name is > > inferred to be the one abstract method in that type. If not, then you > must > > have a method name. However, things are only this trivial if the type > also > > lists all generics information, and then we not only have the > considerable > > burden of writing probably reptitive type information anytime we want to > > write a closure, it also won't solve the explosion of types in > > ParallelArray. Thus, clearly, inference needs to be a lot more > complicated > > than this for it to be a passable closures implementation for java. > > > > With some work on the JVM, it should be relatively simple to detect the > > following situation: > > > > 1. Some method runs Integer.valueOf() whilst constructing a stack frame > to > > be passed off to another method via one of the INVOKE opcodes (if this is > > difficult to detect, the compiler can perhaps emit a special call or > opcode > > to 'box' an integer this way). This is an immutable property for any > given > > parameter in any given INVOKEx opcode. > > > > 2. The invoked method's very first act when it first touches the object > at > > this stack position is to call intValue() on it. This is an immutable > > property for any given parameter in any given method. > > > > That's, in other words, the situation where one method calls another with > a > > primitive. Once this feature is in, which isn't useful just for closures > but > > a boon to the JVM in general, then it becomes possible to eliminate a lot > of > > types in ParallelArray's Ops; instead of having: > > > > Ops.DoubleMaxReducer > > Ops.IntMaxReducer > > Ops.LongMaxReducer > > Ops.MaxReducer > > > > You'd just have: > > > > Ops.MaxReducer. > > > > All methods in the ParallelArray library that currently take an > > Ops.DoubleMaxReducer will instead take an Ops.MaxReducer, and the > > JVM will make sure that performance is not adversely affected as long as > the > > caller also works with a direct double. If the caller doesn't, this can > only > > be described as a win, as in such a situation the caller either has to do > a > > runtime if/elseif block or use MaxReducer without the benefit of > JVM > > optimized autobox/unbox elimination. > > > > If that's done, then "all" that's left is inferring that something like > > this: > > > > Ops.MaxReducer x = #(double a, double b) { ... } > > > > or even harder, this: > > > > ParallelDoubleArray a = ...; > > a.cumulate(#(double a, double b) { ... }, 0); > > > > is legal, and in fact means that the closure is in fact implementing > > Ops.MaxReducer#invoke(double a, double b), that (in the second > > case), the intended target method signature is > > ParallelDoubleArray.cumulate(Ops.MaxReducer, double), and of > course > > that using primitive types in generics is legal in the first place. > > > > This again has the property that its certainly not trivial to do so, but > the > > benefit of doing it extends well beyond merely closures. For example, > even > > with full blown BGGA closures, I don't see how you could ever write > > something like: > > > > new TreeSet({int a, int b => ...}); > > > > whereas with a scheme to allow primitive types as type arguments, you > could > > not only write such a thing, it would be simpler: > > > > new TreeSet(#(int a, int b)( ... )); > > > > How to allow primitives in generics declarations has been researched, and > > abandoned, before, but if we can solve this problem today then function > > types can be dropped. The cost/value analysis of trying to make it work > has > > perhaps changed now - simplying closures considerably is now added to the > > value. > > > > > > A very rough sketch on how to go about allowing primitive types in > generics: > > > > For Type _Parameters_ nothing needs to change. Writing class X > int> makes no sense. > > > > For Type _Arguments_, 'int' is legal anywhere Integer would be. As far as > > type bounds go, this means 'int' will take the place of either something > > like (including just ?/T, which is of course shorthand > > for ? extends Object), or something like . As an > > implementation, all locations where a type "int" is used to fit any > > parameterized type are by the compiler turned into Integer, and > appropriate > > box/unbox code is inserted at an appropriate place (Per field access for > > fields, at the top of a method for parameters, and right before invoking > / > > setting when invoking methods / setting fields). The generated conversion > > code for outgoing calls is simple (just wrap with Integer.valueOf), but > for > > incoming changes (from Integer to int), it needs to do both .intValue() > and > > a nullcheck. Sequential Integer.valueOf/.intValue calls, even across > > invokes, would be eliminated by the JVM, so the performance hit should be > > negligible in most cases. > > > > Apply the same reasoning for all the other primitives. > > > > > > If anyone remembers or has a link to the problems inherent in allowing > > primitives in generics, that would be useful for this discussion. > > > > > > Staggered release of these features (closures itself in JDK7, allowing > > primitives in generics in JDK8) is possibly but a great solution; > changing > > an API to go from e.g. cumulate(DoubleMaxReducer r, double base) to > > cumulate(MaxReducer r, double base) is backwards compatible, but > > only if both methods continue to exist simultaneously, and the > > DoubleMaxReducer type can never be deleted. That's clearly not an optimal > > situation. > > > > > > Of course, if function types are an base requirement than forget I > mentioned > > it. > > > > --Reinier Zwitserloot > > > > > > > > On Thu, Feb 18, 2010 at 2:06 AM, Stephen Colebourne < > scolebourne at joda.org>wrote: > > > >> Recent threads have indicated the difficulties of adding function types > >> to the language. > >> > >> - difficult/impossible to integrate with arrays (an omission that would > >> certainly be a surprise to many I suspect) > >> > >> - difficult to integrate with varargs > >> > >> - difficult to implement (various strategies suggested) > >> > >> - difficult syntax choices (difficult to find something that is easy to > >> read, given Java's painful checked exceptions) > >> > >> - debated invocation syntax - the func.() syntax > >> > >> > >> One alternative is to omit function types from JDK 7, and only include > >> conversion to SAM interface types. It would be a compile time error to > >> declare a lambda expression/block that did not convert to a SAM. > >> > >> Function types could then be included in JDK 8 if a valid approach was > >> available (ie. it gives the appearance of needing much more in depth > >> study than the timescale allows) > >> > >> Pros: > >> - reduces risk of making the wrong decision in the short JDK 7 > time-frame > >> > >> - avoids all the issues above (AFAIK) > >> > >> - doesn't restrict adding function types later (we'd need some proof of > >> this) > >> > >> - reduces the initial learning curve > >> > >> Cons: > >> - doesn't tackle the problem of fork-joins excess of interfaces, which > >> is declared as a major project goal > >> > >> - results in multiple, incompatible SAM interfaces for the same concept > >> > >> - no automatic handling of co/contra variance > >> > >> > >> I therefore ask (in a positive way!) as to whether function types are a > >> fixed requirement of project lambda? Or whether the SAM-conversion-only > >> strategy would be considered for JDK 7? (My concern is to avoid the > >> experience of generic wildcards where it was rather rushed in at the > >> last minute by all accounts) > >> > >> I'm really looking for a simple answer from Alex rather than a debate on > >> this... > >> > >> Stephen > >> > >> > >> > >> > >> > > > > > From jjb at google.com Wed Feb 17 23:26:55 2010 From: jjb at google.com (Joshua Bloch) Date: Wed, 17 Feb 2010 23:26:55 -0800 Subject: Function types versus arrays In-Reply-To: <15e8b9d21002172156g37f98afbie1e9695bebb62a2@mail.gmail.com> References: <3dd3f56a1002081452o36d2b695kc766842ec2e8348b@mail.gmail.com> <15e8b9d21002171448p7177fe3crfd64a2846307c13d@mail.gmail.com> <17b2302a1002171459n6212bb2em458174add2f83afd@mail.gmail.com> <15e8b9d21002171510ldb6ad0dh4e85f866cd96fbc2@mail.gmail.com> <17b2302a1002171647l4e4b62fep582ec4cd84a1c0e3@mail.gmail.com> <15e8b9d21002171654p7b4ec961u4b36f033577789aa@mail.gmail.com> <17b2302a1002171759p41ac3359v7e74d9ccb98a612f@mail.gmail.com> <15e8b9d21002171818j69b3f6a4s141b1af8fb75de78@mail.gmail.com> <17b2302a1002171856w5813a062g36cb7330b2610398@mail.gmail.com> <15e8b9d21002172156g37f98afbie1e9695bebb62a2@mail.gmail.com> Message-ID: <17b2302a1002172326o1fe6f470mc7beb96e98f6223a@mail.gmail.com> Neal, Thanks. This example makes the problem clear. Josh On Wed, Feb 17, 2010 at 9:56 PM, Neal Gafter wrote: > On Wed, Feb 17, 2010 at 6:56 PM, Joshua Bloch wrote: > > Actually this is a bit too formal/abstract for my tastes. Show me some > code > > that would break. I'm looking for something along the lines of the "Why > > generic array creation is illegal" example on page 120 of Effective Java > > (Second Ed.). (I believe it was you who first showed me this example.) > > Since reification of function types is visible by its effect in arrays > (project lambda hasn't yet described the reflective APIs), we can > illustrate the problem this way: > > class Ex { > static #String()[] array = new #String()[1]; > static Object[] objectArray = array; > static void impossible() { > final T t = null; > objectArray[0] = #()(t); // assign a function of type #T() > into objectArray > } > } > > Now, if we call Ex.impossible(), the code should run without > problems (the lambda inside is of type #String()). > > If we call Ex.impossible(), the code must throw an array > store exception (the lambda inside is of type #Integer()). > > Both calls must generate the exact same bytecode due to erasure of > generics and the existing binary compatibility rules. > > Since the same code must behave differently at runtime due to > information that is unavailable at runtime, we have a logical > contradiction. > From mthornton at optrak.co.uk Thu Feb 18 00:47:52 2010 From: mthornton at optrak.co.uk (Mark Thornton) Date: Thu, 18 Feb 2010 08:47:52 +0000 Subject: Are function types a requirement? In-Reply-To: <560fb5ed1002172006n40851460i3548077b3a74c1e6@mail.gmail.com> References: <4B7C92A6.9020008@joda.org> <560fb5ed1002172006n40851460i3548077b3a74c1e6@mail.gmail.com> Message-ID: <4B7CFEB8.4030509@optrak.co.uk> Reinier Zwitserloot wrote: > How to allow primitives in generics declarations has been researched, and > abandoned, before, but if we can solve this problem today then function > types can be dropped. The cost/value analysis of trying to make it work has > perhaps changed now - simplying closures considerably is now added to the > value. > > ... > > If anyone remembers or has a link to the problems inherent in allowing > primitives in generics, that would be useful for this discussion. > I found it difficult to do consistently without also reifying generics. You have to reify the primitives so that f(int) is distinct from f(double). Then you really want to reify the rest as well. At least that is my amateur take on it. Mark Thornton From reinier at zwitserloot.com Thu Feb 18 01:04:57 2010 From: reinier at zwitserloot.com (Reinier Zwitserloot) Date: Thu, 18 Feb 2010 10:04:57 +0100 Subject: Are function types a requirement? In-Reply-To: <4B7CFEB8.4030509@optrak.co.uk> References: <4B7C92A6.9020008@joda.org> <560fb5ed1002172006n40851460i3548077b3a74c1e6@mail.gmail.com> <4B7CFEB8.4030509@optrak.co.uk> Message-ID: <560fb5ed1002180104p196df93bgf91ceb70785bc983@mail.gmail.com> Well, you could get around that by stating that primitives aren't generified _unless_ a primitive type is used to fulfill a type variable. Thus, in something like: public class Test extends ArrayList { public int indexOf(int value) { ... } } the 'int' in 'int value' is being used to fulfill the generics bound, so it's considered to be "Integer" instead. Of course, that would be very weird indeed when code is working directly with Test, so it should ALSO stand for int, which means that's actually 2 methods - indexOf(Integer value) as well as indexOf(int value). Even if this duality concerns the return type this is possible (this is already done when tightening the return type in a subclass - javac generates 2 methods, one of which is a wrapper to the other). of course, if you have something like: public T foo(T param1, T param2, T param3, T param4, ...) {} the permutations required would be bordering on the silly, so this is where integrating primitives into the generics type system becomes quite complicated, and you'll need to take shortcuts, such as: Only 1 version with all-primitives, and one version with all-wrappers is generated (and marked as synthetic, or some simile thereof to tell IDEs and other code viewing tools not to list that method). An alternative is that there really is only int indexOf(Integer x), with method resolution taking care (via autoboxing) of the rest, but this would result in some weirdness when viewing the class in for example an IDE auto-complete dialog; you'd see int indexOf(Integer) even though you wrote int indexOf(int)! With such a system f(int) and f(double) will always remain distinct - there's no way to have a supertype that's got f(T) and f(V) defined in such a way that this is itself legal (T and V would have to erase to different base types) but trying to implement them with primitive types would lead to a signature collision. --Reinier Zwitserloot On Thu, Feb 18, 2010 at 9:47 AM, Mark Thornton wrote: > Reinier Zwitserloot wrote: > >> How to allow primitives in generics declarations has been researched, and >> abandoned, before, but if we can solve this problem today then function >> types can be dropped. The cost/value analysis of trying to make it work >> has >> perhaps changed now - simplying closures considerably is now added to the >> value. >> >> ... >> >> >> If anyone remembers or has a link to the problems inherent in allowing >> primitives in generics, that would be useful for this discussion. >> >> > I found it difficult to do consistently without also reifying generics. You > have to reify the primitives so that f(int) is distinct from f(double). Then > you really want to reify the rest as well. At least that is my amateur take > on it. > > Mark Thornton > > > From howard.lovatt at iee.org Thu Feb 18 01:44:53 2010 From: howard.lovatt at iee.org (Howard Lovatt) Date: Thu, 18 Feb 2010 20:44:53 +1100 Subject: Function types versus arrays In-Reply-To: <15e8b9d21002162126l26f89047q22d25e953d55be6@mail.gmail.com> References: <3dd3f56a1002081452o36d2b695kc766842ec2e8348b@mail.gmail.com> <3dd3f56a1002081944p6d6e0119g181fb968d7da747a@mail.gmail.com> <15e8b9d21002082129i21a209cel925ad8b6ba3313cd@mail.gmail.com> <3dd3f56a1002082201l304a5473ge2859fcc52032913@mail.gmail.com> <560fb5ed1002122022u27f0ed29x44d7f821ddb44ab5@mail.gmail.com> <15e8b9d21002122239p158cf4d3vfc45faaa2b48dced@mail.gmail.com> <3dd3f56a1002151357m2d5b13bs5e078c69d604e31b@mail.gmail.com> <15e8b9d21002151448i2ca40f8fpbf40c70caffeab2c@mail.gmail.com> <3dd3f56a1002162058j7f6e248cj197705273c2054b3@mail.gmail.com> <15e8b9d21002162126l26f89047q22d25e953d55be6@mail.gmail.com> Message-ID: <3dd3f56a1002180144o6aa115ddi45e56a2e1d3d0071@mail.gmail.com> Hi Neal, On 17 February 2010 16:26, Neal Gafter wrote: > Howard- > > Can you please show the implementation of > _Array_From_void_throws_ChangedCharSetException_SAXParseException_UnkownHostException__To_void_throws_AWTException_IOException_SAXException.instance? Sure: public final static class _Array_From_void_throws_ChangedCharSetException_UnknownHostException__To_void_throws_ChangedCharSetException_SAXParseException_UnknownHostException { public static _Callable_void_throws_ChangedCharSetException_SAXParseException_UnknownHostException[] instance(final _Callable_void_throws_ChangedCharSetException_UnknownHostException[] from) { if (from == null) { return null; } final _Callable_void_throws_ChangedCharSetException_SAXParseException_UnknownHostException[] to = new _Callable_void_throws_ChangedCharSetException_SAXParseException_UnknownHostException[from.length]; for (int i = 0; i < from.length; i++) { to[i] = _From_void_throws_ChangedCharSetException_UnkownHostException__To_void_throws_ChangedCharSetException_SAXParseException_UnkownHostException.instance(from[i]); } return to; } } public final static class _From_void_throws_ChangedCharSetException_UnkownHostException__To_void_throws_ChangedCharSetException_SAXParseException_UnkownHostException extends _Callable_void_throws_ChangedCharSetException_SAXParseException_UnknownHostException implements Wrapper { private final _Callable_void_throws_ChangedCharSetException_UnknownHostException f; private _From_void_throws_ChangedCharSetException_UnkownHostException__To_void_throws_ChangedCharSetException_SAXParseException_UnkownHostException(final _Callable_void_throws_ChangedCharSetException_UnknownHostException f) { this.f = f; } public static _From_void_throws_ChangedCharSetException_UnkownHostException__To_void_throws_ChangedCharSetException_SAXParseException_UnkownHostException instance(final _Callable_void_throws_ChangedCharSetException_UnknownHostException from) { if (from == null) { return null; } final _Callable_void_throws_ChangedCharSetException_UnknownHostException f = (_Callable_void_throws_ChangedCharSetException_UnknownHostException)Wrappers.unWrap(from); return new _From_void_throws_ChangedCharSetException_UnkownHostException__To_void_throws_ChangedCharSetException_SAXParseException_UnkownHostException(f); } @Override public void callPrimitive() throws ChangedCharSetException, SAXParseException, UnknownHostException { f.callPrimitive(); } @Override public Object getWrappee() { return f; } @Override public int hashCode() { return f.hashCode(); } } public final static class _Array_From_void_throws_ChangedCharSetException_SAXParseException_UnknownHostException__To_void_throws_AWTException_IOException_SAXException { public static _Callable_void_throws_AWTException_IOException_SAXException[] instance(final _Callable_void_throws_ChangedCharSetException_SAXParseException_UnknownHostException[] from) { if (from == null) { return null; } final _Callable_void_throws_AWTException_IOException_SAXException[] to = new _Callable_void_throws_AWTException_IOException_SAXException[from.length]; for (int i = 0; i < from.length; i++) { to[i] = _From_void_throws_ChangedCharSetException_SAXParseException_UnknownHostException__To_void_throws_AWTException_IOException_SAXException.instance(from[i]); } return to; } } public final static class _From_void_throws_ChangedCharSetException_SAXParseException_UnknownHostException__To_void_throws_AWTException_IOException_SAXException extends _Callable_void_throws_AWTException_IOException_SAXException implements Wrapper { private final _Callable_void_throws_ChangedCharSetException_SAXParseException_UnknownHostException f; private _From_void_throws_ChangedCharSetException_SAXParseException_UnknownHostException__To_void_throws_AWTException_IOException_SAXException(final _Callable_void_throws_ChangedCharSetException_SAXParseException_UnknownHostException f) { this.f = f; } public static _From_void_throws_ChangedCharSetException_SAXParseException_UnknownHostException__To_void_throws_AWTException_IOException_SAXException instance(final _Callable_void_throws_ChangedCharSetException_SAXParseException_UnknownHostException from) { if (from == null) { return null; } final _Callable_void_throws_ChangedCharSetException_SAXParseException_UnknownHostException f = (_Callable_void_throws_ChangedCharSetException_SAXParseException_UnknownHostException)Wrappers.unWrap(from); return new _From_void_throws_ChangedCharSetException_SAXParseException_UnknownHostException__To_void_throws_AWTException_IOException_SAXException(f); } @Override public void callPrimitive() throws AWTException, IOException, SAXException { f.callPrimitive(); } @Override public Object getWrappee() { return f; } @Override public int hashCode() { return f.hashCode(); } } // #void()(throws IOException, AWTException, SAXException) abstract static class _Callable_void_throws_AWTException_IOException_SAXException implements _Callable_0_throws_1 { public Void call() throws AWTException, IOException, SAXException { callPrimitive(); return null; } public abstract void callPrimitive() throws AWTException, IOException, SAXException; } // #void()(throws ChangedCharSetException, UnknownHostException, SAXParseException) abstract static class _Callable_void_throws_ChangedCharSetException_SAXParseException_UnknownHostException implements _Callable_0_throws_1 { public Void call() throws ChangedCharSetException, SAXParseException, UnknownHostException { callPrimitive(); return null; } public abstract void callPrimitive() throws ChangedCharSetException, SAXParseException, UnknownHostException; } // #void()(throws ChangedCharSetException, UnknownHostException) abstract static class _Callable_void_throws_ChangedCharSetException_UnknownHostException implements _Callable_0_throws_1 { public Void call() throws ChangedCharSetException, UnknownHostException { callPrimitive(); return null; } public abstract void callPrimitive() throws ChangedCharSetException, UnknownHostException; } // Generics can't declare an arbitrary number of exceptions (incl. 0), therefore use // _Callable_0_throws_1 which throws IOException public interface _Callable_0_throws_1 { R call() throws E; } Cheers, -- Howard. > > Cheers, > Neal > > On Tue, Feb 16, 2010 at 8:58 PM, Howard Lovatt wrote: >> (the above translation is an extension of sections 5.2.2, 5.4, and 4.4) and: >> >>> ? #void()(throws IOException, AWTException, SAXException)[] array2 = array1; >> >> becomes: >> >> _Callable_void_throws_AWTException_IOException_SAXException[] array2 = >> _Array_From_void_throws_ChangedCharSetException_SAXParseException_UnkownHostException__To_void_throws_AWTException_IOException_SAXException.instance( >> array1 ); > > ______________________________________________________________________ > This email has been scanned by the MessageLabs Email Security System. > For more information please visit http://www.messagelabs.com/email > ______________________________________________________________________ > -- -- Howard. From howard.lovatt at gmail.com Thu Feb 18 01:52:02 2010 From: howard.lovatt at gmail.com (Howard Lovatt) Date: Thu, 18 Feb 2010 20:52:02 +1100 Subject: Function types versus arrays Message-ID: <3dd3f56a1002180152rc3fedffy3263804b0c129585@mail.gmail.com> Hi, > Ignorant question from a lurker ... > > Would it be possible to reify just those function types that don't > have generics in their type signature? If this were the case, would > that allow arrays of functions so long as the functions do not have > generics in their signature? I would consider that consistent with the > existing restriction against arrays of generics. This is what http://www.artima.com/weblogs/viewpost.jsp?thread=278567 suggests. -- Howard. From howard.lovatt at iee.org Thu Feb 18 02:17:32 2010 From: howard.lovatt at iee.org (Howard Lovatt) Date: Thu, 18 Feb 2010 21:17:32 +1100 Subject: Function types versus arrays Message-ID: <3dd3f56a1002180217r34bfe5e7g9ef5f4d436415d95@mail.gmail.com> Hi John, Your examples in: http://www.artima.com/weblogs/viewpost.jsp?thread=278567 Are: // hard to infer A/T, must return concrete #(Object x)(x) // static #T(A) identityFunction() { return #(A x)(x); } static _Callable_1 identityFunction() { return new _Callable_1() { @Override public T call(A x) { return x; } }; } // Don't really understand this - what does type.cast does? However, don't need typecast for constantFunction. // static #T() constantFunction(T x) { return type.cast( #()(x) ); } static _Callable_0 constantFunction(final T x) { return new _Callable_0() { @Override public T call() { return x; } }; } -- Howard. From howard.lovatt at iee.org Thu Feb 18 02:28:08 2010 From: howard.lovatt at iee.org (Howard Lovatt) Date: Thu, 18 Feb 2010 21:28:08 +1100 Subject: Function types versus arrays Message-ID: <3dd3f56a1002180228x6e5c6542o3d62d110136e555@mail.gmail.com> Hi Josh, My reasons for discussing the implementation are: 1. If you can demonstrate that you can do the difficult cases then arguments along the lines of "we shouldn't have lambda arrays because we can't implement them" are negated. 2. I was interested in fast implementations, particularly for primitives and particularly for parallel processing 3. It was fun to see if it was possible (a number of people had said you couldn't do reified lambdas) With regard to my suggestion: http://www.artima.com/weblogs/viewpost.jsp?thread=278567 This suggestion fits very well with Java. The main exception is that: #int() il = #int()(1); #long() ll = il; ll != il // since ll is il wrapped ll.equals(il) == true // equals is implemented to compared the unwrapped lambdas This is similar to String or none cached Integers etc. Therefore hopefully OK. -- Howard. From howard.lovatt at iee.org Thu Feb 18 02:41:13 2010 From: howard.lovatt at iee.org (Howard Lovatt) Date: Thu, 18 Feb 2010 21:41:13 +1100 Subject: Function types versus arrays Message-ID: <3dd3f56a1002180241p2e88faf2w9ef94ecf51978e0e@mail.gmail.com> The example: class Ex { static #String()[] array = new #String()[1]; static Object[] objectArray = array; static void impossible() { final T t = null; objectArray[0] = #()(t); // assign a function of type #T() into objectArray } } Using reified lambdas: http://www.artima.com/weblogs/viewpost.jsp?thread=278567 Would allow both Ex.impossible() and Ex.impossible() since null is always assignable to #String(). This is similar to: class Ex { static String[] array = new String[1]; static Object[] objectArray = array; static void impossible() { final T t = null; objectArray[0] = t; // assign a T into objectArray } } Therefore the reified lambda behavior would be consistent with generics (maybe not liked behavior - but expected!). -- Howard. From howard.lovatt at iee.org Thu Feb 18 03:12:23 2010 From: howard.lovatt at iee.org (Howard Lovatt) Date: Thu, 18 Feb 2010 22:12:23 +1100 Subject: Function types versus arrays Message-ID: <3dd3f56a1002180312j4a17556eg4a4e48fa9dc91140@mail.gmail.com> Hi Alex, > I find it very difficult to follow Howard's scheme, since some is at > Artima and some is sprinkled around this mailing list. The reason for using Artima was primarily that I find Pipermail very limited. The important stuff is all at Artima. I can send my source code for the examples if you wish (I have already done so for Fredrik ?hrstr?m from the JRocket team who was interested in how lambdas might be optimized). > I am surprised at > the assumptions that suddenly appear from time to time, e.g. "For the > moment lets assume that generics were extended so that you could have a > vararg type construct". I was aware that checked exceptions would require extensions to generics and also the original strawman didn't include checked exceptions. Hence I didn't include them in the original proposal, but to answer on of Neal's questions I needed to explain my thinking on that subject. > I worry about the boot loader having to > auto-generate classes, since loaders are already under heavy > modification for module systems. I haven't studied this, I was hoping that any changes would be orthogonal and hence both possible simultaneously. > And it all feels similar to NextGen's > approach to reification, though no proper comparison has ever been made. The approach is similar, which I find encouraging, since NextGen is known to work. One trivial difference is that they use a class file generated by the compiler as a template that has markers in it that are replaced at load time, in my scheme the compiler simply emits the name of the required class (which encodes the necessary types). A more substantive difference is that backward compatibility is not an issue and this allows the reification scheme some flexibility. > For these reasons, I welcome Neal's continued analysis of the scheme on > lambda-dev, even if it sometimes looks like the implementation focus is > getting out of hand. I too welcome Neal's, and anyone else's, contribution - keep the tricky examples coming! -- Howard. From neal at gafter.com Thu Feb 18 07:36:37 2010 From: neal at gafter.com (Neal Gafter) Date: Thu, 18 Feb 2010 07:36:37 -0800 Subject: Function types versus arrays In-Reply-To: <3dd3f56a1002180144o6aa115ddi45e56a2e1d3d0071@mail.gmail.com> References: <3dd3f56a1002081452o36d2b695kc766842ec2e8348b@mail.gmail.com> <15e8b9d21002082129i21a209cel925ad8b6ba3313cd@mail.gmail.com> <3dd3f56a1002082201l304a5473ge2859fcc52032913@mail.gmail.com> <560fb5ed1002122022u27f0ed29x44d7f821ddb44ab5@mail.gmail.com> <15e8b9d21002122239p158cf4d3vfc45faaa2b48dced@mail.gmail.com> <3dd3f56a1002151357m2d5b13bs5e078c69d604e31b@mail.gmail.com> <15e8b9d21002151448i2ca40f8fpbf40c70caffeab2c@mail.gmail.com> <3dd3f56a1002162058j7f6e248cj197705273c2054b3@mail.gmail.com> <15e8b9d21002162126l26f89047q22d25e953d55be6@mail.gmail.com> <3dd3f56a1002180144o6aa115ddi45e56a2e1d3d0071@mail.gmail.com> Message-ID: <15e8b9d21002180736t14ad9bdavb5830c18dfffd14b@mail.gmail.com> On Thu, Feb 18, 2010 at 1:44 AM, Howard Lovatt wrote: > Hi Neal, > > On 17 February 2010 16:26, Neal Gafter wrote: >> Howard- >> >> Can you please show the implementation of >> _Array_From_void_throws_ChangedCharSetException_SAXParseException_UnkownHostException__To_void_throws_AWTException_IOException_SAXException.instance? > > Sure: This implementation is incorrect, because the two arrays must act as one. This method is supposed to implement the subtype relationship between two array types. Assigning to an element of one array value must modify the corresponding element of the array value which is its subtype. But with your implementation, the two arrays are disconnected. From neal at gafter.com Thu Feb 18 09:45:44 2010 From: neal at gafter.com (Neal Gafter) Date: Thu, 18 Feb 2010 09:45:44 -0800 Subject: Function types versus arrays In-Reply-To: <17b2302a1002171459n6212bb2em458174add2f83afd@mail.gmail.com> References: <3dd3f56a1002081452o36d2b695kc766842ec2e8348b@mail.gmail.com> <3dd3f56a1002151357m2d5b13bs5e078c69d604e31b@mail.gmail.com> <15e8b9d21002151448i2ca40f8fpbf40c70caffeab2c@mail.gmail.com> <3dd3f56a1002162058j7f6e248cj197705273c2054b3@mail.gmail.com> <15e8b9d21002162126l26f89047q22d25e953d55be6@mail.gmail.com> <17b2302a1002170925i28e0e7ceu33c30f9c4172998a@mail.gmail.com> <15e8b9d21002170939n43e51bfeie7fa2a01a5cab4c9@mail.gmail.com> <17b2302a1002171439o467b40f0lf46d96f004b2feca@mail.gmail.com> <15e8b9d21002171448p7177fe3crfd64a2846307c13d@mail.gmail.com> <17b2302a1002171459n6212bb2em458174add2f83afd@mail.gmail.com> Message-ID: <15e8b9d21002180945m5ed8cd97mb3dd7d1088005de6@mail.gmail.com> On Wed, Feb 17, 2010 at 2:59 PM, Joshua Bloch wrote: > consider a method that takes a bunch of #int(int) functions and > returns their composition: > ?? ?#int(int) compose(#int(int)... fns) { ? } > This should definitely be legal, and shouldn't generate a warning > (especially given that we're providing a means to eliminate this sort of > warning on generic varargs parameters in Java 7). This doesn't conflict with any of the restrictions I proposed. From scolebourne at joda.org Thu Feb 18 10:35:08 2010 From: scolebourne at joda.org (Stephen Colebourne) Date: Thu, 18 Feb 2010 18:35:08 +0000 Subject: Are function types a requirement? In-Reply-To: <4B7CA25E.80909@sun.com> References: <4B7C92A6.9020008@joda.org> <4B7CA25E.80909@sun.com> Message-ID: <4b4f45e01002181035u433ce04eu33fb04c2411002ca@mail.gmail.com> Alex, thanks for the reply. I think we all agree that we don't want any "What were they thinking?" outcome! Based on this, I may (if it proves sound) suggest an alternative option to function types in a few days. Stephen On 18 February 2010 02:13, Alex Buckley wrote: > You have summed up the situation perfectly. > > Anonymous inner classes would be much better loved without the 'final' > gotcha. I would hate for lambdas to fall into the same "What were they > thinking?" bucket over a technically minor detail like function types. > Furthermore, function types in themselves still leave Doug needing > duplicate ParallelArrays to get fast primitive array implementations. > > Counterpoint: one could say that function types + upgraded anonymous > inner classes (without the 'final' restriction) + lambda conversion > would be fine. No lambda expressions needed. No arguments over 'this'. > Compatible with adding method references. > > Let's remember that this OpenJDK project is an experiment, with a small > mailing list (~150 people), and without interaction from many valued JCP > partners. So I don't see the need to throw out function types today, > barely six weeks after the project started being active. > > Alex > > Stephen Colebourne wrote: >> Recent threads have indicated the difficulties of adding function types >> to the language. >> >> - difficult/impossible to integrate with arrays (an omission that would >> certainly be a surprise to many I suspect) >> >> - difficult to integrate with varargs >> >> - difficult to implement (various strategies suggested) >> >> - difficult syntax choices (difficult to find something that is easy to >> read, given Java's painful checked exceptions) >> >> - debated invocation syntax - the func.() syntax >> >> >> One alternative is to omit function types from JDK 7, and only include >> conversion to SAM interface types. It would be a compile time error to >> declare a lambda expression/block that did not convert to a SAM. >> >> Function types could then be included in JDK 8 if a valid approach was >> available (ie. it gives the appearance of needing much more in depth >> study than the timescale allows) >> >> Pros: >> - reduces risk of making the wrong decision in the short JDK 7 time-frame >> >> - avoids all the issues above (AFAIK) >> >> - doesn't restrict adding function types later (we'd need some proof of >> this) >> >> - reduces the initial learning curve >> >> Cons: >> - doesn't tackle the problem of fork-joins excess of interfaces, which >> is declared as a major project goal >> >> - results in multiple, incompatible SAM interfaces for the same concept >> >> - no automatic handling of co/contra variance >> >> >> I therefore ask (in a positive way!) as to whether function types are a >> fixed requirement of project lambda? Or whether the SAM-conversion-only >> strategy would be considered for JDK 7? (My concern is to avoid the >> experience of generic wildcards where it was rather rushed in at the >> last minute by all accounts) >> >> I'm really looking for a simple answer from Alex rather than a debate on >> this... >> >> Stephen >> >> >> >> > > From jjb at google.com Thu Feb 18 11:44:05 2010 From: jjb at google.com (Joshua Bloch) Date: Thu, 18 Feb 2010 11:44:05 -0800 Subject: Function types versus arrays In-Reply-To: <15e8b9d21002180945m5ed8cd97mb3dd7d1088005de6@mail.gmail.com> References: <3dd3f56a1002081452o36d2b695kc766842ec2e8348b@mail.gmail.com> <15e8b9d21002151448i2ca40f8fpbf40c70caffeab2c@mail.gmail.com> <3dd3f56a1002162058j7f6e248cj197705273c2054b3@mail.gmail.com> <15e8b9d21002162126l26f89047q22d25e953d55be6@mail.gmail.com> <17b2302a1002170925i28e0e7ceu33c30f9c4172998a@mail.gmail.com> <15e8b9d21002170939n43e51bfeie7fa2a01a5cab4c9@mail.gmail.com> <17b2302a1002171439o467b40f0lf46d96f004b2feca@mail.gmail.com> <15e8b9d21002171448p7177fe3crfd64a2846307c13d@mail.gmail.com> <17b2302a1002171459n6212bb2em458174add2f83afd@mail.gmail.com> <15e8b9d21002180945m5ed8cd97mb3dd7d1088005de6@mail.gmail.com> Message-ID: <17b2302a1002181144w3873b491q7795ce4768f88330@mail.gmail.com> Neal, Perhaps I'm being naive, but it appears to me that allowing: #int(int) compose(#int(int)... fns) { } implies the existence of arrays of (erased) function types. I thought you were arguing against the very existence of types representing arrays of function types. Alex's suggestion that these types be treated analogously to arrays of other nonreifiable types (i.e., most generic and parameterized types) makes good sense to me. Josh On Thu, Feb 18, 2010 at 9:45 AM, Neal Gafter wrote: > On Wed, Feb 17, 2010 at 2:59 PM, Joshua Bloch wrote: > > consider a method that takes a bunch of #int(int) functions and > > returns their composition: > > #int(int) compose(#int(int)... fns) { } > > This should definitely be legal, and shouldn't generate a warning > > (especially given that we're providing a means to eliminate this sort of > > warning on generic varargs parameters in Java 7). > > This doesn't conflict with any of the restrictions I proposed. > From Alex.Buckley at Sun.COM Thu Feb 18 12:16:02 2010 From: Alex.Buckley at Sun.COM (Alex Buckley) Date: Thu, 18 Feb 2010 12:16:02 -0800 Subject: Are function types a requirement? In-Reply-To: <560fb5ed1002172201s1c1eae5am4dd7d8f8627e7eae@mail.gmail.com> References: <4B7C92A6.9020008@joda.org> <560fb5ed1002172006n40851460i3548077b3a74c1e6@mail.gmail.com> <15e8b9d21002172016g3482dcaao3e94f47b6666efe3@mail.gmail.com> <560fb5ed1002172201s1c1eae5am4dd7d8f8627e7eae@mail.gmail.com> Message-ID: <4B7DA002.1080501@sun.com> Reinier Zwitserloot wrote: > On a different note, generified varargs are legal. You just get a warning. This is effectively incorrect. A varargs formal parameter that is parameterized ought to be illegal. (At least where varargs are implemented with arrays.) It breaks the property that static type correctness implies dynamic type safety. We allow such a parameter only for convenience. A warning indicates that you are being allowed to hang yourself for your own convenience. The word "just" has no place here. What you propose below (and it's really very clever) is to strengthen the static type system in the body of a varargs method. Your rules are so strong that if the body is statically type correct, then it will be dynamically type safe. You have achieved this by basically banning any language construct whose static typechecking involves subtype rules. For example, aliasing the varargs formal parameter through an Object[] - the usual way to lose dynamic type safety - is banned in your static type system, and code that did it would cause a warning. I discussed restrictions of this kind extensively with Josh Bloch and Bob Lee before Bob's Coin proposal. The summary is that I'm not a fan, which is why the proposal doesn't change the static type system and thus requires a very strong programmer declaration that the method body will be dynamically type safe promise promise promise. (Strictly speaking, I'm not a fan of building this alternative static type system *on top of array-based varargs*. You get your strong static typechecking for free with covariant generics. That is, if a varargs formal parameter of type Foo... was translated to List>, the method body would be suitably restricted. Redefining varargs in this way is not in scope for Project Lambda.) All the above applies whether the varargs formal parameter is a parameterized type or a function type. Neither kind of type is going to be reified in the near future. Alex > However, that warning is moving to the target method (it's one of project > coin's features). I proposed at the time that this warning too can be > eliminated if the following holds: > > The array variable is only read from, iterated over (foreach), or a > method/field access (.length, .toString()) done on it. > > Or, to be more specific: > > 1. The array variable is not used in any expression, even as a primitive > (Object o = varargsParam;), except for: > > 1A. LHS of a method invocation, so, varargsParam.toString() is acceptable > (not much use in doing it, but it won't cause the warning). > > 1B. LHS of a field access, so, varargsParam.length is okay. > > 1C. Used as a primitive expression as the iterable in a foreach, so, for (T > x : varargsParam) is okay. > > 2. array assignment to the varargsParam does not occur. So, > varargsParam[expr] = whatever; means the warning appears. > > > If the varargs method adheres to all these rules, then the warning is not > shown. None of this seems prohibitive to scan for (none of it requires type > resolution, for example), it's sufficient for type failures due to lack of > reification to be impossible, and (guesstimating) it'll cover 99%+ of all > uses of varargs out there. From howard.lovatt at gmail.com Thu Feb 18 12:39:07 2010 From: howard.lovatt at gmail.com (Howard Lovatt) Date: Fri, 19 Feb 2010 07:39:07 +1100 Subject: Function types versus arrays In-Reply-To: <15e8b9d21002180736t14ad9bdavb5830c18dfffd14b@mail.gmail.com> References: <3dd3f56a1002081452o36d2b695kc766842ec2e8348b@mail.gmail.com> <15e8b9d21002082129i21a209cel925ad8b6ba3313cd@mail.gmail.com> <3dd3f56a1002082201l304a5473ge2859fcc52032913@mail.gmail.com> <560fb5ed1002122022u27f0ed29x44d7f821ddb44ab5@mail.gmail.com> <15e8b9d21002122239p158cf4d3vfc45faaa2b48dced@mail.gmail.com> <3dd3f56a1002151357m2d5b13bs5e078c69d604e31b@mail.gmail.com> <15e8b9d21002151448i2ca40f8fpbf40c70caffeab2c@mail.gmail.com> <3dd3f56a1002162058j7f6e248cj197705273c2054b3@mail.gmail.com> <15e8b9d21002162126l26f89047q22d25e953d55be6@mail.gmail.com> <3dd3f56a1002180144o6aa115ddi45e56a2e1d3d0071@mail.gmail.com> <15e8b9d21002180736t14ad9bdavb5830c18dfffd14b@mail.gmail.com> Message-ID: Neal, Good point, something I had missed. I will think on it. Thanks for finding a bug, -- Howard On 19/02/2010, at 2:36 AM, Neal Gafter wrote: > On Thu, Feb 18, 2010 at 1:44 AM, Howard Lovatt > wrote: >> Hi Neal, >> >> On 17 February 2010 16:26, Neal Gafter wrote: >>> Howard- >>> >>> Can you please show the implementation of >>> _Array_From_void_throws_ChangedCharSetException_SAXParseException_UnkownHostException__To_void_throws_AWTException_IOException_SAXException.instance? >> >> Sure: > > This implementation is incorrect, because the two arrays must act as > one. This method is supposed to implement the subtype relationship > between two array types. Assigning to an element of one array value > must modify the corresponding element of the array value which is its > subtype. But with your implementation, the two arrays are > disconnected. > > ______________________________________________________________________ > This email has been scanned by the MessageLabs Email Security System. > For more information please visit http://www.messagelabs.com/email > ______________________________________________________________________ From tronicek at fit.cvut.cz Thu Feb 18 12:49:48 2010 From: tronicek at fit.cvut.cz (Zdenek Tronicek) Date: Thu, 18 Feb 2010 21:49:48 +0100 Subject: Function types versus arrays In-Reply-To: <17b2302a1002181144w3873b491q7795ce4768f88330@mail.gmail.com> References: <3dd3f56a1002081452o36d2b695kc766842ec2e8348b@mail.gmail.com> <15e8b9d21002151448i2ca40f8fpbf40c70caffeab2c@mail.gmail.com> <3dd3f56a1002162058j7f6e248cj197705273c2054b3@mail.gmail.com> <15e8b9d21002162126l26f89047q22d25e953d55be6@mail.gmail.com> <17b2302a1002170925i28e0e7ceu33c30f9c4172998a@mail.gmail.com> <15e8b9d21002170939n43e51bfeie7fa2a01a5cab4c9@mail.gmail.com> <17b2302a1002171439o467b40f0lf46d96f004b2feca@mail.gmail.com> <15e8b9d21002171448p7177fe3crfd64a2846307c13d@mail.gmail.com> <17b2302a1002171459n6212bb2em458174add2f83afd@mail.gmail.com> <15e8b9d21002180945m5ed8cd97mb3dd7d1088005de6@mail.gmail.com> <17b2302a1002181144w3873b491q7795ce4768f88330@mail.gmail.com> Message-ID: <810216bb6f4d7fe6c015fe6532bb2b4e.squirrel@imap.fit.cvut.cz> Could you have arrays allowed in JVM and disallowed in the language? On the JVM level, function types are implemented as interfaces. So, #int(int)... fns would have been compiled as array but you were not allowed to write #int(int)[] fns. Z. -- Zdenek Tronicek FIT CTU in Prague Joshua Bloch napsal(a): > Neal, > > Perhaps I'm being naive, but it appears to me that allowing: > > #int(int) compose(#int(int)... fns) { } > > implies the existence of arrays of (erased) function types. I thought you > were arguing against the very existence of types representing arrays of > function types. Alex's suggestion that these types be treated analogously > to > arrays of other nonreifiable types (i.e., most generic and parameterized > types) makes good sense to me. > > Josh > > On Thu, Feb 18, 2010 at 9:45 AM, Neal Gafter wrote: > >> On Wed, Feb 17, 2010 at 2:59 PM, Joshua Bloch wrote: >> > consider a method that takes a bunch of #int(int) functions and >> > returns their composition: >> > #int(int) compose(#int(int)... fns) { } >> > This should definitely be legal, and shouldn't generate a warning >> > (especially given that we're providing a means to eliminate this sort >> of >> > warning on generic varargs parameters in Java 7). >> >> This doesn't conflict with any of the restrictions I proposed. >> > > From Alex.Buckley at Sun.COM Thu Feb 18 12:56:12 2010 From: Alex.Buckley at Sun.COM (Alex Buckley) Date: Thu, 18 Feb 2010 12:56:12 -0800 Subject: Function types versus arrays In-Reply-To: <810216bb6f4d7fe6c015fe6532bb2b4e.squirrel@imap.fit.cvut.cz> References: <3dd3f56a1002081452o36d2b695kc766842ec2e8348b@mail.gmail.com> <15e8b9d21002151448i2ca40f8fpbf40c70caffeab2c@mail.gmail.com> <3dd3f56a1002162058j7f6e248cj197705273c2054b3@mail.gmail.com> <15e8b9d21002162126l26f89047q22d25e953d55be6@mail.gmail.com> <17b2302a1002170925i28e0e7ceu33c30f9c4172998a@mail.gmail.com> <15e8b9d21002170939n43e51bfeie7fa2a01a5cab4c9@mail.gmail.com> <17b2302a1002171439o467b40f0lf46d96f004b2feca@mail.gmail.com> <15e8b9d21002171448p7177fe3crfd64a2846307c13d@mail.gmail.com> <17b2302a1002171459n6212bb2em458174add2f83afd@mail.gmail.com> <15e8b9d21002180945m5ed8cd97mb3dd7d1088005de6@mail.gmail.com> <17b2302a1002181144w3873b491q7795ce4768f88330@mail.gmail.com> <810216bb6f4d7fe6c015fe6532bb2b4e.squirrel@imap.fit.cvut.cz> Message-ID: <4B7DA96C.4020502@sun.com> No. 1) No-one says function types are implemented as interfaces in Lambda. 2) Even if function types were implemented as interfaces, they would be implemented with generic interfaces, and then lack of reification for generics bites you. Other mails explain the problem. Alex Zdenek Tronicek wrote: > Could you have arrays allowed in JVM and disallowed in the language? > On the JVM level, function types are implemented as interfaces. So, > > #int(int)... fns > > would have been compiled as array but you were not allowed to write > > #int(int)[] fns. > > Z. From tronicek at fit.cvut.cz Thu Feb 18 13:12:32 2010 From: tronicek at fit.cvut.cz (Zdenek Tronicek) Date: Thu, 18 Feb 2010 22:12:32 +0100 Subject: Function types versus arrays In-Reply-To: <4B7DA96C.4020502@sun.com> References: <3dd3f56a1002081452o36d2b695kc766842ec2e8348b@mail.gmail.com> <15e8b9d21002151448i2ca40f8fpbf40c70caffeab2c@mail.gmail.com> <3dd3f56a1002162058j7f6e248cj197705273c2054b3@mail.gmail.com> <15e8b9d21002162126l26f89047q22d25e953d55be6@mail.gmail.com> <17b2302a1002170925i28e0e7ceu33c30f9c4172998a@mail.gmail.com> <15e8b9d21002170939n43e51bfeie7fa2a01a5cab4c9@mail.gmail.com> <17b2302a1002171439o467b40f0lf46d96f004b2feca@mail.gmail.com> <15e8b9d21002171448p7177fe3crfd64a2846307c13d@mail.gmail.com> <17b2302a1002171459n6212bb2em458174add2f83afd@mail.gmail.com> <15e8b9d21002180945m5ed8cd97mb3dd7d1088005de6@mail.gmail.com> <17b2302a1002181144w3873b491q7795ce4768f88330@mail.gmail.com> <810216bb6f4d7fe6c015fe6532bb2b4e.squirrel@imap.fit.cvut.cz> <4B7DA96C.4020502@sun.com> Message-ID: <2df8aa7177a3bc2890979ae7709cf194.squirrel@imap.fit.cvut.cz> Really? If arrays of function type are not allowed, you are naturally not allowed to use indexing on varargs: #int(int) compose(#int(int)... fns) { fns[0] = ... // this is not allowed } But you are right, this has been discussed here. Z. -- Zdenek Tronicek FIT CTU in Prague Alex Buckley napsal(a): > No. > > 1) No-one says function types are implemented as interfaces in Lambda. > > 2) Even if function types were implemented as interfaces, they would be > implemented with generic interfaces, and then lack of reification for > generics bites you. Other mails explain the problem. > > Alex > > Zdenek Tronicek wrote: >> Could you have arrays allowed in JVM and disallowed in the language? >> On the JVM level, function types are implemented as interfaces. So, >> >> #int(int)... fns >> >> would have been compiled as array but you were not allowed to write >> >> #int(int)[] fns. >> >> Z. > From Alex.Buckley at Sun.COM Thu Feb 18 13:31:54 2010 From: Alex.Buckley at Sun.COM (Alex Buckley) Date: Thu, 18 Feb 2010 13:31:54 -0800 Subject: Function types versus arrays In-Reply-To: <2df8aa7177a3bc2890979ae7709cf194.squirrel@imap.fit.cvut.cz> References: <3dd3f56a1002081452o36d2b695kc766842ec2e8348b@mail.gmail.com> <15e8b9d21002151448i2ca40f8fpbf40c70caffeab2c@mail.gmail.com> <3dd3f56a1002162058j7f6e248cj197705273c2054b3@mail.gmail.com> <15e8b9d21002162126l26f89047q22d25e953d55be6@mail.gmail.com> <17b2302a1002170925i28e0e7ceu33c30f9c4172998a@mail.gmail.com> <15e8b9d21002170939n43e51bfeie7fa2a01a5cab4c9@mail.gmail.com> <17b2302a1002171439o467b40f0lf46d96f004b2feca@mail.gmail.com> <15e8b9d21002171448p7177fe3crfd64a2846307c13d@mail.gmail.com> <17b2302a1002171459n6212bb2em458174add2f83afd@mail.gmail.com> <15e8b9d21002180945m5ed8cd97mb3dd7d1088005de6@mail.gmail.com> <17b2302a1002181144w3873b491q7795ce4768f88330@mail.gmail.com> <810216bb6f4d7fe6c015fe6532bb2b4e.squirrel@imap.fit.cvut.cz> <4B7DA96C.4020502@sun.com> <2df8aa7177a3bc2890979ae7709cf194.squirrel@imap.fit.cvut.cz> Message-ID: <4B7DB1CA.3040907@sun.com> As I have said elsewhere, everything that applies to parameterized types applies to function types. Varargs of function types will be allowed with the same warnings (and mechanisms to turn off warnings) as varargs of parameterized types. Alex Zdenek Tronicek wrote: > Really? If arrays of function type are not allowed, you are naturally not > allowed to use indexing on varargs: > > #int(int) compose(#int(int)... fns) { > fns[0] = ... // this is not allowed > } > > But you are right, this has been discussed here. > > Z. From reinier at zwitserloot.com Thu Feb 18 15:56:50 2010 From: reinier at zwitserloot.com (Reinier Zwitserloot) Date: Fri, 19 Feb 2010 00:56:50 +0100 Subject: Are function types a requirement? In-Reply-To: <4B7DA002.1080501@sun.com> References: <4B7C92A6.9020008@joda.org> <560fb5ed1002172006n40851460i3548077b3a74c1e6@mail.gmail.com> <15e8b9d21002172016g3482dcaao3e94f47b6666efe3@mail.gmail.com> <560fb5ed1002172201s1c1eae5am4dd7d8f8627e7eae@mail.gmail.com> <4B7DA002.1080501@sun.com> Message-ID: <560fb5ed1002181556p61e3dabr472c0842dba8572@mail.gmail.com> Sure, alex, varargs being generified is a superior solution on the face of it compared to my somewhat hacky 'disallow anything that would lose dynamic type safety', but, it's pragmatic. I've made the following assumptions: 1. Changing varargs to become List (well, generics, could be anything) based is very very unlikely as there's no way to do it without breaking backwards compatibility. 2. While my rules may seem arbitrary when you don't know why they are being applied, the enjoy the advantage that 99.99999% of all actual usage of varargs out there already adheres to them; in my experience the only thing anyone ever does with varargs params is check them against null occasionally, and other than that loop through them either via .length / array acces, or foreach. In the exceedingly rare cases where my rules are broken, the warning is definitely warranted, and should probably be reworded in more stern language, treating a vargs parameter as a standard array and doing dynamic type modifications on it _really_ is going to bite you in the behind. Perhaps the proposal can be reframed to say that the compiler assumes the varargs parameter is in fact like generics, and when treating it that way leads to type errors, the warning is generated. If it doesn't, no warning is needed. That may help keep the size of the JLS change managable. Either way, it solves the varargs problem for the *vast majority* of real-world varargs use in a totally transparent way, and it can be implemented without any worry about backwards compatibility. That's gotta worth some points. --Reinier Zwitserloot On Thu, Feb 18, 2010 at 9:16 PM, Alex Buckley wrote: > Reinier Zwitserloot wrote: > > On a different note, generified varargs are legal. You just get a > warning. > > This is effectively incorrect. > > A varargs formal parameter that is parameterized ought to be illegal. > (At least where varargs are implemented with arrays.) > > It breaks the property that static type correctness implies dynamic type > safety. We allow such a parameter only for convenience. A warning > indicates that you are being allowed to hang yourself for your own > convenience. The word "just" has no place here. > > What you propose below (and it's really very clever) is to strengthen > the static type system in the body of a varargs method. Your rules are > so strong that if the body is statically type correct, then it will be > dynamically type safe. You have achieved this by basically banning any > language construct whose static typechecking involves subtype rules. For > example, aliasing the varargs formal parameter through an Object[] - the > usual way to lose dynamic type safety - is banned in your static type > system, and code that did it would cause a warning. > > I discussed restrictions of this kind extensively with Josh Bloch and > Bob Lee before Bob's Coin proposal. The summary is that I'm not a fan, > which is why the proposal doesn't change the static type system and thus > requires a very strong programmer declaration that the method body will > be dynamically type safe promise promise promise. > > (Strictly speaking, I'm not a fan of building this alternative static > type system *on top of array-based varargs*. You get your strong static > typechecking for free with covariant generics. That is, if a varargs > formal parameter of type Foo... was translated to List Foo>, the method body would be suitably restricted. Redefining > varargs in this way is not in scope for Project Lambda.) > > All the above applies whether the varargs formal parameter is a > parameterized type or a function type. Neither kind of type is going to > be reified in the near future. > > Alex > > > However, that warning is moving to the target method (it's one of project > > coin's features). I proposed at the time that this warning too can be > > eliminated if the following holds: > > > > The array variable is only read from, iterated over (foreach), or a > > method/field access (.length, .toString()) done on it. > > > > Or, to be more specific: > > > > 1. The array variable is not used in any expression, even as a primitive > > (Object o = varargsParam;), except for: > > > > 1A. LHS of a method invocation, so, varargsParam.toString() is > acceptable > > (not much use in doing it, but it won't cause the warning). > > > > 1B. LHS of a field access, so, varargsParam.length is okay. > > > > 1C. Used as a primitive expression as the iterable in a foreach, so, for > (T > > x : varargsParam) is okay. > > > > 2. array assignment to the varargsParam does not occur. So, > > varargsParam[expr] = whatever; means the warning appears. > > > > > > If the varargs method adheres to all these rules, then the warning is not > > shown. None of this seems prohibitive to scan for (none of it requires > type > > resolution, for example), it's sufficient for type failures due to lack > of > > reification to be impossible, and (guesstimating) it'll cover 99%+ of all > > uses of varargs out there. > > From Alex.Buckley at Sun.COM Thu Feb 18 17:10:12 2010 From: Alex.Buckley at Sun.COM (Alex Buckley) Date: Thu, 18 Feb 2010 17:10:12 -0800 Subject: Project Lambda: Java Language Specification draft 0.1.5 In-Reply-To: <15e8b9d21002121904u55b47769w9449c113709dc54a@mail.gmail.com> References: <4B75FEA2.9080904@sun.com> <15e8b9d21002121904u55b47769w9449c113709dc54a@mail.gmail.com> Message-ID: <4B7DE4F4.6030603@sun.com> Neal, Thanks for your comments. I agree with most and have either resolved them, or clarified why they don't apply, in the draft spec. Some points remain: Neal Gafter wrote: > - It is unclear what the type of "this" is in an expression lambda. Is > it forbidden? > > - I wonder if the identity of a lambda can change spuriously. For > example, if "this" refers to the lambda expression, can it be expected > to have the same value on each invocation of the same lambda > expression? In other words, > > #Object() self = #()this; > assert self.() == self.(); // guaranteed?? > > - Similar question if the lambda has been converted to a SAM > > Object() self1 = #()this; > Callable self2 = self1; > assert self1.() == self2.call(); // guaranteed?? If 'this' is banned in an expression lambda, and in returns of a statement lambda, then a lambda instance's identity cannot be exposed, and the question is moot. > - It isn't clear why you would want special rules for recursive > lambdas in this revision of the spec, given that one could just invoke > "this" inside the lambda. Or, failing that, just use a method or an > anonymous inner class. To which special rules do you refer? > - The spec "It is a compile-time error to modify the value of an > effectively-final variable in the body of a lambda expression." is > meaningless. If there were an assignment to a variable from an > enclosing scope in a lambda, the variable would not be definitely > unassigned at that point, and so by definition the variable would not > be effectively-final. So this rule could never be applied in the way > you appear to intend. True. How about: It is a compile-time error to modify the value of a variable that is effectively-final outside the body of a lambda expression, from inside the body of the lambda expression. Alex From jjb at google.com Thu Feb 18 17:24:06 2010 From: jjb at google.com (Joshua Bloch) Date: Thu, 18 Feb 2010 17:24:06 -0800 Subject: Project Lambda: Java Language Specification draft 0.1.5 In-Reply-To: <4B7DE4F4.6030603@sun.com> References: <4B75FEA2.9080904@sun.com> <15e8b9d21002121904u55b47769w9449c113709dc54a@mail.gmail.com> <4B7DE4F4.6030603@sun.com> Message-ID: <17b2302a1002181724k40ce8cc4vc88987037ce42413@mail.gmail.com> Alex, On Thu, Feb 18, 2010 at 5:10 PM, Alex Buckley wrote: > Neal, > > Thanks for your comments. I agree with most and have either resolved > them, or clarified why they don't apply, in the draft spec. Some points > remain: > > Neal Gafter wrote: > > - It is unclear what the type of "this" is in an expression lambda. Is > > it forbidden? > > > > - I wonder if the identity of a lambda can change spuriously. For > > example, if "this" refers to the lambda expression, can it be expected > > to have the same value on each invocation of the same lambda > > expression? In other words, > > > > #Object() self = #()this; > > assert self.() == self.(); // guaranteed?? > > > > - Similar question if the lambda has been converted to a SAM > > > > Object() self1 = #()this; > > Callable self2 = self1; > > assert self1.() == self2.call(); // guaranteed?? > > If 'this' is banned in an expression lambda, and in returns of a > statement lambda, then a lambda instance's identity cannot be exposed, > and the question is moot. > I don't believe this (no pun intended). The ban on "this" in returns from a statement lambda was to prevent circularities in the type inference, and it's easy to propagate the object reference: #Object() whoAmI = #{Object me = this; return me;} But I would hope that the answer to Neal's question is yes: the identity of a lambda cannot change. Josh > > From neal at gafter.com Thu Feb 18 18:19:04 2010 From: neal at gafter.com (Neal Gafter) Date: Thu, 18 Feb 2010 18:19:04 -0800 Subject: Are function types a requirement? In-Reply-To: <560fb5ed1002181556p61e3dabr472c0842dba8572@mail.gmail.com> References: <4B7C92A6.9020008@joda.org> <560fb5ed1002172006n40851460i3548077b3a74c1e6@mail.gmail.com> <15e8b9d21002172016g3482dcaao3e94f47b6666efe3@mail.gmail.com> <560fb5ed1002172201s1c1eae5am4dd7d8f8627e7eae@mail.gmail.com> <4B7DA002.1080501@sun.com> <560fb5ed1002181556p61e3dabr472c0842dba8572@mail.gmail.com> Message-ID: <15e8b9d21002181819w2671803fwe72131ec98abedb6@mail.gmail.com> On Thu, Feb 18, 2010 at 3:56 PM, Reinier Zwitserloot wrote: > 2. While my rules may seem arbitrary when you don't know why they are being > applied, the enjoy the advantage that 99.99999% of all actual usage of > varargs out there already adheres to them; in my experience the only thing > anyone ever does with varargs params is check them against null > occasionally, and other than that loop through them either via .length / > array acces, or foreach. In my experience, it is not uncommon to have one varargs method "chain" to another varargs method by passing the array along. I suspect your "99.99999%" statistic is inaccurate. From neal at gafter.com Thu Feb 18 18:42:06 2010 From: neal at gafter.com (Neal Gafter) Date: Thu, 18 Feb 2010 18:42:06 -0800 Subject: Project Lambda: Java Language Specification draft 0.1.5 In-Reply-To: <4B7DE4F4.6030603@sun.com> References: <4B75FEA2.9080904@sun.com> <15e8b9d21002121904u55b47769w9449c113709dc54a@mail.gmail.com> <4B7DE4F4.6030603@sun.com> Message-ID: <15e8b9d21002181842x5d23b139u4e77606dc988dd85@mail.gmail.com> On Thu, Feb 18, 2010 at 5:10 PM, Alex Buckley wrote: > If 'this' is banned in an expression lambda, and in returns of a > statement lambda, then a lambda instance's identity cannot be exposed, > and the question is moot. As Josh pointed out, if "this" refers to the lambda anywhere inside the lambda, then its identity is observable. Another problem with having "this" refer to the lambda that we did not discuss before is that it does not compose well. Specifically, a lambda written within another lambda has no way to refer to "this" of the enclosing lambda. >> - It isn't clear why you would want special rules for recursive >> lambdas in this revision of the spec, given that one could just invoke >> "this" inside the lambda. ?Or, failing that, just use a method or an >> anonymous inner class. > > To which special rules do you refer? I'm referring to the discussion of possible rules, including the phrase "Current options for self-reference include:" >> - The spec "It is a compile-time error to modify the value of an >> effectively-final variable in the body of a lambda expression." is >> meaningless. ?If there were an assignment to a variable from an >> enclosing scope in a lambda, the variable would not be definitely >> unassigned at that point, and so by definition the variable would not >> be effectively-final. ?So this rule could never be applied in the way >> you appear to intend. > > True. How about: It is a compile-time error to modify the value of a > variable that is effectively-final outside the body of a lambda > expression, from inside the body of the lambda expression. Again, unnecessary, because such a variable does not fit the definition of effectively final. Specifically, the variable is not definitely unassigned at the point of its assignment (in the lambda), and so it is not effectively final. I think it is best just to delete this rule as being redundant with other more precise rules. From jjb at google.com Thu Feb 18 19:17:29 2010 From: jjb at google.com (Joshua Bloch) Date: Thu, 18 Feb 2010 19:17:29 -0800 Subject: Project Lambda: Java Language Specification draft 0.1.5 In-Reply-To: <15e8b9d21002181842x5d23b139u4e77606dc988dd85@mail.gmail.com> References: <4B75FEA2.9080904@sun.com> <15e8b9d21002121904u55b47769w9449c113709dc54a@mail.gmail.com> <4B7DE4F4.6030603@sun.com> <15e8b9d21002181842x5d23b139u4e77606dc988dd85@mail.gmail.com> Message-ID: <17b2302a1002181917y6ec7982bq61eb122667ea63b4@mail.gmail.com> Neal, On Thu, Feb 18, 2010 at 6:42 PM, Neal Gafter wrote: > On Thu, Feb 18, 2010 at 5:10 PM, Alex Buckley > wrote: > > If 'this' is banned in an expression lambda, and in returns of a > > statement lambda, then a lambda instance's identity cannot be exposed, > > and the question is moot. > > As Josh pointed out, if "this" refers to the lambda anywhere inside > the lambda, then its identity is observable. > > Another problem with having "this" refer to the lambda I don't see the observability of "this" as a problem; I see it as a desirable feature. that we did not > discuss before is that it does not compose well. Specifically, a > lambda written within another lambda has no way to refer to "this" of > the enclosing lambda. > I see this as a corner case, with a workaround: the enclosing lambda can stash its identity in order to make it available to the enclosed lambda. Regards, Josh From neal at gafter.com Thu Feb 18 20:25:06 2010 From: neal at gafter.com (Neal Gafter) Date: Thu, 18 Feb 2010 20:25:06 -0800 Subject: Project Lambda: Java Language Specification draft 0.1.5 In-Reply-To: <17b2302a1002181917y6ec7982bq61eb122667ea63b4@mail.gmail.com> References: <4B75FEA2.9080904@sun.com> <15e8b9d21002121904u55b47769w9449c113709dc54a@mail.gmail.com> <4B7DE4F4.6030603@sun.com> <15e8b9d21002181842x5d23b139u4e77606dc988dd85@mail.gmail.com> <17b2302a1002181917y6ec7982bq61eb122667ea63b4@mail.gmail.com> Message-ID: <15e8b9d21002182025i3bd00c31ue8bdb4b03ba557c3@mail.gmail.com> On Thu, Feb 18, 2010 at 7:17 PM, Joshua Bloch wrote: >> Another problem with having "this" refer to the lambda that we did not >> discuss before is that it does not compose well. ?Specifically, a >> lambda written within another lambda has no way to refer to "this" of >> the enclosing lambda. > I see this as a corner case, with a workaround: the enclosing lambda can > stash its identity in order to make it available to the enclosed lambda. Alternatively, we can eliminate it as a corner case altogether, by making "stashing" a function value in a variable the only way to get at it by some name. From neal at gafter.com Thu Feb 18 20:51:20 2010 From: neal at gafter.com (Neal Gafter) Date: Thu, 18 Feb 2010 20:51:20 -0800 Subject: Performance implications of the lambda draft specification Message-ID: <15e8b9d21002182051j3099a8c4td54273e30ed7a516@mail.gmail.com> An unfortunate consequence of a closures specification/implementation would be that it is twice as costly to create as an "equivalent" anonymous inner class, and twice as costly to invoke as an "equivalent" anonymous inner class. But that appears to be where the specification in its current form is taking us. The performance costs arise from a number of features of the current specification (1) The requirement that "this" inside a lambda refers to the lambda itself; (3) The likelihood of implementing the specification using MethodHandle for function types; (2) The desire to support SAM classes; and finally (4) The lambda conversion as a way of converting a value of function type to a SAM type. The creation cost I'm concerned about is double allocation: first allocating an object of the function type, and then allocating an object of the SAM type. The invocation cost I'm concerned about is double invocation: the client's invocation of the SAM's method, and then the invocation of the function object inside the compiler-generated implementation of the SAM. By contrast, these problems don't arise in BGGA because there is no function object (or function type) distinct from the SAM, and the lambda conversion occurs at compile-time (not runtime). Cheers, Neal From jjb at google.com Thu Feb 18 20:58:01 2010 From: jjb at google.com (Joshua Bloch) Date: Thu, 18 Feb 2010 20:58:01 -0800 Subject: Lambda Conversion Considered Harmful? Message-ID: <17b2302a1002182058k72e04d04h8fa85c2b409288dc@mail.gmail.com> As I'm sure you're aware, I'm very much in favor of the goal of the Lambda conversion: the new syntax (or something like it) should be usable in combination with existing constructs that require a "function object" such as a Comparator or (perhaps) a TimerTask. But I'm increasingly skeptical that the Lambda conversion (as described in the proposal) is the best way to achieve this. I'm skeptical for two reasons. The first concerns the semantics. Simply put, function types just get in the way when what you want is a SAM type. They raise all sorts of thorny issues, many of which we've been discussing recently. Is the function-typed object the same object as the SAM-typed object, or are there two objects involved? Is there any way to access the members of the SAM type? And so on. The second reason I'm skeptical concerns performance. The latest spec draft says this: Implementing lambda conversion will likely generate new objects to wrap or > delegate to the lambda expression. This is acceptable in assignment and > method invocation conversion contexts, but not in a casting conversion > context. (See > http://mail.openjdk.java.net/pipermail/lambda-dev/2010-January/000449.html > ) I fear that this will have disastrous performance consequences. Presented with a lovely new syntax, programmers will naturally use it, e.g., to make Comparators: Arrays.sort(a, #(String s1, String s2)(return s1.length - s2.length)); But the comparator's sole method is invoked in the inner loop of the sort! The slowdown caused by the delegation will add to the constant factor of the (n log n) sort. This could be very costly! I believe it violates a fundamental tenet of Java's design, which has been called *transparency*: magic should be kept to a minimum, and the code should reflect its performance (to the extent that that's feasible in this day and age). The for-each loop, by contrast, does not create an Iterator when traversing an array, so it runs as fast as the traditional for loop that it replaces. This was very important to me when we were designing the for-each construct, and I'm glad we did it the way we did. Therefore, I think we should *very *seriously consider an alternative design that uses a slightly different syntax when furnishing instances of SAM types that it does when furnishing instances of function types. Call them SAM-lambdas and function-lambads (for the time being). Perhaps the SAM lambdas could take the type as a mandatory argument, to simplify the compiler's job: Arrays.sort(a, #Comparator(String s1, String s2)(return s1.length - s2.length)); I have no idea whether this is the best syntax, or even an acceptable syntax. The important point is that we need something that: (1) Interoperates with existing methods that require "function objects" (2) Provides performance equal to current idioms, and (3) Provide conciseness close to that of lambdas (as specified in the current proposal) As a clarification, I am not arguing for the elimination of function types; rather I'm arguing that they get in the way of "interoperable function objects." so we need two separate syntaxes, one for function types and one for SAM types. Josh From opinali at gmail.com Fri Feb 19 05:44:16 2010 From: opinali at gmail.com (Osvaldo Doederlein) Date: Fri, 19 Feb 2010 11:44:16 -0200 Subject: Lambda Conversion Considered Harmful? In-Reply-To: <17b2302a1002182058k72e04d04h8fa85c2b409288dc@mail.gmail.com> References: <17b2302a1002182058k72e04d04h8fa85c2b409288dc@mail.gmail.com> Message-ID: Wow, it's great being able to agree with both Josh and Neal, this will raise my karma in at least 10 points. ;-) We definitely cannot have any perf regression for any lambda usage that replaces existing idioms, not even "only" for SAM type use-cases that could be sweeped under the rug of "backwards compatibility" - in practice this use-case will be vastly predominant for many years to come. I have exposed the opinion that lambdas could be a great opportunity to have _better_ performance than traditional idioms - either by allowing the compiler to aggressive inline control abstractions and other uses of literal, "soft" lambdas; or because lambdas may be compiled into MethodHandles which may be lightweight compared to bulky synthetic classes - but if this is not possible for any reason, we must at the very least follow Hippocrates' rule "first, do no harm" performance-wise. Considering this, my -1 to the hideously convoluted techniques that I see in proposals related to function types vs. arrays and lambda reification. The hacker inside me admires this ingenuity (assuming that it works), but the smell of complexity and bloat is just unbearable - and I though inner classes were bad. Call me old-fashioned, but if a couple lines of lambda code is desugared into multiple pages of Java code - even if most of it is adapter methods that can be reused in a few other usage sites with identical signatures - that's Just Wrong. Believe me, lambdas will be D.O.A. if, when SE7 ships, people start blogging that they updated some app that uses a lot of inner classes to use lambdas, and the resulting JAR file was +50% the old size. (Profile-driven, dynamic translation could be better, if it doesn't introduce new issues.) A+ Osvaldo 2010/2/19 Joshua Bloch > As I'm sure you're aware, I'm very much in favor of the goal of the Lambda > conversion: the new syntax (or something like it) should be usable in > combination with existing constructs that require a "function object" such > as a Comparator or (perhaps) a TimerTask. But I'm increasingly skeptical > that the Lambda conversion (as described in the proposal) is the best way > to > achieve this. > > I'm skeptical for two reasons. The first concerns the semantics. Simply > put, > function types just get in the way when what you want is a SAM type. They > raise all sorts of thorny issues, many of which we've been discussing > recently. Is the function-typed object the same object as the SAM-typed > object, or are there two objects involved? Is there any way to access the > members of the SAM type? And so on. > > The second reason I'm skeptical concerns performance. The latest spec > draft > says this: > > Implementing lambda conversion will likely generate new objects to wrap or > > delegate to the lambda expression. This is acceptable in assignment and > > method invocation conversion contexts, but not in a casting conversion > > context. (See > > > http://mail.openjdk.java.net/pipermail/lambda-dev/2010-January/000449.html > > ) > > > I fear that this will have disastrous performance consequences. Presented > with a lovely new syntax, programmers will naturally use it, e.g., to make > Comparators: > > Arrays.sort(a, #(String s1, String s2)(return s1.length - s2.length)); > > But the comparator's sole method is invoked in the inner loop of the sort! > The slowdown caused by the delegation will add to the constant factor of > the > (n log n) sort. This could be very costly! I believe it violates a > fundamental tenet of Java's design, which has been called *transparency*: > magic should be kept to a minimum, and the code should reflect its > performance (to the extent that that's feasible in this day and age). The > for-each loop, by contrast, does not create an Iterator when traversing an > array, so it runs as fast as the traditional for loop that it replaces. > This > was very important to me when we were designing the for-each construct, and > I'm glad we did it the way we did. > > Therefore, I think we should *very *seriously consider an alternative > design > that uses a slightly different syntax when furnishing instances of SAM > types > that it does when furnishing instances of function types. Call them > SAM-lambdas and function-lambads (for the time being). Perhaps the SAM > lambdas could take the type as a mandatory argument, to simplify the > compiler's job: > > Arrays.sort(a, #Comparator(String s1, String s2)(return s1.length - > s2.length)); > > I have no idea whether this is the best syntax, or even an acceptable > syntax. The important point is that we need something that: > > (1) Interoperates with existing methods that require "function objects" > (2) Provides performance equal to current idioms, and > (3) Provide conciseness close to that of lambdas (as specified in the > current proposal) > > As a clarification, I am not arguing for the elimination of function types; > rather I'm arguing that they get in the way of "interoperable function > objects." so we need two separate syntaxes, one for function types and one > for SAM types. > > Josh > > From takeshi10 at gmail.com Fri Feb 19 08:26:59 2010 From: takeshi10 at gmail.com (Marcelo Fukushima) Date: Fri, 19 Feb 2010 14:26:59 -0200 Subject: Performance implications of the lambda draft specification In-Reply-To: <15e8b9d21002182051j3099a8c4td54273e30ed7a516@mail.gmail.com> References: <15e8b9d21002182051j3099a8c4td54273e30ed7a516@mail.gmail.com> Message-ID: <7288749d1002190826w6086cd47he6467a1dbe010791@mail.gmail.com> are these concerns problematic in practice, though? i was under the impression that almost all the use cases for converting a function to a SAM would be optimized by hotspot anyway (stack allocation and method inlining), no? on the other hand, convertion at compile time seens feasible in the current (draft) spec, right? On Fri, Feb 19, 2010 at 2:51 AM, Neal Gafter wrote: > An unfortunate consequence of a closures specification/implementation > would be that it is twice as costly to create as an "equivalent" > anonymous inner class, and twice as costly to invoke as an > "equivalent" anonymous inner class. But that appears to be where the > specification in its current form is taking us. > > The performance costs arise from a number of features of the current > specification > (1) The requirement that "this" inside a lambda refers to the lambda > itself; > (3) The likelihood of implementing the specification using > MethodHandle for function types; > (2) The desire to support SAM classes; and finally > (4) The lambda conversion as a way of converting a value of function > type to a SAM type. > > The creation cost I'm concerned about is double allocation: first > allocating an object of the function type, and then allocating an > object of the SAM type. The invocation cost I'm concerned about is > double invocation: the client's invocation of the SAM's method, and > then the invocation of the function object inside the > compiler-generated implementation of the SAM. > > By contrast, these problems don't arise in BGGA because there is no > function object (or function type) distinct from the SAM, and the > lambda conversion occurs at compile-time (not runtime). > > Cheers, > Neal > > -- http://mapsdev.blogspot.com/ Marcelo Takeshi Fukushima From neal at gafter.com Fri Feb 19 09:18:50 2010 From: neal at gafter.com (Neal Gafter) Date: Fri, 19 Feb 2010 09:18:50 -0800 Subject: Performance implications of the lambda draft specification In-Reply-To: <7288749d1002190826w6086cd47he6467a1dbe010791@mail.gmail.com> References: <15e8b9d21002182051j3099a8c4td54273e30ed7a516@mail.gmail.com> <7288749d1002190826w6086cd47he6467a1dbe010791@mail.gmail.com> Message-ID: <15e8b9d21002190918o29f5b540p1adce98d1f80e929@mail.gmail.com> On Fri, Feb 19, 2010 at 8:26 AM, Marcelo Fukushima wrote: > are these concerns problematic in practice, though? i was under the > impression that almost all the use cases for converting a function to a SAM > would be optimized by hotspot anyway (stack allocation and method inlining), > no? > on the other hand, convertion at compile time seens feasible in the current > (draft) spec, right? These concerns are real. Optimizing two levels of nested object creation away is less likely than optimizing one level of object creation away. Conversion at compile-time can only be done in very restricted circumstances (for example, debugging disabled, no reference to "this", etc) and would be the only nontrivial optimization in javac. I think we're much better off not starting with a factor of two handicap. Cheers, Neal From scolebourne at joda.org Fri Feb 19 09:28:22 2010 From: scolebourne at joda.org (Stephen Colebourne) Date: Fri, 19 Feb 2010 17:28:22 +0000 Subject: Lightweight interfaces instead of function types Message-ID: <4b4f45e01002190928x5230dfeek2c1490dedccc71fd@mail.gmail.com> This is an outline of an idea that has appeared before wrt function types - lightweight interfaces. http://www.jroller.com/scolebourne/entry/closures_lightweight_interfaces_instead_of http://weblogs.java.net/blog/forax/archive/2008/01/yet_another_clo_1.html The time has come to mention the concept on this list. I'll express the "proposal" in informal language for now: As indicated previously, function types have issues in their interactions with arrays. Given, the lack of general reification, it also appears that reified function types are unlikely. By comparison, SAM interfaces have no such difficulties. You can hold an array of SAM interfaces safely today. If the SAM interface is generified, then there may be some associated warnings, but developers are familiar with those, and the effects on the type system are known. As such, the aim of this "proposal" is to achieve most (not all) of the goals of function types via SAM interfaces. 1) Lightweight interface: (The keyword interface is reused to save adding a new keyword) A lightweight interface consists of the keyword interface, an interface name and an interface method signature (minus the method name): interface FileJoiner String(Reader...) throws IOException; interface MathsOperation int(int, int); interface Comparer int(T, T); A lightweight interface has a direct translation to a standard SAM, which adds wildcards to generics: public interface FileJoiner extends Functor { String invoke(Reader...) throws IOException; } public interface MathsOperation extends Functor { int invoke(int, int); } public interface Comparer extends Functor { int invoke(? extends T, ? extends T); } The public/protected/package/module/private scope for the interface is inferred by usage (if its only used in a protected method signature, then the interface only needs to be protected, etc). I've added a no-method functor marker interface, but that may not be required. It may be necessary to prevent classes from implementing lightweight interfaces (which would favour a new keyword instead of interface). 2) Usage: The lightweight interface is used in place of a function type in a method/variable declaration. It is expected that the lightweight interface is defined just above the annotations on the method: interface Collector String(String...); interface Counter long(Collector, char); void storeHelloWorldCount(Counter counter, Collector collector) { database.storeCount( counter.invoke( collector.invoke("Hello", "World"), '-')); } The lightweight interface can also be defined within a method body: public int process() { interface MathsCombiner int(int,int); MathsCombiner add = #(int a, int b) {return a + b;}; MathsCombiner mul = #(int a, int b) {return a * b;}; return mul.invoke(add.invoke(2, 3), add.invoke(3, 4)); } As a lightweight interface is effectively equivalent to a SAM interface, the implementation of lambda expressions and blocks is effectively similar to the creation of an anonymous inner class (although I'll maintain my opposition to inner-class-scoped "this"). Other implementations may be possible too. 3) Conversions: There is a boxing conversion from a lightweight interfaces to an equivalent lightweight interfaces or SAM (but not from SAM interfaces in general). interface IntTransformer int(String); public IntTransformer constantProvider(int val) { return #(String str) val; } interface IntParser int(String); public String format(IntParser parser, String text) { return Integer.toString(parser.invoke(text)); } public void myApplicationMethod() { sysOut( format( constantProvider(6) ) ); } In this example, constantProvider() returns an IntTransformer, which is passed to format() which requires an IntParser. Because the source is a lightweight interface, it is boxed to allow it to be passed to format(). The equals() method is defined to work as == on the base/unboxed object. There may be opportunity to avoid physical boxing by dynamically adding the target interface to the source type (something that might be a useful JVM change for dynamic languages). Finally, its important to note that function types still exist in the compiler (as they do today). But they only exist for validation type conversion and boxing. 4) Arrays: Given Neals example: class Ex { static #String()[] array = new #String()[1]; static Object[] objectArray = array; static void impossible() { final T t = null; objectArray[0] = #() t; // assign type #T() } } this becomes: class Ex { interface StringProvider String() static StringProvider[] array = new StringProvider[1]; static Object[] objectArray = array; static void impossible() { final T t = null; objectArray[0] = #() t; } } Now, this fails to compile. There is no publicly available function type for #() t to be represented as, thus it cannot be assigned to Object (or into Object[]). There now needs to be a mechanism to "cast" to a StringProvider: objectArray[0] = (StringProvider) #() t; This "cast" would have enough information to either allow or deny this at compile or runtime to generate an error or warning. (Detail lacking here...!) Summary ------- Pros: - simple to learn/understand by developers (easy to guess the meaning without tuition) - "Transparant" mapping to existing syntax avoiding performance issues - Works with arrays (as far as SAMs work today) - Provides a meaningful name (and potentially documentation) for each part of the function type - Avoids issues of function types that take/return function types having very hard-to-read syntax - Should be implementable in JDK 7 timeline Cons: - Requires developers to name everything - Doesn't handle exception transparancy - Provides only a limited amount of interaction with co-contra variance - Cannot assign lambdas to Object without going via a suitable SAM/lightweight interface - Restricts implementation choices As always, its a compromise, but one worth considering. Stephen From jkuhnert at gmail.com Fri Feb 19 09:44:09 2010 From: jkuhnert at gmail.com (Jesse Kuhnert) Date: Fri, 19 Feb 2010 12:44:09 -0500 Subject: Lambda Conversion Considered Harmful? In-Reply-To: References: <17b2302a1002182058k72e04d04h8fa85c2b409288dc@mail.gmail.com> Message-ID: <7926817e1002190944p69511f6dhc92eb1ba1113dd55@mail.gmail.com> Would it be feasible/crazy talk to suggest toying with the idea of starting with the (assuming it is workable / feasible from a technical perspective) BGGA proposal to get things going on the right foot and attempt to modify / remove / etc things suit the general need of this project? The syntax alone could take almost as much time as figuring out how / what functionality to provide. Possibly.. On Fri, Feb 19, 2010 at 8:44 AM, Osvaldo Doederlein wrote: > Wow, it's great being able to agree with both Josh and Neal, this will raise > my karma in at least 10 points. ;-) We definitely cannot have any perf > regression for any lambda usage that replaces existing idioms, not even > "only" for SAM type use-cases that could be sweeped under the rug of > "backwards compatibility" - in practice this use-case will be vastly > predominant for many years to come. > > I have exposed the opinion that lambdas could be a great opportunity to have > _better_ performance than traditional idioms - either by allowing the > compiler to aggressive inline control abstractions and other uses of > literal, "soft" lambdas; or because lambdas may be compiled into > MethodHandles which may be lightweight compared to bulky synthetic classes - > but if this is not possible for any reason, we must at the very least follow > Hippocrates' rule "first, do no harm" performance-wise. > > Considering this, my -1 to the hideously convoluted techniques that I see in > proposals related to function types vs. arrays and lambda reification. The > hacker inside me admires this ingenuity (assuming that it works), but the > smell of complexity and bloat is just unbearable - and I though inner > classes were bad. Call me old-fashioned, but if a couple lines of lambda > code is desugared into multiple pages of Java code - even if most of it is > adapter methods that can be reused in a few other usage sites with identical > signatures - that's Just Wrong. Believe me, lambdas will be D.O.A. if, when > SE7 ships, people start blogging that they updated some app that uses a lot > of inner classes to use lambdas, and the resulting JAR file was +50% the old > size. (Profile-driven, dynamic translation could be better, if it doesn't > introduce new issues.) > > A+ > Osvaldo > > 2010/2/19 Joshua Bloch > >> As I'm sure you're aware, I'm very much in favor of the goal of the Lambda >> conversion: the new syntax (or something like it) should be usable in >> combination with existing constructs that require a "function object" such >> as a Comparator or (perhaps) a TimerTask. ?But I'm increasingly skeptical >> that the Lambda conversion (as described in the proposal) is the best way >> to >> achieve this. >> >> I'm skeptical for two reasons. The first concerns the semantics. Simply >> put, >> function types just get in the way when what you want is a SAM type. They >> raise all sorts of thorny issues, many of which we've been discussing >> recently. Is the function-typed object the same object as the SAM-typed >> object, or are there two objects involved? Is there any way to access the >> members of the SAM type? And so on. >> >> The second reason I'm skeptical concerns performance. ?The latest spec >> draft >> says this: >> >> Implementing lambda conversion will likely generate new objects to wrap or >> > delegate to the lambda expression. This is acceptable in assignment and >> > method invocation conversion contexts, but not in a casting conversion >> > context. (See >> > >> http://mail.openjdk.java.net/pipermail/lambda-dev/2010-January/000449.html >> > ) >> >> >> I fear that this will have disastrous performance consequences. Presented >> with a lovely new syntax, programmers will naturally use it, e.g., to make >> Comparators: >> >> ? ?Arrays.sort(a, #(String s1, String s2)(return s1.length - s2.length)); >> >> But the comparator's sole method is invoked in the inner loop of the sort! >> The slowdown caused by the delegation will add to the constant factor of >> the >> (n log n) sort. This could be very costly! I believe it violates a >> fundamental tenet of Java's design, which has been called *transparency*: >> ?magic should be kept to a minimum, and the code should reflect its >> performance (to the extent that that's feasible in this day and age). The >> for-each loop, by contrast, does not create an Iterator when traversing an >> array, so it runs as fast as the traditional for loop that it replaces. >> This >> was very important to me when we were designing the for-each construct, and >> I'm glad we did it the way we did. >> >> Therefore, I think we should *very *seriously consider an alternative >> design >> that uses a slightly different syntax when furnishing instances of SAM >> types >> that it does when furnishing instances of function types. ?Call them >> SAM-lambdas and function-lambads (for the time being). Perhaps the SAM >> lambdas could take the type as a mandatory argument, to simplify the >> compiler's job: >> >> ? ?Arrays.sort(a, #Comparator(String s1, String s2)(return s1.length - >> s2.length)); >> >> I have no idea whether this is the best syntax, or even an acceptable >> syntax. ?The important point is that we need something that: >> >> ? (1) Interoperates with existing methods that require "function objects" >> ? (2) Provides performance equal to current idioms, and >> ? (3) Provide conciseness close to that of lambdas (as specified in the >> current proposal) >> >> As a clarification, I am not arguing for the elimination of function types; >> rather I'm arguing that they get in the way of "interoperable function >> objects." so we need two separate syntaxes, one for function types and one >> for SAM types. >> >> ? ? ? ? ? ? Josh >> >> > > From neal at gafter.com Fri Feb 19 09:47:09 2010 From: neal at gafter.com (Neal Gafter) Date: Fri, 19 Feb 2010 09:47:09 -0800 Subject: Lightweight interfaces instead of function types In-Reply-To: <4b4f45e01002190928x5230dfeek2c1490dedccc71fd@mail.gmail.com> References: <4b4f45e01002190928x5230dfeek2c1490dedccc71fd@mail.gmail.com> Message-ID: <15e8b9d21002190947p4af23210o1e2e5d280144da60@mail.gmail.com> On Fri, Feb 19, 2010 at 9:28 AM, Stephen Colebourne wrote: > By comparison, SAM interfaces have no such difficulties. You can hold > an array of SAM interfaces safely today. If the SAM interface is > generified, then there may be some associated warnings, but developers > are familiar with those, and the effects on the type system are known. Today: Tmp.java:5: [error] generic array creation Callable[] array = new Callable[10]; ^ This is a hard error, not a warning. I have problems with your new reference conversions among interface types that are not subtypes of each other. They have all of the same problems as the lambda conversion that Josh and I are complaining about, and more. Fleshing out your proposal and its implications would likely show it to be more complex than the entire lambda proposal today. Cheers, Neal From mthornton at optrak.co.uk Fri Feb 19 09:55:42 2010 From: mthornton at optrak.co.uk (Mark Thornton) Date: Fri, 19 Feb 2010 17:55:42 +0000 Subject: Lightweight interfaces instead of function types In-Reply-To: <4b4f45e01002190928x5230dfeek2c1490dedccc71fd@mail.gmail.com> References: <4b4f45e01002190928x5230dfeek2c1490dedccc71fd@mail.gmail.com> Message-ID: <4B7ED09E.3090203@optrak.co.uk> Stephen Colebourne wrote: > Cons: > - Requires developers to name everything > Which still leaves an API like ParallelArray with 81 of them, all with names that do little more than repeat the types of their parameters. Mark From scolebourne at joda.org Fri Feb 19 10:15:38 2010 From: scolebourne at joda.org (Stephen Colebourne) Date: Fri, 19 Feb 2010 18:15:38 +0000 Subject: Lightweight interfaces instead of function types In-Reply-To: <15e8b9d21002190947p4af23210o1e2e5d280144da60@mail.gmail.com> References: <4b4f45e01002190928x5230dfeek2c1490dedccc71fd@mail.gmail.com> <15e8b9d21002190947p4af23210o1e2e5d280144da60@mail.gmail.com> Message-ID: <4b4f45e01002191015w4341c46dia6a548fe4bd5730f@mail.gmail.com> On 19 February 2010 17:47, Neal Gafter wrote: > Tmp.java:5: [error] generic array creation > ? ? ? ?Callable[] array = new Callable[10]; > This is a hard error, not a warning. True. But if I want a Callable[] I write: Callable[] array = new Callable[10]; This works, and is safe /in practice/ (rather than type-system safe). > I have problems with your new reference conversions among interface > types that are not subtypes of each other. ?They have all of the same > problems as the lambda conversion that Josh and I are complaining > about, and more. Wrapping an anonymous inner class implementing interface B around an object implementing interface A is a known concept in current Java. Yes there are downsides (the object identity changes, mitigated by the equals method rule) but overall, the key is to have a known language change complexity. On 19 February 2010 17:55, Mark Thornton wrote: > Stephen Colebourne wrote: >> Cons: >> - Requires developers to name everything > Which still leaves an API like ParallelArray with 81 of them, all with names > that do little more than repeat the types of their parameters. Yep. But at least they could be defined in one .java file using just 81 lines of code ;-) Again, its a trade-off. The benefit for ParallelArray is that Predicate or Transformer is IMO more meaningful to someone just casually coming to the API than a random function type. Stephen From opinali at gmail.com Fri Feb 19 10:56:25 2010 From: opinali at gmail.com (Osvaldo Doederlein) Date: Fri, 19 Feb 2010 16:56:25 -0200 Subject: Performance implications of the lambda draft specification In-Reply-To: <15e8b9d21002190918o29f5b540p1adce98d1f80e929@mail.gmail.com> References: <15e8b9d21002182051j3099a8c4td54273e30ed7a516@mail.gmail.com> <7288749d1002190826w6086cd47he6467a1dbe010791@mail.gmail.com> <15e8b9d21002190918o29f5b540p1adce98d1f80e929@mail.gmail.com> Message-ID: Just to add some 2c... a good rule of thumb IMHO, is planning to _enable_ optimizations (i.e., after higher-priority design aspects are satisfied, pick remaining choices to make optimizations possible/easier); but at the same time, NOT planning to _depend_ on any new optimization, unless it's something that we can really, really rely on. Brian Goetz has once [in]famously said "Go ahead, make a mess", wrt Java memory management. I think the intention was good: preventing non-expert developers from trying to out-smart modern garbage collectors with garage-quality object pools and such. But taken literally, the advise of making a mess, with blind faith on advanced compiler/runtime tech to undo that mess, is a disaster. We are always optimistic about cool new optimizations. But let's do a quick reality check on scalar replacement / stack allocation: - Has taken ages to be implemented in HotSpot. IIRC this was planned for 5.0 and maybe even earlier, and always moved to the next release. This means, this stuff is hard, and even though it's confimed to ship with JDK7-FCS, I won't trust on it completely (may ship with "Early Access" quality, may ship with some limitations, etc., so maybe we'll wait another year of maintenance updates to really rely on it). - Will ship only for HotSpot Server. There is no estimation of how long we must wait to get this in HotSpot Client. I'd bet in the JDK8-FCS timeframe - and this is because I am optimist. - Will not be available on sub-desktop runtimes (JavaME, or whatever ME evolves into) for even longer time. - Wherever and whenever available, this optimization may have some extra tradeoffs, at least extra JIT overheads. - This is a particularly "fragile" optimization, that may be lost after trivial, hard-to-spot code changes. For example, m1() allocates an object that only "escapes" as a parameter in a call to m2(), but we still get scalar replacement because m2() is inlined; however, after some trivial maintenance m2() becomes slightly bigger than the inlining budget for m1()'s callsite, then we lose the inlining and also the scalar replacement, and maybe get a order-of-magnitude slowdown. (Most specific optimizations, and remarkably optimization chains, exhibit this kind of fragility -- optimization is by nature brittle -- but the problem is more severe if we have entire language features that take such optimization as a precondition to not suck perf-wise.) Having said that I'm enthusiastic about the upcoming EA-based scalar replacement capability of JDK7, I've been testing it for a while. But much less enthusiastic about designing a language feature that is very important, and would depend on that optimization. A+ Osvaldo 2010/2/19 Neal Gafter > On Fri, Feb 19, 2010 at 8:26 AM, Marcelo Fukushima > wrote: > > are these concerns problematic in practice, though? i was under the > > impression that almost all the use cases for converting a function to a > SAM > > would be optimized by hotspot anyway (stack allocation and method > inlining), > > no? > > on the other hand, convertion at compile time seens feasible in the > current > > (draft) spec, right? > > These concerns are real. Optimizing two levels of nested object > creation away is less likely than optimizing one level of object > creation away. Conversion at compile-time can only be done in very > restricted circumstances (for example, debugging disabled, no > reference to "this", etc) and would be the only nontrivial > optimization in javac. I think we're much better off not starting > with a factor of two handicap. > > Cheers, > Neal > > From jjb at google.com Fri Feb 19 11:18:40 2010 From: jjb at google.com (Joshua Bloch) Date: Fri, 19 Feb 2010 11:18:40 -0800 Subject: Lambda Conversion Considered Harmful? In-Reply-To: <7926817e1002190944p69511f6dhc92eb1ba1113dd55@mail.gmail.com> References: <17b2302a1002182058k72e04d04h8fa85c2b409288dc@mail.gmail.com> <7926817e1002190944p69511f6dhc92eb1ba1113dd55@mail.gmail.com> Message-ID: <17b2302a1002191118n37713a1av283a456050d839f1@mail.gmail.com> Jesse, I don't think so. The BGGA proposal also uses the lambda conversion to produce SAM-type (interface only) implementations. I believe the lambda conversion is just getting in our way. I really do think we should explore a design where we have two distinct syntaxes: one for "structural function objects" (lambdas in the current proposal) and one for "nominal function objects" (SAM-type instances). Josh On Fri, Feb 19, 2010 at 9:44 AM, Jesse Kuhnert wrote: > Would it be feasible/crazy talk to suggest toying with the idea of > starting with the (assuming it is workable / feasible from a technical > perspective) BGGA proposal to get things going on the right foot and > attempt to modify / remove / etc things suit the general need of this > project? The syntax alone could take almost as much time as figuring > out how / what functionality to provide. Possibly.. > > On Fri, Feb 19, 2010 at 8:44 AM, Osvaldo Doederlein > wrote: > > Wow, it's great being able to agree with both Josh and Neal, this will > raise > > my karma in at least 10 points. ;-) We definitely cannot have any perf > > regression for any lambda usage that replaces existing idioms, not even > > "only" for SAM type use-cases that could be sweeped under the rug of > > "backwards compatibility" - in practice this use-case will be vastly > > predominant for many years to come. > > > > I have exposed the opinion that lambdas could be a great opportunity to > have > > _better_ performance than traditional idioms - either by allowing the > > compiler to aggressive inline control abstractions and other uses of > > literal, "soft" lambdas; or because lambdas may be compiled into > > MethodHandles which may be lightweight compared to bulky synthetic > classes - > > but if this is not possible for any reason, we must at the very least > follow > > Hippocrates' rule "first, do no harm" performance-wise. > > > > Considering this, my -1 to the hideously convoluted techniques that I see > in > > proposals related to function types vs. arrays and lambda reification. > The > > hacker inside me admires this ingenuity (assuming that it works), but the > > smell of complexity and bloat is just unbearable - and I though inner > > classes were bad. Call me old-fashioned, but if a couple lines of lambda > > code is desugared into multiple pages of Java code - even if most of it > is > > adapter methods that can be reused in a few other usage sites with > identical > > signatures - that's Just Wrong. Believe me, lambdas will be D.O.A. if, > when > > SE7 ships, people start blogging that they updated some app that uses a > lot > > of inner classes to use lambdas, and the resulting JAR file was +50% the > old > > size. (Profile-driven, dynamic translation could be better, if it doesn't > > introduce new issues.) > > > > A+ > > Osvaldo > > > > 2010/2/19 Joshua Bloch > > > >> As I'm sure you're aware, I'm very much in favor of the goal of the > Lambda > >> conversion: the new syntax (or something like it) should be usable in > >> combination with existing constructs that require a "function object" > such > >> as a Comparator or (perhaps) a TimerTask. But I'm increasingly > skeptical > >> that the Lambda conversion (as described in the proposal) is the best > way > >> to > >> achieve this. > >> > >> I'm skeptical for two reasons. The first concerns the semantics. Simply > >> put, > >> function types just get in the way when what you want is a SAM type. > They > >> raise all sorts of thorny issues, many of which we've been discussing > >> recently. Is the function-typed object the same object as the SAM-typed > >> object, or are there two objects involved? Is there any way to access > the > >> members of the SAM type? And so on. > >> > >> The second reason I'm skeptical concerns performance. The latest spec > >> draft > >> says this: > >> > >> Implementing lambda conversion will likely generate new objects to wrap > or > >> > delegate to the lambda expression. This is acceptable in assignment > and > >> > method invocation conversion contexts, but not in a casting conversion > >> > context. (See > >> > > >> > http://mail.openjdk.java.net/pipermail/lambda-dev/2010-January/000449.html > >> > ) > >> > >> > >> I fear that this will have disastrous performance consequences. > Presented > >> with a lovely new syntax, programmers will naturally use it, e.g., to > make > >> Comparators: > >> > >> Arrays.sort(a, #(String s1, String s2)(return s1.length - > s2.length)); > >> > >> But the comparator's sole method is invoked in the inner loop of the > sort! > >> The slowdown caused by the delegation will add to the constant factor of > >> the > >> (n log n) sort. This could be very costly! I believe it violates a > >> fundamental tenet of Java's design, which has been called > *transparency*: > >> magic should be kept to a minimum, and the code should reflect its > >> performance (to the extent that that's feasible in this day and age). > The > >> for-each loop, by contrast, does not create an Iterator when traversing > an > >> array, so it runs as fast as the traditional for loop that it replaces. > >> This > >> was very important to me when we were designing the for-each construct, > and > >> I'm glad we did it the way we did. > >> > >> Therefore, I think we should *very *seriously consider an alternative > >> design > >> that uses a slightly different syntax when furnishing instances of SAM > >> types > >> that it does when furnishing instances of function types. Call them > >> SAM-lambdas and function-lambads (for the time being). Perhaps the SAM > >> lambdas could take the type as a mandatory argument, to simplify the > >> compiler's job: > >> > >> Arrays.sort(a, #Comparator(String s1, String s2)(return s1.length - > >> s2.length)); > >> > >> I have no idea whether this is the best syntax, or even an acceptable > >> syntax. The important point is that we need something that: > >> > >> (1) Interoperates with existing methods that require "function > objects" > >> (2) Provides performance equal to current idioms, and > >> (3) Provide conciseness close to that of lambdas (as specified in the > >> current proposal) > >> > >> As a clarification, I am not arguing for the elimination of function > types; > >> rather I'm arguing that they get in the way of "interoperable function > >> objects." so we need two separate syntaxes, one for function types and > one > >> for SAM types. > >> > >> Josh > >> > >> > > > > > From neal at gafter.com Fri Feb 19 11:22:34 2010 From: neal at gafter.com (Neal Gafter) Date: Fri, 19 Feb 2010 11:22:34 -0800 Subject: Lambda Conversion Considered Harmful? In-Reply-To: <17b2302a1002191118n37713a1av283a456050d839f1@mail.gmail.com> References: <17b2302a1002182058k72e04d04h8fa85c2b409288dc@mail.gmail.com> <7926817e1002190944p69511f6dhc92eb1ba1113dd55@mail.gmail.com> <17b2302a1002191118n37713a1av283a456050d839f1@mail.gmail.com> Message-ID: <15e8b9d21002191122v5f99435cxe113fed90471be0@mail.gmail.com> On Fri, Feb 19, 2010 at 11:18 AM, Joshua Bloch wrote: > I don't think so. ?The BGGA proposal also uses the lambda conversion to > produce SAM-type (interface only) implementations. Not so. What BGGA calls the lambda conversion is completely unrelated to what project lambda calls the lambda conversion. I don't know why project lambda uses that name, by the way, since it converts a function type to a SAM type; lambdas have nothing to do with it. >?I believe the lambda > conversion is just getting in our way. ?I really do think we should explore > a design where we have two distinct syntaxes: one for "structural function > objects" (lambdas in the current proposal) and one for "nominal function > objects" (SAM-type instances). BGGA demonstrates that you can do both with one syntax, and without the performance hit of the current draft's lambda conversion. From jjb at google.com Fri Feb 19 11:52:17 2010 From: jjb at google.com (Joshua Bloch) Date: Fri, 19 Feb 2010 11:52:17 -0800 Subject: Lambda Conversion Considered Harmful? In-Reply-To: <15e8b9d21002191122v5f99435cxe113fed90471be0@mail.gmail.com> References: <17b2302a1002182058k72e04d04h8fa85c2b409288dc@mail.gmail.com> <7926817e1002190944p69511f6dhc92eb1ba1113dd55@mail.gmail.com> <17b2302a1002191118n37713a1av283a456050d839f1@mail.gmail.com> <15e8b9d21002191122v5f99435cxe113fed90471be0@mail.gmail.com> Message-ID: <17b2302a1002191152t6b1eca3dg755312c422681492@mail.gmail.com> Neal, On Fri, Feb 19, 2010 at 11:22 AM, Neal Gafter wrote: > On Fri, Feb 19, 2010 at 11:18 AM, Joshua Bloch wrote: > > I don't think so. The BGGA proposal also uses the lambda conversion to > > produce SAM-type (interface only) implementations. > > Not so. What BGGA calls the lambda conversion is completely unrelated > to what project lambda calls the lambda conversion. I don't know why > project lambda uses that name, by the way, since it converts a > function type to a SAM type; lambdas have nothing to do with it. > > > I believe the lambda > > conversion is just getting in our way. I really do think we should > explore > > a design where we have two distinct syntaxes: one for "structural > function > > objects" (lambdas in the current proposal) and one for "nominal function > > objects" (SAM-type instances). > > BGGA demonstrates that you can do both with one syntax, and without > the performance hit of the current draft's lambda conversion. > As per my previous e-mail, I'm worried about the semantics of lambda conversions (both project lambda's and BGGA's) as well as the performance. I consider the lack of an accurate "this" to be a serious issue, as well as the inability to work on class types and access class members. Also it's confusing to have a widening reference conversion from one interface type to another when the latter ("wider") interface doesn't include a method present on the former (invoke). Josh From neal at gafter.com Fri Feb 19 12:12:06 2010 From: neal at gafter.com (Neal Gafter) Date: Fri, 19 Feb 2010 12:12:06 -0800 Subject: Lambda Conversion Considered Harmful? In-Reply-To: <17b2302a1002191152t6b1eca3dg755312c422681492@mail.gmail.com> References: <17b2302a1002182058k72e04d04h8fa85c2b409288dc@mail.gmail.com> <7926817e1002190944p69511f6dhc92eb1ba1113dd55@mail.gmail.com> <17b2302a1002191118n37713a1av283a456050d839f1@mail.gmail.com> <15e8b9d21002191122v5f99435cxe113fed90471be0@mail.gmail.com> <17b2302a1002191152t6b1eca3dg755312c422681492@mail.gmail.com> Message-ID: <15e8b9d21002191212o3ebb1e9bre36f07ea61d9dcac@mail.gmail.com> On Fri, Feb 19, 2010 at 11:52 AM, Joshua Bloch wrote: > ... it's confusing to have a widening reference conversion from one interface type to > another when the latter ("wider") interface doesn't include a method present > on the former (invoke). BGGA's lambda conversion does not convert from an interface type. From neal at gafter.com Fri Feb 19 13:48:22 2010 From: neal at gafter.com (Neal Gafter) Date: Fri, 19 Feb 2010 13:48:22 -0800 Subject: Lambda Conversion Considered Harmful? In-Reply-To: <15e8b9d21002191212o3ebb1e9bre36f07ea61d9dcac@mail.gmail.com> References: <17b2302a1002182058k72e04d04h8fa85c2b409288dc@mail.gmail.com> <7926817e1002190944p69511f6dhc92eb1ba1113dd55@mail.gmail.com> <17b2302a1002191118n37713a1av283a456050d839f1@mail.gmail.com> <15e8b9d21002191122v5f99435cxe113fed90471be0@mail.gmail.com> <17b2302a1002191152t6b1eca3dg755312c422681492@mail.gmail.com> <15e8b9d21002191212o3ebb1e9bre36f07ea61d9dcac@mail.gmail.com> Message-ID: <15e8b9d21002191348r2969150y66c2f7c87ab2c708@mail.gmail.com> Well, we can add support for function types and lambda expressions, ala BGGA (with interoperability for SAM interfaces). Or we can add a more concise anonymous inner class syntax, ala CICE. Or we can do both, as you propose, which is more complex than either alternative. The current draft tries (and fails) to strike a technical compromise between the two, and in so doing undermines the value of both alternatives. (We can rule out "neither" in our exploration of the design space, since we already have a precise specification and implementation of that option (i.e. Java SE 6) to compare against whatever project lambda comes up with.) From markmahieu at googlemail.com Fri Feb 19 13:57:03 2010 From: markmahieu at googlemail.com (Mark Mahieu) Date: Fri, 19 Feb 2010 21:57:03 +0000 Subject: Lambda Conversion Considered Harmful? In-Reply-To: <17b2302a1002191152t6b1eca3dg755312c422681492@mail.gmail.com> References: <17b2302a1002182058k72e04d04h8fa85c2b409288dc@mail.gmail.com> <7926817e1002190944p69511f6dhc92eb1ba1113dd55@mail.gmail.com> <17b2302a1002191118n37713a1av283a456050d839f1@mail.gmail.com> <15e8b9d21002191122v5f99435cxe113fed90471be0@mail.gmail.com> <17b2302a1002191152t6b1eca3dg755312c422681492@mail.gmail.com> Message-ID: <2F4A6E13-4D3F-4E08-A6F2-DE031C673C64@googlemail.com> Hi Josh, On 19 Feb 2010, at 19:52, Joshua Bloch wrote: > > Also it's > confusing to have a widening reference conversion from one interface type to > another when the latter ("wider") interface doesn't include a method present > on the former (invoke). I'm confused by what you're saying here. I'd have thought it should be considered perfectly normal for the wider type not to include a method on the type being converted from. Mind you, I've never been tempted to think of lambda conversions as widening conversions, personally. > Josh Mark From Alex.Buckley at Sun.COM Fri Feb 19 14:09:35 2010 From: Alex.Buckley at Sun.COM (Alex Buckley) Date: Fri, 19 Feb 2010 14:09:35 -0800 Subject: Lambda Conversion Considered Harmful? In-Reply-To: <15e8b9d21002191122v5f99435cxe113fed90471be0@mail.gmail.com> References: <17b2302a1002182058k72e04d04h8fa85c2b409288dc@mail.gmail.com> <7926817e1002190944p69511f6dhc92eb1ba1113dd55@mail.gmail.com> <17b2302a1002191118n37713a1av283a456050d839f1@mail.gmail.com> <15e8b9d21002191122v5f99435cxe113fed90471be0@mail.gmail.com> Message-ID: <4B7F0C1F.5050906@sun.com> Neal Gafter wrote: > On Fri, Feb 19, 2010 at 11:18 AM, Joshua Bloch wrote: >> I don't think so. The BGGA proposal also uses the lambda conversion to >> produce SAM-type (interface only) implementations. > > Not so. What BGGA calls the lambda conversion is completely unrelated > to what project lambda calls the lambda conversion. I don't know why > project lambda uses that name, by the way, since it converts a > function type to a SAM type; lambdas have nothing to do with it. A conversion in the Java language is a mapping from one type to another type. BGGA/CfJ describes a mapping from an expression to a type, so I don't know why you call it a conversion. >> I believe the lambda >> conversion is just getting in our way. I really do think we should explore >> a design where we have two distinct syntaxes: one for "structural function >> objects" (lambdas in the current proposal) and one for "nominal function >> objects" (SAM-type instances). > > BGGA demonstrates that you can do both with one syntax, and without > the performance hit of the current draft's lambda conversion. It is irregular to support "conversion" from only a literal lambda expression to a SAM type, rather than from any value of function typre to a SAM type. Alex From Alex.Buckley at Sun.COM Fri Feb 19 14:09:54 2010 From: Alex.Buckley at Sun.COM (Alex Buckley) Date: Fri, 19 Feb 2010 14:09:54 -0800 Subject: Lambda Conversion Considered Harmful? In-Reply-To: <15e8b9d21002191212o3ebb1e9bre36f07ea61d9dcac@mail.gmail.com> References: <17b2302a1002182058k72e04d04h8fa85c2b409288dc@mail.gmail.com> <7926817e1002190944p69511f6dhc92eb1ba1113dd55@mail.gmail.com> <17b2302a1002191118n37713a1av283a456050d839f1@mail.gmail.com> <15e8b9d21002191122v5f99435cxe113fed90471be0@mail.gmail.com> <17b2302a1002191152t6b1eca3dg755312c422681492@mail.gmail.com> <15e8b9d21002191212o3ebb1e9bre36f07ea61d9dcac@mail.gmail.com> Message-ID: <4B7F0C32.2000401@sun.com> Neal Gafter wrote: > On Fri, Feb 19, 2010 at 11:52 AM, Joshua Bloch wrote: >> ... it's confusing to have a widening reference conversion from one interface type to >> another when the latter ("wider") interface doesn't include a method present >> on the former (invoke). > > BGGA's lambda conversion does not convert from an interface type. This is a bug, not a feature. Alex From Alex.Buckley at Sun.COM Fri Feb 19 14:13:41 2010 From: Alex.Buckley at Sun.COM (Alex Buckley) Date: Fri, 19 Feb 2010 14:13:41 -0800 Subject: Lambda Conversion Considered Harmful? In-Reply-To: <15e8b9d21002191348r2969150y66c2f7c87ab2c708@mail.gmail.com> References: <17b2302a1002182058k72e04d04h8fa85c2b409288dc@mail.gmail.com> <7926817e1002190944p69511f6dhc92eb1ba1113dd55@mail.gmail.com> <17b2302a1002191118n37713a1av283a456050d839f1@mail.gmail.com> <15e8b9d21002191122v5f99435cxe113fed90471be0@mail.gmail.com> <17b2302a1002191152t6b1eca3dg755312c422681492@mail.gmail.com> <15e8b9d21002191212o3ebb1e9bre36f07ea61d9dcac@mail.gmail.com> <15e8b9d21002191348r2969150y66c2f7c87ab2c708@mail.gmail.com> Message-ID: <4B7F0D15.1080900@sun.com> Neal Gafter wrote: > Well, we can add support for function types and lambda expressions, ala BGGA > (with interoperability for SAM interfaces). > > Or we can add a more concise anonymous inner class syntax, ala CICE. > > Or we can do both, as you propose, which is more complex than either > alternative. > > The current draft tries (and fails) to strike a technical compromise between > the two, and in so doing undermines the value of both alternatives. The current draft tries to do both. The performance implications may yet be undesirable, in which case we may restrict the "conversion" to be from lambda expressions only. Alex From peter.levart at gmail.com Fri Feb 19 14:40:52 2010 From: peter.levart at gmail.com (Peter Levart) Date: Fri, 19 Feb 2010 23:40:52 +0100 Subject: Lambda Conversion Considered Harmful? In-Reply-To: <17b2302a1002182058k72e04d04h8fa85c2b409288dc@mail.gmail.com> References: <17b2302a1002182058k72e04d04h8fa85c2b409288dc@mail.gmail.com> Message-ID: <201002192340.52895.peter.levart@gmail.com> From the 0.1.5 spec: On Saturday 13 February 2010 02:21:38 Alex Buckley wrote: > [5.2] Assignment Conversion > > Assignment contexts allow the use of one of the following: > - a lambda conversion (5.1.14). > > [5.3] Method Invocation Conversion > > Method invocation contexts allow the use of one of the following: > - a lambda conversion (5.1.14). > > /* > Implementing lambda conversion will likely generate new objects to > wrap or delegate to the lambda expression. This is acceptable in > assignment and method invocation conversion contexts, but not in a > casting conversion context. (See > http://mail.openjdk.java.net/pipermail/lambda-dev/2010-January/000449.html) > */ > I don't think we can allow assignment contexts or method invocation contexts to perform conversion if the conversion is to SAM classes (not interfaces) which wrap function objects. The abused TimerTask example from above URL could be rewritten to exhibit those two conversion contexts to produce misleading source code. This is the original example: Timer timer = new Timer(); #void() task = #() { ....; ((TimerTask)this).cancel(); } timer.schedule((TimerTask)task, 1000L, 1000L); Now, if only casting conversion wasn't allowed, one could write this: Timer timer = new Timer(); #void() task = #() { ....; TimerTask tt = this; tt.cancel(); } timer.schedule(task, 1000L, 1000L); So if we disallow assignment conversion too, one could write this: static void doCancel(TimerTask tt) { tt.cancel(); } ... Timer timer = new Timer(); #void() task = #() { ....; doCancel(this); } timer.schedule(task, 1000L, 1000L); All above examples are misleading if each conversion to SAM class instantiates new instance. I think we have to disallow conversion to SAM classes (if implemented by wrapping function objects). Regards, Peter From reinier at zwitserloot.com Fri Feb 19 15:04:31 2010 From: reinier at zwitserloot.com (Reinier Zwitserloot) Date: Sat, 20 Feb 2010 00:04:31 +0100 Subject: Are function types a requirement? In-Reply-To: <15e8b9d21002181819w2671803fwe72131ec98abedb6@mail.gmail.com> References: <4B7C92A6.9020008@joda.org> <560fb5ed1002172006n40851460i3548077b3a74c1e6@mail.gmail.com> <15e8b9d21002172016g3482dcaao3e94f47b6666efe3@mail.gmail.com> <560fb5ed1002172201s1c1eae5am4dd7d8f8627e7eae@mail.gmail.com> <4B7DA002.1080501@sun.com> <560fb5ed1002181556p61e3dabr472c0842dba8572@mail.gmail.com> <15e8b9d21002181819w2671803fwe72131ec98abedb6@mail.gmail.com> Message-ID: <560fb5ed1002191504o59b0c369rdf45d984d69b77e3@mail.gmail.com> I believe I mentioned that chaining varargs into another varargs is actually allowed (and, as the same rules apply to the chained method, does not weaken type safety, unless you mix compilation units compiled by different versions of java, an act which has always had ample opportunity for uncaught type errors). If not, that's an omission in putting it down in this thread, it's always been part of this set of rules. --Reinier Zwitserloot On Fri, Feb 19, 2010 at 3:19 AM, Neal Gafter wrote: > On Thu, Feb 18, 2010 at 3:56 PM, Reinier Zwitserloot > wrote: > > 2. While my rules may seem arbitrary when you don't know why they are > being > > applied, the enjoy the advantage that 99.99999% of all actual usage of > > varargs out there already adheres to them; in my experience the only > thing > > anyone ever does with varargs params is check them against null > > occasionally, and other than that loop through them either via .length / > > array acces, or foreach. > > In my experience, it is not uncommon to have one varargs method > "chain" to another varargs method by passing the array along. I > suspect your "99.99999%" statistic is inaccurate. > From reinier at zwitserloot.com Fri Feb 19 15:09:59 2010 From: reinier at zwitserloot.com (Reinier Zwitserloot) Date: Sat, 20 Feb 2010 00:09:59 +0100 Subject: Lambda Conversion Considered Harmful? In-Reply-To: <17b2302a1002182058k72e04d04h8fa85c2b409288dc@mail.gmail.com> References: <17b2302a1002182058k72e04d04h8fa85c2b409288dc@mail.gmail.com> Message-ID: <560fb5ed1002191509h3318dc54rcfd669ce167cbd5@mail.gmail.com> If the #Comparator(String a, String b)... syntax survives, may I suggest putting the type name _before_ the hash? This opens the door to implementing non-SAMs with the closure syntax. For example: WindowAdapter#windowClosing(WindowEvent e) {} where the method name (the identifier between hash and open paren) is optional only if the Type name (to the left of the hash) resolves to a SAM type. The reason for setting up the syntax this way is mostly the same reason SAM conversion is being looked at in the first place - to be useful to existing libraries, many of which won't change their API or do a complete core redesign of their library just to accomodate closures. For example, I'm not so sure that the swing library is going to be updated with an all-new event handling system so that you can e.g. implement mouseOver with a closure. --Reinier Zwitserloot On Fri, Feb 19, 2010 at 5:58 AM, Joshua Bloch wrote: > As I'm sure you're aware, I'm very much in favor of the goal of the Lambda > conversion: the new syntax (or something like it) should be usable in > combination with existing constructs that require a "function object" such > as a Comparator or (perhaps) a TimerTask. But I'm increasingly skeptical > that the Lambda conversion (as described in the proposal) is the best way > to > achieve this. > > I'm skeptical for two reasons. The first concerns the semantics. Simply > put, > function types just get in the way when what you want is a SAM type. They > raise all sorts of thorny issues, many of which we've been discussing > recently. Is the function-typed object the same object as the SAM-typed > object, or are there two objects involved? Is there any way to access the > members of the SAM type? And so on. > > The second reason I'm skeptical concerns performance. The latest spec > draft > says this: > > Implementing lambda conversion will likely generate new objects to wrap or > > delegate to the lambda expression. This is acceptable in assignment and > > method invocation conversion contexts, but not in a casting conversion > > context. (See > > > http://mail.openjdk.java.net/pipermail/lambda-dev/2010-January/000449.html > > ) > > > I fear that this will have disastrous performance consequences. Presented > with a lovely new syntax, programmers will naturally use it, e.g., to make > Comparators: > > Arrays.sort(a, #(String s1, String s2)(return s1.length - s2.length)); > > But the comparator's sole method is invoked in the inner loop of the sort! > The slowdown caused by the delegation will add to the constant factor of > the > (n log n) sort. This could be very costly! I believe it violates a > fundamental tenet of Java's design, which has been called *transparency*: > magic should be kept to a minimum, and the code should reflect its > performance (to the extent that that's feasible in this day and age). The > for-each loop, by contrast, does not create an Iterator when traversing an > array, so it runs as fast as the traditional for loop that it replaces. > This > was very important to me when we were designing the for-each construct, and > I'm glad we did it the way we did. > > Therefore, I think we should *very *seriously consider an alternative > design > that uses a slightly different syntax when furnishing instances of SAM > types > that it does when furnishing instances of function types. Call them > SAM-lambdas and function-lambads (for the time being). Perhaps the SAM > lambdas could take the type as a mandatory argument, to simplify the > compiler's job: > > Arrays.sort(a, #Comparator(String s1, String s2)(return s1.length - > s2.length)); > > I have no idea whether this is the best syntax, or even an acceptable > syntax. The important point is that we need something that: > > (1) Interoperates with existing methods that require "function objects" > (2) Provides performance equal to current idioms, and > (3) Provide conciseness close to that of lambdas (as specified in the > current proposal) > > As a clarification, I am not arguing for the elimination of function types; > rather I'm arguing that they get in the way of "interoperable function > objects." so we need two separate syntaxes, one for function types and one > for SAM types. > > Josh > > From reinier at zwitserloot.com Fri Feb 19 15:13:48 2010 From: reinier at zwitserloot.com (Reinier Zwitserloot) Date: Sat, 20 Feb 2010 00:13:48 +0100 Subject: Lightweight interfaces instead of function types In-Reply-To: <4b4f45e01002191015w4341c46dia6a548fe4bd5730f@mail.gmail.com> References: <4b4f45e01002190928x5230dfeek2c1490dedccc71fd@mail.gmail.com> <15e8b9d21002190947p4af23210o1e2e5d280144da60@mail.gmail.com> <4b4f45e01002191015w4341c46dia6a548fe4bd5730f@mail.gmail.com> Message-ID: <560fb5ed1002191513kdd2d44ar65ae9f2a9b25170d@mail.gmail.com> On Fri, Feb 19, 2010 at 7:15 PM, Stephen Colebourne wrote: > > Again, its a trade-off. The benefit for ParallelArray is that > Predicate or Transformer is IMO more meaningful to someone just > casually coming to the API than a random function type. > > +1 I'm still not entirely clear on the obsession of getting rid of _every_ type listed in ParallelArrays' Ops.java. DoubleMaxReducer as well as IntMaxReducer, that does feel rather awkward, but having a concept called a MaxReducer in the first place, that seems to mostly be a force for good. From neal at gafter.com Fri Feb 19 15:21:14 2010 From: neal at gafter.com (Neal Gafter) Date: Fri, 19 Feb 2010 15:21:14 -0800 Subject: Lambda Conversion Considered Harmful? In-Reply-To: <4B7F0C1F.5050906@sun.com> References: <17b2302a1002182058k72e04d04h8fa85c2b409288dc@mail.gmail.com> <7926817e1002190944p69511f6dhc92eb1ba1113dd55@mail.gmail.com> <17b2302a1002191118n37713a1av283a456050d839f1@mail.gmail.com> <15e8b9d21002191122v5f99435cxe113fed90471be0@mail.gmail.com> <4B7F0C1F.5050906@sun.com> Message-ID: <15e8b9d21002191521s4f1e5462v9fb1c94ead99735c@mail.gmail.com> On Fri, Feb 19, 2010 at 2:09 PM, Alex Buckley wrote: > Neal Gafter wrote: > > On Fri, Feb 19, 2010 at 11:18 AM, Joshua Bloch wrote: > >> I don't think so. The BGGA proposal also uses the lambda conversion to > >> produce SAM-type (interface only) implementations. > > > > Not so. What BGGA calls the lambda conversion is completely unrelated > > to what project lambda calls the lambda conversion. I don't know why > > project lambda uses that name, by the way, since it converts a > > function type to a SAM type; lambdas have nothing to do with it. > > A conversion in the Java language is a mapping from one type to another > type. Not quite. A mapping associates a distinct element in the codomain with each value in the domain. A conversion, on the other hand, is a kind of binary relation between types. > BGGA/CfJ describes a mapping from an expression to a type, so I > don't know why you call it a conversion. > The type of a lambda expression in CfJ is not denotable in source, but that isn't the same as saying that it doesn't have a type. The type of a lambda expression in CfJ is defined by the set of declared argument types, result expressions, and thrown types of the lambda's body. The lambda conversion is a binary relation all of whose pairs are the type of a lambda expression on the left and a SAM interface on the right. There are other types in the Java language that are not denotable, for example intersection types (JLS3 4.9), infinite/recursive types (JLS3 15.12.2.7), and int constants whose "value .. is representable in the type of the variable" (JLS3 5.2). Those types are used in defining conversions too. > BGGA demonstrates that you can do both with one syntax, and without > > the performance hit of the current draft's lambda conversion. > > It is irregular to support "conversion" from only a literal lambda > expression to a SAM type, rather than from any value of function typre > to a SAM type. > CfJ (and BGGA) doesn't define any conversions from any values of function type to any other types. Those conversions are subtype relations already defined by the JLS. In CfJ, a function type *is* a SAM type; they are both handled precisely the same way by the type system and the conversions, uniformly. I agree that it would be irregular for you to take this approach as long as you continue to define lambda expressions to be of function type, and make that visible in the language through a "this". From neal at gafter.com Fri Feb 19 15:23:04 2010 From: neal at gafter.com (Neal Gafter) Date: Fri, 19 Feb 2010 15:23:04 -0800 Subject: Lambda Conversion Considered Harmful? In-Reply-To: <4B7F0C32.2000401@sun.com> References: <17b2302a1002182058k72e04d04h8fa85c2b409288dc@mail.gmail.com> <7926817e1002190944p69511f6dhc92eb1ba1113dd55@mail.gmail.com> <17b2302a1002191118n37713a1av283a456050d839f1@mail.gmail.com> <15e8b9d21002191122v5f99435cxe113fed90471be0@mail.gmail.com> <17b2302a1002191152t6b1eca3dg755312c422681492@mail.gmail.com> <15e8b9d21002191212o3ebb1e9bre36f07ea61d9dcac@mail.gmail.com> <4B7F0C32.2000401@sun.com> Message-ID: <15e8b9d21002191523m60dba7c9u6d3842f5da959ea6@mail.gmail.com> On Fri, Feb 19, 2010 at 2:09 PM, Alex Buckley wrote: > Neal Gafter wrote: > > BGGA's lambda conversion does not convert from an interface type. > > This is a bug, not a feature. > I can't make any sense of this response. From mthornton at optrak.co.uk Fri Feb 19 15:27:05 2010 From: mthornton at optrak.co.uk (Mark Thornton) Date: Fri, 19 Feb 2010 23:27:05 +0000 Subject: Lightweight interfaces instead of function types In-Reply-To: <560fb5ed1002191513kdd2d44ar65ae9f2a9b25170d@mail.gmail.com> References: <4b4f45e01002190928x5230dfeek2c1490dedccc71fd@mail.gmail.com> <15e8b9d21002190947p4af23210o1e2e5d280144da60@mail.gmail.com> <4b4f45e01002191015w4341c46dia6a548fe4bd5730f@mail.gmail.com> <560fb5ed1002191513kdd2d44ar65ae9f2a9b25170d@mail.gmail.com> Message-ID: <4B7F1E49.2000306@optrak.co.uk> Reinier Zwitserloot wrote: > On Fri, Feb 19, 2010 at 7:15 PM, Stephen Colebourne wrote: > > >> Again, its a trade-off. The benefit for ParallelArray is that >> Predicate or Transformer is IMO more meaningful to someone just >> casually coming to the API than a random function type. >> >> >> > +1 > > I'm still not entirely clear on the obsession of getting rid of _every_ type > listed in ParallelArrays' Ops.java. DoubleMaxReducer as well as > IntMaxReducer, that does feel rather awkward, but having a concept called a > MaxReducer in the first place, that seems to mostly be a force for good. > > It is the ones which enumerate their primitive type parameters that smell (and constitute most of the 81 as result of the combinatorial explosion of possibilities). Mark From neal at gafter.com Fri Feb 19 15:31:27 2010 From: neal at gafter.com (Neal Gafter) Date: Fri, 19 Feb 2010 15:31:27 -0800 Subject: Are function types a requirement? In-Reply-To: <560fb5ed1002191504o59b0c369rdf45d984d69b77e3@mail.gmail.com> References: <4B7C92A6.9020008@joda.org> <560fb5ed1002172006n40851460i3548077b3a74c1e6@mail.gmail.com> <15e8b9d21002172016g3482dcaao3e94f47b6666efe3@mail.gmail.com> <560fb5ed1002172201s1c1eae5am4dd7d8f8627e7eae@mail.gmail.com> <4B7DA002.1080501@sun.com> <560fb5ed1002181556p61e3dabr472c0842dba8572@mail.gmail.com> <15e8b9d21002181819w2671803fwe72131ec98abedb6@mail.gmail.com> <560fb5ed1002191504o59b0c369rdf45d984d69b77e3@mail.gmail.com> Message-ID: <15e8b9d21002191531k2791165co32fe00233bb2c3fe@mail.gmail.com> On Fri, Feb 19, 2010 at 3:04 PM, Reinier Zwitserloot < reinier at zwitserloot.com> wrote: > I believe I mentioned that chaining varargs into another varargs is > actually allowed (and, as the same rules apply to the chained method, does > not weaken type safety, unless you mix compilation units compiled by > different versions of java, an act which has always had ample opportunity > for uncaught type errors). If not, that's an omission in putting it down in > this thread, it's always been part of this set of rules. > I'm afraid it does weaken type safety. A varargs method can override, and be overridden by, a non-varargs method. From peter.levart at gmail.com Fri Feb 19 15:32:13 2010 From: peter.levart at gmail.com (Peter Levart) Date: Sat, 20 Feb 2010 00:32:13 +0100 Subject: Lambda Conversion Considered Harmful? In-Reply-To: <17b2302a1002191152t6b1eca3dg755312c422681492@mail.gmail.com> References: <17b2302a1002182058k72e04d04h8fa85c2b409288dc@mail.gmail.com> <15e8b9d21002191122v5f99435cxe113fed90471be0@mail.gmail.com> <17b2302a1002191152t6b1eca3dg755312c422681492@mail.gmail.com> Message-ID: <201002200032.13144.peter.levart@gmail.com> On Friday 19 February 2010 20:52:17 Joshua Bloch wrote: > As per my previous e-mail, I'm worried about the semantics of lambda > conversions (both project lambda's and BGGA's) as well as the performance. > I consider the lack of an accurate "this" to be a serious issue, as well > as the inability to work on class types and access class members. Also > it's confusing to have a widening reference conversion from one interface > type to another when the latter ("wider") interface doesn't include a > method present on the former (invoke). > > Josh > There is a way to support "conversion" of lambda expression to SAM class (and/or interface) and at the same time have "this" refer to lambda instance of function (not SAM) type. This precludes interfaces as the implementation mechanism of function types - the same object can both extend SAM class and implement function type. If you want to access members of SAM type from within lambda body, you simply cast "this" to SAM type. There might be another "hacky" variant which uses special objects as function typed objects (MethodHandle) and goes like this: - lambda conversion is performed at compile time (like BGGA) and includes possible conversion to SAM classes. Each such generated SAM sub-class contains a reference to a function object (MethoHandle) constructed at SAM instance construction time and which delegates to SAM's method (not the other way around). "this" inside lambda refers to the function object. "this" can be cast only to SAM type (or supertype) to which lambda expression was converted and such cast returns the SAM instance (not the function object). So we only have 2 instances at any time. I call it "hacky" because the following expression inside lambda body would be false: this == (SamType)this; But the following would be true: (SamType)this == (SamType)this; Regards, Peter From Alex.Buckley at Sun.COM Fri Feb 19 15:34:17 2010 From: Alex.Buckley at Sun.COM (Alex Buckley) Date: Fri, 19 Feb 2010 15:34:17 -0800 Subject: Lambda Conversion Considered Harmful? In-Reply-To: <15e8b9d21002191521s4f1e5462v9fb1c94ead99735c@mail.gmail.com> References: <17b2302a1002182058k72e04d04h8fa85c2b409288dc@mail.gmail.com> <7926817e1002190944p69511f6dhc92eb1ba1113dd55@mail.gmail.com> <17b2302a1002191118n37713a1av283a456050d839f1@mail.gmail.com> <15e8b9d21002191122v5f99435cxe113fed90471be0@mail.gmail.com> <4B7F0C1F.5050906@sun.com> <15e8b9d21002191521s4f1e5462v9fb1c94ead99735c@mail.gmail.com> Message-ID: <4B7F1FF9.3030608@sun.com> Neal Gafter wrote: > The type of a lambda expression in CfJ is not denotable in source, but > that isn't the same as saying that it doesn't have a type. The type of > a lambda expression in CfJ is defined by the set of declared argument > types, result expressions, and thrown types of the lambda's body. The > lambda conversion is a binary relation all of whose pairs are the type > of a lambda expression on the left and a SAM interface on the right. While CfJ already avoids saying that the type of a lambda expression is a function type, it would be nice if the paragraph above was incorporated into the CfJ specification. > > BGGA demonstrates that you can do both with one syntax, and without > > the performance hit of the current draft's lambda conversion. > > It is irregular to support "conversion" from only a literal lambda > expression to a SAM type, rather than from any value of function typre > to a SAM type. > > CfJ (and BGGA) doesn't define any conversions from any values of > function type to any other types. Those conversions are subtype > relations already defined by the JLS. In CfJ, a function type *is* a > SAM type; they are both handled precisely the same way by the type > system and the conversions, uniformly. OK. And just so everyone is clear: the Lambda spec does not mandate that a function type is a SAM type. If you were implementing BGGA today, would you look at using MethodHandles? > I agree that it would be irregular for you to take this approach as > long as you continue to define lambda expressions to be of function > type, and make that visible in the language through a "this". Yes. Alex From jjb at google.com Fri Feb 19 15:36:36 2010 From: jjb at google.com (Joshua Bloch) Date: Fri, 19 Feb 2010 15:36:36 -0800 Subject: Lambda Conversion Considered Harmful? In-Reply-To: <4B7F0C1F.5050906@sun.com> References: <17b2302a1002182058k72e04d04h8fa85c2b409288dc@mail.gmail.com> <7926817e1002190944p69511f6dhc92eb1ba1113dd55@mail.gmail.com> <17b2302a1002191118n37713a1av283a456050d839f1@mail.gmail.com> <15e8b9d21002191122v5f99435cxe113fed90471be0@mail.gmail.com> <4B7F0C1F.5050906@sun.com> Message-ID: <17b2302a1002191536t705318dem88ab86a00861942a@mail.gmail.com> On Fri, Feb 19, 2010 at 2:09 PM, Alex Buckley wrote: > Neal Gafter wrote: > > On Fri, Feb 19, 2010 at 11:18 AM, Joshua Bloch wrote: > >> I don't think so. The BGGA proposal also uses the lambda conversion to > >> produce SAM-type (interface only) implementations. > > > > Not so. What BGGA calls the lambda conversion is completely unrelated > > to what project lambda calls the lambda conversion. I don't know why > > project lambda uses that name, by the way, since it converts a > > function type to a SAM type; lambdas have nothing to do with it. > > A conversion in the Java language is a mapping from one type to another > type. BGGA/CfJ describes a mapping from an expression to a type, so I > don't know why you call it a conversion. Just to echo Alex's concerns, I was fooled by this, hence my statement that it was converting from one interface to another. JLS 3 (Section 5.1.5)says " A *widening reference* conversion exists from any type *S* to any type *T*, provided *S* is a subtype (?4.10) of *T*." The lambda conversion described in CfJ does not meet this specification, so it's wrong to call it a widening reference conversion. Josh From neal at gafter.com Fri Feb 19 15:49:56 2010 From: neal at gafter.com (Neal Gafter) Date: Fri, 19 Feb 2010 15:49:56 -0800 Subject: Lambda Conversion Considered Harmful? In-Reply-To: <4B7F1FF9.3030608@sun.com> References: <17b2302a1002182058k72e04d04h8fa85c2b409288dc@mail.gmail.com> <7926817e1002190944p69511f6dhc92eb1ba1113dd55@mail.gmail.com> <17b2302a1002191118n37713a1av283a456050d839f1@mail.gmail.com> <15e8b9d21002191122v5f99435cxe113fed90471be0@mail.gmail.com> <4B7F0C1F.5050906@sun.com> <15e8b9d21002191521s4f1e5462v9fb1c94ead99735c@mail.gmail.com> <4B7F1FF9.3030608@sun.com> Message-ID: <15e8b9d21002191549k5a580c80q63561631564a65@mail.gmail.com> On Fri, Feb 19, 2010 at 3:34 PM, Alex Buckley wrote: > Neal Gafter wrote: > >> The type of a lambda expression in CfJ is not denotable in source, but >> that isn't the same as saying that it doesn't have a type. The type of a >> lambda expression in CfJ is defined by the set of declared argument types, >> result expressions, and thrown types of the lambda's body. The lambda >> conversion is a binary relation all of whose pairs are the type of a lambda >> expression on the left and a SAM interface on the right. >> > > While CfJ already avoids saying that the type of a lambda expression is a > function type, it would be nice if the paragraph above was incorporated into > the CfJ specification. Done. > OK. And just so everyone is clear: the Lambda spec does not mandate that a > function type is a SAM type. If you were implementing BGGA today, would you > look at using MethodHandles? Absolutely. And then the lambda conversion would be defined from lambda expressions to SAM interfaces *and* from lambda expressions to function types. But likely not from function types to SAM interfaces (though the programmer could use a method reference to do that, or a separate conversion could be defined). From peter.levart at gmail.com Fri Feb 19 15:53:18 2010 From: peter.levart at gmail.com (Peter Levart) Date: Sat, 20 Feb 2010 00:53:18 +0100 Subject: Lambda Conversion Considered Harmful? In-Reply-To: <201002200032.13144.peter.levart@gmail.com> References: <17b2302a1002182058k72e04d04h8fa85c2b409288dc@mail.gmail.com> <17b2302a1002191152t6b1eca3dg755312c422681492@mail.gmail.com> <201002200032.13144.peter.levart@gmail.com> Message-ID: <201002200053.18817.peter.levart@gmail.com> On Saturday 20 February 2010 00:32:13 Peter Levart wrote: > There is a way to support "conversion" of lambda expression to SAM class > (and/or interface) and at the same time have "this" refer to lambda > instance of function (not SAM) type. This precludes interfaces as the Oh, I have to go to sleep! change "precludes" -> "mandates" > implementation mechanism of function types - the same object can both > extend SAM class and implement function type. If you want to access > members of SAM type from within lambda body, you simply cast "this" to SAM > type. > From neal at gafter.com Fri Feb 19 15:56:26 2010 From: neal at gafter.com (Neal Gafter) Date: Fri, 19 Feb 2010 15:56:26 -0800 Subject: Lambda Conversion Considered Harmful? In-Reply-To: <17b2302a1002191536t705318dem88ab86a00861942a@mail.gmail.com> References: <17b2302a1002182058k72e04d04h8fa85c2b409288dc@mail.gmail.com> <7926817e1002190944p69511f6dhc92eb1ba1113dd55@mail.gmail.com> <17b2302a1002191118n37713a1av283a456050d839f1@mail.gmail.com> <15e8b9d21002191122v5f99435cxe113fed90471be0@mail.gmail.com> <4B7F0C1F.5050906@sun.com> <17b2302a1002191536t705318dem88ab86a00861942a@mail.gmail.com> Message-ID: <15e8b9d21002191556j67e3687fy552564f4ad52744@mail.gmail.com> On Fri, Feb 19, 2010 at 3:36 PM, Joshua Bloch wrote: > Just to echo Alex's concerns, I was fooled by this, hence my statement that > it was converting from one interface to another. JLS 3 (Section > 5.1.5)< > http://java.sun.com/docs/books/jls/third_edition/html/conversions.html > >says > " > A *widening reference* conversion exists from any type *S* to any type *T*, > provided *S* is a subtype > (?4.10)< > http://java.sun.com/docs/books/jls/third_edition/html/typesValues.html#120403 > > > of *T*." The lambda conversion described in CfJ does not meet this > specification, so it's wrong to call it a widening reference conversion. > This is clearly a bug in the JLS, otherwise the widening conversion from byte to int would be a widening reference conversion. CfJ's lambda conversion is defined as a widening conversion, but it isn't a reference conversion. From neal at gafter.com Fri Feb 19 15:58:14 2010 From: neal at gafter.com (Neal Gafter) Date: Fri, 19 Feb 2010 15:58:14 -0800 Subject: Lambda Conversion Considered Harmful? In-Reply-To: <15e8b9d21002191556j67e3687fy552564f4ad52744@mail.gmail.com> References: <17b2302a1002182058k72e04d04h8fa85c2b409288dc@mail.gmail.com> <7926817e1002190944p69511f6dhc92eb1ba1113dd55@mail.gmail.com> <17b2302a1002191118n37713a1av283a456050d839f1@mail.gmail.com> <15e8b9d21002191122v5f99435cxe113fed90471be0@mail.gmail.com> <4B7F0C1F.5050906@sun.com> <17b2302a1002191536t705318dem88ab86a00861942a@mail.gmail.com> <15e8b9d21002191556j67e3687fy552564f4ad52744@mail.gmail.com> Message-ID: <15e8b9d21002191558o588392d6g109cf892683db7df@mail.gmail.com> On Fri, Feb 19, 2010 at 3:56 PM, Neal Gafter wrote: > On Fri, Feb 19, 2010 at 3:36 PM, Joshua Bloch wrote: > >> Just to echo Alex's concerns, I was fooled by this, hence my statement >> that >> it was converting from one interface to another. JLS 3 (Section >> 5.1.5)< >> http://java.sun.com/docs/books/jls/third_edition/html/conversions.html >> >says >> " >> A *widening reference* conversion exists from any type *S* to any type >> *T*, >> provided *S* is a subtype >> (?4.10)< >> http://java.sun.com/docs/books/jls/third_edition/html/typesValues.html#120403 >> > >> of *T*." The lambda conversion described in CfJ does not meet this >> specification, so it's wrong to call it a widening reference conversion. >> > > This is clearly a bug in the JLS, otherwise the widening conversion from > byte to int would be a widening reference conversion. > > CfJ's lambda conversion is defined as a widening conversion, but it isn't a > reference conversion. > Just to clarify: yes, this implies that CfJ's lambda expression is a subtype of the SAM to which it is converted. From jjb at google.com Fri Feb 19 16:13:03 2010 From: jjb at google.com (Joshua Bloch) Date: Fri, 19 Feb 2010 16:13:03 -0800 Subject: Lambda Conversion Considered Harmful? In-Reply-To: <15e8b9d21002191556j67e3687fy552564f4ad52744@mail.gmail.com> References: <17b2302a1002182058k72e04d04h8fa85c2b409288dc@mail.gmail.com> <7926817e1002190944p69511f6dhc92eb1ba1113dd55@mail.gmail.com> <17b2302a1002191118n37713a1av283a456050d839f1@mail.gmail.com> <15e8b9d21002191122v5f99435cxe113fed90471be0@mail.gmail.com> <4B7F0C1F.5050906@sun.com> <17b2302a1002191536t705318dem88ab86a00861942a@mail.gmail.com> <15e8b9d21002191556j67e3687fy552564f4ad52744@mail.gmail.com> Message-ID: <17b2302a1002191613l40c36803i4bc9bbb4df9c0305@mail.gmail.com> Neal, On Fri, Feb 19, 2010 at 3:56 PM, Neal Gafter wrote: > On Fri, Feb 19, 2010 at 3:36 PM, Joshua Bloch wrote: > >> Just to echo Alex's concerns, I was fooled by this, hence my statement >> that >> it was converting from one interface to another. JLS 3 (Section >> 5.1.5)< >> http://java.sun.com/docs/books/jls/third_edition/html/conversions.html >> >says >> >> " >> A *widening reference* conversion exists from any type *S* to any type >> *T*, >> provided *S* is a subtype >> (?4.10)< >> http://java.sun.com/docs/books/jls/third_edition/html/typesValues.html#120403 >> > >> >> of *T*." The lambda conversion described in CfJ does not meet this >> specification, so it's wrong to call it a widening reference conversion. >> > > This is clearly a bug in the JLS, otherwise the widening conversion from > byte to int would be a widening reference conversion. > > I'd argue that the bug is that byte is defined to be a subtype of int. This wasn't true in earlier editions of the JLS, as I recall;) Josh From scolebourne at joda.org Fri Feb 19 16:44:28 2010 From: scolebourne at joda.org (Stephen Colebourne) Date: Sat, 20 Feb 2010 00:44:28 +0000 Subject: Lightweight interfaces instead of function types In-Reply-To: <4B7F1E49.2000306@optrak.co.uk> References: <4b4f45e01002190928x5230dfeek2c1490dedccc71fd@mail.gmail.com> <15e8b9d21002190947p4af23210o1e2e5d280144da60@mail.gmail.com> <4b4f45e01002191015w4341c46dia6a548fe4bd5730f@mail.gmail.com> <560fb5ed1002191513kdd2d44ar65ae9f2a9b25170d@mail.gmail.com> <4B7F1E49.2000306@optrak.co.uk> Message-ID: <4b4f45e01002191644p4942c27p2cd04ae99dc19394@mail.gmail.com> On 19 February 2010 23:27, Mark Thornton wrote: > It is the ones which enumerate their primitive type parameters that smell > (and constitute most of the 81 as result of the combinatorial explosion of > possibilities). I did consider whether this might be handled more cleverly by allowing the lightweight interface to auto-generate multiple real interfaces (or one with more than one method). (ie. for Reducer to automatically handle the combinations of primitive types) I suspect getting this to work would be tricky... I'd also note that fork-join will probably need multiple versions of the implementation code anyway in order to make use of any primitives passed in, so the issue does seem slightly moot. One alternative option is to consider the lightweight interface to have a name scoped to the associated method. Fork-join has this: static Ops.LongPredicate andPredicate(Ops.LongPredicate first, Ops.LongPredicate second) {...} static Ops.DoublePredicate andPredicate(Ops.DoublePredicate first, Ops.DoublePredicate second) {...} whereas with method-scoped lightweight interface names you could write: interface Predicate boolean(long) static Predicate andPredicate(Predicate first, Predicate second) {...} interface Predicate boolean(double) static Predicate andPredicate(Predicate first, Predicate second) {...} With the compiler renaming the matching interfaces to make them unique. Of course this is close to function types (which just embed the signature). I raise it to show that there are ways to avoid writing the 81interfaces up front (by writing it on demand). As always, the route this project takes is a choice. (To be clear, I agree that the interfaces should describe what they do, not what the argument types are. However, this isn't a simple problem to solve - every option has a compromise) Stephen From john at milsson.nu Fri Feb 19 16:54:38 2010 From: john at milsson.nu (John Nilsson) Date: Sat, 20 Feb 2010 01:54:38 +0100 Subject: Performance implications of the lambda draft specification In-Reply-To: References: <15e8b9d21002182051j3099a8c4td54273e30ed7a516@mail.gmail.com> <7288749d1002190826w6086cd47he6467a1dbe010791@mail.gmail.com> <15e8b9d21002190918o29f5b540p1adce98d1f80e929@mail.gmail.com> Message-ID: On Fri, Feb 19, 2010 at 7:56 PM, Osvaldo Doederlein wrote: > Having said that I'm enthusiastic about the upcoming EA-based scalar > replacement capability of JDK7, I've been testing it for a while. But much > less enthusiastic about designing a language feature that is very > important, > and would depend on that optimization. OTOH other JVM languages are already dependent on such optimizations. Doing the same for Java would only add one more to the family. If this leads to more resources being spent on such optimizations all JVM languages would benefit. BR, John From jjb at google.com Fri Feb 19 17:09:14 2010 From: jjb at google.com (Joshua Bloch) Date: Fri, 19 Feb 2010 17:09:14 -0800 Subject: Performance implications of the lambda draft specification In-Reply-To: References: <15e8b9d21002182051j3099a8c4td54273e30ed7a516@mail.gmail.com> <7288749d1002190826w6086cd47he6467a1dbe010791@mail.gmail.com> <15e8b9d21002190918o29f5b540p1adce98d1f80e929@mail.gmail.com> Message-ID: <17b2302a1002191709k3d949802wd2343537410526b7@mail.gmail.com> John, I've learned the hard way that this line of reasoning can lead to pain. When I designed the collections framework, I was assured that the cost of creating and using an iterator to traverse an ArrayList would vanish due to VM-wonderfulness. That was 13 years ago. I'm still waiting. Josh On Fri, Feb 19, 2010 at 4:54 PM, John Nilsson wrote: > On Fri, Feb 19, 2010 at 7:56 PM, Osvaldo Doederlein >wrote: > > > Having said that I'm enthusiastic about the upcoming EA-based scalar > > replacement capability of JDK7, I've been testing it for a while. But > much > > less enthusiastic about designing a language feature that is very > > important, > > and would depend on that optimization. > > > OTOH other JVM languages are already dependent on such optimizations. Doing > the same for Java would only add one more to the family. If this leads to > more resources being spent on such optimizations all JVM languages would > benefit. > > BR, > John > > From neal at gafter.com Fri Feb 19 18:15:53 2010 From: neal at gafter.com (Neal Gafter) Date: Fri, 19 Feb 2010 18:15:53 -0800 Subject: Lambda Conversion Considered Harmful? In-Reply-To: <17b2302a1002191613l40c36803i4bc9bbb4df9c0305@mail.gmail.com> References: <17b2302a1002182058k72e04d04h8fa85c2b409288dc@mail.gmail.com> <7926817e1002190944p69511f6dhc92eb1ba1113dd55@mail.gmail.com> <17b2302a1002191118n37713a1av283a456050d839f1@mail.gmail.com> <15e8b9d21002191122v5f99435cxe113fed90471be0@mail.gmail.com> <4B7F0C1F.5050906@sun.com> <17b2302a1002191536t705318dem88ab86a00861942a@mail.gmail.com> <15e8b9d21002191556j67e3687fy552564f4ad52744@mail.gmail.com> <17b2302a1002191613l40c36803i4bc9bbb4df9c0305@mail.gmail.com> Message-ID: <15e8b9d21002191815x63cd54eck69c04bbd6cbc7640@mail.gmail.com> On Fri, Feb 19, 2010 at 4:13 PM, Joshua Bloch wrote: > I'd argue that the bug is that byte is defined to be a subtype of int. ?This > wasn't true in earlier editions of the JLS, as I recall;) Earlier editions didn't define "subtype". From jjb at google.com Sat Feb 20 00:11:06 2010 From: jjb at google.com (Joshua Bloch) Date: Sat, 20 Feb 2010 00:11:06 -0800 Subject: Lambda Conversion Considered Harmful? In-Reply-To: <15e8b9d21002191815x63cd54eck69c04bbd6cbc7640@mail.gmail.com> References: <17b2302a1002182058k72e04d04h8fa85c2b409288dc@mail.gmail.com> <7926817e1002190944p69511f6dhc92eb1ba1113dd55@mail.gmail.com> <17b2302a1002191118n37713a1av283a456050d839f1@mail.gmail.com> <15e8b9d21002191122v5f99435cxe113fed90471be0@mail.gmail.com> <4B7F0C1F.5050906@sun.com> <17b2302a1002191536t705318dem88ab86a00861942a@mail.gmail.com> <15e8b9d21002191556j67e3687fy552564f4ad52744@mail.gmail.com> <17b2302a1002191613l40c36803i4bc9bbb4df9c0305@mail.gmail.com> <15e8b9d21002191815x63cd54eck69c04bbd6cbc7640@mail.gmail.com> Message-ID: <17b2302a1002200011x361a5704g83cc445a9aa9705f@mail.gmail.com> Neal, On Fri, Feb 19, 2010 at 6:15 PM, Neal Gafter wrote: > On Fri, Feb 19, 2010 at 4:13 PM, Joshua Bloch wrote: > > I'd argue that the bug is that byte is defined to be a subtype of int. > This > > wasn't true in earlier editions of the JLS, as I recall;) > > Earlier editions didn't define "subtype". > They may not have defined the term, but they did use it, for example: Assignment to an array component of reference type (?10.10, ?15.13, ?15.26.1). The type-checking rules allow the array type S[] to be treated as > a subtype of T[] if S is a subtype of T, but this requires a run-time check > for assignment to an array component, similar to the check performed for a cast. from page 320 of the Second Ed. Josh From reinier at zwitserloot.com Sat Feb 20 15:22:47 2010 From: reinier at zwitserloot.com (Reinier Zwitserloot) Date: Sun, 21 Feb 2010 00:22:47 +0100 Subject: Are function types a requirement? In-Reply-To: <15e8b9d21002191531k2791165co32fe00233bb2c3fe@mail.gmail.com> References: <4B7C92A6.9020008@joda.org> <560fb5ed1002172006n40851460i3548077b3a74c1e6@mail.gmail.com> <15e8b9d21002172016g3482dcaao3e94f47b6666efe3@mail.gmail.com> <560fb5ed1002172201s1c1eae5am4dd7d8f8627e7eae@mail.gmail.com> <4B7DA002.1080501@sun.com> <560fb5ed1002181556p61e3dabr472c0842dba8572@mail.gmail.com> <15e8b9d21002181819w2671803fwe72131ec98abedb6@mail.gmail.com> <560fb5ed1002191504o59b0c369rdf45d984d69b77e3@mail.gmail.com> <15e8b9d21002191531k2791165co32fe00233bb2c3fe@mail.gmail.com> Message-ID: <560fb5ed1002201522j7b0f01casee3699adf0bc53c3@mail.gmail.com> ... but the compiler catches this. class B { void m(String... a) {} } class C extends B { void m(String[] a) {} } new C().m("a", "b"); is legal. It's not exactly a leap of faith to consider that the compiler already knows that C's m() method is virtually varargs, and therefore gets the same checks applied to it. At worst they get a warning, not a compiler error. If this is a bridge too far for the spec, then just drop it. That means you get a warning when you pass a varargs on to another varargs, and the strength of the proposal drops from eliminating completely unnecessary warnings from 99.9999% of the time to merely 95% of the time. Worth it? Oh yes. --Reinier Zwitserloot On Sat, Feb 20, 2010 at 12:31 AM, Neal Gafter wrote: > On Fri, Feb 19, 2010 at 3:04 PM, Reinier Zwitserloot < > reinier at zwitserloot.com> wrote: > >> I believe I mentioned that chaining varargs into another varargs is >> actually allowed (and, as the same rules apply to the chained method, does >> not weaken type safety, unless you mix compilation units compiled by >> different versions of java, an act which has always had ample opportunity >> for uncaught type errors). If not, that's an omission in putting it down in >> this thread, it's always been part of this set of rules. >> > > I'm afraid it does weaken type safety. A varargs method can override, and > be overridden by, a non-varargs method. > From Alex.Buckley at Sun.COM Sat Feb 20 22:46:56 2010 From: Alex.Buckley at Sun.COM (Alex Buckley) Date: Sat, 20 Feb 2010 22:46:56 -0800 Subject: Lightweight interfaces instead of function types In-Reply-To: <4b4f45e01002191015w4341c46dia6a548fe4bd5730f@mail.gmail.com> References: <4b4f45e01002190928x5230dfeek2c1490dedccc71fd@mail.gmail.com> <15e8b9d21002190947p4af23210o1e2e5d280144da60@mail.gmail.com> <4b4f45e01002191015w4341c46dia6a548fe4bd5730f@mail.gmail.com> Message-ID: <4B80D6E0.103@sun.com> Stephen Colebourne wrote: > On 19 February 2010 17:47, Neal Gafter wrote: >> Tmp.java:5: [error] generic array creation >> Callable[] array = new Callable[10]; >> This is a hard error, not a warning. > > True. But if I want a Callable[] I write: > > Callable[] array = new Callable[10]; > > This works, and is safe /in practice/ (rather than type-system safe). No, it isn't safe in practice. It causes heap pollution which can cause programs to crash. It is a hack allowed because migration compatibility was so very, very important. (It is not particularly helpful of IDE vendors to hide the unchecked warning by default.) Reified generics don't help either; they just make the topic 5x more complicated. > Again, its a trade-off. The benefit for ParallelArray is that > Predicate or Transformer is IMO more meaningful to someone just > casually coming to the API than a random function type. I actually agree with this. Long nominal type names are the Java way. Unfortunately, we don't have reified generics over primitive type arguments (Predicate, Ops.Reducer). This is because we don't know how to implement them efficiently, even if strides have been made over the past decade w.r.t. implementing reified generics over reference type arguments. Reifying function types as MethodHandles still has no specialization for primitive types, so won't give Doug what he needs. Long story short: first-class functions in the Java language may be doomed without fully reified structural types in the Java VM. (Other languages can avoid primitive types or box them to their hearts' content, but again, that doesn't help Java programmer or Doug's library.) Hence my "peering over the edge of a very tall cliff" comment last week. Alex From neal at gafter.com Sat Feb 20 23:37:55 2010 From: neal at gafter.com (Neal Gafter) Date: Sat, 20 Feb 2010 23:37:55 -0800 Subject: Lightweight interfaces instead of function types In-Reply-To: <4b4f45e01002191015w4341c46dia6a548fe4bd5730f@mail.gmail.com> References: <4b4f45e01002190928x5230dfeek2c1490dedccc71fd@mail.gmail.com> <15e8b9d21002190947p4af23210o1e2e5d280144da60@mail.gmail.com> <4b4f45e01002191015w4341c46dia6a548fe4bd5730f@mail.gmail.com> Message-ID: <15e8b9d21002202337u208f039aue7d6b6279da4ecc1@mail.gmail.com> On Fri, Feb 19, 2010 at 10:15 AM, Stephen Colebourne wrote: > On 19 February 2010 17:47, Neal Gafter wrote: > > Tmp.java:5: [error] generic array creation > > Callable[] array = new Callable[10]; > > This is a hard error, not a warning. > > True. But if I want a Callable[] I write: > > Callable[] array = new Callable[10]; > > This works, and is safe /in practice/ (rather than type-system safe). > It is because people like you have been teaching this "safe in practice" attitude that people write code like that... code that would break hard if generics were ever reified. The existence of so much code that would break is a principal obstacle to actually reifying generics. From Alex.Buckley at Sun.COM Sun Feb 21 01:47:31 2010 From: Alex.Buckley at Sun.COM (Alex Buckley) Date: Sun, 21 Feb 2010 01:47:31 -0800 Subject: Lambda Conversion Considered Harmful? In-Reply-To: <17b2302a1002200011x361a5704g83cc445a9aa9705f@mail.gmail.com> References: <17b2302a1002182058k72e04d04h8fa85c2b409288dc@mail.gmail.com> <7926817e1002190944p69511f6dhc92eb1ba1113dd55@mail.gmail.com> <17b2302a1002191118n37713a1av283a456050d839f1@mail.gmail.com> <15e8b9d21002191122v5f99435cxe113fed90471be0@mail.gmail.com> <4B7F0C1F.5050906@sun.com> <17b2302a1002191536t705318dem88ab86a00861942a@mail.gmail.com> <15e8b9d21002191556j67e3687fy552564f4ad52744@mail.gmail.com> <17b2302a1002191613l40c36803i4bc9bbb4df9c0305@mail.gmail.com> <15e8b9d21002191815x63cd54eck69c04bbd6cbc7640@mail.gmail.com> <17b2302a1002200011x361a5704g83cc445a9aa9705f@mail.gmail.com> Message-ID: <4B810133.4070901@sun.com> Joshua Bloch wrote: > On Fri, Feb 19, 2010 at 6:15 PM, Neal Gafter wrote: > >> On Fri, Feb 19, 2010 at 4:13 PM, Joshua Bloch wrote: >>> I'd argue that the bug is that byte is defined to be a subtype of int. >> This >>> wasn't true in earlier editions of the JLS, as I recall;) >> Earlier editions didn't define "subtype". >> > > They may not have defined the term, but they did use it, for example: > > Assignment to an array component of reference type (?10.10, ?15.13, > ?15.26.1). The type-checking rules allow the array type S[] to be treated as > a subtype of T[] if S is a subtype of T, but this requires a run-time check > for assignment to an array component, similar to the check performed for a cast. > > from page 320 of the Second Ed. That's the only occurrence of the term, and it's basically in non-normative text. (There is a paper waiting to be written on specification of a specification.) To the issue, I have noted that JLS3 5.1.5 should be restricted to *reference* types S and T. It's just good practice to be precise. For primitive types, JLS2 did not describe subtyping. It is irregular for JLS3 to both carry over the enumeration of primitive conversions from JLS2 (5.1.2/3) and also enumerate a primitive subtype hierarchy (4.10.1). I propose keeping 4.10.1 and having 5.1.2/3 defer to it, adding the interesting narrowing conversions of short->char, char->short, and char->byte to 5.1.3 of course. Alex From forax at univ-mlv.fr Sun Feb 21 03:58:50 2010 From: forax at univ-mlv.fr (=?ISO-8859-1?Q?R=E9mi_Forax?=) Date: Sun, 21 Feb 2010 12:58:50 +0100 Subject: Lightweight interfaces instead of function types In-Reply-To: <15e8b9d21002202337u208f039aue7d6b6279da4ecc1@mail.gmail.com> References: <4b4f45e01002190928x5230dfeek2c1490dedccc71fd@mail.gmail.com> <15e8b9d21002190947p4af23210o1e2e5d280144da60@mail.gmail.com> <4b4f45e01002191015w4341c46dia6a548fe4bd5730f@mail.gmail.com> <15e8b9d21002202337u208f039aue7d6b6279da4ecc1@mail.gmail.com> Message-ID: <4B811FFA.1030206@univ-mlv.fr> Le 21/02/2010 08:37, Neal Gafter a ?crit : > On Fri, Feb 19, 2010 at 10:15 AM, Stephen Colebourne > wrote: > > >> On 19 February 2010 17:47, Neal Gafter wrote: >> >>> Tmp.java:5: [error] generic array creation >>> Callable[] array = new Callable[10]; >>> This is a hard error, not a warning. >>> >> True. But if I want a Callable[] I write: >> >> Callable[] array = new Callable[10]; >> >> This works, and is safe /in practice/ (rather than type-system safe). >> >> > It is because people like you have been teaching this "safe in practice" > attitude that people write code like that... code that would break hard if > generics were ever reified. The existence of so much code that would break > is a principal obstacle to actually reifying generics. > It's not safe ? it raises a warning. R?mi From scolebourne at joda.org Sun Feb 21 03:58:13 2010 From: scolebourne at joda.org (Stephen Colebourne) Date: Sun, 21 Feb 2010 11:58:13 +0000 Subject: Lightweight interfaces instead of function types In-Reply-To: <4B80D6E0.103@sun.com> References: <4b4f45e01002190928x5230dfeek2c1490dedccc71fd@mail.gmail.com> <15e8b9d21002190947p4af23210o1e2e5d280144da60@mail.gmail.com> <4b4f45e01002191015w4341c46dia6a548fe4bd5730f@mail.gmail.com> <4B80D6E0.103@sun.com> Message-ID: <4b4f45e01002210358u1859ec59tcfdeee9084f4b90f@mail.gmail.com> >> Callable[] array = new Callable[10]; >> This works, and is safe /in practice/ (rather than type-system safe). On 21 February 2010 06:46, Alex Buckley wrote: > No, it isn't safe in practice. It causes heap pollution which can cause > programs to crash. It is a hack allowed because migration compatibility > was so very, very important. (It is not particularly helpful of IDE > vendors to hide the unchecked warning by default.) Reified generics > don't help either; they just make the topic 5x more complicated. On 21 February 2010 07:37, Neal Gafter wrote: > It is because people like you have been teaching this "safe in practice" > attitude that people write code like that... code that would break hard if > generics were ever reified. The existence of so much code that would break > is a principal obstacle to actually reifying generics. The "safe in practice" point is that for the vast majority of developers, for the vast majority of time, this will never be an issue. Arrays are relatively rarely assigned to another array, and arrays are relatively rarely used in method signatures Its also important to say that most developers don't understand the detailed reason why it has to produce a warning, they just know that lots of things in generics suck. I expect the sequence of events to be something like: Developer writes: Callable[] = new Callable[1]; Gets compile error. Curses sucky generics. Thinks "how can I work around generics". Thinks "this would work without generics". Changes code: Callable[] = new Callable[1]; Gets warning. Curses sucky generics. Clicks IDE "get rid of warnings" button. Carries on with what they were actually trying to do. The developer is just interested in getting their task done, not keeping the compiler happy. Taken from that perspective, its not unreasonable to wonder if the error and warning actually accomplished anything? I'd also note that varargs use arrays, and with the Coin fix we will see more generified arrays, not less. More broadly, it has been noted by more than one person that ClassCastExceptions from ungenerified objects/collections or ArrayStoreExceptions have always been a rarity in real code by comparison to NullPointerExceptions (ie. why aren't we tackling NPEs???). I'm not trying to start a debate on this - its partly philosophical. But I do feel its very important to make the point that, IMO, most developers don't worry about these things to the extent that language designers do. Writing Callable[] array = new Callable[10]; has become just a necessary part of using Java today. Stephen From mthornton at optrak.co.uk Sun Feb 21 04:10:38 2010 From: mthornton at optrak.co.uk (Mark Thornton) Date: Sun, 21 Feb 2010 12:10:38 +0000 Subject: Lightweight interfaces instead of function types In-Reply-To: <4b4f45e01002210358u1859ec59tcfdeee9084f4b90f@mail.gmail.com> References: <4b4f45e01002190928x5230dfeek2c1490dedccc71fd@mail.gmail.com> <15e8b9d21002190947p4af23210o1e2e5d280144da60@mail.gmail.com> <4b4f45e01002191015w4341c46dia6a548fe4bd5730f@mail.gmail.com> <4B80D6E0.103@sun.com> <4b4f45e01002210358u1859ec59tcfdeee9084f4b90f@mail.gmail.com> Message-ID: <4B8122BE.4050805@optrak.co.uk> Stephen Colebourne wrote: > I'm not trying to start a debate on this - its partly philosophical. > But I do feel its very important to make the point that, IMO, most > developers don't worry about these things to the extent that language > designers do. Writing Callable[] array = new Callable[10]; has > become just a necessary part of using Java today. > > Common yes, necessary no. Leaving the LHS as Callable[] is safe, but requires more casts. The varargs case isn't equivalent because were generics to be reified it would be fixed in one place (the compiler). Mark From forax at univ-mlv.fr Sun Feb 21 04:22:22 2010 From: forax at univ-mlv.fr (=?ISO-8859-1?Q?R=E9mi_Forax?=) Date: Sun, 21 Feb 2010 13:22:22 +0100 Subject: Lightweight interfaces instead of function types In-Reply-To: <4B80D6E0.103@sun.com> References: <4b4f45e01002190928x5230dfeek2c1490dedccc71fd@mail.gmail.com> <15e8b9d21002190947p4af23210o1e2e5d280144da60@mail.gmail.com> <4b4f45e01002191015w4341c46dia6a548fe4bd5730f@mail.gmail.com> <4B80D6E0.103@sun.com> Message-ID: <4B81257E.8010205@univ-mlv.fr> Le 21/02/2010 07:46, Alex Buckley a ?crit : [...] > Reifying function types as MethodHandles still has no specialization for > primitive types, so won't give Doug what he needs. Yes and no, Method handles have a specialized method invoke. MethodHandle mh = ldc virtual indexOf(char)int; int index = mh.invoke("foo", 'f'); invocation in this example doesn't require any boxing. > Long story short: > first-class functions in the Java language may be doomed without fully > reified structural types in the Java VM. (Other languages can avoid > primitive types or box them to their hearts' content, but again, that > doesn't help Java programmer or Doug's library.) > > Hence my "peering over the edge of a very tall cliff" comment last week. > > Alex > R?mi From oallouch at free.fr Sun Feb 21 05:36:45 2010 From: oallouch at free.fr (Olivier Allouch) Date: Sun, 21 Feb 2010 14:36:45 +0100 Subject: Lightweight interfaces instead of function types In-Reply-To: <4b4f45e01002210358u1859ec59tcfdeee9084f4b90f@mail.gmail.com> References: <4b4f45e01002190928x5230dfeek2c1490dedccc71fd@mail.gmail.com> <15e8b9d21002190947p4af23210o1e2e5d280144da60@mail.gmail.com> <4b4f45e01002191015w4341c46dia6a548fe4bd5730f@mail.gmail.com> <4B80D6E0.103@sun.com> <4b4f45e01002210358u1859ec59tcfdeee9084f4b90f@mail.gmail.com> Message-ID: <4B8136ED.5000709@free.fr> +1000 Designing an API is like desigining an innovative UI, you have to be in the user's mind. You can say "every user is different" or "a coder can learn and adapt", but they (we) usually don't. We learn by using. Btw, ditching expressions is an excellent idea. It will avoid "where the hell is the return" reflexes. Le 21/02/2010 12:58, Stephen Colebourne a ?crit : >>> Callable[] array = new Callable[10]; >>> This works, and is safe /in practice/ (rather than type-system safe). >>> > On 21 February 2010 06:46, Alex Buckley wrote: > >> No, it isn't safe in practice. It causes heap pollution which can cause >> programs to crash. It is a hack allowed because migration compatibility >> was so very, very important. (It is not particularly helpful of IDE >> vendors to hide the unchecked warning by default.) Reified generics >> don't help either; they just make the topic 5x more complicated. >> > On 21 February 2010 07:37, Neal Gafter wrote: > >> It is because people like you have been teaching this "safe in practice" >> attitude that people write code like that... code that would break hard if >> generics were ever reified. The existence of so much code that would break >> is a principal obstacle to actually reifying generics. >> > The "safe in practice" point is that for the vast majority of > developers, for the vast majority of time, this will never be an > issue. Arrays are relatively rarely assigned to another array, and > arrays are relatively rarely used in method signatures > > Its also important to say that most developers don't understand the > detailed reason why it has to produce a warning, they just know that > lots of things in generics suck. I expect the sequence of events to be > something like: > > Developer writes: > Callable[] = new Callable[1]; > Gets compile error. > Curses sucky generics. > Thinks "how can I work around generics". > Thinks "this would work without generics". > Changes code: > Callable[] = new Callable[1]; > Gets warning. > Curses sucky generics. > Clicks IDE "get rid of warnings" button. > Carries on with what they were actually trying to do. > > The developer is just interested in getting their task done, not > keeping the compiler happy. Taken from that perspective, its not > unreasonable to wonder if the error and warning actually accomplished > anything? > > I'd also note that varargs use arrays, and with the Coin fix we will > see more generified arrays, not less. > > More broadly, it has been noted by more than one person that > ClassCastExceptions from ungenerified objects/collections or > ArrayStoreExceptions have always been a rarity in real code by > comparison to NullPointerExceptions (ie. why aren't we tackling > NPEs???). > > I'm not trying to start a debate on this - its partly philosophical. > But I do feel its very important to make the point that, IMO, most > developers don't worry about these things to the extent that language > designers do. Writing Callable[] array = new Callable[10]; has > become just a necessary part of using Java today. > > Stephen > > From forax at univ-mlv.fr Sun Feb 21 07:26:08 2010 From: forax at univ-mlv.fr (=?ISO-8859-1?Q?R=E9mi_Forax?=) Date: Sun, 21 Feb 2010 16:26:08 +0100 Subject: this should not refer to the lambda Message-ID: <4B815090.6060708@univ-mlv.fr> I've a bad feeling about 'this' referring to the lambda an not to the enclosing object. I've tried to summarize my grievances, some had been already reported by Josh and Neal. 1) only few existing inner classes contains one recursive method so adding a special syntax for a corner case doesn't worth it. Moreover, recursive method often have more parameters, a counter, a mutable object etc that are used during the recursion. but not needed outside. This will not fit well with lambda which often must respect a given signature. 2) If 'this' allow to reference the current lambda, for completeness, what is the syntax to reference an enclosing lambda in a lambda ? 3) if 'this' refers to the lambda, it doesn't play well with type inference. 4) Here is a snippet with use lambda conversion, function type and lambda. public class MyPanel extends JPanel { void createChangeColorButton(Color color, #void(JPanel,Color) action) { JButton button = new JButton(); button.setBackground(color); button.add(this); button.addActionListener(#(ActionEvent e) { action.invoke(this, color); }); } public static void main(String[] args) { MyPanel panel = new MyPanel(); panel.createChangeButton(Color.RED, #(JPanel panel, Color color) { panel.setBackground(color); }); panel.createChangeButton(Color.BLUE, #(JPanel panel, Color color) { panel.setForeground(color); }); ... } } A lambda statement really looks like a block and not like an object creation. If 'this' refers to the lambda itself, it's error prone. In the example, color and action are not declared final, because there are not modified. This trick let the developer to care less about scopes. But if 'this' refers to the lambda, scopes are important. The 'this as the current lambda' feature seems to go in the wrong direction. For my students, I would love to explain lambda without having to explain inner classes and anonymous classes. Even if Java developers are used to use inner classes, don't forget that scoping rules are complex to understand for rookies (particularly if your first language is C and not Javascript :) Lambdas are by itself a concept which is not simple to grasp, please don't pollute it with unnecessary feature. Just a last example: I don't want to explain why 'this' doesn't mean the same thing in the two codes: Object[] array = { this }; // here this doesn't refer to the array #Object() fun = #() (this); // here this refer to the lambda ?? R?mi From alex.blewitt at gmail.com Sun Feb 21 08:44:18 2010 From: alex.blewitt at gmail.com (Alex Blewitt) Date: Sun, 21 Feb 2010 16:44:18 +0000 Subject: this should not refer to the lambda In-Reply-To: <4B815090.6060708@univ-mlv.fr> References: <4B815090.6060708@univ-mlv.fr> Message-ID: <9AA3F77A-0214-4296-9735-C77E00A58045@gmail.com> On 21 Feb 2010, at 15:26, R?mi Forax wrote: > Just a last example: I don't want to explain why 'this' doesn't mean > the same thing in the two codes: > > Object[] array = { this }; // here this doesn't refer to the array > > #Object() fun = #() (this); // here this refer to the lambda ?? This argument is somewhat tautologous. "I'd have difficulty in explaining it because I don't like what it means". > Object o = new Runnable() { void run() { this;} }// this refers to the runnable, not the enclosing object. The # is a lot more like the Runnable than the block case; you are (potentially) instantiating a new "thing" (it can throw OOM) and that thing has reference semantics, much like an instance would. A block cannot throw an OOM nor does it have reference semantics. Note that I am not assuming any particular implementation of # but these effects follow from the fact that # may be a closure over local state (thus requiring a binding context to be created) and by virtue that a lambda may be passed around as a reference. Even if there is a third type of top-level element added to the language, the it-doesn't-fit-my-mental-model-of-what-I'd-like-it-to-be is not a good reason one way or another. What we have to ensure is that we explain it in a way that people can conceptualise, and to ensure that the # is treated as a psuedo-new operator to ensure that it doesn't become a source of memory leaks. (I use pseudo-new here because it shouldn't always create a new instance, particularly when no local scope is captured.) But having a potentially instance-creating, reference based type and then disallowing the use of the operator in Java which is used to refer to self-instances seems (to me) inconsistent with the language as is (though of course, this is about changing the language). And, as noted elsewhere, the ablility to reference the enclosing type is not hindered in any way if "this" does refer to the lambda. As for the nested lambda case, this is also possible (either implicitly due to local state capture or explicitly) without having to resort to language gymnastics like letting an RHS expression refer to the LHS type. public class A { public class B { public void foo() { #() { // Need to add this line if needed Object me = this; #() { A.this; // A instance B.this; // B instance me; // outer lambda this; // inner lambda } } } } } It seems this doesn't prevent anything and doesn't require self-refs to be designed as: Object me = #() { me} There is no other construct in Jaca where you can do: Object foo = ... foo ... Since on the RHS foo isn't in scope. And it'll cause merry hell for IDEs which are built to assume that the RHS can't refer to an LHS assignment - a not unreasonable assumption in any language. (This may also break refactoring concepts as well; whilst there will obviously extensions which will be needed to treat lambdas in the general case, the being able to refer to the LHS from the RHS is the primary issue.) Still, I suppose we couldn't screw up more than generics, so if we do allow RHS to use LHS then it will just be another future pain point for Java programmers of the future. Alex From grev at miginfocom.com Sun Feb 21 08:50:50 2010 From: grev at miginfocom.com (Mikael Grev) Date: Sun, 21 Feb 2010 17:50:50 +0100 Subject: this should not refer to the lambda In-Reply-To: <4B815090.6060708@univ-mlv.fr> References: <4B815090.6060708@univ-mlv.fr> Message-ID: <4CD9532E-8B8C-4CD2-96DE-478358B4EF90@miginfocom.com> I concur. If Java is going down the closure/lambda path, and not just making anonymous inner classes easier to write, it makes sense to cater for the more used use case. Pureness is only in the minds of the compiler experts and of no concern for the general developer. We should also be smart here. Which choice will produce the least flack? I'm pretty sure that is the path with this referencing the outer context. Java needs bloggers to stay possitive and the bad press will be because they are too cumbersome to use or too little to late (as always). Cheers, Mikael On 21 feb 2010, at 16.26, R?mi Forax wrote: > I've a bad feeling about 'this' referring to the lambda an not to the > enclosing object. > I've tried to summarize my grievances, some had been already > reported by > Josh and Neal. > > 1) only few existing inner classes contains one recursive method > so adding a special syntax for a corner case doesn't worth it. > Moreover, recursive method often have more parameters, > a counter, a mutable object etc that are used during the recursion. > but not needed outside. This will not fit well with lambda > which often must respect a given signature. > > 2) If 'this' allow to reference the current lambda, > for completeness, what is the syntax to reference > an enclosing lambda in a lambda ? > > 3) if 'this' refers to the lambda, it doesn't play well with type > inference. > > 4) Here is a snippet with use lambda conversion, function type > and lambda. > > public class MyPanel extends JPanel { > void createChangeColorButton(Color color, #void(JPanel,Color) > action) { > JButton button = new JButton(); > button.setBackground(color); > button.add(this); > button.addActionListener(#(ActionEvent e) { > action.invoke(this, color); > }); > } > > public static void main(String[] args) { > MyPanel panel = new MyPanel(); > panel.createChangeButton(Color.RED, #(JPanel panel, Color color) { > panel.setBackground(color); > }); > panel.createChangeButton(Color.BLUE, #(JPanel panel, Color > color) { > panel.setForeground(color); > }); > ... > } > } > > A lambda statement really looks like a block and not like an > object > creation. > If 'this' refers to the lambda itself, it's error prone. > > In the example, color and action are not declared final, > because there are not modified. > This trick let the developer to care less about scopes. > But if 'this' refers to the lambda, scopes are important. > The 'this as the current lambda' feature seems to go in the wrong > direction. > > For my students, I would love to explain lambda without having to > explain > inner classes and anonymous classes. > Even if Java developers are used to use inner classes, > don't forget that scoping rules are complex to understand for rookies > (particularly if your first language is C and not Javascript :) > Lambdas are by itself a concept which is not simple to grasp, > please don't pollute it with unnecessary feature. > > Just a last example: I don't want to explain why 'this' doesn't mean > the same thing in the two codes: > > Object[] array = { this }; // here this doesn't refer to the array > > #Object() fun = #() (this); // here this refer to the lambda ?? > > > R?mi From mthornton at optrak.co.uk Sun Feb 21 09:05:34 2010 From: mthornton at optrak.co.uk (Mark Thornton) Date: Sun, 21 Feb 2010 17:05:34 +0000 Subject: this should not refer to the lambda In-Reply-To: <9AA3F77A-0214-4296-9735-C77E00A58045@gmail.com> References: <4B815090.6060708@univ-mlv.fr> <9AA3F77A-0214-4296-9735-C77E00A58045@gmail.com> Message-ID: <4B8167DE.3070208@optrak.co.uk> Alex Blewitt wrote: > The # is a lot more like the Runnable than the block case; you are > (potentially) instantiating a new "thing" (it can throw OOM) and that > thing has reference semantics, much like an instance would. A block > cannot throw an OOM nor does it have reference semantics. > If a block declares variables, is it permissable for VM to allocate space for them on entry to the block instead of at method entry? Some VM's dynamically resize stacks, thus potentially transforming a stack overflow into OOM. Mark From neal at gafter.com Sun Feb 21 10:08:56 2010 From: neal at gafter.com (Neal Gafter) Date: Sun, 21 Feb 2010 10:08:56 -0800 Subject: Lightweight interfaces instead of function types In-Reply-To: <4b4f45e01002210358u1859ec59tcfdeee9084f4b90f@mail.gmail.com> References: <4b4f45e01002190928x5230dfeek2c1490dedccc71fd@mail.gmail.com> <15e8b9d21002190947p4af23210o1e2e5d280144da60@mail.gmail.com> <4b4f45e01002191015w4341c46dia6a548fe4bd5730f@mail.gmail.com> <4B80D6E0.103@sun.com> <4b4f45e01002210358u1859ec59tcfdeee9084f4b90f@mail.gmail.com> Message-ID: <15e8b9d21002211008x5d50fc92u40d5619966e7fd38@mail.gmail.com> On Sun, Feb 21, 2010 at 3:58 AM, Stephen Colebourne wrote: > More broadly, it has been noted by more than one person that > ClassCastExceptions from ungenerified objects/collections or > ArrayStoreExceptions have always been a rarity in real code by > comparison to NullPointerExceptions (ie. why aren't we tackling > NPEs???). > I'm trying to figure out how this statement fits into the discussion. Are you suggesting that because dynamic type safety is hardly ever violated in practice, there would be little value in ever reifying generics (compared to, for example, the jsr308 work in JDK7 to help eliminate NPEs using annotations)? If so, that would explain your eagerness to encourage coding patterns that would undermine it. From forax at univ-mlv.fr Sun Feb 21 10:51:57 2010 From: forax at univ-mlv.fr (=?UTF-8?B?UsOpbWkgRm9yYXg=?=) Date: Sun, 21 Feb 2010 19:51:57 +0100 Subject: this should not refer to the lambda In-Reply-To: <9AA3F77A-0214-4296-9735-C77E00A58045@gmail.com> References: <4B815090.6060708@univ-mlv.fr> <9AA3F77A-0214-4296-9735-C77E00A58045@gmail.com> Message-ID: <4B8180CD.3090900@univ-mlv.fr> Le 21/02/2010 17:44, Alex Blewitt a ?crit : > On 21 Feb 2010, at 15:26, R?mi Forax wrote: > >> Just a last example: I don't want to explain why 'this' doesn't mean >> the same thing in the two codes: >> >> Object[] array = { this }; // here this doesn't refer to the array >> >> #Object() fun = #() (this); // here this refer to the lambda ?? > > This argument is somewhat tautologous. "I'd have difficulty in > explaining it because I don't like what it means". I have no difficulty to explain it. I don't want to explain it. There is different way to weight a feature, my way is to try to evaluate how many hours it will take to explain it. The other useful metric is in which order you have to explain the feature. By example, enum is a complex feature in Java because you have to first explain interface and anonymous class. I think it's possible to explain lambda *before* anonymous class as a lightweight anonymous function. > >> > Object o = new Runnable() { void run() { this;} }// this refers to the > runnable, not the enclosing object. > > The # is a lot more like the Runnable than the block case; you are > (potentially) instantiating a new "thing" (it can throw OOM) and that > thing has reference semantics, much like an instance would. A block > cannot throw an OOM nor does it have reference semantics. > > Note that I am not assuming any particular implementation of # but > these effects follow from the fact that # may be a closure over local > state (thus requiring a binding context to be created) and by virtue > that a lambda may be passed around as a reference. > > Even if there is a third type of top-level element added to the > language, the it-doesn't-fit-my-mental-model-of-what-I'd-like-it-to-be > is not a good reason one way or another. What we have to ensure is > that we explain it in a way that people can conceptualise, and to > ensure that the # is treated as a psuedo-new operator to ensure that > it doesn't become a source of memory leaks. > > (I use pseudo-new here because it shouldn't always create a new > instance, particularly when no local scope is captured.) > > But having a potentially instance-creating, reference based type and > then disallowing the use of the operator in Java which is used to > refer to self-instances seems (to me) inconsistent with the language > as is (though of course, this is about changing the language). > > And, as noted elsewhere, the ablility to reference the enclosing type > is not hindered in any way if "this" does refer to the lambda. As for > the nested lambda case, this is also possible (either implicitly due > to local state capture or explicitly) without having to resort to > language gymnastics like letting an RHS expression refer to the LHS type. > > public class A { > public class B { > public void foo() { > #() { > // Need to add this line if needed > Object me = this; > #() { > A.this; // A instance > B.this; // B instance > me; // outer lambda > this; // inner lambda > } > } > } > } > } > > It seems this doesn't prevent anything and doesn't require self-refs > to be designed as: > > Object me = #() { me} > > There is no other construct in Jaca where you can do: > > Object foo = ... foo ... > > Since on the RHS foo isn't in scope. And it'll cause merry hell for > IDEs which are built to assume that the RHS can't refer to an LHS > assignment - a not unreasonable assumption in any language. (This may > also break refactoring concepts as well; whilst there will obviously > extensions which will be needed to treat lambdas in the general case, > the being able to refer to the LHS from the RHS is the primary issue.) > > Still, I suppose we couldn't screw up more than generics, so if we do > allow RHS to use LHS then it will just be another future pain point > for Java programmers of the future. Alex, try to compile: int a =a; and take a look to the error message. > > Alex > R?mi From alex.blewitt at gmail.com Sun Feb 21 11:11:12 2010 From: alex.blewitt at gmail.com (Alex Blewitt) Date: Sun, 21 Feb 2010 19:11:12 +0000 Subject: this should not refer to the lambda In-Reply-To: <4B8180CD.3090900@univ-mlv.fr> References: <4B815090.6060708@univ-mlv.fr> <9AA3F77A-0214-4296-9735-C77E00A58045@gmail.com> <4B8180CD.3090900@univ-mlv.fr> Message-ID: <636fd28e1002211111y54a44250v414f5ef0affce419@mail.gmail.com> On Sun, Feb 21, 2010 at 6:51 PM, R?mi Forax wrote: >> This argument is somewhat tautologous. "I'd have difficulty in explaining >> it because I don't like what it means". > > I have no difficulty to explain it. I don't want to explain it. I'm not sure that is a technical reason to not include something. > Alex, try to compile: > int a =a; > and take a look to the error message. Test.java:2: illegal forward reference int a=a; ^ That's the point - it's an illegal reference. Yet, were 'this' used to describe only the enclosing object instance (and not the lambda instance), you wouldn't be able to do recursive lambdas. This prints out a countdown from n to 0: #(i) { System.out.println(i); if(i>0) this.(i-1) }; So without 'this' referring to the enclosing lambda, you can't do this. An earlier proposal would be to have an illegal forward reference like: countdown = #(i) { System.out.println(i); if(i>0) countdown.(i-1) } So, you either allow 'this' as referring to the lambda (to support the recursive case), or disallow 'this' to refer to the lambda; at which point, you either disallow recursive cases or use hackery forward references to enable recursion. Given that the example I posted earlier lets your refer to any number of nested lambdas, and that in the most case, the compiler should be smart enough to resolve the correct enclosing scope where necessary (i.e. you won't need to put 'this' in that frequently, since it will work up the stack to find it, much like inner classes work today) - what exactly is the problem with letting 'this' refer to the lambda? The alternative seems to be to allow 'this' to refer to the lambda, and then use 'A.this' when you want to access the enclosing classes 'this'. I doubt many Java programmers would find that difficult, especially since that's how inner classes work today. I have yet to see a technical argument of what can be done if you disallow 'this' as a lambda reference, other than having to type in n+1 extra characters should you ever want to refer to the enclosing scope. Alex From neal at gafter.com Sun Feb 21 11:48:42 2010 From: neal at gafter.com (Neal Gafter) Date: Sun, 21 Feb 2010 11:48:42 -0800 Subject: this should not refer to the lambda In-Reply-To: <636fd28e1002211111y54a44250v414f5ef0affce419@mail.gmail.com> References: <4B815090.6060708@univ-mlv.fr> <9AA3F77A-0214-4296-9735-C77E00A58045@gmail.com> <4B8180CD.3090900@univ-mlv.fr> <636fd28e1002211111y54a44250v414f5ef0affce419@mail.gmail.com> Message-ID: <15e8b9d21002211148p13fb1250w8271d7c1ec5210f4@mail.gmail.com> On Sun, Feb 21, 2010 at 11:11 AM, Alex Blewitt wrote: > Yet, were 'this' used to > describe only the enclosing object instance (and not the lambda > instance), you wouldn't be able to do recursive lambdas. > We've already shown both that the need for this is rare and that there are a number of ways to do it without having "this" refer to the lambda itself. I have yet to see a technical argument of what can be done if you > disallow 'this' as a lambda reference, other than having to type in > n+1 extra characters should you ever want to refer to the enclosing > scope. > There are no "what can be done" arguments on either side: Java is Turing-complete and will remain so. The technical arguments for transparency of semantics for lambdas have been made in many contexts over the years. A good place to start is with Steele's "Lambda the Ultimate..." series of papers. From alex.blewitt at gmail.com Sun Feb 21 12:29:13 2010 From: alex.blewitt at gmail.com (Alex Blewitt) Date: Sun, 21 Feb 2010 20:29:13 +0000 Subject: this should not refer to the lambda In-Reply-To: <15e8b9d21002211148p13fb1250w8271d7c1ec5210f4@mail.gmail.com> References: <4B815090.6060708@univ-mlv.fr> <9AA3F77A-0214-4296-9735-C77E00A58045@gmail.com> <4B8180CD.3090900@univ-mlv.fr> <636fd28e1002211111y54a44250v414f5ef0affce419@mail.gmail.com> <15e8b9d21002211148p13fb1250w8271d7c1ec5210f4@mail.gmail.com> Message-ID: <636fd28e1002211229l2f56285eg573d4c9ffee97a49@mail.gmail.com> On Sun, Feb 21, 2010 at 7:48 PM, Neal Gafter wrote: > On Sun, Feb 21, 2010 at 11:11 AM, Alex Blewitt > wrote: >> >> Yet, were 'this' used to >> describe only the enclosing object instance (and not the lambda >> instance), you wouldn't be able to do recursive lambdas. > > We've already shown both that the need for this is rare and that there are a > number of ways to do it without having "this" refer to the lambda itself. I've also shown that it's possible to refer to inner/outer classes, as well as nested lambdas, whilst using 'this' to refer to the lambda: http://mail.openjdk.java.net/pipermail/lambda-dev/2010-February/000823.html I don't recall seing a suggestion of how to implement a recursive lambda (which is 'rare' but not 'extinct') without breaking the illegal forward reference issue that I mentioned previously. I could have missed something; can you forward me the link of where that is done? > The technical arguments for > transparency of semantics for lambdas have been made in many contexts over > the years.? A good place to start is with Steele's "Lambda the Ultimate..." > series of papers. And yet, they've not been made/summarised here; nor is everyone convinced of the 'transparency' argument. Which is not to say it's not important to some; but let's not forget that if the issue boils down to 'we should be able to surround body X of code with a lambda without having to do anything else' then there are tools in compilers that let you achieve such refactorings (and adding 'this.' where necessary). Note that several cases exist where surrounding (or removing) bodies of code can introduce side effects; for example, in-lining a method can introduce the same named variable in a block of code; IDEs detect this and warn/prevent its use. Alex From peter.levart at gmail.com Sun Feb 21 13:50:36 2010 From: peter.levart at gmail.com (Peter Levart) Date: Sun, 21 Feb 2010 22:50:36 +0100 Subject: this should not refer to the lambda In-Reply-To: <636fd28e1002211111y54a44250v414f5ef0affce419@mail.gmail.com> References: <4B815090.6060708@univ-mlv.fr> <4B8180CD.3090900@univ-mlv.fr> <636fd28e1002211111y54a44250v414f5ef0affce419@mail.gmail.com> Message-ID: <201002212250.36999.peter.levart@gmail.com> On Sunday 21 February 2010 20:11:12 Alex Blewitt wrote: > This prints out a countdown from n to 0: > > #(i) { System.out.println(i); if(i>0) this.(i-1) }; > > So without 'this' referring to the enclosing lambda, you can't do > this. An earlier proposal would be to have an illegal forward > reference like: > > countdown = #(i) { System.out.println(i); if(i>0) countdown.(i-1) } > > So, you either allow 'this' as referring to the lambda (to support the > recursive case), or disallow 'this' to refer to the lambda; at which > point, you either disallow recursive cases or use hackery forward > references to enable recursion. #void(int) countdown = #(int i) { System.out.println(i); if(i>0) countdown.(i-1) } ...could be defined as a special function typed variable initializer (like for example array initializer). Plain and simple. No big deal. > > Given that the example I posted earlier lets your refer to any number > of nested lambdas, and that in the most case, the compiler should be > smart enough to resolve the correct enclosing scope where necessary > (i.e. you won't need to put 'this' in that frequently, since it will > work up the stack to find it, much like inner classes work today) - > what exactly is the problem with letting 'this' refer to the lambda? There is an implicit "this" which is much more important than explicit "this". If you make "this" refer to the lambda instance (of function type) and function types are reference types (subtypes of Object) then, to be consistent, you must treat unqualified Object members (hashCode, equals, wait*, notify*, getClass, toString) refer to the members of lambda instance too. Code statistics computed by some participants of this list show that you rarely need that in anonymous inner SAMs today. Quite the contrary, you often need to refer to Object members of enclosing instance. By making it necessary to qualify those references, you open the door to subtle bugs (some discovered just by inspecting the places in JDK sources where qualified "this" was used, see: http://mail.openjdk.java.net/pipermail/lambda-dev/2010- February/000550.html). I doubt that Object methods would have any utility being invoked on lambda instances: - equals/hashCode (hm, very dangerous, since identity of lambda is unspecified) - wait/notify (hm, very dangerous, since identity of lambda is unspecified) - getClass (of no use - might allways be MethodHandle) - toString (of no use either - might allways be the same) On the contrary, they still have the same utility when invoked on an enclosing object instance. By forcing the programmer to qualify them (i.e. MyOuterClass.this.equals(...)) you have forced her to type more for the majority of usecases. More importantly, NO programmer is qualifying references to other non-Object members of enclosing class (she is not a masochist is she?) when she's writing the body of a single method of an anonymous inner SAM instance. She's thinking in terms of outer class members at that time, so she must be very careful not to select any of the inner's Object methods by mistake. > > The alternative seems to be to allow 'this' to refer to the lambda, > and then use 'A.this' when you want to access the enclosing classes > 'this'. I doubt many Java programmers would find that difficult, > especially since that's how inner classes work today. They work that way, because they are classes. With known syntactic structure. And yet anonymous inner instances of SAM interfaces would often be better of with "this" referring to enclosing class. This of course is not possible since it would be inconsistent with other anonymous/named/local/nested/top classes. Lambda expressions, on the other hand, are syntactically something entirely different. We can escape the metal jacket with them. > > I have yet to see a technical argument of what can be done if you > disallow 'this' as a lambda reference, other than having to type in > n+1 extra characters should you ever want to refer to the enclosing > scope. I think I showed that "this" referring to lambda instance has a miniscule usage compared to the usage of implicit unqualified "this" (references to Object methods of enclosing instance). Tackling the problems of explicit "this" and implicit "this" (unqualified Object methods) separately - i.e. making "this" refer to lambda instance but unqualified Object methods refer to enclosing class - would be a big mistake. It would simply be inconsistent. Regards, Peter From forax at univ-mlv.fr Sun Feb 21 14:00:45 2010 From: forax at univ-mlv.fr (=?UTF-8?B?UsOpbWkgRm9yYXg=?=) Date: Sun, 21 Feb 2010 23:00:45 +0100 Subject: this should not refer to the lambda In-Reply-To: <636fd28e1002211111y54a44250v414f5ef0affce419@mail.gmail.com> References: <4B815090.6060708@univ-mlv.fr> <9AA3F77A-0214-4296-9735-C77E00A58045@gmail.com> <4B8180CD.3090900@univ-mlv.fr> <636fd28e1002211111y54a44250v414f5ef0affce419@mail.gmail.com> Message-ID: <4B81AD0D.1070208@univ-mlv.fr> Le 21/02/2010 20:11, Alex Blewitt a ?crit : > On Sun, Feb 21, 2010 at 6:51 PM, R?mi Forax wrote: > >>> This argument is somewhat tautologous. "I'd have difficulty in explaining >>> it because I don't like what it means". >>> >> I have no difficulty to explain it. I don't want to explain it. >> > I'm not sure that is a technical reason to not include something. > > >> Alex, try to compile: >> int a =a; >> and take a look to the error message. >> > Test.java:2: illegal forward reference > int a=a; > ^ > > That's the point - it's an illegal reference. Yet, were 'this' used to > describe only the enclosing object instance (and not the lambda > instance), you wouldn't be able to do recursive lambdas. > The message is not something like "can't find symbol". The compiler knows what 'a' means. > This prints out a countdown from n to 0: > > #(i) { System.out.println(i); if(i>0) this.(i-1) }; > > So without 'this' referring to the enclosing lambda, you can't do > this. An earlier proposal would be to have an illegal forward > reference like: > > countdown = #(i) { System.out.println(i); if(i>0) countdown.(i-1) } > > So, you either allow 'this' as referring to the lambda (to support the > recursive case), or disallow 'this' to refer to the lambda; at which > point, you either disallow recursive cases or use hackery forward > references to enable recursion. > > Given that the example I posted earlier lets your refer to any number > of nested lambdas, and that in the most case, the compiler should be > smart enough to resolve the correct enclosing scope where necessary > (i.e. you won't need to put 'this' in that frequently, since it will > work up the stack to find it, much like inner classes work today) - > what exactly is the problem with letting 'this' refer to the lambda? > > The alternative seems to be to allow 'this' to refer to the lambda, > and then use 'A.this' when you want to access the enclosing classes > 'this'. I doubt many Java programmers would find that difficult, > especially since that's how inner classes work today. > > I have yet to see a technical argument of what can be done if you > disallow 'this' as a lambda reference, other than having to type in > n+1 extra characters should you ever want to refer to the enclosing > scope. > > Alex > R?mi From neal at gafter.com Sun Feb 21 13:56:13 2010 From: neal at gafter.com (Neal Gafter) Date: Sun, 21 Feb 2010 13:56:13 -0800 Subject: this should not refer to the lambda In-Reply-To: <636fd28e1002211229l2f56285eg573d4c9ffee97a49@mail.gmail.com> References: <4B815090.6060708@univ-mlv.fr> <9AA3F77A-0214-4296-9735-C77E00A58045@gmail.com> <4B8180CD.3090900@univ-mlv.fr> <636fd28e1002211111y54a44250v414f5ef0affce419@mail.gmail.com> <15e8b9d21002211148p13fb1250w8271d7c1ec5210f4@mail.gmail.com> <636fd28e1002211229l2f56285eg573d4c9ffee97a49@mail.gmail.com> Message-ID: <15e8b9d21002211356t7ffe4b1cg3e1855621c795118@mail.gmail.com> On Sun, Feb 21, 2010 at 12:29 PM, Alex Blewitt wrote: > I don't recall seing a suggestion of how to implement a recursive > lambda (which is 'rare' but not 'extinct') without breaking the > illegal forward reference issue that I mentioned previously. I could > have missed something; can you forward me the link of where that is > done? Sure: http://mail.openjdk.java.net/pipermail/lambda-dev/2010-January/000432.html Just to be clear, the one minor adjustment to the definite assignment rules that is needed if you want to allow, for example #int(int) fac = #(int x)(x <= 1 ? 1 : x * fac.(x-1)); is trivial to specify (when the right-hand-side of a variable initialization is a lambda expression, the variable is definitely assigned before the lambda expression) and trivial to implement, and has been implemented in BGGA. This option has the advantage that it avoids the need to specify the lambda's result type. > > The technical arguments for > > transparency of semantics for lambdas have been made in many contexts over > > the years. A good place to start is with Steele's "Lambda the Ultimate..." > > series of papers. > And yet, they've not been made/summarised here; ... Some of them have, actually, but rather than continuing to predigest the information for you, perhaps you would care to take a stab at it. From alex.blewitt at gmail.com Sun Feb 21 14:18:01 2010 From: alex.blewitt at gmail.com (Alex Blewitt) Date: Sun, 21 Feb 2010 22:18:01 +0000 Subject: this should not refer to the lambda In-Reply-To: <15e8b9d21002211356t7ffe4b1cg3e1855621c795118@mail.gmail.com> References: <4B815090.6060708@univ-mlv.fr> <9AA3F77A-0214-4296-9735-C77E00A58045@gmail.com> <4B8180CD.3090900@univ-mlv.fr> <636fd28e1002211111y54a44250v414f5ef0affce419@mail.gmail.com> <15e8b9d21002211148p13fb1250w8271d7c1ec5210f4@mail.gmail.com> <636fd28e1002211229l2f56285eg573d4c9ffee97a49@mail.gmail.com> <15e8b9d21002211356t7ffe4b1cg3e1855621c795118@mail.gmail.com> Message-ID: <6D725A58-6874-4366-9500-A07C6EA84837@gmail.com> On 21 Feb 2010, at 21:56, Neal Gafter wrote: > On Sun, Feb 21, 2010 at 12:29 PM, Alex Blewitt wrote: >> I don't recall seing a suggestion of how to implement a recursive >> lambda (which is 'rare' but not 'extinct') without breaking the >> illegal forward reference issue that I mentioned previously. I could >> have missed something; can you forward me the link of where that is >> done? > > Sure: http://mail.openjdk.java.net/pipermail/lambda-dev/2010-January/000432.html > > Just to be clear, the one minor adjustment to the definite assignment > rules that is needed if you want to allow, for example > > #int(int) fac = #(int x)(x <= 1 ? 1 : x * fac.(x-1)); This is the example of the breaking the forward reference which I cited in my post you originally replied to. Specifically: http://mail.openjdk.java.net/pipermail/lambda-dev/2010-February/000823.html It seems this doesn't prevent anything and doesn't require self-refs to be designed as: Object me = #() { me } There is no other construct in Jaca [sic] where you can do: Object foo = ... foo ... Whether it's easy to implement is somewhat orthogonal to whether a language should be able to define a variable assignment with a LHS that refers to itself in the RHS. So, as noted, it appears to come to: * 'this' refers to the lambda, or * 'this' refers to enclosing object, and * recursive lambdas are disallowed, or * we break the language model such that special-cases of LHS assignments can refer to the LHS in the RHS >>> The technical arguments for >>> transparency of semantics for lambdas have been made in many contexts over >>> the years. A good place to start is with Steele's "Lambda the Ultimate..." >>> series of papers. >> And yet, they've not been made/summarised here; ... > > Some of them have, actually, but rather than continuing to predigest > the information for you, perhaps you would care to take a stab at it. If the point boils down to 'it should be possible to surround any code with a lambda block and execution', i.e. that: --- 8< --- foo; and #() { foo; }.(); // define, then execute, the lambda --- 8< --- are the same thing and can be done with no other changes, then the fact is that it's already impossible to do this. --- 8< --- int sum = 0; for(int i=0;i<10;i++ ) sum += i System.out.println(sum); -> int sum = 0; #() { for(int i=0;i<10;i++ ) sum += i }.(); // define, then execute, the lambda System.out.println(sum); --- 8< --- This will fail to compile for the (obvious) reason that 'sum' is not final. Therefore, there are some expressions such that it is not possible to surround with a lambda block/execute statement; which is my understanding of the 'transparency' argument. I may, of course, have misunderstood the details of 'transparency' here - but if the argument lies that 'this' refers to other references, and as such, an 'accidental' capture of something with a lambda (as in here) would fall into the category of problematic statements. Please let me know if I have misunderstood/misrepresented the transparency argument here. Alex From alex.blewitt at gmail.com Sun Feb 21 14:28:35 2010 From: alex.blewitt at gmail.com (Alex Blewitt) Date: Sun, 21 Feb 2010 22:28:35 +0000 Subject: this should not refer to the lambda In-Reply-To: <201002212250.36999.peter.levart@gmail.com> References: <4B815090.6060708@univ-mlv.fr> <4B8180CD.3090900@univ-mlv.fr> <636fd28e1002211111y54a44250v414f5ef0affce419@mail.gmail.com> <201002212250.36999.peter.levart@gmail.com> Message-ID: On 21 Feb 2010, at 21:50, Peter Levart wrote: > On Sunday 21 February 2010 20:11:12 Alex Blewitt wrote: >> So, you either allow 'this' as referring to the lambda (to support the >> recursive case), or disallow 'this' to refer to the lambda; at which >> point, you either disallow recursive cases or use hackery forward >> references to enable recursion. > > #void(int) countdown = #(int i) { System.out.println(i); if(i>0) countdown.(i-1) } > > ...could be defined as a special function typed variable initializer (like for example array > initializer). Plain and simple. No big deal. Right, so this would be 'disallow recursive case' option. > There is an implicit "this" which is much more important than explicit "this". > > If you make "this" refer to the lambda instance (of function type) and function types are > reference types (subtypes of Object) then, to be consistent, you must treat unqualified Object > members (hashCode, equals, wait*, notify*, getClass, toString) refer to the members of lambda > instance too. OK. That doesn't sound like a problem. After all, you can do those with arrays as well, right? > Code statistics computed by some participants of this list show that you rarely > need that in anonymous inner SAMs today. Rarely, but not never. > Quite the contrary, you often need to refer to > Object members of enclosing instance. By making it necessary to qualify those references, you > open the door to subtle bugs (some discovered just by inspecting the places in JDK sources where > qualified "this" was used, see: http://mail.openjdk.java.net/pipermail/lambda-dev/2010- > February/000550.html). Agreed. But you can still refer to members of the enclosing instance using an explicit qualified this, as I've noted. > I doubt that Object methods would have any utility being invoked on lambda instances: I agree with you. But then, they don't make any sense on arrays either. Note also that regardless of what 'this' is implemented as - or even if it refers to a MethodHandle - it will still be possible to invoke those methods on the reference type. #() foo = #() { }; foo.toString(); foo.wait(); //etc. int[] bar = {1,2,3}; bar.toString(); bar.wait(); // etc > On the contrary, they still have the same utility when invoked on an enclosing object instance. > By forcing the programmer to qualify them (i.e. MyOuterClass.this.equals(...)) you have forced > her to type more for the majority of usecases. No, because I doubt that the majority of usecases will use Object-level methods on the enclosing object. And for non-Object-level methods, you won't need to do this either. Arguably, a compiler warning of a self-reference to toString/equals etc. inside a lambda could be generated to warn in those cases where it's needed. > More importantly, NO programmer is qualifying references to other non-Object members of > enclosing class (she is not a masochist is she?) when she's writing the body of a single method > of an anonymous inner SAM instance. She's thinking in terms of outer class members at that time, > so she must be very careful not to select any of the inner's Object methods by mistake. Your example of inner classes - where people are ware of these things already - fits well into the model where the lambda is the 'this' as well. The programmer will be thinking in those same terms should 'this' refer to the lambda. > They work that way, because they are classes. With known syntactic structure. And there's no (good) reason here posted yet as to why an equivalent conceptual model cannot work for lambdas, regardless of their implementation. > We can escape the metal jacket with them. What exactly, is the metal jacket? I do not understand what kind of restrictions there are, or what cannot be done. Please enlighten me, because if there's something obvious I'm missing ... well, I'm missing it. > I think I showed that "this" referring to lambda instance has a miniscule usage compared to the > usage of implicit unqualified "this" (references to Object methods of enclosing instance). > Tackling the problems of explicit "this" and implicit "this" (unqualified Object methods) > separately - i.e. making "this" refer to lambda instance but unqualified Object methods refer to > enclosing class - would be a big mistake. It would simply be inconsistent. It is just as arguable that using 'this' to refer to something other than its enclosing instance is inconsistent. The difference here is what the enclosing instance means. For a number of programmers used to inner classes, they may consider a lambda to work in the same way. But there is inconsistency in only permitting recursive lambdas by modifying the language structure such that in certain places, the LHS of an element can be defined with an assignment to a RHS which refers to the LHS. Alex From alex.blewitt at gmail.com Sun Feb 21 14:30:02 2010 From: alex.blewitt at gmail.com (Alex Blewitt) Date: Sun, 21 Feb 2010 22:30:02 +0000 Subject: this should not refer to the lambda In-Reply-To: <4B81AD0D.1070208@univ-mlv.fr> References: <4B815090.6060708@univ-mlv.fr> <9AA3F77A-0214-4296-9735-C77E00A58045@gmail.com> <4B8180CD.3090900@univ-mlv.fr> <636fd28e1002211111y54a44250v414f5ef0affce419@mail.gmail.com> <4B81AD0D.1070208@univ-mlv.fr> Message-ID: <74E5C689-8A16-4397-BA00-CDD581588E77@gmail.com> On 21 Feb 2010, at 22:00, R?mi Forax wrote: > Le 21/02/2010 20:11, Alex Blewitt a ?crit : >>> Alex, try to compile: >>> int a =a; >>> and take a look to the error message. >>> >> Test.java:2: illegal forward reference >> int a=a; >> ^ >> >> That's the point - it's an illegal reference. Yet, were 'this' used to >> describe only the enclosing object instance (and not the lambda >> instance), you wouldn't be able to do recursive lambdas. >> > > The message is not something like "can't find symbol". > The compiler knows what 'a' means. Knows, and disallows. That's why I don't believe recursive lambdas, defined in the same way, should be allowed either. If that's the case, then the decision as to what 'this' means directly affects whether recursive lambdas are possible. Alex From neal at gafter.com Sun Feb 21 14:57:35 2010 From: neal at gafter.com (Neal Gafter) Date: Sun, 21 Feb 2010 14:57:35 -0800 Subject: this should not refer to the lambda In-Reply-To: <6D725A58-6874-4366-9500-A07C6EA84837@gmail.com> References: <4B815090.6060708@univ-mlv.fr> <9AA3F77A-0214-4296-9735-C77E00A58045@gmail.com> <4B8180CD.3090900@univ-mlv.fr> <636fd28e1002211111y54a44250v414f5ef0affce419@mail.gmail.com> <15e8b9d21002211148p13fb1250w8271d7c1ec5210f4@mail.gmail.com> <636fd28e1002211229l2f56285eg573d4c9ffee97a49@mail.gmail.com> <15e8b9d21002211356t7ffe4b1cg3e1855621c795118@mail.gmail.com> <6D725A58-6874-4366-9500-A07C6EA84837@gmail.com> Message-ID: <15e8b9d21002211457m415653d8ie3bbdf60b29e7335@mail.gmail.com> On Sun, Feb 21, 2010 at 2:18 PM, Alex Blewitt wrote: > Whether it's easy to implement is somewhat orthogonal to whether a language > should be able to define a variable assignment with a LHS that refers to > itself in the RHS. So, as noted, it appears to come to: > * 'this' refers to the lambda, or > * 'this' refers to enclosing object, and > ??* recursive lambdas are disallowed, or > ??* we break the language model such that special-cases of LHS assignments > can refer to the LHS in the RHS I don't know what you mean by "break the language model". The task at hand is to define the language. The observation that this is different than existing constructs doesn't necessarily make it bad. You correctly frame the question as to whether the language "should" allow this. We have technical reasons in its favor, and philosophical reasons against. > If the point boils down to 'it should be possible to surround any code with > a lambda block and execution', [and the original code] are the same thing > and can be done with no other changes, then the fact is > that it's already impossible to do this. The fact that things are bad doesn't mean we should make them worse. Project lambda may not provide complete transparency, but that doesn't undermine the value of transparency in the construct where project lambda has a choice of making an aspect transparent or not. Your example of nonfinal variable capture failing to be transparent is why the issue of access to nonfinal variables continues to be an issue for the spec. Cheers, Neal From alex.blewitt at gmail.com Sun Feb 21 15:25:30 2010 From: alex.blewitt at gmail.com (Alex Blewitt) Date: Sun, 21 Feb 2010 23:25:30 +0000 Subject: this should not refer to the lambda In-Reply-To: <15e8b9d21002211457m415653d8ie3bbdf60b29e7335@mail.gmail.com> References: <4B815090.6060708@univ-mlv.fr> <9AA3F77A-0214-4296-9735-C77E00A58045@gmail.com> <4B8180CD.3090900@univ-mlv.fr> <636fd28e1002211111y54a44250v414f5ef0affce419@mail.gmail.com> <15e8b9d21002211148p13fb1250w8271d7c1ec5210f4@mail.gmail.com> <636fd28e1002211229l2f56285eg573d4c9ffee97a49@mail.gmail.com> <15e8b9d21002211356t7ffe4b1cg3e1855621c795118@mail.gmail.com> <6D725A58-6874-4366-9500-A07C6EA84837@gmail.com> <15e8b9d21002211457m415653d8ie3bbdf60b29e7335@mail.gmail.com> Message-ID: <7F45FFCD-EA40-4DAD-860C-656B98833DDC@gmail.com> On 21 Feb 2010, at 22:57, Neal Gafter wrote: > On Sun, Feb 21, 2010 at 2:18 PM, Alex Blewitt wrote: >> Whether it's easy to implement is somewhat orthogonal to whether a language >> should be able to define a variable assignment with a LHS that refers to >> itself in the RHS. So, as noted, it appears to come to: >> * 'this' refers to the lambda, or >> * 'this' refers to enclosing object, and >> * recursive lambdas are disallowed, or >> * we break the language model such that special-cases of LHS assignments >> can refer to the LHS in the RHS > > I don't know what you mean by "break the language model". I don't know any construct in Java - or for that matter, other languages - where the RHS of an assignment expression can refer to the lHS of the assignment expression, as you proposed in http://mail.openjdk.java.net/pipermail/lambda-dev/2010-February/000833.html #int(int) fac = #(int x)(x <= 1 ? 1 : x * fac.(x-1)); Putting aside the types/contents of the expression, we basically have: ... fac = ... fac ... I have a philosophical reason against allowing that construct. So, if we disallow this, we disallow recursive lambdas (assuming this=enclosing object, instead of enclosing lambda). Hence the point 'recursive lambdas are disallowed, or ...' above. > The task at hand is to define the language. The observation that this is > different than existing constructs doesn't necessarily make it bad. That's true, but I wasn't saying that it was different ergo it was bad. I was saying that we should not allow any expression to be defined in terms of the LHS that it is being assigned to. It would involve fairly hairy changes to the 'Simple assignment operator' section in the JLS. http://java.sun.com/docs/books/jls/second_edition/html/expressions.doc.html#5295 For example, would you be able to assign a value in a chain? ... foo = ... fac = #() { ... fac } ... fac = ... foo = #() { ... fac } Arguably these are unlikely to come up, but the language should be able to defend such uses under the JLS, regardless of implementation chosen. >> If the point boils down to 'it should be possible to surround any code with >> a lambda block and execution', [and the original code] are the same thing >> and can be done with no other changes, then the fact is >> that it's already impossible to do this. > > The fact that things are bad doesn't mean we should make them worse. > Project lambda may not provide complete transparency, but that doesn't > undermine the value of transparency in the construct where project > lambda has a choice of making an aspect transparent or not. As I understand it (and I may be wrong; please correct me if so), the 'transparency' argument requires: * Treating 'this' as the enclosing object, not the enclosing lambda * Having 'return' exit the enclosing method, and not the lambda * Having 'break/continue' operate in the enclosing method scope, and not the lambda * Allowing mutable access to anything in scope, not just (effectively) final variables My understanding of the latest spec is that all bar the first have been decided against as it stands in the current draft. > Your example of nonfinal variable capture failing to be transparent is why > the issue of access to nonfinal variables continues to be an issue for the spec. Yes, and this is why we are still discussing; though as someone wiser than myself has already noted, we are just providing input for those at Oracle who will make the ultimate call. What is certain is that at least some people will be disappointed :-) If we end up in a situation where the transparency opportunities have already been blown by virtue of the break/continue/return/mutable access, then it seems that blowing transparency via 'this' does not add additional disadvantages to the proposal as it stands. On the other hand, if mutable access and control flow (including returns) are permitted as per the transparency argument, then I would concur that for consistency's sake, it would make sense for 'this' to act transparently as well. I suspect others would agree that there's limited benefit in only providing partial transparency; if that's the case, then none and some are equivalent. (Others may disagree, though.) Alex From neal at gafter.com Sun Feb 21 15:49:33 2010 From: neal at gafter.com (Neal Gafter) Date: Sun, 21 Feb 2010 15:49:33 -0800 Subject: this should not refer to the lambda In-Reply-To: <7F45FFCD-EA40-4DAD-860C-656B98833DDC@gmail.com> References: <4B815090.6060708@univ-mlv.fr> <9AA3F77A-0214-4296-9735-C77E00A58045@gmail.com> <4B8180CD.3090900@univ-mlv.fr> <636fd28e1002211111y54a44250v414f5ef0affce419@mail.gmail.com> <15e8b9d21002211148p13fb1250w8271d7c1ec5210f4@mail.gmail.com> <636fd28e1002211229l2f56285eg573d4c9ffee97a49@mail.gmail.com> <15e8b9d21002211356t7ffe4b1cg3e1855621c795118@mail.gmail.com> <6D725A58-6874-4366-9500-A07C6EA84837@gmail.com> <15e8b9d21002211457m415653d8ie3bbdf60b29e7335@mail.gmail.com> <7F45FFCD-EA40-4DAD-860C-656B98833DDC@gmail.com> Message-ID: <15e8b9d21002211549n1c1d2b06re6e2e7c12e22b30f@mail.gmail.com> On Sun, Feb 21, 2010 at 3:25 PM, Alex Blewitt wrote: >> I don't know what you mean by "break the language model". > > I don't know any construct in Java - or for that matter, other languages - where the > RHS of an assignment expression can refer to the lHS of the assignment expression, > as ?you proposed in http://mail.openjdk.java.net/pipermail/lambda-dev/2010-February/000833.html I didn't propose anything for the assignment expression. I proposed something about the initialization in a variable declaration. For examples see Haskell, or lisp's letrec, to pick two obvious candidates. Just because you don't know about it doesn't mean it's a bad idea. >> The task at hand is to define the language. ?The observation that this is >> different than existing constructs doesn't necessarily make it bad. > > That's true, but I wasn't saying that it was different ergo it was bad. I was saying that we > should not allow any expression to be defined in terms of the LHS that it is being assigned to. > It would involve fairly hairy changes to the 'Simple assignment operator' section in the JLS. No, it has nothing to do with the simple assignment operator. It would only be a special case for variable initialization. > As I understand it (and I may be wrong; please correct me if so), the 'transparency' argument requires: Transparency is a litmus test. It suggests, not requires. > If we end up in a situation where the transparency opportunities have already been > blown by virtue of the break/continue/return/mutable access, then it seems that > blowing transparency via 'this' does not add additional disadvantages to the proposal as it stands. That's not the case. The advantages of transparency accrue (or the disadvantages accrue) for each element that satisfies it (or fails to satisfy it). Cheers, Neal From alex.blewitt at gmail.com Sun Feb 21 16:15:44 2010 From: alex.blewitt at gmail.com (Alex Blewitt) Date: Mon, 22 Feb 2010 00:15:44 +0000 Subject: this should not refer to the lambda In-Reply-To: <15e8b9d21002211549n1c1d2b06re6e2e7c12e22b30f@mail.gmail.com> References: <4B815090.6060708@univ-mlv.fr> <9AA3F77A-0214-4296-9735-C77E00A58045@gmail.com> <4B8180CD.3090900@univ-mlv.fr> <636fd28e1002211111y54a44250v414f5ef0affce419@mail.gmail.com> <15e8b9d21002211148p13fb1250w8271d7c1ec5210f4@mail.gmail.com> <636fd28e1002211229l2f56285eg573d4c9ffee97a49@mail.gmail.com> <15e8b9d21002211356t7ffe4b1cg3e1855621c795118@mail.gmail.com> <6D725A58-6874-4366-9500-A07C6EA84837@gmail.com> <15e8b9d21002211457m415653d8ie3bbdf60b29e7335@mail.gmail.com> <7F45FFCD-EA40-4DAD-860C-656B98833DDC@gmail.com> <15e8b9d21002211549n1c1d2b06re6e2e7c12e22b30f@mail.gmail.com> Message-ID: On 21 Feb 2010, at 23:49, Neal Gafter wrote: > On Sun, Feb 21, 2010 at 3:25 PM, Alex Blewitt wrote: >>> I don't know what you mean by "break the language model". >> >> I don't know any construct in Java - or for that matter, other languages - where the >> RHS of an assignment expression can refer to the lHS of the assignment expression, >> as you proposed in http://mail.openjdk.java.net/pipermail/lambda-dev/2010-February/000833.html > > I didn't propose anything for the assignment expression. I proposed > something about the initialization in a variable declaration. The argument still stands for initialisation in the Java language. There are no other variable initialisation types where the RHS can refer to the LHS. So instead of the 'simple assignment', it would be 'local variable declaration statements', and 'definite assignment', which states that a variable must have a value before it is referenced. It would be a challenge (though not impossible) to weave in the requirements for allowing this to be violated for initialisation of lambdas. http://java.sun.com/docs/books/jls/second_edition/html/defAssign.doc.html#25979 I'm more familiar with ML (and Prolog) than Haskell and Lisp, but clearly they allow functions to be defined recursively. The difference is that the function definition is not strictly assignment; this example is from Wikipedia: http://en.wikipedia.org/wiki/ML_(programming_language) fun fac 0 = 1 | fac n = n * fac (n-1) Although this uses = in the syntax, it's not really assignment (you can't assign 'fac' twice, obviously). The Haskell version appears to be equivalent: http://en.wikipedia.org/wiki/Haskell_(programming_language) -- using recursion factorial 0 = 1 factorial n = n * factorial (n - 1) Here, 'factorial' is not actually being assigned twice. The fact that it's possible to write both these statements in a different language does not necessarily imply that the assignment (or initialisation) operators in Java should be able to handle the syntax of this. > For examples see Haskell, or lisp's letrec, to pick two obvious > candidates. Just because you don't know about it doesn't mean it's a > bad idea. That's true, but I do know about it in other languages; if not necessarily the ones you provided. >> As I understand it (and I may be wrong; please correct me if so), the 'transparency' argument requires: > > Transparency is a litmus test. It suggests, not requires. A litmus test results in a binary result. It detects the acidic or alkaline conditions. It does not suggest or require that the solution be acidic or alkaline. http://en.wikipedia.org/wiki/Litmus_test > That's not the case. The advantages of transparency accrue (or the > disadvantages accrue) for each element that satisfies it (or fails to > satisfy it). For some, the advantages and disadvantages are reversed. Alex From peter.levart at gmail.com Sun Feb 21 16:33:39 2010 From: peter.levart at gmail.com (Peter Levart) Date: Mon, 22 Feb 2010 01:33:39 +0100 Subject: this should not refer to the lambda In-Reply-To: References: <4B815090.6060708@univ-mlv.fr> <201002212250.36999.peter.levart@gmail.com> Message-ID: <201002220133.39735.peter.levart@gmail.com> On Sunday 21 February 2010 23:28:35 Alex Blewitt wrote: > On 21 Feb 2010, at 21:50, Peter Levart wrote: > > #void(int) countdown = #(int i) { System.out.println(i); if(i>0) > > countdown.(i-1) } > > > > ...could be defined as a special function typed variable initializer > > (like for example array initializer). Plain and simple. No big deal. > > Right, so this would be 'disallow recursive case' option. No, this would allow it - I was talking about the simple solution (not tackling the problems of a more general solution that John Rose proposed where the right side could contain an arbitrary function typed expression) - the array initializer was presented as an example of a syntax which is contextually limited. > > > There is an implicit "this" which is much more important than explicit > > "this". > > > > If you make "this" refer to the lambda instance (of function type) and > > function types are reference types (subtypes of Object) then, to be > > consistent, you must treat unqualified Object members (hashCode, equals, > > wait*, notify*, getClass, toString) refer to the members of lambda > > instance too. > > OK. That doesn't sound like a problem. After all, you can do those with > arrays as well, right? I don't understand you. You can do what with arrays? Did you mean array initializer? "this" as well as unqualified Object method invocations inside array initializer refer to enclosing class instance - not to array instance... > > > Code statistics computed by some participants of this list show that you > > rarely need that in anonymous inner SAMs today. > > Rarely, but not never. > > > Quite the contrary, you often need to refer to > > Object members of enclosing instance. By making it necessary to qualify > > those references, you open the door to subtle bugs (some discovered just > > by inspecting the places in JDK sources where qualified "this" was used, > > see: http://mail.openjdk.java.net/pipermail/lambda-dev/2010- > > February/000550.html). > > Agreed. But you can still refer to members of the enclosing instance using > an explicit qualified this, as I've noted. You can, but that's not the point. I was trying to present how often you would need to refer to unqualified "this" representing lambda instance compared to how often you would need to refer to unqualified Object methods representing methods of the enclosing class. Then I asserted that these two "desires" are mutually exclusive. One of them has to be dropped. They both have a workaround. > > > I doubt that Object methods would have any utility being invoked on > > lambda instances: > > I agree with you. But then, they don't make any sense on arrays either. They make more sense on arrays. But who's talking of arrays anyway? > > Note also that regardless of what 'this' is implemented as - or even if it > refers to a MethodHandle - it will still be possible to invoke those > methods on the reference type. > > #() foo = #() { }; > > foo.toString(); > foo.wait(); //etc. > > int[] bar = {1,2,3}; > bar.toString(); > bar.wait(); // etc Yes, but you missed the point again. I was trying to present that implicit "this" inside lambda has no sense/utility as invoking Object methods on lambda instance is fruitless. I was not comparing it to invoking Object methods on arrays. I was comparing it to invoking Object methods on enclosing class which has more sense/utility. > > > On the contrary, they still have the same utility when invoked on an > > enclosing object instance. By forcing the programmer to qualify them > > (i.e. MyOuterClass.this.equals(...)) you have forced her to type more for > > the majority of usecases. > > No, because I doubt that the majority of usecases will use Object-level > methods on the enclosing object. Compared to usecases of explicit "this" refering to lambda instance? I think yes. Remember? We're comparing just those two mutually exclusive features. > And for non-Object-level methods, you > won't need to do this either. You won't and you don't. What's the point? > Arguably, a compiler warning of a > self-reference to toString/equals etc. inside a lambda could be generated > to warn in those cases where it's needed. I don't think a warning is a solution for a badly designed feature. > > > More importantly, NO programmer is qualifying references to other > > non-Object members of enclosing class (she is not a masochist is she?) > > when she's writing the body of a single method of an anonymous inner SAM > > instance. She's thinking in terms of outer class members at that time, so > > she must be very careful not to select any of the inner's Object methods > > by mistake. > > Your example of inner classes - where people are ware of these things > already - fits well into the model where the lambda is the 'this' as well. > The programmer will be thinking in those same terms should 'this' refer to > the lambda. Yes, and there's the catch. She'll be thinking in those terms, and forget about Object methods being in scope and that refer to lambda's methods. > > > They work that way, because they are classes. With known syntactic > > structure. > > And there's no (good) reason here posted yet as to why an equivalent > conceptual model cannot work for lambdas, regardless of their > implementation. There are some good reasons I presented here. You just have to see them. > > We can escape the metal jacket with them. > > What exactly, is the metal jacket? http://en.wikipedia.org/wiki/Full_metal_jacket_bullet The shape of classes is defined by their jacket - it can not be reformed. Lambdas can have their own form - they are like hot lead at the moment - why are you trying to pour them into class' jacket? > I do not understand what kind of > restrictions there are, or what cannot be done. Please enlighten me, > because if there's something obvious I'm missing ... well, I'm missing it. It's simple. Unqualified java.lang.Object method invocations must refer to the same instance as unqualified "this". Not following this rule would render Java inconsistent. The desire to have "this" refer to lambda instance conflicts with the desire to invoke (unqualified) Object methods on enclosing class. We have to weight which of the two is more important. I think that statistics presented so far show that invoking Object methods on enclosing class is more frequent that referring to explicit "this" in anonymous inner SAM instances. Which one to drop in lambdas? Both have a workaround. > > > I think I showed that "this" referring to lambda instance has a miniscule > > usage compared to the usage of implicit unqualified "this" (references to > > Object methods of enclosing instance). Tackling the problems of explicit > > "this" and implicit "this" (unqualified Object methods) separately - i.e. > > making "this" refer to lambda instance but unqualified Object methods > > refer to enclosing class - would be a big mistake. It would simply be > > inconsistent. > > It is just as arguable that using 'this' to refer to something other than > its enclosing instance is inconsistent. The difference here is what the > enclosing instance means. For a number of programmers used to inner > classes, they may consider a lambda to work in the same way. ...until they are told that this is not so. They might be confused at first. They will have to learn new rules (which are actually simpler). They will find that these rules make sense. And they will like those rules. And finally they will applaud. > But there is > inconsistency in only permitting recursive lambdas by modifying the > language structure such that in certain places, the LHS of an element can > be defined with an assignment to a RHS which refers to the LHS. This is positive inconsistency. An exception which allows something that is generally not allowed. Positive in sense that you are positively surprised when you discover it. Much better than negative inconsistency: an exception which prohibits something that is generally allowed (arrays of function types?). But the worst is irregular inconsistency: the same thing that has different meaning in two different places: synchronized (this) { while (waiting) wait(); } Regardless of what "this" refers to, the above idiom should work (not only compile) everywhere! > > Alex > Regards, Peter From markmahieu at googlemail.com Sun Feb 21 16:53:06 2010 From: markmahieu at googlemail.com (Mark Mahieu) Date: Mon, 22 Feb 2010 00:53:06 +0000 Subject: this should not refer to the lambda In-Reply-To: References: <4B815090.6060708@univ-mlv.fr> <9AA3F77A-0214-4296-9735-C77E00A58045@gmail.com> <4B8180CD.3090900@univ-mlv.fr> <636fd28e1002211111y54a44250v414f5ef0affce419@mail.gmail.com> <15e8b9d21002211148p13fb1250w8271d7c1ec5210f4@mail.gmail.com> <636fd28e1002211229l2f56285eg573d4c9ffee97a49@mail.gmail.com> <15e8b9d21002211356t7ffe4b1cg3e1855621c795118@mail.gmail.com> <6D725A58-6874-4366-9500-A07C6EA84837@gmail.com> <15e8b9d21002211457m415653d8ie3bbdf60b29e7335@mail.gmail.com> <7F45FFCD-EA40-4DAD-860C-656B98833DDC@gmail.com> <15e8b9d21002211549n1c1d2b06re6e2e7c12e22b30f@mail.gmail.com> Message-ID: On 22 Feb 2010, at 00:15, Alex Blewitt wrote: > On 21 Feb 2010, at 23:49, Neal Gafter wrote: > >> On Sun, Feb 21, 2010 at 3:25 PM, Alex Blewitt wrote: >>>> I don't know what you mean by "break the language model". >>> >>> I don't know any construct in Java - or for that matter, other languages - where the >>> RHS of an assignment expression can refer to the lHS of the assignment expression, >>> as you proposed in http://mail.openjdk.java.net/pipermail/lambda-dev/2010-February/000833.html >> >> I didn't propose anything for the assignment expression. I proposed >> something about the initialization in a variable declaration. > > The argument still stands for initialisation in the Java language. There are no other variable initialisation types where the RHS can refer to the LHS. class Foo { Runnable r = new Runnable() { public void run() { r.run(); } }; } That's an example in which it's fine for the RHS to refer to the LHS. The same may well apply for a local variable declaration where the RHS is a lambda expression (depending on how they're defined). Regards, Mark From alex.blewitt at gmail.com Sun Feb 21 17:06:46 2010 From: alex.blewitt at gmail.com (Alex Blewitt) Date: Mon, 22 Feb 2010 01:06:46 +0000 Subject: this should not refer to the lambda In-Reply-To: References: <4B815090.6060708@univ-mlv.fr> <9AA3F77A-0214-4296-9735-C77E00A58045@gmail.com> <4B8180CD.3090900@univ-mlv.fr> <636fd28e1002211111y54a44250v414f5ef0affce419@mail.gmail.com> <15e8b9d21002211148p13fb1250w8271d7c1ec5210f4@mail.gmail.com> <636fd28e1002211229l2f56285eg573d4c9ffee97a49@mail.gmail.com> <15e8b9d21002211356t7ffe4b1cg3e1855621c795118@mail.gmail.com> <6D725A58-6874-4366-9500-A07C6EA84837@gmail.com> <15e8b9d21002211457m415653d8ie3bbdf60b29e7335@mail.gmail.com> <7F45FFCD-EA40-4DAD-860C-656B98833DDC@gmail.com> <15e8b9d21002211549n1c1d2b06re6e2e7c12e22b30f@mail.gmail.com> Message-ID: On 22 Feb 2010, at 00:53, Mark Mahieu wrote: > class Foo { > Runnable r = new Runnable() { > public void run() { > r.run(); > } > }; > } Thanks for both Mark and Vladimir for correcting me. I note though the example above works because the field (r.run()) is actually a field reference to the instance of the enclosing class; and the field of the class is known and defined (statically) whilst the instance is bound by a dynamic lookup at runtime. If it's converted inside an instance method, then the code fails to compile: class Foo { void bar() { final Runnable r = new Runnable() { public void run() { r.run(); } }; } } Foo.java:5: variable r might not have been initialized r.run(); ^ 1 error However, I do accept that it is possible to define initialisers in Java which are self-referential; so I withdraw my earlier objections to the issue of recursive lambdas at that ground. Alex From neal at gafter.com Sun Feb 21 22:25:31 2010 From: neal at gafter.com (Neal Gafter) Date: Sun, 21 Feb 2010 22:25:31 -0800 Subject: MethodHandle vs function types Message-ID: <15e8b9d21002212225y700af772jd71bfe2a018ef598@mail.gmail.com> When function types are represented as generic interfaces, as in BGGA, the generated implementation must cast each reference argument from Object (the type of the argument in the function's erased type) to the declared type of the lambda argument. For example, in the lambda expression #(Integer i) (i) the bytecode for the body of this lambda must do something like this #(Object _i) { Integer i = (Integer)_i; // copy the erased parameter to a variable with the proper type return i; } Similarly, values received from a lambda invocation must be cast by the caller to the return type of the function. For example, this #Integer() function = ...; Integer x = function.(); would be translated in the generated bytecode to something like this #Object() function = ...; Integer x = (Integer)function.(); These translations are described in the JLS and were implemented in JDK5 as part of the "erasure" scheme for generics. Some people are displeased about these casts (and their performance impact) remaining in the code, and would like the implementation of project lambda to avoid them. One approach that has been suggested is erasing function types to MethodHandle instead of interfaces. Unfortunately, MethodHandle does not implement the subtyping scheme for values of function type. That necessitates one of three approaches. Either (1) code is used to implement the subtype conversion, or (2) the same erasure scheme that would have been used with interfaces be used with MethodHandle (the method handle uses Object in place of all reference types), or (3) the MethodHandle primitive be relaxed to support the function subtying scheme. Approach (1) doesn't work for various reasons. As we've already seen by the failure of Howard Lovatt's attempt to reify function types, we must preserve reference identity for all widening reference conversions. Approach (2) retains the very disadvantage that MethodHandle was meant to solve. Of course, MethodHandle may come with other advantages that make it still worth using. Approach (3) is outside the scope of project lambda to address (it is an issue for jsr292), but I think it would be a very good idea for us to encourage them to work on it. Cheers, Neal From fweimer at bfk.de Mon Feb 22 01:33:22 2010 From: fweimer at bfk.de (Florian Weimer) Date: Mon, 22 Feb 2010 09:33:22 +0000 Subject: this should not refer to the lambda In-Reply-To: <7F45FFCD-EA40-4DAD-860C-656B98833DDC@gmail.com> (Alex Blewitt's message of "Sun\, 21 Feb 2010 23\:25\:30 +0000") References: <4B815090.6060708@univ-mlv.fr> <9AA3F77A-0214-4296-9735-C77E00A58045@gmail.com> <4B8180CD.3090900@univ-mlv.fr> <636fd28e1002211111y54a44250v414f5ef0affce419@mail.gmail.com> <15e8b9d21002211148p13fb1250w8271d7c1ec5210f4@mail.gmail.com> <636fd28e1002211229l2f56285eg573d4c9ffee97a49@mail.gmail.com> <15e8b9d21002211356t7ffe4b1cg3e1855621c795118@mail.gmail.com> <6D725A58-6874-4366-9500-A07C6EA84837@gmail.com> <15e8b9d21002211457m415653d8ie3bbdf60b29e7335@mail.gmail.com> <7F45FFCD-EA40-4DAD-860C-656B98833DDC@gmail.com> Message-ID: <82d3zxaa3h.fsf@mid.bfk.de> * Alex Blewitt: > I don't know any construct in Java - or for that matter, other languages - where the RHS of an assignment expression can refer to the lHS of the assignment expression, as you proposed in http://mail.openjdk.java.net/pipermail/lambda-dev/2010-February/000833.html C and C++ support this, as do Haskell, Objective Caml and Standard ML. (This is about expressions in initializers, of course.) For Ada, special syntax has been suggested to refer to the LHS from the RHS in assignment statements, to achieve the brevity of operators such as += without the (perceived) lack of clarity. -- Florian Weimer BFK edv-consulting GmbH http://www.bfk.de/ Kriegsstra?e 100 tel: +49-721-96201-1 D-76133 Karlsruhe fax: +49-721-96201-99 From John.Rose at Sun.COM Mon Feb 22 01:54:35 2010 From: John.Rose at Sun.COM (John Rose) Date: Mon, 22 Feb 2010 01:54:35 -0800 Subject: MethodHandle vs function types In-Reply-To: <15e8b9d21002212225y700af772jd71bfe2a018ef598@mail.gmail.com> References: <15e8b9d21002212225y700af772jd71bfe2a018ef598@mail.gmail.com> Message-ID: <01939DAE-5609-45B4-A09A-5C6D368A675F@sun.com> On Feb 21, 2010, at 10:25 PM, Neal Gafter wrote: > Approach (3) is outside the scope of project lambda to address (it is an > issue for jsr292), but I think it would be a very good idea for us to > encourage them to work on it. As you know, the original JSR 292 EDR specifies that methods handle invocations are non-variant; the signatures of caller and callee must agree exactly. Thus, a (String)->Object method handle cannot be invoked as (Object)->String, nor vice versa. Based on comments from several people (including you?thanks!) the committee has agreed to provide a method handle invocation called "invokeGeneric" that provides on-the-fly reference conversions and boxing/unboxing conversions (N.B. but not primitive conversions). The original "invoke" is now called "invokeExact", and never provides implicit conversions; it is useful for tuned library code, but unforgiving of function type shifts. The names of these can be adjusted one more time if there are strong reasons. I am now working on the OpenJDK reference implementation of invokeGeneric and related functionality. The conversions provided by invokeGeneric are not enough to implement every conceivable "natural" function type conversion (as derived from Java's method calling conversions via substitution experiments). But they are enough (I hope) to provide infrastructure for a useful system of implicit and explicit function type conversions. As with generics, there may need to be occasional unchecked operations. But I think a static type system can be devised which will sit on top of method handle types and which can guarantee (for many kinds of useful code, and in the absence of unchecked operations) that an invokeGeneric will not throw CCE on any argument or return value. Of course, arrays are (as always) a special case, with their special typing rules and special bytecodes which no one wants to revisit. The 292 EG has seen no need (to date) to specify a mechanism for defining arrays of strongly typed method handles. We probably cannot offer any help with integrating new types (of lambdas or methods handles or both) with arrays. To me, extended array types would make more sense as a JVM support mechanism specified by Project Lambda and/or Project Coin. It seems remotely possible that a scheme for extension methods, with modest JVM support, could integrate Object[] and List enough to provide hybrid lists with array behavior, or hybrid arrays with list methods, which could then be programmed to perform the expected function type checks. At present it's just speculation. Throws are also a special case. As with normal methods, exception declarations do not play a role in method handle linkage, and java.dyn.MethodType does not have a "throws" property, only "returnType" and "parameterTypes". This will need some fudging since any reasonable static account of function types will have to check throws as well as returns. Your choices are to erase throws, making dynamic type checks throw-agnostic, or to reify throws in MethodTypes or a related data structure. The 292 EG is willing to consider incremental changes that would remove barriers to usability of method handles for lambdas, since it is obviously to everyone's advantage to unify these mechanisms where possible. For throws, this might mean hardwiring a purely advisory attribute into MethodType. More likely, we might provide some way to decorate method handle types with throws and other annotations. This would require modest cooperation with MethodType, something like the way java.lang.Class cooperates with generics by implementing the interface Type. For constant lambdas, JSR 292 can probably help with a new ldc type, which provides a compact way to "take the address of" any method. Of course, non-constant lambdas (closures) can be created from constant (curried) functions by partial application. Partial application is a fundamental feature of method handles; this is one of the key differentiators from java.lang.reflect.Method, which cannot be partially applied (or otherwise made into closures). The partial application API (insertArgument, bind) is in the original JSR 292 EDR, and the new ldc types are in our current plans. For SAM types (a contentious topic!) JSR 292 could perhaps help by defining library APIs (or other mechanisms?) for converting between method handles and SAM references. Rather than requiring a stereotyped bytecode adapter for every statically detected SAM conversion, a library API would (a) provide a compact way to express such conversions and (b) a location for JVM vendors to concentrate optimization work. With helpful best wishes, -- John From alex.blewitt at gmail.com Mon Feb 22 01:54:47 2010 From: alex.blewitt at gmail.com (Alex Blewitt) Date: Mon, 22 Feb 2010 09:54:47 +0000 Subject: Recursive lambdas Message-ID: <73BDC50C-B19F-4A0E-B20D-68733ED068A4@gmail.com> It has been suggested that recursive lambdas can be defined without any assumption of 'this' as follows: #int(int) fac = #(int x)(x <= 1 ? 1 : x * fac.(x-1)); There may be confusion in the case of 'fac' being overwritten in the remainder of the body: #int(int) fac = #(int x)(x <= 1 ? 1 : x * fac.(x-1)); #int(int) other = #(int x)( fac.(x) ); // Not used in this example, see below #int(int) old = fac; fac = #(int x)(0); old.()// What does (should) old.() result in? In the case of a lambda-this, the answer would be clear; it would return the factorial as before. However, ther are two ways of interpreting the execution of old.() here: * The embedded 'fac' retains its old reference from when it was defined * The embedded 'fac' is interpreted in the current scope, and so returns 0 This is a contrived example, but now let's hoist these to member variables of a class: class Foo { #int(int) fac = #(int x)(x <= 1 ? 1 : x * fac.(x-1)); #int(int) other = #(int x)( fac.(x) ); Foo() { #int(int) old = fac; fac = #(int x)(0); old.(); // What does this return? } } One might argue that it's the same result. But in this case, 'fac' can refer to one of two things: * The lambda-this * The reference to this.fac as a field in the class Under the first interpretation, the effect is the same as before - it prints the factorial correctly. However, under the second interpretation, it will print 0. The 'other' here demonstrates the problem. We have two lambdas, both with recursive calls to fac.(x) in them; however, the 'other' will always refer to the field 'fac' in the enclosing scope by reference rather than by value (will it?) Mark's example yesterday suggests that it might: class Foo { Runnable r = new Runnable() { public void run() { r.run(); // refers to Foo.this.r, and so will see changes to r } }; } One can special-case this (or treat it as an oddity that no-one will worry about) but I thought I'd bring it up in the result of Mark's post. Alex From peter.levart at marand.si Mon Feb 22 02:14:45 2010 From: peter.levart at marand.si (Peter Levart) Date: Mon, 22 Feb 2010 11:14:45 +0100 Subject: MethodHandle vs function types In-Reply-To: <15e8b9d21002212225y700af772jd71bfe2a018ef598@mail.gmail.com> References: <15e8b9d21002212225y700af772jd71bfe2a018ef598@mail.gmail.com> Message-ID: <201002221114.45680.peter.levart@marand.si> On Monday 22 February 2010 07:25:31 Neal Gafter wrote: > When function types are represented as generic interfaces, as in BGGA, the > generated implementation must cast each reference argument from Object (the > type of the argument in the function's erased type) to the declared type of > the lambda argument. For example, in the lambda expression > > #(Integer i) (i) > > the bytecode for the body of this lambda must do something like this > > #(Object _i) { > Integer i = (Integer)_i; // copy the erased parameter to a variable with > the proper type > return i; > } > > Similarly, values received from a lambda invocation must be cast by the > caller to the return type of the function. For example, this > > #Integer() function = ...; > Integer x = function.(); > > would be translated in the generated bytecode to something like this > > #Object() function = ...; > Integer x = (Integer)function.(); > > These translations are described in the JLS and were implemented in JDK5 as > part of the "erasure" scheme for generics. > > Some people are displeased about these casts (and their performance impact) > remaining in the code, and would like the implementation of project lambda > to avoid them. One approach that has been suggested is erasing function > types to MethodHandle instead of interfaces. Unfortunately, MethodHandle > does not implement the subtyping scheme for values of function type. That > necessitates one of three approaches. Either (1) code is used to implement > the subtype conversion, or (2) the same erasure scheme that would have been > used with interfaces be used with MethodHandle (the method handle uses > Object in place of all reference types), or (3) the MethodHandle primitive > be relaxed to support the function subtying scheme. > > Approach (1) doesn't work for various reasons. As we've already seen by the > failure of Howard Lovatt's attempt to reify function types, we must preserve > reference identity for all widening reference conversions. > > Approach (2) retains the very disadvantage that MethodHandle was meant to > solve. Of course, MethodHandle may come with other advantages that make it > still worth using. Yes, the problem, as I see it, is that code must invoke MethodHandle with exact types of arguments and return type which reflect the types of arguments/return type of the method that it delegates to. Those types can not always be known for every call site at compile time, so code would have to "adapt" the method handle at every call site. The question is what is the overhead of such adaptation. Considering the semantics of for example MethodHandles.convertArguments(MethodHandle, MethodType) that could do the adaptation, I imagine the overhead be at least the overhead of casting each argument and return value (as per generics approach). The other overhead that should be compared is the overhead of capturing free variables. The MethodHandle approach, I imagine, is to be using MethodHandles.insertArgument(MethodHandle, int, Object) semantics for each captured reference (primitives have to be boxed?) or a special capturing context object would have to be constructed for each lambda which negates the purpose of MethodHandles not having to generate specialized classes for each lambda. One could argue that lambda construction overhead is not so important as each lambda is potentially executed multiple times, but I imagine that capturing many values via "inserArgument" constructs a "chain" of method handles which at invocation time delegate to each other. I may be wrong but is this delegation without overhead? > > Approach (3) is outside the scope of project lambda to address (it is an > issue for jsr292), but I think it would be a very good idea for us to > encourage them to work on it. I agree and I think they are already very busy tackling those problems. > > Cheers, > Neal > > Regards, Peter From Alex.Buckley at Sun.COM Mon Feb 22 02:20:11 2010 From: Alex.Buckley at Sun.COM (Alex Buckley) Date: Mon, 22 Feb 2010 02:20:11 -0800 Subject: Recursive lambdas In-Reply-To: <73BDC50C-B19F-4A0E-B20D-68733ED068A4@gmail.com> References: <73BDC50C-B19F-4A0E-B20D-68733ED068A4@gmail.com> Message-ID: <4B825A5B.4020102@sun.com> The ability for a lambda expression to reference itself during initialization is noted in the 0.1.5 spec draft, but not actually specified. However, the answer is clear: the value of a variable of function type is a reference. fac.(...) makes reference to the lambda instance resulting from evaluation of #(int x)(x<=1...). This holds whether fac is a local variable or an instance variable. As always, extravagant aliasing is best avoided. Alex Alex Blewitt wrote: > It has been suggested that recursive lambdas can be defined without any assumption of 'this' as follows: > > #int(int) fac = #(int x)(x <= 1 ? 1 : x * fac.(x-1)); > > There may be confusion in the case of 'fac' being overwritten in the remainder of the body: > > #int(int) fac = #(int x)(x <= 1 ? 1 : x * fac.(x-1)); > #int(int) other = #(int x)( fac.(x) ); // Not used in this example, see below > #int(int) old = fac; > fac = #(int x)(0); > old.()// > > What does (should) old.() result in? In the case of a lambda-this, the answer would be clear; it would return the factorial as before. However, ther are two ways of interpreting the execution of old.() here: > > * The embedded 'fac' retains its old reference from when it was defined > * The embedded 'fac' is interpreted in the current scope, and so returns 0 > > This is a contrived example, but now let's hoist these to member variables of a class: > > class Foo { > #int(int) fac = #(int x)(x <= 1 ? 1 : x * fac.(x-1)); > #int(int) other = #(int x)( fac.(x) ); > Foo() { > #int(int) old = fac; > fac = #(int x)(0); > old.(); // What does this return? > } > } > > One might argue that it's the same result. But in this case, 'fac' can refer to one of two things: > > * The lambda-this > * The reference to this.fac as a field in the class > > Under the first interpretation, the effect is the same as before - it prints the factorial correctly. However, under the second interpretation, it will print 0. > > The 'other' here demonstrates the problem. We have two lambdas, both with recursive calls to fac.(x) in them; however, the 'other' will always refer to the field 'fac' in the enclosing scope by reference rather than by value (will it?) Mark's example yesterday suggests that it might: > > class Foo { > Runnable r = new Runnable() { > public void run() { > r.run(); // refers to Foo.this.r, and so will see changes to r > } > }; > } > > One can special-case this (or treat it as an oddity that no-one will worry about) but I thought I'd bring it up in the result of Mark's post. > > Alex > From alex.blewitt at gmail.com Mon Feb 22 02:42:29 2010 From: alex.blewitt at gmail.com (Alex Blewitt) Date: Mon, 22 Feb 2010 10:42:29 +0000 Subject: Recursive lambdas In-Reply-To: <4B825A5B.4020102@sun.com> References: <73BDC50C-B19F-4A0E-B20D-68733ED068A4@gmail.com> <4B825A5B.4020102@sun.com> Message-ID: On 22 Feb 2010, at 10:20, Alex Buckley wrote: > The ability for a lambda expression to reference itself during > initialization is noted in the 0.1.5 spec draft, but not actually > specified. However, the answer is clear: the value of a variable of > function type is a reference. fac.(...) makes reference to the lambda > instance resulting from evaluation of #(int x)(x<=1...). This holds > whether fac is a local variable or an instance variable. As always, > extravagant aliasing is best avoided. That would make sense (and avoid hairy problems) but note that we now have .fac() referred to in potentially two different ways here: > class Foo { > #int(int) fac = #(int x)(x <= 1 ? 1 : x * fac.(x-1)); > #int(int) other = #(int x)( fac.(x) ); > } In the definition of 'other', how is 'fac' captured? If it's by reference to the instance variable, then it's possible that changes to fac could be observed by other. If it's by reference to the value of fac at the point of capture, then it's by (current) value of fac at the time. Note that 'effectively final' doesn't help here; even if no other code in Foo mutates fac, that doesn't prevent external code from mutating fac. If fac were defined as final, then the problem goes away; but I can see this pattern being used functionally in Java code: class Op { #bool(Object) filter = #(Object o) { return true }; #Object[](Object[]) subset = #(Object[] o) { r = new Object[o.length]; for(int i=0;i To summarize, there is three ways to have recursive lambda: 1) No recursive lambda: If you need a recursive function, create it has a private method and call it in the lambda (this is my favorite choice :) static int sum(int x) { return (x==0)?0:x + sum(x-1); } #int(int) a= #int(int x) { return sum(x); }; 2) this refers to the current lambda In that case you can use this.() to do the recursive call. #int(int) a= #int(int x) { return (x==0)?0:x + this.(x-1); }; 3) allow a reference to a lambda to be used inside a lambda #int(int) a= #int(int x) { return (x==0)?0:x + a.(x-1); }; How to implement solution 2 or 3 with method handles ? First, it's not simple because a method handle is an object that contains a function pointer to a method, so there is no dedicated class. You have to first create the method handle and then inject itself to be available in the body of the function referenced. Because injecting the method handle creates a new method handle so you have a eggs and chicken problem that you can solve with an array. static int $lambda$1(MethodHandle[] mh, int x) { return (x==0)?0:x + mh[0].invokeGenerics(x-1); } MethodHandle mh = #$lambda$1(MethodHandle, int) MethodHandle[] array = {null}; mh = mh.bind(array); array[0] = mh; Rather ugly, isn't it ? cheers, R?mi From scolebourne at joda.org Mon Feb 22 05:16:22 2010 From: scolebourne at joda.org (Stephen Colebourne) Date: Mon, 22 Feb 2010 13:16:22 +0000 Subject: Lightweight interfaces instead of function types In-Reply-To: <15e8b9d21002211008x5d50fc92u40d5619966e7fd38@mail.gmail.com> References: <4b4f45e01002190928x5230dfeek2c1490dedccc71fd@mail.gmail.com> <15e8b9d21002190947p4af23210o1e2e5d280144da60@mail.gmail.com> <4b4f45e01002191015w4341c46dia6a548fe4bd5730f@mail.gmail.com> <4B80D6E0.103@sun.com> <4b4f45e01002210358u1859ec59tcfdeee9084f4b90f@mail.gmail.com> <15e8b9d21002211008x5d50fc92u40d5619966e7fd38@mail.gmail.com> Message-ID: <4b4f45e01002220516s7a29fddx99d7c08400f7e23c@mail.gmail.com> On 21 February 2010 18:08, Neal Gafter wrote: > On Sun, Feb 21, 2010 at 3:58 AM, Stephen Colebourne wrote: >> More broadly, it has been noted by more than one person that >> ClassCastExceptions from ungenerified objects/collections or >> ArrayStoreExceptions have always been a rarity in real code by >> comparison to NullPointerExceptions (ie. why aren't we tackling >> NPEs???). > > I'm trying to figure out how this statement fits into the discussion.? Are > you suggesting that because dynamic type safety is hardly ever violated in > practice, there would be little value in ever reifying generics (compared > to, for example, the jsr308 work in JDK7 to help eliminate NPEs using > annotations)?? If so, that would explain your eagerness to encourage coding > patterns that would undermine it. Were we back pre JDK 1.5, I would be advocating tackling the NPE problem (and properties) before tackling the CCE problem (generics). This wouldn't have affected the addition of foreach loops: Collection coll = new ArrayList(); coll.add("Hello"); for (String str : coll) { ... } The above does not compile in JDK 1.5/1.6, but it could. It simply requires the language spec to be a little less "holier than thou". Doing so would be result in code no less safe than that previously written in JDK 1.4. (I didn't bother mentioning this in Coin, as I believe it had 0% chance of being picked). At this point I see generics as just another verbose problem to work around in Java, that frequently behaves in apparently weird ways and often makes me curse out loud. I'm afraid I have no faith that reification would make matters better. BTW, I consider JSR-305 annotations to be a nasty hack to the NPE problem. To answer your question - this affects Lambda's discussion because this project has to choose a compromise. There is no perfect solution (do we break arrays, or avoid function types, or only use SAMs, or...). I'm "helpfully" suggesting that the damage caused by CCE/ASE has been overstated, relative to NPE, which might affect which compromise gets chosen. Stephen From neal at gafter.com Mon Feb 22 07:26:18 2010 From: neal at gafter.com (Neal Gafter) Date: Mon, 22 Feb 2010 07:26:18 -0800 Subject: MethodHandle vs function types In-Reply-To: <01939DAE-5609-45B4-A09A-5C6D368A675F@sun.com> References: <15e8b9d21002212225y700af772jd71bfe2a018ef598@mail.gmail.com> <01939DAE-5609-45B4-A09A-5C6D368A675F@sun.com> Message-ID: <15e8b9d21002220726u596d4d00v1e5dfbfddedee3ea@mail.gmail.com> On Mon, Feb 22, 2010 at 1:54 AM, John Rose wrote: > Based on comments from several people (including you?thanks!) the committee > has agreed to provide a method handle invocation called "invokeGeneric" that > provides on-the-fly reference conversions and boxing/unboxing conversions > (N.B. but not primitive conversions). > Why the boxing/unboxing conversions? Those aren't part of the proposed function subtype relation or the function subtype relation in any existing language. Is this to help support language interop between Java and languages that have only the reference version of primitives? From neal at gafter.com Mon Feb 22 07:35:21 2010 From: neal at gafter.com (Neal Gafter) Date: Mon, 22 Feb 2010 07:35:21 -0800 Subject: Recursive lambdas In-Reply-To: <4B825A5B.4020102@sun.com> References: <73BDC50C-B19F-4A0E-B20D-68733ED068A4@gmail.com> <4B825A5B.4020102@sun.com> Message-ID: <15e8b9d21002220735p1c12e1f0lfcc874740b640224@mail.gmail.com> On Mon, Feb 22, 2010 at 2:20 AM, Alex Buckley wrote: > The ability for a lambda expression to reference itself during > initialization is noted in the 0.1.5 spec draft, but not actually > specified. However, the answer is clear: the value of a variable of > function type is a reference. fac.(...) makes reference to the lambda > instance resulting from evaluation of #(int x)(x<=1...). This holds > whether fac is a local variable or an instance variable. As always, > extravagant aliasing is best avoided. > The first example is illegal because the body of the lambda uses a variable that is not effectively final. But presuming it were legal (or marked with "shared" or annotated with @Shared, or whatever)... I thought that after you reassigned a variable, evaluating an expression that is a reference to the variable should reflect the new value of the variable. That is true whether or not the variable is declared inside a lambda, outside a lambda, or as a field. Your response seems to suggest some other semantics of variable assignment and variable reference. From forax at univ-mlv.fr Mon Feb 22 08:14:37 2010 From: forax at univ-mlv.fr (=?windows-1252?Q?R=E9mi_Forax?=) Date: Mon, 22 Feb 2010 17:14:37 +0100 Subject: MethodHandle vs function types In-Reply-To: <15e8b9d21002220726u596d4d00v1e5dfbfddedee3ea@mail.gmail.com> References: <15e8b9d21002212225y700af772jd71bfe2a018ef598@mail.gmail.com> <01939DAE-5609-45B4-A09A-5C6D368A675F@sun.com> <15e8b9d21002220726u596d4d00v1e5dfbfddedee3ea@mail.gmail.com> Message-ID: <4B82AD6D.4040008@univ-mlv.fr> Le 22/02/2010 16:26, Neal Gafter a ?crit : > On Mon, Feb 22, 2010 at 1:54 AM, John Rose wrote: > > >> Based on comments from several people (including you?thanks!) the committee >> has agreed to provide a method handle invocation called "invokeGeneric" that >> provides on-the-fly reference conversions and boxing/unboxing conversions >> (N.B. but not primitive conversions). >> >> > Why the boxing/unboxing conversions? Those aren't part of the proposed > function subtype relation or the function subtype relation in any existing > language. Is this to help support language interop between Java and > languages that have only the reference version of primitives? > Yes, this also allows to replace java.lang.reflect.Method invocation by MethodHandle + invokeGeneric because it defines the same semantics. R?mi From neal at gafter.com Mon Feb 22 08:26:19 2010 From: neal at gafter.com (Neal Gafter) Date: Mon, 22 Feb 2010 08:26:19 -0800 Subject: Implementing recursive lambda with MethodHandle In-Reply-To: <4B826FEF.4070605@univ-mlv.fr> References: <4B826FEF.4070605@univ-mlv.fr> Message-ID: <15e8b9d21002220826t1a666395w69f9138c62de1c32@mail.gmail.com> On Mon, Feb 22, 2010 at 3:52 AM, R?mi Forax wrote: > How to implement solution 2 or 3 with method handles ? > First, it's not simple because a method handle is an > object that contains a function pointer to a method, > so there is no dedicated class. > > You have to first create the method handle and then inject itself to be > available > in the body of the function referenced. > Because injecting the method handle creates a new method handle > so you have a eggs and chicken problem that you can solve with an array. > > Rather ugly, isn't it ? > Sure is ugly. As a practical matter, I'd expect the compiler to use a frame object rather than an array, as it would for @Shared variables, but it's still ugly. I have two suggestions to address this: (1) Don't make "this" refer to the function from within the function. That minimizes how often the compiler would do this. (2) Don't look at the generated code. ;-) Cheers, Neal From forax at univ-mlv.fr Mon Feb 22 08:31:48 2010 From: forax at univ-mlv.fr (=?windows-1252?Q?R=E9mi_Forax?=) Date: Mon, 22 Feb 2010 17:31:48 +0100 Subject: MethodHandle vs function types In-Reply-To: <01939DAE-5609-45B4-A09A-5C6D368A675F@sun.com> References: <15e8b9d21002212225y700af772jd71bfe2a018ef598@mail.gmail.com> <01939DAE-5609-45B4-A09A-5C6D368A675F@sun.com> Message-ID: <4B82B174.70400@univ-mlv.fr> Le 22/02/2010 10:54, John Rose a ?crit : [...] > For SAM types (a contentious topic!) JSR 292 could perhaps help by defining library APIs (or other mechanisms?) for converting between method handles and SAM references. Rather than requiring a stereotyped bytecode adapter for every statically detected SAM conversion, a library API would (a) provide a compact way to express such conversions and (b) a location for JVM vendors to concentrate optimization work. > > With helpful best wishes, > -- John > I really like that idea ! Instead of letting the compiler to produce bytecodes in order create an object implementing an interface from a MethodHandle, a VM can create a lightweight class with an itable containing the same pointer as the one contained in the MethodHandle. R?mi From Alex.Buckley at Sun.COM Mon Feb 22 08:34:19 2010 From: Alex.Buckley at Sun.COM (Alex Buckley) Date: Mon, 22 Feb 2010 08:34:19 -0800 Subject: Recursive lambdas In-Reply-To: <15e8b9d21002220735p1c12e1f0lfcc874740b640224@mail.gmail.com> References: <73BDC50C-B19F-4A0E-B20D-68733ED068A4@gmail.com> <4B825A5B.4020102@sun.com> <15e8b9d21002220735p1c12e1f0lfcc874740b640224@mail.gmail.com> Message-ID: <4B82B20B.8080200@sun.com> Neal Gafter wrote: > I thought that after you reassigned a variable, evaluating an expression > that is a reference to the variable should reflect the new value of the > variable. That is true whether or not the variable is declared inside a > lambda, outside a lambda, or as a field. Your response seems to suggest > some other semantics of variable assignment and variable reference. The self-reference to 'fac' could well justify a special binding for 'fac' within its initialization. The binding of 'fac' in the initialization of 'other' would work the usual way. Alex From jjb at google.com Mon Feb 22 09:49:07 2010 From: jjb at google.com (Joshua Bloch) Date: Mon, 22 Feb 2010 09:49:07 -0800 Subject: Implementing recursive lambda with MethodHandle In-Reply-To: <4B826FEF.4070605@univ-mlv.fr> References: <4B826FEF.4070605@univ-mlv.fr> Message-ID: <17b2302a1002220949h5754454u6f7fe2603219c5b@mail.gmail.com> R?mi, Sorry to sound so negative today, but I don't like any of these. They all force you to define a variable or method in the scope enclosing the lambda, cluttering up this scope. One of the things that I love about for-lops (as opposed, for example, to while loops) is that they allow you to declare index variables and bounds within the scope of the loop. I don't consider the recursive lambda case to be all that important (definitely a corner case), but I much prefer to use a local variable declared within the scope of the lambda: #i(int i) { #int(int) thisFn = this; return i == 0 ? 1 : i * thisFn.(i - 1); }; This does not require any fancy (i.e., recursive) type inference and does hot pollute the surrounding scope with an unused local variable or method declaration. Josh On Mon, Feb 22, 2010 at 3:52 AM, R?mi Forax wrote: > To summarize, there is three ways to have recursive lambda: > 1) No recursive lambda: > If you need a recursive function, create it has a private method > and call it in the lambda > (this is my favorite choice :) > > static int sum(int x) { > return (x==0)?0:x + sum(x-1); > } > #int(int) a= #int(int x) { > return sum(x); > }; > > 2) this refers to the current lambda > In that case you can use this.() to do the recursive call. > #int(int) a= #int(int x) { > return (x==0)?0:x + this.(x-1); > }; > > 3) allow a reference to a lambda to be used inside a lambda > #int(int) a= #int(int x) { > return (x==0)?0:x + a.(x-1); > }; > > How to implement solution 2 or 3 with method handles ? > First, it's not simple because a method handle is an > object that contains a function pointer to a method, > so there is no dedicated class. > > You have to first create the method handle and then inject itself to be > available > in the body of the function referenced. > Because injecting the method handle creates a new method handle > so you have a eggs and chicken problem that you can solve with an array. > > static int $lambda$1(MethodHandle[] mh, int x) { > return (x==0)?0:x + mh[0].invokeGenerics(x-1); > } > > MethodHandle mh = #$lambda$1(MethodHandle, int) > MethodHandle[] array = {null}; > mh = mh.bind(array); > array[0] = mh; > > Rather ugly, isn't it ? > cheers, > R?mi > > > From alex.blewitt at gmail.com Mon Feb 22 10:11:21 2010 From: alex.blewitt at gmail.com (Alex Blewitt) Date: Mon, 22 Feb 2010 18:11:21 +0000 Subject: Implementing recursive lambda with MethodHandle In-Reply-To: <17b2302a1002220949h5754454u6f7fe2603219c5b@mail.gmail.com> References: <4B826FEF.4070605@univ-mlv.fr> <17b2302a1002220949h5754454u6f7fe2603219c5b@mail.gmail.com> Message-ID: <39560A26-EFA0-4762-8B7D-28AE4B667B59@gmail.com> On Feb 22, 2010, at 17:49, Joshua Bloch wrote: > Sorry to sound so negative today, but I don't like any of these. > They all > force you to define a variable or method in the scope enclosing the > lambda, > cluttering up this scope. Remi wrote: > 2) this refers to the current lambda > In that case you can use this.() to do the recursive call. > #int(int) a= #int(int x) { > return (x==0)?0:x + this.(x-1); > }; If 'this' refers to the current lambda, you don't need to pollute the scope. The assignment to 'a' here has no effect; it could be used as a direct value, e.g.: doSomethingWithLambda( #int(int x) { return (x==0)?0:x+this.(x-1) ); I'm not sure why a temporary assignment to 'thisFn' is an better than using 'this' in place - on the other hand, if one wanted to, one could do both the above and the below. > #i(int i) { > #int(int) thisFn = this; > return i == 0 ? 1 : i * thisFn.(i - 1); > }; > > This does not require any fancy (i.e., recursive) type inference and > does > [not] pollute the surrounding scope with an unused local variable or > method > declaration. I personally agree that the assignment of the lambda to a variable in order to be able to use it recursively feels like clutter. It's also worth noting that the recursion isn't the only form; it could be used as a dependency injection handle as well: #(Object target,#() fn) { target.visit(this,fn); }; This could be used to e.g. implement a monad or a strategy for doing depth/breadth-first processing of a set of nodes: interface INode { INode[] children(); } headFirst = #(Node n, #(Node) process ) { process(n); for(child:n.children()) this.(child,process) } headLast = #(Node n, #(Node) process ) { for(child:n.children()) this. (child,process); process(n) } Finally, the code-search tools that Neal provided show what can be done with existing Java objects. However, in these cases, it's far more likely that they'd be captured as static methods on a named class, rather than being able to pass in (potentially recursive) functions as arguments to other processes. As well as considering how it might be used with existing code, it probably makes sense to look into how it might be used in the future as well. Alex From John.Rose at Sun.COM Mon Feb 22 10:35:32 2010 From: John.Rose at Sun.COM (John Rose) Date: Mon, 22 Feb 2010 10:35:32 -0800 Subject: Implementing recursive lambda with MethodHandle In-Reply-To: <4B826FEF.4070605@univ-mlv.fr> References: <4B826FEF.4070605@univ-mlv.fr> Message-ID: <6E101962-67EA-4834-BF6D-097E3CB4FBD6@sun.com> On Feb 22, 2010, at 3:52 AM, R?mi Forax wrote: > Rather ugly, isn't it ? Yes, but it is totally standard for such things. Self-recursion of functions requires some sort of uplevel patchable self-reference, whether via an object or an array. Lisp dialects do such things routinely for (mutually) recursive functions. I assume more modern functional languages do something similar to "close the loop". This technique, strong and ugly, scales naturally to important use case of *several* mutually recursive local functions (LETREC, LET/DEFINE, etc.). -- John From jjb at google.com Mon Feb 22 10:38:37 2010 From: jjb at google.com (Joshua Bloch) Date: Mon, 22 Feb 2010 10:38:37 -0800 Subject: Implementing recursive lambda with MethodHandle In-Reply-To: <39560A26-EFA0-4762-8B7D-28AE4B667B59@gmail.com> References: <4B826FEF.4070605@univ-mlv.fr> <17b2302a1002220949h5754454u6f7fe2603219c5b@mail.gmail.com> <39560A26-EFA0-4762-8B7D-28AE4B667B59@gmail.com> Message-ID: <17b2302a1002221038l268dff63wba991fb9c225a4a8@mail.gmail.com> Alex, On Mon, Feb 22, 2010 at 10:11 AM, Alex Blewitt wrote: > > I'm not sure why a temporary assignment to 'thisFn' is an better than using > 'this' in place - on the other hand, if one wanted to, one could do both the > above and the below. > > #i(int i) { >> #int(int) thisFn = this; >> return i == 0 ? 1 : i * thisFn.(i - 1); >> }; >> >> It eliminates the circularity in the definition of the type of the lambda: it's clearly #int(int). It's visual clutter, but at least it doesn't clutter up the enclosing namespace. Josh From forax at univ-mlv.fr Mon Feb 22 10:50:33 2010 From: forax at univ-mlv.fr (=?ISO-8859-1?Q?R=E9mi_Forax?=) Date: Mon, 22 Feb 2010 19:50:33 +0100 Subject: Implementing recursive lambda with MethodHandle In-Reply-To: <17b2302a1002221038l268dff63wba991fb9c225a4a8@mail.gmail.com> References: <4B826FEF.4070605@univ-mlv.fr> <17b2302a1002220949h5754454u6f7fe2603219c5b@mail.gmail.com> <39560A26-EFA0-4762-8B7D-28AE4B667B59@gmail.com> <17b2302a1002221038l268dff63wba991fb9c225a4a8@mail.gmail.com> Message-ID: <4B82D1F9.5040907@univ-mlv.fr> Le 22/02/2010 19:38, Joshua Bloch a ?crit : > Alex, > > On Mon, Feb 22, 2010 at 10:11 AM, Alex Blewitt > wrote: > > > I'm not sure why a temporary assignment to 'thisFn' is an better > than using 'this' in place - on the other hand, if one wanted to, > one could do both the above and the below. > > #i(int i) { > #int(int) thisFn = this; > return i == 0 ? 1 : i * thisFn.(i - 1); > }; > > > It eliminates the circularity in the definition of the type of the > lambda: it's clearly #int(int). It's visual clutter, but at least it > doesn't clutter up the enclosing namespace. > > Josh This one doesn't clutter up the enclosing scope too :) #(int i) { return new Object() { int sum(int x) { return (x==0)?0:x + sum(x-1); } }.sum(x); }; R?mi From neal at gafter.com Mon Feb 22 11:02:04 2010 From: neal at gafter.com (Neal Gafter) Date: Mon, 22 Feb 2010 11:02:04 -0800 Subject: Recursive lambdas In-Reply-To: <4B82B20B.8080200@sun.com> References: <73BDC50C-B19F-4A0E-B20D-68733ED068A4@gmail.com> <4B825A5B.4020102@sun.com> <15e8b9d21002220735p1c12e1f0lfcc874740b640224@mail.gmail.com> <4B82B20B.8080200@sun.com> Message-ID: <15e8b9d21002221102h1d49a4e3m35aab7f250bcbb70@mail.gmail.com> On Mon, Feb 22, 2010 at 8:34 AM, Alex Buckley wrote: > Neal Gafter wrote: > > I thought that after you reassigned a variable, evaluating an expression > > that is a reference to the variable should reflect the new value of the > > variable. That is true whether or not the variable is declared inside a > > lambda, outside a lambda, or as a field. Your response seems to suggest > > some other semantics of variable assignment and variable reference. > > The self-reference to 'fac' could well justify a special binding for > 'fac' within its initialization. The binding of 'fac' in the > initialization of 'other' would work the usual way. > Assuming you're not joking, this sounds like a bad idea. From jjb at google.com Mon Feb 22 11:28:21 2010 From: jjb at google.com (Joshua Bloch) Date: Mon, 22 Feb 2010 11:28:21 -0800 Subject: Implementing recursive lambda with MethodHandle In-Reply-To: <4B82D1F9.5040907@univ-mlv.fr> References: <4B826FEF.4070605@univ-mlv.fr> <17b2302a1002220949h5754454u6f7fe2603219c5b@mail.gmail.com> <39560A26-EFA0-4762-8B7D-28AE4B667B59@gmail.com> <17b2302a1002221038l268dff63wba991fb9c225a4a8@mail.gmail.com> <4B82D1F9.5040907@univ-mlv.fr> Message-ID: <17b2302a1002221128i41fa44fbp5cda06b6cc784f13@mail.gmail.com> R?mi, On Mon, Feb 22, 2010 at 10:50 AM, R?mi Forax wrote: > > This one doesn't clutter up the enclosing scope too :) > > #(int i) { > return new Object() { > int sum(int x) { > return (x==0)?0:x + sum(x-1); > } > }.sum(x); > }; > > Yes, but it uses an (ugly) anonymous class initialization expression. The whole idea of this JSR is to eliminate the need for these beasts. Josh From lk at teamten.com Mon Feb 22 11:46:26 2010 From: lk at teamten.com (Lawrence Kesteloot) Date: Mon, 22 Feb 2010 11:46:26 -0800 Subject: Implementing recursive lambda with MethodHandle In-Reply-To: <17b2302a1002221128i41fa44fbp5cda06b6cc784f13@mail.gmail.com> References: <4B826FEF.4070605@univ-mlv.fr> <17b2302a1002220949h5754454u6f7fe2603219c5b@mail.gmail.com> <39560A26-EFA0-4762-8B7D-28AE4B667B59@gmail.com> <17b2302a1002221038l268dff63wba991fb9c225a4a8@mail.gmail.com> <4B82D1F9.5040907@univ-mlv.fr> <17b2302a1002221128i41fa44fbp5cda06b6cc784f13@mail.gmail.com> Message-ID: <997cab101002221146u534a88dj5be46fab00da0aa@mail.gmail.com> On Mon, Feb 22, 2010 at 11:28 AM, Joshua Bloch wrote: > Yes, but it uses an (ugly) anonymous class initialization expression. ?The > whole idea of this JSR is to eliminate the need for these beasts. It concerns me that so much of the discussion on this list centers around the use case of the programmer writing factorial functions. Every recursive function I've ever written was too complex to fit into a lambda. They were all part of some data structure object where I had proper methods. If we forbid lambdas from being recursive I suspect it would have little real-world impact, and it would be easy to explain since they're anonymous. (Unlike the recent suggestion to make bindings to the lambda's LHS variable be different than bindings to other free variables.) I think our time is better spent discussing Doug's preference for a parallel-safe construct and whether that's the kind of lambda we want in this general-purpose language. Josh, perhaps I missed it, but I don't understand why it's so important to you that "this" refer to the function object. (Other than for a recursive call.) Can you provide a few use cases? Lawrence From neal at gafter.com Mon Feb 22 11:48:08 2010 From: neal at gafter.com (Neal Gafter) Date: Mon, 22 Feb 2010 11:48:08 -0800 Subject: Implementing recursive lambda with MethodHandle In-Reply-To: <17b2302a1002221128i41fa44fbp5cda06b6cc784f13@mail.gmail.com> References: <4B826FEF.4070605@univ-mlv.fr> <17b2302a1002220949h5754454u6f7fe2603219c5b@mail.gmail.com> <39560A26-EFA0-4762-8B7D-28AE4B667B59@gmail.com> <17b2302a1002221038l268dff63wba991fb9c225a4a8@mail.gmail.com> <4B82D1F9.5040907@univ-mlv.fr> <17b2302a1002221128i41fa44fbp5cda06b6cc784f13@mail.gmail.com> Message-ID: <15e8b9d21002221148y8382f6dlb689117738118d92@mail.gmail.com> On Mon, Feb 22, 2010 at 11:28 AM, Joshua Bloch wrote: > Yes, but it uses an (ugly) anonymous class initialization expression. The > whole idea of this JSR is to eliminate the need for these beasts. > That is definitely not the idea of this JSR. We are not "eliminating" the need for anonymous inner classes. We are providing a good alternative for a huge number of use cases. These recursive functions are a vanishingly small fraction of the identified use cases. From pbenedict at apache.org Mon Feb 22 11:57:30 2010 From: pbenedict at apache.org (Paul Benedict) Date: Mon, 22 Feb 2010 13:57:30 -0600 Subject: Implementing recursive lambda with MethodHandle In-Reply-To: <15e8b9d21002221148y8382f6dlb689117738118d92@mail.gmail.com> References: <4B826FEF.4070605@univ-mlv.fr> <17b2302a1002220949h5754454u6f7fe2603219c5b@mail.gmail.com> <39560A26-EFA0-4762-8B7D-28AE4B667B59@gmail.com> <17b2302a1002221038l268dff63wba991fb9c225a4a8@mail.gmail.com> <4B82D1F9.5040907@univ-mlv.fr> <17b2302a1002221128i41fa44fbp5cda06b6cc784f13@mail.gmail.com> <15e8b9d21002221148y8382f6dlb689117738118d92@mail.gmail.com> Message-ID: On Mon, Feb 22, 2010 at 1:48 PM, Neal Gafter wrote: > On Mon, Feb 22, 2010 at 11:28 AM, Joshua Bloch wrote: > >> Yes, but it uses an (ugly) anonymous class initialization expression. ?The >> whole idea of this JSR is to eliminate the need for these beasts. >> > > That is definitely not the idea of this JSR. ?We are not "eliminating" the > need for anonymous inner classes. ?We are providing a good alternative for a > huge number of use cases. ?These recursive functions are a vanishingly small > fraction of the identified use cases. > Neal, I think Josh was saying the JSR is to eliminate the anonymous class syntax for lambdas -- not eliminate the syntax in general. Paul From peter.levart at gmail.com Mon Feb 22 12:08:15 2010 From: peter.levart at gmail.com (Peter Levart) Date: Mon, 22 Feb 2010 21:08:15 +0100 Subject: Implementing recursive lambda with MethodHandle In-Reply-To: <4B826FEF.4070605@univ-mlv.fr> References: <4B826FEF.4070605@univ-mlv.fr> Message-ID: <201002222108.15569.peter.levart@gmail.com> On Monday 22 February 2010 12:52:15 R?mi Forax wrote: > 2) this refers to the current lambda > In that case you can use this.() to do the recursive call. > #int(int) a= #int(int x) { > return (x==0)?0:x + this.(x-1); > }; > > 3) allow a reference to a lambda to be used inside a lambda > #int(int) a= #int(int x) { > return (x==0)?0:x + a.(x-1); > }; > > How to implement solution 2 or 3 with method handles ? > First, it's not simple because a method handle is an > object that contains a function pointer to a method, > so there is no dedicated class. > > You have to first create the method handle and then inject itself to be > available > in the body of the function referenced. > Because injecting the method handle creates a new method handle > If you're just invoking the function (either this.(...) or a.(...)) the compiler could translate that to simple invocation of a method (which is the target of the method handle). If you're passing the value to someone, then that's a different story... Peter From jjb at google.com Mon Feb 22 12:19:45 2010 From: jjb at google.com (Joshua Bloch) Date: Mon, 22 Feb 2010 12:19:45 -0800 Subject: Implementing recursive lambda with MethodHandle In-Reply-To: References: <4B826FEF.4070605@univ-mlv.fr> <17b2302a1002220949h5754454u6f7fe2603219c5b@mail.gmail.com> <39560A26-EFA0-4762-8B7D-28AE4B667B59@gmail.com> <17b2302a1002221038l268dff63wba991fb9c225a4a8@mail.gmail.com> <4B82D1F9.5040907@univ-mlv.fr> <17b2302a1002221128i41fa44fbp5cda06b6cc784f13@mail.gmail.com> <15e8b9d21002221148y8382f6dlb689117738118d92@mail.gmail.com> Message-ID: <17b2302a1002221219v1678540dw2a669f8dd2ebe734@mail.gmail.com> Paul, On Mon, Feb 22, 2010 at 11:57 AM, Paul Benedict wrote: > On Mon, Feb 22, 2010 at 1:48 PM, Neal Gafter wrote: > > On Mon, Feb 22, 2010 at 11:28 AM, Joshua Bloch wrote: > > > >> Yes, but it uses an (ugly) anonymous class initialization expression. > The > >> whole idea of this JSR is to eliminate the need for these beasts. > >> > > > > That is definitely not the idea of this JSR. We are not "eliminating" > the > > need for anonymous inner classes. We are providing a good alternative > for a > > huge number of use cases. These recursive functions are a vanishingly > small > > fraction of the identified use cases. > > > > Neal, I think Josh was saying the JSR is to eliminate the anonymous > class syntax for lambdas -- not eliminate the syntax in general. Exactly so. Josh From jjb at google.com Mon Feb 22 12:32:02 2010 From: jjb at google.com (Joshua Bloch) Date: Mon, 22 Feb 2010 12:32:02 -0800 Subject: Implementing recursive lambda with MethodHandle In-Reply-To: <997cab101002221146u534a88dj5be46fab00da0aa@mail.gmail.com> References: <4B826FEF.4070605@univ-mlv.fr> <17b2302a1002220949h5754454u6f7fe2603219c5b@mail.gmail.com> <39560A26-EFA0-4762-8B7D-28AE4B667B59@gmail.com> <17b2302a1002221038l268dff63wba991fb9c225a4a8@mail.gmail.com> <4B82D1F9.5040907@univ-mlv.fr> <17b2302a1002221128i41fa44fbp5cda06b6cc784f13@mail.gmail.com> <997cab101002221146u534a88dj5be46fab00da0aa@mail.gmail.com> Message-ID: <17b2302a1002221232r63beb12s5fe70709cc400bc8@mail.gmail.com> Lawrence, On Mon, Feb 22, 2010 at 11:46 AM, Lawrence Kesteloot wrote: > On Mon, Feb 22, 2010 at 11:28 AM, Joshua Bloch wrote: > > Yes, but it uses an (ugly) anonymous class initialization expression. > The > > whole idea of this JSR is to eliminate the need for these beasts. > > It concerns me that so much of the discussion on this list centers > around the use case of the programmer writing factorial functions. > Every recursive function I've ever written was too complex to fit into > a lambda. I've written plenty of shorter ones, but I do believe this (recursive lambdas) is a corner case. > Josh, perhaps I missed it, but I don't understand why it's so > important to you that "this" refer to the function object. (Other than > for a recursive call.) Can you provide a few use cases? As I've said a few times, I believe it's the right thing to do from a consistency standpoint. Also it's strictly more powerful than having an unqualified "this" refer to the enclosing instance (if any): you can always get access to the enclosing instance using a qualified this. As for use cases, I expect that a lambda might register itself or deregister itself for some sort of callback (e.g., change notification). Josh From jjb at google.com Mon Feb 22 12:38:17 2010 From: jjb at google.com (Joshua Bloch) Date: Mon, 22 Feb 2010 12:38:17 -0800 Subject: Nice to @Share? Message-ID: <17b2302a1002221238p2db47f2albcd2e6bb23ddafab@mail.gmail.com> Some say it's nice to @Share, but not me. Draft 0.1.5 of the Project Lambda spec contains some discussion of an @Shared annotation to "allow unqualified references to upplevel variables (all sorts) if they are marked @Shared." There are several reasons that I believe this not a good idea: (1) For fields (instance or static) this annotation is not required today (in anonymouns class instance creation expressions), and I believe that programmers are likely to see it as an added annoyance (like the current situation with final variables). (2) For local variables, I see it as too much magic: @Shared int i; has a totally different implementation from: int i; The latter creates a "container object" on the heap, the former doesn't. The semantics (in particular, the lifetime) and the performance are likely to differ significantly. (3) I believe it's an abuse of annotations. Some may liken it to @Override, but I believe this is specious. The @Override annotation merely expresses a compile-time assertion that the compiler is able to verify. The proposed @Shared annotation changes the semantics of the language. (4) If, in spite of all the above, it is the consesnsus that such an annotation is still a good idea, I believe the name @Shared could hardly be more misleading. In practical terms, it means "this variable must *not* be shared (without additional synchronization)." It would typically be used when the programmer could assert that the variable in question would never be sharead among multiple threads. Josh From pbenedict at apache.org Mon Feb 22 13:05:59 2010 From: pbenedict at apache.org (Paul Benedict) Date: Mon, 22 Feb 2010 15:05:59 -0600 Subject: Nice to @Share? In-Reply-To: <17b2302a1002221238p2db47f2albcd2e6bb23ddafab@mail.gmail.com> References: <17b2302a1002221238p2db47f2albcd2e6bb23ddafab@mail.gmail.com> Message-ID: I agree with Josh and I strongly agree with his third point. I thought Sun (Oracle) had the philosophy that annotations should not alter the semantics of the language? An annotation that tells the compiler to turns allocation from stack to heap sounds unpalatable. This kind of behavior should be controlled by a (context-sensitive) keyword. On Mon, Feb 22, 2010 at 2:38 PM, Joshua Bloch wrote: > Some say it's nice to @Share, but not me. Draft 0.1.5 of the Project Lambda > spec contains some discussion of an @Shared annotation to "allow unqualified > references to upplevel variables (all sorts) if they are marked @Shared." > ?There are several reasons that I believe this not a good idea: > > (1) For fields (instance or static) this annotation is not required today > (in anonymouns class instance creation expressions), and I believe that > programmers are likely to see it as an added annoyance (like the current > situation with final variables). > > (2) For local variables, I see it as too much magic: > > ? ?@Shared int i; > > ?has a totally different implementation from: > > ? ?int i; > > The latter creates a "container object" on the heap, the former doesn't. The > semantics (in particular, the lifetime) and the performance are likely to > differ significantly. > > (3) I believe it's an abuse of annotations. ?Some may liken it to @Override, > but I believe this is specious. The @Override annotation merely expresses a > compile-time assertion that the compiler is able to verify. The proposed > @Shared annotation changes the semantics of the language. > > (4) If, in spite of all the above, it is the consesnsus that such an > annotation is still a good idea, I believe the name @Shared could hardly be > more misleading. In practical terms, it means "this variable must *not* be > shared (without additional synchronization)." It would typically be used > when the programmer could assert that the variable in question would never > be sharead among multiple threads. > > ? ? ? ? ? ?Josh > > From tronicek at fit.cvut.cz Mon Feb 22 13:07:26 2010 From: tronicek at fit.cvut.cz (Zdenek Tronicek) Date: Mon, 22 Feb 2010 22:07:26 +0100 Subject: Nice to @Share? In-Reply-To: <17b2302a1002221238p2db47f2albcd2e6bb23ddafab@mail.gmail.com> References: <17b2302a1002221238p2db47f2albcd2e6bb23ddafab@mail.gmail.com> Message-ID: Joshua Bloch napsal(a): > There are several reasons that I believe this not a good idea: > > (1) For fields (instance or static) this annotation is not required today > (in anonymouns class instance creation expressions), and I believe that > programmers are likely to see it as an added annoyance (like the current > situation with final variables). I think the annotation is only for local variables. As far as I know, in the BGGA prototype it is not used for fields. > (2) For local variables, I see it as too much magic: > > @Shared int i; > > has a totally different implementation from: > > int i; > > The latter creates a "container object" on the heap, the former doesn't. > The > semantics (in particular, the lifetime) and the performance are likely to > differ significantly. In the BGGA prototype, the annotation was meant as confirmation "I am aware that this variable may be shared between threads". It does not change the semantics because the variable can be captured even if it does not have this annotation. In such case, the compiler emitted warning that the captured variable is not annotated. Z. -- Zdenek Tronicek FIT CTU in Prague > (3) I believe it's an abuse of annotations. Some may liken it to > @Override, > but I believe this is specious. The @Override annotation merely expresses > a > compile-time assertion that the compiler is able to verify. The proposed > @Shared annotation changes the semantics of the language. > > (4) If, in spite of all the above, it is the consesnsus that such an > annotation is still a good idea, I believe the name @Shared could hardly > be > more misleading. In practical terms, it means "this variable must *not* be > shared (without additional synchronization)." It would typically be used > when the programmer could assert that the variable in question would never > be sharead among multiple threads. From neal at gafter.com Mon Feb 22 13:10:25 2010 From: neal at gafter.com (Neal Gafter) Date: Mon, 22 Feb 2010 13:10:25 -0800 Subject: Implementing recursive lambda with MethodHandle In-Reply-To: <17b2302a1002221219v1678540dw2a669f8dd2ebe734@mail.gmail.com> References: <4B826FEF.4070605@univ-mlv.fr> <17b2302a1002220949h5754454u6f7fe2603219c5b@mail.gmail.com> <39560A26-EFA0-4762-8B7D-28AE4B667B59@gmail.com> <17b2302a1002221038l268dff63wba991fb9c225a4a8@mail.gmail.com> <4B82D1F9.5040907@univ-mlv.fr> <17b2302a1002221128i41fa44fbp5cda06b6cc784f13@mail.gmail.com> <15e8b9d21002221148y8382f6dlb689117738118d92@mail.gmail.com> <17b2302a1002221219v1678540dw2a669f8dd2ebe734@mail.gmail.com> Message-ID: <15e8b9d21002221310x1f054fc2y4292022172d9abd6@mail.gmail.com> On Mon, Feb 22, 2010 at 12:19 PM, Joshua Bloch wrote: > On Mon, Feb 22, 2010 at 1:48 PM, Neal Gafter wrote: >> > That is definitely not the idea of this JSR. We are not "eliminating" >> the >> > need for anonymous inner classes. We are providing a good alternative >> for a >> > huge number of use cases. These recursive functions are a vanishingly >> small >> > fraction of the identified use cases. >> > >> >> Neal, I think Josh was saying the JSR is to eliminate the anonymous >> class syntax for lambdas -- not eliminate the syntax in general. > > > Exactly so. > Well, Java doesn't have any "lambdas" today, so to make sense of this we'd have to refer to what other languages call lambda. Doing so, we find a clear answer to the question of what it means for a lambda to be recursive, and how the recursion is expressed. From neal at gafter.com Mon Feb 22 13:31:48 2010 From: neal at gafter.com (Neal Gafter) Date: Mon, 22 Feb 2010 13:31:48 -0800 Subject: Nice to @Share? In-Reply-To: <17b2302a1002221238p2db47f2albcd2e6bb23ddafab@mail.gmail.com> References: <17b2302a1002221238p2db47f2albcd2e6bb23ddafab@mail.gmail.com> Message-ID: <15e8b9d21002221331p6be3edb6wa8ee62a3fe89565b@mail.gmail.com> I agree with Josh on most of this. An annotation could be used as long as it does not change the semantics.? If it does change the semantics (e.g. making something legal that would otherwise be illegal, or vice versa) then an annotation is not the right mechanism. His issues (1), (2), and (3) can be addressed by making violations of the rule a warning rather than an error, as in CfJ . As for the name, "shared" doesn't mean that the variable is shared between threads. It means that the variable may be shared between the method (or lambda) in which it is declared and any nested lambdas. Re 'In practical terms, it means "this variable must *not* be shared (without additional synchronization)."', this doesn't really make much sense to me; all variables (whether marked @Shared or not) must not be shared between threads (without additional synchronization). Our comments are on text that is in the commentary in the current draft. I presume that some upcoming revision of the draft spec will provide some way to access variables from the enclosing scope that are not morally final, but I'm willing to wait to see that revision to pass judgment. If no such support appears, I expect to complain. From jjb at google.com Mon Feb 22 13:37:55 2010 From: jjb at google.com (Joshua Bloch) Date: Mon, 22 Feb 2010 13:37:55 -0800 Subject: Nice to @Share? In-Reply-To: References: <17b2302a1002221238p2db47f2albcd2e6bb23ddafab@mail.gmail.com> Message-ID: <17b2302a1002221337v17ca9586r3efcc2f643dd78ff@mail.gmail.com> Zdenek, Hi! On Mon, Feb 22, 2010 at 1:07 PM, Zdenek Tronicek wrote: > Joshua Bloch napsal(a): > > There are several reasons that I believe this not a good idea: > > > > (1) For fields (instance or static) this annotation is not required today > > (in anonymouns class instance creation expressions), and I believe that > > programmers are likely to see it as an added annoyance (like the current > > situation with final variables). > > I think the annotation is only for local variables. As far as I know, in > the BGGA prototype it is not used for fields. > That may have been the case in some BGGA proposal(s), but the prose that I quoted (which included the phrase "all kinds of variables") came straight from a discussion section of the vurrent (0.1.5) draft of the Project Lambda proposal. > > > (2) For local variables, I see it as too much magic: > > > > @Shared int i; > > > > has a totally different implementation from: > > > > int i; > > > > The latter creates a "container object" on the heap, the former doesn't. > > The > > semantics (in particular, the lifetime) and the performance are likely to > > differ significantly. > > In the BGGA prototype, the annotation was meant as confirmation "I am > aware that this variable may be shared between threads". It does not > change the semantics because the variable can be captured even if it does > not have this annotation. In such case, the compiler emitted warning that > the captured variable is not annotated. > Understood. But the BGGA proposal was different in many ways, as it had very different goals. In this proposal, one practical alternative (as per the current normative text) is to ban outright the capture of local variables that are not final or effectively final. This requires programmers to use the well-known single-element array (or wrapper object) idiom in the rare case where they really do want to close over a mutable local variable. If we decide that we really must provide a more concise syntax (and I'm not at all sure that we should), then I believe a new modifier is justified (not an annotation), and that "shared" is a poor choice of name. I'm not sure what a good name would be, but I won't think about until we've decided that we really want such a thing. Think of it as lazy evaluation;) Josh From jjb at google.com Mon Feb 22 13:52:07 2010 From: jjb at google.com (Joshua Bloch) Date: Mon, 22 Feb 2010 13:52:07 -0800 Subject: Nice to @Share? In-Reply-To: <15e8b9d21002221331p6be3edb6wa8ee62a3fe89565b@mail.gmail.com> References: <17b2302a1002221238p2db47f2albcd2e6bb23ddafab@mail.gmail.com> <15e8b9d21002221331p6be3edb6wa8ee62a3fe89565b@mail.gmail.com> Message-ID: <17b2302a1002221352w3c878e49r730f43c5f26287e@mail.gmail.com> Neal, On Mon, Feb 22, 2010 at 1:31 PM, Neal Gafter wrote: > I agree with Josh on most of this. Twice in a week! Will wonders never cease? > > As for the name, "shared" doesn't mean that the variable is shared > between threads. It means that the variable may be shared between the > method (or lambda) in which it is declared and any nested lambdas. > I understand, but I believe that programmers would be very confused by this, because of the ambiguity in the possible meaning of shared. > Re 'In practical terms, it means "this variable must *not* be shared > (without additional synchronization)."', this doesn't really make much > sense to me; all variables (whether marked @Shared or not) must not be > shared between threads (without additional synchronization). > Not so. Shared *mutable* state must be synchronized. If it's not shared *or not mutable*, no synchronization is necessary. Josh From neal at gafter.com Mon Feb 22 13:54:41 2010 From: neal at gafter.com (Neal Gafter) Date: Mon, 22 Feb 2010 13:54:41 -0800 Subject: Nice to @Share? In-Reply-To: <17b2302a1002221352w3c878e49r730f43c5f26287e@mail.gmail.com> References: <17b2302a1002221238p2db47f2albcd2e6bb23ddafab@mail.gmail.com> <15e8b9d21002221331p6be3edb6wa8ee62a3fe89565b@mail.gmail.com> <17b2302a1002221352w3c878e49r730f43c5f26287e@mail.gmail.com> Message-ID: <15e8b9d21002221354n3780c69bkc10d42d382abaabb@mail.gmail.com> On Mon, Feb 22, 2010 at 1:52 PM, Joshua Bloch wrote: >> Re 'In practical terms, it means "this variable must *not* be shared >> (without additional synchronization)."', this doesn't really make much >> sense to me; all variables (whether marked @Shared or not) must not be >> shared between threads (without additional synchronization). > > Not so. Shared mutable state must be synchronized. ?If it's not shared or > not mutable, no synchronization is necessary. Variables that are marked @Shared (or whatever) are not necessarily mutated (or shared between threads); synchronization is orthogonal. From mthornton at optrak.co.uk Mon Feb 22 13:59:53 2010 From: mthornton at optrak.co.uk (Mark Thornton) Date: Mon, 22 Feb 2010 21:59:53 +0000 Subject: Nice to @Share? In-Reply-To: <17b2302a1002221352w3c878e49r730f43c5f26287e@mail.gmail.com> References: <17b2302a1002221238p2db47f2albcd2e6bb23ddafab@mail.gmail.com> <15e8b9d21002221331p6be3edb6wa8ee62a3fe89565b@mail.gmail.com> <17b2302a1002221352w3c878e49r730f43c5f26287e@mail.gmail.com> Message-ID: <4B82FE59.40808@optrak.co.uk> Joshua Bloch wrote: > Not so. Shared *mutable* state must be synchronized. If it's not shared *or > not mutable*, no synchronization is necessary. > > Josh > I understood that non mutable state actually has to be declared final if you want to share it across threads reliably without using synchronization. Merely not modifying it isn't sufficient. Mark From markmahieu at googlemail.com Mon Feb 22 14:02:23 2010 From: markmahieu at googlemail.com (Mark Mahieu) Date: Mon, 22 Feb 2010 22:02:23 +0000 Subject: Nice to @Share? In-Reply-To: <17b2302a1002221337v17ca9586r3efcc2f643dd78ff@mail.gmail.com> References: <17b2302a1002221238p2db47f2albcd2e6bb23ddafab@mail.gmail.com> <17b2302a1002221337v17ca9586r3efcc2f643dd78ff@mail.gmail.com> Message-ID: <8A83C7D6-4F56-42A1-82CC-EF2B6CFB33A5@googlemail.com> On 22 Feb 2010, at 21:37, Joshua Bloch wrote: > This requires programmers to use > the well-known single-element array (or wrapper object) idiom in the rare > case where they really do want to close over a mutable local variable. That's an interesting point actually - how rare is it? My gut feeling is that it's rather less rare than some of the other things we've discussed, but it might be useful to find out. Whilst producing the statistics which Alex requested a couple of weeks ago, I noticed a few classes which might have been good candidates for lambdas but for the fact that they contained an additional field updated by the 'primary' method, along with a getter method to retrieve its value. Mark From peter.kitt.reilly at gmail.com Mon Feb 22 14:08:16 2010 From: peter.kitt.reilly at gmail.com (Peter Reilly) Date: Mon, 22 Feb 2010 22:08:16 +0000 Subject: Nice to @Share? In-Reply-To: <8A83C7D6-4F56-42A1-82CC-EF2B6CFB33A5@googlemail.com> References: <17b2302a1002221238p2db47f2albcd2e6bb23ddafab@mail.gmail.com> <17b2302a1002221337v17ca9586r3efcc2f643dd78ff@mail.gmail.com> <8A83C7D6-4F56-42A1-82CC-EF2B6CFB33A5@googlemail.com> Message-ID: This is why AtomicInteger was invented Peter On Mon, Feb 22, 2010 at 10:02 PM, Mark Mahieu wrote: > > On 22 Feb 2010, at 21:37, Joshua Bloch wrote: > >> This requires programmers to use >> the well-known single-element array (or wrapper object) idiom in the rare >> case where they really do want to close over a mutable local variable. > > That's an interesting point actually - how rare is it? ?My gut feeling is that it's rather less rare than some of the other things we've discussed, but it might be useful to find out. > > Whilst producing the statistics which Alex requested a couple of weeks ago, I noticed a few classes which might have been good candidates for lambdas but for the fact that they contained an additional field updated by the 'primary' method, along with a getter method to retrieve its value. > > Mark > > > From jjb at google.com Mon Feb 22 14:29:13 2010 From: jjb at google.com (Joshua Bloch) Date: Mon, 22 Feb 2010 14:29:13 -0800 Subject: Nice to @Share? In-Reply-To: <4B82FE59.40808@optrak.co.uk> References: <17b2302a1002221238p2db47f2albcd2e6bb23ddafab@mail.gmail.com> <15e8b9d21002221331p6be3edb6wa8ee62a3fe89565b@mail.gmail.com> <17b2302a1002221352w3c878e49r730f43c5f26287e@mail.gmail.com> <4B82FE59.40808@optrak.co.uk> Message-ID: <17b2302a1002221429l3b83034aif24099f8da377176@mail.gmail.com> Mark, On Mon, Feb 22, 2010 at 1:59 PM, Mark Thornton wrote: > Joshua Bloch wrote: > >> Not so. Shared *mutable* state must be synchronized. If it's not shared >> *or >> not mutable*, no synchronization is necessary. >> >> Josh >> >> > I understood that non mutable state actually has to be declared final if > you want to share it across threads reliably without using synchronization. > Merely not modifying it isn't sufficient. This is a complex topic. It turns out that if an object is "strongly immutable" (like String), you can share references to it with no synchronization whatsoever.If you're morbidly curious, this is a good place to start: http://www.cs.umd.edu/~pugh/java/memoryModel/ . Josh From markmahieu at googlemail.com Mon Feb 22 14:34:47 2010 From: markmahieu at googlemail.com (Mark Mahieu) Date: Mon, 22 Feb 2010 22:34:47 +0000 Subject: Nice to @Share? In-Reply-To: References: <17b2302a1002221238p2db47f2albcd2e6bb23ddafab@mail.gmail.com> <17b2302a1002221337v17ca9586r3efcc2f643dd78ff@mail.gmail.com> <8A83C7D6-4F56-42A1-82CC-EF2B6CFB33A5@googlemail.com> Message-ID: <49516B11-68AF-41D6-B4E2-1D73945FD398@googlemail.com> Well I don't know what the reasoning was behind the cases I spotted, but the ones I recall were synchronously invoked from a single thread, so there were a number of viable options, including the (well known, as Josh points out) local single-element array variable. Perhaps the programmer(s) in question simply found the technique more tasteful than other forms of wrapper. Mark On 22 Feb 2010, at 22:08, Peter Reilly wrote: > This is why AtomicInteger was invented > > Peter > > On Mon, Feb 22, 2010 at 10:02 PM, Mark Mahieu wrote: >> >> On 22 Feb 2010, at 21:37, Joshua Bloch wrote: >> >>> This requires programmers to use >>> the well-known single-element array (or wrapper object) idiom in the rare >>> case where they really do want to close over a mutable local variable. >> >> That's an interesting point actually - how rare is it? My gut feeling is that it's rather less rare than some of the other things we've discussed, but it might be useful to find out. >> >> Whilst producing the statistics which Alex requested a couple of weeks ago, I noticed a few classes which might have been good candidates for lambdas but for the fact that they contained an additional field updated by the 'primary' method, along with a getter method to retrieve its value. >> >> Mark >> >> >> From tronicek at fit.cvut.cz Tue Feb 23 00:08:22 2010 From: tronicek at fit.cvut.cz (Zdenek Tronicek) Date: Tue, 23 Feb 2010 09:08:22 +0100 Subject: Nice to @Share? Message-ID: <0c3eb5b5e95f5744fed565e0fcfec6ca.squirrel@imap.fit.cvut.cz> Joshua Bloch napsal(a): >> I understood that non mutable state actually has to be declared final if >> you want to share it across threads reliably without using >> synchronization. >> Merely not modifying it isn't sufficient. > This is a complex topic. It turns out that if an object is "strongly immutable" (like String), you can share references to it with no synchronization whatsoever.If you're morbidly curious, this is a good place > to start: http://www.cs.umd.edu/~pugh/java/memoryModel/ . In some sense, Mark is right. Of course, you can share strongly immutable objects without any synchronization. But how to share the reference? If one thread stores a reference to a shared variable, how do you ensure that such change is visible in another thread? You need a barrier and this is probably where the hint "make the shared variables final" comes from. Z. -- Zdenek Tronicek FIT CTU in Prague From jjb at google.com Tue Feb 23 00:23:46 2010 From: jjb at google.com (Joshua Bloch) Date: Tue, 23 Feb 2010 00:23:46 -0800 Subject: Nice to @Share? In-Reply-To: <0c3eb5b5e95f5744fed565e0fcfec6ca.squirrel@imap.fit.cvut.cz> References: <0c3eb5b5e95f5744fed565e0fcfec6ca.squirrel@imap.fit.cvut.cz> Message-ID: <17b2302a1002230023p5507de6x9f4d8a7bf1ddec91@mail.gmail.com> Zdenek, On Tue, Feb 23, 2010 at 12:08 AM, Zdenek Tronicek wrote: > Joshua Bloch napsal(a): > >> I understood that non mutable state actually has to be declared final > if > >> you want to share it across threads reliably without using > >> synchronization. > >> Merely not modifying it isn't sufficient. > > This is a complex topic. It turns out that if an object is "strongly > immutable" (like String), you can share references to it with no > synchronization whatsoever.If you're morbidly curious, this is a good > place > > to start: http://www.cs.umd.edu/~pugh/java/memoryModel/ . > > In some sense, Mark is right. Of course, you can share strongly immutable > objects without any synchronization. But how to share the reference? If > one thread stores a reference to a shared variable, how do you ensure that > such change is visible in another thread? You need a barrier and this is > probably where the hint "make the shared variables final" comes from. Believe it or not, you don't. For example, you can have one thread create a new String, jam a reference to it into a public static variable, and another thread can safely read the String. You should pretty much never do this, but it was a constraint in the design of the memory model: Java relies on immutability of String (and Integer, and the like), even if references are shared willy-nilly. That said, ordinary humans should "synchronize" all access to shared mutable state (where synchronize is broadly defined). "Just because you can doesn't mean you should." Josh From mcnepp02 at googlemail.com Tue Feb 23 00:33:24 2010 From: mcnepp02 at googlemail.com (Gernot Neppert) Date: Tue, 23 Feb 2010 08:33:24 +0000 Subject: Library Solution possible? (was: Nice to @Share?) Message-ID: <1f5b87141002230033n56942b3ay1ad1a3032f38a54a@mail.gmail.com> 2010/2/22 Joshua Bloch : > that are not final or effectively final. This requires programmers to use > the well-known single-element array (or wrapper object) idiom in the rare > case where they really do want to close over a mutable local variable. If we > decide that we really must provide a more concise syntax (and I'm not at all > sure that we should), then I believe a new modifier is justified (not an > annotation), and that "shared" is a poor choice of name. I'm not sure what a > good name would be, but I won't think about until we've decided that we > really want such a thing. Think of it as lazy evaluation;) I'm also not convinved that we need a more concise snytax for capturing non-final variables at all. Quite rarely have I've been wanting to modify a local variable from within an anonymous inner class. Half of them have been asynchronous invocations, and that was before the advent of the Executor-framework in Java 5 with its concept of "Futures". The other half were synchronous "closure-like" Predicates or Transformations in order to process Collections, and those were almost always completely free of side-effects, which I consider good practice anyhow, BTW. In the rare remaining cases I was sometimes tempted to (ab)use the "Holder" classes from the package org.omg.CORBA. This makes me ask: Can we come up with a Standard Library solution for mutable local objects? Something like "java.util.MutableVariable" From abies at adres.pl Tue Feb 23 01:06:02 2010 From: abies at adres.pl (abies at adres.pl) Date: Tue, 23 Feb 2010 10:06:02 +0100 Subject: Nice to @Share? In-Reply-To: 17b2302a1002230023p5507de6x9f4d8a7bf1ddec91@mail.gmail.com Message-ID: "Joshua Bloch" napisa?(a): > Believe it or not, you don't. For example, you can have one thread create a > new String, jam a reference to it into a public static variable, and another > thread can safely read the String. You should pretty much never do this, > but it was a constraint in the design of the memory model: Java relies on > immutability of String (and Integer, and the like), even if references are > shared willy-nilly. I think that original point was about visibility of the reference update, not about seeing initialized/not-initialized String itself (which indeed is guaranteed in case of proper immutable objects). If both threads are already started and there is no synchronization point between them, thread2 is allowed to NEVER see the update to non-volatile static field done by thread1. Question is, does it matter to us? I'm not sure if it is possible to do something and then dispatch work on that something to multiple threads using any of the concurrency facilities without performing implicit synchronization. Putting any kind of task to executor/thread pool will create some synchronization point on one of the queues (unless there is some CAS no-lock no-membarrier mojo involved there which will be well over top of my head...). This means we get consistent state at the beginning. If you mutate the variable in non-trivial way without synchronization/locking from multiple threads afterward, you are in the mess anyway, updates visible or not. Only interesting non-synchronized case would be probably something like boolean initialized to false (or reference from null to something), which is then turned to 'true' if one of the threads hits some condition. But again, in this case, you will probably want to wait for all threads to finish their work before checking the boolean - and again, this 'waiting' will implicitly make the memory right. If you just loop over boolean without explicitly checking if threads are finished... I don't think you deserve a cookie. Regards, Artur Biesiadowski From Alex.Buckley at Sun.COM Tue Feb 23 01:22:36 2010 From: Alex.Buckley at Sun.COM (Alex Buckley) Date: Tue, 23 Feb 2010 01:22:36 -0800 Subject: Nice to @Share? In-Reply-To: <17b2302a1002221238p2db47f2albcd2e6bb23ddafab@mail.gmail.com> References: <17b2302a1002221238p2db47f2albcd2e6bb23ddafab@mail.gmail.com> Message-ID: <4B839E5C.5010502@sun.com> As Neal said, @Shared is mentioned in non-normative text lifted directly from John Rose's mail. I have never thought it appropriate for an annotation to turn a local variable into a shared variable. That is, once a method-scoped variable is morally "shared", it isn't a local variable (stack-allocated, single-thread-visibility, no volatility) at all. It's an eighth kind of variable in JLS 4.12.3. (And being 'final' is orthogonal.) A restricted keyword - suggestions welcome - is more than appropriate. Alex Joshua Bloch wrote: > Some say it's nice to @Share, but not me. Draft 0.1.5 of the Project Lambda > spec contains some discussion of an @Shared annotation to "allow unqualified > references to upplevel variables (all sorts) if they are marked @Shared." > There are several reasons that I believe this not a good idea: > > (1) For fields (instance or static) this annotation is not required today > (in anonymouns class instance creation expressions), and I believe that > programmers are likely to see it as an added annoyance (like the current > situation with final variables). > > (2) For local variables, I see it as too much magic: > > @Shared int i; > > has a totally different implementation from: > > int i; > > The latter creates a "container object" on the heap, the former doesn't. The > semantics (in particular, the lifetime) and the performance are likely to > differ significantly. > > (3) I believe it's an abuse of annotations. Some may liken it to @Override, > but I believe this is specious. The @Override annotation merely expresses a > compile-time assertion that the compiler is able to verify. The proposed > @Shared annotation changes the semantics of the language. > > (4) If, in spite of all the above, it is the consesnsus that such an > annotation is still a good idea, I believe the name @Shared could hardly be > more misleading. In practical terms, it means "this variable must *not* be > shared (without additional synchronization)." It would typically be used > when the programmer could assert that the variable in question would never > be sharead among multiple threads. > > Josh > From David.Moss at ubs.com Tue Feb 23 01:24:23 2010 From: David.Moss at ubs.com (David.Moss at ubs.com) Date: Tue, 23 Feb 2010 10:24:23 +0100 Subject: Implementing recursive lambda with MethodHandle In-Reply-To: <15e8b9d21002221310x1f054fc2y4292022172d9abd6@mail.gmail.com> References: <4B826FEF.4070605@univ-mlv.fr><17b2302a1002220949h5754454u6f7fe2603219c5b@mail.gmail.com> <39560A26-EFA0-4762-8B7D-28AE4B667B59@gmail.com><17b2302a1002221038l268dff63wba991fb9c225a4a8@mail.gmail.com> <4B82D1F9.5040907@univ-mlv.fr><17b2302a1002221128i41fa44fbp5cda06b6cc784f13@mail.gmail.com> <15e8b9d21002221148y8382f6dlb689117738118d92@mail.gmail.com> <17b2302a1002221219v1678540dw2a669f8dd2ebe734@mail.gmail.com> <15e8b9d21002221310x1f054fc2y4292022172d9abd6@mail.gmail.com> Message-ID: <9FF0EECF0E58D849A9D537E324DDDC0D0452EFA7@NZURC102PEX1.ubsw.net> Hi, I don't think 'this' should refer to the lambda. A lambda is a function belonging to it's enclosing scope (class), this means that 'this' needs to refer to the enclosing scope in order to be semantically correct. An additional syntax needs to be defined to refer to the lambda itself. A few options OTTOMH: Public class C { /* * 1- Create a closure on the lambda variable. * * In this example, lambda1 is closed on at declaration * time and remains a pointer to the function regardless * of what the outer scope does to it's lambda1 variable. * However, this approach potentially has consequences * outside of the function scope, but better suited for * the closures-dev list to discuss if such an approach * were to be taken. */ #int(int) lambda1 = #int(int x) { return x == 0 ? 1 : x * lambda1.(x - 1); }; /* * 2- Reference the variable. * * Similar to the above, but weaker, as the enclosing * scope can change the value of 'lambda' making it * point to a different function. Note that this may be * desired behaviour and should be allowed, but should * not be the only means of accessing the function. */ #int(int) lambda2 = #int(int x) { return x == 0 ? 1 : x * this.lambda2.(x - 1); }; /* * 3- Force recursive function variables to be final. * * The final declaration could be implied, here it is * explicit just for the sake of clarity. */ final #int(int) lambda3 = #int(int x) { return x == 0 ? 1 : x * lambda3.(x - 1); }; /* * 4- Omit the variable name altogether. * * Create new syntax that behaves similarly to 'this' * within function scope. The important thing about * this approach is that it allows fully anonymous * recursive functions, and as such is my preferred * approach. */ int result1 = #int(int x) { return x == 0 ? 1 : x * .(x - 1); }.(10); // Another example of the same, but using '#' as the // reference to the function as I know a lot of people // will be unhappy with the previous syntax (though it // makes no difference): int result2 = #int(int x) { return x == 0 ? 1 : x * #.(x - 1); }.(10); } Personally, I would allow for 2 and 4 (and potentially 1, if lexical closures are a desired feature). Kind Regards, David. -- EUR2 EG.1168 [1923 96757] SST: DL-SST-DEV "Where's the Kaboom? There was supposed to be an Earth-shattering Kaboom!" - Marvin the Martian "Don't worry about what anybody else is going to do. The best way to predict the future is to invent it." - Alan Kay "there is this amazingly powerful thing writhing around in there that will basically do everything I could possibly ask of it if only I knew how." - Ben Butler-Cole Based on the present E-Mail exchange, and/or on the agreement reached with you, respectively, UBS is entitled to contact you via insecure E-Mail: (a) E-Mails contain substantial risks such as lack of confidentiality, manipulation of content and sender, misdirection, viruses etc. UBS does not accept any liability for damages arising from use of E-mail. Accordingly, UBS recommends to abstain from sending any sensitive information via E-Mail, from forwarding the text received when submitting reply E-Mails and recommends to manually capture the E-Mail address in every instance. If you should wish to verify the content of this message, please request a hard-copy version. (b) In principle, UBS does not accept any (purchase) orders, cancellation of orders or authorizations etc. via E-mail. If UBS receives such E-Mails, UBS is not obliged to expressly decline them. If you have received this E-Mail by mistake or do not wish to be contacted by E-Mail in the future, you are kindly asked to inform UBS accordingly. Any E-Mail received by mistake (including all its annexes) needs to be destroyed and the content may not be forwarded nor disclosed to any further persons. c) This message is provided for informational purposes and should not be construed as a solicitation or offer to buy or sell any securities or related financial instruments. UBS reserves the right to retain all messages. Messages are protected and accessed only in legally justified cases. From neal at gafter.com Tue Feb 23 01:39:40 2010 From: neal at gafter.com (Neal Gafter) Date: Tue, 23 Feb 2010 01:39:40 -0800 Subject: Nice to @Share? In-Reply-To: <4B839E5C.5010502@sun.com> References: <17b2302a1002221238p2db47f2albcd2e6bb23ddafab@mail.gmail.com> <4B839E5C.5010502@sun.com> Message-ID: <15e8b9d21002230139k561e32d5x8454a1e69350c868@mail.gmail.com> On Tue, Feb 23, 2010 at 1:22 AM, Alex Buckley wrote: > As Neal said, @Shared is mentioned in non-normative text lifted directly > from John Rose's mail. I have never thought it appropriate for an > annotation to turn a local variable into a shared variable. That is, > once a method-scoped variable is morally "shared", it isn't a local > variable (stack-allocated, single-thread-visibility, no volatility) at > all. It's an eighth kind of variable in JLS 4.12.3. (And being 'final' > is orthogonal.) A restricted keyword - suggestions welcome - is more > than appropriate. The definition of "local variable" in (?4.12.3) is "Local variables are declared by local variable declaration statements (?14.4). Whenever the flow of control enters a block (?14.2) or for statement (?14.14), a new variable is created for each local variable declared in a local variable declaration statement immediately contained within that block or for statement. A local variable declaration statement may contain an expression which initializes the variable. The local variable with an initializing expression is not initialized, however, until the local variable declaration statement that declares it is executed. (The rules of definite assignment (?16) prevent the value of a local variable from being used before it has been initialized or otherwise assigned a value.) The local variable effectively ceases to exist when the execution of the block or for statement is complete." The only part of that requiring rewording in the presence of lambdas that can access local variables from the enclosing scope is the last sentence. The rewording is required whether or not such variables are final. In fact, that last sentence appears to conflict with the fact that final variables can be accessed in an anonymous inner class after the scope containing the variable has completed. A new keyword, while better than what programmers have to do today, is still boilerplate. From reinier at zwitserloot.com Tue Feb 23 02:00:14 2010 From: reinier at zwitserloot.com (Reinier Zwitserloot) Date: Tue, 23 Feb 2010 11:00:14 +0100 Subject: Nice to @Share? In-Reply-To: <15e8b9d21002230139k561e32d5x8454a1e69350c868@mail.gmail.com> References: <17b2302a1002221238p2db47f2albcd2e6bb23ddafab@mail.gmail.com> <4B839E5C.5010502@sun.com> <15e8b9d21002230139k561e32d5x8454a1e69350c868@mail.gmail.com> Message-ID: <560fb5ed1002230200r30026d18gdaea464471159c7d@mail.gmail.com> If anyone is going to run an analysis of the single-element array pattern to have mutable local variables accessible from anonymous inner classes, keep the following in mind: single-element array is just one way to do it. 2 other ways: - AtomicInteger, AtomicReference, and friends. - A custom Pointer class: public class Pointer { public T value; public Pointer(T initial) { this.value = initial; } } this class may not be called Pointer and could look quite different. - Turning the local variable into a field instead (with all the pain that entails for threading issues, but, nevertheless, it is one possible way around this issue). As with many forays into existing code to see if a certain feature is needed or not, the very fact that a feature does not exist means code has been designed to work around the lack of said feature. --Reinier Zwitserloot On Tue, Feb 23, 2010 at 10:39 AM, Neal Gafter wrote: > On Tue, Feb 23, 2010 at 1:22 AM, Alex Buckley > wrote: > > As Neal said, @Shared is mentioned in non-normative text lifted directly > > from John Rose's mail. I have never thought it appropriate for an > > annotation to turn a local variable into a shared variable. That is, > > once a method-scoped variable is morally "shared", it isn't a local > > variable (stack-allocated, single-thread-visibility, no volatility) at > > all. It's an eighth kind of variable in JLS 4.12.3. (And being 'final' > > is orthogonal.) A restricted keyword - suggestions welcome - is more > > than appropriate. > > The definition of "local variable" in (?4.12.3) is "Local variables > are declared by local variable declaration statements (?14.4). > Whenever the flow of control enters a block (?14.2) or for statement > (?14.14), a new variable is created for each local variable declared > in a local variable declaration statement immediately contained within > that block or for statement. A local variable declaration statement > may contain an expression which initializes the variable. The local > variable with an initializing expression is not initialized, however, > until the local variable declaration statement that declares it is > executed. (The rules of definite assignment (?16) prevent the value of > a local variable from being used before it has been initialized or > otherwise assigned a value.) The local variable effectively ceases to > exist when the execution of the block or for statement is complete." > > The only part of that requiring rewording in the presence of lambdas > that can access local variables from the enclosing scope is the last > sentence. The rewording is required whether or not such variables are > final. In fact, that last sentence appears to conflict with the fact > that final variables can be accessed in an anonymous inner class after > the scope containing the variable has completed. > > A new keyword, while better than what programmers have to do today, is > still boilerplate. > > From reinier at zwitserloot.com Tue Feb 23 02:03:00 2010 From: reinier at zwitserloot.com (Reinier Zwitserloot) Date: Tue, 23 Feb 2010 11:03:00 +0100 Subject: Nice to @Share? In-Reply-To: <4B839E5C.5010502@sun.com> References: <17b2302a1002221238p2db47f2albcd2e6bb23ddafab@mail.gmail.com> <4B839E5C.5010502@sun.com> Message-ID: <560fb5ed1002230203p94ff249j5e81c8bb1db99bfa@mail.gmail.com> "public" as a keyword on local variable declarations and method parameters avoids the need to mess with (restricted) new keywords, is backwards compatible, and there is some (but admittedly not a lot) semantic harmony between public fields and methods, and local variables that end up declared on the heap, potentially accessible by multiple threads, etc. I'm not sure if there is a single short word that covers both the meaning and repercussions of letting a local variable be accessible by lambdas. In that case, 'public's intrinsic benefits, i.e. it being an existing non-restricted keyword, makes for a strong case. Second point: Consider allowing access from anonymous inner classes as well? Just because we'll have shiny new lambdas doesn't mean anonymous classes are going to go away or start gathering dust! --Reinier Zwitserloot On Tue, Feb 23, 2010 at 10:22 AM, Alex Buckley wrote: > As Neal said, @Shared is mentioned in non-normative text lifted directly > from John Rose's mail. I have never thought it appropriate for an > annotation to turn a local variable into a shared variable. That is, > once a method-scoped variable is morally "shared", it isn't a local > variable (stack-allocated, single-thread-visibility, no volatility) at > all. It's an eighth kind of variable in JLS 4.12.3. (And being 'final' > is orthogonal.) A restricted keyword - suggestions welcome - is more > than appropriate. > > Alex > > Joshua Bloch wrote: > > Some say it's nice to @Share, but not me. Draft 0.1.5 of the Project > Lambda > > spec contains some discussion of an @Shared annotation to "allow > unqualified > > references to upplevel variables (all sorts) if they are marked @Shared." > > There are several reasons that I believe this not a good idea: > > > > (1) For fields (instance or static) this annotation is not required today > > (in anonymouns class instance creation expressions), and I believe that > > programmers are likely to see it as an added annoyance (like the current > > situation with final variables). > > > > (2) For local variables, I see it as too much magic: > > > > @Shared int i; > > > > has a totally different implementation from: > > > > int i; > > > > The latter creates a "container object" on the heap, the former doesn't. > The > > semantics (in particular, the lifetime) and the performance are likely to > > differ significantly. > > > > (3) I believe it's an abuse of annotations. Some may liken it to > @Override, > > but I believe this is specious. The @Override annotation merely expresses > a > > compile-time assertion that the compiler is able to verify. The proposed > > @Shared annotation changes the semantics of the language. > > > > (4) If, in spite of all the above, it is the consesnsus that such an > > annotation is still a good idea, I believe the name @Shared could hardly > be > > more misleading. In practical terms, it means "this variable must *not* > be > > shared (without additional synchronization)." It would typically be used > > when the programmer could assert that the variable in question would > never > > be sharead among multiple threads. > > > > Josh > > > > From reinier at zwitserloot.com Tue Feb 23 02:11:40 2010 From: reinier at zwitserloot.com (Reinier Zwitserloot) Date: Tue, 23 Feb 2010 11:11:40 +0100 Subject: Library Solution possible? (was: Nice to @Share?) In-Reply-To: <1f5b87141002230033n56942b3ay1ad1a3032f38a54a@mail.gmail.com> References: <1f5b87141002230033n56942b3ay1ad1a3032f38a54a@mail.gmail.com> Message-ID: <560fb5ed1002230211u18861e0drd4754177c08eed98@mail.gmail.com> Sure. It already exists though: java.util.concurrent.atomic.AtomicReference. Yes, you get the extra 'baggage' of the thread safety in the .get() and .set() methods that AtomicReference has, but this would seem to be a good thing, given that by sharing across lambdas you open the door to threading issues. On the other hand, past experiences in java have always resulted in creating a mirror non-thread-safe object (StringBuilder vs. StringBuffer, ArrayList vs. Vector, HashMap vs. Hashtable), for performance reasons. I always use AtomicReference (or AtomicInteger and variants) when I need mutable access to parent scope from an anonymous inner class literal. --Reinier Zwitserloot On Tue, Feb 23, 2010 at 9:33 AM, Gernot Neppert wrote: > 2010/2/22 Joshua Bloch : > > that are not final or effectively final. This requires programmers to use > > the well-known single-element array (or wrapper object) idiom in the rare > > case where they really do want to close over a mutable local variable. If > we > > decide that we really must provide a more concise syntax (and I'm not at > all > > sure that we should), then I believe a new modifier is justified (not an > > annotation), and that "shared" is a poor choice of name. I'm not sure > what a > > good name would be, but I won't think about until we've decided that we > > really want such a thing. Think of it as lazy evaluation;) > > I'm also not convinved that we need a more concise snytax for > capturing non-final variables at all. > > Quite rarely have I've been wanting to modify a local variable from > within an anonymous inner class. > Half of them have been asynchronous invocations, and that was before > the advent of the Executor-framework in Java 5 with its concept of > "Futures". > The other half were synchronous "closure-like" Predicates or > Transformations in order to process Collections, and those were almost > always completely free of side-effects, which I consider good practice > anyhow, BTW. > > In the rare remaining cases I was sometimes tempted to (ab)use the > "Holder" classes from the package org.omg.CORBA. > This makes me ask: > Can we come up with a Standard Library solution for mutable local objects? > Something like "java.util.MutableVariable" > > From mcnepp02 at googlemail.com Tue Feb 23 02:46:03 2010 From: mcnepp02 at googlemail.com (Gernot Neppert) Date: Tue, 23 Feb 2010 10:46:03 +0000 Subject: Library Solution possible? (was: Nice to @Share?) In-Reply-To: <560fb5ed1002230211u18861e0drd4754177c08eed98@mail.gmail.com> References: <1f5b87141002230033n56942b3ay1ad1a3032f38a54a@mail.gmail.com> <560fb5ed1002230211u18861e0drd4754177c08eed98@mail.gmail.com> Message-ID: <1f5b87141002230246r6b09df4fk2b4834b32c1236da@mail.gmail.com> 2010/2/23 Reinier Zwitserloot : > Sure. It already exists though: > java.util.concurrent.atomic.AtomicReference. > Yes, you get the extra 'baggage' of the thread safety in the .get() and > .set() methods that AtomicReference has, but this would seem to be a good > thing, given that by sharing across lambdas you open the door to threading > issues. On the other hand, past experiences in java have always resulted in > creating a mirror non-thread-safe object (StringBuilder vs. StringBuffer, > ArrayList vs. Vector, HashMap vs. Hashtable), for performance reasons. > I always use AtomicReference (or AtomicInteger and variants) when I need > mutable access to parent scope from an anonymous inner class literal. > --Reinier Zwitserloot Oh dear, of course you're right: java.util.concurrent.atomic already contains the classes that I was proposing! Don't know how that could have escaped me... I don't think we'd need another family of non-thread-safe classes, since the performance overhead should be neglible, especially if only "get" and "set" would be used. From alex.blewitt at gmail.com Tue Feb 23 03:03:05 2010 From: alex.blewitt at gmail.com (Alex Blewitt) Date: Tue, 23 Feb 2010 11:03:05 +0000 Subject: Implementing recursive lambda with MethodHandle In-Reply-To: <9FF0EECF0E58D849A9D537E324DDDC0D0452EFA7@NZURC102PEX1.ubsw.net> References: <4B826FEF.4070605@univ-mlv.fr><17b2302a1002220949h5754454u6f7fe2603219c5b@mail.gmail.com> <39560A26-EFA0-4762-8B7D-28AE4B667B59@gmail.com><17b2302a1002221038l268dff63wba991fb9c225a4a8@mail.gmail.com> <4B82D1F9.5040907@univ-mlv.fr><17b2302a1002221128i41fa44fbp5cda06b6cc784f13@mail.gmail.com> <15e8b9d21002221148y8382f6dlb689117738118d92@mail.gmail.com> <17b2302a1002221219v1678540dw2a669f8dd2ebe734@mail.gmail.com> <15e8b9d21002221310x1f054fc2y4292022172d9abd6@mail.gmail.com> <9FF0EECF0E58D849A9D537E324DDDC0D0452EFA7@NZURC102PEX1.ubsw.net> Message-ID: <6C819E7F-7249-4689-B1D7-EDB910201738@gmail.com> On 23 Feb 2010, at 09:24, wrote: > I don't think 'this' should refer to the lambda. > > A lambda is a function belonging to it's enclosing scope (class), this > means that 'this' needs to refer to the enclosing scope in order to be > semantically correct. Only if 'this' is defined as referring to the enclosing class. If 'this' is defined as referring to the lambda itself, then there is no semantic error. > An additional syntax needs to be defined to refer to the lambda itself. Whilst that may be the case, your code is solely focusing on 'simple' recursion, where the target of the lambda is a lambda invocation. That isn't necessarily the case. For example, one may use lambdas to provide an inversion of control, where the lambda itself is passed on as an argument: process = #(Node n) { n.accept( process ) }; Simply omitting the target of a method invocation won't help here, because we need to pass the (self) reference rather than invoking internally. In other words, any proposed syntax should be able to act as the target of a lambda invocation (i.e. foo.()) as well as the lambda reference (i.e. foo). > Public class C { > /* > * 1- Create a closure on the lambda variable. That is one of the proposals so far. > /* > * 2- Reference the variable. You're referring to it with 'this.lambda2', which only works if the lambda is an instance field of an enclosing class. It wouldn't work for a local defined lambda. One could put further restrictions (like recursive lambdas can only be defined at the top scope) but this seems an artificial and unnecessary restriction. > /* > * 3- Force recursive function variables to be final. > * This would prevent a redefinition case. Questions still arise with the question of mutable lambda references captured by other lambdas. > > /* > * 4- Omit the variable name altogether. This fails to allow the lambda to be passed by reference to other lambdas/functions/methods/classes inside the lambda. > Personally, I would allow for 2 and 4 (and potentially 1, if lexical > closures are a desired feature). The problem with 2 is that it only works for instance fields; the problem with 4 is that it prevents passing the lambda by reference. We could create a new keyword, like 'lambda', which acts as the self-lambda reference, but it seems a lot of work rather than just re-targeting 'this' to refer to the enclosing lambda scope, and allowing Outer.this to refer to the enclosing class instance. Alex From peter.levart at marand.si Tue Feb 23 03:06:02 2010 From: peter.levart at marand.si (Peter Levart) Date: Tue, 23 Feb 2010 12:06:02 +0100 Subject: Nice to @Share? In-Reply-To: <0c3eb5b5e95f5744fed565e0fcfec6ca.squirrel@imap.fit.cvut.cz> References: <0c3eb5b5e95f5744fed565e0fcfec6ca.squirrel@imap.fit.cvut.cz> Message-ID: <201002231206.02787.peter.levart@marand.si> On Tuesday 23 February 2010 09:08:22 Zdenek Tronicek wrote: > In some sense, Mark is right. Of course, you can share strongly immutable > objects without any synchronization. But how to share the reference? If > one thread stores a reference to a shared variable, how do you ensure that > such change is visible in another thread? You need a barrier and this is > probably where the hint "make the shared variables final" comes from. > > Z. > -- > Zdenek Tronicek > FIT CTU in Prague > The barrier is usually made when you dispatch the reference to function to some other thread, so you don't have to worry about it... Peter From mcnepp02 at googlemail.com Tue Feb 23 03:09:21 2010 From: mcnepp02 at googlemail.com (Gernot Neppert) Date: Tue, 23 Feb 2010 11:09:21 +0000 Subject: Nice to @Share? In-Reply-To: <560fb5ed1002230203p94ff249j5e81c8bb1db99bfa@mail.gmail.com> References: <17b2302a1002221238p2db47f2albcd2e6bb23ddafab@mail.gmail.com> <4B839E5C.5010502@sun.com> <560fb5ed1002230203p94ff249j5e81c8bb1db99bfa@mail.gmail.com> Message-ID: <1f5b87141002230309i334a2ac5la8704559359e244@mail.gmail.com> 2010/2/23 Reinier Zwitserloot : > "public" as a keyword on local variable declarations and method parameters > avoids the need to mess with (restricted) new keywords, is backwards > compatible, and there is some (but admittedly not a lot) semantic harmony > between public fields and methods, and local variables that end up declared > on the heap, potentially accessible by multiple threads, etc > Well, at least it is more self-explanatory than the proposed annotation @Shared. And, as you say, having a "real" keyword for a modifier with such vast implications seems appropriate! But I wouldn't allow it on method parameters, though. Modifying method parameters directly is bad coding style, IMHO. One could argue that method parameters should have been "final" by default in the first place. Apart from that, it worries me a little that in the future there would be 2 ways of achieving access to local variables from anonymous inner classes + lambdas: "final" and "public". You'd have to teach programmers that if they needed read access only, they could use both but were strongly encouraged to use "final". Then you'd have to go into detail to explain why ("declaring a variable public entails the creation of a special-class heap object which will be auto-boxed and unboxed every time the variable is being accessed")... From Alex.Buckley at Sun.COM Tue Feb 23 03:17:33 2010 From: Alex.Buckley at Sun.COM (Alex Buckley) Date: Tue, 23 Feb 2010 03:17:33 -0800 Subject: Nice to @Share? In-Reply-To: <15e8b9d21002230139k561e32d5x8454a1e69350c868@mail.gmail.com> References: <17b2302a1002221238p2db47f2albcd2e6bb23ddafab@mail.gmail.com> <4B839E5C.5010502@sun.com> <15e8b9d21002230139k561e32d5x8454a1e69350c868@mail.gmail.com> Message-ID: <4B83B94D.2070504@sun.com> Neal Gafter wrote: > The definition of "local variable" in (?4.12.3) is ... > The only part of that requiring rewording in the presence of lambdas > that can access local variables from the enclosing scope is the last > sentence. The rewording is required whether or not such variables are > final. In fact, that last sentence appears to conflict with the fact > that final variables can be accessed in an anonymous inner class after > the scope containing the variable has completed. Consider also JLS3 17.4.1: "Local variables ... are never shared between threads and are unaffected by the memory model." A variable hoisted on to the heap is not a local variable, even if its declaration form looks similar to a local variable declaration statement. For example, you could morally have a volatile shared variable, whereas you cannot have a volatile local variable. Alex From David.Moss at ubs.com Tue Feb 23 03:22:30 2010 From: David.Moss at ubs.com (David.Moss at ubs.com) Date: Tue, 23 Feb 2010 12:22:30 +0100 Subject: Implementing recursive lambda with MethodHandle In-Reply-To: <6C819E7F-7249-4689-B1D7-EDB910201738@gmail.com> References: <4B826FEF.4070605@univ-mlv.fr><17b2302a1002220949h5754454u6f7fe2603219c5b@mail.gmail.com> <39560A26-EFA0-4762-8B7D-28AE4B667B59@gmail.com><17b2302a1002221038l268dff63wba991fb9c225a4a8@mail.gmail.com> <4B82D1F9.5040907@univ-mlv.fr><17b2302a1002221128i41fa44fbp5cda06b6cc784f13@mail.gmail.com> <15e8b9d21002221148y8382f6dlb689117738118d92@mail.gmail.com> <17b2302a1002221219v1678540dw2a669f8dd2ebe734@mail.gmail.com> <15e8b9d21002221310x1f054fc2y4292022172d9abd6@mail.gmail.com> <9FF0EECF0E58D849A9D537E324DDDC0D0452EFA7@NZURC102PEX1.ubsw.net> <6C819E7F-7249-4689-B1D7-EDB910201738@gmail.com> Message-ID: <9FF0EECF0E58D849A9D537E324DDDC0D0452F0E1@NZURC102PEX1.ubsw.net> >> Personally, I would allow for 2 and 4 (and potentially 1, if lexical >> closures are a desired feature). > > The problem with 2 is that it only works for instance fields; the > problem with 4 is that it prevents passing the lambda by reference. > > We could create a new keyword, like 'lambda', which acts as the > self-lambda reference, but it seems a lot of work rather than just > re-targeting 'this' to refer to the enclosing lambda scope, and > allowing Outer.this to refer to the enclosing class instance. Lexical closures +1 New 'lambda'/'self'/'#' keyword +1 Optional name omission within function body +1 But, Retargeting 'this' -1 Because I think it is a really bad idea. Kind Regards, David. -- EUR2 EG.1168 [1923 96757] SST: DL-SST-DEV "Where's the Kaboom? There was supposed to be an Earth-shattering Kaboom!" - Marvin the Martian "Don't worry about what anybody else is going to do. The best way to predict the future is to invent it." - Alan Kay "there is this amazingly powerful thing writhing around in there that will basically do everything I could possibly ask of it if only I knew how." - Ben Butler-Cole Based on the present E-Mail exchange, and/or on the agreement reached with you, respectively, UBS is entitled to contact you via insecure E-Mail: (a) E-Mails contain substantial risks such as lack of confidentiality, manipulation of content and sender, misdirection, viruses etc. UBS does not accept any liability for damages arising from use of E-mail. Accordingly, UBS recommends to abstain from sending any sensitive information via E-Mail, from forwarding the text received when submitting reply E-Mails and recommends to manually capture the E-Mail address in every instance. If you should wish to verify the content of this message, please request a hard-copy version. (b) In principle, UBS does not accept any (purchase) orders, cancellation of orders or authorizations etc. via E-mail. If UBS receives such E-Mails, UBS is not obliged to expressly decline them. If you have received this E-Mail by mistake or do not wish to be contacted by E-Mail in the future, you are kindly asked to inform UBS accordingly. Any E-Mail received by mistake (including all its annexes) needs to be destroyed and the content may not be forwarded nor disclosed to any further persons. c) This message is provided for informational purposes and should not be construed as a solicitation or offer to buy or sell any securities or related financial instruments. UBS reserves the right to retain all messages. Messages are protected and accessed only in legally justified cases. From alex.blewitt at gmail.com Tue Feb 23 03:49:58 2010 From: alex.blewitt at gmail.com (Alex Blewitt) Date: Tue, 23 Feb 2010 11:49:58 +0000 Subject: Implementing recursive lambda with MethodHandle In-Reply-To: <9FF0EECF0E58D849A9D537E324DDDC0D0452F0E1@NZURC102PEX1.ubsw.net> References: <4B826FEF.4070605@univ-mlv.fr><17b2302a1002220949h5754454u6f7fe2603219c5b@mail.gmail.com> <39560A26-EFA0-4762-8B7D-28AE4B667B59@gmail.com><17b2302a1002221038l268dff63wba991fb9c225a4a8@mail.gmail.com> <4B82D1F9.5040907@univ-mlv.fr><17b2302a1002221128i41fa44fbp5cda06b6cc784f13@mail.gmail.com> <15e8b9d21002221148y8382f6dlb689117738118d92@mail.gmail.com> <17b2302a1002221219v1678540dw2a669f8dd2ebe734@mail.gmail.com> <15e8b9d21002221310x1f054fc2y4292022172d9abd6@mail.gmail.com> <9FF0EECF0E58D849A9D537E324DDDC0D0452EFA7@NZURC102PEX1.ubsw.net> <6C819E7F-7249-4689-B1D7-EDB910201738@gmail.com> <9FF0EECF0E58D849A9D537E324DDDC0D0452F0E1@NZURC102PEX1.ubsw.net> Message-ID: On 23 Feb 2010, at 11:22, wrote: > But, > Retargeting 'this' -1 > > Because I think it is a really bad idea. There are others that share your opinion on this list. :-) But if we've blown transparency with the lack of (non-final) variable capture, returns/continue/break acting differently from within a lambda than outside, there doesn't appear to be any argument against having 'this' as meaning something different inside a scope as well, other than personal expectations. Note also that existing Java developers are familiar with their being multiple 'this' inside nested inner classes, and I'm sure the same objections came up at that time. If we end up introducing the transparency for 'returns/continue/break' then I'm all for consistency in using 'this' to be transparently referring to the enclosing class instance. But equally, if we don't have transparency - and thus, can never surround a block of code without potentially inspecting it for changes - then it seems that we're suffering an awful lot of contortions based on personal opinion. Lastly, we still can't use 'this' transparently in the 0.1.5 spec, since 'this' is explicitly disallowed at the current time. Alex From peter.levart at marand.si Tue Feb 23 04:27:27 2010 From: peter.levart at marand.si (Peter Levart) Date: Tue, 23 Feb 2010 13:27:27 +0100 Subject: Implementing recursive lambda with MethodHandle In-Reply-To: References: <4B826FEF.4070605@univ-mlv.fr> <9FF0EECF0E58D849A9D537E324DDDC0D0452F0E1@NZURC102PEX1.ubsw.net> Message-ID: <201002231327.27880.peter.levart@marand.si> On Tuesday 23 February 2010 12:49:58 Alex Blewitt wrote: > On 23 Feb 2010, at 11:22, wrote: > > > But, > > Retargeting 'this' -1 > > > > Because I think it is a really bad idea. > > There are others that share your opinion on this list. :-) > > But if we've blown transparency with the lack of (non-final) variable capture, returns/continue/break acting differently from within a lambda than outside, These are all restrictions that make such code illegal (doesn't compile) and not, as you claim, "acting differently"... > there doesn't appear to be any argument against having 'this' as meaning something different inside a scope as well, other than personal expectations. Now this is something that will compile, one way or another. It's just semantics of "this" that are questioned. What's the difference? A big one. Restrictions can be lifted at a later time without backwards compatibility breakage, while the semantics of "this" can never change once defined. > > Lastly, we still can't use 'this' transparently in the 0.1.5 spec, since 'this' is explicitly disallowed at the current time. > Only in expression lambdas or return statements of statement lambdas, If I remember correctly. Regards, Peter From forax at univ-mlv.fr Tue Feb 23 05:23:35 2010 From: forax at univ-mlv.fr (=?ISO-8859-1?Q?R=E9mi_Forax?=) Date: Tue, 23 Feb 2010 14:23:35 +0100 Subject: Nice to @Share? In-Reply-To: <560fb5ed1002230203p94ff249j5e81c8bb1db99bfa@mail.gmail.com> References: <17b2302a1002221238p2db47f2albcd2e6bb23ddafab@mail.gmail.com> <4B839E5C.5010502@sun.com> <560fb5ed1002230203p94ff249j5e81c8bb1db99bfa@mail.gmail.com> Message-ID: <4B83D6D7.8030809@univ-mlv.fr> Le 23/02/2010 11:03, Reinier Zwitserloot a ?crit : > "public" as a keyword on local variable declarations and method parameters > avoids the need to mess with (restricted) new keywords, is backwards > compatible, Reiner, I think you can introduce any keyword before a local declaration without breaking compatibility. So the best is to choose a keyword that means what you really want. By the way, I am very happy with final (or implicit final). When I code and find that I need a non-final local variable, I first try to rewrite the code to avoid to use non-final before trying to use a mutable object. void foreach(List list, #void(T) fun) { for(String element: list) fun.invoke(element); } ... @shared int length = 0; foreach(alist, #(String element){ length += element.length; }); can be rewritten as: int reduce(List list, #int(T, int) fun, int acc) { for(String element: list) acc+= fun.invoke(element); } return acc; } ... int length = reduce(alist, #(String element, int acc) { return element.length + acc; }); In my opinion, @Shared is not that important if you provide APIs to easily play with collections. [...] > --Reinier Zwitserloot > R?mi From David.Moss at ubs.com Tue Feb 23 05:43:32 2010 From: David.Moss at ubs.com (David.Moss at ubs.com) Date: Tue, 23 Feb 2010 14:43:32 +0100 Subject: Nice to @Share? In-Reply-To: <4B83D6D7.8030809@univ-mlv.fr> References: <17b2302a1002221238p2db47f2albcd2e6bb23ddafab@mail.gmail.com> <4B839E5C.5010502@sun.com><560fb5ed1002230203p94ff249j5e81c8bb1db99bfa@mail.gmail.com> <4B83D6D7.8030809@univ-mlv.fr> Message-ID: <9FF0EECF0E58D849A9D537E324DDDC0D045C5151@NZURC102PEX1.ubsw.net> > By the way, I am very happy with final (or implicit final). > When I code and find that I need a non-final local variable, > I first try to rewrite the code to avoid to use non-final > before trying to use a mutable object. Where des this leave closures? i.e.: what would we need to make the following code do what it is meant to: public class C { public #int() getSequenceGenerator(int start, int increment) { return #int() { int ret = start; start += increment; return ret; }; } } Kind Regards, David. -- EUR2 EG.1168 [1923 96757] SST: DL-SST-DEV "Where's the Kaboom? There was supposed to be an Earth-shattering Kaboom!" - Marvin the Martian "Don't worry about what anybody else is going to do. The best way to predict the future is to invent it." - Alan Kay "there is this amazingly powerful thing writhing around in there that will basically do everything I could possibly ask of it if only I knew how." - Ben Butler-Cole Based on the present E-Mail exchange, and/or on the agreement reached with you, respectively, UBS is entitled to contact you via insecure E-Mail: (a) E-Mails contain substantial risks such as lack of confidentiality, manipulation of content and sender, misdirection, viruses etc. UBS does not accept any liability for damages arising from use of E-mail. Accordingly, UBS recommends to abstain from sending any sensitive information via E-Mail, from forwarding the text received when submitting reply E-Mails and recommends to manually capture the E-Mail address in every instance. If you should wish to verify the content of this message, please request a hard-copy version. (b) In principle, UBS does not accept any (purchase) orders, cancellation of orders or authorizations etc. via E-mail. If UBS receives such E-Mails, UBS is not obliged to expressly decline them. If you have received this E-Mail by mistake or do not wish to be contacted by E-Mail in the future, you are kindly asked to inform UBS accordingly. Any E-Mail received by mistake (including all its annexes) needs to be destroyed and the content may not be forwarded nor disclosed to any further persons. c) This message is provided for informational purposes and should not be construed as a solicitation or offer to buy or sell any securities or related financial instruments. UBS reserves the right to retain all messages. Messages are protected and accessed only in legally justified cases. From alex.blewitt at gmail.com Tue Feb 23 05:49:45 2010 From: alex.blewitt at gmail.com (Alex Blewitt) Date: Tue, 23 Feb 2010 13:49:45 +0000 Subject: Implementing recursive lambda with MethodHandle In-Reply-To: <201002231327.27880.peter.levart@marand.si> References: <4B826FEF.4070605@univ-mlv.fr> <9FF0EECF0E58D849A9D537E324DDDC0D0452F0E1@NZURC102PEX1.ubsw.net> <201002231327.27880.peter.levart@marand.si> Message-ID: <1C0A9286-265F-4F9A-942D-3CF0D546923E@gmail.com> On 23 Feb 2010, at 12:27, Peter Levart wrote: >> But if we've blown transparency with the lack of (non-final) variable capture, returns/continue/break acting differently from within a lambda than outside, > > These are all restrictions that make such code illegal (doesn't compile) and not, as you claim, "acting differently"... That's not strictly true. You can still use break/continue/returns inside the lambda; but they have potentially different meaning htan before. (Jumping to a label outside may be a compile error; but return-from-lambda will be compilable and be different than return-from-enclosing-emthod.) >> there doesn't appear to be any argument against having 'this' as meaning something different inside a scope as well, other than personal expectations. > > Now this is something that will compile, one way or another. It's just semantics of "this" that are questioned. Like the semantics of 'return' being questioned as well? >> Lastly, we still can't use 'this' transparently in the 0.1.5 spec, since 'this' is explicitly disallowed at the current time. > > Only in expression lambdas or return statements of statement lambdas, If I remember correctly. This still means there are code segments which cannot be converted into an (expression) lambda, so the transparency argument is already broken for those cases. Alex From Alex.Buckley at Sun.COM Tue Feb 23 05:51:37 2010 From: Alex.Buckley at Sun.COM (Alex Buckley) Date: Tue, 23 Feb 2010 05:51:37 -0800 Subject: Nice to @Share? In-Reply-To: <4B83D6D7.8030809@univ-mlv.fr> References: <17b2302a1002221238p2db47f2albcd2e6bb23ddafab@mail.gmail.com> <4B839E5C.5010502@sun.com> <560fb5ed1002230203p94ff249j5e81c8bb1db99bfa@mail.gmail.com> <4B83D6D7.8030809@univ-mlv.fr> Message-ID: <4B83DD69.2060207@sun.com> I would love to have a library of examples that show lambdas using tail recursion rather than imperative side effects. You get entry #0 and a credit in "The Java Programming Language (5th Edition)". Does anyone else have examples? Alex R?mi Forax wrote: > By the way, I am very happy with final (or implicit final). > When I code and find that I need a non-final local variable, > I first try to rewrite the code to avoid to use non-final > before trying to use a mutable object. > > void foreach(List list, #void(T) fun) { > for(String element: list) > fun.invoke(element); > } > ... > @shared int length = 0; > foreach(alist, #(String element){ > length += element.length; > }); > > can be rewritten as: > > int reduce(List list, #int(T, int) fun, int acc) { > for(String element: list) > acc+= fun.invoke(element); > } > return acc; > } > ... > int length = reduce(alist, #(String element, int acc) { > return element.length + acc; > }); > > In my opinion, @Shared is not that important if you provide APIs > to easily play with collections. > > [...] > >> --Reinier Zwitserloot >> > > R?mi > From Alex.Buckley at Sun.COM Tue Feb 23 06:00:08 2010 From: Alex.Buckley at Sun.COM (Alex Buckley) Date: Tue, 23 Feb 2010 06:00:08 -0800 Subject: Nice to @Share? In-Reply-To: <9FF0EECF0E58D849A9D537E324DDDC0D045C5151@NZURC102PEX1.ubsw.net> References: <17b2302a1002221238p2db47f2albcd2e6bb23ddafab@mail.gmail.com> <4B839E5C.5010502@sun.com> <560fb5ed1002230203p94ff249j5e81c8bb1db99bfa@mail.gmail.com> <4B83D6D7.8030809@univ-mlv.fr> <9FF0EECF0E58D849A9D537E324DDDC0D045C5151@NZURC102PEX1.ubsw.net> Message-ID: <4B83DF68.1000706@sun.com> I think the use of 'start' was rather what Fredrik warned about in http://blogs.oracle.com/ohrstrom/2009/08/using_methodhandles_to_reconci.html What possible effect can mutating 'start' have? Alex David.Moss at ubs.com wrote: >> By the way, I am very happy with final (or implicit final). >> When I code and find that I need a non-final local variable, >> I first try to rewrite the code to avoid to use non-final >> before trying to use a mutable object. > > Where des this leave closures? > > i.e.: what would we need to make the following code do what it is meant to: > > public class C { > public #int() getSequenceGenerator(int start, int increment) { > return #int() { > int ret = start; > start += increment; > return ret; > }; > } > } > > > Kind Regards, > > David. > > -- > EUR2 EG.1168 [1923 96757] > SST: DL-SST-DEV > > "Where's the Kaboom? There was supposed to be an Earth-shattering Kaboom!" - Marvin the Martian > > "Don't worry about what anybody else is going to do. The best way to predict the future is to invent it." - Alan Kay > > "there is this amazingly powerful thing writhing around in there that will basically do everything I could possibly ask of it if only I knew how." - Ben Butler-Cole > > Based on the present E-Mail exchange, and/or on the agreement reached > with you, respectively, UBS is entitled to contact you via insecure > E-Mail: > (a) E-Mails contain substantial risks such as lack of confidentiality, > manipulation of content and sender, misdirection, viruses etc. UBS > does not accept any liability for damages arising from use of > E-mail. Accordingly, UBS recommends to abstain from sending any > sensitive information via E-Mail, from forwarding the text received > when submitting reply E-Mails and recommends to manually capture the > E-Mail address in every instance. If you should wish to verify the > content of this message, please request a hard-copy version. > (b) In principle, UBS does not accept any (purchase) orders, > cancellation of orders or authorizations etc. via E-mail. If UBS > receives such E-Mails, UBS is not obliged to expressly decline them. > If you have received this E-Mail by mistake or do not wish to be > contacted by E-Mail in the future, you are kindly asked to inform UBS > accordingly. Any E-Mail received by mistake (including all its annexes) > needs to be destroyed and the content may not be forwarded nor disclosed > to any further persons. > c) This message is provided for informational purposes and should not be > construed as a solicitation or offer to buy or sell any securities or > related financial instruments. > > UBS reserves the right to retain all messages. Messages are protected > and accessed only in legally justified cases. > From David.Moss at ubs.com Tue Feb 23 06:21:24 2010 From: David.Moss at ubs.com (David.Moss at ubs.com) Date: Tue, 23 Feb 2010 15:21:24 +0100 Subject: Nice to @Share? In-Reply-To: <4B83DF68.1000706@sun.com> References: <17b2302a1002221238p2db47f2albcd2e6bb23ddafab@mail.gmail.com><4B839E5C.5010502@sun.com><560fb5ed1002230203p94ff249j5e81c8bb1db99bfa@mail.gmail.com><4B83D6D7.8030809@univ-mlv.fr><9FF0EECF0E58D849A9D537E324DDDC0D045C5151@NZURC102PEX1.ubsw.net> <4B83DF68.1000706@sun.com> Message-ID: <9FF0EECF0E58D849A9D537E324DDDC0D045C51B2@NZURC102PEX1.ubsw.net> > I think the use of 'start' was rather what Fredrik warned about in > http://blogs.oracle.com/ohrstrom/2009/08/using_methodhandles_to_reconci. html > > What possible effect can mutating 'start' have? > >> >> public class C { >> public #int() getSequenceGenerator(int start, int increment) { >> return #int() { >> int ret = start; >> start += increment; >> return ret; >> }; >> } >> } >> Simple example following from the example above: public class D { public static void main(String...args) { C c = new C(); #int() accumulator = c.getSequenceGenerator(1, 2); while(accumulator.() < 200) { // do some stuff 100 times. I know, silly // example, but you can see why a mutable // 'start' parameter is necessary in 'C'. } } } Kind Regards, David. -- EUR2 EG.1168 [1923 96757] SST: DL-SST-DEV "Where's the Kaboom? There was supposed to be an Earth-shattering Kaboom!" - Marvin the Martian "Don't worry about what anybody else is going to do. The best way to predict the future is to invent it." - Alan Kay "there is this amazingly powerful thing writhing around in there that will basically do everything I could possibly ask of it if only I knew how." - Ben Butler-Cole Based on the present E-Mail exchange, and/or on the agreement reached with you, respectively, UBS is entitled to contact you via insecure E-Mail: (a) E-Mails contain substantial risks such as lack of confidentiality, manipulation of content and sender, misdirection, viruses etc. UBS does not accept any liability for damages arising from use of E-mail. Accordingly, UBS recommends to abstain from sending any sensitive information via E-Mail, from forwarding the text received when submitting reply E-Mails and recommends to manually capture the E-Mail address in every instance. If you should wish to verify the content of this message, please request a hard-copy version. (b) In principle, UBS does not accept any (purchase) orders, cancellation of orders or authorizations etc. via E-mail. If UBS receives such E-Mails, UBS is not obliged to expressly decline them. If you have received this E-Mail by mistake or do not wish to be contacted by E-Mail in the future, you are kindly asked to inform UBS accordingly. Any E-Mail received by mistake (including all its annexes) needs to be destroyed and the content may not be forwarded nor disclosed to any further persons. c) This message is provided for informational purposes and should not be construed as a solicitation or offer to buy or sell any securities or related financial instruments. UBS reserves the right to retain all messages. Messages are protected and accessed only in legally justified cases. From peter.levart at marand.si Tue Feb 23 07:01:47 2010 From: peter.levart at marand.si (Peter Levart) Date: Tue, 23 Feb 2010 16:01:47 +0100 Subject: Implementing recursive lambda with MethodHandle In-Reply-To: <1C0A9286-265F-4F9A-942D-3CF0D546923E@gmail.com> References: <4B826FEF.4070605@univ-mlv.fr> <201002231327.27880.peter.levart@marand.si> <1C0A9286-265F-4F9A-942D-3CF0D546923E@gmail.com> Message-ID: <201002231601.47428.peter.levart@marand.si> On Tuesday 23 February 2010 14:49:45 Alex Blewitt wrote: > > On 23 Feb 2010, at 12:27, Peter Levart wrote: > > >> But if we've blown transparency with the lack of (non-final) variable capture, returns/continue/break acting differently from within a lambda than outside, > > > > These are all restrictions that make such code illegal (doesn't compile) and not, as you claim, "acting differently"... > > That's not strictly true. You can still use break/continue/returns inside the lambda; but they have potentially different meaning htan before. (Jumping to a label outside may be a compile error; but return-from-lambda will be compilable and be different than return-from-enclosing-emthod.) Yeah, the unfortunate return. > > >> there doesn't appear to be any argument against having 'this' as meaning something different inside a scope as well, other than personal expectations. > > > > Now this is something that will compile, one way or another. It's just semantics of "this" that are questioned. > > Like the semantics of 'return' being questioned as well? Unfortunately there is no elegant way to perform transparent early returns from lambdas other than inventing new keyword (which is still not transparent when nesting) or some awkward lambda-labeling mechanism with awkward labeled value returning construct (for example: return label : value;). There was a moment of "hope" when Alex Buckley considered dropping statement lambdas. Expression lambdas combined with block expressions (see CfJ 0.6b http://www.javac.info/closures-v06b.html) would be a way to go. Early returns from such block expressions could be performed by combining two existing Java features: Deffinite Assignment rules + labeled statement/labeled break. But as 0.1.5 spec stands currently, statement lambdas are still there with this/return emulating this/return in annon inner classes and block expression is not part of the spec. > > >> Lastly, we still can't use 'this' transparently in the 0.1.5 spec, since 'this' is explicitly disallowed at the current time. > > > > Only in expression lambdas or return statements of statement lambdas, If I remember correctly. > > This still means there are code segments which cannot be converted into an (expression) lambda, so the transparency argument is already broken for those cases. That's because "this" is refering to the lambda instance and therefore the compiler has trouble to infer lambda's return type. If it was refering to enclosing class instance then the restrictions to use "this" in expression lambdas and return statements of statement lambdas could be lifted. > > Alex Regards, Peter From fredrik.ohrstrom at oracle.com Tue Feb 23 07:13:38 2010 From: fredrik.ohrstrom at oracle.com (=?ISO-8859-1?Q?Fredrik_=D6hrstr=F6m?=) Date: Tue, 23 Feb 2010 16:13:38 +0100 Subject: Nice to @Share? In-Reply-To: <4B83DF68.1000706@sun.com> References: <17b2302a1002221238p2db47f2albcd2e6bb23ddafab@mail.gmail.com> <4B839E5C.5010502@sun.com> <560fb5ed1002230203p94ff249j5e81c8bb1db99bfa@mail.gmail.com> <4B83D6D7.8030809@univ-mlv.fr> <9FF0EECF0E58D849A9D537E324DDDC0D045C5151@NZURC102PEX1.ubsw.net> <4B83DF68.1000706@sun.com> Message-ID: <4B83F0A2.4050200@oracle.com> I cannot see that "start" is different from any other local variable that is expected to be mutable when accessed from within a closure. I.e. the bytecode will use getfield/putfield instead of aload/astore and the object that it will do getfield/putfield on must potentially be an environment object separate from the closure object. The initialization of the field must be a copy from the argument of course. This BGGA example shows that the environment object that contains start must exist separately from any closures created using that environment. (There are simpler situations where the closure object can serve as the env object.) When v == false, then the env will not be live after gen has exited. When v == true it will stay alive as long as the closure stays alive since the closure holds on to it. public class Test { public static void main(String... args) { {==>int} f = gen(42); System.err.println(""+f.invoke()); System.err.println(""+f.invoke()); } static volatile boolean v = false; static {==>int} gen(int start) { if (v) { return {==> start++; start }; } else { start++; System.err.println("start="+start); return null; } } } When v=true, it prints 43, 44 and when v=false it print 43 and throws NPE. I am not sure that I really warned about this, my point is that the environmental object is inevitable because closures == objects and vice versa. This is not necessarily a bad thing. //Fredrik Alex Buckley skrev: > I think the use of 'start' was rather what Fredrik warned about in > http://blogs.oracle.com/ohrstrom/2009/08/using_methodhandles_to_reconci.html > > What possible effect can mutating 'start' have? > > Alex > > David.Moss at ubs.com wrote: > >>> By the way, I am very happy with final (or implicit final). >>> When I code and find that I need a non-final local variable, >>> I first try to rewrite the code to avoid to use non-final >>> before trying to use a mutable object. >>> >> Where des this leave closures? >> >> i.e.: what would we need to make the following code do what it is meant to: >> >> public class C { >> public #int() getSequenceGenerator(int start, int increment) { >> return #int() { >> int ret = start; >> start += increment; >> return ret; >> }; >> } >> } >> >> >> Kind Regards, >> >> David. >> >> -- From mcnepp02 at googlemail.com Tue Feb 23 07:21:50 2010 From: mcnepp02 at googlemail.com (Gernot Neppert) Date: Tue, 23 Feb 2010 15:21:50 +0000 Subject: Nice to @Share? In-Reply-To: <9FF0EECF0E58D849A9D537E324DDDC0D045C5151@NZURC102PEX1.ubsw.net> References: <17b2302a1002221238p2db47f2albcd2e6bb23ddafab@mail.gmail.com> <4B839E5C.5010502@sun.com> <560fb5ed1002230203p94ff249j5e81c8bb1db99bfa@mail.gmail.com> <4B83D6D7.8030809@univ-mlv.fr> <9FF0EECF0E58D849A9D537E324DDDC0D045C5151@NZURC102PEX1.ubsw.net> Message-ID: <1f5b87141002230721m1be461a8nbecb331011427093@mail.gmail.com> 2010/2/23 : > i.e.: what would we need to make the following code do what it is meant to: > > public class C { > ? ?public #int() getSequenceGenerator(int start, int increment) { > ? ? ? ?return #int() { > ? ? ? ? ? ?int ret = start; > ? ? ? ? ? ?start += increment; > ? ? ? ? ? ?return ret; > ? ? ? ?}; > ? ?} > } public interface IntSequence { public int next(); } public class C { public IntSequence() getSequenceGenerator(final int start, final int increment) { return new IntSequence() { int current = start; public int next() { int ret = current; current += increment; return ret; }; } } The point I'm trying to make: The concept of a "Sequence generator" is obviously inherently stateful. Every invocation of its parameterless method is meant to yield a different value. IMHO, this is best modelled by means of a designated interface, not by a lambda. From peter.levart at marand.si Tue Feb 23 07:32:45 2010 From: peter.levart at marand.si (Peter Levart) Date: Tue, 23 Feb 2010 16:32:45 +0100 Subject: Nice to @Share? In-Reply-To: <9FF0EECF0E58D849A9D537E324DDDC0D045C5151@NZURC102PEX1.ubsw.net> References: <17b2302a1002221238p2db47f2albcd2e6bb23ddafab@mail.gmail.com> <4B83D6D7.8030809@univ-mlv.fr> <9FF0EECF0E58D849A9D537E324DDDC0D045C5151@NZURC102PEX1.ubsw.net> Message-ID: <201002231632.45394.peter.levart@marand.si> On Tuesday 23 February 2010 14:43:32 David.Moss at ubs.com wrote: > > By the way, I am very happy with final (or implicit final). > > When I code and find that I need a non-final local variable, > > I first try to rewrite the code to avoid to use non-final > > before trying to use a mutable object. > > Where des this leave closures? > > i.e.: what would we need to make the following code do what it is meant to: > > public class C { > public #int() getSequenceGenerator(int start, int increment) { > return #int() { > int ret = start; > start += increment; > return ret; > }; > } > } > Perhaps, diagonal to what Josh suggested about having special syntax to support producing instances of SAM subclasses, e.g.: #Callable(String s) { System.out.println(s); } we might need special syntax to support producing instances of function type but with full class-like features: public class C { public #int() getSequenceGenerator(final int start, final int increment) { return new #int() { int seq = start; { try { return seq; } finally { seq += increment; } } }; } } > > Kind Regards, > > David. > Regards, Peter From David.Moss at ubs.com Tue Feb 23 07:30:35 2010 From: David.Moss at ubs.com (David.Moss at ubs.com) Date: Tue, 23 Feb 2010 16:30:35 +0100 Subject: Nice to @Share? In-Reply-To: <1f5b87141002230721m1be461a8nbecb331011427093@mail.gmail.com> References: <17b2302a1002221238p2db47f2albcd2e6bb23ddafab@mail.gmail.com><4B839E5C.5010502@sun.com><560fb5ed1002230203p94ff249j5e81c8bb1db99bfa@mail.gmail.com><4B83D6D7.8030809@univ-mlv.fr><9FF0EECF0E58D849A9D537E324DDDC0D045C5151@NZURC102PEX1.ubsw.net> <1f5b87141002230721m1be461a8nbecb331011427093@mail.gmail.com> Message-ID: <9FF0EECF0E58D849A9D537E324DDDC0D045C5291@NZURC102PEX1.ubsw.net> > The point I'm trying to make: > The concept of a "Sequence generator" is obviously inherently > stateful. Every invocation of its parameterless method is > meant to yield a different value. IMHO, this is best modelled > by means of a designated interface, not by a lambda. The point I'm trying to make is that closures need to carry an environment, and that implies that they are able to hold state. I chose a sequence generator to demonstrate this, but the argument is not about the example, but rather of how the implications (stateful closures) are to be implemented in a meaningful manner. Kind Regards, David. -- EUR2 EG.1168 [1923 96757] SST: DL-SST-DEV "Where's the Kaboom? There was supposed to be an Earth-shattering Kaboom!" - Marvin the Martian "Don't worry about what anybody else is going to do. The best way to predict the future is to invent it." - Alan Kay "there is this amazingly powerful thing writhing around in there that will basically do everything I could possibly ask of it if only I knew how." - Ben Butler-Cole Based on the present E-Mail exchange, and/or on the agreement reached with you, respectively, UBS is entitled to contact you via insecure E-Mail: (a) E-Mails contain substantial risks such as lack of confidentiality, manipulation of content and sender, misdirection, viruses etc. UBS does not accept any liability for damages arising from use of E-mail. Accordingly, UBS recommends to abstain from sending any sensitive information via E-Mail, from forwarding the text received when submitting reply E-Mails and recommends to manually capture the E-Mail address in every instance. If you should wish to verify the content of this message, please request a hard-copy version. (b) In principle, UBS does not accept any (purchase) orders, cancellation of orders or authorizations etc. via E-mail. If UBS receives such E-Mails, UBS is not obliged to expressly decline them. If you have received this E-Mail by mistake or do not wish to be contacted by E-Mail in the future, you are kindly asked to inform UBS accordingly. Any E-Mail received by mistake (including all its annexes) needs to be destroyed and the content may not be forwarded nor disclosed to any further persons. c) This message is provided for informational purposes and should not be construed as a solicitation or offer to buy or sell any securities or related financial instruments. UBS reserves the right to retain all messages. Messages are protected and accessed only in legally justified cases. From neal at gafter.com Tue Feb 23 08:03:38 2010 From: neal at gafter.com (Neal Gafter) Date: Tue, 23 Feb 2010 08:03:38 -0800 Subject: Nice to @Share? In-Reply-To: <4B83B94D.2070504@sun.com> References: <17b2302a1002221238p2db47f2albcd2e6bb23ddafab@mail.gmail.com> <4B839E5C.5010502@sun.com> <15e8b9d21002230139k561e32d5x8454a1e69350c868@mail.gmail.com> <4B83B94D.2070504@sun.com> Message-ID: <15e8b9d21002230803x561358c5p94c60614cc01d358@mail.gmail.com> On Tue, Feb 23, 2010 at 3:17 AM, Alex Buckley wrote: > Neal Gafter wrote: >> The definition of "local variable" in (?4.12.3) is ... > >> The only part of that requiring rewording in the presence of lambdas >> that can access local variables from the enclosing scope is the last >> sentence. ?The rewording is required whether or not such variables are >> final. ?In fact, that last sentence appears to conflict with the fact >> that final variables can be accessed in an anonymous inner class after >> the scope containing the variable has completed. > > Consider also JLS3 17.4.1: "Local variables ... are never shared between > threads and are unaffected by the memory model." A variable hoisted on > to the heap is not a local variable, even if its declaration form looks > similar to a local variable declaration statement. For example, you > could morally have a volatile shared variable, whereas you cannot have a > volatile local variable. Right... that sentence would have to be removed, and "volatile" allowed on local variables. From peter.levart at marand.si Tue Feb 23 08:23:09 2010 From: peter.levart at marand.si (Peter Levart) Date: Tue, 23 Feb 2010 17:23:09 +0100 Subject: Nice to @Share? In-Reply-To: <9FF0EECF0E58D849A9D537E324DDDC0D045C5151@NZURC102PEX1.ubsw.net> References: <17b2302a1002221238p2db47f2albcd2e6bb23ddafab@mail.gmail.com> <4B83D6D7.8030809@univ-mlv.fr> <9FF0EECF0E58D849A9D537E324DDDC0D045C5151@NZURC102PEX1.ubsw.net> Message-ID: <201002231723.09369.peter.levart@marand.si> On Tuesday 23 February 2010 14:43:32 David.Moss at ubs.com wrote: > > By the way, I am very happy with final (or implicit final). > > When I code and find that I need a non-final local variable, > > I first try to rewrite the code to avoid to use non-final > > before trying to use a mutable object. > > Where des this leave closures? > > i.e.: what would we need to make the following code do what it is meant to: > > public class C { > public #int() getSequenceGenerator(int start, int increment) { > return #int() { > int ret = start; > start += increment; > return ret; > }; > } > } > > Well, there is a workarround if you want to capture arbitrary state by copy and still not have to define any extra classes or stash each wariable in it's own wrapper: public class C { public #int() getSequenceGenerator(final int start, final int increment) { return new Object() { int seq = start; final #int() fn = #() { try { return seq; } finally { seq+=increment; } } }.fn; } } The benefit being that it is obvious what code does... Peter From David.Moss at ubs.com Tue Feb 23 08:30:24 2010 From: David.Moss at ubs.com (David.Moss at ubs.com) Date: Tue, 23 Feb 2010 17:30:24 +0100 Subject: Nice to @Share? In-Reply-To: <201002231723.09369.peter.levart@marand.si> References: <17b2302a1002221238p2db47f2albcd2e6bb23ddafab@mail.gmail.com> <4B83D6D7.8030809@univ-mlv.fr> <9FF0EECF0E58D849A9D537E324DDDC0D045C5151@NZURC102PEX1.ubsw.net> <201002231723.09369.peter.levart@marand.si> Message-ID: <9FF0EECF0E58D849A9D537E324DDDC0D045C5332@NZURC102PEX1.ubsw.net> > The benefit being that it is obvious what code does... I'd argue that it is obscuring (in a very nasty way) what the intention of the closure actually is. Kind Regards, David. -- EUR2 EG.1168 [1923 96757] SST: DL-SST-DEV "Where's the Kaboom? There was supposed to be an Earth-shattering Kaboom!" - Marvin the Martian "Don't worry about what anybody else is going to do. The best way to predict the future is to invent it." - Alan Kay "there is this amazingly powerful thing writhing around in there that will basically do everything I could possibly ask of it if only I knew how." - Ben Butler-Cole Based on the present E-Mail exchange, and/or on the agreement reached with you, respectively, UBS is entitled to contact you via insecure E-Mail: (a) E-Mails contain substantial risks such as lack of confidentiality, manipulation of content and sender, misdirection, viruses etc. UBS does not accept any liability for damages arising from use of E-mail. Accordingly, UBS recommends to abstain from sending any sensitive information via E-Mail, from forwarding the text received when submitting reply E-Mails and recommends to manually capture the E-Mail address in every instance. If you should wish to verify the content of this message, please request a hard-copy version. (b) In principle, UBS does not accept any (purchase) orders, cancellation of orders or authorizations etc. via E-mail. If UBS receives such E-Mails, UBS is not obliged to expressly decline them. If you have received this E-Mail by mistake or do not wish to be contacted by E-Mail in the future, you are kindly asked to inform UBS accordingly. Any E-Mail received by mistake (including all its annexes) needs to be destroyed and the content may not be forwarded nor disclosed to any further persons. c) This message is provided for informational purposes and should not be construed as a solicitation or offer to buy or sell any securities or related financial instruments. UBS reserves the right to retain all messages. Messages are protected and accessed only in legally justified cases. From peter.levart at marand.si Tue Feb 23 08:41:32 2010 From: peter.levart at marand.si (Peter Levart) Date: Tue, 23 Feb 2010 17:41:32 +0100 Subject: Look ma, recursive lambda without "this" or bending the rules Message-ID: <201002231741.32923.peter.levart@marand.si> Just for fun... #int(int) factorial = new Object() { #int(int) f; { f = #(int x)(x == 0 ? 1 : x*f.(x-1)); } }.f; Peter From neal at gafter.com Tue Feb 23 09:03:27 2010 From: neal at gafter.com (Neal Gafter) Date: Tue, 23 Feb 2010 09:03:27 -0800 Subject: Nice to @Share? In-Reply-To: <4B83B94D.2070504@sun.com> References: <17b2302a1002221238p2db47f2albcd2e6bb23ddafab@mail.gmail.com> <4B839E5C.5010502@sun.com> <15e8b9d21002230139k561e32d5x8454a1e69350c868@mail.gmail.com> <4B83B94D.2070504@sun.com> Message-ID: <15e8b9d21002230903k55fc56cav63d1736b64768a8@mail.gmail.com> On Tue, Feb 23, 2010 at 3:17 AM, Alex Buckley wrote: > Consider also JLS3 17.4.1: "Local variables ... are never shared between > threads and are unaffected by the memory model." On second thought, that sentence appears to be a spec bug. Local variables can indeed be shared between threads, though today that requires the variable be final and used in an inner class. The memory model doesn't say what the behavior should be (another bug), but I expect section 17.5 should be generalized to talk about "final variables" instead of just "final fields". Otherwise the JLS is simply not correct today. Your description of local variables being "stack allocated" does not match the JLS description. According to the JLS, local variables are allocated in the activation frame. That may be a stack, a heap, or some combination of the two. There is no requirement regarding the timing of deallocation of these frames. From Alex.Buckley at Sun.COM Tue Feb 23 09:34:16 2010 From: Alex.Buckley at Sun.COM (Alex Buckley) Date: Tue, 23 Feb 2010 09:34:16 -0800 Subject: Nice to @Share? In-Reply-To: <4B83F0A2.4050200@oracle.com> References: <17b2302a1002221238p2db47f2albcd2e6bb23ddafab@mail.gmail.com> <4B839E5C.5010502@sun.com> <560fb5ed1002230203p94ff249j5e81c8bb1db99bfa@mail.gmail.com> <4B83D6D7.8030809@univ-mlv.fr> <9FF0EECF0E58D849A9D537E324DDDC0D045C5151@NZURC102PEX1.ubsw.net> <4B83DF68.1000706@sun.com> <4B83F0A2.4050200@oracle.com> Message-ID: <4B841198.7000703@sun.com> You are right. I did not ask a good question, but you (and David and Gernot and Peter) have provided good answers. Alex Fredrik ?hrstr?m wrote: > I cannot see that "start" is different from any other local variable > that is expected to be mutable when accessed from within a closure. I.e. > the bytecode will use getfield/putfield instead of aload/astore and the > object that it will do getfield/putfield on must potentially be an > environment object separate from the closure object. The initialization > of the field must be a copy from the argument of course. > > This BGGA example shows that the environment object that contains start > must exist separately from any closures created using that environment. > (There are simpler situations where the closure object can serve as the > env object.) When v == false, then the env will not be live after gen > has exited. When v == true it will stay alive as long as the closure > stays alive since the closure holds on to it. > > public class Test > { > public static void main(String... args) > { > {==>int} f = gen(42); > System.err.println(""+f.invoke()); > System.err.println(""+f.invoke()); > } > > static volatile boolean v = false; > > static {==>int} gen(int start) { > > if (v) { > return {==> start++; start }; > } else { > start++; > System.err.println("start="+start); > return null; > } > } > } > > When v=true, it prints 43, 44 and when v=false it print 43 and throws NPE. > > I am not sure that I really warned about this, my point is that the > environmental object is inevitable because closures == objects and vice > versa. This is not necessarily a bad thing. > > //Fredrik > > Alex Buckley skrev: >> I think the use of 'start' was rather what Fredrik warned about in >> http://blogs.oracle.com/ohrstrom/2009/08/using_methodhandles_to_reconci.html >> >> What possible effect can mutating 'start' have? >> >> Alex >> >> David.Moss at ubs.com wrote: >> >>>> By the way, I am very happy with final (or implicit final). >>>> When I code and find that I need a non-final local variable, >>>> I first try to rewrite the code to avoid to use non-final >>>> before trying to use a mutable object. >>>> >>> Where des this leave closures? >>> >>> i.e.: what would we need to make the following code do what it is meant to: >>> >>> public class C { >>> public #int() getSequenceGenerator(int start, int increment) { >>> return #int() { >>> int ret = start; >>> start += increment; >>> return ret; >>> }; >>> } >>> } >>> >>> >>> Kind Regards, >>> >>> David. >>> >>> -- > From alex.blewitt at gmail.com Tue Feb 23 10:03:28 2010 From: alex.blewitt at gmail.com (Alex Blewitt) Date: Tue, 23 Feb 2010 18:03:28 +0000 Subject: Look ma, recursive lambda without "this" or bending the rules In-Reply-To: <201002231741.32923.peter.levart@marand.si> References: <201002231741.32923.peter.levart@marand.si> Message-ID: On 23 Feb 2010, at 16:41, Peter Levart wrote: > Just for fun... Wow. And people are complaining that "final" is too much to type ... and, as if that wasn't enough, creating a new instance and anonymous class to do your bidding. It's a pity there isn't some four-character string we can't use for this. > Alex From john at milsson.nu Tue Feb 23 11:06:17 2010 From: john at milsson.nu (John Nilsson) Date: Tue, 23 Feb 2010 20:06:17 +0100 Subject: Implementing recursive lambda with MethodHandle In-Reply-To: <6C819E7F-7249-4689-B1D7-EDB910201738@gmail.com> References: <4B826FEF.4070605@univ-mlv.fr> <17b2302a1002221038l268dff63wba991fb9c225a4a8@mail.gmail.com> <4B82D1F9.5040907@univ-mlv.fr> <17b2302a1002221128i41fa44fbp5cda06b6cc784f13@mail.gmail.com> <15e8b9d21002221148y8382f6dlb689117738118d92@mail.gmail.com> <17b2302a1002221219v1678540dw2a669f8dd2ebe734@mail.gmail.com> <15e8b9d21002221310x1f054fc2y4292022172d9abd6@mail.gmail.com> <9FF0EECF0E58D849A9D537E324DDDC0D0452EFA7@NZURC102PEX1.ubsw.net> <6C819E7F-7249-4689-B1D7-EDB910201738@gmail.com> Message-ID: On Tue, Feb 23, 2010 at 12:03 PM, Alex Blewitt wrote: > We could create a new keyword, like 'lambda', which acts as the self-lambda > reference, but it seems a lot of work rather than just re-targeting 'this' > to refer to the enclosing lambda scope, and allowing Outer.this to refer to > the enclosing class instance. > Has it been decided that lambdas will be instances of Object? To me it seems that a lambda is fundamentally different from an object which in it self could warrant a different keyword than this. I'd vote for . (period) being that keyword. #int(int) lambda = .; This could also be extended to a nice symetric syntax for lifting methods to lambda-references. #int(int) lambda = SomeClass.someStaticMethod; or #int(int) lambda = .someMethodInScope; BR, John From jjb at google.com Tue Feb 23 11:08:10 2010 From: jjb at google.com (Joshua Bloch) Date: Tue, 23 Feb 2010 11:08:10 -0800 Subject: Nice to @Share? In-Reply-To: <15e8b9d21002230139k561e32d5x8454a1e69350c868@mail.gmail.com> References: <17b2302a1002221238p2db47f2albcd2e6bb23ddafab@mail.gmail.com> <4B839E5C.5010502@sun.com> <15e8b9d21002230139k561e32d5x8454a1e69350c868@mail.gmail.com> Message-ID: <17b2302a1002231108o151a64fcje3c853478eb8ee65@mail.gmail.com> Neal, On Tue, Feb 23, 2010 at 1:39 AM, Neal Gafter wrote: The definition of "local variable" in (?4.12.3) is "Local variables > are declared by local variable declaration statements (?14.4). > ....... The local variable effectively ceases to > exist when the execution of the block or for statement is complete." > > The only part of that requiring rewording in the presence of lambdas > that can access local variables from the enclosing scope is the last > sentence. It's only one sentence, but it's a *huge* conceptual change, and not one I see as justifiable. > > A new keyword, while better than what programmers have to do today, is > still boilerplate. > No it isn't. If we decide that lambdas are permitted to reference mutable local variables (which I believe to be a bad idea), the code should contain a glaring visual cue that this is being done. Boilerplate is junk that you'd rather not see but the language requires. This is something I'd desperately want to see. It's like the volatile keyword, only moreso; it means "dragons lie here." That said, I'm more than happy with AtomicXxxx or the single-element array hack. Josh From neal at gafter.com Tue Feb 23 11:17:31 2010 From: neal at gafter.com (Neal Gafter) Date: Tue, 23 Feb 2010 11:17:31 -0800 Subject: Nice to @Share? In-Reply-To: <17b2302a1002231108o151a64fcje3c853478eb8ee65@mail.gmail.com> References: <17b2302a1002221238p2db47f2albcd2e6bb23ddafab@mail.gmail.com> <4B839E5C.5010502@sun.com> <15e8b9d21002230139k561e32d5x8454a1e69350c868@mail.gmail.com> <17b2302a1002231108o151a64fcje3c853478eb8ee65@mail.gmail.com> Message-ID: <15e8b9d21002231117y760dc081me52dc4587ec8fb9f@mail.gmail.com> On Tue, Feb 23, 2010 at 11:08 AM, Joshua Bloch wrote: > Neal, > > On Tue, Feb 23, 2010 at 1:39 AM, Neal Gafter wrote: > > The definition of "local variable" in (?4.12.3) is "Local variables >> are declared by local variable declaration statements (?14.4). >> ....... The local variable effectively ceases to >> exist when the execution of the block or for statement is complete." >> >> The only part of that requiring rewording in the presence of lambdas >> that can access local variables from the enclosing scope is the last >> sentence. > > > It's only one sentence, but it's a *huge* conceptual change, and not one I > see as justifiable. Too bad, because the sentence is already incorrect. Local variables that happen to be final and used in an inner class do not cease to exist when the execution of the block is complete. Removing the "final" requirement doesn't change whether or not that sentence is correct. From jjb at google.com Tue Feb 23 11:19:50 2010 From: jjb at google.com (Joshua Bloch) Date: Tue, 23 Feb 2010 11:19:50 -0800 Subject: Implementing recursive lambda with MethodHandle In-Reply-To: References: <4B826FEF.4070605@univ-mlv.fr> <4B82D1F9.5040907@univ-mlv.fr> <17b2302a1002221128i41fa44fbp5cda06b6cc784f13@mail.gmail.com> <15e8b9d21002221148y8382f6dlb689117738118d92@mail.gmail.com> <17b2302a1002221219v1678540dw2a669f8dd2ebe734@mail.gmail.com> <15e8b9d21002221310x1f054fc2y4292022172d9abd6@mail.gmail.com> <9FF0EECF0E58D849A9D537E324DDDC0D0452EFA7@NZURC102PEX1.ubsw.net> <6C819E7F-7249-4689-B1D7-EDB910201738@gmail.com> Message-ID: <17b2302a1002231119k2c7ded43obaf34f2ae8940692@mail.gmail.com> John, On Tue, Feb 23, 2010 at 11:06 AM, John Nilsson wrote: > On Tue, Feb 23, 2010 at 12:03 PM, Alex Blewitt >wrote: > > > We could create a new keyword, like 'lambda', which acts as the > self-lambda > > reference, but it seems a lot of work rather than just re-targeting > 'this' > > to refer to the enclosing lambda scope, and allowing Outer.this to refer > to > > the enclosing class instance. > > > > Has it been decided that lambdas will be instances of Object? To me it > seems > that a lambda is fundamentally different from an object I respectfully disagree. Java already has a bipartite type system (with object rerference types and primitive types). The *last* thing we should do is to make it tripartite. People already complain about the primitive/object distinction. We must avoid adding significant new complexity to an already complex language. Java's type system is among the most delicate and complex parts of the language. Anything we add has to interoperate seamlessly and predictably with what's already there/ Josh From neal at gafter.com Tue Feb 23 11:23:06 2010 From: neal at gafter.com (Neal Gafter) Date: Tue, 23 Feb 2010 11:23:06 -0800 Subject: Implementing recursive lambda with MethodHandle In-Reply-To: <17b2302a1002231119k2c7ded43obaf34f2ae8940692@mail.gmail.com> References: <4B826FEF.4070605@univ-mlv.fr> <17b2302a1002221128i41fa44fbp5cda06b6cc784f13@mail.gmail.com> <15e8b9d21002221148y8382f6dlb689117738118d92@mail.gmail.com> <17b2302a1002221219v1678540dw2a669f8dd2ebe734@mail.gmail.com> <15e8b9d21002221310x1f054fc2y4292022172d9abd6@mail.gmail.com> <9FF0EECF0E58D849A9D537E324DDDC0D0452EFA7@NZURC102PEX1.ubsw.net> <6C819E7F-7249-4689-B1D7-EDB910201738@gmail.com> <17b2302a1002231119k2c7ded43obaf34f2ae8940692@mail.gmail.com> Message-ID: <15e8b9d21002231123t47bc8904h3784aaa83de429d@mail.gmail.com> On Tue, Feb 23, 2010 at 11:19 AM, Joshua Bloch wrote: > On Tue, Feb 23, 2010 at 11:06 AM, John Nilsson wrote: >> Has it been decided that lambdas will be instances of Object? To me it >> seems >> that a lambda is fundamentally different from an object > > > I respectfully disagree. ?Java already has a bipartite type system (with > object rerference types and primitive types). ?The *last* thing we should do > is to make it tripartite. ?People already complain about the > primitive/object distinction. ?We must avoid adding significant new > complexity to an already complex language. Java's type system is among the > most delicate and complex parts of the language. Anything we add has to > interoperate seamlessly and predictably with what's already there/ I think John was saying that lambdas are fundamentally different from object creation expressions (e.g. anonymous inner classes), which I agree with. I also agree with you that function types must be reference types. From dl at cs.oswego.edu Tue Feb 23 11:51:41 2010 From: dl at cs.oswego.edu (Doug Lea) Date: Tue, 23 Feb 2010 14:51:41 -0500 Subject: Nice to @Share? In-Reply-To: <15e8b9d21002230903k55fc56cav63d1736b64768a8@mail.gmail.com> References: <17b2302a1002221238p2db47f2albcd2e6bb23ddafab@mail.gmail.com> <4B839E5C.5010502@sun.com> <15e8b9d21002230139k561e32d5x8454a1e69350c868@mail.gmail.com> <4B83B94D.2070504@sun.com> <15e8b9d21002230903k55fc56cav63d1736b64768a8@mail.gmail.com> Message-ID: <4B8431CD.3050004@cs.oswego.edu> On 02/23/10 12:03, Neal Gafter wrote: > On Tue, Feb 23, 2010 at 3:17 AM, Alex Buckley wrote: >> Consider also JLS3 17.4.1: "Local variables ... are never shared between >> threads and are unaffected by the memory model." > > On second thought, that sentence appears to be a spec bug. Local > variables can indeed be shared between threads, though today that > requires the variable be final and used in an inner class. The memory > model doesn't say what the behavior should be (another bug), but I > expect section 17.5 should be generalized to talk about "final > variables" instead of just "final fields". Otherwise the JLS is > simply not correct today. > One cannot define a processor- or language-level memory model without reference to a category of variables that are intrinsically local to a processor/thread. Without this, the memory model would need to be defined at least one level lower, via bytecode or register mappings, which would not be doing anyone a favor. So if the syntactic category of "local variables" isn't sufficiently precise, the JLS should be clarified. One alternative is to document the translation scheme for introducing new variables in inner classes with capturing values, in which case case you wouldn't need to call these "locals", and could deduce their safety by virtue of the other JMM rules. Of course, the absolute knowledge that a local is in fact thread-private is critical in implementing many concurrent algorithms. It is frightening to contemplate how to program without this. -Doug From neal at gafter.com Tue Feb 23 12:06:29 2010 From: neal at gafter.com (Neal Gafter) Date: Tue, 23 Feb 2010 12:06:29 -0800 Subject: Nice to @Share? In-Reply-To: <4B8431CD.3050004@cs.oswego.edu> References: <17b2302a1002221238p2db47f2albcd2e6bb23ddafab@mail.gmail.com> <4B839E5C.5010502@sun.com> <15e8b9d21002230139k561e32d5x8454a1e69350c868@mail.gmail.com> <4B83B94D.2070504@sun.com> <15e8b9d21002230903k55fc56cav63d1736b64768a8@mail.gmail.com> <4B8431CD.3050004@cs.oswego.edu> Message-ID: <15e8b9d21002231206l24dbe186jb3c7e9d341deb310@mail.gmail.com> On Tue, Feb 23, 2010 at 11:51 AM, Doug Lea
wrote: > One cannot define a processor- or language-level memory > model without reference to a category of variables that are > intrinsically local to a processor/thread. Why not? It seems to me that widening the scope of the description of "shared variables" (17.4.1) to include all variables - even those that are not actually shared between threads - gives precisely the correct semantics: if the variable is only used in one thread, then references to it are intra-thread actions and therefore don't cause conflicts (17.4.2); on the other hand if the variable is final (e.g. a local variable used in an anonymous class) then the guarantees of "final variable semantics" (17.5) would apply. Moreover, this simplifies the specification, because the distinction need no longer be made. And in the process of simplifying it, the specification goes from being incorrect and incomplete (for the case of locals used in anonymous classes) to correct and more complete. > Of course, the absolute knowledge that a local is in fact > thread-private is critical in implementing many concurrent > algorithms. It is frightening to contemplate how to > program ?without this. None of those concurrent algorithms would be undermined in any way by allowing lambdas, or anonymous inner classes, to share local variables from enclosing scopes. From dl at cs.oswego.edu Tue Feb 23 12:20:33 2010 From: dl at cs.oswego.edu (Doug Lea) Date: Tue, 23 Feb 2010 15:20:33 -0500 Subject: Nice to @Share? In-Reply-To: <15e8b9d21002231206l24dbe186jb3c7e9d341deb310@mail.gmail.com> References: <17b2302a1002221238p2db47f2albcd2e6bb23ddafab@mail.gmail.com> <4B839E5C.5010502@sun.com> <15e8b9d21002230139k561e32d5x8454a1e69350c868@mail.gmail.com> <4B83B94D.2070504@sun.com> <15e8b9d21002230903k55fc56cav63d1736b64768a8@mail.gmail.com> <4B8431CD.3050004@cs.oswego.edu> <15e8b9d21002231206l24dbe186jb3c7e9d341deb310@mail.gmail.com> Message-ID: <4B843891.4080002@cs.oswego.edu> On 02/23/10 15:06, Neal Gafter wrote: > On Tue, Feb 23, 2010 at 11:51 AM, Doug Lea
wrote: >> One cannot define a processor- or language-level memory >> model without reference to a category of variables that are >> intrinsically local to a processor/thread. > > Why not? Axiomatic locality is needed for the sake of modular specification and reasoning. Otherwise memory properties could only be validated in the presence of global program analyses proving that a variable is only accessed in one thread. And given dynamic loading, one cannot in general perform global analysis. > >> Of course, the absolute knowledge that a local is in fact >> thread-private is critical in implementing many concurrent >> algorithms. It is frightening to contemplate how to >> program without this. > > None of those concurrent algorithms would be undermined in any way by > allowing lambdas, or anonymous inner classes, to share local variables > from enclosing scopes. > But rarely verifiably or provably so. -Doug From neal at gafter.com Tue Feb 23 12:46:30 2010 From: neal at gafter.com (Neal Gafter) Date: Tue, 23 Feb 2010 12:46:30 -0800 Subject: Nice to @Share? In-Reply-To: <4B843891.4080002@cs.oswego.edu> References: <17b2302a1002221238p2db47f2albcd2e6bb23ddafab@mail.gmail.com> <4B839E5C.5010502@sun.com> <15e8b9d21002230139k561e32d5x8454a1e69350c868@mail.gmail.com> <4B83B94D.2070504@sun.com> <15e8b9d21002230903k55fc56cav63d1736b64768a8@mail.gmail.com> <4B8431CD.3050004@cs.oswego.edu> <15e8b9d21002231206l24dbe186jb3c7e9d341deb310@mail.gmail.com> <4B843891.4080002@cs.oswego.edu> Message-ID: <15e8b9d21002231246w659aa3b5i8167c48338953a3a@mail.gmail.com> On Tue, Feb 23, 2010 at 12:20 PM, Doug Lea
wrote: >>> One cannot define a processor- or language-level memory >>> model without reference to a category of variables that are >>> intrinsically local to a processor/thread. >> >> Why not? > > Axiomatic locality is needed for the sake of modular > specification and reasoning. Otherwise memory properties > could only be validated in the presence of global program > analyses proving that a variable is only accessed in one > thread. And given dynamic loading, one cannot in general > perform global analysis. In the case of local variables that are *not* accessed within inner classes or lambdas, or which are declared final, no axioms are required. Neither is global analysis required. Nor is the reasoning undermined by dynamic loading. So neither modular specification nor modular reasoning is undermined by treating locals the same as fields in the memory model. The specification would be simpler, and so reasoning would be simpler. >>> Of course, the absolute knowledge that a local is in fact >>> thread-private is critical in implementing many concurrent >>> algorithms. It is frightening to contemplate how to >>> program ?without this. >> >> None of those concurrent algorithms would be undermined in any way by >> allowing lambdas, or anonymous inner classes, to share local variables >> from enclosing scopes. > > But rarely verifiably or provably so. Not rarely - always! Fields that are not used in lambdas or inner classes are verifiably and provably not shared between threads. Final variables have guarantees in the memory model too. These properties are locally verifiable and provable always. From dl at cs.oswego.edu Tue Feb 23 12:57:23 2010 From: dl at cs.oswego.edu (Doug Lea) Date: Tue, 23 Feb 2010 15:57:23 -0500 Subject: Nice to @Share? In-Reply-To: <15e8b9d21002231246w659aa3b5i8167c48338953a3a@mail.gmail.com> References: <17b2302a1002221238p2db47f2albcd2e6bb23ddafab@mail.gmail.com> <4B839E5C.5010502@sun.com> <15e8b9d21002230139k561e32d5x8454a1e69350c868@mail.gmail.com> <4B83B94D.2070504@sun.com> <15e8b9d21002230903k55fc56cav63d1736b64768a8@mail.gmail.com> <4B8431CD.3050004@cs.oswego.edu> <15e8b9d21002231206l24dbe186jb3c7e9d341deb310@mail.gmail.com> <4B843891.4080002@cs.oswego.edu> <15e8b9d21002231246w659aa3b5i8167c48338953a3a@mail.gmail.com> Message-ID: <4B844133.10509@cs.oswego.edu> On 02/23/10 15:46, Neal Gafter wrote: > Fields that are not used in lambdas or inner > classes are verifiably and provably not shared between threads. Could you suggest a term and definition of these things so it can be used in your requested clarification of JLS sec 17.4? > Final variables have guarantees in the memory model too. Weaker and messier ones. -Doug From jkuhnert at gmail.com Tue Feb 23 13:36:10 2010 From: jkuhnert at gmail.com (Jesse Kuhnert) Date: Tue, 23 Feb 2010 16:36:10 -0500 Subject: Nice to @Share? In-Reply-To: <4B844133.10509@cs.oswego.edu> References: <17b2302a1002221238p2db47f2albcd2e6bb23ddafab@mail.gmail.com> <4B839E5C.5010502@sun.com> <15e8b9d21002230139k561e32d5x8454a1e69350c868@mail.gmail.com> <4B83B94D.2070504@sun.com> <15e8b9d21002230903k55fc56cav63d1736b64768a8@mail.gmail.com> <4B8431CD.3050004@cs.oswego.edu> <15e8b9d21002231206l24dbe186jb3c7e9d341deb310@mail.gmail.com> <4B843891.4080002@cs.oswego.edu> <15e8b9d21002231246w659aa3b5i8167c48338953a3a@mail.gmail.com> <4B844133.10509@cs.oswego.edu> Message-ID: <7926817e1002231336v1c289699gad43f9287db5f3a3@mail.gmail.com> Forgive me for throwing random noise in here, but the basic gist of things as I understand them is that you would like to be able to add parallel operations in to some of the core areas like Collections.sort() / etc in some kind of behind the scenes automated way and can't really do that if you can't be sure it wouldn't produce random breakage for people not expecting that behavior at runtime? If the root issue is that the general safety of variable access is already something that developers have to purposefully address with more thread-safe code then anything done here will be sort of un-pure from a language perspective (unless it solves the root issue, whatever / however that is or means. don't yell at me please ;) ) whether it is addressed via new keywords or @Shared or whatnot. As long as those doors are being opened, you could also do an even hackier sort of solution by defining some sort of java.util.concurrent annotation in general which a developer can decorate their lambda/etc with to let whatever parallel / thread safety minded api's that are interested know that they understand the thread safety issues and this one is "ok to do what you will with" ? At least then the annotation use would continue to provide meta only information ant not necessarily change the runtime. (other than possibly being picked up by interested parallel parties) At least that could prolong or postpone figuring this issue out more fully until such point as everyone feels very comfortable with it, just because it sounds like it's turning out to be a very tough nut to solve to everyone's satisfaction here. On Tue, Feb 23, 2010 at 3:57 PM, Doug Lea
wrote: > On 02/23/10 15:46, Neal Gafter wrote: > >> ?Fields that are not used in lambdas or inner >> classes are verifiably and provably not shared between threads. > > Could you suggest a term and definition of these things so > it can be used in your requested clarification of JLS sec 17.4? > >> Final variables have guarantees in the memory model too. > > Weaker and messier ones. > > -Doug > > From john at milsson.nu Tue Feb 23 13:55:33 2010 From: john at milsson.nu (John Nilsson) Date: Tue, 23 Feb 2010 22:55:33 +0100 Subject: Implementing recursive lambda with MethodHandle In-Reply-To: <15e8b9d21002231123t47bc8904h3784aaa83de429d@mail.gmail.com> References: <4B826FEF.4070605@univ-mlv.fr> <15e8b9d21002221148y8382f6dlb689117738118d92@mail.gmail.com> <17b2302a1002221219v1678540dw2a669f8dd2ebe734@mail.gmail.com> <15e8b9d21002221310x1f054fc2y4292022172d9abd6@mail.gmail.com> <9FF0EECF0E58D849A9D537E324DDDC0D0452EFA7@NZURC102PEX1.ubsw.net> <6C819E7F-7249-4689-B1D7-EDB910201738@gmail.com> <17b2302a1002231119k2c7ded43obaf34f2ae8940692@mail.gmail.com> <15e8b9d21002231123t47bc8904h3784aaa83de429d@mail.gmail.com> Message-ID: On Tue, Feb 23, 2010 at 8:23 PM, Neal Gafter wrote: > On Tue, Feb 23, 2010 at 11:19 AM, Joshua Bloch wrote: > > On Tue, Feb 23, 2010 at 11:06 AM, John Nilsson wrote: > >> Has it been decided that lambdas will be instances of Object? To me it > >> seems > >> that a lambda is fundamentally different from an object > > > > > > I respectfully disagree. Java already has a bipartite type system (with > > object rerference types and primitive types). The *last* thing we should > do > > is to make it tripartite. People already complain about the > > primitive/object distinction. We must avoid adding significant new > > complexity to an already complex language. Java's type system is among > the > > most delicate and complex parts of the language. Anything we add has to > > interoperate seamlessly and predictably with what's already there/ > > I think John was saying that lambdas are fundamentally different from > object creation expressions (e.g. anonymous inner classes), which I > agree with. I also agree with you that function types must be > reference types. > I realize that it would be hairy to sort out how to do this cleanly. I just wanted to air the question. What I was thinking is that "Object" imply the existence of equals, hashCode, toString, getClass or any other stuff associated with "normal" objects. I don't expect any of this to exist for lambdas so why force them to? I think of lambdas as closer to methods than objects, or rather, I think of methods as lambdas attached to objects. As some would say, it is a has-a, not an is-a, relationship ;-) BR, John From neal at gafter.com Tue Feb 23 15:35:12 2010 From: neal at gafter.com (Neal Gafter) Date: Tue, 23 Feb 2010 15:35:12 -0800 Subject: Nice to @Share? In-Reply-To: <4B844133.10509@cs.oswego.edu> References: <17b2302a1002221238p2db47f2albcd2e6bb23ddafab@mail.gmail.com> <4B839E5C.5010502@sun.com> <15e8b9d21002230139k561e32d5x8454a1e69350c868@mail.gmail.com> <4B83B94D.2070504@sun.com> <15e8b9d21002230903k55fc56cav63d1736b64768a8@mail.gmail.com> <4B8431CD.3050004@cs.oswego.edu> <15e8b9d21002231206l24dbe186jb3c7e9d341deb310@mail.gmail.com> <4B843891.4080002@cs.oswego.edu> <15e8b9d21002231246w659aa3b5i8167c48338953a3a@mail.gmail.com> <4B844133.10509@cs.oswego.edu> Message-ID: <15e8b9d21002231535w77732284wc43a0727c1d68b58@mail.gmail.com> On Tue, Feb 23, 2010 at 12:57 PM, Doug Lea
wrote: > On 02/23/10 15:46, Neal Gafter wrote: > >> ?Fields that are not used in lambdas or inner >> classes are verifiably and provably not shared between threads. > > Could you suggest a term and definition of these things so > it can be used in your requested clarification of JLS sec 17.4? Yes: I'd just call them "variables" (17.4.1), and include static fields, instance fields, array elements, local variables, formal parameters, and exception handler parameters under that term. The term "heap memory" can be removed, as it serves no purpose in the spec. The term "shared memory" should be defined as "memory that is shared between threads". >> Final variables have guarantees in the memory model too. > > Weaker and messier ones. Weaker and messier than what? The language spec does not currently have any other guarantees for final local variables that are shared between threads; if these rules do not apply, then what does? As far as I can tell, for the limited ways in which final local variables can be used, the guarantees are just as strong as you could hope for. From neal at gafter.com Tue Feb 23 15:58:38 2010 From: neal at gafter.com (Neal Gafter) Date: Tue, 23 Feb 2010 15:58:38 -0800 Subject: Implementing recursive lambda with MethodHandle In-Reply-To: References: <4B826FEF.4070605@univ-mlv.fr> <17b2302a1002221219v1678540dw2a669f8dd2ebe734@mail.gmail.com> <15e8b9d21002221310x1f054fc2y4292022172d9abd6@mail.gmail.com> <9FF0EECF0E58D849A9D537E324DDDC0D0452EFA7@NZURC102PEX1.ubsw.net> <6C819E7F-7249-4689-B1D7-EDB910201738@gmail.com> <17b2302a1002231119k2c7ded43obaf34f2ae8940692@mail.gmail.com> <15e8b9d21002231123t47bc8904h3784aaa83de429d@mail.gmail.com> Message-ID: <15e8b9d21002231558m857e18bg7df0ea746a0cc366@mail.gmail.com> On Tue, Feb 23, 2010 at 1:55 PM, John Nilsson wrote: > I realize that it would be hairy to sort out how to do this cleanly. I just > wanted to air the question. > What I was thinking is that "Object" imply the existence of equals, > hashCode, toString, getClass or any other stuff associated with "normal" > objects. I don't expect any of this to exist for lambdas so why force them > to? I think of lambdas as closer to methods than objects, or rather, I think > of methods as lambdas attached to objects. > As some would say, it is a has-a, not an is-a, relationship ;-) I think it is fine for values of function type to inherit these methods from Object, and for getClass() to return MethodHandle.class or the class of some compiler-generated implementation object. From dl at cs.oswego.edu Tue Feb 23 16:11:02 2010 From: dl at cs.oswego.edu (Doug Lea) Date: Tue, 23 Feb 2010 19:11:02 -0500 Subject: Nice to @Share? In-Reply-To: <15e8b9d21002231535w77732284wc43a0727c1d68b58@mail.gmail.com> References: <17b2302a1002221238p2db47f2albcd2e6bb23ddafab@mail.gmail.com> <4B839E5C.5010502@sun.com> <15e8b9d21002230139k561e32d5x8454a1e69350c868@mail.gmail.com> <4B83B94D.2070504@sun.com> <15e8b9d21002230903k55fc56cav63d1736b64768a8@mail.gmail.com> <4B8431CD.3050004@cs.oswego.edu> <15e8b9d21002231206l24dbe186jb3c7e9d341deb310@mail.gmail.com> <4B843891.4080002@cs.oswego.edu> <15e8b9d21002231246w659aa3b5i8167c48338953a3a@mail.gmail.com> <4B844133.10509@cs.oswego.edu> <15e8b9d21002231535w77732284wc43a0727c1d68b58@mail.gmail.com> Message-ID: <4B846E96.702@cs.oswego.edu> On 02/23/10 18:35, Neal Gafter wrote: > On Tue, Feb 23, 2010 at 12:57 PM, Doug Lea
wrote: >> On 02/23/10 15:46, Neal Gafter wrote: >> >>> Fields that are not used in lambdas or inner >>> classes are verifiably and provably not shared between threads. >> >> Could you suggest a term and definition of these things so >> it can be used in your requested clarification of JLS sec 17.4? > > Yes: I'd just call them "variables" (17.4.1), and include static > fields, instance fields, array elements, local variables, formal > parameters, and exception handler parameters under that term. I'm running out of ways to say this so will stop after this last try: The only way we know to phrase a usable memory ordering spec entails distinguishing variables that CANNOT be shared across threads from those that happen not to be shared in a given program execution. There is nothing especially Java-ish about this. Specs for other languages and processors take the same form -- we must be able to describe states strictly from the point of view of a given processor/thread, unconditionally with respect to possible multithreaded program executions. >>> Final variables have guarantees in the memory model too. >> >> Weaker and messier ones. > > Weaker and messier than what? Than those for local variables. For example, if "this" leaks in a constructor, a reader might see pre-initialized values of a final field. So to verify some property of a use of a final field of some object, you must know something about its constructor. In contrast, properties of local variables are unconditional. -Doug From neal at gafter.com Tue Feb 23 16:26:07 2010 From: neal at gafter.com (Neal Gafter) Date: Tue, 23 Feb 2010 16:26:07 -0800 Subject: Nice to @Share? In-Reply-To: <4B846E96.702@cs.oswego.edu> References: <17b2302a1002221238p2db47f2albcd2e6bb23ddafab@mail.gmail.com> <4B83B94D.2070504@sun.com> <15e8b9d21002230903k55fc56cav63d1736b64768a8@mail.gmail.com> <4B8431CD.3050004@cs.oswego.edu> <15e8b9d21002231206l24dbe186jb3c7e9d341deb310@mail.gmail.com> <4B843891.4080002@cs.oswego.edu> <15e8b9d21002231246w659aa3b5i8167c48338953a3a@mail.gmail.com> <4B844133.10509@cs.oswego.edu> <15e8b9d21002231535w77732284wc43a0727c1d68b58@mail.gmail.com> <4B846E96.702@cs.oswego.edu> Message-ID: <15e8b9d21002231626t60c15a24h1011566414b2caee@mail.gmail.com> On Tue, Feb 23, 2010 at 4:11 PM, Doug Lea
wrote: > The only way we know to phrase a usable memory ordering spec > entails distinguishing variables that CANNOT be shared > across threads from those that happen not to be > shared in a given program execution. There is nothing > especially Java-ish about this. Specs for other languages > and processors take the same form -- we must be able to > describe states strictly from the point of view of > a given processor/thread, unconditionally with respect > to possible multithreaded program executions. You keep saying that you only know how to do it with an extra axiom, and that others do it that way too, but in fact (1) The Java spec does not have the axiom you require, and (2) The guarantees for other "shared" variables serve precisely the purpose you need for local variables. There, now you know how to do it. >>>> Final variables have guarantees in the memory model too. >>> >>> Weaker and messier ones. >> >> Weaker and messier than what? > > Than those for local variables. For example, if > "this" leaks in a constructor, a reader might see > pre-initialized values of a final field. So to > verify some property of a use of a final field > of some object, you must know something about > its constructor. In contrast, properties of > local variables are unconditional. Right, because local variables cannot leak before they are initialized; the definite assignment rules ensure that, and so you don't need to know anything more about the code that initializes them. The stronger property for local variables that you require falls out of the general rules for shared variables coupled with the definite assignment rules for locals. No separate axioms are required, nor are any stronger guarantees currently in the spec. Cheers, Neal From dl at cs.oswego.edu Tue Feb 23 17:09:50 2010 From: dl at cs.oswego.edu (Doug Lea) Date: Tue, 23 Feb 2010 20:09:50 -0500 Subject: Nice to @Share? In-Reply-To: <15e8b9d21002231626t60c15a24h1011566414b2caee@mail.gmail.com> References: <17b2302a1002221238p2db47f2albcd2e6bb23ddafab@mail.gmail.com> <4B83B94D.2070504@sun.com> <15e8b9d21002230903k55fc56cav63d1736b64768a8@mail.gmail.com> <4B8431CD.3050004@cs.oswego.edu> <15e8b9d21002231206l24dbe186jb3c7e9d341deb310@mail.gmail.com> <4B843891.4080002@cs.oswego.edu> <15e8b9d21002231246w659aa3b5i8167c48338953a3a@mail.gmail.com> <4B844133.10509@cs.oswego.edu> <15e8b9d21002231535w77732284wc43a0727c1d68b58@mail.gmail.com> <4B846E96.702@cs.oswego.edu> <15e8b9d21002231626t60c15a24h1011566414b2caee@mail.gmail.com> Message-ID: <4B847C5E.40603@cs.oswego.edu> On 02/23/10 19:26, Neal Gafter wrote: >> Than those for local variables. For example, if >> "this" leaks in a constructor, a reader might see >> pre-initialized values of a final field. So to >> verify some property of a use of a final field >> of some object, you must know something about >> its constructor. In contrast, properties of >> local variables are unconditional. > > Right, because local variables cannot leak before they are > initialized... True at some level, but for a memory model, the the critical difference is that reading a value written by another processor entails a cooperative protocol (thus the possible values read depend on this protocol). Reading a value written by your own processor does not. This also shows up at implementation level. So, for example, writes to final fields may require memory fence instructions on some processors, and even reads of them do on others (happily no common ones though). It is hard to get a handle on such specs and implementation guidance without the foothold of locals. -Doug From neal at gafter.com Tue Feb 23 18:25:26 2010 From: neal at gafter.com (Neal Gafter) Date: Tue, 23 Feb 2010 18:25:26 -0800 Subject: Nice to @Share? In-Reply-To: <4B847C5E.40603@cs.oswego.edu> References: <17b2302a1002221238p2db47f2albcd2e6bb23ddafab@mail.gmail.com> <4B8431CD.3050004@cs.oswego.edu> <15e8b9d21002231206l24dbe186jb3c7e9d341deb310@mail.gmail.com> <4B843891.4080002@cs.oswego.edu> <15e8b9d21002231246w659aa3b5i8167c48338953a3a@mail.gmail.com> <4B844133.10509@cs.oswego.edu> <15e8b9d21002231535w77732284wc43a0727c1d68b58@mail.gmail.com> <4B846E96.702@cs.oswego.edu> <15e8b9d21002231626t60c15a24h1011566414b2caee@mail.gmail.com> <4B847C5E.40603@cs.oswego.edu> Message-ID: <15e8b9d21002231825k6de99c85n4271596787b1e977@mail.gmail.com> On Tue, Feb 23, 2010 at 5:09 PM, Doug Lea
wrote: > On 02/23/10 19:26, Neal Gafter wrote: >>> >>> Than those for local variables. For example, if >>> "this" leaks in a constructor, a reader might see >>> pre-initialized values of a final field. So to >>> verify some property of a use of a final field >>> of some object, you must know something about >>> its constructor. In contrast, properties of >>> local variables are unconditional. >> >> Right, because local variables cannot leak before they are >> initialized... > > True at some level, but for a memory model, the > the critical difference is that reading a value written > by another processor entails a cooperative protocol > (thus the possible values read depend on this protocol). > Reading a value written by your own processor does not. > This also shows up at implementation level. > So, for example, writes to final fields may require > memory fence instructions on some processors, and > even reads of them do on others (happily no common > ones though). It is hard to get a handle on such > specs and implementation guidance without the foothold > of locals. OK, I can certainly see that 17.5.1 as written only works for instance fields (and not locals). The corresponding rule for a local would be that the write executed by the method in which the local variable is declared happens-before any reads in other threads. I had hoped to infer that from the transitive closure of the definite assignment rule for the local happening before the anonymous instance is created (due to definite assignment rules) and the write to a final field (capturing the local in the anonymous instance) following that in the same thread, combined with the happens-before relationship defined by 17.5.1 applied to that field. Unfortunately, the JLS's specification for how constructors of anonymous instances work (15.9.5.1) is not correct in the presence of captured locals. Still, I think it would be better to fix the JLS bugs that prevent this logic from working rather than adding axioms that should be consequences of existing rules. From Alex.Buckley at Sun.COM Wed Feb 24 09:52:10 2010 From: Alex.Buckley at Sun.COM (Alex Buckley) Date: Wed, 24 Feb 2010 09:52:10 -0800 Subject: Nice to @Share? In-Reply-To: <15e8b9d21002231117y760dc081me52dc4587ec8fb9f@mail.gmail.com> References: <17b2302a1002221238p2db47f2albcd2e6bb23ddafab@mail.gmail.com> <4B839E5C.5010502@sun.com> <15e8b9d21002230139k561e32d5x8454a1e69350c868@mail.gmail.com> <17b2302a1002231108o151a64fcje3c853478eb8ee65@mail.gmail.com> <15e8b9d21002231117y760dc081me52dc4587ec8fb9f@mail.gmail.com> Message-ID: <4B85674A.3040302@sun.com> Neal Gafter wrote: > Too bad, because the sentence is already incorrect. Local variables > that happen to be final and used in an inner class do not cease to > exist when the execution of the block is complete. Local variables that are final and accessed from an anon.inner class are compiled to non-local variables, right? They are local from the p.o.v. of JLS3 1-16 but not local from the p.o.v. of the JMM, which ended up as JLS3 17. A "shared local variable" defined in 1-16 would no more be a local variable in 17 than a "final local variable" is a local variable in 17. I pointed out 17.4.1 to show that "local variable" has always been assumed to mean the same in 1-16 as in 17. But the assumption is incorrect, as I realized for 'shared variables' and you realized for 'final local variables'. 17 could be changed to treat all method-scoped variables in the language as local variables in the JMM, but that will not happen. As you indicated, JLS3 15.12.4.5 is a masterpiece of wording that puts local variables in an "activation frame", separate from the stack. Shared variables will live in the frame along with final and non-final local variables. But it seems to me that shared variables are a more dangerous language construct than final local variables, and deserve to be called out in the language. JMM-inspired assumptions around locality should be stripped away with extreme prejudice. Come to think of it, JLS3 8.1.3 would ideally say something other than "local variable" in: "Any local variable, formal method parameter or exception handler parameter used but not declared in an inner class must be declared final. Any local variable, used but not declared in an inner class must be definitely assigned (?16) before the body of the inner class." Alex From john at milsson.nu Wed Feb 24 10:17:28 2010 From: john at milsson.nu (John Nilsson) Date: Wed, 24 Feb 2010 19:17:28 +0100 Subject: Implementing recursive lambda with MethodHandle In-Reply-To: <15e8b9d21002231558m857e18bg7df0ea746a0cc366@mail.gmail.com> References: <4B826FEF.4070605@univ-mlv.fr> <17b2302a1002221219v1678540dw2a669f8dd2ebe734@mail.gmail.com> <15e8b9d21002221310x1f054fc2y4292022172d9abd6@mail.gmail.com> <9FF0EECF0E58D849A9D537E324DDDC0D0452EFA7@NZURC102PEX1.ubsw.net> <6C819E7F-7249-4689-B1D7-EDB910201738@gmail.com> <17b2302a1002231119k2c7ded43obaf34f2ae8940692@mail.gmail.com> <15e8b9d21002231123t47bc8904h3784aaa83de429d@mail.gmail.com> <15e8b9d21002231558m857e18bg7df0ea746a0cc366@mail.gmail.com> Message-ID: On Wed, Feb 24, 2010 at 12:58 AM, Neal Gafter wrote: > I think it is fine for values of function type to inherit these > methods from Object, and for getClass() to return MethodHandle.class > or the class of some compiler-generated implementation object. > Fair enough. Just to finish the train of thought. I feared that having lambdas be objects will constrain the runtime representation options in a manner that will exclude some optimizations or just add unwanted overhead. OTOH I wouldn't have the first clue about how the JVM actually handles this so I wouldn't be surprised to be wrong. In the end I would want to use lots of lambdas for all kinds of stuff without fear of adding performance issues. BR, John From neal at gafter.com Wed Feb 24 10:41:23 2010 From: neal at gafter.com (Neal Gafter) Date: Wed, 24 Feb 2010 10:41:23 -0800 Subject: Nice to @Share? In-Reply-To: <4B85674A.3040302@sun.com> References: <17b2302a1002221238p2db47f2albcd2e6bb23ddafab@mail.gmail.com> <4B839E5C.5010502@sun.com> <15e8b9d21002230139k561e32d5x8454a1e69350c868@mail.gmail.com> <17b2302a1002231108o151a64fcje3c853478eb8ee65@mail.gmail.com> <15e8b9d21002231117y760dc081me52dc4587ec8fb9f@mail.gmail.com> <4B85674A.3040302@sun.com> Message-ID: <15e8b9d21002241041u2c721c6al128eb565d258b211@mail.gmail.com> On Wed, Feb 24, 2010 at 9:52 AM, Alex Buckley wrote: > Neal Gafter wrote: >> Too bad, because the sentence is already incorrect. ?Local variables >> that happen to be final and used in an inner class do not cease to >> exist when the execution of the block is complete. > > Local variables that are final and accessed from an anon.inner class are > compiled to non-local variables, right? Right, but unfortunately the JLS doesn't say that (yet). > They are local from the p.o.v. > of JLS3 1-16 but not local from the p.o.v. of the JMM, which ended up as > JLS3 17. Right, that is how it should be. > A "shared local variable" defined in 1-16 would no more be a local > variable in 17 than a "final local variable" is a local variable in 17. Agreed. > I pointed out 17.4.1 to show that "local variable" has always been > assumed to mean the same in 1-16 as in 17. But the assumption is > incorrect, as I realized for 'shared variables' and you realized for > 'final local variables'. 17 could be changed to treat all method-scoped > variables in the language as local variables in the JMM, but that will > not happen. Excellent! > As you indicated, JLS3 15.12.4.5 is a masterpiece of wording that puts > local variables in an "activation frame", separate from the stack. > Shared variables will live in the frame along with final and non-final > local variables. But it seems to me that shared variables are a more > dangerous language construct than final local variables, and deserve to > be called out in the language. That makes sense, presuming that calling out is really required to get the semantics right. If its value is primarily tutorial, then it does not belong in normative text. From howard.lovatt at gmail.com Wed Feb 24 12:53:18 2010 From: howard.lovatt at gmail.com (Howard Lovatt) Date: Thu, 25 Feb 2010 07:53:18 +1100 Subject: Not Dead Yet (was: MethodHandle vs function types) Message-ID: <3dd3f56a1002241253n413f1747pa16ae898b06f5fb5@mail.gmail.com> Neal Gafter Said > Approach (1) doesn't work for various reasons. As we've already seen by the > failure of Howard Lovatt's attempt to reify function types, we must preserve > reference identity for all widening reference conversions. I would say this is overstating the problems. To amplify their are two areas in which reference identity are not conserved: 1. If you have: #String(Object) so = #(Object o)(o.toString()); #Object(String) os = so; Then os == so is false, but os.equals(so) is true. This is little different than comparing many reference types, e.g. String, and therefore I think OK. 2. If, as my suggestion is currently written, you have: #String(Object)[] soa = new #String(Object)[1]; #Object(String)[] osa = soa; Then osa == soa is false. This is indeed a problem since osa[0] = #(Object o)(o.toString()) will not set soa[0] also. As I said in a previous email, I am thinking about the best solution to this. One possibility is that the above example is translated into: _Callable_1[] soa = (_Callable_1[]) new _Callable_1[1]; _Callable_1[] osa = soa; IE arrays are erased not reified. This is probably OK, but there may well be a better solution and I am still considering other options. Therefore I would say, with apologies to Monty Python, that reified lambdas are "not dead yet". -- Howard. From alex.blewitt at gmail.com Wed Feb 24 13:24:03 2010 From: alex.blewitt at gmail.com (Alex Blewitt) Date: Wed, 24 Feb 2010 21:24:03 +0000 Subject: Effectively final effective? Message-ID: The Lambda 0.1.5 spec introduces 'effectively final', as a way of getting final-like variables but without having to type the letters f i n a l into the code. I'd argue that adding the keyword is no more or less difficult than it was when inner classes were introduced, and that modern IDEs will automatically prompt you to change the variable to final with a single click if used in an inner class. Furthermore, I've never found it difficult to add it in when needed; but it does encode (in the source) the fact that it's supposed to be final, which then discourages changing it elsewhere. An effectively final variable loses that information, which might lead to developers wondering why subsequent assignments don't work or cause compilation errors elsewhere. My concern is that by introducing the 'effectively final' change in with the Lambda spec, we're increasing the scope of the change - and therefore the potential for issues. Lastly, it's not clear whether effectively final would also apply to inner classes, and whether this would be seen for a good or bad change given the above. Alex From john at milsson.nu Wed Feb 24 13:52:50 2010 From: john at milsson.nu (John Nilsson) Date: Wed, 24 Feb 2010 22:52:50 +0100 Subject: Effectively final effective? In-Reply-To: References: Message-ID: On Wed, Feb 24, 2010 at 10:24 PM, Alex Blewitt wrote: > The Lambda 0.1.5 spec introduces 'effectively final', as a way of getting > final-like variables but without having to type the letters f i n a l into > the code. > Isn't it also about being able to mutate the variable outside the closure? IOW it's effectively final inside the closure, but the variable is free to be rebound outside it. So it's actually about not having to type final T myFinalT = mutableT; ? BR, John From jjb at google.com Wed Feb 24 14:12:48 2010 From: jjb at google.com (Joshua Bloch) Date: Wed, 24 Feb 2010 14:12:48 -0800 Subject: Effectively final effective? In-Reply-To: References: Message-ID: <17b2302a1002241412s283a07c6p8fedc75106e38dbf@mail.gmail.com> Alex, On Wed, Feb 24, 2010 at 1:24 PM, Alex Blewitt wrote: > The Lambda 0.1.5 spec introduces 'effectively final', as a way of getting > final-like variables but without having to type the letters f i n a l into > the code. > > I'd argue that adding the keyword is no more or less difficult than it was > when inner classes were introduced, and that modern IDEs will automatically > prompt you to change the variable to final with a single click if used in an > inner class. > I agree, but most people hate this aspect of anonymous classes, and we can fix it. It's vaguely analogous to what we're doing with the "diamond operator" (<>) for parametrized type instance creation. It's not about how difficult it is to add the modifier final, but what it does to the code. Seeing final, e.g., on a method parameter declaration so that it can be passed to a "function object" (currently, an anonymous class instance) is visual noise. Of course this is a matter of taste, but I do believe this represents the consensus opinion among Java programmers. Josh From neal at gafter.com Wed Feb 24 14:49:04 2010 From: neal at gafter.com (Neal Gafter) Date: Wed, 24 Feb 2010 14:49:04 -0800 Subject: Effectively final effective? In-Reply-To: <17b2302a1002241412s283a07c6p8fedc75106e38dbf@mail.gmail.com> References: <17b2302a1002241412s283a07c6p8fedc75106e38dbf@mail.gmail.com> Message-ID: <15e8b9d21002241449y69b555e0ie89db6b07e650254@mail.gmail.com> On Wed, Feb 24, 2010 at 2:12 PM, Joshua Bloch wrote: >> I'd argue that adding the keyword is no more or less difficult than it was >> when inner classes were introduced, and that modern IDEs will automatically >> prompt you to change the variable to final with a single click if used in an >> inner class. > > I agree, but most people hate this aspect of anonymous classes, and we can > fix it. It's vaguely analogous to what we're doing with the "diamond > operator" (<>) for parametrized type instance creation. It isn't just the visual noise that people find so bothersome. The restriction itself is also bothersome, and harder to work around. From mthornton at optrak.co.uk Wed Feb 24 15:04:48 2010 From: mthornton at optrak.co.uk (Mark Thornton) Date: Wed, 24 Feb 2010 23:04:48 +0000 Subject: Effectively final effective? In-Reply-To: <15e8b9d21002241449y69b555e0ie89db6b07e650254@mail.gmail.com> References: <17b2302a1002241412s283a07c6p8fedc75106e38dbf@mail.gmail.com> <15e8b9d21002241449y69b555e0ie89db6b07e650254@mail.gmail.com> Message-ID: <4B85B090.9090706@optrak.co.uk> Neal Gafter wrote: > On Wed, Feb 24, 2010 at 2:12 PM, Joshua Bloch wrote: > >>> I'd argue that adding the keyword is no more or less difficult than it was >>> when inner classes were introduced, and that modern IDEs will automatically >>> prompt you to change the variable to final with a single click if used in an >>> inner class. >>> >> I agree, but most people hate this aspect of anonymous classes, and we can >> fix it. It's vaguely analogous to what we're doing with the "diamond >> operator" (<>) for parametrized type instance creation. >> > > It isn't just the visual noise that people find so bothersome. The > restriction itself is also bothersome, and harder to work around. > > Yet would not many developers be surprised to discover that they couldn't change an apparently ordinary local variable because it had been captured by a lambda (or anonymous inner class) some lines previously. Mark From alex.blewitt at gmail.com Wed Feb 24 15:16:29 2010 From: alex.blewitt at gmail.com (Alex Blewitt) Date: Wed, 24 Feb 2010 23:16:29 +0000 Subject: Effectively final effective? In-Reply-To: <4B85B090.9090706@optrak.co.uk> References: <17b2302a1002241412s283a07c6p8fedc75106e38dbf@mail.gmail.com> <15e8b9d21002241449y69b555e0ie89db6b07e650254@mail.gmail.com> <4B85B090.9090706@optrak.co.uk> Message-ID: <3F515628-F551-4D13-B703-AA9B41957F4E@gmail.com> On 24 Feb 2010, at 23:04, Mark Thornton wrote: > Neal Gafter wrote: >> It isn't just the visual noise that people find so bothersome. The >> restriction itself is also bothersome, and harder to work around. > Yet would not many developers be surprised to discover that they > couldn't change an apparently ordinary local variable because it had > been captured by a lambda (or anonymous inner class) some lines previously. That's my concern too. The fact that it has to be final (save tricks like one-element arrays, atomic refs etc.) is part of documenting that restriction. Simply allowing it to be omitted doesn't change the fact that it is final. Would it be worthwhile asking the question of a larger audience to get their thoughts? Or have those at Sun/Oracle already done so at some point? As with 'most' other people, the set of who 'most' is is an interesting question, particularly since I wonder if the question had been asked in any other language than English :-) Alex From mthornton at optrak.co.uk Wed Feb 24 15:24:34 2010 From: mthornton at optrak.co.uk (Mark Thornton) Date: Wed, 24 Feb 2010 23:24:34 +0000 Subject: Effectively final effective? In-Reply-To: <3F515628-F551-4D13-B703-AA9B41957F4E@gmail.com> References: <17b2302a1002241412s283a07c6p8fedc75106e38dbf@mail.gmail.com> <15e8b9d21002241449y69b555e0ie89db6b07e650254@mail.gmail.com> <4B85B090.9090706@optrak.co.uk> <3F515628-F551-4D13-B703-AA9B41957F4E@gmail.com> Message-ID: <4B85B532.5040709@optrak.co.uk> Alex Blewitt wrote: > On 24 Feb 2010, at 23:04, Mark Thornton wrote: > > >> Neal Gafter wrote: >> >>> It isn't just the visual noise that people find so bothersome. The >>> restriction itself is also bothersome, and harder to work around. >>> >> Yet would not many developers be surprised to discover that they >> couldn't change an apparently ordinary local variable because it had >> been captured by a lambda (or anonymous inner class) some lines previously. >> > > That's my concern too. The fact that it has to be final (save tricks like one-element arrays, atomic refs etc.) is part of documenting that restriction. Simply allowing it to be omitted doesn't change the fact that it is final. > > Would it be worthwhile asking the question of a larger audience to get their thoughts? Or have those at Sun/Oracle already done so at some point? As with 'most' other people, the set of who 'most' is is an interesting question, particularly since I wonder if the question had been asked in any other language than English :-) > > Alex Another alternative would be to allow the variables to be mutated, but where a lambda (or inner class) captured a variable that was not effectively final, it would also implement a marker interface that concurrent libraries could detect. Concurrent code could either throw an exception or fall back to a sequential algorithm if the supplied lambda implemented the marker interface. If this approach is viable it would be more flexible and surprise fewer developers. Admittedly it leaves the onus on concurrent library developers to insert the check. Mark From abies at adres.pl Wed Feb 24 15:37:40 2010 From: abies at adres.pl (Artur Biesiadowski) Date: Thu, 25 Feb 2010 00:37:40 +0100 Subject: Effectively final effective? In-Reply-To: <4B85B090.9090706@optrak.co.uk> References: <17b2302a1002241412s283a07c6p8fedc75106e38dbf@mail.gmail.com> <15e8b9d21002241449y69b555e0ie89db6b07e650254@mail.gmail.com> <4B85B090.9090706@optrak.co.uk> Message-ID: <4B85B844.9050507@adres.pl> Mark Thornton wrote: > > Yet would not many developers be surprised to discover that they > couldn't change an apparently ordinary local variable because it had > been captured by a lambda (or anonymous inner class) some lines previously. > > What about yet another solution ? ------------------ int x = 1; Runnable x = new Runnable() {public void run() { use(x); // you can use x as capture at moment of creation here //x++; // illegal, x is not allowed be modified from anonymous/lamba scopes }}; x = 2; // you CAN mutate it further in outside scope x.run(); // use(1) will be called inside, mutation of x is not visible in already captured environment ----------------- I think this has all the benefits: - no need to write final anymore to just capture the local variable - existing source code still works without any changes - works with index variables of 'for' as expected, even if lambdas/classes created inside are executed later - you can get very clear warning when trying to mutate x inside lambda/anonymous class ("Cannot modify local variable x of enclosing scope from within lambda/anonymous class") Only tricky part is that somebody might try to communicate with lambda/anonymous class through local variable, hoping that assignment x=2 above will become visible at some point inside already captured closure. Still, I think that this is use case complicated enough that people who will try it can be explained why it is not working. Regards, Artur Biesiadowski From lk at teamten.com Wed Feb 24 15:37:21 2010 From: lk at teamten.com (Lawrence Kesteloot) Date: Wed, 24 Feb 2010 15:37:21 -0800 Subject: Effectively final effective? In-Reply-To: <3F515628-F551-4D13-B703-AA9B41957F4E@gmail.com> References: <17b2302a1002241412s283a07c6p8fedc75106e38dbf@mail.gmail.com> <15e8b9d21002241449y69b555e0ie89db6b07e650254@mail.gmail.com> <4B85B090.9090706@optrak.co.uk> <3F515628-F551-4D13-B703-AA9B41957F4E@gmail.com> Message-ID: <997cab101002241537n2a5fdaa2nfcb3ed4b9e02a368@mail.gmail.com> Let's come up with a semi-realistic example then. Something like: public #int(String,String) getClippedSortFunction(int clipLength) { return #(String a,String b) { String aClipped = a.substring(0, Math.min(clipLength, a.length())); String bClipped = b.substring(0, Math.min(clipLength, b.length())); return aClipped.compareTo(bClipped); }; } The "clipLength" is effectively final, so its value is copied into the lambda and the sort happens fast. But then someone comes along later and inserts this line at the top of the getClippedSortFunction() method: clipLength = Math.max(1, clipLength); The parameter is no longer effectively final, so the programmer gets a warning (?), which he overrides with a click of the mouse in an IDE (adding @Shared or whatever), and "clipLength" is now in the heap and the sort slows down. Did I get that right? The "final" modifier has bugged me in the past with anonymous inner classes, but we really have no experience allowing large number of programmers to write code without the restriction. For all we know it would cause more confusion or more errors. Lawrence On Wed, Feb 24, 2010 at 3:16 PM, Alex Blewitt wrote: > On 24 Feb 2010, at 23:04, Mark Thornton wrote: > >> Neal Gafter wrote: >>> It isn't just the visual noise that people find so bothersome. ?The >>> restriction itself is also bothersome, and harder to work around. >> Yet would not many developers be surprised to discover that they >> couldn't change an apparently ordinary local variable because it had >> been captured by a lambda (or anonymous inner class) some lines previously. > > That's my concern too. The fact that it has to be final (save tricks like one-element arrays, atomic refs etc.) is part of documenting that restriction. Simply allowing it to be omitted doesn't change the fact that it is final. > > Would it be worthwhile asking the question of a larger audience to get their thoughts? Or have those at Sun/Oracle already done so at some point? As with 'most' other people, the set of who 'most' is is an interesting question, particularly since I wonder if the question had been asked in any other language than English :-) > > Alex > > From neal at gafter.com Wed Feb 24 16:34:33 2010 From: neal at gafter.com (Neal Gafter) Date: Wed, 24 Feb 2010 16:34:33 -0800 Subject: Effectively final effective? In-Reply-To: <997cab101002241537n2a5fdaa2nfcb3ed4b9e02a368@mail.gmail.com> References: <17b2302a1002241412s283a07c6p8fedc75106e38dbf@mail.gmail.com> <15e8b9d21002241449y69b555e0ie89db6b07e650254@mail.gmail.com> <4B85B090.9090706@optrak.co.uk> <3F515628-F551-4D13-B703-AA9B41957F4E@gmail.com> <997cab101002241537n2a5fdaa2nfcb3ed4b9e02a368@mail.gmail.com> Message-ID: <15e8b9d21002241634s53a3ba0csf3f274e59f836835@mail.gmail.com> On Wed, Feb 24, 2010 at 3:37 PM, Lawrence Kesteloot wrote: > The "final" modifier has bugged me in the past with anonymous inner > classes, but we really have no experience allowing large number of > programmers to write code without the restriction. For all we know it > would cause more confusion or more errors. Scala and C# and Ruby programmers, and programmers in many other languages, live without this restriction. Perhaps we should ask them. In my experience it is a blessing compared to the Java alternatives. One common pattern I've seen is the equivalent of Type1 result1 = null; Type2 result2 = null; doSomeApi(#(){ result1 = someComputation(); result2 = otherComputation(); }); doSomethingWith(result1, result2); where doSomeApi might be something like Java's AccessController.doPrivileged(PrivilegedAction). Cheers, Neal From lk at teamten.com Wed Feb 24 17:10:34 2010 From: lk at teamten.com (Lawrence Kesteloot) Date: Wed, 24 Feb 2010 17:10:34 -0800 Subject: Effectively final effective? In-Reply-To: <15e8b9d21002241634s53a3ba0csf3f274e59f836835@mail.gmail.com> References: <17b2302a1002241412s283a07c6p8fedc75106e38dbf@mail.gmail.com> <15e8b9d21002241449y69b555e0ie89db6b07e650254@mail.gmail.com> <4B85B090.9090706@optrak.co.uk> <3F515628-F551-4D13-B703-AA9B41957F4E@gmail.com> <997cab101002241537n2a5fdaa2nfcb3ed4b9e02a368@mail.gmail.com> <15e8b9d21002241634s53a3ba0csf3f274e59f836835@mail.gmail.com> Message-ID: <997cab101002241710w5e0ff7d2l8fd02265ac29a06f@mail.gmail.com> On Wed, Feb 24, 2010 at 4:34 PM, Neal Gafter wrote: > Scala and C# and Ruby programmers, and programmers in many other > languages, live without this restriction. ?Perhaps we should ask them. > ?In my experience it is a blessing compared to the Java alternatives. That's fine, but you're implicitly agreeing that you're okay with my innocent Math.max() assignment changing the implementation from a value-copy to a heap-store, along with the potential performance implications. I believe that's what triggered this thread in the first place. I think Alex Blewitt was saying that the "final" keyword forces one implementation over another, and it's therefore a useful indicator. Also, the paragraph you responded to was not about writable access to free variables from lambdas, but about CICE-style effective-final changes to anonymous inner classes. That's what we have no experience with. Lawrence From neal at gafter.com Wed Feb 24 17:39:13 2010 From: neal at gafter.com (Neal Gafter) Date: Wed, 24 Feb 2010 17:39:13 -0800 Subject: Effectively final effective? In-Reply-To: <997cab101002241710w5e0ff7d2l8fd02265ac29a06f@mail.gmail.com> References: <17b2302a1002241412s283a07c6p8fedc75106e38dbf@mail.gmail.com> <15e8b9d21002241449y69b555e0ie89db6b07e650254@mail.gmail.com> <4B85B090.9090706@optrak.co.uk> <3F515628-F551-4D13-B703-AA9B41957F4E@gmail.com> <997cab101002241537n2a5fdaa2nfcb3ed4b9e02a368@mail.gmail.com> <15e8b9d21002241634s53a3ba0csf3f274e59f836835@mail.gmail.com> <997cab101002241710w5e0ff7d2l8fd02265ac29a06f@mail.gmail.com> Message-ID: <15e8b9d21002241739x59d0f78cp39743d0b87cefa11@mail.gmail.com> On Wed, Feb 24, 2010 at 5:10 PM, Lawrence Kesteloot wrote: > On Wed, Feb 24, 2010 at 4:34 PM, Neal Gafter wrote: >> Scala and C# and Ruby programmers, and programmers in many other >> languages, live without this restriction. ?Perhaps we should ask them. >> ?In my experience it is a blessing compared to the Java alternatives. > > That's fine, but you're implicitly agreeing that you're okay with my > innocent Math.max() assignment changing the implementation from a > value-copy to a heap-store, along with the potential performance > implications. No; the compiler should be free to use any code sequence that implements the semantics required by the language. In this case, the compiler should be able to prove that a copy is correct, and should be free to generate code that way. From i30817 at gmail.com Wed Feb 24 20:40:10 2010 From: i30817 at gmail.com (Paulo Levi) Date: Thu, 25 Feb 2010 04:40:10 +0000 Subject: Effectively final effective? In-Reply-To: <15e8b9d21002241739x59d0f78cp39743d0b87cefa11@mail.gmail.com> References: <17b2302a1002241412s283a07c6p8fedc75106e38dbf@mail.gmail.com> <15e8b9d21002241449y69b555e0ie89db6b07e650254@mail.gmail.com> <4B85B090.9090706@optrak.co.uk> <3F515628-F551-4D13-B703-AA9B41957F4E@gmail.com> <997cab101002241537n2a5fdaa2nfcb3ed4b9e02a368@mail.gmail.com> <15e8b9d21002241634s53a3ba0csf3f274e59f836835@mail.gmail.com> <997cab101002241710w5e0ff7d2l8fd02265ac29a06f@mail.gmail.com> <15e8b9d21002241739x59d0f78cp39743d0b87cefa11@mail.gmail.com> Message-ID: <212322091002242040t53672c0bm383e8e075c7efda3@mail.gmail.com> Isn't final just a security measure for concurrency here? I think Neal had the right idea in the closures proposal to use different syntax, so a api that is expected to run in another thread doesn't have to worry if their users are careful enough to copy the variables and not expect to use the updated values. Something like void m( ##(int, int) m2){...} Never bothered about final myself, especially with all the possible workarounds ParentClass.this.var, arrays, return functors etc) From alex.blewitt at gmail.com Thu Feb 25 00:21:24 2010 From: alex.blewitt at gmail.com (Alex Blewitt) Date: Thu, 25 Feb 2010 08:21:24 +0000 Subject: Effectively final effective? In-Reply-To: <15e8b9d21002241634s53a3ba0csf3f274e59f836835@mail.gmail.com> References: <17b2302a1002241412s283a07c6p8fedc75106e38dbf@mail.gmail.com> <15e8b9d21002241449y69b555e0ie89db6b07e650254@mail.gmail.com> <4B85B090.9090706@optrak.co.uk> <3F515628-F551-4D13-B703-AA9B41957F4E@gmail.com> <997cab101002241537n2a5fdaa2nfcb3ed4b9e02a368@mail.gmail.com> <15e8b9d21002241634s53a3ba0csf3f274e59f836835@mail.gmail.com> Message-ID: <636fd28e1002250021n3a6876beu95e1deb45bee34bb@mail.gmail.com> On Thu, Feb 25, 2010 at 12:34 AM, Neal Gafter wrote: > Scala and C# and Ruby programmers, and programmers in many other > languages, live without this restriction. ?Perhaps we should ask them. > ?In my experience it is a blessing compared to the Java alternatives. Scala has 'val' and 'var', the former of which is final whilst the latter is mutable. There is a strong preference for using 'val' where possible in Scala programs (and in general, for creating immutable objects). Secondly, the fact that Scala can mutate captured variables (whereas Java cannot, in the current version of the spec) is a key difference between the two approaches. SImply omitting the keyword in Java does not enable muation. So the question is not 'would you prefer them to be mutable' but rather 'given that it is final, would you prefer to explicitly declare that for less capable programmers to read later'. Programs are written for other humans to understand, not for a computer to understand. Alex From tronicek at fit.cvut.cz Thu Feb 25 00:46:52 2010 From: tronicek at fit.cvut.cz (Zdenek Tronicek) Date: Thu, 25 Feb 2010 09:46:52 +0100 Subject: Effectively final effective? In-Reply-To: <636fd28e1002250021n3a6876beu95e1deb45bee34bb@mail.gmail.com> References: <17b2302a1002241412s283a07c6p8fedc75106e38dbf@mail.gmail.com> <15e8b9d21002241449y69b555e0ie89db6b07e650254@mail.gmail.com> <4B85B090.9090706@optrak.co.uk> <3F515628-F551-4D13-B703-AA9B41957F4E@gmail.com> <997cab101002241537n2a5fdaa2nfcb3ed4b9e02a368@mail.gmail.com> <15e8b9d21002241634s53a3ba0csf3f274e59f836835@mail.gmail.com> <636fd28e1002250021n3a6876beu95e1deb45bee34bb@mail.gmail.com> Message-ID: <4310c67c9c1b7d4f7fbadc4ebed2393b.squirrel@imap.fit.cvut.cz> Alex Blewitt napsal(a): > Secondly, the fact that Scala can mutate captured variables (whereas > Java cannot, in the current version of the spec) is a key difference > between the two approaches. SImply omitting the keyword in Java does > not enable muation. So the question is not 'would you prefer them to > be mutable' but rather 'given that it is final, would you prefer to > explicitly declare that for less capable programmers to read later'. > Programs are written for other humans to understand, not for a > computer to understand. Sure. We should prefer easy-to-read to easy-to-write. And this not only from myself: listen to Barbara Liskov at http://www.infoq.com/presentations/liskov-power-of-abstraction. But I am in doubt here, if 'final' improves readability. Z. -- Zdenek Tronicek FIT CTU in Prague From peter.levart at marand.si Thu Feb 25 01:00:30 2010 From: peter.levart at marand.si (Peter Levart) Date: Thu, 25 Feb 2010 10:00:30 +0100 Subject: Effectively final effective? In-Reply-To: <15e8b9d21002241634s53a3ba0csf3f274e59f836835@mail.gmail.com> References: <997cab101002241537n2a5fdaa2nfcb3ed4b9e02a368@mail.gmail.com> <15e8b9d21002241634s53a3ba0csf3f274e59f836835@mail.gmail.com> Message-ID: <201002251000.30778.peter.levart@marand.si> On Thursday 25 February 2010 01:34:33 Neal Gafter wrote: > On Wed, Feb 24, 2010 at 3:37 PM, Lawrence Kesteloot wrote: > > The "final" modifier has bugged me in the past with anonymous inner > > classes, but we really have no experience allowing large number of > > programmers to write code without the restriction. For all we know it > > would cause more confusion or more errors. > > Scala and C# and Ruby programmers, and programmers in many other > languages, live without this restriction. Perhaps we should ask them. > In my experience it is a blessing compared to the Java alternatives. > > One common pattern I've seen is the equivalent of > > Type1 result1 = null; > Type2 result2 = null; > doSomeApi(#(){ result1 = someComputation(); result2 = otherComputation(); }); > doSomethingWith(result1, result2); > > where doSomeApi might be something like Java's > AccessController.doPrivileged(PrivilegedAction). > > Cheers, > Neal First I'd like to say that I'm all for "transparent" access to variables from lambdas. It's realy the most natural way to program that way and I would like to see it implemented. But if concerned powers decide it is too much of a risk, then there should be something in the language to support explicit denotation of variable wrappers so that awkward workarrounds are not needed. For example, how do you find this: final *volatile Type1 result1 = &null; final *Type2 result2 = &null; doSomeApi(#(){ *result1 = someComputation(); *result2 = otherComputation(); }); doSomethingWith(*result1, *result2); to be compiled as: final java.gen.VolatileObjectReference result1 = new java.gen.VolatileObjectReference(null); final java.gen.ObjectReference result2 = new java.gen.ObjectReference(null); doSomeApi(#(){ result1.referent = someComputation(); result2.referent = otherComputation(); }); doSomethingWith(result1.referent, result2.referent); or is this awkward too? Regards, Peter From i30817 at gmail.com Thu Feb 25 01:08:49 2010 From: i30817 at gmail.com (Paulo Levi) Date: Thu, 25 Feb 2010 09:08:49 +0000 Subject: Effectively final effective? In-Reply-To: <201002251000.30778.peter.levart@marand.si> References: <997cab101002241537n2a5fdaa2nfcb3ed4b9e02a368@mail.gmail.com> <15e8b9d21002241634s53a3ba0csf3f274e59f836835@mail.gmail.com> <201002251000.30778.peter.levart@marand.si> Message-ID: <212322091002250108u38390071wf26a05790bf7c7f9@mail.gmail.com> I ask is accessing a "result" like that going to use Futures for all variables? Only way i see to make the threaded case in the function receiving the lambda to work. Blocking things like that makes this useless for most use cases. (i think this idea is a mistake and too much magic) Besides java has always emphasized one return value, and that is a good idea, even if i cursed it myself sometimes. That "return" style seems more motivated by a avoidance of return wrappers or workarounds for void lambdas than anything else. From i30817 at gmail.com Thu Feb 25 01:13:11 2010 From: i30817 at gmail.com (Paulo Levi) Date: Thu, 25 Feb 2010 09:13:11 +0000 Subject: Effectively final effective? In-Reply-To: <212322091002250108u38390071wf26a05790bf7c7f9@mail.gmail.com> References: <997cab101002241537n2a5fdaa2nfcb3ed4b9e02a368@mail.gmail.com> <15e8b9d21002241634s53a3ba0csf3f274e59f836835@mail.gmail.com> <201002251000.30778.peter.levart@marand.si> <212322091002250108u38390071wf26a05790bf7c7f9@mail.gmail.com> Message-ID: <212322091002250113y4f31881dg871ac7a7edb1d40e@mail.gmail.com> A aside. It's not the caller that knows how the lambda is going to be used, but the function that receives it. You can either forbid unsafe usages, or allow them but the function needs a way to say it is safe. Just spouting my not very considered opinions. Will shut up now. From markmahieu at googlemail.com Thu Feb 25 01:28:17 2010 From: markmahieu at googlemail.com (Mark Mahieu) Date: Thu, 25 Feb 2010 09:28:17 +0000 Subject: Effectively final effective? In-Reply-To: <212322091002250113y4f31881dg871ac7a7edb1d40e@mail.gmail.com> References: <997cab101002241537n2a5fdaa2nfcb3ed4b9e02a368@mail.gmail.com> <15e8b9d21002241634s53a3ba0csf3f274e59f836835@mail.gmail.com> <201002251000.30778.peter.levart@marand.si> <212322091002250108u38390071wf26a05790bf7c7f9@mail.gmail.com> <212322091002250113y4f31881dg871ac7a7edb1d40e@mail.gmail.com> Message-ID: <9A600150-83D8-42C0-AD87-E74A3CDE867B@googlemail.com> On 25 Feb 2010, at 09:13, Paulo Levi wrote: > A aside. It's not the caller that knows how the lambda is going to be used, > but the function that receives it. You can either forbid unsafe usages, or > allow them but the function needs a way to say it is safe. > > Just spouting my not very considered opinions. Will shut up now. > That's a perfectly valid line of thinking. It does make for a more complex approach, but perhaps that's not unwarranted. BGGA explored some options in that direction. One big question is where to encode the distinction between 'safe' and 'unsafe'; the temptation is to burden the type system with the responsibility of carrying yet another concept, which isn't without its costs. Mark From peter.levart at marand.si Thu Feb 25 08:20:36 2010 From: peter.levart at marand.si (Peter Levart) Date: Thu, 25 Feb 2010 17:20:36 +0100 Subject: Effectively final effective? In-Reply-To: <9A600150-83D8-42C0-AD87-E74A3CDE867B@googlemail.com> References: <212322091002250113y4f31881dg871ac7a7edb1d40e@mail.gmail.com> <9A600150-83D8-42C0-AD87-E74A3CDE867B@googlemail.com> Message-ID: <201002251720.36358.peter.levart@marand.si> On Thursday 25 February 2010 10:28:17 Mark Mahieu wrote: > > On 25 Feb 2010, at 09:13, Paulo Levi wrote: > > > A aside. It's not the caller that knows how the lambda is going to be used, > > but the function that receives it. You can either forbid unsafe usages, or > > allow them but the function needs a way to say it is safe. > > > > Just spouting my not very considered opinions. Will shut up now. > > > > That's a perfectly valid line of thinking. It does make for a more complex approach, but perhaps that's not unwarranted. BGGA explored some options in that direction. > > One big question is where to encode the distinction between 'safe' and 'unsafe'; the temptation is to burden the type system with the responsibility of carrying yet another concept, which isn't without its costs. > > Mark > JSR 308 could be extended to support that. Regards, Peter From reinier at zwitserloot.com Thu Feb 25 08:44:44 2010 From: reinier at zwitserloot.com (Reinier Zwitserloot) Date: Thu, 25 Feb 2010 17:44:44 +0100 Subject: Effectively final effective? In-Reply-To: <201002251000.30778.peter.levart@marand.si> References: <997cab101002241537n2a5fdaa2nfcb3ed4b9e02a368@mail.gmail.com> <15e8b9d21002241634s53a3ba0csf3f274e59f836835@mail.gmail.com> <201002251000.30778.peter.levart@marand.si> Message-ID: <560fb5ed1002250844x54659e94gfa83327bce9d5e38@mail.gmail.com> On Thu, Feb 25, 2010 at 10:00 AM, Peter Levart wrote: > For example, how do you find this: > > final *volatile Type1 result1 = &null; > final *Type2 result2 = &null; > doSomeApi(#(){ *result1 = someComputation(); *result2 = > otherComputation(); }); > doSomethingWith(*result1, *result2); > > It's still more than a month until April 1st. Introducing Both * and & makes java look like C which is certainly going to cause the sky to fall down. It's purely a perception problem of course, but a perception problem is still a problem. Ignoring the C-ishness of it, You're now committing both prefix * and prefix & for this use, which means not only that other future features can no longer use it, but also that parsing java code becomes more complicated, resulting in syntax errors become more obscure (as its harder for the compiler to figure out what you did mean). All this to avoid some minor effort for a use case that's relatively rare and should always be treated with much care. Hence I judge this proposal as a fantastically bad idea. NB1: The minor workaorund is to use AtomicReference/AtomicInteger and friends. NB2: Without an explicit 'safe / unsafe' closure concept, turning local variables into heap-allocated non-thread safe variables is a major change and something that you should always be careful with. *with* an explicit 'safe / unsafe' closures concept, the solution is of course to state that safe closures may mutate variables from their enclosing scope, and for the compiler to sort out how to make it happen. This idea is therefore either dangerous or unnecessary, depending on if safe/unsafe closures somehow make it into lambda-dev. From reinier at zwitserloot.com Thu Feb 25 08:57:45 2010 From: reinier at zwitserloot.com (Reinier Zwitserloot) Date: Thu, 25 Feb 2010 17:57:45 +0100 Subject: Effectively final effective? In-Reply-To: <4B85B844.9050507@adres.pl> References: <17b2302a1002241412s283a07c6p8fedc75106e38dbf@mail.gmail.com> <15e8b9d21002241449y69b555e0ie89db6b07e650254@mail.gmail.com> <4B85B090.9090706@optrak.co.uk> <4B85B844.9050507@adres.pl> Message-ID: <560fb5ed1002250857i42a63aa2u4d3c1cebc74762b@mail.gmail.com> On Thu, Feb 25, 2010 at 12:37 AM, Artur Biesiadowski wrote: > What about yet another solution ? > > ------------------ > int x = 1; > > Runnable x = new Runnable() {public void run() { > use(x); // you can use x as capture at moment of creation here > //x++; // illegal, x is not allowed be modified from anonymous/lamba > scopes > }}; > > x = 2; // you CAN mutate it further in outside scope > > x.run(); // use(1) will be called inside, mutation of x is not visible > in already captured environment > Doing something which causes the compiler (which in practice almost always means: Your editor, right then and there as you make the mistake) to emit a clear and direct error/warning message is so vastly superior to a situation where something occurs which a programmer may not realize, and is invisible until runtime, but there's no warning or error, it's incomparable. That last situation is always bad and always leads to many hours of bug hunting. A single error of that kind is obviously worse than 1000 errors where your editor/compiler instantly points out what went wrong and points at the exact position where you need to apply a trivial fix. And yet that's what you are proposing: To avoid a simple, obvious, and direct error message ("this variable is implicitly final because it is accessed from a closure; if you'd like to mutate it, use AtomicReference"), you've set up a situation where you implicitly create a copy of a variable, and changing the one does not reflect into the other. Horrible, _horrible_ idea. There's no problem with implicit final. The argument that an explicit 'final' makes things for readable doesn't really fly; you can use the same reasoning to say that imports should go away (as well as the implicit import java.lang.*) because "java.lang.String" is less ambiguous and less magical than "String". Which I presume everyone realizes would be a pretty bad idea. *Every* single character of code in a source file is both noise and signal. You do the math on how much noise vs. how much signal there is to consider whether making it explicit is a good idea. This clearly doesn't cut the mustard; it's going to be 100% noise in many cases, and a rather weak signal in the remaining few. The situation is really very simple, either: 1. The implicit finalness is irrelevant, because nobody is mutating the variable. Knowing that a certain variable is final when it never gets modified in the first place in a scope that is self-contained (i.e. a local variable that is never modified) is utterly moot. 2. The implicit finalness is relevant (in that someone is looking at code with the intention of adding a new line of code that mutates an implicit final), but the error in thinking of that programmer is pointed out _immediately_, and the error message explains exactly what happened. He can fix it in 5 seconds, and IDEs will most likely offer an instant 'hoist into AtomicX construct' as well. Said differently, instead of having a publically visible 'final' cluttering up your source file, you get a "Just In Time" completely automatic final keyword: The moment you actually attempt to write a line that mutates an implicit final, the 'final' keyword "appears" just in time (it actually takes the form of an immediate IDE error, but this is in many ways even better: It's pointing right at where you are typing! - the actual variable declaration may not even be on your screen). From peter.levart at marand.si Thu Feb 25 09:19:10 2010 From: peter.levart at marand.si (Peter Levart) Date: Thu, 25 Feb 2010 18:19:10 +0100 Subject: Effectively final effective? In-Reply-To: <560fb5ed1002250844x54659e94gfa83327bce9d5e38@mail.gmail.com> References: <201002251000.30778.peter.levart@marand.si> <560fb5ed1002250844x54659e94gfa83327bce9d5e38@mail.gmail.com> Message-ID: <201002251819.10312.peter.levart@marand.si> On Thursday 25 February 2010 17:44:44 Reinier Zwitserloot wrote: > It's still more than a month until April 1st. Sorry, It was another bad idea. I apologize. Now, seriously: > .... > > NB2: Without an explicit 'safe / unsafe' closure concept, turning local > variables into heap-allocated non-thread safe variables is a major change > and something that you should always be careful with. *with* an explicit > 'safe / unsafe' closures concept, the solution is of course to state that > safe closures may mutate variables from their enclosing scope, and for the You meant 'unsafe', didn't you? > compiler to sort out how to make it happen. This idea is therefore either > dangerous or unnecessary, depending on if safe/unsafe closures somehow make > it into lambda-dev. > Would a '@Safe' qualifier as a JSR 308 checker plug-in be enough to warn against 'unsafe' functions? Peter From reinier at zwitserloot.com Thu Feb 25 09:19:23 2010 From: reinier at zwitserloot.com (Reinier Zwitserloot) Date: Thu, 25 Feb 2010 18:19:23 +0100 Subject: Nice to @Share? In-Reply-To: <4B83D6D7.8030809@univ-mlv.fr> References: <17b2302a1002221238p2db47f2albcd2e6bb23ddafab@mail.gmail.com> <4B839E5C.5010502@sun.com> <560fb5ed1002230203p94ff249j5e81c8bb1db99bfa@mail.gmail.com> <4B83D6D7.8030809@univ-mlv.fr> Message-ID: <560fb5ed1002250919x210a30aag50d92557d77d5b6b@mail.gmail.com> On Tue, Feb 23, 2010 at 2:23 PM, R?mi Forax wrote: > > Reiner, > I think you can introduce any keyword before a local declaration without > breaking compatibility. > So the best is to choose a keyword that means what you really want. > That's not really true - lets say the keyword becomes 'modifiable', then this would be legal: modifiable modifiable modifiable; and backwards compatible, but it won't be readable and will cause anyone seeing it to do a double-take. It's not that big of a deal, the amount of types out there called 'modifiable' (with lower casing) is probably 0, and not that many variables will be called 'modifiable' either, but just proving a point. It's also somewhat easier for the parser to generate sane error messages if the amount of context-sensitive keywords are kept to a minimum (if modifiable is used wrongly and it is a context sensitive keyword, then the parser must erroneously conclude that it is not a keyword, and likely attempt a spectacularly wrong interpretation of what you were trying to write, resulting in a suboptimal parse error. By overriding the meaning of keywords, you run similar risks, but these seem somewhat smaller to me - even if the parser can't tell what kind of 'public' you are talking about, enumerating the possible meanings is more straightforward). These are all relatively insignificant arguments, but the more I think about it, the more 'public' seems like a good name for it, and the fact that it's an existing keyword is a coincidence. The best alternative keyword I've come up with is 'modifiable', but that's somewhat hard to type (also a relatively insignificant argument, but a decent tie breaker) and still doesn't really cover the meaning that well. The meaning is better covered with "nonfinal" (to highlight that you explicitly do not intend for it to be implicitly final), but that feels a bit awkward. > > By the way, I am very happy with final (or implicit final). > When I code and find that I need a non-final local variable, > I first try to rewrite the code to avoid to use non-final > before trying to use a mutable object. > > void foreach(List list, #void(T) fun) { > for(String element: list) > fun.invoke(element); > } > ... > @shared int length = 0; > foreach(alist, #(String element){ > length += element.length; > }); > > can be rewritten as: > > int reduce(List list, #int(T, int) fun, int acc) { > for(String element: list) > acc+= fun.invoke(element); > } > return acc; > } > ... > int length = reduce(alist, #(String element, int acc) { > return element.length + acc; > }); > > In my opinion, @Shared is not that important if you provide APIs > to easily play with collections. > > [...] > > > --Reinier Zwitserloot > > > > R?mi > > From reinier at zwitserloot.com Thu Feb 25 09:28:20 2010 From: reinier at zwitserloot.com (Reinier Zwitserloot) Date: Thu, 25 Feb 2010 18:28:20 +0100 Subject: Effectively final effective? In-Reply-To: <201002251819.10312.peter.levart@marand.si> References: <201002251000.30778.peter.levart@marand.si> <560fb5ed1002250844x54659e94gfa83327bce9d5e38@mail.gmail.com> <201002251819.10312.peter.levart@marand.si> Message-ID: <560fb5ed1002250928r2b5c1794gf0b8a9ea3ef7d927@mail.gmail.com> Er, I guess. So, "safe" means that the _closure_ is safe to be transported into different threads, and "unsafe" means that the closure cannot be transported? I tend to think from the side of the callee, so I mixed them up. Until every Joe, Tom, and Mary java programmer run the checker framework, An "@Safe" qualifier isn't going to change the notion that doing unexpected magic (either turning a variable into a heap-allocated concept, _or_ making an implicit copy) is going to result in folks spending many hours hunting for obscure bugs. Also, if a variable is turned into a heap allocation, does it then become legal to put "volatile" on a local variable declaration? What should happen if a variable isn't hoisted into the heap? Just ignore the volatile? Emit a warning or error that this isn't making any sense? What's so bad about either a keyword ('public', 'nonfinal', 'modifiable', or some such), or usage of AtomicReference (both of those explicitly pointed out by the compiler/IDE error message when you attempt mutation of an implicit final)? Without safe / unsafe closures, one must presume closures are not restricted to the current thread, and as a result making a variable mutable and shared across the closure boundary is a very very big deal, and warrants some sort of explicit flag. --Reinier Zwitserloot On Thu, Feb 25, 2010 at 6:19 PM, Peter Levart wrote: > On Thursday 25 February 2010 17:44:44 Reinier Zwitserloot wrote: > > It's still more than a month until April 1st. > > Sorry, It was another bad idea. I apologize. > > Now, seriously: > > > .... > > > > NB2: Without an explicit 'safe / unsafe' closure concept, turning local > > variables into heap-allocated non-thread safe variables is a major change > > and something that you should always be careful with. *with* an explicit > > 'safe / unsafe' closures concept, the solution is of course to state that > > safe closures may mutate variables from their enclosing scope, and for > the > You meant 'unsafe', didn't you? > > compiler to sort out how to make it happen. This idea is therefore either > > dangerous or unnecessary, depending on if safe/unsafe closures somehow > make > > it into lambda-dev. > > > > Would a '@Safe' qualifier as a JSR 308 checker plug-in be enough to warn > against 'unsafe' functions? > > Peter > From peter.levart at marand.si Thu Feb 25 09:44:31 2010 From: peter.levart at marand.si (Peter Levart) Date: Thu, 25 Feb 2010 18:44:31 +0100 Subject: Effectively final effective? In-Reply-To: <560fb5ed1002250928r2b5c1794gf0b8a9ea3ef7d927@mail.gmail.com> References: <201002251819.10312.peter.levart@marand.si> <560fb5ed1002250928r2b5c1794gf0b8a9ea3ef7d927@mail.gmail.com> Message-ID: <201002251844.31532.peter.levart@marand.si> On Thursday 25 February 2010 18:28:20 Reinier Zwitserloot wrote: > Er, I guess. So, "safe" means that the _closure_ is safe to be transported > into different threads, and "unsafe" means that the closure cannot be > transported? > > I tend to think from the side of the callee, so I mixed them up. > > Until every Joe, Tom, and Mary java programmer run the checker framework, An > "@Safe" qualifier isn't going to change the notion that doing unexpected > magic (either turning a variable into a heap-allocated concept, _or_ making > an implicit copy) is going to result in folks spending many hours hunting > for obscure bugs. As I understand, JSR 308 will be integrated into JDK7 and '@Safe' checker could be enabled by default. The writers of concurrent execution frameworks (such as fork/join or ParallelArray) would just have to mark their APIs with '@Safe' annotations and a special checker's AnnotatedTypeFactory would analyze the AST to deduce implicitly if a particular function-typed expression is @Safe or not... > > Also, if a variable is turned into a heap allocation, does it then become > legal to put "volatile" on a local variable declaration? What should happen > if a variable isn't hoisted into the heap? Just ignore the volatile? Emit a > warning or error that this isn't making any sense? > > What's so bad about either a keyword ('public', 'nonfinal', 'modifiable', or > some such), or usage of AtomicReference (both of those explicitly pointed > out by the compiler/IDE error message when you attempt mutation of an > implicit final)? Without safe / unsafe closures, one must presume closures > are not restricted to the current thread, and as a result making a variable > mutable and shared across the closure boundary is a very very big deal, and > warrants some sort of explicit flag. > > --Reinier Zwitserloot > Peter From reinier at zwitserloot.com Thu Feb 25 10:55:09 2010 From: reinier at zwitserloot.com (Reinier Zwitserloot) Date: Thu, 25 Feb 2010 19:55:09 +0100 Subject: Effectively final effective? In-Reply-To: <201002251844.31532.peter.levart@marand.si> References: <201002251819.10312.peter.levart@marand.si> <560fb5ed1002250928r2b5c1794gf0b8a9ea3ef7d927@mail.gmail.com> <201002251844.31532.peter.levart@marand.si> Message-ID: <560fb5ed1002251055n29b77c22i722ca1af2c6ec5cc@mail.gmail.com> The way I understand JSR308, the only thing that's going to be integrated into JDK7 is the ability to stick annotations on arbitrary types. The Checker Framework right now already "allows" this by reparsing your source with a custom build of JDK7 which parses this: List x; As if the annotation wasn't commented out. The actual checker framework itself, as well as the batch of annotations + processors that ship with it 'out of the box', aren't going to be in JDK7, I'm pretty sure. --Reinier Zwitserloot On Thu, Feb 25, 2010 at 6:44 PM, Peter Levart wrote: > On Thursday 25 February 2010 18:28:20 Reinier Zwitserloot wrote: > > Er, I guess. So, "safe" means that the _closure_ is safe to be > transported > > into different threads, and "unsafe" means that the closure cannot be > > transported? > > > > I tend to think from the side of the callee, so I mixed them up. > > > > Until every Joe, Tom, and Mary java programmer run the checker framework, > An > > "@Safe" qualifier isn't going to change the notion that doing unexpected > > magic (either turning a variable into a heap-allocated concept, _or_ > making > > an implicit copy) is going to result in folks spending many hours hunting > > for obscure bugs. > > As I understand, JSR 308 will be integrated into JDK7 and '@Safe' checker > could be enabled by default. The writers of concurrent execution frameworks > (such as fork/join or ParallelArray) would just have to mark their APIs with > '@Safe' annotations and a special checker's AnnotatedTypeFactory would > analyze the AST to deduce implicitly if a particular function-typed > expression is @Safe or not... > > > > > Also, if a variable is turned into a heap allocation, does it then become > > legal to put "volatile" on a local variable declaration? What should > happen > > if a variable isn't hoisted into the heap? Just ignore the volatile? Emit > a > > warning or error that this isn't making any sense? > > > > What's so bad about either a keyword ('public', 'nonfinal', 'modifiable', > or > > some such), or usage of AtomicReference (both of those explicitly pointed > > out by the compiler/IDE error message when you attempt mutation of an > > implicit final)? Without safe / unsafe closures, one must presume > closures > > are not restricted to the current thread, and as a result making a > variable > > mutable and shared across the closure boundary is a very very big deal, > and > > warrants some sort of explicit flag. > > > > --Reinier Zwitserloot > > > > Peter > From john at milsson.nu Thu Feb 25 12:26:48 2010 From: john at milsson.nu (John Nilsson) Date: Thu, 25 Feb 2010 21:26:48 +0100 Subject: Effectively final effective? In-Reply-To: <560fb5ed1002250857i42a63aa2u4d3c1cebc74762b@mail.gmail.com> References: <17b2302a1002241412s283a07c6p8fedc75106e38dbf@mail.gmail.com> <15e8b9d21002241449y69b555e0ie89db6b07e650254@mail.gmail.com> <4B85B090.9090706@optrak.co.uk> <4B85B844.9050507@adres.pl> <560fb5ed1002250857i42a63aa2u4d3c1cebc74762b@mail.gmail.com> Message-ID: On Thu, Feb 25, 2010 at 5:57 PM, Reinier Zwitserloot < reinier at zwitserloot.com> wrote: > And yet that's what you are proposing: To avoid a simple, obvious, and > direct error message ("this variable is implicitly final because it is > accessed from a closure; if you'd like to mutate it, use AtomicReference"), > you've set up a situation where you implicitly create a copy of a variable, > and changing the one does not reflect into the other. Horrible, _horrible_ > idea. > Just to add to the noise. I strongly disagree. The implicit copy is the most natural and intuitive semantics of closures in Java IMHO. Getting rid of "final" just to save some keystrokes sounds very bad to me. This is one of the best keywords in the language! In argument lists it may seem somewhat superfluous given that you never expect arguments to be mutated anyway. So I'm all for adding a rule to Java which makes arguments implicitly final (I know this won't happen, but I would support it). However when it comes to local variables I demand from everyone in my team to always declare every single one of them final. To me it is bad style to use mutable local variables unless you have a damn good reason for it. And most of the times, you don't. This policy makes code easy to read and maintain. I don't have to execute the whole code block in my head to find out what that variable is used for, I simply looks at it's definition. Then there is another aspect to it. In my IDE (Eclipse) keywords are coloured different from types and variable names. It is also the case that final is a constant in every variable declaration, while the type and name may change. So the coloured keyword final in a local scope screams "Variable declaration here!". This also contributes to readability. If anything, for this very reason, I would like to see the keyword mutable added to the language, and required as a prefix for non-final declarations. So to summarize, to me the only reason to _not_ use the final-keyword is to signal "!DANGER! here be dragons around" when declaring a variable. And one reason for wanting to do so in the context of lambdas is to allow mutability of the variable, even if it is to be used as effectively final inside the lambda. As Java is today I can only imagine one use-case for this though (but that can be either because of lack of experience on my part, or lack of lambdas in Java) and that is for the annoying tendancy for having to declare Files and Streams and such mutable to have the variable available for the final-block when closing. But I hear this is being addressed elsewhere in Java7. BR, John From abies at adres.pl Thu Feb 25 12:38:38 2010 From: abies at adres.pl (Artur Biesiadowski) Date: Thu, 25 Feb 2010 21:38:38 +0100 Subject: Effectively final effective? In-Reply-To: <560fb5ed1002250857i42a63aa2u4d3c1cebc74762b@mail.gmail.com> References: <17b2302a1002241412s283a07c6p8fedc75106e38dbf@mail.gmail.com> <15e8b9d21002241449y69b555e0ie89db6b07e650254@mail.gmail.com> <4B85B090.9090706@optrak.co.uk> <4B85B844.9050507@adres.pl> <560fb5ed1002250857i42a63aa2u4d3c1cebc74762b@mail.gmail.com> Message-ID: <4B86DFCE.7020304@adres.pl> Reinier Zwitserloot wrote: > And yet that's what you are proposing: To avoid a simple, obvious, and > direct error message ("this variable is implicitly final because it is > accessed from a closure; if you'd like to mutate it, use > AtomicReference"), you've set up a situation where you implicitly > create a copy of a variable, and changing the one does not reflect > into the other. Horrible, _horrible_ idea. I'm suggesting making it effectively final in the scope of lambda/anonymous class, just not outside of it. As far as bugs or understanding the concept is concerned, I do not see any fundamental difference between int i; for (i =0; i < 10; i++ ) { list.add(new Integer(i)); } print(list); and int i; for (i =0; i < 10; i++ ) { list.add(new Object() { public String toString() { return Integer.toString(i);}}); } print(list); In both cases, value of the variable is captured at the moment it appears in source code. Same way as you will not expect to see ten of '10' in first case, you won't expect to see ten '10' in second case. Still, you are arguing that while first case is obvious and will not cause bugs, second case HAS to be written int i; for (i =0; i < 10; i++ ) { final int j = i; list.add(new Object() { public String toString() { return Integer.toString(j);}}); } print(list); (with final possibly being removed if I understand your position correctly). Can you imagine explaining to newcomers in java (after final is removed) that they have to do int j =i; in the loop to pass the variable inside, because if they would NOT reassign it to different name, they would make a mistake of expecting 10x10 as the output? Example with loop is one of the main cases where I'm really angry about current rules (as I use final modifier for the most of normal, non-loop variables anyway) - of course, in real life it is bit more complicated :) I think that having similar rules for method arguments and variables passed into lambdas/anonymous classes is not bad thing (after all, in both cases you are doing a method/function call, just code is inlined in case of lambda/anonymous class as opposed to normal method call). Given something like @Shared would be accepted, all attempts to modify variable from inside closed scope would result in warning (make it shared or use in read mode only), but would not impose limits on the outside. Going further this route, to achieve symmetry between the inlined methods and normal methods, I could even imagine this evolving into by-reference passing for java. You would declare method void swap(@Shared int a, @Shared int b); and then you could pass variables declared as @Shared to swap, knowing that they will be mutated inside your scope. If passed into non-shared receiver, current value would be used. I would probably disallow automagical 'sharing' of normal variable if it is used as argument to method expecting @Shared - it should be compilation error (as @Shared will be most probably just a syntax sugar around AtomicXYZ or WrappedXYZ). Anyway, I'm not advocating particular solution for @Shared here - just showing that there is a certain symmetry between method calls and lambda instantiations and I don't think it is unreasonable to expect closing over CURRENT value of variable being natural. Regards, Artur Biesiadowski From jkuhnert at gmail.com Thu Feb 25 12:39:30 2010 From: jkuhnert at gmail.com (Jesse Kuhnert) Date: Thu, 25 Feb 2010 15:39:30 -0500 Subject: Effectively final effective? In-Reply-To: References: <17b2302a1002241412s283a07c6p8fedc75106e38dbf@mail.gmail.com> <15e8b9d21002241449y69b555e0ie89db6b07e650254@mail.gmail.com> <4B85B090.9090706@optrak.co.uk> <4B85B844.9050507@adres.pl> <560fb5ed1002250857i42a63aa2u4d3c1cebc74762b@mail.gmail.com> Message-ID: <7926817e1002251239g7745a353x878bd996447eba14@mail.gmail.com> Only a very small and obvious note to add. Whether the project description is lambda expressions / closures / etc, when it makes its way out in to the real world everyone will probably refer to them as closures whether anyone likes it or not. Which means they will likely also immediately compare usage here against other languages to figure out how it works - or try to assume their way through it based on existing experience.. It might be a good idea to keep syntax overhead and all other changes with this in mind when making decisions. Not that that makes it any easier or the above statements strictly true for the general case, but sounds pretty close. On Thu, Feb 25, 2010 at 3:26 PM, John Nilsson wrote: > On Thu, Feb 25, 2010 at 5:57 PM, Reinier Zwitserloot < > reinier at zwitserloot.com> wrote: > However when it comes to local variables I demand from everyone in my team > to always declare every single one of them final. To me it is bad style to > use mutable local variables unless you have a damn good reason for it. And > most of the times, you don't. This policy makes code easy to read and > maintain. I don't have to execute the whole code block in my head to find > out what that variable is used for, I simply looks at it's definition. > BR, > John > > From reinier at zwitserloot.com Thu Feb 25 13:26:58 2010 From: reinier at zwitserloot.com (Reinier Zwitserloot) Date: Thu, 25 Feb 2010 22:26:58 +0100 Subject: Effectively final effective? In-Reply-To: References: <17b2302a1002241412s283a07c6p8fedc75106e38dbf@mail.gmail.com> <15e8b9d21002241449y69b555e0ie89db6b07e650254@mail.gmail.com> <4B85B090.9090706@optrak.co.uk> <4B85B844.9050507@adres.pl> <560fb5ed1002250857i42a63aa2u4d3c1cebc74762b@mail.gmail.com> Message-ID: <560fb5ed1002251326q6d12c59wa3ca36f496a620c9@mail.gmail.com> On Thu, Feb 25, 2010 at 9:26 PM, John Nilsson wrote: > Just to add to the noise. I strongly disagree. The implicit copy is the > most natural and intuitive semantics of closures in Java IMHO. > > I don't think silently making a copy is intuitive. Which disproves that it is intuitive in the first place. Instead of saying things are "intuitive" without explaining why, perhaps try some arguments that aren't based on gut feeling. Getting rid of "final" just to save some keystrokes sounds very bad to me. > This is one of the best keywords in the language! > I think you're confusing "keyword" with "feature". If you actually meant the keyword, presumably your later spiel about how "final" is useful to you because it indicates a local variable declaration, you're not making a consistent argument. In that same spiel you point out that your IDE has fancy colouring capabilities. One of the many capabilities you can build into any IDE is for it to highlight certain constructs. Such as variable declarations. > However when it comes to local variables I demand from everyone in my team > to always declare every single one of them final. > Good lord. Glad that I don't work for you. Do you force your team to write "java.lang.String" instead of just "String" too? > To me it is bad style to use mutable local variables unless you have a damn > good reason for it. > for (int i = 0; i < 10; i++) { ... } And most of the times, you don't. [have an excuse to make a local variable > non-final] > Balderdash. What's even more amazing is that you think the keyword 'final' is a good idea in the first place, if you've marked 95%+ of all your parameters and local variables with it. That's some awesome amount of noise right there. Stuff being final by default and requiring a keyword to make them non-final is effectively what we're debating here, so you're not making much sense. This policy makes code easy to read and maintain. > I disagree. A casual glance at thousands of open source java projects indicate I'm certainly not the only one. Let's hypothesize that you're on to something. This notion hasn't seeped very far through the greater java community yet, and now you are advocating that new language design features should presume everyone feels and codes the same way. I take it you now realize this is ridiculous. From john at milsson.nu Thu Feb 25 13:56:23 2010 From: john at milsson.nu (John Nilsson) Date: Thu, 25 Feb 2010 22:56:23 +0100 Subject: Effectively final effective? In-Reply-To: <560fb5ed1002251326q6d12c59wa3ca36f496a620c9@mail.gmail.com> References: <17b2302a1002241412s283a07c6p8fedc75106e38dbf@mail.gmail.com> <15e8b9d21002241449y69b555e0ie89db6b07e650254@mail.gmail.com> <4B85B090.9090706@optrak.co.uk> <4B85B844.9050507@adres.pl> <560fb5ed1002250857i42a63aa2u4d3c1cebc74762b@mail.gmail.com> <560fb5ed1002251326q6d12c59wa3ca36f496a620c9@mail.gmail.com> Message-ID: I think we just disagree ;-) On Thu, Feb 25, 2010 at 10:26 PM, Reinier Zwitserloot < reinier at zwitserloot.com> wrote: > On Thu, Feb 25, 2010 at 9:26 PM, John Nilsson wrote: > >> Just to add to the noise. I strongly disagree. The implicit copy is the >> most natural and intuitive semantics of closures in Java IMHO. >> > I don't think silently making a copy is intuitive. Which disproves that it > is intuitive in the first place. Instead of saying things are "intuitive" > without explaining why, perhaps try some arguments that aren't based on gut > feeling. > Which would be the IMHO part of that statement. For arguments I refer to earlier mail from Artur Biesiadowski. But as an example, it is already possible, and at least to me perfectly intuitive, to write for(final T t : ts). > Getting rid of "final" just to save some keystrokes sounds very bad to me. >> This is one of the best keywords in the language! >> > I think you're confusing "keyword" with "feature". If you actually meant > the keyword, presumably your later spiel about how "final" is useful to you > because it indicates a local variable declaration, you're not making a > consistent argument. In that same spiel you point out that your IDE has > fancy colouring capabilities. One of the many capabilities you can build > into any IDE is for it to highlight certain constructs. Such as variable > declarations. > Consistent or not, it's my take on the issue. And as always, YMMV. Yes you are correct that this is possible. But as it stands now, it is not necessary, I get two features at the cost of one. Immutability and visibility. > However when it comes to local variables I demand from everyone in my team >> to always declare every single one of them final. >> > > Good lord. Glad that I don't work for you. Do you force your team to write > "java.lang.String" instead of just "String" too? > Of course not. And if you think that is comparable maybe you misunderstand me. It is not simply for the syntax. By demanding immutability you also force people to think about what they are doing and write somewhat coherent code. And as with all rules, there are exceptions of course. YMMV but it works for us. > To me it is bad style to use mutable local variables unless you have a >> damn good reason for it. >> > for (int i = 0; i < 10; i++) { > ... > } > This is one of those odd examples that actual works fine despite mutability. But it is the same as with method arguments. You don't expect people to do stupid things like mutate i inside the body of the loop. Also, if you have the need for this construct you are either doing something wrong, or are a sufficiently competent coder to know what you are doing, in which case I trust you to write good code from the beginning. > And most of the times, you don't. [have an excuse to make a local variable >> non-final] >> > > Balderdash. What's even more amazing is that you think the keyword 'final' > is a good idea in the first place, if you've marked 95%+ of all your > parameters and local variables with it. That's some awesome amount of noise > right there. Stuff being final by default and requiring a keyword to make > them non-final is effectively what we're debating here, so you're not making > much sense. > If that was the case I would support it wholeheartedly. Replacing final with mutable across the language works fine with me. This is not what is being discussed though, what is being discussed is an isolated exception to the general syntax of the language. One of the reasons I like Scala is for the symetry of val/var/def three simple letters for each construct. In Java you instead have either fairly long or no keyword at all, and I happen to dislike the no-keyword version. > This policy makes code easy to read and maintain. >> > > I disagree. A casual glance at thousands of open source java projects > indicate I'm certainly not the only one. Let's hypothesize that you're on to > something. This notion hasn't seeped very far through the greater java > community yet, and now you are advocating that new language design features > should presume everyone feels and codes the same way. I take it you now > realize this is ridiculous. > Even if this notion hasn't spread throughout the Java world yet. The general notion of immutability seems very accepted in the functional world, and it looks like it is gaining traction as a result of the concurrency preparations taking place everywhere. I simply present a data point to highlight that there are radically different opinions from yours on this matter (but it can't be that radically different because I actually like project Lombok). I am fully aware that I am just that, one data point. BR, John From i30817 at gmail.com Thu Feb 25 18:46:26 2010 From: i30817 at gmail.com (Paulo Levi) Date: Fri, 26 Feb 2010 02:46:26 +0000 Subject: Effectively final effective? In-Reply-To: References: <17b2302a1002241412s283a07c6p8fedc75106e38dbf@mail.gmail.com> <15e8b9d21002241449y69b555e0ie89db6b07e650254@mail.gmail.com> <4B85B090.9090706@optrak.co.uk> <4B85B844.9050507@adres.pl> <560fb5ed1002250857i42a63aa2u4d3c1cebc74762b@mail.gmail.com> <560fb5ed1002251326q6d12c59wa3ca36f496a620c9@mail.gmail.com> Message-ID: <212322091002251846k397efc92rc10e1462175cd524@mail.gmail.com> Final should have been the default, non null should have been in the language (and default), constructors should have not been allowed to call protected methods, closures should have come on day one, reification should have been included in generics, i miss map and reduce on arrays & lists, i like list generators and RIIA and would love it in java. The discussion is if it is worthwhile to allow final by default for captured variables in lambdas considering that a) anonymous classes have the opposite behavior, so things get slightly inconsistent. b) it is safer for threaded use. From jkuhnert at gmail.com Thu Feb 25 19:25:59 2010 From: jkuhnert at gmail.com (Jesse Kuhnert) Date: Thu, 25 Feb 2010 22:25:59 -0500 Subject: Effectively final effective? In-Reply-To: <212322091002251846k397efc92rc10e1462175cd524@mail.gmail.com> References: <17b2302a1002241412s283a07c6p8fedc75106e38dbf@mail.gmail.com> <15e8b9d21002241449y69b555e0ie89db6b07e650254@mail.gmail.com> <4B85B090.9090706@optrak.co.uk> <4B85B844.9050507@adres.pl> <560fb5ed1002250857i42a63aa2u4d3c1cebc74762b@mail.gmail.com> <560fb5ed1002251326q6d12c59wa3ca36f496a620c9@mail.gmail.com> <212322091002251846k397efc92rc10e1462175cd524@mail.gmail.com> Message-ID: <7926817e1002251925md1f3686k2742d81a75c60d5f@mail.gmail.com> If anonymous inner classes are something everyone thinks shouldn't be there anyway then they shouldn't be considered too strictly. Shouldn't they become obsolete with this project anyways? The threading issues do sound like something worthwhile exploring very thoroughly though, given the predominent environment the language runs in. Tough choice there.. On Thursday, February 25, 2010, Paulo Levi wrote: > Final should have been the default, non null should have been in the > language (and default), constructors should have not been allowed to call > protected methods, closures should have come on day one, reification should > have been included in generics, i miss map and reduce on arrays & lists, i > like list generators and RIIA and would love it in java. > > The discussion is if it is worthwhile to allow final by default for captured > variables in lambdas considering that > a) anonymous classes have the opposite behavior, so things get slightly > inconsistent. > b) it is safer for threaded use. > > From i30817 at gmail.com Thu Feb 25 19:43:54 2010 From: i30817 at gmail.com (Paulo Levi) Date: Fri, 26 Feb 2010 03:43:54 +0000 Subject: Effectively final effective? In-Reply-To: <7926817e1002251925md1f3686k2742d81a75c60d5f@mail.gmail.com> References: <15e8b9d21002241449y69b555e0ie89db6b07e650254@mail.gmail.com> <4B85B090.9090706@optrak.co.uk> <4B85B844.9050507@adres.pl> <560fb5ed1002250857i42a63aa2u4d3c1cebc74762b@mail.gmail.com> <560fb5ed1002251326q6d12c59wa3ca36f496a620c9@mail.gmail.com> <212322091002251846k397efc92rc10e1462175cd524@mail.gmail.com> <7926817e1002251925md1f3686k2742d81a75c60d5f@mail.gmail.com> Message-ID: <212322091002251943i7f48f1bbh69f2864084ca6942@mail.gmail.com> On Fri, Feb 26, 2010 at 3:25 AM, Jesse Kuhnert wrote: > If anonymous inner classes are something everyone thinks shouldn't be > there anyway then they shouldn't be considered too strictly. Shouldn't > they become obsolete with this project anyways? > > Not for multiple methods templates (like mouselistener). I use inner classes then, but many do not. From jkuhnert at gmail.com Thu Feb 25 19:59:53 2010 From: jkuhnert at gmail.com (Jesse Kuhnert) Date: Thu, 25 Feb 2010 22:59:53 -0500 Subject: Effectively final effective? In-Reply-To: <212322091002251943i7f48f1bbh69f2864084ca6942@mail.gmail.com> References: <4B85B090.9090706@optrak.co.uk> <4B85B844.9050507@adres.pl> <560fb5ed1002250857i42a63aa2u4d3c1cebc74762b@mail.gmail.com> <560fb5ed1002251326q6d12c59wa3ca36f496a620c9@mail.gmail.com> <212322091002251846k397efc92rc10e1462175cd524@mail.gmail.com> <7926817e1002251925md1f3686k2742d81a75c60d5f@mail.gmail.com> <212322091002251943i7f48f1bbh69f2864084ca6942@mail.gmail.com> Message-ID: <7926817e1002251959w59aeb7e8ya308005721c541fc@mail.gmail.com> That does make some sense. They could also be improved with the closure change but then that's a fundamental change. No experience watching language change features discussed before but I find it hard to think of as being the same as "API" changes wrt backwards compatibility.. Compromising the "best possible" for that doesn't completely sound assuring. Mire like java 2.0 if you were numbering it realistically instead of marketing or that-sounds-too-nerdy speak. On Thursday, February 25, 2010, Paulo Levi wrote: > > > On Fri, Feb 26, 2010 at 3:25 AM, Jesse Kuhnert wrote: > > > If anonymous inner classes are something everyone thinks shouldn't be > there anyway then they shouldn't be considered too strictly. Shouldn't > they become obsolete with this project anyways? > > Not for multiple methods templates (like mouselistener). > I use inner classes then, but many do not. > > > > From neal at gafter.com Thu Feb 25 22:52:17 2010 From: neal at gafter.com (Neal Gafter) Date: Thu, 25 Feb 2010 22:52:17 -0800 Subject: Method References Message-ID: <15e8b9d21002252252t45faaa9fq6859222b31a6052f@mail.gmail.com> Method References (ala CfJ 0.6a) would be a valuable addition to Project Lambda's specification. While the additional specification text is fairly small, it enable a number of useful idioms that would otherwise be quite verbose. The most obvious use case will be lambdas that would otherwise have to be written this way #(Type1 arg1, Type1 arg2, ...)(EnclosingClass.this.method(arg1, arg2, ...)) because they could be written like this this#method(Type1, Type2, ...) This simplifies a common pattern seen in code today that registers callbacks. From neal at gafter.com Thu Feb 25 23:11:06 2010 From: neal at gafter.com (Neal Gafter) Date: Thu, 25 Feb 2010 23:11:06 -0800 Subject: Exception Transparency Message-ID: <15e8b9d21002252311i3de2e97cs80b1495acfebeef6@mail.gmail.com> Exception Transparency, as it appears in BGGA and CfJ, would be a valuable addition to project lambda.? It can be usefully applied in most applications of higher-order functions. Take, for example, a lambda-enabled version of the Google Collections filtering method: Iterable filter( ??? Iterable input, ??? #bool(T) predicate) { ... } or, using my preferred syntax Iterable filter( ??? Iterable input, ??? (T)->bool predicate) { ... } This method returns an iterable that includes only those elements from the input iterable that pass the predicate. Unfortunately, if your predicate includes any code that might throw a checked exception (for example, looking up something in a database where SqlException might be thrown as a result) you cannot simply use this method.? At best, you can write your predicate so that it catches all checked exceptions and wraps them in unchecked exceptions.? Then, you had better remember to catch those unchecked exceptions in the caller if you need to handle them; the compiler won't help you remember to do so, as for checked exceptions.? In short, the absence of support for exception transparency in this API forces clients to undermine the exception safety feature of the Java language.? Without language support for exception transparency, API authors have no choice but to write their APIs this way. If exception transparency were added to the lambda specification, this method could be written this way: Iterable filter( ??? Iterable input, ??? #bool(T)(throws X) predicate) throws X { ... } or, using my preferred syntax Iterable filter( ??? Iterable input, ??? (T throws X)->bool predicate) throws X { ... } With this exception-transparent method, the caller can pass in a lambda as a predicate that throws any checked exceptions.? If any exceptions are thrown by the predicate, they propagate out to the caller of filter.? And the language requires the compiler to diagnose any uncaught checked exceptions at compile-time. Without explicit language support, it is not practical to write APIs that can act this way.? If API authors cannot use function types to write exception-transparent APIs, and clients of those APIs cannot use checked exceptions, we have effectively deprecated checked exceptions. I hope the project lambda specification is modified to include support for exception transparency. From i30817 at gmail.com Thu Feb 25 23:22:56 2010 From: i30817 at gmail.com (Paulo Levi) Date: Fri, 26 Feb 2010 07:22:56 +0000 Subject: Exception Transparency In-Reply-To: <15e8b9d21002252311i3de2e97cs80b1495acfebeef6@mail.gmail.com> References: <15e8b9d21002252311i3de2e97cs80b1495acfebeef6@mail.gmail.com> Message-ID: <212322091002252322u151a9635rcb4d03f52cc4061d@mail.gmail.com> Agree completely. The syntax is a extension of the parametric type inference that already exists in methods (so people already use it), so this is just the removal of a limitation -> exceptions can be parametric too. From i30817 at gmail.com Thu Feb 25 23:27:33 2010 From: i30817 at gmail.com (Paulo Levi) Date: Fri, 26 Feb 2010 07:27:33 +0000 Subject: Method References In-Reply-To: <15e8b9d21002252252t45faaa9fq6859222b31a6052f@mail.gmail.com> References: <15e8b9d21002252252t45faaa9fq6859222b31a6052f@mail.gmail.com> Message-ID: <212322091002252327s149a38abg5815d5794084df6c@mail.gmail.com> What about not "this" objects. In fact would work on constructors or does it need the reference. If constructors worked it would be a lightweight way to lazy initialize unknown objects (already possible with callable i guess). From neal at gafter.com Thu Feb 25 23:36:12 2010 From: neal at gafter.com (Neal Gafter) Date: Thu, 25 Feb 2010 23:36:12 -0800 Subject: Accessing non-final local variables from a lambda expression Message-ID: <15e8b9d21002252336o2f7280d2v157a9acd38ab29da@mail.gmail.com> I experimented a bit with a lambda-ized version of a modified Google Collections. Specifically, I was playing with the method Iterable filter( Iterable input, Predicate predicate) { ... } Using this method and an anonymous inner class, one can write a method that returns every other value in its input sequence: // before Iterable everyOther(Iterable input) { return filter(input, new Predicate{ boolean flag = true; public boolean apply(T input) { return (flag ^= true); } }); } The equivalent function-ized version of the API would be (expressed in my preferred syntax): Iterable filter( Iterable input, (T)->boolean predicate) { ... } But now I have a problem implementing everyOther. The only way to create a value of function type is to write a lambda expression, but where to put the flag? In BGGA, CfJ, and other languages with lambdas, one could write it simply like this Iterable everyOther(Iterable input) { boolean flag = true; return filter(input, (T t)->(flag ^= true)); } But project lambda forbids access to mutable local variables from the enclosing scope. I came up with the following workable approach // after Iterable everyOther(Iterable input) { class Alternator { boolean flag = true; boolean invoke() { return (flag ^= true); } } final Alternator alternate = new Alternator(); return filter(input, #(T t)(alternate.invoke()); } Comparing the before and the after, it appears that project lambda didn't do us any favors here. The "after" code is far worse. In our attempt to make the concurrent use cases "easier not to get wrong", we've placed a pile of obstacles before every programmer who uses function types in sequential code. It would be better to give a warning (not error) when a lambda accesses a mutable local variable from an enclosing scope, and provide one or more easy ways for programmers to suppress the warning. See CfJ < http://www.javac.info/closures-v06a.html> for my preferred approach. From neal at gafter.com Thu Feb 25 23:40:59 2010 From: neal at gafter.com (Neal Gafter) Date: Thu, 25 Feb 2010 23:40:59 -0800 Subject: Method References In-Reply-To: <212322091002252327s149a38abg5815d5794084df6c@mail.gmail.com> References: <15e8b9d21002252252t45faaa9fq6859222b31a6052f@mail.gmail.com> <212322091002252327s149a38abg5815d5794084df6c@mail.gmail.com> Message-ID: <15e8b9d21002252340l6e500883j7fbef7434063a6bd@mail.gmail.com> On Thu, Feb 25, 2010 at 11:27 PM, Paulo Levi wrote: > What about not "this" objects. > See CfJ for my specific proposal, under the section heading "Method References". The part before # is an arbitrary expression. In fact would work on constructors or does it need the reference. > I don't think constructors are a frequent enough use case that explicit support for them is needed as a language feature, but that could be done too. From lk at teamten.com Fri Feb 26 00:23:42 2010 From: lk at teamten.com (Lawrence Kesteloot) Date: Fri, 26 Feb 2010 00:23:42 -0800 Subject: Accessing non-final local variables from a lambda expression In-Reply-To: <15e8b9d21002252336o2f7280d2v157a9acd38ab29da@mail.gmail.com> References: <15e8b9d21002252336o2f7280d2v157a9acd38ab29da@mail.gmail.com> Message-ID: <997cab101002260023t570987beob147c83288da4a23@mail.gmail.com> > In BGGA, CfJ, and other languages with lambdas, one could write it > simply like this > > Iterable everyOther(Iterable input) { > ? ?boolean flag = true; > ? ?return filter(input, (T t)->(flag ^= true)); > } > > But project lambda forbids access to mutable local variables from the > enclosing scope. Iterable everyOther(Iterable input) { final boolean[] flag = new boolean[] { true }; return filter(input, (T t)->(flag[0] ^= true)); } It's not beautiful, but not the end of the world either. And the performance is comparable, since in any case you have to create and dereference some heap object. Lawrence From forax at univ-mlv.fr Fri Feb 26 00:42:04 2010 From: forax at univ-mlv.fr (=?ISO-8859-1?Q?R=E9mi_Forax?=) Date: Fri, 26 Feb 2010 09:42:04 +0100 Subject: Method References In-Reply-To: <15e8b9d21002252340l6e500883j7fbef7434063a6bd@mail.gmail.com> References: <15e8b9d21002252252t45faaa9fq6859222b31a6052f@mail.gmail.com> <212322091002252327s149a38abg5815d5794084df6c@mail.gmail.com> <15e8b9d21002252340l6e500883j7fbef7434063a6bd@mail.gmail.com> Message-ID: <4B87895C.9020208@univ-mlv.fr> Le 26/02/2010 08:40, Neal Gafter a ?crit : > On Thu, Feb 25, 2010 at 11:27 PM, Paulo Levi wrote: > > >> What about not "this" objects. >> >> > See CfJ for my specific proposal, > under the section heading "Method References". The part before # is an > arbitrary expression. > > In fact would work on constructors or does it need the reference. > >> > I don't think constructors are a frequent enough use case that explicit > support for them is needed as a language feature, but that could be done > too. > Having constructors enable to write the Design Pattern abstract factory in few lines. (reusing the example from wikipedia about abstract factory) interface GUIFactory{ public Button createButton(); } class Application{ public Application(GUIFactory factory){ Button button= factory.createButton(); button.paint(); } } public class ApplicationRunner{ public static void main(String[] args) { new Application(createOsSpecificFactory()); } public static GUIFactory createOsSpecificFactory() { int sys= readFromConfigFile("OS_TYPE"); if (sys== 0) { return WinButton#new(); } else { return OSXButton#new(); } } } R?mi From forax at univ-mlv.fr Fri Feb 26 00:44:07 2010 From: forax at univ-mlv.fr (=?ISO-8859-1?Q?R=E9mi_Forax?=) Date: Fri, 26 Feb 2010 09:44:07 +0100 Subject: Accessing non-final local variables from a lambda expression In-Reply-To: <997cab101002260023t570987beob147c83288da4a23@mail.gmail.com> References: <15e8b9d21002252336o2f7280d2v157a9acd38ab29da@mail.gmail.com> <997cab101002260023t570987beob147c83288da4a23@mail.gmail.com> Message-ID: <4B8789D7.1000804@univ-mlv.fr> Le 26/02/2010 09:23, Lawrence Kesteloot a ?crit : >> In BGGA, CfJ, and other languages with lambdas, one could write it >> simply like this >> >> Iterable everyOther(Iterable input) { >> boolean flag = true; >> return filter(input, (T t)->(flag ^= true)); >> } >> >> But project lambda forbids access to mutable local variables from the >> enclosing scope. >> > Iterable everyOther(Iterable input) { > final boolean[] flag = new boolean[] { true }; > return filter(input, (T t)->(flag[0] ^= true)); > } > > It's not beautiful, but not the end of the world either. And the > performance is comparable, since in any case you have to create and > dereference some heap object. > +1 > Lawrence R?mi From fredrik.ohrstrom at oracle.com Fri Feb 26 01:01:26 2010 From: fredrik.ohrstrom at oracle.com (=?ISO-8859-1?Q?Fredrik_=D6hrstr=F6m?=) Date: Fri, 26 Feb 2010 10:01:26 +0100 Subject: Method References In-Reply-To: <15e8b9d21002252252t45faaa9fq6859222b31a6052f@mail.gmail.com> References: <15e8b9d21002252252t45faaa9fq6859222b31a6052f@mail.gmail.com> Message-ID: <4B878DE6.4070100@oracle.com> Neal Gafter skrev: > Method References (ala CfJ 0.6a) would be a valuable addition to Project > Lambda's specification. While the additional specification text is fairly > small, it enable a number of useful idioms that would otherwise be quite > verbose. > > The most obvious use case will be lambdas that would otherwise have to be > written this way > > #(Type1 arg1, Type1 arg2, ...)(EnclosingClass.this.method(arg1, arg2, ...)) > > because they could be written like this > > this#method(Type1, Type2, ...) > > This simplifies a common pattern seen in code today that registers > callbacks Yes, this is a very important and common use case. If the parentheses and types are unnecessary, for example there is only one single method with that particular name, please allow the reference to be created without the types. Ie: cb = this#saveState; will work as expected as long as there is only a single saveState in this. //Fredrik From forax at univ-mlv.fr Fri Feb 26 02:45:06 2010 From: forax at univ-mlv.fr (=?ISO-8859-1?Q?R=E9mi_Forax?=) Date: Fri, 26 Feb 2010 11:45:06 +0100 Subject: Method References In-Reply-To: <4B878DE6.4070100@oracle.com> References: <15e8b9d21002252252t45faaa9fq6859222b31a6052f@mail.gmail.com> <4B878DE6.4070100@oracle.com> Message-ID: <4B87A632.4010100@univ-mlv.fr> Le 26/02/2010 10:01, Fredrik ?hrstr?m a ?crit : > Neal Gafter skrev: > >> Method References (ala CfJ 0.6a) would be a valuable addition to Project >> Lambda's specification. While the additional specification text is fairly >> small, it enable a number of useful idioms that would otherwise be quite >> verbose. >> >> The most obvious use case will be lambdas that would otherwise have to be >> written this way >> >> #(Type1 arg1, Type1 arg2, ...)(EnclosingClass.this.method(arg1, arg2, ...)) >> >> because they could be written like this >> >> this#method(Type1, Type2, ...) >> >> This simplifies a common pattern seen in code today that registers >> callbacks >> > Yes, this is a very important and common use case. If the parentheses > and types are unnecessary, for example there is only one single method > with that particular name, please allow the reference to be created > without the types. Ie: > > cb = this#saveState; > > will work as expected as long as there is only a single saveState in this. > > //Fredrik > Hi Fredrik, there is a source compatibility issue if you allow to not specify the type. class A { void m(String s) { ... } } .... ref = A#m; Now suppose I want to add a new method m(Integer), because String and Integer an unrelated, currently there is no source compatibility issue. If reference can be created without types, this change is no more compatible because the compiler should raise an error. R?mi From fredrik.ohrstrom at oracle.com Fri Feb 26 04:45:39 2010 From: fredrik.ohrstrom at oracle.com (=?ISO-8859-1?Q?Fredrik_=D6hrstr=F6m?=) Date: Fri, 26 Feb 2010 13:45:39 +0100 Subject: Method References In-Reply-To: <4B87A632.4010100@univ-mlv.fr> References: <15e8b9d21002252252t45faaa9fq6859222b31a6052f@mail.gmail.com> <4B878DE6.4070100@oracle.com> <4B87A632.4010100@univ-mlv.fr> Message-ID: <4B87C273.8000509@oracle.com> R?mi Forax skrev: > Hi Fredrik, > there is a source compatibility issue if you allow to not specify the type. > > Hi R?mi, there is a source compatibility issue if you allow to change the name of a method.... ;-) > If reference can be created without types, > this change is no more compatible because the compiler should raise an > error. > Obviously you can change the source code so that it no longer is possible to deduce which method to lookup without the types. But if you do, then you will get an error and you will have to specify the type. This is not a problem. //Fredrik From reinier at zwitserloot.com Fri Feb 26 05:09:54 2010 From: reinier at zwitserloot.com (Reinier Zwitserloot) Date: Fri, 26 Feb 2010 14:09:54 +0100 Subject: Effectively final effective? In-Reply-To: <7926817e1002251925md1f3686k2742d81a75c60d5f@mail.gmail.com> References: <15e8b9d21002241449y69b555e0ie89db6b07e650254@mail.gmail.com> <4B85B090.9090706@optrak.co.uk> <4B85B844.9050507@adres.pl> <560fb5ed1002250857i42a63aa2u4d3c1cebc74762b@mail.gmail.com> <560fb5ed1002251326q6d12c59wa3ca36f496a620c9@mail.gmail.com> <212322091002251846k397efc92rc10e1462175cd524@mail.gmail.com> <7926817e1002251925md1f3686k2742d81a75c60d5f@mail.gmail.com> Message-ID: <560fb5ed1002260509l3f5e2d98td49a5d959c5e3df5@mail.gmail.com> On Fri, Feb 26, 2010 at 4:25 AM, Jesse Kuhnert wrote: > If anonymous inner classes are something everyone thinks shouldn't be > there anyway then they shouldn't be considered too strictly. Shouldn't > they become obsolete with this project anyways? > > No. Anonymous Inner Classes will become obsolete as a way to do closure-esque things. They will not be obsolete for e.g. inline specification of for example a WindowListener, or really any other on-the-fly specification of an interface or (usually abstract) class in a place that really requires an object and not a function. > The threading issues do sound like something worthwhile exploring very > thoroughly though, given the predominent environment the language > runs in. Tough choice there.. > There are so many different sentiments floating around that its hard to tell the forest from the trees, but as far as I understand it, these are the three sentiments I've heard: 1. Variables accessed in closures should be silently moved into the heap. I haven't yet heard anything from this camp about what to do with "volatile", which would be a useful construct to them have on such local variables. This is somewhat tricky when the closure is run in another thread. 2. Variables accessed in closures will be made implicitly final. Any attempt to mutate them, either in the closure or outside of it, generates a compiler error. This camp is furthermore split into two: One camp wants a keyword of some sort to get #1 behaviour (so, _explicit_ movement to the heap), the other says that AtomicReference, AtomicInteger, and friends are good enough. 3. Variables accessed in closures will be silently copied, with the copy inside the closure being final, but the variable outside of it being mutable (but, as the closure gets a copy, any mutations don't show up in the closure). I've filed this away as crackpot, where it'll hopefully remain, as this is going to be very surprising to a lot of programmers and the only way this surprise is going to become obvious is after many hours spent debugging. 4. As with anonymous inner classes, variables cannot be accessed at all inside closures unless they are marked explicitly final. I'd say that with no distinction between safe and unsafe closures, Only option #2 (either variant), with option #4 as a very distant second, is sensible, and with safe/unsafe closures, Option #1 for unsafe closures, and either #2 or #4 for safe closures, seems reasonable. Exactly how to proceed is no clear to me, as this discussion seems to be heading into a painting the bike shed direction. NB: Jesse, regardless of the pros and cons of making a backwards incompatible java 2.0 release, Java 7 most assuredly is not going to be such a release, so whatever ideas you come up with, if they aren't backwards compatible, the only way its going to happen is for closures to not make it into Java7, and for Java8 to be a Java 2.0 release, which even then is very very unlikely. Mark Reinhold explicitly mentioned during Devoxx that he considers any attempt to make Java 2.0 to be rife with second version syndrome. > On Thursday, February 25, 2010, Paulo Levi wrote: > > Final should have been the default, non null should have been in the > > language (and default), constructors should have not been allowed to call > > protected methods, closures should have come on day one, reification > should > > have been included in generics, i miss map and reduce on arrays & lists, > i > > like list generators and RIIA and would love it in java. > > > > The discussion is if it is worthwhile to allow final by default for > captured > > variables in lambdas considering that > > a) anonymous classes have the opposite behavior, so things get slightly > > inconsistent. > > b) it is safer for threaded use. > > > > > > From reinier at zwitserloot.com Fri Feb 26 05:26:30 2010 From: reinier at zwitserloot.com (Reinier Zwitserloot) Date: Fri, 26 Feb 2010 14:26:30 +0100 Subject: Accessing non-final local variables from a lambda expression In-Reply-To: <4B8789D7.1000804@univ-mlv.fr> References: <15e8b9d21002252336o2f7280d2v157a9acd38ab29da@mail.gmail.com> <997cab101002260023t570987beob147c83288da4a23@mail.gmail.com> <4B8789D7.1000804@univ-mlv.fr> Message-ID: <560fb5ed1002260526p7947bc95lcafce224f0e21463@mail.gmail.com> Neal, this isn't a valid argument in favour of automatic heap allocation, because you've just made a needlessly complicated example, skipping the obvious way to do it with the new closure proposal. Here's yet another alternative, which gets a little bit weird because AtomicBoolean has no getAndFlip method, but the fact that your alternator could get away with "return flag ^= foo;" was a lucky coincidence, and inline assignment is a style most java shops don't allow in the first place. Iterable everyOther(Iterable input) { AtomicBoolean flag = new AtomicBoolean(); return filter(input, #(T t) (return flag.getAndSet(!flag.get())); } or in a somewhat more verbose but easier to read fashion: Iterable everyOther(Iterable input) { AtomicBoolean flag = new AtomicBoolean(); return filter(input, #(T t) { boolean val = flag.get(); flag.set(!val); return val; }); } Alternatively, with a keyword to move allocation to the heap, it becomes much simpler still: Iterable everyOther(Iterable input) { nonfinal boolean flag = true; return filter(input, #(T t) (flag ^= true)); } All of which look concise, and easy to read to me. They are easy to write too, especially if the error message you get when you try it your way explicitly points out the nonfinal / AtomicBoolean solution, which it certainly can (and should!) do. As a final point, your code isn't even "correct", in the sense that no java shop in their right mind would let it pass. As you yourself said, it generates a warning. Code shouldn't generate any warnings. We already have a mechanism for turning warnings off, so we should use it here to be consistent. Thus, your example would become, rewritten to the status quo syntax: Iterable everyOther(Iterable input) { @SuppressWarnings("heapallocation") boolean flag = true; return filter(input, (T t) (flag ^= true)); } That's clearly inferior to the previous choice, and in my book still inferior to the AtomicBoolean solution. --Reinier Zwitserloot On Fri, Feb 26, 2010 at 9:44 AM, R?mi Forax wrote: > Le 26/02/2010 09:23, Lawrence Kesteloot a ?crit : > >> In BGGA, CfJ, and other languages with lambdas, one could write it > >> simply like this > >> > >> Iterable everyOther(Iterable input) { > >> boolean flag = true; > >> return filter(input, (T t)->(flag ^= true)); > >> } > >> > >> But project lambda forbids access to mutable local variables from the > >> enclosing scope. > >> > > Iterable everyOther(Iterable input) { > > final boolean[] flag = new boolean[] { true }; > > return filter(input, (T t)->(flag[0] ^= true)); > > } > > > > It's not beautiful, but not the end of the world either. And the > > performance is comparable, since in any case you have to create and > > dereference some heap object. > > > > +1 > > > Lawrence > > R?mi > > From fatih at coskuns-castle.de Fri Feb 26 05:56:47 2010 From: fatih at coskuns-castle.de (Fatih Coskun) Date: Fri, 26 Feb 2010 14:56:47 +0100 Subject: Accessing non-final local variables from a lambda expression In-Reply-To: References: Message-ID: <4B87D31F.7050306@coskuns-castle.de> > >> In BGGA, CfJ, and other languages with lambdas, one could write it >> simply like this >> >> Iterable everyOther(Iterable input) { >> ? ?boolean flag = true; >> ? ?return filter(input, (T t)->(flag ^= true)); >> } >> >> But project lambda forbids access to mutable local variables from the >> enclosing scope. >> > > Iterable everyOther(Iterable input) { > final boolean[] flag = new boolean[] { true }; > return filter(input, (T t)->(flag[0] ^= true)); > } > > It's not beautiful, but not the end of the world either. And the > performance is comparable, since in any case you have to create and > dereference some heap object. > > > How about adding at least some syntactic sugar for that "array-wrapper-idiom"? The code looks really ugly, syntactic sugar would make it look better. Fatih From forax at univ-mlv.fr Fri Feb 26 06:06:09 2010 From: forax at univ-mlv.fr (=?ISO-8859-1?Q?R=E9mi_Forax?=) Date: Fri, 26 Feb 2010 15:06:09 +0100 Subject: Accessing non-final local variables from a lambda expression In-Reply-To: <4B87D31F.7050306@coskuns-castle.de> References: <4B87D31F.7050306@coskuns-castle.de> Message-ID: <4B87D551.5050506@univ-mlv.fr> Le 26/02/2010 14:56, Fatih Coskun a ?crit : > >> >> >>> In BGGA, CfJ, and other languages with lambdas, one could write it >>> simply like this >>> >>> Iterable everyOther(Iterable input) { >>> ? ?boolean flag = true; >>> ? ?return filter(input, (T t)->(flag ^= true)); >>> } >>> >>> But project lambda forbids access to mutable local variables from the >>> enclosing scope. >>> >>> >> Iterable everyOther(Iterable input) { >> final boolean[] flag = new boolean[] { true }; >> return filter(input, (T t)->(flag[0] ^= true)); >> } >> >> It's not beautiful, but not the end of the world either. And the >> performance is comparable, since in any case you have to create and >> dereference some heap object. >> >> >> >> > > How about adding at least some syntactic sugar for that > "array-wrapper-idiom"? The code looks really ugly, syntactic sugar would > make it look better. > > Fatih You're right it's ugly but at least there is no black magic somewhere. The problem with a a syntactic sugar for shared state is that there is more than one way to 'unsugar' it and those ways define very different semantics. Should we let users use a gun that sometimes explode ? R?mi From jkuhnert at gmail.com Fri Feb 26 07:14:02 2010 From: jkuhnert at gmail.com (Jesse Kuhnert) Date: Fri, 26 Feb 2010 10:14:02 -0500 Subject: Accessing non-final local variables from a lambda expression In-Reply-To: <560fb5ed1002260526p7947bc95lcafce224f0e21463@mail.gmail.com> References: <15e8b9d21002252336o2f7280d2v157a9acd38ab29da@mail.gmail.com> <997cab101002260023t570987beob147c83288da4a23@mail.gmail.com> <4B8789D7.1000804@univ-mlv.fr> <560fb5ed1002260526p7947bc95lcafce224f0e21463@mail.gmail.com> Message-ID: <7926817e1002260714h51189a45qccf1e62c1f58f1b5@mail.gmail.com> Only if you're filtering concurrently. You sound very sure of your opinions given that the intention is to make it as "usable" and safe as possible for the general developer. It's a new feature! Why is it so hard to introduce it properly without crippling it from the beginning? On Fri, Feb 26, 2010 at 8:26 AM, Reinier Zwitserloot wrote: > > Iterable everyOther(Iterable input) { > ? @SuppressWarnings("heapallocation") > ? boolean flag = true; > ? return filter(input, (T t) (flag ^= true)); > } > > That's clearly inferior to the previous choice, and in my book still > inferior to the AtomicBoolean solution. > > --Reinier Zwitserloot > > > > On Fri, Feb 26, 2010 at 9:44 AM, R?mi Forax wrote: > >> Le 26/02/2010 09:23, Lawrence Kesteloot a ?crit : >> >> In BGGA, CfJ, and other languages with lambdas, one could write it >> >> simply like this >> >> >> >> ?Iterable ?everyOther(Iterable ?input) { >> >> ? ? boolean flag = true; >> >> ? ? return filter(input, (T t)->(flag ^= true)); >> >> } >> >> >> >> But project lambda forbids access to mutable local variables from the >> >> enclosing scope. >> >> >> > ?Iterable ?everyOther(Iterable ?input) { >> > ? ? final boolean[] flag = new boolean[] { true }; >> > ? ? return filter(input, (T t)->(flag[0] ^= true)); >> > } >> > >> > It's not beautiful, but not the end of the world either. And the >> > performance is comparable, since in any case you have to create and >> > dereference some heap object. >> > >> >> +1 >> >> > Lawrence >> >> R?mi >> >> > > From reinier at zwitserloot.com Fri Feb 26 07:59:51 2010 From: reinier at zwitserloot.com (Reinier Zwitserloot) Date: Fri, 26 Feb 2010 16:59:51 +0100 Subject: Accessing non-final local variables from a lambda expression In-Reply-To: <7926817e1002260714h51189a45qccf1e62c1f58f1b5@mail.gmail.com> References: <15e8b9d21002252336o2f7280d2v157a9acd38ab29da@mail.gmail.com> <997cab101002260023t570987beob147c83288da4a23@mail.gmail.com> <4B8789D7.1000804@univ-mlv.fr> <560fb5ed1002260526p7947bc95lcafce224f0e21463@mail.gmail.com> <7926817e1002260714h51189a45qccf1e62c1f58f1b5@mail.gmail.com> Message-ID: <560fb5ed1002260759j1f99d3cdka6dc4b91bb403688@mail.gmail.com> Nah; I avoid @SuppressWarnings as much as I can. I'd gladly rewrite to AtomicBoolean even if nothing concurrent is happening to avoid it. More to the point, I notice this behaviour in almost every other codebase I see, so it seems to be a bad idea to basically force folks to write copious amounts of @SuppressWarnings to (warning/error-free) use heap-hosted local variables. Pragmatically speaking, if this kind of construct is common enough, then a "nonfinal" or similar explicit "please allocate me on the heap" keyword would be good. If it's not very common, AtomicX will do. --Reinier Zwitserloot On Fri, Feb 26, 2010 at 4:14 PM, Jesse Kuhnert wrote: > Only if you're filtering concurrently. > > You sound very sure of your opinions given that the intention is to > make it as "usable" and safe as possible for the general developer. > It's a new feature! Why is it so hard to introduce it properly without > crippling it from the beginning? > > On Fri, Feb 26, 2010 at 8:26 AM, Reinier Zwitserloot > wrote: > > > > > Iterable everyOther(Iterable input) { > > @SuppressWarnings("heapallocation") > > boolean flag = true; > > return filter(input, (T t) (flag ^= true)); > > } > > > > That's clearly inferior to the previous choice, and in my book still > > inferior to the AtomicBoolean solution. > > > > --Reinier Zwitserloot > > > > > > > > On Fri, Feb 26, 2010 at 9:44 AM, R?mi Forax wrote: > > > >> Le 26/02/2010 09:23, Lawrence Kesteloot a ?crit : > >> >> In BGGA, CfJ, and other languages with lambdas, one could write it > >> >> simply like this > >> >> > >> >> Iterable everyOther(Iterable input) { > >> >> boolean flag = true; > >> >> return filter(input, (T t)->(flag ^= true)); > >> >> } > >> >> > >> >> But project lambda forbids access to mutable local variables from the > >> >> enclosing scope. > >> >> > >> > Iterable everyOther(Iterable input) { > >> > final boolean[] flag = new boolean[] { true }; > >> > return filter(input, (T t)->(flag[0] ^= true)); > >> > } > >> > > >> > It's not beautiful, but not the end of the world either. And the > >> > performance is comparable, since in any case you have to create and > >> > dereference some heap object. > >> > > >> > >> +1 > >> > >> > Lawrence > >> > >> R?mi > >> > >> > > > > > From peter.levart at marand.si Fri Feb 26 08:01:24 2010 From: peter.levart at marand.si (Peter Levart) Date: Fri, 26 Feb 2010 17:01:24 +0100 Subject: Accessing non-final local variables from a lambda expression In-Reply-To: <997cab101002260023t570987beob147c83288da4a23@mail.gmail.com> References: <15e8b9d21002252336o2f7280d2v157a9acd38ab29da@mail.gmail.com> <997cab101002260023t570987beob147c83288da4a23@mail.gmail.com> Message-ID: <201002261701.24155.peter.levart@marand.si> On Friday 26 February 2010 09:23:42 Lawrence Kesteloot wrote: > > In BGGA, CfJ, and other languages with lambdas, one could write it > > simply like this > > > > Iterable everyOther(Iterable input) { > > boolean flag = true; > > return filter(input, (T t)->(flag ^= true)); > > } > > > > But project lambda forbids access to mutable local variables from the > > enclosing scope. > > Iterable everyOther(Iterable input) { > final boolean[] flag = new boolean[] { true }; > return filter(input, (T t)->(flag[0] ^= true)); > } > > It's not beautiful, but not the end of the world either. And the > performance is comparable, since in any case you have to create and > dereference some heap object. > The problem with this approach (from the performance perspective) is when you capture several mutable local variables. Having a separate wrapper object for each of them is an overhead which can be avoided if the compiler constructs a single frame object for you. Regarding @Shared annotation on captured mutable local variables to prevent compiler warning, I think that it would be better to approach the problem a little differently. There are perfectly valid use-cases (like the one above) where accessing a mutable local variable is safe. For such cases it should not be necessary to use @Shared annotation on a variable to prevent a warning. The only way for a mutable local variable to escape the method is via capture from a closure whose reference escapes the method. So a better approach would be to annotate function type (and SAM type) variables (instance/static/local/parameters) that hold references to closures, not mutable local variables accessed from closures. A simple approach with a single annotation @Safe (meaning: multi thread safe) could work as follows: void test() { boolean flag = true; // OK ()->boolean flipFlop = ()->(flag ^= true); // warning: assignment of unsafe function to @Safe variable @Safe ()->boolean safeFlipFlop = ()->(flag ^= true); // OK: compiler determines lambda is safe (doesn't access mutable local vars) @Safe ()->void safeHello = ()->(System.out.println("Hello!"); void); flipFlop = safeFlipFlop; // OK safeFlipFlop = flipFlop; // warning: assignment of unsafe function to @Safe variable int count = 0; Object lock = new Object(); // OK: user suppresses the warning since she knows her code is safe @SuppressWarnings("unsafe") @Safe ()->int safeCounter = ()->(int c; synchronized(lock) { c = count++; } c); } All JDK methods/constructors accepting SAM types as parameters and invoking them from other threads (like new Thread, Executor.submit, ...) would have to add @Safe to their SAM parameters and any such 3rd party APIs would have to be retrofitted with annotations too. That's a drawback of this approach but It might still be better than @Shared on local vars. If users are forced to shut-up the compiler in all valid use-cases where it is perfectly safe to access mutable local vars, then they might develop a habit of putting @Shared annotation on every local variable accessed in lambdas without verifying that passing such lambdas to different methods (threads) is actually safe. On the other hand, when using @Safe on function/SAM variables, warnings would be less frequent, more accurate and would therefore have a better chance of being considered. The issue with this approach is whether to treat accesses to instance/static fields of outer classes and invocations of methods as safe or unsafe. In general it is unsafe, but it is allowed by default in inner classes so if we just want to issue warnings for the "new dangerous feature", then they should be treated as safe. The general approach to checking the multi-thread-safe code would of course extend to methods and be orthogonal to lambdas, but If we just want to play safe with new lambda features then the simple approach described above could be enough. Regards, Peter From neal at gafter.com Fri Feb 26 08:15:42 2010 From: neal at gafter.com (Neal Gafter) Date: Fri, 26 Feb 2010 08:15:42 -0800 Subject: Method References In-Reply-To: <4B87A632.4010100@univ-mlv.fr> References: <15e8b9d21002252252t45faaa9fq6859222b31a6052f@mail.gmail.com> <4B878DE6.4070100@oracle.com> <4B87A632.4010100@univ-mlv.fr> Message-ID: <15e8b9d21002260815y6c48f7a9n2a70ca5e0352274d@mail.gmail.com> On Fri, Feb 26, 2010 at 2:45 AM, R?mi Forax wrote: > there is a source compatibility issue if you allow to not specify the type. > > class A { > void m(String s) { ... } > } > .... > ref = A#m; > > Now suppose I want to add a new method m(Integer), > Adding a new method to a class is not a source-compatible change today. However, it is a binary-compatible change today and both with and without method references and with or without the proposed method reference argument inference. From jjb at google.com Fri Feb 26 08:40:47 2010 From: jjb at google.com (Joshua Bloch) Date: Fri, 26 Feb 2010 08:40:47 -0800 Subject: Method References In-Reply-To: <15e8b9d21002260815y6c48f7a9n2a70ca5e0352274d@mail.gmail.com> References: <15e8b9d21002252252t45faaa9fq6859222b31a6052f@mail.gmail.com> <4B878DE6.4070100@oracle.com> <4B87A632.4010100@univ-mlv.fr> <15e8b9d21002260815y6c48f7a9n2a70ca5e0352274d@mail.gmail.com> Message-ID: <17b2302a1002260840u3318d99ep9561e6fd1963fd7a@mail.gmail.com> I agree that method references would be a fine addition to the proposal, and I like the proposed syntax. I believe it's important that method references work for constructors and static methods as well as instance methods. I am very much in favor of allowing the user to omit the type information when it's unambiguous. In fact, is their any reason to use "this" (as proposed by Fredrick): cb = this#saveState; or could we allow: cb = #saveState; Not only is this more succinct, but it mirrors the rules for method invocation, allowing "reasoning by analogy" and reducing programmer astonishment. Josh On Fri, Feb 26, 2010 at 8:15 AM, Neal Gafter wrote: > On Fri, Feb 26, 2010 at 2:45 AM, R?mi Forax wrote: > > > there is a source compatibility issue if you allow to not specify the > type. > > > > class A { > > void m(String s) { ... } > > } > > .... > > ref = A#m; > > > > Now suppose I want to add a new method m(Integer), > > > > Adding a new method to a class is not a source-compatible change today. > However, it is a binary-compatible change today and both with and without > method references and with or without the proposed method reference > argument > inference. > > From neal at gafter.com Fri Feb 26 08:55:01 2010 From: neal at gafter.com (Neal Gafter) Date: Fri, 26 Feb 2010 08:55:01 -0800 Subject: Method References In-Reply-To: <17b2302a1002260840u3318d99ep9561e6fd1963fd7a@mail.gmail.com> References: <15e8b9d21002252252t45faaa9fq6859222b31a6052f@mail.gmail.com> <4B878DE6.4070100@oracle.com> <4B87A632.4010100@univ-mlv.fr> <15e8b9d21002260815y6c48f7a9n2a70ca5e0352274d@mail.gmail.com> <17b2302a1002260840u3318d99ep9561e6fd1963fd7a@mail.gmail.com> Message-ID: <15e8b9d21002260855s6e12aebdk684c74f86e070739@mail.gmail.com> On Fri, Feb 26, 2010 at 8:40 AM, Joshua Bloch wrote: > I agree that method references would be a fine addition to the proposal, > and > I like the proposed syntax. I believe it's important that method references > work for constructors and static methods as well as instance methods. > > I am very much in favor of allowing the user to omit the type information > when it's unambiguous. In fact, is their any reason to use "this" (as > proposed by Fredrick): > > cb = this#saveState; > > or could we allow: > > cb = #saveState; > > Not only is this more succinct, but it mirrors the rules for method > invocation, allowing "reasoning by analogy" and reducing programmer > astonishment. > I worry that it looks too much like a lambda or function type, but if we use my proposed syntax for lambda then there is no conflict. "Unambiguous" can't possibly refer to that term in the overload-resolution sense. From schulz at the-loom.de Fri Feb 26 09:18:34 2010 From: schulz at the-loom.de (Stefan Schulz) Date: Fri, 26 Feb 2010 18:18:34 +0100 Subject: Method References In-Reply-To: <17b2302a1002260840u3318d99ep9561e6fd1963fd7a@mail.gmail.com> References: <15e8b9d21002252252t45faaa9fq6859222b31a6052f@mail.gmail.com> <4B878DE6.4070100@oracle.com> <4B87A632.4010100@univ-mlv.fr> <15e8b9d21002260815y6c48f7a9n2a70ca5e0352274d@mail.gmail.com> <17b2302a1002260840u3318d99ep9561e6fd1963fd7a@mail.gmail.com> Message-ID: <4B88026A.3030606@the-loom.de> Am 26.02.2010 17:40, schrieb Joshua Bloch: > I agree that method references would be a fine addition to the proposal, and > I like the proposed syntax. I believe it's important that method references > work for constructors and static methods as well as instance methods. > > I am very much in favor of allowing the user to omit the type information > when it's unambiguous. In fact, is their any reason to use "this" (as > proposed by Fredrick): > > cb = this#saveState; > > or could we allow: > > cb = #saveState; > > Not only is this more succinct, but it mirrors the rules for method > invocation, allowing "reasoning by analogy" and reducing programmer > astonishment. We had this proposal in FCM [1] from early beginnings, so it may be no wonder to see me give a big +1 ;) As lambas do neither have a leading identifier nor would the match a type-only signature of a method reference, "this" would not be needed (at least for the compiler), but not forbidden either, as the leading primary might refer to another object or class. We had them combined with literals. Especially because of field literals, I am in no favor of the type-less variant, as these would bar adding field literals (or require some other awkward syntax), which are also beneficial (e.g. for Beans, replacing string-based bindings). Cheers, Stefan [1] http://docs.google.com/Doc?id=ddhp95vd_6hg3qhc From forax at univ-mlv.fr Fri Feb 26 10:00:55 2010 From: forax at univ-mlv.fr (=?ISO-8859-1?Q?R=E9mi_Forax?=) Date: Fri, 26 Feb 2010 19:00:55 +0100 Subject: Method References In-Reply-To: <15e8b9d21002260855s6e12aebdk684c74f86e070739@mail.gmail.com> References: <15e8b9d21002252252t45faaa9fq6859222b31a6052f@mail.gmail.com> <4B878DE6.4070100@oracle.com> <4B87A632.4010100@univ-mlv.fr> <15e8b9d21002260815y6c48f7a9n2a70ca5e0352274d@mail.gmail.com> <17b2302a1002260840u3318d99ep9561e6fd1963fd7a@mail.gmail.com> <15e8b9d21002260855s6e12aebdk684c74f86e070739@mail.gmail.com> Message-ID: <4B880C57.5070206@univ-mlv.fr> Le 26/02/2010 17:55, Neal Gafter a ?crit : > On Fri, Feb 26, 2010 at 8:40 AM, Joshua Bloch wrote: > > >> I agree that method references would be a fine addition to the proposal, >> and >> I like the proposed syntax. I believe it's important that method references >> work for constructors and static methods as well as instance methods. >> >> I am very much in favor of allowing the user to omit the type information >> when it's unambiguous. In fact, is their any reason to use "this" (as >> proposed by Fredrick): >> >> cb = this#saveState; >> >> or could we allow: >> >> cb = #saveState; >> >> Not only is this more succinct, but it mirrors the rules for method >> invocation, allowing "reasoning by analogy" and reducing programmer >> astonishment. >> >> > I worry that it looks too much like a lambda or function type, but if we use > my proposed syntax for lambda then there is no conflict. > > "Unambiguous" can't possibly refer to that term in the overload-resolution > sense. > Perl zealots will be jealous of the true power of Java ! class A { A[] A(A a) { return new A[]{a}; } A f() { return #A.((A#A(A).(A.A(#A()(null).())[0]))[0])[0]; } } R?mi From forax at univ-mlv.fr Fri Feb 26 10:11:13 2010 From: forax at univ-mlv.fr (=?ISO-8859-1?Q?R=E9mi_Forax?=) Date: Fri, 26 Feb 2010 19:11:13 +0100 Subject: Method References In-Reply-To: <15e8b9d21002260815y6c48f7a9n2a70ca5e0352274d@mail.gmail.com> References: <15e8b9d21002252252t45faaa9fq6859222b31a6052f@mail.gmail.com> <4B878DE6.4070100@oracle.com> <4B87A632.4010100@univ-mlv.fr> <15e8b9d21002260815y6c48f7a9n2a70ca5e0352274d@mail.gmail.com> Message-ID: <4B880EC1.2000603@univ-mlv.fr> Le 26/02/2010 17:15, Neal Gafter a ?crit : > On Fri, Feb 26, 2010 at 2:45 AM, R?mi Forax > wrote: > > there is a source compatibility issue if you allow to not specify > the type. > > class A { > void m(String s) { ... } > } > .... > ref = A#m; > > Now suppose I want to add a new method m(Integer), > > > Adding a new method to a class is not a source-compatible change today. Oups. > However, it is a binary-compatible change today and both with and > without method references and with or without the proposed method > reference argument inference. Yes, overloading is resolved at compile time. How do you expect to resolve generic method call ? class A { static T identity(T t) { ... } } A#identity ??? In that case, is A#identity.("foo") equivalent to A.identity("foo") ? R?mi From neal at gafter.com Fri Feb 26 10:10:02 2010 From: neal at gafter.com (Neal Gafter) Date: Fri, 26 Feb 2010 10:10:02 -0800 Subject: Method References In-Reply-To: <4B880EC1.2000603@univ-mlv.fr> References: <15e8b9d21002252252t45faaa9fq6859222b31a6052f@mail.gmail.com> <4B878DE6.4070100@oracle.com> <4B87A632.4010100@univ-mlv.fr> <15e8b9d21002260815y6c48f7a9n2a70ca5e0352274d@mail.gmail.com> <4B880EC1.2000603@univ-mlv.fr> Message-ID: <15e8b9d21002261010v1ea9e0e1s1ce084980852da24@mail.gmail.com> On Fri, Feb 26, 2010 at 10:11 AM, R?mi Forax wrote: > How do you expect to resolve generic method call ? > > class A { > ? static T identity(T t) { ... } > } > > A#identity?? ??? > > > In that case, is A#identity.("foo") equivalent to A.identity("foo") ? That is not a syntax I proposed to support. My specification is at . Cheers, Neal From fredrik.ohrstrom at oracle.com Fri Feb 26 11:26:42 2010 From: fredrik.ohrstrom at oracle.com (=?ISO-8859-1?Q?Fredrik_=D6hrstr=F6m?=) Date: Fri, 26 Feb 2010 20:26:42 +0100 Subject: Method References In-Reply-To: <17b2302a1002260840u3318d99ep9561e6fd1963fd7a@mail.gmail.com> References: <15e8b9d21002252252t45faaa9fq6859222b31a6052f@mail.gmail.com> <4B878DE6.4070100@oracle.com> <4B87A632.4010100@univ-mlv.fr> <15e8b9d21002260815y6c48f7a9n2a70ca5e0352274d@mail.gmail.com> <17b2302a1002260840u3318d99ep9561e6fd1963fd7a@mail.gmail.com> Message-ID: <4B882072.1040303@oracle.com> Joshua Bloch wrote: > or could we allow: > > cb = #saveState; > > When you bind a callback to your own code in the constructur, then you want to use my_button.onClick(this#saveState); When you bind a callback between two objects that you have created then you want to use: model.onChange(view#notify); Sometimes you are interested in acquiring a function that encapsulates the virtual dispatch of a particular message (ie method name). I call this a "stored message". This could be acquired using: MyApp#saveState and would be equivalent to the lamda: #(MyApp a) a.saveState() You might also want to refer to the static function: MyApp#staticSaveState So, should #saveState be: 1) this#saveState 2) MyApp#saveState 3) MyApp#saveState that is static 4) Not allowed at all. I would like the following three expressions to be identical (sed s/invoke//g if you like that better, I don't): reziver.message(); (#message).invoke(reziver); (reziver#message).invoke(); From this follows that #saveState will give you a stored message if saveState uniquely identifies a method in the enclosing class. If #saveState uniquely identifies a static function, then it will be that static function. //Fredrik > Not only is this more succinct, but it mirrors the rules for method > invocation, allowing "reasoning by analogy" and reducing programmer > astonishment. > > Josh > > On Fri, Feb 26, 2010 at 8:15 AM, Neal Gafter wrote: > > >> On Fri, Feb 26, 2010 at 2:45 AM, R?mi Forax wrote: >> >> >>> there is a source compatibility issue if you allow to not specify the >>> >> type. >> >>> class A { >>> void m(String s) { ... } >>> } >>> .... >>> ref = A#m; >>> >>> Now suppose I want to add a new method m(Integer), >>> >>> >> Adding a new method to a class is not a source-compatible change today. >> However, it is a binary-compatible change today and both with and without >> method references and with or without the proposed method reference >> argument >> inference. >> >> >> > > From neal at gafter.com Fri Feb 26 11:33:04 2010 From: neal at gafter.com (Neal Gafter) Date: Fri, 26 Feb 2010 11:33:04 -0800 Subject: Method References In-Reply-To: <4B882072.1040303@oracle.com> References: <15e8b9d21002252252t45faaa9fq6859222b31a6052f@mail.gmail.com> <4B878DE6.4070100@oracle.com> <4B87A632.4010100@univ-mlv.fr> <15e8b9d21002260815y6c48f7a9n2a70ca5e0352274d@mail.gmail.com> <17b2302a1002260840u3318d99ep9561e6fd1963fd7a@mail.gmail.com> <4B882072.1040303@oracle.com> Message-ID: <15e8b9d21002261133l1351afd2n7040f48f7c594f9@mail.gmail.com> Fredrik- I agree. The ambiguity between the receiver being bound and the receiver being unbound in a method reference is the reason we had made the receiver explicit. This issue doesn't arise with ordinary method invocations, because in that case it is impossible to have the receiver unbound. It isn't easy to make the shorthand "default" to one or another behavior either. In a static context only the unbound method reference makes sense, but in a nonstatic context both are possible. Cheers, Neal On Fri, Feb 26, 2010 at 11:26 AM, Fredrik ?hrstr?m wrote: > Joshua Bloch wrote: >> or could we allow: >> >> ? ? cb = #saveState; >> >> > When you bind a callback to your own code in the constructur, then you want > to use > ?my_button.onClick(this#saveState); > > When you bind a callback between two objects that you have created then you > want to use: > ?model.onChange(view#notify); > > Sometimes you are interested in acquiring a function that encapsulates > the virtual dispatch > of a particular message (ie method name). I call this a "stored > message". This could be > acquired using: > ?MyApp#saveState > and would be equivalent to the lamda: > ?#(MyApp a) a.saveState() > > You might also want to refer to the static function: > ?MyApp#staticSaveState > > So, should #saveState ?be: > ?1) this#saveState > ?2) MyApp#saveState > ?3) MyApp#saveState that is static > ?4) Not allowed at all. > > I would like the following three expressions to be identical (sed > s/invoke//g if you like that better, I don't): > > reziver.message(); > (#message).invoke(reziver); > (reziver#message).invoke(); > > ?From this follows that #saveState will give you a stored message if > saveState uniquely identifies a method in the enclosing class. If > #saveState uniquely identifies a static function, then it will be that > static function. > > //Fredrik > >> Not only is this more succinct, but it mirrors the rules for method >> invocation, allowing "reasoning by analogy" and reducing programmer >> astonishment. >> >> ? ? ? ? ? ? Josh >> >> On Fri, Feb 26, 2010 at 8:15 AM, Neal Gafter wrote: >> >> >>> On Fri, Feb 26, 2010 at 2:45 AM, R?mi Forax wrote: >>> >>> >>>> there is a source compatibility issue if you allow to not specify the >>>> >>> type. >>> >>>> class A { >>>> ? void m(String s) { ... } >>>> } >>>> .... >>>> ref = A#m; >>>> >>>> Now suppose I want to add a new method m(Integer), >>>> >>>> >>> Adding a new method to a class is not a source-compatible change today. >>> ?However, it is a binary-compatible change today and both with and without >>> method references and with or without the proposed method reference >>> argument >>> inference. >>> >>> >>> >> >> > > > From Alex.Buckley at Sun.COM Fri Feb 26 11:37:32 2010 From: Alex.Buckley at Sun.COM (Alex Buckley) Date: Fri, 26 Feb 2010 11:37:32 -0800 Subject: Accessing non-final local variables from a lambda expression In-Reply-To: <15e8b9d21002252336o2f7280d2v157a9acd38ab29da@mail.gmail.com> References: <15e8b9d21002252336o2f7280d2v157a9acd38ab29da@mail.gmail.com> Message-ID: <4B8822FC.7000802@sun.com> You appear to have missed shared variables in the project strawman. I explicitly note in the draft spec that it does not yet incorporate shared variables, so this discussion is moot. Alex Neal Gafter wrote: > I experimented a bit with a lambda-ized version of a modified Google > Collections. Specifically, I was playing with the method > > Iterable filter( > Iterable input, > Predicate predicate) > { ... } > > Using this method and an anonymous inner class, one can write a method that > returns every other value in its input sequence: > > // before > Iterable everyOther(Iterable input) { > return filter(input, new Predicate{ > boolean flag = true; > public boolean apply(T input) { > return (flag ^= true); > } > }); > } > > The equivalent function-ized version of the API would be (expressed in my > preferred syntax): > > Iterable filter( > Iterable input, > (T)->boolean predicate) > { ... } > > But now I have a problem implementing everyOther. The only way to create a > value of function type is to write a lambda expression, but where to put the > flag? In BGGA, CfJ, and other languages with lambdas, one could write it > simply like this > > Iterable everyOther(Iterable input) { > boolean flag = true; > return filter(input, (T t)->(flag ^= true)); > } > > But project lambda forbids access to mutable local variables from the > enclosing scope. I came up with the following workable approach > > // after > Iterable everyOther(Iterable input) { > class Alternator { > boolean flag = true; > boolean invoke() { > return (flag ^= true); > } > } > final Alternator alternate = new Alternator(); > return filter(input, #(T t)(alternate.invoke()); > } > > Comparing the before and the after, it appears that project lambda didn't do > us any favors here. The "after" code is far worse. In our attempt to make > the concurrent use cases "easier not to get wrong", we've placed a pile of > obstacles before every programmer who uses function types in sequential > code. > > It would be better to give a warning (not error) when a lambda accesses a > mutable local variable from an enclosing scope, and provide one or more easy > ways for programmers to suppress the warning. See CfJ < > http://www.javac.info/closures-v06a.html> for my preferred approach. > From alex.blewitt at gmail.com Fri Feb 26 11:46:23 2010 From: alex.blewitt at gmail.com (Alex Blewitt) Date: Fri, 26 Feb 2010 19:46:23 +0000 Subject: Effectively final effective? In-Reply-To: <560fb5ed1002260509l3f5e2d98td49a5d959c5e3df5@mail.gmail.com> References: <4B85B090.9090706@optrak.co.uk> <4B85B844.9050507@adres.pl> <560fb5ed1002250857i42a63aa2u4d3c1cebc74762b@mail.gmail.com> <560fb5ed1002251326q6d12c59wa3ca36f496a620c9@mail.gmail.com> <212322091002251846k397efc92rc10e1462175cd524@mail.gmail.com> <7926817e1002251925md1f3686k2742d81a75c60d5f@mail.gmail.com> <560fb5ed1002260509l3f5e2d98td49a5d959c5e3df5@mail.gmail.com> Message-ID: <636fd28e1002261146p36910745n4d424a3145535181@mail.gmail.com> On Fri, Feb 26, 2010 at 1:09 PM, Reinier Zwitserloot wrote: > NB: Jesse, regardless of the pros and cons of making a backwards > incompatible java 2.0 release, Java 7 most assuredly is not going to be such > a release. One problem with the change of semantics of 'effective final', if it applies to inner classes, is that it introduces the possibility of Java programs compiled with Java 1.7 to be incompatible with one compiled for 1.6. Clearly, one cannot use pre-1.7 features (closures, coin features) when compiling on a 1.6 compiler (or, using a -source 1.6 flag), but it will mean that developers will be able to write: public class Outer { public void method() { boolean nonFinal = true; new Inner { boolean foo; void other() { foo = nonFinal; } } } } On a 1.7 compiler, this will compile without error. On a 1.6 compiler, the lack of 'final' on the 'nonFinal' variable will cause a compile time error. So it's still possible to be backwardly incompatible, even if not using any of the 'new' features like lambdas, method references, coin syntax improvements etc. Alex From jjb at google.com Fri Feb 26 12:27:59 2010 From: jjb at google.com (Joshua Bloch) Date: Fri, 26 Feb 2010 12:27:59 -0800 Subject: Exception Transparency In-Reply-To: <212322091002252322u151a9635rcb4d03f52cc4061d@mail.gmail.com> References: <15e8b9d21002252311i3de2e97cs80b1495acfebeef6@mail.gmail.com> <212322091002252322u151a9635rcb4d03f52cc4061d@mail.gmail.com> Message-ID: <17b2302a1002261227r372e1e2n59aae849ea32e70c@mail.gmail.com> I too agree that exception transparency is a good thing -- "throws X" is an empty promise without it.I tried (unsuccessfully) to convince the generics EG to support this feature when generics were first added to the language:( Sadly, many APIs have already been written in a fashion that probably won't be able to take advantage of exception transparency if we add it to the language. Exhibit A is java.util.concurrent.Callable, which has pervasive effects on the Executor framework: public interface Callable { public abstract V call() throws Exception; } Might it be possible to add exception transparency in some fashion that allows us to retrofit Callable, or is that asking too much? Either way, I believe it's not too late to add this feature to the language and I hope that we can do it. Josh P.S. I'd be curious to know if the term exception transparency was used before '97. John Rose and I used the term when anonymous classes were added to Java. On Thu, Feb 25, 2010 at 11:22 PM, Paulo Levi wrote: > Agree completely. The syntax is a extension of the parametric type > inference > that already exists in methods (so people already use it), so this is just > the removal of a limitation -> exceptions can be parametric too. > > From john at milsson.nu Fri Feb 26 12:32:45 2010 From: john at milsson.nu (John Nilsson) Date: Fri, 26 Feb 2010 21:32:45 +0100 Subject: Effectively final effective? In-Reply-To: <560fb5ed1002260509l3f5e2d98td49a5d959c5e3df5@mail.gmail.com> References: <4B85B090.9090706@optrak.co.uk> <4B85B844.9050507@adres.pl> <560fb5ed1002250857i42a63aa2u4d3c1cebc74762b@mail.gmail.com> <560fb5ed1002251326q6d12c59wa3ca36f496a620c9@mail.gmail.com> <212322091002251846k397efc92rc10e1462175cd524@mail.gmail.com> <7926817e1002251925md1f3686k2742d81a75c60d5f@mail.gmail.com> <560fb5ed1002260509l3f5e2d98td49a5d959c5e3df5@mail.gmail.com> Message-ID: On Fri, Feb 26, 2010 at 2:09 PM, Reinier Zwitserloot < reinier at zwitserloot.com> wrote: > 3. Variables accessed in closures will be silently copied, with the copy > inside the closure being final, but the variable outside of it being > mutable > (but, as the closure gets a copy, any mutations don't show up in the > closure). I've filed this away as crackpot, where it'll hopefully remain, > as > this is going to be very surprising to a lot of programmers and the only > way > this surprise is going to become obvious is after many hours spent > debugging. > So before this is filed away, let me try again and explain why this is not surprising and in fact the obvious behaviour. The variable from the outside scope is "captured" at the time then the lambda is created. In the same manner as any variable is carried into another scope there and then. int a = 1; int b = 0; { b = a; } a = 2; //At this point b is expected to be 1, the value of a inside the block. It is not expected that b has somehow been equated with the a variable just because there is an equal sign implying that relation. In the same manner you expect for(Integer i : asList(1,2,3)) aList.add(i); to produce a list containing 1,2,3 and not a list containing 3 (the last value of i) So there is already a precedent for a semantic of capturing a value, not a variable. Now in ordinary blocks we are free to mutate variables from the outside scope inside the inner scope. However as these blocks are exectued in the same order as they are declared this is not a problem. With lambdas and inner classes the definition is decoupled from the execution, which means that the outer scope might not even exist at the time of execution. In previous versions of Java this was solved only allowing final variables inside inner classes. As they can't be mutated in either scope there is no mutation order to care about. So by continuing this line of reasoning, we simply continue to view them as immutable inside the inner class/lambda but relax the requirement that they are mutable in the defining scope. Thinking that it is the value that is captured, not the variable. BR, John From neal at gafter.com Fri Feb 26 12:47:29 2010 From: neal at gafter.com (Neal Gafter) Date: Fri, 26 Feb 2010 12:47:29 -0800 Subject: Accessing non-final local variables from a lambda expression In-Reply-To: <4B8822FC.7000802@sun.com> References: <15e8b9d21002252336o2f7280d2v157a9acd38ab29da@mail.gmail.com> <4B8822FC.7000802@sun.com> Message-ID: <15e8b9d21002261247q5c8baad9l478dc135dbdb6767@mail.gmail.com> On Fri, Feb 26, 2010 at 11:37 AM, Alex Buckley wrote: > You appear to have missed shared variables in the project strawman. I > explicitly note in the draft spec that it does not yet incorporate > shared variables, so this discussion is moot. I did not miss that note. The discussion is not moot precisely because shared variables have not yet been incorporated, and the note in the draft leaves the impression that no decision has yet been made on their status. From Alex.Buckley at Sun.COM Fri Feb 26 13:16:51 2010 From: Alex.Buckley at Sun.COM (Alex Buckley) Date: Fri, 26 Feb 2010 13:16:51 -0800 Subject: Accessing non-final local variables from a lambda expression In-Reply-To: <15e8b9d21002261247q5c8baad9l478dc135dbdb6767@mail.gmail.com> References: <15e8b9d21002252336o2f7280d2v157a9acd38ab29da@mail.gmail.com> <4B8822FC.7000802@sun.com> <15e8b9d21002261247q5c8baad9l478dc135dbdb6767@mail.gmail.com> Message-ID: <4B883A43.4020909@sun.com> Neal Gafter wrote: > I did not miss that note. The discussion is not moot precisely > because shared variables have not yet been incorporated, and the note > in the draft leaves the impression that no decision has yet been made > on their status. Shared variables are in the strawman proposal that guides this project. The draft spec explicitly identifies that they are not yet specified: "I am holding off shared variables for now". If you think that means they may never be included, despite being in the strawman, you could have asked before asserting that "project lambda didn't do us any favors here". Alex From john at milsson.nu Fri Feb 26 14:59:25 2010 From: john at milsson.nu (John Nilsson) Date: Fri, 26 Feb 2010 23:59:25 +0100 Subject: Accessing non-final local variables from a lambda expression In-Reply-To: <15e8b9d21002252336o2f7280d2v157a9acd38ab29da@mail.gmail.com> References: <15e8b9d21002252336o2f7280d2v157a9acd38ab29da@mail.gmail.com> Message-ID: On Fri, Feb 26, 2010 at 8:36 AM, Neal Gafter wrote: > Iterable everyOther(Iterable input) { > class Alternator { > boolean flag = true; > boolean invoke() { > return (flag ^= true); > } > } > final Alternator alternate = new Alternator(); > return filter(input, #(T t)(alternate.invoke()); > } > Why not something like this instead? Iterable everyOther(Iterable input) { return zipWithIndex(input).filter(#(long i, T t)(i%2==0)).map(#(long i, T t)(t)); } Point being, wouldn't it be better, from a concurrency point of view, to encourage a functional style of programming instead of focusing on supporting shared mutable state? BR, John From forax at univ-mlv.fr Fri Feb 26 15:30:44 2010 From: forax at univ-mlv.fr (=?ISO-8859-1?Q?R=E9mi_Forax?=) Date: Sat, 27 Feb 2010 00:30:44 +0100 Subject: Method References In-Reply-To: <15e8b9d21002261010v1ea9e0e1s1ce084980852da24@mail.gmail.com> References: <15e8b9d21002252252t45faaa9fq6859222b31a6052f@mail.gmail.com> <4B878DE6.4070100@oracle.com> <4B87A632.4010100@univ-mlv.fr> <15e8b9d21002260815y6c48f7a9n2a70ca5e0352274d@mail.gmail.com> <4B880EC1.2000603@univ-mlv.fr> <15e8b9d21002261010v1ea9e0e1s1ce084980852da24@mail.gmail.com> Message-ID: <4B8859A4.9060402@univ-mlv.fr> Le 26/02/2010 19:10, Neal Gafter a ?crit : > On Fri, Feb 26, 2010 at 10:11 AM, R?mi Forax wrote: > >> How do you expect to resolve generic method call ? >> >> class A { >> static T identity(T t) { ... } >> } >> >> A#identity ??? >> >> >> In that case, is A#identity.("foo") equivalent to A.identity("foo") ? >> > That is not a syntax I proposed to support. My specification is at > . > > Cheers, > Neal > Sorry Neal, My mail was not clear, I've no problem with your specification. the question was for Fredrik (the receiver of my first mail). R?mi From neal at gafter.com Fri Feb 26 15:41:20 2010 From: neal at gafter.com (Neal Gafter) Date: Fri, 26 Feb 2010 15:41:20 -0800 Subject: Accessing non-final local variables from a lambda expression In-Reply-To: References: <15e8b9d21002252336o2f7280d2v157a9acd38ab29da@mail.gmail.com> Message-ID: <15e8b9d21002261541v575de9ebqb6e5cf35bb7a456c@mail.gmail.com> On Fri, Feb 26, 2010 at 2:59 PM, John Nilsson wrote: > Why not something like this instead? > Iterable everyOther(Iterable input) { > ?? ?return zipWithIndex(input).filter(#(long i, T t)(i%2==0)).map(#(long i, > T t)(t)); > } Very clever. > Point being, wouldn't it be better, from a concurrency point of view, to > encourage a functional style of programming instead of focusing on > supporting shared mutable state? Where it's practical, yes. But there is a difference between encouraging a functional style and discouraging an imperative style. I admire the solution, but it isn't realistic to expect people to come up with it without significantly more time invested than is required for the simple solution. Most programmers have a job to do and a life outside work. From lk at teamten.com Fri Feb 26 15:53:08 2010 From: lk at teamten.com (Lawrence Kesteloot) Date: Fri, 26 Feb 2010 15:53:08 -0800 Subject: Accessing non-final local variables from a lambda expression In-Reply-To: <15e8b9d21002261541v575de9ebqb6e5cf35bb7a456c@mail.gmail.com> References: <15e8b9d21002252336o2f7280d2v157a9acd38ab29da@mail.gmail.com> <15e8b9d21002261541v575de9ebqb6e5cf35bb7a456c@mail.gmail.com> Message-ID: <997cab101002261553o4082b9a3i1c2426fb1c52ae05@mail.gmail.com> Neal, > Where it's practical, yes. ?But there is a difference between > encouraging a functional style and discouraging an imperative style. > I admire the solution, but it isn't realistic to expect people to come > up with it without significantly more time invested than is required > for the simple solution. ?Most programmers have a job to do and a life > outside work. John's solution is pretty standard in Python, where you'd write: >>> letters = ['a', 'b', 'c', 'd', 'e'] >>> [letter for index, letter in enumerate(letters) if index % 2 == 0] ['a', 'c', 'e'] This solution creates an extra object (the tuple) per iteration. John's solution requires a parallel API (one that deals with two-parameter predicates). The performance of the former would be fine for most people and may become mainstream enough that people wouldn't have to "come up with it", it'd be an idiom, as it is in Python. The Java equivalent would be something like: Iterable everyOther(Iterable input) { return makeEnumerated(input) .filter(#(Enumerated it)(it.getIndex()%2==0)) .map(#(Enumerated it)(it.getValue())); } Lawrence From Alex.Buckley at Sun.COM Fri Feb 26 15:59:14 2010 From: Alex.Buckley at Sun.COM (Alex Buckley) Date: Fri, 26 Feb 2010 15:59:14 -0800 Subject: A reminder of Project Lambda's scope Message-ID: <4B886052.5070305@sun.com> http://cr.openjdk.java.net/~mr/lambda/straw-man/ Where features in the strawman have made it into the draft spec so far, such as function types and lambda conversion, they are fair game for discussion. There is no need to lobby for mutable up-level variables, method references, exception transparency, extension methods, etc, since they are all mentioned in the strawman. The draft spec will get round to them in good time. Before anyone asks for a schedule, I don't have one. Alex From john at milsson.nu Fri Feb 26 16:07:20 2010 From: john at milsson.nu (John Nilsson) Date: Sat, 27 Feb 2010 01:07:20 +0100 Subject: Accessing non-final local variables from a lambda expression In-Reply-To: <15e8b9d21002261541v575de9ebqb6e5cf35bb7a456c@mail.gmail.com> References: <15e8b9d21002252336o2f7280d2v157a9acd38ab29da@mail.gmail.com> <15e8b9d21002261541v575de9ebqb6e5cf35bb7a456c@mail.gmail.com> Message-ID: On Sat, Feb 27, 2010 at 12:41 AM, Neal Gafter wrote: > Where it's practical, yes. But there is a difference between > encouraging a functional style and discouraging an imperative style. > I admire the solution, but it isn't realistic to expect people to come > up with it without significantly more time invested than is required > for the simple solution. Most programmers have a job to do and a life > outside work. For sure. But this is the same people we expect to learn to use lambdas. Which will be a struggle either way. As it will be possible to use one element arrays, or AtomicInteger or other variants to get access to shared mutable state this isn't really a problem. And maybe it's better if it looks somewhat cludgy. While the version I proposed still looks like it has some syntactic overhead, maybe this could improved with the proper attention? BR, John From neal at gafter.com Fri Feb 26 17:48:42 2010 From: neal at gafter.com (Neal Gafter) Date: Fri, 26 Feb 2010 17:48:42 -0800 Subject: A syntax option (function types versus arrays) Message-ID: <15e8b9d21002261748u77d29847ob56e4544263b5d1c@mail.gmail.com> The enclosed proposal for an alternative syntax, previously published on my blog, is included here by request so that it may formally be considered a contribution to project lambda.? This is the option that we previously discussed, which would look something like this: (int,int)->int add = (int x, int y)->x+y; Cheers, Neal From markmahieu at googlemail.com Fri Feb 26 18:13:24 2010 From: markmahieu at googlemail.com (Mark Mahieu) Date: Sat, 27 Feb 2010 02:13:24 +0000 Subject: A syntax option (function types versus arrays) In-Reply-To: <15e8b9d21002261748u77d29847ob56e4544263b5d1c@mail.gmail.com> References: <15e8b9d21002261748u77d29847ob56e4544263b5d1c@mail.gmail.com> Message-ID: <21208A19-213F-47CC-A062-4CF038651FA4@googlemail.com> On 27 Feb 2010, at 01:48, Neal Gafter wrote: > The enclosed proposal for an alternative syntax, previously published > on my blog, is included here by request so that it may formally be > considered a contribution to project lambda. This is the option that > we previously discussed, which would look something like this: > > (int,int)->int add = (int x, int y)->x+y; > > Cheers, > Neal > Was the enclosed proposal meant to be an attachment? If so, it didn't make it. Mark From i30817 at gmail.com Fri Feb 26 18:30:31 2010 From: i30817 at gmail.com (Paulo Levi) Date: Sat, 27 Feb 2010 02:30:31 +0000 Subject: Accessing non-final local variables from a lambda expression In-Reply-To: References: <15e8b9d21002252336o2f7280d2v157a9acd38ab29da@mail.gmail.com> <15e8b9d21002261541v575de9ebqb6e5cf35bb7a456c@mail.gmail.com> Message-ID: <212322091002261830l2f0c754bi443425758fb4c9de@mail.gmail.com> I ask again, why not move the , into the method that uses the lambda: Iterable filter( Iterable input, Predicate predicate) { ... } ok: int a = 0; filter(list, (int i){ a+= i; return true; }); Iterable parallelFilter( Iterable input, lambdaFinal Predicate predicate) { ... } syntax error: int a = 0; filter(list, (int i){ a+= i; return true; }); If the trype is in the variable being used that is the decision of the user, not of the library designer. The user must know (remember) that the function he calls is parallel and that shared state is non-deterministic. From neal at gafter.com Fri Feb 26 20:08:02 2010 From: neal at gafter.com (Neal Gafter) Date: Fri, 26 Feb 2010 20:08:02 -0800 Subject: A syntax option (function types versus arrays) In-Reply-To: <21208A19-213F-47CC-A062-4CF038651FA4@googlemail.com> References: <15e8b9d21002261748u77d29847ob56e4544263b5d1c@mail.gmail.com> <21208A19-213F-47CC-A062-4CF038651FA4@googlemail.com> Message-ID: <15e8b9d21002262008u6f947137v57069bd875c57242@mail.gmail.com> *A Syntax Option for Project Lambda (Closures for Java) * It was noted recently on the Project Lambda mailing listthat allowing arrays of function type would undermine the type system. The reason for this is a combination of Java's covariant arrays, the natural subtypes among function types (they are covariant on return type and contravariant on argument types), exception checking, and the erasure implementation of generics. We certainly can't remove covariant arrays or checked exceptions, and removing the subtype relationship among function types really reduces their utility. Unfortunately, it is almost certainly too late to reify generics too. Although you can't have an array of function type, there is no problem having, for example, an *ArrayList* of a function type. So while we might prefer not to have this restriction, it won't be much of a problem in practice. There are many ways in which arrays of function type would undermine the type system, but to give you some flavor of the problem, the following is a method written assuming that arrays of function type are allowed. This method must fail in some way - either fail to compile, or throw a * ClassCastException*, or an *ArrayStoreException*, or something. In all of the implementations being considered, this method would throw *IOException*, even though that isn't declared in the throws clause of the method. We have undermined Java's exception checking without even a cast! *public** void main(String[] args) {* * #void()[] arrayOfFunction = new #void()[1];* * Object[] objectArray = arrayOfFunction;* * objectArray[0] = #(){ throw new IOException(); };* * arrayOfFunction[0].(); // invoke the function out of the array* *}* The realization that supporting arrays of function type would introduce a loophole in the type system makes it possible to consider some very nice syntax options that were previously eliminated because there was no syntactic way to write an array of function type. But if we disallow arrays of function type, those options can be considered. My favorite of the alternatives is based on this grammar *FunctionType*: *(* *TypeList*opt *Throws*opt *)* *->* *ResultType* *Expression*: *LambdaExpression* *LambdaExpression*: *(* *FormalParameterList*opt *)* *->* *Expression* A function type is written with the argument types between parentheses, then a right arrow (dash greater-than), and then the result type. The most important distinction between this proposal and the existing draft specification is that this proposal places the result type after the argument types, instead of before. The benefits of its simplicity compared to the current specification are most obvious when you combine function types and lambdas with other features. Here is a simple example to show how the two proposals look. The example is currying . The following method takes as a parameter a function of two arguments, and returns a function of one argument that returns a function of one argument. Applying the resulting function to one value, and then the result of that to another value, should have the same effect as applying the original function to both values. To make it interesting, we make the argument and result types generic, and we allow the function to throw an exception. Using the currently proposed syntax for Project Lambda, the code would look something like this: *static** * *##V(U)(throws X)(T) curry(#V(T,U)(throws X) function) {* * return #(T t)(#(U u)(function.(t,u)));* *}* On the other hand, with the proposal described above it looks something like this: *static** * *(T)->(U throws X)->V curry((T,U throws X)->V function) {* * return (T t)->(U u)->function.(t,u);* *}* I've intentionally selected an example that uses the new features heavily, so either may take a few moments of studying to understand. But I claim the latter is much easier to follow than the former, even once you've become familiar with the syntax. You can easily write a function that yields an array as its result *()-**>int[] arrayLambda = ()->new int[]{1, 2, 3};* With this proposed syntax there is no way to write an array of functions. But that is exactly what we concluded could not be made to work within the type system anyway. From neal at gafter.com Fri Feb 26 21:23:37 2010 From: neal at gafter.com (Neal Gafter) Date: Fri, 26 Feb 2010 21:23:37 -0800 Subject: Extermal references: Closures for the Java Programming Language (v0.5) a.k.a BGGA Message-ID: <15e8b9d21002262123y2a926047ob596f8fada9d4c4d@mail.gmail.com> Alex asked me to include BGGA and other documents in messages on this list, so that they may formally be considered contributions to project lambda. Enclosed please find: Closures for the Java Programming Language (v0.5) also known as BGGA It may also be found at *Changes in this revision: * - Changed the terminology slightly to emphasize the distinction between a closure literal and the result of evaluating it, which is a closure instance. - Corrected the with example. - Specified the class-file format for communicating throws type parameters. - Added grammar rules for throws type parameters and type arguments. - The *Future Directions* section has been moved to a separate *Open Issues * document. - This specification now describes only the *functional* version of the proposed language feature. - Added support for control abstractions that act as loopsby binding the meaning of break and continue. - The package for system-generated function type interfaces is now java.lang.function. Closures for the Java Programming Language (v0.5) *Gilad Bracha, Neal Gafter, James Gosling, Peter von der Ah?* Modern programming languages provide a mixture of primitives for composing programs. Most notably Scheme, Smaltalk, Ruby, and Scala have direct language support for parameterized delayed-execution blocks of code, variously called *lambda*, *anonymous functions*, or *closures*. These provide a natural way to express some kinds of abstractions that are currently quite awkward to express in Java. For programming in the small, anonymous functions allow one to abstract an algorithm over a piece of code; that is, they allow one to more easily extract the common parts of two almost-identical pieces of code. For programming in the large, anonymous functions support APIs that express an algorithm abstracted over some computational aspect of the algorithm. For example, they enable writing methods that act as library-defined control constructs. Closure Literals We introduce a syntactic form for constructing an anonymus function value: *Primary:**ClosureLiteral**ClosureLiteral*:*{* *FormalParametersopt* *=>* * BlockStatementsopt* *Expressionopt* *}* Evaluating the closure literal expression results in a *closure instance *. A closure instance is converted to some object type by a *closure conversion *. In the nominal version of the specification, it is a compile-time error if a closure literal appears in a context where it is not subject to a closure conversion. In the functional version of the specification, if a closure literal is not subject to a closure conversion it is converted to the *corresponding function type* of the closure literal, which is the function type with: identical argument types; a return type that is the type of the final expression, if one exists, or java.lang.Unreachable if the closure literal's body cannot complete normally, or void otherwise; and a throws type list corresponding to the checked exception types that can be thrown from the body of the closure literal. The conversion, in either case,occurs entirely at compile-time. *Example*: the following closure literal takes two integers and yields their sum: {int x, int y => x+y} A closure literal captures a block of code - the block statements and the expression - parameterized by the closure literal's formal parameters. All free lexical bindings - that is, lexical bindings not defined within the closure literal - are bound at the time of evaluation of the closure literal to their meaning in the lexical context in which the closure literal appears. Free lexical bindings include references to variables from enclosing scopes, and the meaning of this, break, continue, and return. Evaluating the closure literal does not cause the statements or expression to be evaluated, but packages them up at runtime with a representation of the lexical context to be invoked later. *Rationale*: One purpose for closure literals is to allow a programmer to refactor common code into a shared utility, with the difference between the use sites being abstracted into a closure literal by the client. The code to be abstracted sometimes contains a break, continue, or return statement. This need not be an obstacle to the transformation. One implication of the specification is that a break or continue statement appearing within a closure literal's body may transfer to a matching enclosing statement. A return statement always returns from the nearest enclosing method or constructor. A function may outlive the target of control transfers appearing within it. At runtime, if a break statement is executed that would transfer control out of a statement that is no longer executing, or is executing in another thread, the VM throws a new unchecked exception, UnmatchedNonlocalTransfer. Similarly, an UnmatchedNonlocalTransfer is thrown when a continue statement attempts to complete a loop iteration that is not executing in the current thread. Finally, an UnmatchedNonlocalTransfer is thrown when a returnstatement is executed if the method invocation to which the return statement would transfer control is not on the stack of the current thread. Function Types We introduce a syntactic form for a *function type*: *Type:**FunctionType**FunctionType*:*{* *Typesopt* *=>* *Type* *Throwsopt* * }* *Types:**Type Type **,* *Types* Informally, a function type describes the set of functions that accept a given list of argument types, result in a value of the given type, and may throw the indicated checked exception types. *Example*: the following assigns to the local variable plus a function that computes the sum of its two int arguments: {int,int=>int} plus = {int x, int y => x+y}; A function type is translated into an instantiation of a generic system-generated interface type with a single method. The interface has a type parameter for each argument type and return type that is a reference type, and one additional type parameter for the exception signature. The generic instance of the interface used to represent a function type has covariant wildcards for the type arguments representing the return type and exception type, and contravariant wildcards for function argument types, except when a function type is used in the context of a declared supertype, in which case the type arguments appear without wildcards. *[This paragraph is a placeholder for a more precise description, including the name of the interface, required for interoperability among compilers. We expect the generated interfaces to be in the package java.lang.function, which will be closed to user-defined code, and the naming scheme to be an extension of the JVM method signature scheme.] * *Example*. The variable declaration {int,String=>Number throws IOException} xyzzy; is translated into interface Function1 { // system-generated R invoke(int x1, A2 x2) throws E; } Function1 xyzzy; With the exception that the generated interface name and parameter names would be synthetic, and the compiler, runtime system, and debuggers should display these types using the function type syntax rather than in terms of the translation involving generics and wildcards. Because a function type *is* an interface type, it can be extended by a user-defined interface and it can be implemented by a user-defined class. This translation scheme causes a function type { *T0* ... *Tn* => *Tr* throws *E0*, ... *Em* } to be defined as a system-generated interface type with a single abstract method *Tr* invoke(*T0* x0, ... *Tn* xn) throws *E0*, ... *Em*; that obeys the following subtyping relationship: A function type { *T0* ... *Tn* => *Tr* throws *E0*, ... *Em* } is a subtype of function type { *U0* ... *Ul* => *Ur* throws *X0*, ... *Xk* } iff all of the following hold: - Either - *Tr* is the same as *Ur* or - Both are reference types and *Tr* is a subtype of *Ur*; - *n *is equal to *l*; - for every* i* in *0..n*, either *Ti* and *Ui* are the same types or both are reference types and* Ui* is a subtype of*Ti*; and - for every *j* in *0..m*, there exists an *h* in *0..k* such that *Ej*is a subtype of *Xh*. In English, these rules are - Function types may have covariant returns; - A subtype function must accept the same number of arguments as its supertype; - Function types may have contravariant argument types; and - The subtype function may not throw any checked exception type not declared in the supertype. While the subtype rules for function types may at first glance appear arcane, they are defined this way for very good reason: this is the classical *arrow rule*, which has proven itself indispensible in languages that offer support for higher-order programming. And while the rules seem complex, function types do not add complexity to Java's type system because function types and their subtype relations can be understood as a straightforward application of generics and wildcards (existing constructs). >From the programmer's perspective, function types just "do the right thing." Closure Conversion A closure literal may be assigned to a variable or parameter of any compatible interface type by the *closure conversion*: There is a *closure conversion* from a closure literal to every interface type that has a single method *m* such that the closure literal is compatible with* m*. A closure literal is *compatible with* a method *m* iff all of the following hold: - Either - The closure literal has no final expression and the method *m* has return type void or java.lang.Void; or - The closure literal has a final expression and there is an assignment conversion from its type to the return type of *m*; or - The body of the closure literal cannot complete normally. - *m* has the same number of arguments as the closure literal. - For each corresponding argument position in the argument lists**, both argument types are the same. - Every exception type that can be thrown by the body of the closure literal is a subtype of some exception type in the throws clause of *m*. The closure conversion supports converting a function that yields no result to an interface whose method returns java.lang.Void. In this case a nullvalue is returned from the function. This is necessary to write APIs that support *completion transparency *: that is, in which an invocation of the API can return normally if and only if the function that is passed to it can complete normally. If the target of the closure conversion extends the marker interface java.lang.RestrictedFunction then - it is a compile-time error if the function being converted contains a break, continue or return statement whose target is outside the closure literal; and - it is a compile-time error if the function being converted refers to a non-final local variable declared in an enclosing scope. The closure conversion to a "restricted" interface only allows functions that obey the current restrictions on anonymous instances. The motivation for this is to help catch inadvertent use of non-local control flow in situations where it would be inappropriate. Examples would be when the function is passed to another thread to run asynchronously, or stored in a data structure to be invoked later. *Example*: We can create a closure instance that adds two to its argument like this: interface IntFunction { int invoke(int i); } IntFunction plus2 = {int x => x+2}; Alternatively, using function types: {int=>int} plus2 = {int x => x+2}; We can use the existing Executor framework to run a closure instance in the background: void sayHello(java.util.concurrent.Executor ex) { ex.execute({=> System.out.println("hello"); }); } Exception type parameters To support exception transparency (that is, the design of control-invocation APIs in which the compiler can infer that exceptions thrown by the closure are thrown by the API), we add a new type of generic formal type argument to receive a set of thrown types. *[This deserves a more formal treatment]* *TypeParameter:*throws* TypeVariable* *TypeBoundopt**ActualTypeArgument*:* throws* *ExceptionList* *ExceptionList*:*ReferenceType* *ReferenceType** **|* *ExceptionList* What follows is an example of how the proposal would be used to write an exception-transparent version of a method that locks and then unlocks a java.util.concurrent.Lock, before and after a user-provided block of code. In the nominal version of the proposal: interface Block { T invoke() throws E; } public static T withLock(Lock lock, Block block) throws E { lock.lock(); try { return block.invoke(); } finally { lock.unlock(); } } Or using the functional version of the proposal: public static T withLock(Lock lock, {=>T throws E} block) throws E { lock.lock(); try { return block.invoke(); } finally { lock.unlock(); } } This can be used as follows: withLock(lock, {=> System.out.println("hello"); }); This uses a newly introduced form of generic type parameter. The type parameter E is declared to be used in throws clauses. If the extends clause is omitted, a type parameter declared with the throws keyword is considered to extend Exception (instead of Object, which is the default for other type parameters). This type parameter accepts multiple exception types. For example, if you invoke it with a block that can throw IOException or NumberFormatException the type parameter E would be given as IOException|NumberFormatException. In those rare cases that you want to use explicit type parameters with multiple thrown types, the keyword throws is required in the invocation, like this: Locks.withLock(lock, {=> System.out.println("hello"); }); You can think of this kind of type parameter accepting disjunction, "or" types. That is to allow it to match the exception signature of a function that throws any number of different checked exceptions. If the block throws no exceptions, the type argument would be the type null (see below). Type parameters declared this way can be used only in a throws clause or as a type argument to a generic method, interface, or class whose type argument was declared this way too. Throws type parameters are indicated in class files by the presence of the character '^' preceding the type parameter. We introduce a meaning for the keyword null as a type name; it designates the type of the expression null. A class literal for the type of null is null.class. These are necessary to allow reflection, type inference, and closure literals to work for functions that result in the value null or that throw no checked exceptions. We also add the non-instantiable class java.lang.Null as a placeholder, and its static member field TYPE as a synonym for null.class. Definite assignment The body of a closure literal may not assign to a final variable declared outside the closure literal. A closure literal does not affect the DA/DU status of any free variables it names. A f ree variable referenced inside a closure literal receives its initial DA state from the DA state of the variable at the point where the closure literal appears. The type Unreachable We add the non-instantiable type java.lang.Unreachable. Values of this type appear where statements or expressions cannot complete normally. This is necessary to enable writing APIs that provide transparencyfor functions that do not complete normally. Unreachable is a subtype of every obect type. The null type is a subtype of Unreachable. At runtime, a NullPointerException is thrown when a value of type Unreachable would appear. This occurs when a programmer converts nullto Unreachable. *Example*: The following illustrates a function being assigned to a variable of the correct type. interface NullaryFunction { T invoke() throws E; } NullaryFunction thrower = {=> throw new AssertionError(); }; Reachable statements The initial statement of a closure literal is reachable. An expression statement in which the expression is of type Unreachablecannot complete normally. Control invocation syntax A new invocation statement syntax is added to make closures convenient for control abstraction: *ControlInvocationStatement*:for*opt Primary* (* FormalParameters *: * ExpressionListopt* ) *Statement* for*opt Primary* (* ExpressionListopt* ) *Statement* This syntax is a shorthand for the following statement: *Primary* ( *ExpressionList*, {* FormalParametersopt** *=> *Statement* } ); (See the later section *loop abstractions* for the meaning of the optional for keyword) This syntax makes some kinds of function-receiving APIs more convenient to use to compose statements. Note: There is some question of the correct order in the rewriting. On the one hand, the function seems most natural in the last position when not using the abbreviated syntax. On the other hand, that doesn't work well with varargs methods. Which is best remains an open issue. We could use the shorthand to write our previous example this way withLock(lock) { System.out.println("hello"); } Rationale: This is not an expression form for a very good reason: it looks like a statement, and we expect it to be used most commonly as a statement for the purpose of writing APIs that abstract patterns of control. If it were an expression form, an invocation like this would require a trailing semicolon after the close curly brace of a controlled block. Forgetting the semicolon would probably be a common source of error. The syntax is convenient for both synchronous (e.g. see withLock) and asynchronous (e.g. see Executor.execute) use cases. Another example of its use would be a an API that closes a java.io.Closeableafter a user-supplied block of code, discarding any exception from Closeable.close: class OneArgBlock { R invoke(T t) throws E; } R with(T t, OneArgBlock block) throws E { try { return block.invoke(t); } finally { try { t.close(); } catch (IOException ex) {} } } Or using the functional version: R with(T t, {T=>R throws E} block) throws E { try { return block.invoke(t); } finally { try { t.close(); } catch (IOException ex) {} } } We could use the shorthand with this API to close a number of streams at the end of a block of code: with(FileReader in : makeReader()) with(FileWriter out : makeWriter()) { // code using in and out } Loop Abstractions The specification supports writing methods that act as control statements, but when used to support loops the programmer should be able to specify that the statement should capture the meaning of break and continue in a way analogous to the built-in loop statements. For this purpose we allow the forkeyword at the beginning of a control invocation statement (see above), and introduce the use of the keyword for as a modifier on method declarations: *MethodDeclarator*:for*opt* *Identifier* ( *FormalParamterListopt* ) An overriding or implementing method must be declared with the keyword forif and only if the method being overridden or implemented was declared with the keyword for. A control invocation statement must use the for keyword if and only if the method being invoked was declared with the keyword for. Within the controlled statement of a control invocation statement using the keyword for: - An unlabelled break statement breaks from the control invocation statement. - An unlabelled continue statement breaks from the controlled statement. *Example*: The following illustrates a method used to loop through the key-value pairs in a map. The method void for eachEntry(Map map, {K,V=>void throws X} block) throws X { for (Map.Entry entry : map.entrySet()) { block.invoke(entry.getKey(), entry.getValue()); } } allows us to rewrite this void test(Map map) { for (Map.Entry entry : map.entrySet()) { String name = entry.getKey(); Integer value = entry.getValue(); if ("end".equals(name)) break; if (name.startsWith("com.sun.")) continue; System.out.println(name + ":" + value); } } like this void test(Map map) { for eachEntry(String name, Integer value : map) { if ("end".equals(name)) break; if (name.startsWith("com.sun.")) continue; System.out.println(name + ":" + value); } } Acknowledgments The authors would like to thank the following people whose discussions and insight have helped us craft, refine, and improve this proposal: * C. Scott Ananian, Lars Bak, Cedric Beust, Joshua Bloch, Martin Buchholz, Danny Coward, Erik Ernst, R?mi Forax, Christian Plesner Hansen, Kohsuke Kawaguchi, Danny Lagrouw, Doug Lea,"crazy" Bob Lee, Martin Odersky, Tim Peierls, Cris Perdue, John Rose, Ken Russell, Guy Steele, Mads Torgersen, Jan Vitek, *and* Dave Yost.* ------------------------------ [image: Creative Commons License] Closures for Java by Gilad Bracha, Neal Gafter, James Gosling, Peter von der Ah? is licensed under a Creative Commons Attribution-Share Alike 3.0 United States License. From neal at gafter.com Fri Feb 26 21:34:17 2010 From: neal at gafter.com (Neal Gafter) Date: Fri, 26 Feb 2010 21:34:17 -0800 Subject: Extermal references: BGGA Prototype change list Message-ID: <15e8b9d21002262134r49b44b77le6395cf2c0b5fdf@mail.gmail.com> Alex Buckley asked me to include BGGA and other documents in messages on this list, so that they may formally be considered contributions to project lambda. Enclosed please find: a document describing features of the BGGA prototype not in the specification. It can also be found at < http://www.javac.info/PrototypeDifferences.html>. Differences Between the Prototype and the Spec (v0.5) Here is a summary of the differences between the current prototype and the last published specification (v0.5). These changes reflect feedback from the user community and are suitable for inclusion in the next revision of the specification. - Renamed Unreachable to Nothing We adopt the name used by Scala to represent the same concept. - Removed support for the type null We used null as a placeholder for an exception type when none can be thrown. The type Nothing now serves that purpose; null is no longer supported as the name of a type. - Overhauled restricted versus unrestricted In the specification, an interface is considered *restricted* if it extends a marker interface. Unfortunately, the specification only provides a syntax for function type interfaces that are unrestricted. We modified the syntax so that a function type written using the => token designates a restricted function type, while one written using the newly introduced ==> token represents an unrestricted function type. This allows programmers to easily write APIs that restrict (or don't restrict) the operations of closure expressions passed as parameters. - Refined restrictions We modified the distinction between restricted and unrestricted closures. As before, it is not legal to convert an unrestricted closure to a restricted interface type, nor is it legal to break, continue, or returnfrom inside a restricted closure to a target outside the closure. However, a restricted closure is allowed to refer to a non-final local variable from an enclosing scope. In this case a warning is given unless one of the following conditions holds: 1. The variable is not the target of any assignment, or 2. The variable is annotated @Shared It is possible to suppress the warning by annotating some enclosing construct @SuppressWarnings("shared"). - Relaxed the closure conversion In response to user feedback, we've relaxed the relationship between a closure parameter's type and the target interface's parameter type. Rather than requiring them to be of the same type, they are now allowed to be related by an assignment conversion, including boxing or unboxing. - for-qualified method declarations The for keyword on a method declaration, meant to introduce a control abstraction method that works like a loop, is now treated syntactically like a modifier rather than appearing immediately before the method name. This helps make the declaration site more similar to the use site. - Added support for method references We added extensive support for treating a reference to a method as a closure using a newly introduced token #. The syntax is borrowed from the FCM proposal . The semantics are as follows: A method reference written as *Primary* # *Identifier* ( *TypeList* ) where the Primary designates an expression (as opposed to a type) is treated the same as a closure { Type x0, Type x1 ... => tmp.Identifier(x0, x1 ...) } or { Type x0, Type x1 ... => tmp.Identifier(x0, x1 ...); } Where tmp is a temporary value that holds the computed value of the primary expression. The former translation is used when the resolved method has a non-void return type, while the latter is used when the resolved method has a void return type. If the primary resolves to a type, then this is translated to { Type x0, Type x1 ... => Primary.Identifier(x0, x1 ...) } or { Type x0, Type x1 ... => Primary.Identifier(x0, x1 ...); } when the resolved method is static, or { Primary x, Type x0, Type x1 ... => x.Identifier(x0, x1 ...) } or { Primary x, Type x0, Type x1 ... => x.Identifier(x0, x1 ...); } when the resolved method is an instance method. In addition, optional explicit type arguments, between angle brackets, may be placed immediately after the # token. These are used directly in the translated method invocation to resolve the method to be invoked. - Implemented a classfile format for the for qualifier We've impleemnted a class file representation of the for qualifier to support separate compilation. From neal at gafter.com Fri Feb 26 21:37:15 2010 From: neal at gafter.com (Neal Gafter) Date: Fri, 26 Feb 2010 21:37:15 -0800 Subject: Extermal references: Closures for the Java Programming Language (v0.5) issues list Message-ID: <15e8b9d21002262137x6149b465kc22c2599414661ff@mail.gmail.com> Alex Buckley asked me to include BGGA and other documents in messages on this list, so that they may formally be considered contributions to project lambda. Enclosed please find the issues list associated with the BGGA specification. It may also be found at < http://www.javac.info/issues-v05.html>. Closures: Open Issues (v0.5) - Make it easy to yield a result early from a closure. It should be as easy to yield a result early from a closure as returning from a method.** - Should support for currying be directly in the language? Bound method references? - *Handle interfaces like Comparator that define more than one method*, all but one of which will be implemented by a method inherited from Object. The definition of "interface with a single method" should count only methods that would not be implemented by a method in Object and should count multiple methods as one if implementing one of them would implement them all. Mainly, this requires a more precise specification of what it means for an interface to have only a single abstract method. - *Specify mapping from function types to interfaces*: names, parameters, etc. We should fully specify the mapping from function types to system-generated interfaces precisely. - *Type inference*. The rules for type inference need to be augmented to accomodate the inference of exception type parameters. Similarly, the subtype relationships used by the closure conversion should be reflected as well. - *VM support for null* The verifier already supports null, including in the verifiers, but there is no way to specify null as a method argument or return type. We need a way to specify null in a signature (that is, a new signature letter). - *Elided exception type parameters* to help retrofit exception transparency. Perhaps make elided exception type parameters mean the bound. This enables retrofitting existing generic interfaces that don't have a type parameter for the exception, such as java.util.concurrent.Callable, by adding a new generic exception parameter. - *Document the new unchecked Throwables for nonlocal transfers*. - *Should javac detect the most common problematic interactions catching nonlocal transfers? *The most likely case may be completely local within a single method, when a nonlocal transfer has a try-catch between the transfer and its target. - *volatile on locals*. We should say it is allowed. - *How are class literals for function types formed?* Is it {?=>?}.class ? - *The system class loader should dynamically generate function type interfaces. *The interfaces corresponding to function types should be generated on demand by the bootstrap class loader, so they can be shared among all user code. For the prototype, we may have javac generate these interfaces so prototype-generated code can run on stock (JDK5-6) VMs. - *Can or should javac inline some invocations of control abstraction APIs? *If so, how will it decide which ones? Does the inlining happen at the bytecode or source level? Will javac handle break/return/continue efficiently when these are inlined? Or is this something should be left for the VM to handle? (Hotspot already seems to handle the most important cases) - *Do we need any new reflection support for closures? *For function types? For null? Probably not because of the information available in the defining (not bridge) methods. - *Should a labelled for control invocation statement* interact with labelled break and continue? - *Should for be a modifier* (before the return type in a method declaration) or appear next to the name? - *Would explicit VM support for nonlocal transfers improve performance?* - *Should we request VM support for proper tail recursion?* - Should we support (parametric) polymorphic function types? Probably not. ------------------------------ [image: Creative Commons License] Closures for Java by Gilad Bracha, Neal Gafter, James Gosling, Peter von der Ah? is licensed under a Creative Commons Attribution-Share Alike 3.0 United States License. From neal at gafter.com Fri Feb 26 22:12:27 2010 From: neal at gafter.com (Neal Gafter) Date: Fri, 26 Feb 2010 22:12:27 -0800 Subject: External References: Closures for Java (CfJ) version 0.6 part a Message-ID: <15e8b9d21002262212u53279a0ci7faa27690be0a84f@mail.gmail.com> Alex Buckley asked me to include BGGA and other documents in messages on this list, so that they may formally be considered contributions to project lambda. Enclosed please find the Closure for Java specification, version 0.6, part a. It can also be found at . Closures for Java (v0.6a) *Neal Gafter and Peter von der Ah?* We describe a proposed extension of the Java Programming Language that supports deferred execution of snippets of code. These correspond to *lambda expressions* in languages such as Javascript, C#, and Groovy. Lambda Expressions We introduce a syntactic form for constructing an anonymus function value: *Expression:**ExpressionLambda**Primary:**StatementLambda**ExpressionLambda* :# ( *FormalParametersopt* ) *Expression* *StatementLambda*:# ( * FormalParametersopt* ) *BlockStatement*** We refer to these two expression forms, *expression lambdas* and *statement lambdas* as *lambda expressions. * Evaluating a lambda expression does not cause its controlled statement or expression to be evaluated immediately, but rather it builds a *function object* that represents the code. The function object can be stored and invoked later to cause the controlled statement or expression to be evaluated. Within the controlled statement of a statement lambda - A return statement yields a result from the statement lambda. - It is a compile-time error if some break or continue statement transfers control to a target outside the statement lambda. Within the controlled expression or statement - The meanings of names are the same as the meanings of names in the context in which the lambda expression occurs, with the exception that the formal parameters introduce variable names that are in scope within the controlled expression or statement (and may shadow variable names from enclosing scopes). - this and super have the same meaning as they would where the lambda expression appears. - A mandatory warning is required if some local variable or parameter from an enclosing scope is used, unless one of the following conditions holds: 1. The variable is declared final, or 2. The variable is not the target of any assignment (i.e., it is only initialized in its declaration), or 3. The variable is annotated @Shared, or 4. some enclosing construct is annotated @SuppressWarnings("shared"). The type of a lambda expression is defined by the set of declared argument types, result expressions, and thrown types of the lambda's body. While the type of a lambda expression is not denotable in source, a lambda expression may be converted to some object type by a *lambda conversion* (see later). The target of such a conversion it typically a *function type* (see below) or other user-defined interface type. It is a compile-time error if a lambda expression appears in a context where it is not subject to a lambda conversion. The conversion occurs entirely at compile-time. *Example*: the following lambda expression takes two integers and yields their sum: #(int x, int y) x+y Function Types We introduce a syntactic form for a *function type*: *Type:**FunctionType**FunctionType*:*# **Type* ( *TypeListopt** )* * FunctionThrowsopt* *TypeList:**Type Type* , *TypeList**FunctionThrows*:throws *FunctionExceptionTypeList* * FunctionExceptionTypeList:**ExceptionType FunctionExceptionTypeList | ExceptionType * Informally, a function type describes the set of functions that accept a given list of argument types, result in a value of the given type, and may throw the indicated checked exception types. *Example*: the following assigns to the local variable plus a function that computes the sum of its two int arguments: #int(int,int) plus = #(int x, int y) x+y; A function type is defined to have the same meaning as a parameterized interface type of a system-generated interface that has a single method named invoke. *[This paragraph is a placeholder for a more precise description, including the name of the interface, required for interoperability among compilers. We expect the generated interfaces to be in the package java.lang.function, which will be closed to user-defined code, and the naming scheme to be an extension of the JVM method signature scheme.] * While some have proposed to add function types as a first-class extenstion to Java's type system, we don't believe that is necessary. Interfaces are expressive enough to be used to represent function types. Unfortunately, doing so directly in user code is clumsy, error-prone, verbose, and difficult to read and write. The function type notation provides a more readable, convenient, and less error-prone shorthand, and the interfaces to which it is translated provide a standard for interoperability among libraries and languages. *Example*. The variable declaration #int(String) parseInteger; is translated into interface java.lang.function.IO { // system-generated int invoke(A1 x1) throws X; } IO parseInteger; We would expect the compiler, runtime system, and debuggers to display these types using the function type syntax rather than the translation involving generics. Because a function type *is* an interface type, it can be extended by a user-defined interface and it can be implemented by a user-defined class. Our prototype translation scheme causes a function type # *Tr* ( *T0* ... *Tn* ) throws *E0 | ... Em* to be defined as a system-generated parameterized interface type with a single abstract method public *Tr* invoke(*T0* x0, ... *Tn* xn) throws *E0*, ... *Em* By relying on the existing Java subtype rules for generic interfaces, they obey the following subtyping relationship: A function type # *Tr* ( *T0* ... *Tn* ) throws *E0** | ... Em* is a subtype of function type # *Ur* ( *U0* ... *Ul* ) throws *X**0 | ... Xk* iff all of the following hold: - Either - *Tr* is the same as *Ur* or - Both are reference types and *Tr* is a subtype of *Ur*; - *n *is equal to *l*; - for every* i* in *0..n*, either - *Ti* and *Ui* are the same types or - Both are reference types and* Ui* is a subtype of*Ti*; and - for every *j* in *0..m*, there exists an *h* in *0..k* such that *Ej*is a subtype of *Xh*. In English, these rules are - Function types may have covariant returns; - A subtype function must accept the same number of arguments as its supertype; - Function types may have contravariant argument types; and - The subtype function may not throw any checked exception type not declared in the supertype (function subtyping is exception-safe) Lambda Conversion A lambda expression may be converted to any compatible interface type by the *lambda conversion*, which is a kind of widening conversion. There is a *lambda conversion* from a lambda expression to every interface type that has a single method *m* such that the lambda expression is compatible with* m*. A lambda expression is *compatible with* a method *m*iff all of the following hold: - Either - The lambda expression is an expression lambda, and either - there is an assignment conversion from the type of its result expression to the return type of *m*; or - the result expression is a subtype of java.lang.Void and the method *m* has return type void; or - The lambda expression is a statement lambda, and either - its body cannot complete normally and there is an assignment conversion from the type of the expression in each of its return statements to the return type of *m*; or - all of its return statements yield no return value, and the method *m* has return type void or java.lang.Void - *m* has the same number of parameters as the lambda expression. - For each corresponding position in the parameter lists**, both parameter types are the same. - Every checked exception type that can be thrown by the body of the lambda expression is a subtype of some exception type in the throws clause of *m*. *Example*: We can create a function object that adds two to its argument like this: interface IntFunction { int invoke(int i); } IntFunction plus2 = #(int x) x+2; Alternatively, using function types: #int(int) plus2 = #(int x) x+2; We can use the existing Executor framework to run a function object in the background: void sayHello(java.util.concurrent.Executor ex) { ex.execute(#(){ System.out.println("hello"); }); } Method References A *method reference* is an expression form that allows a reference to a method to be used as shorthand for a lambda expression that invokes that method *Primary:**MethodReference**MethodReference:**Primary *# *TypeArgumentsoptIdentifier * ( *TypeListopt* ) When the Primary designates an expression (as opposed to a type) is treated the same as a lambda expression (#(Type x0, Type x1 ...)tmp.Identifier(x0, x1 ...)) or (#(Type x0, Type x1 ...){tmp.Identifier(x0, x1 ...);}) Where tmp is a temporary value that holds the computed value of the primary expression. The former translation is used when the resolved method has a non-void return type, while the latter is used when the resolved method has a void return type. If the primary resolves to a type, then this is translated to (#(Type x0, Type x1 ...)Primary.Identifier(x0, x1 ...)) or (#(Type x0, Type x1 ...){Primary.Identifier(x0, x1 ...);}) when the resolved method is static, or (#(Primary x, Type x0, Type x1 ...)x.Identifier(x0, x1 ...)) or (#(Primary x, Type x0, Type x1 ...){x.Identifier(x0, x1 ...);}) when the resolved method is an instance method. Exception type parameters To support exception transparency (that is, the design of APIs in which the compiler can infer that exceptions thrown by a lambda expression are thrown by the API), we add a new type of generic formal type argument to receive a set of thrown types. *[This deserves a more formal treatment]* *TypeParameter:*throws* TypeVariable* *TypeBoundopt**ActualTypeArgument*:* throws* *ExceptionList* *ExceptionList*:*ReferenceType* *ReferenceType** **|* *ExceptionList* What follows is an example of how the proposal would be used to write an exception-transparent version of a method that locks and then unlocks a java.util.concurrent.Lock, before and after a user-provided block of code. Given public static T withLock(Lock lock, #T()throws E block) throws E { lock.lock(); try { return block.invoke(); } finally { lock.unlock(); } } This can be used as follows: withLock(lock, #(){ System.out.println("hello"); }); This uses a newly introduced form of generic type parameter. The type parameter E is declared to be used in throws clauses. If the extends clause is omitted, a type parameter declared with the throws keyword is considered to extend Throwable (instead of Object, which is the default for other type parameters). This type parameter accepts multiple exception types. For example, if you invoke it with a block that can throw IOException or NumberFormatException the type parameter E would be given as IOException|NumberFormatException. In those rare cases that you want to use explicit type parameters with multiple thrown types, the keyword throws is required in the invocation, like this: Locks.withLock(lock, #(){ System.out.println("hello"); }); You can think of this kind of type parameter accepting disjunction, "or" types. That is to allow it to match the exception signature of a function that throws any number of different checked exceptions. If the block throws no exceptions, the type argument would be the type Nothing (see below). Type parameters declared this way can be used only in a throws clause or as a type argument to a generic method, interface, or class whose type argument was declared this way too. Throws type parameters are indicated in class file format by the presence of the character '^' preceding the type parameter. Definite assignment A variable V is (un)assigned after a lambda expression iff V is (un)assigned before the lambda expression A variable V used in a lambda expression's body but declared outside the lambda is - definitely assigned before the lambda expression's body iff V is definitely assigned before the lambda expression; and - not definitely unassigned before the lambda expression's body. The return statement A return statement returns control to the invoker of the nearest enclosing method, constructor, *or* *statement lambda*. The type Nothing We add the non-instantiable type java.lang.Nothing. Values of this type appear where statements or expressions cannot complete normally. This is necessary to enable writing APIs that provide transparencyfor functions that do not complete normally. Nothing is a subtype of every obect type. The null type is a subtype of Nothing. At runtime, a NullPointerException is thrown when a value of type Nothing would appear. This occurs when a programmer converts null to Nothing. *Example*: The following illustrates a function being assigned to a variable of the correct type. interface NullaryFunction { T invoke() throws E; } NullaryFunction thrower = #(){ throw new AssertionError(); }; Reachable statements The initial statement of a statement lambda is reachable. An expression statement in which the expression is of type Nothing cannot complete normally. Acknowledgments The authors would like to thank the following people whose discussions and insight have helped us craft, refine, and improve this proposal: * C. Scott Ananian, Lars Bak, Cedric Beust, Joshua Bloch, Gilad Bracha, Martin Buchholz, Alex Buckley, Maurizio Cimadamore, Ricky Clarkson, Stephen Colebourne, Danny Coward, Luc Duponcheel, Erik Ernst, R?mi Forax, James Gosling, Christian Plesner Hansen, Cay Horstmann, Kohsuke Kawaguchi, Danny Lagrouw, Doug Lea, Peter Levart, "crazy" Bob Lee, Mark Mahieu, Niklas Matthies, Tony Morris, Martin Odersky, Terence Parr, Tim Peierls, Cris Perdue, John Rose, Ken Russell, Stefan Schulz, Guy Steele, Mads Torgersen, Zdenek Tronicek, Jan Vitek, *and* Dave Yost. * Minor changes since 0.6 first published - Added *Peter von der Ah?* as a coauthor. - In the *lambda expressions* section, separated two semantic bullets that only apply to statement lambdas. - The VM signature character for throws type parameters has been changed from # to ^. - Other minor formatting improvements and additions to the acknowledgments. - A "throws" type parameter implicitly extends Throwable, not Exception. - The lambda conversion allows an expression lambda yielding java.lang.Void to be compatible with a method returning void - Within a lambda expression, super has the same meaning as it would in the enclosing scope. Technically, this is not a change to the meaning of the specification, because it "goes without saying". - Reworded the definite assignment section to use JLS terminology. - Clarified that the type of a lambda expression is not denotable. Changes in this revision (0.6) compared to BGGA 0.5 - The *control invocatio**n syntax* has been moved to a separate specification. - The term *closure literal* has been replaced by *lambda expression*. - We overhauled the syntax for lambda expressions, borrowing loosely from Clang. There are now two forms of lambda expression: an *expression lambda* has a controlled expression, while a *statement lambda* has a controlled statement. - The term *closure object* has been replaced by *function object*. - The term *closure conversion* has been replaced by *lambda conversion*. - Added new semantics for the return statement: it can now be used to return from a statement lambda. - Renamed java.lang.Unreachable to java.lang.Nothing ala Scala . - Removed support for the type name null. The previous version used nullas a placeholder for an exception type when none can be thrown. The type Nothing now serves that purpose. - Overhauled restricted. The concepts of *restricted* and *unrestricted*function types and closures have been removed. Now, all lambda expressions are restricted. The equivalent of the previous specification's unrestricted closure may be passed to a method by using the *control invocation syntax * (0.6b). - Added support for *method references*: We added support for treating a reference to a method as a function using a newly introduced token #. The syntax is borrowed from cross-references in javadocand the FCM proposal . Changes that need to go into the prototype - Change the syntax of lambda expressions. - Remove the restricted/unrestricted distinction; remove JDK API retrofits. - Make #int(int) a non-generic interface java.lang.function.II0. - Integrate the closures prototype with recent openjdk javac changes, particularly the diagnostic improvements. - Change the pretty-printing of function types to use the new function type syntax. ------------------------------ [image: Creative Commons License] Closures for Java by Neal Gafter and Peter von der Ah?is licensed under a Creative Commons Attribution-Share Alike 3.0 United States License. From david.goodenough at linkchoose.co.uk Sat Feb 27 02:46:49 2010 From: david.goodenough at linkchoose.co.uk (David Goodenough) Date: Sat, 27 Feb 2010 10:46:49 +0000 Subject: A reminder of Project Lambda's scope In-Reply-To: <4B886052.5070305@sun.com> References: <4B886052.5070305@sun.com> Message-ID: <201002271046.49540.david.goodenough@linkchoose.co.uk> On Friday 26 February 2010, Alex Buckley wrote: > http://cr.openjdk.java.net/~mr/lambda/straw-man/ > > Where features in the strawman have made it into the draft spec so far, > such as function types and lambda conversion, they are fair game for > discussion. > > There is no need to lobby for mutable up-level variables, method > references, exception transparency, extension methods, etc, since they > are all mentioned in the strawman. The draft spec will get round to them > in good time. Before anyone asks for a schedule, I don't have one. > > Alex > Does this mean that obvious extensions of things, like field references being a logical extension of method references, can also be considered? David From neal at gafter.com Sat Feb 27 07:09:27 2010 From: neal at gafter.com (Neal Gafter) Date: Sat, 27 Feb 2010 07:09:27 -0800 Subject: External References: Closures for Java (CfJ) version 0.6 part b Message-ID: <15e8b9d21002270709l4c37bcf3l96b51b4090eab62d@mail.gmail.com> Alex Buckley asked me to include BGGA and other documents in messages on this list, so that they may formally be considered contributions to project lambda. Enclosed please find the Closure for Java specification, version 0.6, part b. It can also be found at . Closures for Java (v0.6b) *Neal Gafter and Peter von der Ah?* Building on *lambda expressions* and *exception type parameters* described in the companion document v0.6a, we describe support for *control abstractions*, which are APIs that can abstract patterns of control. These correspond to *blocks* and *lambda expressions * in languages such as Ruby, Smalltalk, and Scala. Block Expressions We extend the syntax of a parenthesized expression to allow it to be preceded by a sequence of statements. The grammar for a parenthesized expression [JLS 15.8.5] is changed to: ParExpression:*(* *BlockStatementsopt* *Expression* *)* Note that the existing syntax for a parenthesized expression is a special case of this. *Meaning of Expressions*: The specification for a parenthesized expression is modified to describe its new execution semantics: The block statements (if any) are executed in sequence, from left to right. The result of the block expression is: - The type Nothing if there are block statements and the last block statement cannot complete normally; otherwise - The value and type of the final expression *Definite Assignment*: The definite assignment rules for this construct are almost identical to that for the block statement. The definite assignment state before the first block statement is the definite assignment state before the parenthesized expression. The definite assignment state before the subexpression is the definite assignment state following the last block statement. The definite assignment state after the parenthesized expression is the definite assignment state after the contained expression. *Exception Checking*: A parenthesized expression can throw exception type E if any statement or expression immediately contained in it can throw E. *Scoping*: Local variables and classes declared by an immediately contained statement is in scope until the end of the parenthesized expression. Control invocation syntax A new invocation statement syntax is added to make closures convenient for control abstraction: *ControlInvocationStatement*:for*opt Primary* (* FormalParameters *: * ExpressionListopt* ) *Statement* for*opt Primary* (* ExpressionListopt* ) *Statement* This syntax is a shorthand for the following statement: *Primary* *(* *ExpressionList*, *# (* *FormalParametersopt* *) (* *Statement * *(Void)null ) ); * (See the later section *loop abstractions* for the meaning of the optional for keyword) This syntax makes some kinds of function-receiving APIs more convenient to use to compose statements. We could use the shorthand to write our previous example (0.6a) this way withLock(lock) { System.out.println("hello"); } Another example of its use would be a an API that closes a java.io.Closeableafter a user-supplied block of code, discarding any exception from Closeable.close: R with(T t, #R(T) throws E block) throws E { try { return block.invoke(t); } finally { try { t.close(); } catch (IOException ex) {} } } We could use the shorthand with this API to close a number of streams at the end of a block of code: with(FileReader in : makeReader()) with(FileWriter out : makeWriter()) { // code using in and out } Loop Abstractions The specification supports writing methods that act as control statements, but when used to support loops the API implementor should be able to specify that the controlled statement should capture the meaning of break and continue in a way analogous to the built-in loop statements. For this purpose we allow the for keyword at the beginning of a control invocation statement (see above), and introduce the use of the keyword for as a modifier on method declarations: *MethodDeclarator*:for*opt* *Identifier* ( *FormalParamterListopt* ) An overriding or implementing method must be declared with the keyword forif and only if the method being overridden or implemented was declared with the keyword for. A control invocation statement must use the for keyword if and only if the method being invoked was declared with the keyword for. Within the controlled statement of a control invocation statement using the keyword for: - An unlabelled break statement breaks from the control invocation statement. - An unlabelled continue statement breaks from the controlled statement. *Example*: The following illustrates a method used to loop through the key-value pairs in a map. The method void for eachEntry(Map map, #void(K,V) throws X} block) throws X { for (Map.Entry entry : map.entrySet()) { block.invoke(entry.getKey(), entry.getValue()); } } allows us to rewrite this void test(Map map) { for (Map.Entry entry : map.entrySet()) { String name = entry.getKey(); Integer value = entry.getValue(); if ("end".equals(name)) break; if (name.startsWith("com.sun.")) continue; System.out.println(name + ":" + value); } } like this void test(Map map) { for eachEntry(String name, Integer value : map) { if ("end".equals(name)) break; if (name.startsWith("com.sun.")) continue; System.out.println(name + ":" + value); } } Unmatched Control Transfers Control transfers by return, break, or continue can, within an expression lambda, jump to a target outside the body of the lambda. It is possible for such code to be invoked at a time that the target of the transfer is not on the call stack of the current thread. In this case an exception of type UnmatchedTransferException is thrown. package java.lang; /** * Exception thrown when a transfer from within a lambda doesn't have * a matching frame on the stack of the current thread. * * @author gafter */ public class UnmatchedTransferException extends RuntimeException { /** * This constructor is used by compiler-generated code. */ public UnmatchedTransferException(Jump jump); /** * Returns the thread in which the transfer target is executing. */ public Thread thread(); /** * Return the exception to be thrown to cause the transfer to * occur. This can be used to implement concurrent APIs * that support control transfers. See, for example, * http://gafter.blogspot.com/2006/10/concurrent-loops-using-java-closures.html * and http://markmahieu.blogspot.com/2008/08/for-eachconcurrently.html */ public RuntimeException transfer(); } Acknowledgments The authors would like to thank the following people whose discussions and insight have helped us craft, refine, and improve this proposal: * C. Scott Ananian, Lars Bak, Cedric Beust, Joshua Bloch, Gilad Bracha, Martin Buchholz, Alex Buckley, Maurizio Cimadamore, Ricky Clarkson, Stephen Colebourne, Danny Coward, Luc Duponcheel, Erik Ernst, R?mi Forax, James Gosling, Christian Plesner Hansen, Cay Horstmann, Kohsuke Kawaguchi, Danny Lagrouw, Doug Lea, Peter Levart, "crazy" Bob Lee, Mark Mahieu, Niklas Matthies, Tony Morris, Martin Odersky, Terence Parr, Tim Peierls, Cris Perdue, John Rose, Ken Russell, Stefan Schulz, Guy Steele, Mads Torgersen, Zdenek Tronicek, Jan Vitek, *and* Dave Yost. * Changes in this revision (0.6) compared to BGGA 0.5 - The translation of the control invocation statement has been modified to reflect changes in the lambda syntax. - UnmatchedNonlocalTransfer has been renamed UnmatchedTransferException. Changes that need to go into the prototype - Implement block expressions. - Revise the implementation of the control invocation statement as specified. - Implement complete transparency for control flow from a closure. Specifically, it should not be possible to intercept matched control transfers by catching Throwable. ------------------------------ [image: Creative Commons License] Closures for Java by Neal Gafter and Peter von der Ah?is licensed under a Creative Commons Attribution-Share Alike 3.0 United States License. From john at milsson.nu Sat Feb 27 07:10:09 2010 From: john at milsson.nu (John Nilsson) Date: Sat, 27 Feb 2010 16:10:09 +0100 Subject: A syntax option (function types versus arrays) In-Reply-To: <15e8b9d21002262008u6f947137v57069bd875c57242@mail.gmail.com> References: <15e8b9d21002261748u77d29847ob56e4544263b5d1c@mail.gmail.com> <21208A19-213F-47CC-A062-4CF038651FA4@googlemail.com> <15e8b9d21002262008u6f947137v57069bd875c57242@mail.gmail.com> Message-ID: On Sat, Feb 27, 2010 at 5:08 AM, Neal Gafter wrote: > *static** * > > *(T)->(U throws X)->V curry((T,U throws X)->V function) {* > > * return (T t)->(U u)->function.(t,u);* > > *}* > Is it U that can throw X or is it (T)->U that can throw X ? To me the syntax suggests the former, if the latter is intendend maybe the syntax could be tweaked somewhat? (T)-> throws X (U)->V (T)-> ((U)->V) throws X T -X> U -> V BR, John From john at milsson.nu Sat Feb 27 07:14:56 2010 From: john at milsson.nu (John Nilsson) Date: Sat, 27 Feb 2010 16:14:56 +0100 Subject: A syntax option (function types versus arrays) In-Reply-To: References: <15e8b9d21002261748u77d29847ob56e4544263b5d1c@mail.gmail.com> <21208A19-213F-47CC-A062-4CF038651FA4@googlemail.com> <15e8b9d21002262008u6f947137v57069bd875c57242@mail.gmail.com> Message-ID: On Sat, Feb 27, 2010 at 4:10 PM, John Nilsson wrote: > (T)-> throws X (U)->V > (T)-> ((U)->V) throws X > T -X> U -> V > Or, (throws X) could be seen as a return type of its own (replacing Nothing in some instances) So the following are equally correct T -> throws X | (U) -> V T -> ((U) -> V) | throws X Another usage would be public throws X error(String msg)) { throw new ... } which would be the same as public Nothing error(String msg) throws X BR, John From neal at gafter.com Sat Feb 27 07:14:45 2010 From: neal at gafter.com (Neal Gafter) Date: Sat, 27 Feb 2010 07:14:45 -0800 Subject: External References: Issues List for Closures for Java (CfJ) version 0.6 parts a and b Message-ID: <15e8b9d21002270714r4911a9d3ue6233efb1b239149@mail.gmail.com> Alex Buckley asked me to include BGGA and other documents in messages on this list, so that they may formally be considered contributions to project lambda. Enclosed please find the Issues Lists Closure for Java specification, version 0.6, parts a and b. They can also be found at < http://www.javac.info/issues-v06a.html> and < http://www.javac.info/issues-v06b.html>. Closures: Open Issues (v0.6a) - *Can Method Handles be used for Function Types?* It isn't obvious how to make that work. One problem is that Method Handles reify type parameters, but in a way that interferes with function subtyping. - *Can we get rid of the explicit declaration of "throws" type parameters?* The idea would be to use disjuntive type inference whenever the declared bound is a checked exception type. This is not strictly backward compatible, but it's unlikely to break real existing code. We probably can't get rid of "throws" in the type argument, however, due to syntactic ambiguity. - *Disallow @Shared on old-style loop index variables* - *Handle interfaces like Comparator that define more than one method*, all but one of which will be implemented by a method inherited from Object. The definition of "interface with a single method" should count only methods that would not be implemented by a method in Object and should count multiple methods as one if implementing one of them would implement them all. Mainly, this requires a more precise specification of what it means for an interface to have only a single abstract method. - *Specify mapping from function types to interfaces*: names, parameters, etc. We should fully specify the mapping from function types to system-generated interfaces precisely. - *Type inference*. The rules for type inference need to be augmented to accomodate the inference of exception type parameters. Similarly, the subtype relationships used by the closure conversion should be reflected as well. - *Elided exception type parameters* to help retrofit exception transparency. Perhaps make elided exception type parameters mean the bound. This enables retrofitting existing generic interfaces that don't have a type parameter for the exception, such as java.util.concurrent.Callable, by adding a new generic exception parameter. - *How are class literals for function types formed?* Is it #void().class ? If so, how does it work if object types are erased? Is it #?(?).class ? - *The system class loader should dynamically generate function type interfaces. *The interfaces corresponding to function types should be generated on demand by the bootstrap class loader, so they can be shared among all user code. For the prototype, we may have javac generate these interfaces so prototype-generated code can run on stock (JDK5-6) VMs. - *Must the evaluation of a lambda expression produce a fresh object each time?* Hopefully not. If a lambda captures no variables from an enclosing scope, for example, it can be allocated statically. Similarly, in other situations a lambda could be moved out of an inner loop if it doesn't capture any variables declared inside the loop. It would therefore be best if the specification promises nothing about the reference identity of the result of a lambda expression, so such optimizations can be done by the compiler. ------------------------------ [image: Creative Commons License] Closures for Java by Neal Gafter and Peter von der Ah?is licensed under a Creative Commons Attribution-Share Alike 3.0 United States License. Closures: Open Issues (v0.6b) - *The specification does not define the classfile format for the "for" qualifier.* - *The for keyword on a method declaration should be a qualifier.* Meant to introduce a control abstraction method that works like a loop, "for" should be treated syntactically like a modifier rather than appearing immediately before the method name. This helps make the declaration site more similar to the use site. - *Relax the closure conversion.* The closure conversion requires that the parameters of a closure exactly match the parameter types of the interface it is converted to. Experience with the 0.5 prototype suggests that this should be relaxed to an assignment conversion. - *Control transfers from a lambda.* We should describe in nonnormative text the techniques used in the prototype to implement control transfers out of an expression lambda. ------------------------------ [image: Creative Commons License] Closures for Java by Neal Gafter and Peter von der Ah?is licensed under a Creative Commons Attribution-Share Alike 3.0 United States License. From neal at gafter.com Sat Feb 27 07:17:17 2010 From: neal at gafter.com (Neal Gafter) Date: Sat, 27 Feb 2010 07:17:17 -0800 Subject: A syntax option (function types versus arrays) In-Reply-To: References: <15e8b9d21002261748u77d29847ob56e4544263b5d1c@mail.gmail.com> <21208A19-213F-47CC-A062-4CF038651FA4@googlemail.com> <15e8b9d21002262008u6f947137v57069bd875c57242@mail.gmail.com> Message-ID: <15e8b9d21002270717l2396f7d3o91c04fa4e4e81018@mail.gmail.com> On Sat, Feb 27, 2010 at 7:10 AM, John Nilsson wrote: > On Sat, Feb 27, 2010 at 5:08 AM, Neal Gafter wrote: > >> *static** * >> >> >> *(T)->(U throws X)->V curry((T,U throws X)->V function) {* >> >> * return (T t)->(U u)->function.(t,u);* >> >> *}* >> > > Is it U that can throw X or is it (T)->U that can throw X ? > (U throws X) -> V means you might get an exception (instead of a V) when you pass the U. From forax at univ-mlv.fr Sat Feb 27 07:23:33 2010 From: forax at univ-mlv.fr (=?ISO-8859-1?Q?R=E9mi_Forax?=) Date: Sat, 27 Feb 2010 16:23:33 +0100 Subject: Accessing non-final local variables from a lambda expression In-Reply-To: <15e8b9d21002261541v575de9ebqb6e5cf35bb7a456c@mail.gmail.com> References: <15e8b9d21002252336o2f7280d2v157a9acd38ab29da@mail.gmail.com> <15e8b9d21002261541v575de9ebqb6e5cf35bb7a456c@mail.gmail.com> Message-ID: <4B8938F5.3030902@univ-mlv.fr> Le 27/02/2010 00:41, Neal Gafter a ?crit : > On Fri, Feb 26, 2010 at 2:59 PM, John Nilsson wrote: > >> Why not something like this instead? >> Iterable everyOther(Iterable input) { >> return zipWithIndex(input).filter(#(long i, T t)(i%2==0)).map(#(long i, >> T t)(t)); >> } >> > Very clever. > > >> Point being, wouldn't it be better, from a concurrency point of view, to >> encourage a functional style of programming instead of focusing on >> supporting shared mutable state? >> > Where it's practical, yes. But there is a difference between > encouraging a functional style and discouraging an imperative style. > I admire the solution, but it isn't realistic to expect people to come > up with it without significantly more time invested than is required > for the simple solution. Most programmers have a job to do and a life > outside work. > I don't agree, filter and map are simple concepts already used in several other mainstream languages. If you have them in java.util, people will use them. The problem is not to discouraging imperative style but encourage the developers to use the right tools for the right problem. If you want to create (virtually) an iterable from an iterable, a function style programming is better than an imperative one. R?mi From reinier at zwitserloot.com Sat Feb 27 14:05:56 2010 From: reinier at zwitserloot.com (Reinier Zwitserloot) Date: Sat, 27 Feb 2010 23:05:56 +0100 Subject: Accessing non-final local variables from a lambda expression In-Reply-To: <201002261701.24155.peter.levart@marand.si> References: <15e8b9d21002252336o2f7280d2v157a9acd38ab29da@mail.gmail.com> <997cab101002260023t570987beob147c83288da4a23@mail.gmail.com> <201002261701.24155.peter.levart@marand.si> Message-ID: <560fb5ed1002271405j63a3db17s8809eecc2ce1ae2c@mail.gmail.com> On Fri, Feb 26, 2010 at 5:01 PM, Peter Levart wrote: > > The problem with this approach (from the performance perspective) is when > you capture several mutable local variables. Having a separate wrapper > object for each of them is an overhead which can be avoided if the compiler > constructs a single frame object for you. > That's entirely correct, but doing this is, if I understand correctly, very complicated from the perspective of the java memory model. Also, without a very specific sort of unsafe/safe closures I don't see how this can be done: If a closure lives on beyond the lifetime of its parent frame, then obviously the variable cannot _be_ in the parent frame. These variables would have to be hosted in a separate frame that lives on until the method and ALL closures in it (or at least all the ones that access this variable) have been GCed. Right now frames don't have a dependency on the GC so I don't see how this can be done in a way that is more performant than a heap allocation. Also, if a closure is transported to a different thread, the same problem occurs, even if the original method is still 'live'. Frames, as I understand the JVM, aren't shared between threads. Hence I'm assuming in all these discussions about syntax that it boils down to some sugar that hosts the variable on the heap, via e.g. an intermediate array, or perhaps a custom class to do it, which makes it easier for the JVM to recognize this case and eventually perhaps optimize for it. > There are perfectly valid use-cases (like the one above) where accessing a > mutable local variable is safe. For such cases it should not be necessary to > use @Shared annotation on a variable to prevent a warning. The only way for > a mutable local variable to escape the method is via capture from a closure > whose reference escapes the method. So a better approach would be to > annotate function type (and SAM type) variables > (instance/static/local/parameters) that hold references to closures, not > mutable local variables accessed from closures. > In interesting idea, but I don't think the type system is the right place. For example, TreeSet takes a Comparator, but this Comparator is *NOT* transparency-safe; after all, the TreeSet keeps a reference to it and will invoke it each time an item is added to the TreeSet, which may even occur in another thread. On the other hand, that same Comparator type is also used for Collections.sort, but here its entirely transparency-safe. It doesn't escape the sort() call, and the sort() call doesn't do any threading. Instead it would have to be a flag on the method itself, or to be even more precise, a method parameter. An alternative is to just let Comparator rot, and focus on the new (or introduce closure syntax for both, but to avoid the java API hosting many superfluous methods, this optimally should be solved with some sort of automatic adaptation of closures into SAM types, as is part of most proposals including the current status quo). Either way this means it's not really part of the type system then, and you'd have to write: public static void sort(List, @Safe Comparator foo). "@Safe" doesn't feel right for this. I was coincidentally already working on somehing similar to this; I was using the "do" keyword. Either way you still have the problem of _writing_ the closure in the first place: What should happen when the closure does things that are only legit in a @Safe fashion? Should the notion of safety be part of a closure's _type_? Isn't that a big burden on the type system - yet another flag that needs to be tracked? > > A simple approach with a single annotation @Safe (meaning: multi thread > safe) could work as follows: > > void test() > { > boolean flag = true; > > // OK > ()->boolean flipFlop = ()->(flag ^= true); > > // warning: assignment of unsafe function to @Safe variable > @Safe ()->boolean safeFlipFlop = ()->(flag ^= true); > > // OK: compiler determines lambda is safe (doesn't access mutable local > vars) > @Safe ()->void safeHello = ()->(System.out.println("Hello!"); void); > > flipFlop = safeFlipFlop; // OK > safeFlipFlop = flipFlop; // warning: assignment of unsafe function to > @Safe variable > > int count = 0; > Object lock = new Object(); > // OK: user suppresses the warning since she knows her code is safe > @SuppressWarnings("unsafe") > @Safe ()->int safeCounter = ()->(int c; synchronized(lock) { c = count++; > } c); > } > > All JDK methods/constructors accepting SAM types as parameters and invoking > them from other threads (like new Thread, Executor.submit, ...) would have > to add @Safe to their SAM parameters and any such 3rd party APIs would have > to be retrofitted with annotations too. > > That's a drawback of this approach but It might still be better than > @Shared on local vars. If users are forced to shut-up the compiler in all > valid use-cases where it is perfectly safe to access mutable local vars, > then they might develop a habit of putting @Shared annotation on every local > variable accessed in lambdas without verifying that passing such lambdas to > different methods (threads) is actually safe. On the other hand, when using > @Safe on function/SAM variables, warnings would be less frequent, more > accurate and would therefore have a better chance of being considered. > > The issue with this approach is whether to treat accesses to > instance/static fields of outer classes and invocations of methods as safe > or unsafe. In general it is unsafe, but it is allowed by default in inner > classes so if we just want to issue warnings for the "new dangerous > feature", then they should be treated as safe. > > The general approach to checking the multi-thread-safe code would of course > extend to methods and be orthogonal to lambdas, but If we just want to play > safe with new lambda features then the simple approach described above could > be enough. > > Regards, Peter > > From reinier at zwitserloot.com Sat Feb 27 14:12:10 2010 From: reinier at zwitserloot.com (Reinier Zwitserloot) Date: Sat, 27 Feb 2010 23:12:10 +0100 Subject: Method References In-Reply-To: <17b2302a1002260840u3318d99ep9561e6fd1963fd7a@mail.gmail.com> References: <15e8b9d21002252252t45faaa9fq6859222b31a6052f@mail.gmail.com> <4B878DE6.4070100@oracle.com> <4B87A632.4010100@univ-mlv.fr> <15e8b9d21002260815y6c48f7a9n2a70ca5e0352274d@mail.gmail.com> <17b2302a1002260840u3318d99ep9561e6fd1963fd7a@mail.gmail.com> Message-ID: <560fb5ed1002271412n4a729d33h8a7d29896e7d9525@mail.gmail.com> Is this really necessary? If the signature of saveState is: void saveState() { ... } then instead of adding syntax so you can write: #void() saveStateRef = this#saveState; you can just use the existing closures status quo: #void() saveStateRef = #() {saveState();}; The syntax gets more wordy when there are a lot of parameters, and you have to switch from brackets to parens depending on whether the method you're trying to turn into a closure returns void or non-void, and this is clearly inferior to just being able to write "this#saveState", but, two things: 1. Methods aren't closures. They are similar, but with the explicit 'call it in a closure' style, you make this more obvious. 2. The second way means zero additional effort required in the JLS and compiler; you get it for free with closure support. Perhaps said differently, if the closure syntax CANNOT effortlessly wrap around methods, maybe the closure syntax (whatever it ends up being) is no good. --Reinier Zwitserloot On Fri, Feb 26, 2010 at 5:40 PM, Joshua Bloch wrote: > I agree that method references would be a fine addition to the proposal, > and > I like the proposed syntax. I believe it's important that method references > work for constructors and static methods as well as instance methods. > > I am very much in favor of allowing the user to omit the type information > when it's unambiguous. In fact, is their any reason to use "this" (as > proposed by Fredrick): > > cb = this#saveState; > > or could we allow: > > cb = #saveState; > > Not only is this more succinct, but it mirrors the rules for method > invocation, allowing "reasoning by analogy" and reducing programmer > astonishment. > > Josh > > On Fri, Feb 26, 2010 at 8:15 AM, Neal Gafter wrote: > > > On Fri, Feb 26, 2010 at 2:45 AM, R?mi Forax wrote: > > > > > there is a source compatibility issue if you allow to not specify the > > type. > > > > > > class A { > > > void m(String s) { ... } > > > } > > > .... > > > ref = A#m; > > > > > > Now suppose I want to add a new method m(Integer), > > > > > > > Adding a new method to a class is not a source-compatible change today. > > However, it is a binary-compatible change today and both with and > without > > method references and with or without the proposed method reference > > argument > > inference. > > > > > > From reinier at zwitserloot.com Sat Feb 27 14:24:40 2010 From: reinier at zwitserloot.com (Reinier Zwitserloot) Date: Sat, 27 Feb 2010 23:24:40 +0100 Subject: Effectively final effective? In-Reply-To: <636fd28e1002261146p36910745n4d424a3145535181@mail.gmail.com> References: <4B85B844.9050507@adres.pl> <560fb5ed1002250857i42a63aa2u4d3c1cebc74762b@mail.gmail.com> <560fb5ed1002251326q6d12c59wa3ca36f496a620c9@mail.gmail.com> <212322091002251846k397efc92rc10e1462175cd524@mail.gmail.com> <7926817e1002251925md1f3686k2742d81a75c60d5f@mail.gmail.com> <560fb5ed1002260509l3f5e2d98td49a5d959c5e3df5@mail.gmail.com> <636fd28e1002261146p36910745n4d424a3145535181@mail.gmail.com> Message-ID: <560fb5ed1002271424u2240e222hacb1713bf92e8c60@mail.gmail.com> I don't understand. There is no source incompatibility. That is, it is perfectly allright for a new language feature to make previously illegal code, legal. For example, this: for (String x : foo) {} was illegal in java 1.4. It was legal in 1.5. --Reinier Zwitserloot On Fri, Feb 26, 2010 at 8:46 PM, Alex Blewitt wrote: > On Fri, Feb 26, 2010 at 1:09 PM, Reinier Zwitserloot > wrote: > > NB: Jesse, regardless of the pros and cons of making a backwards > > incompatible java 2.0 release, Java 7 most assuredly is not going to be > such > > a release. > > One problem with the change of semantics of 'effective final', if it > applies to inner classes, is that it introduces the possibility of > Java programs compiled with Java 1.7 to be incompatible with one > compiled for 1.6. Clearly, one cannot use pre-1.7 features (closures, > coin features) when compiling on a 1.6 compiler (or, using a -source > 1.6 flag), but it will mean that developers will be able to write: > > public class Outer { > public void method() { > boolean nonFinal = true; > new Inner { > boolean foo; > void other() { > foo = nonFinal; > } > } > } > } > > On a 1.7 compiler, this will compile without error. On a 1.6 compiler, > the lack of 'final' on the 'nonFinal' variable will cause a compile > time error. So it's still possible to be backwardly incompatible, even > if not using any of the 'new' features like lambdas, method > references, coin syntax improvements etc. > > Alex > From reinier at zwitserloot.com Sat Feb 27 14:25:49 2010 From: reinier at zwitserloot.com (Reinier Zwitserloot) Date: Sat, 27 Feb 2010 23:25:49 +0100 Subject: Effectively final effective? In-Reply-To: References: <4B85B844.9050507@adres.pl> <560fb5ed1002250857i42a63aa2u4d3c1cebc74762b@mail.gmail.com> <560fb5ed1002251326q6d12c59wa3ca36f496a620c9@mail.gmail.com> <212322091002251846k397efc92rc10e1462175cd524@mail.gmail.com> <7926817e1002251925md1f3686k2742d81a75c60d5f@mail.gmail.com> <560fb5ed1002260509l3f5e2d98td49a5d959c5e3df5@mail.gmail.com> Message-ID: <560fb5ed1002271425h2d15454foa7ebe827124f8faa@mail.gmail.com> John, you're making the argument that something is so natural it doesn't need explaining.... by explaining it. As long as you need to explain it, it's obviously not the "non-surprising, obvious" behaviour you think it is. --Reinier Zwitserloot On Fri, Feb 26, 2010 at 9:32 PM, John Nilsson wrote: > On Fri, Feb 26, 2010 at 2:09 PM, Reinier Zwitserloot < > reinier at zwitserloot.com> wrote: > >> 3. Variables accessed in closures will be silently copied, with the copy >> inside the closure being final, but the variable outside of it being >> mutable >> (but, as the closure gets a copy, any mutations don't show up in the >> closure). I've filed this away as crackpot, where it'll hopefully remain, >> as >> this is going to be very surprising to a lot of programmers and the only >> way >> this surprise is going to become obvious is after many hours spent >> debugging. >> > > So before this is filed away, let me try again and explain why this is not > surprising and in fact the obvious behaviour. > > The variable from the outside scope is "captured" at the time then the > lambda is created. In the same manner as any variable is carried into > another scope there and then. > > int a = 1; > int b = 0; > { > b = a; > } > a = 2; > //At this point b is expected to be 1, the value of a inside the block. It > is not expected that b has somehow been equated with the a variable just > because there is an equal sign implying that relation. > > In the same manner you expect > for(Integer i : asList(1,2,3)) > aList.add(i); > to produce a list containing 1,2,3 and not a list containing 3 (the last > value of i) > > So there is already a precedent for a semantic of capturing a value, not a > variable. > > > Now in ordinary blocks we are free to mutate variables from the outside > scope inside the inner scope. However as these blocks are exectued in the > same order as they are declared this is not a problem. With lambdas and > inner classes the definition is decoupled from the execution, which means > that the outer scope might not even exist at the time of execution. In > previous versions of Java this was solved only allowing final variables > inside inner classes. As they can't be mutated in either scope there is no > mutation order to care about. > > So by continuing this line of reasoning, we simply continue to view them as > immutable inside the inner class/lambda but relax the requirement that they > are mutable in the defining scope. Thinking that it is the value that is > captured, not the variable. > > BR, > John > From alex.blewitt at gmail.com Sat Feb 27 14:35:11 2010 From: alex.blewitt at gmail.com (Alex Blewitt) Date: Sat, 27 Feb 2010 22:35:11 +0000 Subject: Effectively final effective? In-Reply-To: <560fb5ed1002271424u2240e222hacb1713bf92e8c60@mail.gmail.com> References: <4B85B844.9050507@adres.pl> <560fb5ed1002250857i42a63aa2u4d3c1cebc74762b@mail.gmail.com> <560fb5ed1002251326q6d12c59wa3ca36f496a620c9@mail.gmail.com> <212322091002251846k397efc92rc10e1462175cd524@mail.gmail.com> <7926817e1002251925md1f3686k2742d81a75c60d5f@mail.gmail.com> <560fb5ed1002260509l3f5e2d98td49a5d959c5e3df5@mail.gmail.com> <636fd28e1002261146p36910745n4d424a3145535181@mail.gmail.com> <560fb5ed1002271424u2240e222hacb1713bf92e8c60@mail.gmail.com> Message-ID: On 27 Feb 2010, at 22:24, Reinier Zwitserloot wrote: > I don't understand. There is no source incompatibility. > > That is, it is perfectly allright for a new language feature to make previously illegal code, legal. For example, this: > > for (String x : foo) {} > > was illegal in java 1.4. It was legal in 1.5. That would have been syntactically incorrect in Java 1.4. However, it is not a syntax error to omit a final keyword from a variable captured by an inner class, but a semantic one. The point is that effectively final may result in code being developed outside of new language (syntax) features be semantically incorrect on earlier versions. One could argue that the same is also true of e.g. String.split or other new features (which then fail to compile on older versions of code), so it's in this class of error. However, I wanted to raise the observation that the effective finalness, as applied to inner classes, may result in code which is compatible with 1.7 but not 1.6, even if it doesn't touch any new syntax features like the one you highlighted. Alex From john at milsson.nu Sat Feb 27 15:25:06 2010 From: john at milsson.nu (John Nilsson) Date: Sun, 28 Feb 2010 00:25:06 +0100 Subject: Effectively final effective? In-Reply-To: <560fb5ed1002271425h2d15454foa7ebe827124f8faa@mail.gmail.com> References: <560fb5ed1002250857i42a63aa2u4d3c1cebc74762b@mail.gmail.com> <560fb5ed1002251326q6d12c59wa3ca36f496a620c9@mail.gmail.com> <212322091002251846k397efc92rc10e1462175cd524@mail.gmail.com> <7926817e1002251925md1f3686k2742d81a75c60d5f@mail.gmail.com> <560fb5ed1002260509l3f5e2d98td49a5d959c5e3df5@mail.gmail.com> <560fb5ed1002271425h2d15454foa7ebe827124f8faa@mail.gmail.com> Message-ID: On Sat, Feb 27, 2010 at 11:25 PM, Reinier Zwitserloot < reinier at zwitserloot.com> wrote: > John, you're making the argument that something is so natural it doesn't > need explaining.... by explaining it. > > As long as you need to explain it, it's obviously not the "non-surprising, > obvious" behaviour you think it is. > OTOH the fact that you have a different mental model than I do not make it a "crackpot" argument. It is simply different, and the only way to know for sure is to make some kind of survey. I suspect that my mental model of the semantics will prove to be more common than yours. But right now I have no other proof than pointing out examples of how current Java semantics already match my mental model. BR, John From john at milsson.nu Sat Feb 27 15:36:35 2010 From: john at milsson.nu (John Nilsson) Date: Sun, 28 Feb 2010 00:36:35 +0100 Subject: Effectively final effective? In-Reply-To: <560fb5ed1002260509l3f5e2d98td49a5d959c5e3df5@mail.gmail.com> References: <4B85B090.9090706@optrak.co.uk> <4B85B844.9050507@adres.pl> <560fb5ed1002250857i42a63aa2u4d3c1cebc74762b@mail.gmail.com> <560fb5ed1002251326q6d12c59wa3ca36f496a620c9@mail.gmail.com> <212322091002251846k397efc92rc10e1462175cd524@mail.gmail.com> <7926817e1002251925md1f3686k2742d81a75c60d5f@mail.gmail.com> <560fb5ed1002260509l3f5e2d98td49a5d959c5e3df5@mail.gmail.com> Message-ID: Am I correct in interpreting the 4 sentiments listed below like so: Given this code List<()->int> funs = new ArrayList<>(); for(int i = 0; i < 3; i++) funs.add(()->i) for(()->i f : funs) System.out.print(f.()); The following is the result 1. Prints: 333 2. Compiler error at i++ (mutating a final variable) 3. Prints: 123 4. Compiler error at ()->i (trying to access a non-final var) BR, John On Fri, Feb 26, 2010 at 2:09 PM, Reinier Zwitserloot < reinier at zwitserloot.com> wrote: > There are so many different sentiments floating around that its hard to > tell > the forest from the trees, but as far as I understand it, these are the > three sentiments I've heard: > > 1. Variables accessed in closures should be silently moved into the heap. > I > haven't yet heard anything from this camp about what to do with "volatile", > which would be a useful construct to them have on such local variables. > This > is somewhat tricky when the closure is run in another thread. > > 2. Variables accessed in closures will be made implicitly final. Any > attempt to mutate them, either in the closure or outside of it, generates a > compiler error. This camp is furthermore split into two: One camp wants a > keyword of some sort to get #1 behaviour (so, _explicit_ movement to the > heap), the other says that AtomicReference, AtomicInteger, and friends are > good enough. > > 3. Variables accessed in closures will be silently copied, with the copy > inside the closure being final, but the variable outside of it being > mutable > (but, as the closure gets a copy, any mutations don't show up in the > closure). I've filed this away as crackpot, where it'll hopefully remain, > as > this is going to be very surprising to a lot of programmers and the only > way > this surprise is going to become obvious is after many hours spent > debugging. > > 4. As with anonymous inner classes, variables cannot be accessed at all > inside closures unless they are marked explicitly final. > > > I'd say that with no distinction between safe and unsafe closures, Only > option #2 (either variant), with option #4 as a very distant second, is > sensible, and with safe/unsafe closures, Option #1 for unsafe closures, and > either #2 or #4 for safe closures, seems reasonable. Exactly how to proceed > is no clear to me, as this discussion seems to be heading into a painting > the bike shed direction. > > > NB: Jesse, regardless of the pros and cons of making a backwards > incompatible java 2.0 release, Java 7 most assuredly is not going to be > such > a release, so whatever ideas you come up with, if they aren't backwards > compatible, the only way its going to happen is for closures to not make it > into Java7, and for Java8 to be a Java 2.0 release, which even then is very > very unlikely. Mark Reinhold explicitly mentioned during Devoxx that he > considers any attempt to make Java 2.0 to be rife with second version > syndrome. > > > > On Thursday, February 25, 2010, Paulo Levi wrote: > > > Final should have been the default, non null should have been in the > > > language (and default), constructors should have not been allowed to > call > > > protected methods, closures should have come on day one, reification > > should > > > have been included in generics, i miss map and reduce on arrays & > lists, > > i > > > like list generators and RIIA and would love it in java. > > > > > > The discussion is if it is worthwhile to allow final by default for > > captured > > > variables in lambdas considering that > > > a) anonymous classes have the opposite behavior, so things get slightly > > > inconsistent. > > > b) it is safer for threaded use. > > > > > > > > > > > > From john at milsson.nu Sat Feb 27 15:37:26 2010 From: john at milsson.nu (John Nilsson) Date: Sun, 28 Feb 2010 00:37:26 +0100 Subject: Effectively final effective? In-Reply-To: References: <4B85B844.9050507@adres.pl> <560fb5ed1002250857i42a63aa2u4d3c1cebc74762b@mail.gmail.com> <560fb5ed1002251326q6d12c59wa3ca36f496a620c9@mail.gmail.com> <212322091002251846k397efc92rc10e1462175cd524@mail.gmail.com> <7926817e1002251925md1f3686k2742d81a75c60d5f@mail.gmail.com> <560fb5ed1002260509l3f5e2d98td49a5d959c5e3df5@mail.gmail.com> Message-ID: Please ignore the off my one error... imagine i wrote 222 and 012 respectively... On Sun, Feb 28, 2010 at 12:36 AM, John Nilsson wrote: > Am I correct in interpreting the 4 sentiments listed below like so: > > Given this code > > List<()->int> funs = new ArrayList<>(); > for(int i = 0; i < 3; i++) > funs.add(()->i) > for(()->i f : funs) > System.out.print(f.()); > > The following is the result > > 1. Prints: 333 > 2. Compiler error at i++ (mutating a final variable) > 3. Prints: 123 > 4. Compiler error at ()->i (trying to access a non-final var) > > BR, > John > > On Fri, Feb 26, 2010 at 2:09 PM, Reinier Zwitserloot < > reinier at zwitserloot.com> wrote: > >> There are so many different sentiments floating around that its hard to >> tell >> the forest from the trees, but as far as I understand it, these are the >> three sentiments I've heard: >> >> 1. Variables accessed in closures should be silently moved into the heap. >> I >> haven't yet heard anything from this camp about what to do with >> "volatile", >> which would be a useful construct to them have on such local variables. >> This >> is somewhat tricky when the closure is run in another thread. >> >> 2. Variables accessed in closures will be made implicitly final. Any >> attempt to mutate them, either in the closure or outside of it, generates >> a >> compiler error. This camp is furthermore split into two: One camp wants a >> keyword of some sort to get #1 behaviour (so, _explicit_ movement to the >> heap), the other says that AtomicReference, AtomicInteger, and friends are >> good enough. >> >> 3. Variables accessed in closures will be silently copied, with the copy >> inside the closure being final, but the variable outside of it being >> mutable >> (but, as the closure gets a copy, any mutations don't show up in the >> closure). I've filed this away as crackpot, where it'll hopefully remain, >> as >> this is going to be very surprising to a lot of programmers and the only >> way >> this surprise is going to become obvious is after many hours spent >> debugging. >> >> 4. As with anonymous inner classes, variables cannot be accessed at all >> inside closures unless they are marked explicitly final. >> >> >> I'd say that with no distinction between safe and unsafe closures, Only >> option #2 (either variant), with option #4 as a very distant second, is >> sensible, and with safe/unsafe closures, Option #1 for unsafe closures, >> and >> either #2 or #4 for safe closures, seems reasonable. Exactly how to >> proceed >> is no clear to me, as this discussion seems to be heading into a painting >> the bike shed direction. >> >> >> NB: Jesse, regardless of the pros and cons of making a backwards >> incompatible java 2.0 release, Java 7 most assuredly is not going to be >> such >> a release, so whatever ideas you come up with, if they aren't backwards >> compatible, the only way its going to happen is for closures to not make >> it >> into Java7, and for Java8 to be a Java 2.0 release, which even then is >> very >> very unlikely. Mark Reinhold explicitly mentioned during Devoxx that he >> considers any attempt to make Java 2.0 to be rife with second version >> syndrome. >> >> >> > On Thursday, February 25, 2010, Paulo Levi wrote: >> > > Final should have been the default, non null should have been in the >> > > language (and default), constructors should have not been allowed to >> call >> > > protected methods, closures should have come on day one, reification >> > should >> > > have been included in generics, i miss map and reduce on arrays & >> lists, >> > i >> > > like list generators and RIIA and would love it in java. >> > > >> > > The discussion is if it is worthwhile to allow final by default for >> > captured >> > > variables in lambdas considering that >> > > a) anonymous classes have the opposite behavior, so things get >> slightly >> > > inconsistent. >> > > b) it is safer for threaded use. >> > > >> > > >> > >> > >> >> > From reinier at zwitserloot.com Sat Feb 27 16:10:39 2010 From: reinier at zwitserloot.com (Reinier Zwitserloot) Date: Sun, 28 Feb 2010 01:10:39 +0100 Subject: Effectively final effective? In-Reply-To: References: <560fb5ed1002251326q6d12c59wa3ca36f496a620c9@mail.gmail.com> <212322091002251846k397efc92rc10e1462175cd524@mail.gmail.com> <7926817e1002251925md1f3686k2742d81a75c60d5f@mail.gmail.com> <560fb5ed1002260509l3f5e2d98td49a5d959c5e3df5@mail.gmail.com> <560fb5ed1002271425h2d15454foa7ebe827124f8faa@mail.gmail.com> Message-ID: <560fb5ed1002271610w3f8f22edw6c9b61e96ca556d7@mail.gmail.com> Let's posit for a moment that your model IS a more common viewpoint (something I doubt): So what? I already showed that those people who are confused about my model will see the error of their ways in 2 seconds. The people who are confused about your model, even if there are only a tenth of them (I highly doubt this), need an hour. 60 minutes is a lot more than 2 seconds * 10. I don't know if your interpretation of the four cases is correct; you're using the Neal syntax which I haven't reviewed. Guessing at intent and looking at the output (and correcting for the off-by-1), I believe so. NB: Without hashes as a visual anchor, this is really hard to read. I'm not sure Neal syntax is a good idea, even if _by themselves_ closure types may read more naturally. --Reinier Zwitserloot On Sun, Feb 28, 2010 at 12:25 AM, John Nilsson wrote: > On Sat, Feb 27, 2010 at 11:25 PM, Reinier Zwitserloot < > reinier at zwitserloot.com> wrote: > >> John, you're making the argument that something is so natural it doesn't >> need explaining.... by explaining it. >> >> As long as you need to explain it, it's obviously not the "non-surprising, >> obvious" behaviour you think it is. >> > OTOH the fact that you have a different mental model than I do not make it > a "crackpot" argument. It is simply different, and the only way to know for > sure is to make some kind of survey. I suspect that my mental model of the > semantics will prove to be more common than yours. But right now I have no > other proof than pointing out examples of how current Java semantics already > match my mental model. > > BR, > John > From john at milsson.nu Sat Feb 27 16:32:04 2010 From: john at milsson.nu (John Nilsson) Date: Sun, 28 Feb 2010 01:32:04 +0100 Subject: Effectively final effective? In-Reply-To: <560fb5ed1002271610w3f8f22edw6c9b61e96ca556d7@mail.gmail.com> References: <560fb5ed1002251326q6d12c59wa3ca36f496a620c9@mail.gmail.com> <212322091002251846k397efc92rc10e1462175cd524@mail.gmail.com> <7926817e1002251925md1f3686k2742d81a75c60d5f@mail.gmail.com> <560fb5ed1002260509l3f5e2d98td49a5d959c5e3df5@mail.gmail.com> <560fb5ed1002271425h2d15454foa7ebe827124f8faa@mail.gmail.com> <560fb5ed1002271610w3f8f22edw6c9b61e96ca556d7@mail.gmail.com> Message-ID: On Sun, Feb 28, 2010 at 1:10 AM, Reinier Zwitserloot < reinier at zwitserloot.com> wrote: > So what? I already showed that those people who are confused about my model > will see the error of their ways in 2 seconds. The people who are confused > about your model, even if there are only a tenth of them (I highly doubt > this), need an hour. 60 minutes is a lot more than 2 seconds * 10. > Looking back at the discussion I realize a mistake on my part. When arguing against my suggestions I assumed you argued for the print 222 variant. I now realize that you actually prefer a compiler error. Given the four suggestions I would say that it is more important to avoid the 222 variant then it is to support the 123 variant I was arguing for. With reference to the same argument you made: it leads to surprising results. So if the option is to throw a compiler error at the developer and force them to write an explicit workaround, I guess I actually agree. So then the discussion is rather about what compiler error it is that forces you to write for(int i = 0; i < 3; i++) { final int currI = i; funs.add(()->currI) } ? BR, John From peter.levart at gmail.com Sun Feb 28 01:32:50 2010 From: peter.levart at gmail.com (Peter Levart) Date: Sun, 28 Feb 2010 10:32:50 +0100 Subject: Effectively final effective? In-Reply-To: References: Message-ID: <201002281032.51318.peter.levart@gmail.com> You meant 333 vs. 012 didn't you? Peter On Sunday 28 February 2010 12:37:26 am John Nilsson wrote: > Please ignore the off my one error... imagine i wrote 222 and 012 > respectively... > > On Sun, Feb 28, 2010 at 12:36 AM, John Nilsson wrote: > > Am I correct in interpreting the 4 sentiments listed below like so: > > > > Given this code > > > > List<()->int> funs = new ArrayList<>(); > > for(int i = 0; i < 3; i++) > > > > funs.add(()->i) > > > > for(()->i f : funs) > > > > System.out.print(f.()); > > > > The following is the result > > > > 1. Prints: 333 > > 2. Compiler error at i++ (mutating a final variable) > > 3. Prints: 123 > > 4. Compiler error at ()->i (trying to access a non-final var) > > > > BR, > > John From fredrik.ohrstrom at oracle.com Sun Feb 28 01:39:36 2010 From: fredrik.ohrstrom at oracle.com (=?UTF-8?B?RnJlZHJpayDDlmhyc3Ryw7Zt?=) Date: Sun, 28 Feb 2010 10:39:36 +0100 Subject: Method References In-Reply-To: <560fb5ed1002271412n4a729d33h8a7d29896e7d9525@mail.gmail.com> References: <15e8b9d21002252252t45faaa9fq6859222b31a6052f@mail.gmail.com> <4B878DE6.4070100@oracle.com> <4B87A632.4010100@univ-mlv.fr> <15e8b9d21002260815y6c48f7a9n2a70ca5e0352274d@mail.gmail.com> <17b2302a1002260840u3318d99ep9561e6fd1963fd7a@mail.gmail.com> <560fb5ed1002271412n4a729d33h8a7d29896e7d9525@mail.gmail.com> Message-ID: <4B8A39D8.7000500@oracle.com> Reinier Zwitserloot wrote: > Is this really necessary? If the signature of saveState is: > > void saveState() { > ... > } > > then instead of adding syntax so you can write: > > #void() saveStateRef = this#saveState; > > you can just use the existing closures status quo: > > #void() saveStateRef = #() {saveState();}; > > Is this closure syntax really necessary? You can just use the existing anon.inner.class status quo: Callable saveStateRef = new Callable () { public Void call() { saveState(); return null;}}; **joke-end** Yes, method references are necessary since their use case is extremely common. The most common use case should have the shortest syntax. Also the method ref syntax avoids the risk of binding locals. MethodHandles are in effect method references. Assuming MHs are used to construct closures then, then the building block for closures are method references, not the other way round. Thus the this#saveState will be compiled into the following bytecode: ldc MyApp.saveState()V aload 0 invokevirtual MethodHandle.bindTo(LObject;)LMethodHandle; "ldc non-static-methodref" will create a virtual MethodHandle (aka stored message) with the signature invokeGeneric(MyApp)V when we bind this MethodHandle to an instance (ie this, stored in local var 0) the signature changes to: invokeGeneric()V Neat, eh? //Fredrik From peter.levart at gmail.com Sun Feb 28 02:29:41 2010 From: peter.levart at gmail.com (Peter Levart) Date: Sun, 28 Feb 2010 11:29:41 +0100 Subject: Effectively final effective? In-Reply-To: References: <560fb5ed1002260509l3f5e2d98td49a5d959c5e3df5@mail.gmail.com> Message-ID: <201002281129.41140.peter.levart@gmail.com> The effectively final case of basic for-loop index variables is interesting in itself. Usually we think of a particular basic for-loop construct: for ( LocalVariableDeclaration ; Expression ; ForUpdate ) Statement as a shorthand for: { LocalVariableDeclaration ; while ( Expression ) { Statement ForUpdate ; } } So if Expression or Statement or ForUpdate writes into local index variable then it is clearly not effectively final. Regardless of whether capturing non-final variables is allowed or not, capturing non-final for-loop index variables (by reference of course) is rarely a semantics that is desired so it should be marked with a warning. In that case you would have to rewrite you example either as: for(int i = 0; i < 3; i++) { final int ii = i; funs.add(()->ii); } or: for(int i = 0; i < 3; i++) { @SuppressWarnings("for loop index capture") funs.add(()->i); } Depending whether you want your program to print 012 or 333 (respectively). That's what I think is a good enough solution to this problem. Regards, Peter On Sunday 28 February 2010 12:36:35 am John Nilsson wrote: > Am I correct in interpreting the 4 sentiments listed below like so: > > Given this code > > List<()->int> funs = new ArrayList<>(); > for(int i = 0; i < 3; i++) > funs.add(()->i) > for(()->i f : funs) > System.out.print(f.()); > > The following is the result > > 1. Prints: 333 > 2. Compiler error at i++ (mutating a final variable) > 3. Prints: 123 > 4. Compiler error at ()->i (trying to access a non-final var) > > BR, > John From i30817 at gmail.com Sun Feb 28 06:40:32 2010 From: i30817 at gmail.com (Paulo Levi) Date: Sun, 28 Feb 2010 14:40:32 +0000 Subject: Effectively final effective? In-Reply-To: <201002281129.41140.peter.levart@gmail.com> References: <560fb5ed1002260509l3f5e2d98td49a5d959c5e3df5@mail.gmail.com> <201002281129.41140.peter.levart@gmail.com> Message-ID: <212322091002280640sd01f21h59c8091bd4728bea@mail.gmail.com> Can't think of a situation where i would want to calculate the same computation many times with the last value. Maybe if the computation is lazy and there is a non-pure source of input like a inputstream. From i30817 at gmail.com Sun Feb 28 06:43:32 2010 From: i30817 at gmail.com (Paulo Levi) Date: Sun, 28 Feb 2010 14:43:32 +0000 Subject: Method References In-Reply-To: <4B8A39D8.7000500@oracle.com> References: <15e8b9d21002252252t45faaa9fq6859222b31a6052f@mail.gmail.com> <4B878DE6.4070100@oracle.com> <4B87A632.4010100@univ-mlv.fr> <15e8b9d21002260815y6c48f7a9n2a70ca5e0352274d@mail.gmail.com> <17b2302a1002260840u3318d99ep9561e6fd1963fd7a@mail.gmail.com> <560fb5ed1002271412n4a729d33h8a7d29896e7d9525@mail.gmail.com> <4B8A39D8.7000500@oracle.com> Message-ID: <212322091002280643p60d4562dsbedadd727f9f2966@mail.gmail.com> What about local variables mutated inside of methods? The discussion of capturing local variables applies right? From john at milsson.nu Sun Feb 28 06:59:37 2010 From: john at milsson.nu (John Nilsson) Date: Sun, 28 Feb 2010 15:59:37 +0100 Subject: Effectively final effective? In-Reply-To: <201002281032.51318.peter.levart@gmail.com> References: <201002281032.51318.peter.levart@gmail.com> Message-ID: On Sun, Feb 28, 2010 at 10:32 AM, Peter Levart wrote: > You meant 333 vs. 012 didn't you? > Hahaha. I think I'll just keep away from indexes for a while ;) BR, John From john at milsson.nu Sun Feb 28 07:15:25 2010 From: john at milsson.nu (John Nilsson) Date: Sun, 28 Feb 2010 16:15:25 +0100 Subject: Effectively final effective? In-Reply-To: <201002281129.41140.peter.levart@gmail.com> References: <560fb5ed1002260509l3f5e2d98td49a5d959c5e3df5@mail.gmail.com> <201002281129.41140.peter.levart@gmail.com> Message-ID: On Sun, Feb 28, 2010 at 11:29 AM, Peter Levart wrote: > for(int i = 0; i < 3; i++) { > > final int ii = i; > > funs.add(()->ii); > > } > > or: > > for(int i = 0; i < 3; i++) { > > @SuppressWarnings("for loop index capture") > > funs.add(()->i); > > } > > Depending whether you want your program to print 012 or 333 (respectively). > > That's what I think is a good enough solution to this problem. > > Regards, Peter > Then someone comes along and "refactors" the code from for(int i = 1; i <= 3; i++) //Can I get this right? funs.add(()->i); to for(int i : in(1,2,3)) fins.add(()->i); Accidentally breaking indexing semantics. BR, John From john at milsson.nu Sun Feb 28 07:37:49 2010 From: john at milsson.nu (John Nilsson) Date: Sun, 28 Feb 2010 16:37:49 +0100 Subject: Call chains and method handles Message-ID: It was suggested in some thread of conversation that method references was going to be the most common use case for lambdas. I was thinking of whether this was true for our code base (a 200kloc ERP web application with a swing-like html-framework). As the code is written now I don't think there would be much use for direct method references. The closest thing is a report API that uses strings to compile call chains against a collection of objects. Report r = report(Customer.class,"name","address.street") ... searchButton.onClick(()-> r.show(customer.findByCustomerNo(customerNoField.getText())) I would love to be able to replace those strings with statically verified call chains. As it is now it's too fragile when refactoring. So when Java7 comes we'll probably rewrite that to Report r = report(Customer.class,(Customer c)->c.getName(),(Customer c)->c.getAddress().getStreet()) But now we've actually added boilerplate. MethodRererences could maybe be used to make it Report r = report(Customer.class,cc(Customer#getName),cc(Customer#getAddress,Address#getStreet)) Still lots of boilerplate, and cc wouldn't be type safe, which could be changed by adding some more boiler of course: cc(...).andThen(...) Any thoughts no how this could be improved? Should it be? Ideally I would like to write something no more verbose than Report r = report(Customer.class,#getName,#getAddress#getStreet) BR, John From peter.levart at gmail.com Sun Feb 28 10:35:54 2010 From: peter.levart at gmail.com (Peter Levart) Date: Sun, 28 Feb 2010 19:35:54 +0100 Subject: Effectively final effective? In-Reply-To: <212322091002280640sd01f21h59c8091bd4728bea@mail.gmail.com> References: <201002281129.41140.peter.levart@gmail.com> <212322091002280640sd01f21h59c8091bd4728bea@mail.gmail.com> Message-ID: <201002281935.54633.peter.levart@gmail.com> On Sunday 28 February 2010 03:40:32 pm Paulo Levi wrote: > Can't think of a situation where i would want to calculate the same > computation many times with the last value. Maybe you would want to adjust (increment) current index from within lambda based on some computation. Very ugly I know. Peter From i30817 at gmail.com Sun Feb 28 10:48:26 2010 From: i30817 at gmail.com (Paulo Levi) Date: Sun, 28 Feb 2010 18:48:26 +0000 Subject: Effectively final effective? In-Reply-To: <201002281935.54633.peter.levart@gmail.com> References: <201002281129.41140.peter.levart@gmail.com> <212322091002280640sd01f21h59c8091bd4728bea@mail.gmail.com> <201002281935.54633.peter.levart@gmail.com> Message-ID: <212322091002281048r158bcf96v58f3dce425818184@mail.gmail.com> For numerical values i would just multiply the computation once. That is what multiplication is for, pun intended. For non-pure computations... seems crazy. From peter.levart at gmail.com Sun Feb 28 10:49:38 2010 From: peter.levart at gmail.com (Peter Levart) Date: Sun, 28 Feb 2010 19:49:38 +0100 Subject: Effectively final effective? In-Reply-To: References: <201002281129.41140.peter.levart@gmail.com> Message-ID: <201002281949.38889.peter.levart@gmail.com> On Sunday 28 February 2010 04:15:25 pm John Nilsson wrote: > On Sun, Feb 28, 2010 at 11:29 AM, Peter Levart wrote: > > for(int i = 0; i < 3; i++) { > > > > final int ii = i; > > > > funs.add(()->ii); > > > > } > > > > or: > > > > for(int i = 0; i < 3; i++) { > > > > @SuppressWarnings("for loop index capture") > > > > funs.add(()->i); > > > > } > > > > Depending whether you want your program to print 012 or 333 > > (respectively). > > > > That's what I think is a good enough solution to this problem. > > > > Regards, Peter > > Then someone comes along and "refactors" the code from > > for(int i = 1; i <= 3; i++) //Can I get this right? > funs.add(()->i); > > to > > for(int i : in(1,2,3)) > fins.add(()->i); > > Accidentally breaking indexing semantics. Yeah, I see. Then maybe it is best to just disallow basic for loop index capture. Enhanced for loop variable is another story and it is usually effectively final anyway. Peter > > > BR, > John From peter.levart at gmail.com Sun Feb 28 11:44:08 2010 From: peter.levart at gmail.com (Peter Levart) Date: Sun, 28 Feb 2010 20:44:08 +0100 Subject: Accessing non-final local variables from a lambda expression In-Reply-To: <560fb5ed1002271405j63a3db17s8809eecc2ce1ae2c@mail.gmail.com> References: <15e8b9d21002252336o2f7280d2v157a9acd38ab29da@mail.gmail.com> <201002261701.24155.peter.levart@marand.si> <560fb5ed1002271405j63a3db17s8809eecc2ce1ae2c@mail.gmail.com> Message-ID: <201002282044.09069.peter.levart@marand.si> On Saturday 27 February 2010 11:05:56 pm Reinier Zwitserloot wrote: > On Fri, Feb 26, 2010 at 5:01 PM, Peter Levart wrote: > > The problem with this approach (from the performance perspective) is when > > you capture several mutable local variables. Having a separate wrapper > > object for each of them is an overhead which can be avoided if the > > compiler constructs a single frame object for you. > > That's entirely correct, but doing this is, if I understand correctly, very > complicated from the perspective of the java memory model. Also, without a > very specific sort of unsafe/safe closures I don't see how this can be > done: > > If a closure lives on beyond the lifetime of its parent frame, then > obviously the variable cannot _be_ in the parent frame. These variables > would have to be hosted in a separate frame that lives on until the method > and ALL closures in it (or at least all the ones that access this variable) > have been GCed. Right now frames don't have a dependency on the GC so I > don't see how this can be done in a way that is more performant than a heap > allocation. > > Also, if a closure is transported to a different thread, the same problem > occurs, even if the original method is still 'live'. Frames, as I > understand the JVM, aren't shared between threads. > > Hence I'm assuming in all these discussions about syntax that it boils down > to some sugar that hosts the variable on the heap, via e.g. an intermediate > array, or perhaps a custom class to do it, which makes it easier for the > JVM to recognize this case and eventually perhaps optimize for it. With "frame object" I meant an object, allocated on the heap whose class is constructed by compiler and holds all captured local variables (whether final or non-final). I don't know why but this terminology is used by Neal and others. With MethodHandles, local vars can be captured one-by-one via (MethodHandles.insertArgument) but it might still be better to construct a single "frame object" to hold all of them when their number reaches a certain limit or when at least one of them is non-final. > > > There are perfectly valid use-cases (like the one above) where accessing > > a mutable local variable is safe. For such cases it should not be > > necessary to use @Shared annotation on a variable to prevent a warning. > > The only way for a mutable local variable to escape the method is via > > capture from a closure whose reference escapes the method. So a better > > approach would be to annotate function type (and SAM type) variables > > (instance/static/local/parameters) that hold references to closures, not > > mutable local variables accessed from closures. > > In interesting idea, but I don't think the type system is the right place. > For example, TreeSet takes a Comparator, but this Comparator is *NOT* > transparency-safe; after all, the TreeSet keeps a reference to it and will > invoke it each time an item is added to the TreeSet, which may even occur > in another thread. It is the idea of JSR308 (Type annotations) that you can attach an annotation to any type. These annotations are treated by compiler as additional properties of the type (if types are nouns, type annotations are adjectives). A type with some annotation can be treated by the compiler as a sub-type or a super-type of the type without the annotation. In case of @Safe it would be a sub-type. At run-time these type attributes are of course erased, but they are present in class files for compiler to check them at compile time. JSR308 has other tricks that could be used here. If we had another annotation, let's say @S that would be some kind of generic variable that could be attached to any type or class/interface declaration. For example: @S public interface SortedSet { ... @S Comparator comparator(); @S acts as a sort of aditional SortedSet's generic parameter. A @Safe SortedSet is then an instance of a SortedSet that contains a @Safe Comparator. @Safe-transparent methods are also possible: @S (int)->int pow2(@S (int)->int func) { return (int x)->(func.(func.(x))); } @Safe (int)->int doubler = (int x)->(x+x); @Safe (int)->int quadrupler = pow2(doubler); In short, JSR308 is full of good ideas that could be used for this purpose. The question is whether any such scheme, if implemented, would be part of JLS. I haven't found any reference to @Override annotation in JLS 3 for example. How to prescribe "warning" behavior without specifying it in the JLS? Regards, Peter From i30817 at gmail.com Sun Feb 28 11:51:44 2010 From: i30817 at gmail.com (Paulo Levi) Date: Sun, 28 Feb 2010 19:51:44 +0000 Subject: Accessing non-final local variables from a lambda expression In-Reply-To: <201002282044.09069.peter.levart@marand.si> References: <15e8b9d21002252336o2f7280d2v157a9acd38ab29da@mail.gmail.com> <201002261701.24155.peter.levart@marand.si> <560fb5ed1002271405j63a3db17s8809eecc2ce1ae2c@mail.gmail.com> <201002282044.09069.peter.levart@marand.si> Message-ID: <212322091002281151o7a747bd3y4b6856e840dfe0c0@mail.gmail.com> Seems like cheating. I recommend @Pure instead if you're going to do that. From i30817 at gmail.com Sun Feb 28 11:57:32 2010 From: i30817 at gmail.com (Paulo Levi) Date: Sun, 28 Feb 2010 19:57:32 +0000 Subject: Accessing non-final local variables from a lambda expression In-Reply-To: <212322091002281151o7a747bd3y4b6856e840dfe0c0@mail.gmail.com> References: <15e8b9d21002252336o2f7280d2v157a9acd38ab29da@mail.gmail.com> <201002261701.24155.peter.levart@marand.si> <560fb5ed1002271405j63a3db17s8809eecc2ce1ae2c@mail.gmail.com> <201002282044.09069.peter.levart@marand.si> <212322091002281151o7a747bd3y4b6856e840dfe0c0@mail.gmail.com> Message-ID: <212322091002281157j6e7da5d8v8cb1df3a4acaa407@mail.gmail.com> Besides would that catch the case when a pure function is turned into a "impure" function? Say @Pure public int add(int x){ return 2+x; } .... parallelMap(list, #add(int i)); later in refactoring: @Pure public c = 2; public int add(int x){ return c+x; } Can annotations hijack the type system to this level? From reinier at zwitserloot.com Sun Feb 28 12:17:13 2010 From: reinier at zwitserloot.com (Reinier Zwitserloot) Date: Sun, 28 Feb 2010 21:17:13 +0100 Subject: Accessing non-final local variables from a lambda expression In-Reply-To: <201002282044.09069.peter.levart@marand.si> References: <15e8b9d21002252336o2f7280d2v157a9acd38ab29da@mail.gmail.com> <201002261701.24155.peter.levart@marand.si> <560fb5ed1002271405j63a3db17s8809eecc2ce1ae2c@mail.gmail.com> <201002282044.09069.peter.levart@marand.si> Message-ID: <560fb5ed1002281217w22c56125se9ba31aaef6abcb9@mail.gmail.com> Replies inline. On Sun, Feb 28, 2010 at 8:44 PM, Peter Levart wrote: > With "frame object" I meant an object, allocated on the heap whose class > is constructed by compiler and holds all captured local variables (whether > final or non-final). I don't know why but this terminology is used by Neal > and others. > > With MethodHandles, local vars can be captured one-by-one via > (MethodHandles.insertArgument) but it might still be better to construct a > single "frame object" to hold all of them when their number reaches a > certain limit or when at least one of them is non-final. > So, there's a marginal speed benefit that grows with the number of (effectively) non-final local variables captured, where with capturing 1, the speed benefit is zero? I doubt this is going to be worth the effort to implement. Would it even be faster in the first place? any modification of a shared variable still needs to visit the heap to grab the shared object. Presumably if there are 3 variables all on the heap they'd be part of the same generation and most likely in the same block, which suggests that the odds of a cache miss after the first access is low either way. It is the idea of JSR308 (Type annotations) that you can attach an > annotation to any type. These annotations are treated by compiler as > additional properties of the type (if types are nouns, type annotations are > adjectives). > I don't agree with this notion, but, nevertheless, it remains a standing rule: Annotations cannot change what code does, it can at best change the number and nature of warnings emitted by the compiler. Unless the powers that be want to consider nixing this rule, I don't see how this approach is going to pass muster. > @S public interface SortedSet { > > ... > > @S Comparator comparator(); > > @S acts as a sort of aditional SortedSet's generic parameter. A @Safe > SortedSet is then an instance of a SortedSet that contains a @Safe > Comparator. > Huh? By annotating the interface, I would say the intent is that *ALL* SortedSets are safe. Which isn't necessarily the case, of course. From peter.levart at gmail.com Sun Feb 28 12:19:53 2010 From: peter.levart at gmail.com (Peter Levart) Date: Sun, 28 Feb 2010 21:19:53 +0100 Subject: Accessing non-final local variables from a lambda expression In-Reply-To: <212322091002281157j6e7da5d8v8cb1df3a4acaa407@mail.gmail.com> References: <15e8b9d21002252336o2f7280d2v157a9acd38ab29da@mail.gmail.com> <212322091002281151o7a747bd3y4b6856e840dfe0c0@mail.gmail.com> <212322091002281157j6e7da5d8v8cb1df3a4acaa407@mail.gmail.com> Message-ID: <201002282119.54034.peter.levart@gmail.com> Oh, I haven't got that far. And there's nothing in a method that could render it unsafe considering the criteria I suggested (only capture of non-final local vars by lambda). But I guess if you wanted to extend @Safe tracking to methods (and consequently to method references) compiler would use the same logic as for lambda bodies and attach a 'synthetic' annotation to a method if it determines that it is @Safe. The type of a Method reference would then inherit this annotation from the method the reference refers to. Peter On Sunday 28 February 2010 08:57:32 pm Paulo Levi wrote: > Besides would that catch the case when a pure function is turned into a > "impure" function? > > Say > > @Pure public int add(int x){ > return 2+x; > } > .... > parallelMap(list, #add(int i)); > > > later in refactoring: > @Pure public c = 2; > public int add(int x){ > return c+x; > } > Can annotations hijack the type system to this level? From fredrik.ohrstrom at oracle.com Sun Feb 28 13:16:21 2010 From: fredrik.ohrstrom at oracle.com (=?ISO-8859-1?Q?Fredrik_=D6hrstr=F6m?=) Date: Sun, 28 Feb 2010 22:16:21 +0100 Subject: Accessing non-final local variables from a lambda expression In-Reply-To: <201002282044.09069.peter.levart@marand.si> References: <15e8b9d21002252336o2f7280d2v157a9acd38ab29da@mail.gmail.com> <201002261701.24155.peter.levart@marand.si> <560fb5ed1002271405j63a3db17s8809eecc2ce1ae2c@mail.gmail.com> <201002282044.09069.peter.levart@marand.si> Message-ID: <4B8ADD25.10900@oracle.com> Peter Levart wrote: > With "frame object" I meant an object, allocated on the heap whose class is constructed by > compiler and holds all captured local variables (whether final or non-final). I don't know why but > this terminology is used by Neal and others. > > With MethodHandles, local vars can be captured one-by-one via (MethodHandles.insertArgument) but > it might still be better to construct a single "frame object" to hold all of them when their > number reaches a certain limit or when at least one of them is non-final. > Be aware that insertArgument might very well be implemented as a "closure" using a frame object. See the sample implementation in: http://tinyurl.com/q8v92o#InsertArgument and the appendArgument implementation at the end of http://tinyurl.com/pekyrc //Fredrik From reinier at zwitserloot.com Sun Feb 28 19:38:02 2010 From: reinier at zwitserloot.com (Reinier Zwitserloot) Date: Mon, 1 Mar 2010 04:38:02 +0100 Subject: Effectively final effective? In-Reply-To: <201002281949.38889.peter.levart@gmail.com> References: <201002281129.41140.peter.levart@gmail.com> <201002281949.38889.peter.levart@gmail.com> Message-ID: <560fb5ed1002281938h71df0e2buac930385b7ba911c@mail.gmail.com> Agreed; variable initializers in basic for loops can get special treatment if need be (cannot be captured in closures at all). They are somewhat unique beasts already in the spec. --Reinier Zwitserloot 2010/2/28 Peter Levart > On Sunday 28 February 2010 04:15:25 pm John Nilsson wrote: > > On Sun, Feb 28, 2010 at 11:29 AM, Peter Levart >wrote: > > > for(int i = 0; i < 3; i++) { > > > > > > final int ii = i; > > > > > > funs.add(()->ii); > > > > > > } > > > > > > or: > > > > > > for(int i = 0; i < 3; i++) { > > > > > > @SuppressWarnings("for loop index capture") > > > > > > funs.add(()->i); > > > > > > } > > > > > > Depending whether you want your program to print 012 or 333 > > > (respectively). > > > > > > That's what I think is a good enough solution to this problem. > > > > > > Regards, Peter > > > > Then someone comes along and "refactors" the code from > > > > for(int i = 1; i <= 3; i++) //Can I get this right? > > funs.add(()->i); > > > > to > > > > for(int i : in(1,2,3)) > > fins.add(()->i); > > > > Accidentally breaking indexing semantics. > > Yeah, I see. > > Then maybe it is best to just disallow basic for loop index capture. > Enhanced for loop variable > is another story and it is usually effectively final anyway. > > Peter > > > > > > > BR, > > John > From reinier at zwitserloot.com Sun Feb 28 19:47:28 2010 From: reinier at zwitserloot.com (Reinier Zwitserloot) Date: Mon, 1 Mar 2010 04:47:28 +0100 Subject: Method References In-Reply-To: <4B8A39D8.7000500@oracle.com> References: <15e8b9d21002252252t45faaa9fq6859222b31a6052f@mail.gmail.com> <4B878DE6.4070100@oracle.com> <4B87A632.4010100@univ-mlv.fr> <15e8b9d21002260815y6c48f7a9n2a70ca5e0352274d@mail.gmail.com> <17b2302a1002260840u3318d99ep9561e6fd1963fd7a@mail.gmail.com> <560fb5ed1002271412n4a729d33h8a7d29896e7d9525@mail.gmail.com> <4B8A39D8.7000500@oracle.com> Message-ID: <560fb5ed1002281947p7c2afd2na04ec4e8a1629759@mail.gmail.com> The suggested method reference syntax is 38 characters. My alternate form that uses the status quo closure syntax to wrap a method call into a closure takes 42 characters, which is 10% more. Your 'joke' with anonymous inner classes cannot reasonably be called properly indented, and is 108 characters long, or 184% more. I don't think your slippery slope argument holds water here. You say that method references is "the most common use case". Why do you think so? I doubt I'll use them very much, possibly never. Reinier Zwitserloot wrote: > >> Is this really necessary? If the signature of saveState is: >> >> void saveState() { >> ... >> } >> >> then instead of adding syntax so you can write: >> >> #void() saveStateRef = this#saveState; >> >> you can just use the existing closures status quo: >> >> #void() saveStateRef = #() {saveState();}; >> >> >> > Is this closure syntax really necessary? > You can just use the existing anon.inner.class status quo: > > Callable saveStateRef = new Callable () { > public Void call() { saveState(); return null;}}; > > **joke-end** > > Yes, method references are necessary since their use case is extremely > common. The most common use case should have the shortest syntax. > Also the method ref syntax avoids the risk of binding locals. > > MethodHandles are in effect method references. Assuming MHs are used > to construct closures then, then the building block for closures are method > references, not the other way round. > > Thus the this#saveState will be compiled into the following bytecode: > ldc MyApp.saveState()V > aload 0 > invokevirtual MethodHandle.bindTo(LObject;)LMethodHandle; > > "ldc non-static-methodref" will create a virtual MethodHandle (aka stored > message) with the signature > invokeGeneric(MyApp)V > when we bind this MethodHandle to an instance (ie this, stored in local var > 0) the signature changes to: > invokeGeneric()V > > Neat, eh? > > //Fredrik > >