capturing (or not) mutable local variables

[Mark Mahieu]

Ah, I must have skipped over option (1) somehow when I read your first email; my reply concerned option (2) only, sorry for any confusion.

Personally, I think effectively-final is an improvement, but could be nudged along a little further.  John Rose posted one interesting variation 2 or 3 weeks ago, IIRC.

Mark


On 20 Nov 2010, at 15:21, Mikael Grev wrote:

> I'm only concerned with the usability aspect (as usual) and it is actually simpler for the users the way I suggest.
> 
> The only thing the users have to concern themselves with is that changes to captured variables won't be visible outside the closure (this is actually same for the current effectually-final as well).
> 
> With this loosening of effectively-final this is the only thing the user has to learn. With the current spec the user also has to understand the concept of effectively-final as well as a work around for my code below, which will always be a hack (a new final variable or an array-of-one).
> 
> What differs between 1) and 2) below is whether the user can change the variable after the closure initialization or not. Therefore I would recommend 1) as that is the simplest concept, especially since Java is pass-by-value for methods already.
> 
> So, basically we move complexity from the user (millions) to the compiler write (a handful) which is a good move in my book.
> 
> Effectively-final the way it is done now has a smell of hack to it and won't look good i a few years, much as overprotective language concepts of yesteryear don't look good today.
> 
> Cheers,
> Mikael
> 
> On Nov 20, 2010, at 12:43 PM, Mark Mahieu wrote:
> 
>> I have a bit of a soft spot for this idea, in fact I think I posted something similar a few months ago.
>> 
>> It's quite appealing for certain examples, but I do wonder whether it would mainly just contribute to number of rules a programmer has to deal with in this area of the language.
>> 
>> I think I concluded that this kind of 'loosening' would work better if it were more explicit, perhaps still requiring the programmer to mark the variable final, but by allowing the same variable to be 'redeclared' final rather than having to invent a new one (that's the ugly bit).  I might be wandering off into the weeds now though...
>> 
>> Mark
>> 
>> 
>> On 20 Nov 2010, at 10:37, Mikael Grev wrote:
>> 
>>> Please don't make it possible to mutate the variable from within the closure.
>>> 
>>> 
>>> However, I wouldn't mind if "effectively-final" was extended so that either:
>>> 
>>> 1) The closure works on a copy, giving no restrictions on the initialization and reassignment of the captured variable
>>> 2) Any variable is effectively-final as long as it isn't reassigned after the closure initialization. (reordering should be solvable)
>>> 
>>> The reason for this is that the following isn't that uncommon and "i" needs to be reassigned as another variable to be effectively-final (ugly):
>>> 
>>> int i = 0;
>>> if (something)
>>> 	i = xxx;
>>> 
>>> capture of i
>>> 
>>> Loosening the definition of effectively-final would effectively make the feature easier to use.
>>> 
>>> Cheers,
>>> Mikael
>>> 
>>> 
>>> On Nov 20, 2010, at 10:29 AM, Brian Goetz wrote:
>>> 
>>>>> Hi Brian, I came to you yesterday at devoxx to ask about the rationale
>>>>> behind the decision to not capture mutable local variables.
>>>> 
>>>> I admire your passion and your persistence :)
>>>> 
>>>>> I understand that among several arguments, there is a strong one about
>>>>> preventing people from doing unsafe things. you gave the analogy of
>>>>> the kid, the fence and the pool.
>>>> 
>>>> Safety mechanisms do not need to be perfect to be useful.  Your arguments seem to center around "I can climb the fence, so there's no point in having it."  This argument is, to be blunt, silly.  The design principles of the Java language stress safety, even if there are cases where such safety guards can be "casted away" by a sufficiently motivated user.  
>>>> 
>>>> If you want to capture a not-effectively-final local variable in a closure, then one of the following two things has to happen:
>>>> a) The lifetime of the variable must be extended to the lifetime of the closure
>>>> b) The lifetime of the closure must be shortened to the lifetime of the variable.
>>>> 
>>>> In both cases, we would want to ensure the variable is only accessed from the thread capturing the closure.  In (a), we would be effectively creating a whole new class of variables (in addition to the seven (or eight, depending on how you count) already defined by the JLS); these are no longer local variables and should not look like them.   In both (a) and (b), we would be creating a kind of restricted closure, whose execution is restricted in space (confined to a specific thread) and/or time (confined to the lifetime of the scope in which the variable is declared.)  For concreteness let's call these confined variables and confined lambdas.  
>>>> 
>>>> It is possible to create mechanisms to reify and enforce these various types of confinement (we've discussed them extensively internally.)  However, they add nontrivial complexity to the language.  
>>>> 
>>>> In order to justify the complexity that these new features would generate, there needs to be a compelling use case.  When I explored this issue, I asked a number of people to write down a use case for this.  Every one wrote some form of:
>>>> 
>>>> int sum = 0;
>>>> list.forEach( #{ x -> sum += x.foo() }
>>>> 
>>>> In a parellel world, this idiom is irretrievably broken.  (See Guy Steele's presentation "Organizing Functional Code for Parallel Execution, or, foldl considered slightly harmful.")  People will do this, they will do this without thinking, and their code will be broken.  It is very hard, even for experts, to get this right.  While Java has plenty of other opportunities to create non-thread-safe code, I am not going to create an entirely new and nearly irresistible vector for doing so.  Iteration and side-effects are how we've been trained to do things in Java, but we have to learn to do better.  
>>>> 
>>>> In sum, adding this feature that you (and others) want so badly seems to add up to:
>>>> - Lots of new complexity (e.g., confined variables and lambdas)
>>>> - Nearly irresistible new areas for making errors
>>>> - All to prop up a broken programming model (iterations + side effects)
>>>> 
>>>> To quote a past president: "Wouldn't be prudent."  At this point in time, it feels the risk and complexity outweighs the benefit.  I would rather put the effort into supporting map/reduce-y idioms in the libraries.  The above block is much better as:
>>>> 
>>>> list.reduce( #{ x, y => x+y } )
>>>> or
>>>> list.reduce(Reducers.SUM)
>>>> or
>>>> list.sum()
>>>> 
>>>>> this thought leads to another one. in your example you are assuming
>>>>> that the foreach method will get into multithreaded stuff.
>>>> 
>>>> Not quite: s/will/may/  
>>>> 
>>>>> your foreach impl coud say:
>>>>> 
>>>>> public void forEach(@Multithreaded Foreachable block) {...}
>>>> 
>>>> We explored these issues in the JSR-166 expert group several years ago.  The basic problem is that we don't have a way of reliably expressing "reference to thread-safe object" in the type system (either statically or dynamically), so these things revert to being documentation rather than type assertions.  
>>>> 
>>>>> for that matter, you should probably name your loop method forEachMT.
>>>> 
>>>> Under consideration.  Though my intuition is that (a) such a convention will be hard to stick to and (b) in five years it will probably look silly.  
>>> 
>>> 
> 
>