capturing (or not) mutable local variables

[Reinier Zwitserloot]

Brian's original response to Vincent (quoted below) says almost all that
needs to be said here. A few add-ons:

AtomicReference is the go-to utility if you must have a mutable from outer
state. It would be nice if Atomic* were extended to cover all types. There's
Integer, Long, and Boolean (where'd that come from?). I vote we at least add
Double to this set, and probably the whole set of primitives.

Mutable outer variables can always be added later, but they cannot be taken
away later. However, if they are added, one should be able to declare them
'volatile'. Adding them will also have a rather significant impact on code
meaning: By accessing a mutable from outer, that variable must switch from
frame/stack-based cannot-possibly-cause-thread-issues local variable to
heap-based could-be-anywhere variable that looks like a local variable. I'm
not sure this is worth it. As Brian mentioned, use-cases?

A compromise (which can still be added in JDK9, or later after we get more
experience with lambda, but in time for JDK8) is to offer a keyword of some
sort to explicitly make a mutable outer accessible from lambdas. Possibly
"public" if a (context-sensitive) new keyword is not an option. Another
option is to add it, but always generate a warning to make more visible the
fact that a local is heap-based due to lambda access. This warning can be
suppressed with a custom annotation: "@Shared int x = 10;" - Again, I vote
we don't consider this for now, and revisit this topic later. It won't be
difficult to add (at minimum, the compiler can sugar it into
AtomicReference<X> and wrap all reads/writes into get()/set() calls).

 --Reinier Zwitserloot



On Sat, Nov 20, 2010 at 10:29 AM, Brian Goetz <brian.goetz at oracle.com>wrote:

> > Hi Brian, I came to you yesterday at devoxx to ask about the rationale
> > behind the decision to not capture mutable local variables.
>
> I admire your passion and your persistence :)
>
> > I understand that among several arguments, there is a strong one about
> > preventing people from doing unsafe things. you gave the analogy of
> > the kid, the fence and the pool.
>
> Safety mechanisms do not need to be perfect to be useful.  Your arguments
> seem to center around "I can climb the fence, so there's no point in having
> it."  This argument is, to be blunt, silly.  The design principles of the
> Java language stress safety, even if there are cases where such safety
> guards can be "casted away" by a sufficiently motivated user.
>
> If you want to capture a not-effectively-final local variable in a closure,
> then one of the following two things has to happen:
> a) The lifetime of the variable must be extended to the lifetime of the
> closure
> b) The lifetime of the closure must be shortened to the lifetime of the
> variable.
>
> In both cases, we would want to ensure the variable is only accessed from
> the thread capturing the closure.  In (a), we would be effectively creating
> a whole new class of variables (in addition to the seven (or eight,
> depending on how you count) already defined by the JLS); these are no longer
> local variables and should not look like them.   In both (a) and (b), we
> would be creating a kind of restricted closure, whose execution is
> restricted in space (confined to a specific thread) and/or time (confined to
> the lifetime of the scope in which the variable is declared.)  For
> concreteness let's call these confined variables and confined lambdas.
>
> It is possible to create mechanisms to reify and enforce these various
> types of confinement (we've discussed them extensively internally.)
>  However, they add nontrivial complexity to the language.
>
> In order to justify the complexity that these new features would generate,
> there needs to be a compelling use case.  When I explored this issue, I
> asked a number of people to write down a use case for this.  Every one wrote
> some form of:
>
>  int sum = 0;
>  list.forEach( #{ x -> sum += x.foo() }
>
> In a parellel world, this idiom is irretrievably broken.  (See Guy Steele's
> presentation "Organizing Functional Code for Parallel Execution, or, foldl
> considered slightly harmful.")  People will do this, they will do this
> without thinking, and their code will be broken.  It is very hard, even for
> experts, to get this right.  While Java has plenty of other opportunities to
> create non-thread-safe code, I am not going to create an entirely new and
> nearly irresistible vector for doing so.  Iteration and side-effects are how
> we've been trained to do things in Java, but we have to learn to do better.
>
> In sum, adding this feature that you (and others) want so badly seems to
> add up to:
> - Lots of new complexity (e.g., confined variables and lambdas)
> - Nearly irresistible new areas for making errors
> - All to prop up a broken programming model (iterations + side effects)
>
> To quote a past president: "Wouldn't be prudent."  At this point in time,
> it feels the risk and complexity outweighs the benefit.  I would rather put
> the effort into supporting map/reduce-y idioms in the libraries.  The above
> block is much better as:
>
>  list.reduce( #{ x, y => x+y } )
> or
>  list.reduce(Reducers.SUM)
> or
>  list.sum()
>
> > this thought leads to another one. in your example you are assuming
> > that the foreach method will get into multithreaded stuff.
>
> Not quite: s/will/may/
>
> > your foreach impl coud say:
> >
> > public void forEach(@Multithreaded Foreachable block) {...}
>
> We explored these issues in the JSR-166 expert group several years ago.
>  The basic problem is that we don't have a way of reliably expressing
> "reference to thread-safe object" in the type system (either statically or
> dynamically), so these things revert to being documentation rather than type
> assertions.
>
> > for that matter, you should probably name your loop method forEachMT.
>
> Under consideration.  Though my intuition is that (a) such a convention
> will be hard to stick to and (b) in five years it will probably look silly.
>
>
>
>
>