capturing (or not) mutable local variables

[Jim Mayer]

Actually, I am extremely fond of functional programming and lazy evaluation.
 It's one of my standard "tricks" and I use it a lot in my Java code.
 Haskell (which I've read about but haven't used) gracefully incorporates
the need for interacting with a mutable world by encapsulating "impure"
computations.  I think it would be interesting to embed a functional
language within Java (like a monad in Haskell turned on its head :-)).  The
problem I see with "Brian and Co.'s" approach is they're layering a
functional bit onto a language that is all about mutable state.  What we end
up with is a language that is neither safe to use in a
parallel/functional/multi-core environment nor as expressive as it could be
in a mutable-state environment.

I'd rather see the issue of introducing "pure" functions tackled separately
from the issue of lexical closure in lambda expressions.  I think we could
do that, and it doesn't feel like an "either/or" situation to me.

-- Jim

On Sun, Nov 21, 2010 at 8:45 PM, Howard Lovatt <howard.lovatt at gmail.com>wrote:

> I think Brian and Co. have got this one right; the future is multicore
> and parallel processing, no point introducing something that will be
> an ever increasing pain point. We should be looking to our parallel
> future, not our serial past.
>
> I should state my bias, in that I like a functional approach and I am
> much more influenced by the likes of Haskell than Lisp. I also find
> the argument that you can't have closures without writing to variables
> strange since languages like Haskell are normally treated as having
> closures but don't have write access. Also in Church's Lambda calculus
> there was no write at all!
>
> > ---------- Forwarded message ----------
> > From: Mikael Grev <grev at miginfocom.com>
> > To: lambda-dev at openjdk.java.net
> > Date: Sun, 21 Nov 2010 12:39:03 +0100
> > Subject: Re: capturing (or not) mutable local variables
> > Excellent Brian.
> >
> > It would mean that a single compiler error would be shown when this is
> broken:
> >
> >  - A variable cannot be changed after it has been used in a closure
> initialization.
> >
> > Easy, clear and user friendly. No need for funky hacks in the non-mutable
> case.
> >
> > This kind of capture will not be seen as limited as the current
> effectively-final definition, which only real merit is that it saves six
> keystrokes.
> >
> >
> > And I think there's a big opportunity here as well. The spin doctor in me
> say that if it is carefully communicated by Oracle that 2) is a very
> conscious decision and that the way you should do capture of mutable
> variables is by using an AtomicXxx I think you can almost totally get away
> with it. Maybe even get out on top of other languages since this is very
> multi threading aware from the start.
> > It is very important that all code examples for Lamdas have at least one
> example of a good use of AtomicXxx for mutable variables. Then this becomes
> the standard way of implementing mutability in parallel closures. There
> might even be room for some syntax sweetening of AtomicXxx usage in the
> future, which makes this even more compelling.
> >
> > Cheers,
> > Mikael
> >
> >
> > On Nov 20, 2010, at 21:41 PM, Brian Goetz wrote:
> >
> >> (2) is an interesting idea and worth considering.
> >>
> >> On Nov 20, 2010, at 5:37 AM, Mikael Grev wrote:
> >>
> >>> Please don't make it possible to mutate the variable from within the
> closure.
> >>>
> >>>
> >>> However, I wouldn't mind if "effectively-final" was extended so that
> either:
> >>>
> >>> 1) The closure works on a copy, giving no restrictions on the
> initialization and reassignment of the captured variable
> >>> 2) Any variable is effectively-final as long as it isn't reassigned
> after the closure initialization. (reordering should be solvable)
> >>>
> >>> The reason for this is that the following isn't that uncommon and "i"
> needs to be reassigned as another variable to be effectively-final (ugly):
> >>>
> >>> int i = 0;
> >>> if (something)
> >>>      i = xxx;
> >>>
> >>> capture of i
> >>>
> >>> Loosening the definition of effectively-final would effectively make
> the feature easier to use.
> >>>
> >>> Cheers,
> >>> Mikael
> >>>
> >>>
> >>> On Nov 20, 2010, at 10:29 AM, Brian Goetz wrote:
> >>>
> >>>>> Hi Brian, I came to you yesterday at devoxx to ask about the
> rationale
> >>>>> behind the decision to not capture mutable local variables.
> >>>>
> >>>> I admire your passion and your persistence :)
> >>>>
> >>>>> I understand that among several arguments, there is a strong one
> about
> >>>>> preventing people from doing unsafe things. you gave the analogy of
> >>>>> the kid, the fence and the pool.
> >>>>
> >>>> Safety mechanisms do not need to be perfect to be useful.  Your
> arguments seem to center around "I can climb the fence, so there's no point
> in having it."  This argument is, to be blunt, silly.  The design principles
> of the Java language stress safety, even if there are cases where such
> safety guards can be "casted away" by a sufficiently motivated user.
> >>>>
> >>>> If you want to capture a not-effectively-final local variable in a
> closure, then one of the following two things has to happen:
> >>>> a) The lifetime of the variable must be extended to the lifetime of
> the closure
> >>>> b) The lifetime of the closure must be shortened to the lifetime of
> the variable.
> >>>>
> >>>> In both cases, we would want to ensure the variable is only accessed
> from the thread capturing the closure.  In (a), we would be effectively
> creating a whole new class of variables (in addition to the seven (or eight,
> depending on how you count) already defined by the JLS); these are no longer
> local variables and should not look like them.   In both (a) and (b), we
> would be creating a kind of restricted closure, whose execution is
> restricted in space (confined to a specific thread) and/or time (confined to
> the lifetime of the scope in which the variable is declared.)  For
> concreteness let's call these confined variables and confined lambdas.
> >>>>
> >>>> It is possible to create mechanisms to reify and enforce these various
> types of confinement (we've discussed them extensively internally.)
>  However, they add nontrivial complexity to the language.
> >>>>
> >>>> In order to justify the complexity that these new features would
> generate, there needs to be a compelling use case.  When I explored this
> issue, I asked a number of people to write down a use case for this.  Every
> one wrote some form of:
> >>>>
> >>>> int sum = 0;
> >>>> list.forEach( #{ x -> sum += x.foo() }
> >>>>
> >>>> In a parellel world, this idiom is irretrievably broken.  (See Guy
> Steele's presentation "Organizing Functional Code for Parallel Execution,
> or, foldl considered slightly harmful.")  People will do this, they will do
> this without thinking, and their code will be broken.  It is very hard, even
> for experts, to get this right.  While Java has plenty of other
> opportunities to create non-thread-safe code, I am not going to create an
> entirely new and nearly irresistible vector for doing so.  Iteration and
> side-effects are how we've been trained to do things in Java, but we have to
> learn to do better.
> >>>>
> >>>> In sum, adding this feature that you (and others) want so badly seems
> to add up to:
> >>>> - Lots of new complexity (e.g., confined variables and lambdas)
> >>>> - Nearly irresistible new areas for making errors
> >>>> - All to prop up a broken programming model (iterations + side
> effects)
> >>>>
> >>>> To quote a past president: "Wouldn't be prudent."  At this point in
> time, it feels the risk and complexity outweighs the benefit.  I would
> rather put the effort into supporting map/reduce-y idioms in the libraries.
>  The above block is much better as:
> >>>>
> >>>> list.reduce( #{ x, y => x+y } )
> >>>> or
> >>>> list.reduce(Reducers.SUM)
> >>>> or
> >>>> list.sum()
> >>>>
> >>>>> this thought leads to another one. in your example you are assuming
> >>>>> that the foreach method will get into multithreaded stuff.
> >>>>
> >>>> Not quite: s/will/may/
> >>>>
> >>>>> your foreach impl coud say:
> >>>>>
> >>>>> public void forEach(@Multithreaded Foreachable block) {...}
> >>>>
> >>>> We explored these issues in the JSR-166 expert group several years
> ago.  The basic problem is that we don't have a way of reliably expressing
> "reference to thread-safe object" in the type system (either statically or
> dynamically), so these things revert to being documentation rather than type
> assertions.
> >>>>
> >>>>> for that matter, you should probably name your loop method forEachMT.
> >>>>
> >>>> Under consideration.  Though my intuition is that (a) such a
> convention will be hard to stick to and (b) in five years it will probably
> look silly.
> >>>
> >>>
> >
> >
> >
> >
> > ---------- Forwarded message ----------
> > From: Neal Gafter <neal at gafter.com>
> > To: Alex Blewitt <alex.blewitt at gmail.com>
> > Date: Sun, 21 Nov 2010 07:06:16 -0800
> > Subject: Re: capturing (or not) mutable local variables
> > On Sun, Nov 21, 2010 at 1:21 AM, Alex Blewitt <alex.blewitt at gmail.com
> >wrote:
> >
> >> On 21 Nov 2010, at 02:08, Neal Gafter <neal at gafter.com> wrote:
> >>
> >> > The point of my question is that mutable versus immutable has nothing
> to
> >> do
> >> > with variable lifetime.  Pretending they have anything to do with each
> >> other
> >> > is just unnecessarily muddying the the discussion.
> >>
> >> Except that this assumes it is the same variable inside and outside the
> >> lambda. If it is semantically a different variable (with the lifetime of
> the
> >> closure) which has been assigned a copy of the value from the enclosing
> >> scope then mutability, or lack thereof, is key.
> >>
> >
> > Is it semantically different?  It appears in all ways to be a single
> > variable throughout its scope.  The "lifetime of the variable" isn't part
> of
> > the semantics today, and if you distinguish one part of the variable's
> scope
> > from another by introducing this concept, you haven't added anything
> useful
> > to the description.
> >
> > Do you mean to intend to allow the lambda to change the value of its
> copy?
> > If so, your description would have some meaning.  If not, you're
> suggesting
> > a difference without a distinctions.
> >
> >
> >
> > ---------- Forwarded message ----------
> > From: Brian Goetz <brian.goetz at oracle.com>
> > To: Mikael Grev <grev at miginfocom.com>
> > Date: Sun, 21 Nov 2010 10:50:50 -0500
> > Subject: Re: capturing (or not) mutable local variables
> > You can think of effectively final as a form of type inference.  There
> are lots of aspects of type inference coming in Java (diamond from Coin,
> target typing for lambda formals) and inferring finality is yet another.
> >
> > On 11/21/2010 6:39 AM, Mikael Grev wrote:
> >>
> >> Excellent Brian.
> >>
> >> It would mean that a single compiler error would be shown when this is
> broken:
> >>
> >>   - A variable cannot be changed after it has been used in a closure
> initialization.
> >>
> >> Easy, clear and user friendly. No need for funky hacks in the
> non-mutable case.
> >>
> >> This kind of capture will not be seen as limited as the current
> effectively-final definition, which only real merit is that it saves six
> keystrokes.
> >>
> >>
> >> And I think there's a big opportunity here as well. The spin doctor in
> me say that if it is carefully communicated by Oracle that 2) is a very
> conscious decision and that the way you should do capture of mutable
> variables is by using an AtomicXxx I think you can almost totally get away
> with it. Maybe even get out on top of other languages since this is very
> multi threading aware from the start.
> >> It is very important that all code examples for Lamdas have at least one
> example of a good use of AtomicXxx for mutable variables. Then this becomes
> the standard way of implementing mutability in parallel closures. There
> might even be room for some syntax sweetening of AtomicXxx usage in the
> future, which makes this even more compelling.
> >>
> >> Cheers,
> >> Mikael
> >>
> >>
> >> On Nov 20, 2010, at 21:41 PM, Brian Goetz wrote:
> >>
> >>> (2) is an interesting idea and worth considering.
> >>>
> >>> On Nov 20, 2010, at 5:37 AM, Mikael Grev wrote:
> >>>
> >>>> Please don't make it possible to mutate the variable from within the
> closure.
> >>>>
> >>>>
> >>>> However, I wouldn't mind if "effectively-final" was extended so that
> either:
> >>>>
> >>>> 1) The closure works on a copy, giving no restrictions on the
> initialization and reassignment of the captured variable
> >>>> 2) Any variable is effectively-final as long as it isn't reassigned
> after the closure initialization. (reordering should be solvable)
> >>>>
> >>>> The reason for this is that the following isn't that uncommon and "i"
> needs to be reassigned as another variable to be effectively-final (ugly):
> >>>>
> >>>> int i = 0;
> >>>> if (something)
> >>>>        i = xxx;
> >>>>
> >>>> capture of i
> >>>>
> >>>> Loosening the definition of effectively-final would effectively make
> the feature easier to use.
> >>>>
> >>>> Cheers,
> >>>> Mikael
> >>>>
> >>>>
> >>>> On Nov 20, 2010, at 10:29 AM, Brian Goetz wrote:
> >>>>
> >>>>>> Hi Brian, I came to you yesterday at devoxx to ask about the
> rationale
> >>>>>> behind the decision to not capture mutable local variables.
> >>>>>
> >>>>> I admire your passion and your persistence :)
> >>>>>
> >>>>>> I understand that among several arguments, there is a strong one
> about
> >>>>>> preventing people from doing unsafe things. you gave the analogy of
> >>>>>> the kid, the fence and the pool.
> >>>>>
> >>>>> Safety mechanisms do not need to be perfect to be useful.  Your
> arguments seem to center around "I can climb the fence, so there's no point
> in having it."  This argument is, to be blunt, silly.  The design principles
> of the Java language stress safety, even if there are cases where such
> safety guards can be "casted away" by a sufficiently motivated user.
> >>>>>
> >>>>> If you want to capture a not-effectively-final local variable in a
> closure, then one of the following two things has to happen:
> >>>>> a) The lifetime of the variable must be extended to the lifetime of
> the closure
> >>>>> b) The lifetime of the closure must be shortened to the lifetime of
> the variable.
> >>>>>
> >>>>> In both cases, we would want to ensure the variable is only accessed
> from the thread capturing the closure.  In (a), we would be effectively
> creating a whole new class of variables (in addition to the seven (or eight,
> depending on how you count) already defined by the JLS); these are no longer
> local variables and should not look like them.   In both (a) and (b), we
> would be creating a kind of restricted closure, whose execution is
> restricted in space (confined to a specific thread) and/or time (confined to
> the lifetime of the scope in which the variable is declared.)  For
> concreteness let's call these confined variables and confined lambdas.
> >>>>>
> >>>>> It is possible to create mechanisms to reify and enforce these
> various types of confinement (we've discussed them extensively internally.)
>  However, they add nontrivial complexity to the language.
> >>>>>
> >>>>> In order to justify the complexity that these new features would
> generate, there needs to be a compelling use case.  When I explored this
> issue, I asked a number of people to write down a use case for this.  Every
> one wrote some form of:
> >>>>>
> >>>>> int sum = 0;
> >>>>> list.forEach( #{ x ->  sum += x.foo() }
> >>>>>
> >>>>> In a parellel world, this idiom is irretrievably broken.  (See Guy
> Steele's presentation "Organizing Functional Code for Parallel Execution,
> or, foldl considered slightly harmful.")  People will do this, they will do
> this without thinking, and their code will be broken.  It is very hard, even
> for experts, to get this right.  While Java has plenty of other
> opportunities to create non-thread-safe code, I am not going to create an
> entirely new and nearly irresistible vector for doing so.  Iteration and
> side-effects are how we've been trained to do things in Java, but we have to
> learn to do better.
> >>>>>
> >>>>> In sum, adding this feature that you (and others) want so badly seems
> to add up to:
> >>>>> - Lots of new complexity (e.g., confined variables and lambdas)
> >>>>> - Nearly irresistible new areas for making errors
> >>>>> - All to prop up a broken programming model (iterations + side
> effects)
> >>>>>
> >>>>> To quote a past president: "Wouldn't be prudent."  At this point in
> time, it feels the risk and complexity outweighs the benefit.  I would
> rather put the effort into supporting map/reduce-y idioms in the libraries.
>  The above block is much better as:
> >>>>>
> >>>>> list.reduce( #{ x, y =>  x+y } )
> >>>>> or
> >>>>> list.reduce(Reducers.SUM)
> >>>>> or
> >>>>> list.sum()
> >>>>>
> >>>>>> this thought leads to another one. in your example you are assuming
> >>>>>> that the foreach method will get into multithreaded stuff.
> >>>>>
> >>>>> Not quite: s/will/may/
> >>>>>
> >>>>>> your foreach impl coud say:
> >>>>>>
> >>>>>> public void forEach(@Multithreaded Foreachable block) {...}
> >>>>>
> >>>>> We explored these issues in the JSR-166 expert group several years
> ago.  The basic problem is that we don't have a way of reliably expressing
> "reference to thread-safe object" in the type system (either statically or
> dynamically), so these things revert to being documentation rather than type
> assertions.
> >>>>>
> >>>>>> for that matter, you should probably name your loop method
> forEachMT.
> >>>>>
> >>>>> Under consideration.  Though my intuition is that (a) such a
> convention will be hard to stick to and (b) in five years it will probably
> look silly.
> >>>>
> >>>>
> >>
> >>
> >
> >
> >
>
>
>
> --
>   -- Howard.
>
>