Effectively final

[Jim Mayer]

Hi Brian,

Your post prompted a thought about the naming of library methods.  Since
Java has its history and since it is easy to write non-functional code (call
a method, the evil array hack, etc.), perhaps the methods that are intended
to support fine grain parallelism should call that out in their names.

I'm thinking of 'parallelReduce', ' pReduce', or something like that.  I
suspect that lot of people will mentally model 'reduce' as a shorthand for a
for loop and will assume sequential evaluation.

Such a naming convention would also help avoid confusion with the 'Block'
oriented methods that were discussed recently.  Finally, it would help avoid
problems if mutation of locals was allowed in some future Java release.

On the downside, the library would be more verbose.

Thanks!

-- Jim
On Aug 18, 2011 12:46 PM, "Brian Goetz" <brian.goetz at oracle.com> wrote:
> This argument might work in another world, but ignores the reality when
> applied to Java.
>
> Yes, there *are* other programming models for concurrency that are less
> problematic than "everything is mutable and you have to use exclusion
> and/or partitioning to avoid shooting yourself in the foot."
>
> Yes, these models can be implemented in Java, provided that the
> programmer brings a sufficient degree of discipline to the situation and
> stays to the unenforced rules of the model. And in some of them,
> mutable local capture by lambdas would be convenient.
>
> Yes, people (myself included) have successfully written large, safe,
> concurrent systems in Java using this approach.
>
> Yes, it is a shame that Java chose the "mutation + locking" route to
> concurrency.
>
> But, if we allowed mutable capture, what is the most common idiom that
> would use it? It is virtually guaranteed it will be:
>
> int sum=0;
> x.forEach(#{ a -> sum += a.foo(); });
>
> And this is not one of those safer models that you hope will emerge into
> mainstream popularity in the future. This is making Java less safe, by
> creating a whole new category of serious hazards that didn't exist
> before -- data races on locals.
>
> Given that those safer models have not yet emerged into the mainstream,
> and this sort of unsafe willy-nilly mutation is definitely here, the
> tradeoff simply does not add up.
>
> I've already indicated we may choose to revisit this in the future,
> using an approach where the thread-confinement constraint can be
> enforced by the VM. But we will not be revisiting this now.
>
> I applaud your passion, but all of this is well-covered ground, and
> there's new ground to be covered before we're done. Let's move on.
>
>
> On 8/18/2011 12:00 PM, Tim Fox wrote:
>> On 18/08/11 16:30, Howard Lovatt wrote:
>>> On 16/08/2011, at 10:45 PM, Tim Fox<timvolpe at gmail.com
>>> <mailto:timvolpe at gmail.com>> wrote:
>>>
>>>> The point I was making was in response to your assertion that
>>>> inherently serial code will become less common as time goes on.
>>>> Actors frameworks (and other hybrid frameworks like node.x) are all
>>>> based around inherently serial code, and I don't see them dying out
yet.
>>>
>>> I don't get what you are driving at, you have been posting that
>>> Lambdas should capture mutable local variables and have given Actors
>>> as an example of why that is necessary.
>> No, I didn't give actors as an example of why lambdas should mutate
>> local vars, I gave actors as a counterpoint to your assertion that
>> serial code will become less common as time came on (Actor code is all
>> serial and these frameworks are gaining in popularity)
>>
>> The point I was trying to make, in a nutshell, was this:
>>
>> Conventional wisdom Java concurrency involves letting many threads
>> execute your objects and using synchronization and what-not to make
>> those objects threadsafe. This is difficult. Concurrent programming is
hard.
>>
>> There are other ways of making scalable concurrent systems that don't
>> involve having your code executed by multiple threads concurrently.
>> (Actor model is one example of this, there are others) This is very
>> desirable, since if you don't have to worry about making your code
>> threadsafe, you can open up to a legion of web developers who otherwise
>> would find concurrency too hard.
>>
>> My bet is the latter type of concurrent systems will eventually become
>> more popular. So... if in this future, developers will be writing
>> islands of single threaded coded, then it's safe to use lambdas that
>> mutate local vars in those islands since zero race conditions are
>> guaranteed.
>>
>> So far from being a throwback to the 50s, lambdas which mutate local
>> vars could be a very useful tool for the next generation of frameworks.
>> And if Java does not support such lambdas it will be sidelined as a good
>> programming language for these kinds of frameworks.
>>
>> Allowing such mutation would allow you to implement "node.js-like"
>> frameworks on the JVM. This is already possible in JS/Ruby/Groovy/etc
>> but is currently very clunky in Java because of lack of lambdas - but
>> even with Java 8 proposed lambdas it would still be clunky because of
>> the effectively-final rule.
>>
>>
>>> What I demonstrated, was that for Actors you wouldn't use Lambdas but
>>> classes and therefore your argument that Lambdas should capture local
>>> mutables because of Actors is non sequitur.
>>>
>>> Now you seem to be saying that you were talking about particular
>>> styles of code. Are you arguing about capturing mutables or not?
>>>
>>> PS Capturing a mutable variable in an Actor and passing it via message
>>> to another actor is problematic. Again indicating that Lambdas should
>>> not capture mutable state.
>>>
>>> PPS Also note that Scala has introduced map/reduce style collections
>>> (presumably because people prefer these to Actors - which it already
>>> has). Again map/reduce is best suited to not capturing mutable
variables.
>>>
>>>> Quite the opposite, I see them getting more popular :)
>>>>
>>>> On 16/08/11 16:25, Howard Lovatt wrote:
>>>>> I don't see that the actor or FJ type frameworks are effected much,
>>>>> because the messages are immutable, e.g. a standard actor example:
>>>>>
>>>>> http://java.dzone.com/articles/scala-threadless-concurrent
>>>>>
>>>>> Recoded in Java would be something like this:
>>>>>
>>>>> public class Actor extends FJTask {
>>>>> private Queue messages ...
>>>>> public void accept( final Object message ) ...
>>>>> public Object acceptAndReply( final Object message ) ...
>>>>> public Future<Object> acceptAndReplyLater( final Object message ) ...
>>>>> protected void reply( final Object response ) ...
>>>>> protected Object getMessage() ...
>>>>> }
>>>>>
>>>>> |10.||public class Accumulator ||extends| |Actor {|
>>>>> |11.||public void run() {|
>>>>> |12.||int sum = ||0;|
>>>>> |13.||for(;;) {|
>>>>> |14.|final Object message = getMessage();
>>>>> |15.|if ( message instanceof |Accumulate ) { || sum +=
>>>>> ((Accumulate)message).n; ||}|
>>>>> |16.|else if ( message instanceof |Reset ) { sum = ||0; }|
>>>>> |17.|else if ( message instanceof Total ) { reply(sum); break;
>>>>> |18.||}|
>>>>> |19.||}|
>>>>> |20.||}|
>>>>> |21.||}|
>>>>>
>>>>>
>>>>> The lambdas might prove useful for the messages, that are immutable
>>>>> anyway, but the actors themselves would always be instances of a
>>>>> class because you need to be able to create more actors as the
>>>>> program runs and you can't do this with a lambda (or a closure many
>>>>> other language). As I said at the beginning I don't see that an
>>>>> Actor framework would change much one way or the other if Lambdas
>>>>> could mutate a variable or not.
>>>>>
>>>>> Cheers,
>>>>>
>>>>> -- Howard.
>>>>>
>>>>> Sent from my iPad
>>>>>
>>>>> On 16/08/2011, at 6:57 PM, Tim Fox<timvolpe at gmail.com
>>>>> <mailto:timvolpe at gmail.com>> wrote:
>>>>>
>>>>>> On 16/08/11 12:36, Howard Lovatt wrote:
>>>>>>> I think the correct decision has been made, inherently serial code
>>>>>>> will become less common as time goes on.
>>>>>> The growing popularity of actor model implementations (actors of
>>>>>> course
>>>>>> only allow serial execution of code in the actor), e.g. Erlang and
>>>>>> Akka,
>>>>>> and the growing popularity of languages which are inherently serial
>>>>>> (e.g. JavaScript) are a clear counterpoint to your argument. Also
look
>>>>>> at webworkers, node.js. I could go on.
>>>>>>
>>>>>> I'd argue the current trend is quite opposite to what you see. Non
>>>>>> serial code will become unusual (because concurrency is hard), and
>>>>>> serial code will become the norm (because it's easier to code,
>>>>>> especially for the web developer masses). Systems will scale by
having
>>>>>> multiple "islands" of serial code (e.g. multiple actors in the
>>>>>> actor model).
>>>>>>
>>>>>> Please note that just because all code is executed serially does not
>>>>>> mean you can't exploit parallelism. You can do fork/join type stuff
by
>>>>>> sending messages to other "islands" and reassembling the results as
>>>>>> they
>>>>>> come back.
>>>>>>> Therefore optimizing new features for parallel execution is the
>>>>>>> correct path to take and also constant with Java's past of taking
>>>>>>> existing concepts into the mainstream. When Java was introduced
>>>>>>> the concentration on objects and garbage collection was not
>>>>>>> mainstream and was opposed by many. I think the same will happen
>>>>>>> with mutable data and serial execution and it is nice that Java is
>>>>>>> once again daring to be different and adhering to the mantra that
>>>>>>> less is more.
>>>>>>>
>>>>>>> Cheers,
>>>>>>>
>>>>>>> -- Howard.
>>>>>>>
>>>>>>> Sent from my iPad
>>>>>>>
>>>>>>> On 16/08/2011, at 4:56 PM, Steven Simpson<ss at comp.lancs.ac.uk
>>>>>>> <mailto:ss at comp.lancs.ac.uk>> wrote:
>>>>>>>
>>>>>>>> On 16/08/11 01:51, Stephen Colebourne wrote:
>>>>>>>>
>>>>>>>> [snip: lots of syntax options for permitting mutable locals]
>>>>>>>>> int #total = 0;
>>>>>>>>> list.apply(#{item -> total += item});
>>>>>>>>>
>>>>>>>>> ie. a way to introduce a local variable that can be managed
safely.
>>>>>>>> For the simple example given, you could translate 'total' into an
>>>>>>>> AtomicInteger, but if there are other variables to be accessed,
you'd
>>>>>>>> have to box them together, and make the whole lambda synchronize
>>>>>>>> on it,
>>>>>>>> or at least from the first use of the box to the last. Trying to
>>>>>>>> patch
>>>>>>>> the call site like this doesn't seem to be particularly
>>>>>>>> promising. The
>>>>>>>> alternative is to require List.apply to make additional
>>>>>>>> guarantees about
>>>>>>>> how it executes the lambda.
>>>>>>>>
>>>>>>>> Would it not be better to let List.apply get on with its potential
>>>>>>>> parallelism, and define other methods that do make extra
guarantees,
>>>>>>>> e.g. that the lambda will be executed serially, or even on the
>>>>>>>> caller's
>>>>>>>> thread?
>>>>>>>>
>>>>>>>> Tim's cases include, for example, setTimeout(int, Runnable),
>>>>>>>> which must
>>>>>>>> make such a guarantee, if only informally in its documentation.
>>>>>>>> To be
>>>>>>>> more formal:
>>>>>>>>
>>>>>>>> * Declare setTimeout(int, @Callback Runnable).
>>>>>>>> * When a lambda is assigned to a @Callback Runnable, allow the
>>>>>>>> lambda body to mutate locals (without error or warning).
>>>>>>>> * Don't permit a @Callback Runnable (which is tainted) to be
>>>>>>>> assigned to a plain Runnable (without error or warning).
>>>>>>>>
>>>>>>>> This way, the likes of List.apply don't have to make any
guarantees,
>>>>>>>> requiring the caller to make corresponding ones (automatically
>>>>>>>> achieved
>>>>>>>> by not being generally allowed to mutate locals). Meanwhile,
>>>>>>>> setTimout
>>>>>>>> makes additional guarantees, to the convenience of the caller, who
is
>>>>>>>> specially permitted to mutate locals.
>>>>>>>>
>>>>>>>> Cheers,
>>>>>>>>
>>>>>>>> Steven
>>>>>>>>
>>>>>>
>>>>>>
>>>>
>>
>>
>