Effectively final

[Jim Mayer]

Cool, I like the parallel views better than the naming convention,
especially if sequential versions with the same name are also available.

-- Jim
On Aug 19, 2011 11:07 PM, "Brian Goetz" <brian.goetz at oracle.com> wrote:
> Yep, I think reflecting the parallelism in either the name or the type
> system is pretty much a requirement. Trying to do "transparent
> parallelism" is going to be asking for trouble, just as "transparent
> remoting" was in a previous decade. The name-based approaches that you
> suggest are a possibility, as is having "parallel views":
>
> collection.asParallel().forEach(block)
>
> which is what PLinQ on .Net does. But the user has to ask for it.
> (Scala made a choice similar to .Net.)
>
> Cheers,
> -Brian
>
>
> On 8/19/2011 11:01 PM, Jim Mayer wrote:
>> Hi Brian,
>>
>> Your post prompted a thought about the naming of library methods. Since
>> Java has its history and since it is easy to write non-functional code
>> (call a method, the evil array hack, etc.), perhaps the methods that are
>> intended to support fine grain parallelism should call that out in their
>> names.
>>
>> I'm thinking of 'parallelReduce', ' pReduce', or something like that. I
>> suspect that lot of people will mentally model 'reduce' as a shorthand
>> for a for loop and will assume sequential evaluation.
>>
>> Such a naming convention would also help avoid confusion with the
>> 'Block' oriented methods that were discussed recently. Finally, it
>> would help avoid problems if mutation of locals was allowed in some
>> future Java release.
>>
>> On the downside, the library would be more verbose.
>>
>> Thanks!
>>
>> -- Jim
>>
>> On Aug 18, 2011 12:46 PM, "Brian Goetz" <brian.goetz at oracle.com
>> <mailto:brian.goetz at oracle.com>> wrote:
>> > This argument might work in another world, but ignores the reality when
>> > applied to Java.
>> >
>> > Yes, there *are* other programming models for concurrency that are less
>> > problematic than "everything is mutable and you have to use exclusion
>> > and/or partitioning to avoid shooting yourself in the foot."
>> >
>> > Yes, these models can be implemented in Java, provided that the
>> > programmer brings a sufficient degree of discipline to the situation
and
>> > stays to the unenforced rules of the model. And in some of them,
>> > mutable local capture by lambdas would be convenient.
>> >
>> > Yes, people (myself included) have successfully written large, safe,
>> > concurrent systems in Java using this approach.
>> >
>> > Yes, it is a shame that Java chose the "mutation + locking" route to
>> > concurrency.
>> >
>> > But, if we allowed mutable capture, what is the most common idiom that
>> > would use it? It is virtually guaranteed it will be:
>> >
>> > int sum=0;
>> > x.forEach(#{ a -> sum += a.foo(); });
>> >
>> > And this is not one of those safer models that you hope will emerge
into
>> > mainstream popularity in the future. This is making Java less safe, by
>> > creating a whole new category of serious hazards that didn't exist
>> > before -- data races on locals.
>> >
>> > Given that those safer models have not yet emerged into the mainstream,
>> > and this sort of unsafe willy-nilly mutation is definitely here, the
>> > tradeoff simply does not add up.
>> >
>> > I've already indicated we may choose to revisit this in the future,
>> > using an approach where the thread-confinement constraint can be
>> > enforced by the VM. But we will not be revisiting this now.
>> >
>> > I applaud your passion, but all of this is well-covered ground, and
>> > there's new ground to be covered before we're done. Let's move on.
>> >
>> >
>> > On 8/18/2011 12:00 PM, Tim Fox wrote:
>> >> On 18/08/11 16:30, Howard Lovatt wrote:
>> >>> On 16/08/2011, at 10:45 PM, Tim Fox<timvolpe at gmail.com
>> <mailto:timvolpe at gmail.com>
>> >>> <mailto:timvolpe at gmail.com <mailto:timvolpe at gmail.com>>> wrote:
>> >>>
>> >>>> The point I was making was in response to your assertion that
>> >>>> inherently serial code will become less common as time goes on.
>> >>>> Actors frameworks (and other hybrid frameworks like node.x) are all
>> >>>> based around inherently serial code, and I don't see them dying
>> out yet.
>> >>>
>> >>> I don't get what you are driving at, you have been posting that
>> >>> Lambdas should capture mutable local variables and have given Actors
>> >>> as an example of why that is necessary.
>> >> No, I didn't give actors as an example of why lambdas should mutate
>> >> local vars, I gave actors as a counterpoint to your assertion that
>> >> serial code will become less common as time came on (Actor code is all
>> >> serial and these frameworks are gaining in popularity)
>> >>
>> >> The point I was trying to make, in a nutshell, was this:
>> >>
>> >> Conventional wisdom Java concurrency involves letting many threads
>> >> execute your objects and using synchronization and what-not to make
>> >> those objects threadsafe. This is difficult. Concurrent programming
>> is hard.
>> >>
>> >> There are other ways of making scalable concurrent systems that don't
>> >> involve having your code executed by multiple threads concurrently.
>> >> (Actor model is one example of this, there are others) This is very
>> >> desirable, since if you don't have to worry about making your code
>> >> threadsafe, you can open up to a legion of web developers who
otherwise
>> >> would find concurrency too hard.
>> >>
>> >> My bet is the latter type of concurrent systems will eventually become
>> >> more popular. So... if in this future, developers will be writing
>> >> islands of single threaded coded, then it's safe to use lambdas that
>> >> mutate local vars in those islands since zero race conditions are
>> >> guaranteed.
>> >>
>> >> So far from being a throwback to the 50s, lambdas which mutate local
>> >> vars could be a very useful tool for the next generation of
frameworks.
>> >> And if Java does not support such lambdas it will be sidelined as a
good
>> >> programming language for these kinds of frameworks.
>> >>
>> >> Allowing such mutation would allow you to implement "node.js-like"
>> >> frameworks on the JVM. This is already possible in JS/Ruby/Groovy/etc
>> >> but is currently very clunky in Java because of lack of lambdas - but
>> >> even with Java 8 proposed lambdas it would still be clunky because of
>> >> the effectively-final rule.
>> >>
>> >>
>> >>> What I demonstrated, was that for Actors you wouldn't use Lambdas but
>> >>> classes and therefore your argument that Lambdas should capture local
>> >>> mutables because of Actors is non sequitur.
>> >>>
>> >>> Now you seem to be saying that you were talking about particular
>> >>> styles of code. Are you arguing about capturing mutables or not?
>> >>>
>> >>> PS Capturing a mutable variable in an Actor and passing it via
message
>> >>> to another actor is problematic. Again indicating that Lambdas should
>> >>> not capture mutable state.
>> >>>
>> >>> PPS Also note that Scala has introduced map/reduce style collections
>> >>> (presumably because people prefer these to Actors - which it already
>> >>> has). Again map/reduce is best suited to not capturing mutable
>> variables.
>> >>>
>> >>>> Quite the opposite, I see them getting more popular :)
>> >>>>
>> >>>> On 16/08/11 16:25, Howard Lovatt wrote:
>> >>>>> I don't see that the actor or FJ type frameworks are effected much,
>> >>>>> because the messages are immutable, e.g. a standard actor example:
>> >>>>>
>> >>>>> http://java.dzone.com/articles/scala-threadless-concurrent
>> >>>>>
>> >>>>> Recoded in Java would be something like this:
>> >>>>>
>> >>>>> public class Actor extends FJTask {
>> >>>>> private Queue messages ...
>> >>>>> public void accept( final Object message ) ...
>> >>>>> public Object acceptAndReply( final Object message ) ...
>> >>>>> public Future<Object> acceptAndReplyLater( final Object message )
...
>> >>>>> protected void reply( final Object response ) ...
>> >>>>> protected Object getMessage() ...
>> >>>>> }
>> >>>>>
>> >>>>> |10.||public class Accumulator ||extends| |Actor {|
>> >>>>> |11.||public void run() {|
>> >>>>> |12.||int sum = ||0;|
>> >>>>> |13.||for(;;) {|
>> >>>>> |14.|final Object message = getMessage();
>> >>>>> |15.|if ( message instanceof |Accumulate ) { || sum +=
>> >>>>> ((Accumulate)message).n; ||}|
>> >>>>> |16.|else if ( message instanceof |Reset ) { sum = ||0; }|
>> >>>>> |17.|else if ( message instanceof Total ) { reply(sum); break;
>> >>>>> |18.||}|
>> >>>>> |19.||}|
>> >>>>> |20.||}|
>> >>>>> |21.||}|
>> >>>>>
>> >>>>>
>> >>>>> The lambdas might prove useful for the messages, that are immutable
>> >>>>> anyway, but the actors themselves would always be instances of a
>> >>>>> class because you need to be able to create more actors as the
>> >>>>> program runs and you can't do this with a lambda (or a closure many
>> >>>>> other language). As I said at the beginning I don't see that an
>> >>>>> Actor framework would change much one way or the other if Lambdas
>> >>>>> could mutate a variable or not.
>> >>>>>
>> >>>>> Cheers,
>> >>>>>
>> >>>>> -- Howard.
>> >>>>>
>> >>>>> Sent from my iPad
>> >>>>>
>> >>>>> On 16/08/2011, at 6:57 PM, Tim Fox<timvolpe at gmail.com
>> <mailto:timvolpe at gmail.com>
>> >>>>> <mailto:timvolpe at gmail.com <mailto:timvolpe at gmail.com>>> wrote:
>> >>>>>
>> >>>>>> On 16/08/11 12:36, Howard Lovatt wrote:
>> >>>>>>> I think the correct decision has been made, inherently serial
code
>> >>>>>>> will become less common as time goes on.
>> >>>>>> The growing popularity of actor model implementations (actors of
>> >>>>>> course
>> >>>>>> only allow serial execution of code in the actor), e.g. Erlang and
>> >>>>>> Akka,
>> >>>>>> and the growing popularity of languages which are inherently
serial
>> >>>>>> (e.g. JavaScript) are a clear counterpoint to your argument.
>> Also look
>> >>>>>> at webworkers, node.js. I could go on.
>> >>>>>>
>> >>>>>> I'd argue the current trend is quite opposite to what you see. Non
>> >>>>>> serial code will become unusual (because concurrency is hard), and
>> >>>>>> serial code will become the norm (because it's easier to code,
>> >>>>>> especially for the web developer masses). Systems will scale by
>> having
>> >>>>>> multiple "islands" of serial code (e.g. multiple actors in the
>> >>>>>> actor model).
>> >>>>>>
>> >>>>>> Please note that just because all code is executed serially does
not
>> >>>>>> mean you can't exploit parallelism. You can do fork/join type
>> stuff by
>> >>>>>> sending messages to other "islands" and reassembling the results
as
>> >>>>>> they
>> >>>>>> come back.
>> >>>>>>> Therefore optimizing new features for parallel execution is the
>> >>>>>>> correct path to take and also constant with Java's past of taking
>> >>>>>>> existing concepts into the mainstream. When Java was introduced
>> >>>>>>> the concentration on objects and garbage collection was not
>> >>>>>>> mainstream and was opposed by many. I think the same will happen
>> >>>>>>> with mutable data and serial execution and it is nice that Java
is
>> >>>>>>> once again daring to be different and adhering to the mantra that
>> >>>>>>> less is more.
>> >>>>>>>
>> >>>>>>> Cheers,
>> >>>>>>>
>> >>>>>>> -- Howard.
>> >>>>>>>
>> >>>>>>> Sent from my iPad
>> >>>>>>>
>> >>>>>>> On 16/08/2011, at 4:56 PM, Steven Simpson<ss at comp.lancs.ac.uk
>> <mailto:ss at comp.lancs.ac.uk>
>> >>>>>>> <mailto:ss at comp.lancs.ac.uk <mailto:ss at comp.lancs.ac.uk>>> wrote:
>> >>>>>>>
>> >>>>>>>> On 16/08/11 01:51, Stephen Colebourne wrote:
>> >>>>>>>>
>> >>>>>>>> [snip: lots of syntax options for permitting mutable locals]
>> >>>>>>>>> int #total = 0;
>> >>>>>>>>> list.apply(#{item -> total += item});
>> >>>>>>>>>
>> >>>>>>>>> ie. a way to introduce a local variable that can be managed
>> safely.
>> >>>>>>>> For the simple example given, you could translate 'total' into
an
>> >>>>>>>> AtomicInteger, but if there are other variables to be
>> accessed, you'd
>> >>>>>>>> have to box them together, and make the whole lambda synchronize
>> >>>>>>>> on it,
>> >>>>>>>> or at least from the first use of the box to the last. Trying to
>> >>>>>>>> patch
>> >>>>>>>> the call site like this doesn't seem to be particularly
>> >>>>>>>> promising. The
>> >>>>>>>> alternative is to require List.apply to make additional
>> >>>>>>>> guarantees about
>> >>>>>>>> how it executes the lambda.
>> >>>>>>>>
>> >>>>>>>> Would it not be better to let List.apply get on with its
potential
>> >>>>>>>> parallelism, and define other methods that do make extra
>> guarantees,
>> >>>>>>>> e.g. that the lambda will be executed serially, or even on the
>> >>>>>>>> caller's
>> >>>>>>>> thread?
>> >>>>>>>>
>> >>>>>>>> Tim's cases include, for example, setTimeout(int, Runnable),
>> >>>>>>>> which must
>> >>>>>>>> make such a guarantee, if only informally in its documentation.
>> >>>>>>>> To be
>> >>>>>>>> more formal:
>> >>>>>>>>
>> >>>>>>>> * Declare setTimeout(int, @Callback Runnable).
>> >>>>>>>> * When a lambda is assigned to a @Callback Runnable, allow the
>> >>>>>>>> lambda body to mutate locals (without error or warning).
>> >>>>>>>> * Don't permit a @Callback Runnable (which is tainted) to be
>> >>>>>>>> assigned to a plain Runnable (without error or warning).
>> >>>>>>>>
>> >>>>>>>> This way, the likes of List.apply don't have to make any
>> guarantees,
>> >>>>>>>> requiring the caller to make corresponding ones (automatically
>> >>>>>>>> achieved
>> >>>>>>>> by not being generally allowed to mutate locals). Meanwhile,
>> >>>>>>>> setTimout
>> >>>>>>>> makes additional guarantees, to the convenience of the caller,
>> who is
>> >>>>>>>> specially permitted to mutate locals.
>> >>>>>>>>
>> >>>>>>>> Cheers,
>> >>>>>>>>
>> >>>>>>>> Steven
>> >>>>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>
>> >>
>> >>
>> >