capturing (or not) mutable local variables

[Doug Lea]

Finally reading this discussion...

On 11/21/10 15:18, Per Bothner wrote:
> On 11/20/2010 12:44 PM, Brian Goetz wrote:
>>> I think I'm hearing that mutating a local variable from a closure is just a
>>> bad idea in general right?
>>
>> For a closure that might conceivably be executed in another thread, this is correct.
>
> Agreed - but how is that different from mutating a field or array element
> from a closure that might conceivably be executed in another thread?
> You're just forcing the user to do by hand to work around a restriction.

I think the main goal here is to preserve the
"shallow data race freedom of locals" property of Java:

   Writes to any variable syntactically declared as a local (or
   argument) can never be subject to a data race.

In conjunction with the other main property of Java locals
that they are definitely assigned, this can be used to provide
a simple partial correctness guarantee about the integrity of
reads. Which can then be used as a base step for more
thorough reasoning about safety.

The shallowness arises in case a local is a reference to
an object and is dereferenced (accessed), in which case
programmers have more work to do proving race-freedom
of the accesses. The limitation to shallow safety
seems to be pragmatic decision:
Deeper guarantees would require non-modular checks
(as in, trying to verify that data-race freedom held
across calls to possible virtual methods) as well as
concurrency awareness by front-end compilers, plus
some means of programmers annotating the shapes of
data structures and partitioning, as in those
in DPJ (http://dpj.cs.uiuc.edu/DPJ/Home.html) that
can help check that all in-place updates of elements of
arrays or collections are fully partitioned across threads.
(Aside: Many efficient multicore algorithms entail in-place
updates -- the main advantages of their design (versus other
forms of multiprocessors) stem from their typically faster
access to shared memory.)

The workarounds like using 1-slot arrays add a level of
indirection to subvert the shallow-only provision.
Which seems tolerable. It would be even better to
come up with capture rules that guaranteed this
property while still also allowing more flexible usages.
The suggestion to extend "could be final" to something
akin to having a unique SSA form is one step.
Going even a bit further than this would be hard because
front-end compilers are not thread-aware.
So among other things they cannot detect the canonically
bad idea of using accumulator variables in parallel loops.

However, perhaps the folks arguing for more flexible
usage could discover more powerful yet practical techniques
that would still preserve the shallow data-race freedom
property in more cases, and thus allow some of their target usages.

-Doug