Accessing non-final local variables from a lambda expression

[Reinier Zwitserloot]

On Fri, Feb 26, 2010 at 5:01 PM, Peter Levart <peter.levart at marand.si>wrote:

>
> The problem with this approach (from the performance perspective) is when
> you capture several mutable local variables. Having a separate wrapper
> object for each of them is an overhead which can be avoided if the compiler
> constructs a single frame object for you.
>

That's entirely correct, but doing this is, if I understand correctly, very
complicated from the perspective of the java memory model. Also, without a
very specific sort of unsafe/safe closures I don't see how this can be done:

If a closure lives on beyond the lifetime of its parent frame, then
obviously the variable cannot _be_ in the parent frame. These variables
would have to be hosted in a separate frame that lives on until the method
and ALL closures in it (or at least all the ones that access this variable)
have been GCed. Right now frames don't have a dependency on the GC so I
don't see how this can be done in a way that is more performant than a heap
allocation.

 Also, if a closure is transported to a different thread, the same problem
occurs, even if the original method is still 'live'. Frames, as I understand
the JVM, aren't shared between threads.

Hence I'm assuming in all these discussions about syntax that it boils down
to some sugar that hosts the variable on the heap, via e.g. an intermediate
array, or perhaps a custom class to do it, which makes it easier for the JVM
to recognize this case and eventually perhaps optimize for it.



> There are perfectly valid use-cases (like the one above) where accessing a
> mutable local variable is safe. For such cases it should not be necessary to
> use @Shared annotation on a variable to prevent a warning. The only way for
> a mutable local variable to escape the method is via capture from a closure
> whose reference escapes the method. So a better approach would be to
> annotate function type (and SAM type) variables
> (instance/static/local/parameters) that hold references to closures, not
> mutable local variables accessed from closures.
>

In interesting idea, but I don't think the type system is the right place.
For example, TreeSet takes a Comparator, but this Comparator is *NOT*
transparency-safe; after all, the TreeSet keeps a reference to it and will
invoke it each time an item is added to the TreeSet, which may even occur in
another thread.

On the other hand, that same Comparator type is also used for
Collections.sort, but here its entirely transparency-safe. It doesn't escape
the sort() call, and the sort() call doesn't do any threading.

Instead it would have to be a flag on the method itself, or to be even more
precise, a method parameter. An alternative is to just let Comparator rot,
and focus on the new (or introduce closure syntax for both, but to avoid the
java API hosting many superfluous methods, this optimally should be solved
with some sort of automatic adaptation of closures into SAM types, as is
part of most proposals including the current status quo). Either way this
means it's not really part of the type system then, and you'd have to write:

public static <T> void sort(List<T>, @Safe Comparator<T> foo).

"@Safe" doesn't feel right for this. I was coincidentally already working on
somehing similar to this; I was using the "do" keyword. Either way you still
have the problem of _writing_ the closure in the first place: What should
happen when the closure does things that are only legit in a @Safe fashion?
Should the notion of safety be part of a closure's _type_? Isn't that a big
burden on the type system - yet another flag that needs to be tracked?


>
> A simple approach with a single annotation @Safe (meaning: multi thread
> safe) could work as follows:
>
> void test()
> {
>   boolean flag = true;
>
>   // OK
>   ()->boolean flipFlop = ()->(flag ^= true);
>
>   // warning: assignment of unsafe function to @Safe variable
>   @Safe ()->boolean safeFlipFlop = ()->(flag ^= true);
>
>   // OK: compiler determines lambda is safe (doesn't access mutable local
> vars)
>   @Safe ()->void safeHello = ()->(System.out.println("Hello!"); void);
>
>   flipFlop = safeFlipFlop; // OK
>   safeFlipFlop = flipFlop; // warning: assignment of unsafe function to
> @Safe variable
>
>   int count = 0;
>   Object lock = new Object();
>   // OK: user suppresses the warning since she knows her code is safe
>   @SuppressWarnings("unsafe")
>   @Safe ()->int safeCounter = ()->(int c; synchronized(lock) { c = count++;
> } c);
> }
>
> All JDK methods/constructors accepting SAM types as parameters and invoking
> them from other threads (like new Thread, Executor.submit, ...) would have
> to add @Safe to their SAM parameters and any such 3rd party APIs would have
> to be retrofitted with annotations too.
>
> That's a drawback of this approach but It might still be better than
> @Shared on local vars. If users are forced to shut-up the compiler in all
> valid use-cases where it is perfectly safe to access mutable local vars,
> then they might develop a habit of putting @Shared annotation on every local
> variable accessed in lambdas without verifying that passing such lambdas to
> different methods (threads) is actually safe. On the other hand, when using
> @Safe on function/SAM variables, warnings would be less frequent, more
> accurate and would therefore have a better chance of being considered.
>
> The issue with this approach is whether to treat accesses to
> instance/static fields of outer classes and invocations of methods as safe
> or unsafe. In general it is unsafe, but it is allowed by default in inner
> classes so if we just want to issue warnings for the "new dangerous
> feature", then they should be treated as safe.
>
> The general approach to checking the multi-thread-safe code would of course
> extend to methods and be orthogonal to lambdas, but If we just want to play
> safe with new lambda features then the simple approach described above could
> be enough.
>
> Regards, Peter
>
>