Accessing non-final local variables from a lambda expression

[Peter Levart]

On Saturday 27 February 2010 11:05:56 pm Reinier Zwitserloot wrote:
> On Fri, Feb 26, 2010 at 5:01 PM, Peter Levart <peter.levart at marand.si>wrote:
> > The problem with this approach (from the performance perspective) is when
> > you capture several mutable local variables. Having a separate wrapper
> > object for each of them is an overhead which can be avoided if the
> > compiler constructs a single frame object for you.
> 
> That's entirely correct, but doing this is, if I understand correctly, very
> complicated from the perspective of the java memory model. Also, without a
> very specific sort of unsafe/safe closures I don't see how this can be
> done:
> 
> If a closure lives on beyond the lifetime of its parent frame, then
> obviously the variable cannot _be_ in the parent frame. These variables
> would have to be hosted in a separate frame that lives on until the method
> and ALL closures in it (or at least all the ones that access this variable)
> have been GCed. Right now frames don't have a dependency on the GC so I
> don't see how this can be done in a way that is more performant than a heap
> allocation.
> 
>  Also, if a closure is transported to a different thread, the same problem
> occurs, even if the original method is still 'live'. Frames, as I
> understand the JVM, aren't shared between threads.
> 
> Hence I'm assuming in all these discussions about syntax that it boils down
> to some sugar that hosts the variable on the heap, via e.g. an intermediate
> array, or perhaps a custom class to do it, which makes it easier for the
> JVM to recognize this case and eventually perhaps optimize for it.

With "frame object" I meant an object, allocated on the heap whose class is constructed by 
compiler and holds all captured local variables (whether final or non-final). I don't know why but 
this terminology is used by Neal and others.

With MethodHandles, local vars can be captured one-by-one via (MethodHandles.insertArgument) but 
it might still be better to construct a single "frame object" to hold all of them when their 
number reaches a certain limit or when at least one of them is non-final.

> 
> > There are perfectly valid use-cases (like the one above) where accessing
> > a mutable local variable is safe. For such cases it should not be
> > necessary to use @Shared annotation on a variable to prevent a warning.
> > The only way for a mutable local variable to escape the method is via
> > capture from a closure whose reference escapes the method. So a better
> > approach would be to annotate function type (and SAM type) variables
> > (instance/static/local/parameters) that hold references to closures, not
> > mutable local variables accessed from closures.
> 
> In interesting idea, but I don't think the type system is the right place.
> For example, TreeSet takes a Comparator, but this Comparator is *NOT*
> transparency-safe; after all, the TreeSet keeps a reference to it and will
> invoke it each time an item is added to the TreeSet, which may even occur
> in another thread.

It is the idea of JSR308 (Type annotations) that you can attach an annotation to any type. These 
annotations are treated by compiler as additional properties of the type (if types are nouns, 
type annotations are adjectives). A type with some annotation can be treated by the compiler as 
a sub-type or a super-type of the type without the annotation. In case of @Safe it would be a 
sub-type. At run-time these type attributes are of course erased, but they are present in class 
files for compiler to check them at compile time.

JSR308 has other tricks that could be used here. If we had another annotation, let's say @S that 
would be some kind of generic variable that could be attached to any type or class/interface 
declaration. For example:

@S public interface SortedSet<E> {
  ...
  @S Comparator<? super E> comparator();
 
 
@S acts as a sort of aditional SortedSet's generic parameter. A @Safe SortedSet is then an 
instance of a SortedSet that contains a @Safe Comparator.

@Safe-transparent methods are also possible:

  @S (int)->int pow2(@S (int)->int func) {
    return (int x)->(func.(func.(x)));
  }

  @Safe (int)->int doubler = (int  x)->(x+x);
  @Safe (int)->int quadrupler = pow2(doubler);

In short, JSR308 is full of good ideas that could be used for this purpose. The question is 
whether any such scheme, if implemented, would be part of JLS. I haven't found any reference to 
@Override annotation in JLS 3 for example. How to prescribe "warning" behavior without 
specifying it in the JLS?

Regards, Peter