capturing (or not) mutable local variables

[Vincent Sevel]

Hi Brian, I came to you yesterday at devoxx to ask about the rationale
behind the decision to not capture mutable local variables.
I understand that among several arguments, there is a strong one about
preventing people from doing unsafe things. you gave the analogy of
the kid, the fence and the pool.

if I continue on the analogy, we could say that if the kid disguised
himself (eg: final int[] sum = new int[1]) then you would be probably
happy, although it did not take much to workaround your fence, and
actually people are used to it.

of course your fear completely disappears if the variable is of type
anything as long as it is final. so you do not have problem with that:

class BigGuyThatDoesnotKnowHowToSwim implements Foreachable {
  int sum = 0;

  public void do(Object elt) {
    sum += (Integer) elt;
  }
}

final BigGuyThatDoesnotKnowHowToSwim bigguy = new
BigGuyThatDoesnotKnowHowToSwim();
mylist.foreach(bigguy);

really, from the thread safety perspective, the disguised kid and the
bigguy are not better than the kid. asking if the guy is a kid is
actually the wrong question. you should be asking yourself whether or
not the guy actually knows how to swim, assuming he needs to be able
to. because you have kids that swim better than you and me, and you
have grown up that do not know.

this thought leads to another one. in your example you are assuming
that the foreach method will get into multithreaded stuff. myself, I
did not expect it to be. but if that is the case, truly we want to be
careful about it. but should I assume that any method that accepts a
block can get into MT? and should I design around that assumption? the
basic problem is that you can't figure out what a pool is. so you are
saying there are pools out there, let's protect anything that is
accessible by kids. so you will protect football fields and
playgrounds too.

if you are going that route I could argue you could go even further.
if multithread safety is the issue, and I cannot rely on the designer
to document the limitations of the api, or I cannot rely on the user
to use the api the designer had anticipated, we are actually at risk
on any method that accepts an argument with state.

on that particular issue, it is not about final variables, it is about
passing mutable state that supports the contract set by the api. If
you had told me that your impl was using MT stuff under the cover,
then I would have written a very different block:

Object sync = new Object();
int sum = 0;
list.forEach(#{ e -> synchronized(synch) { sum += e.size(); }});

or if I knew there was a pool I would have written a BigGuyThatKnowsHowToSwim.

in conclusion your fence is not only overprotecting in cases where we
do not need it, but actually it provides protection only in a minority
of cases. so really what you want to do if you are not confident that
people will read the documentation and take responsibility for their
mistakes, you should help them make sure they thought about the issue
when we know there is certainty about it.

your foreach impl coud say:

public void forEach(@Multithreaded Foreachable block) {...}

then I would get a warning or an error as a client if I tried that:

list.forEach(#{ e -> sum += e.size(); });

but the error would disappear if I did:

list.forEach(@ThreadSafe #{ e -> synchronized(synch) { sum += e.size(); }});

the good thing about this is compiler would still complain about the
following, and that, in spite of having the final keyword, which is in
no way a guarantee of thread safety:

final int[] sum = new int[1];
list.forEach(#{ e -> sum[0] += e.size(); });

I believe that if you do multithread your foreach, you will have not
more (or less for that matter) problems with or without capturing
mutable variables. because people will write blocks of code that do
not access local variables, but still manipulate pojos in a non thread
safe way.

for that matter, you should probably name your loop method forEachMT.

as a final point I would say that the issue is not about mutable
variables, it is about carrying clearly the contract constraints to
the user of the api. if the spec stays that way, I am hoping that this
is not based on the pool paradigm. in the jdk we should learn more
from dynamic languages about trust and responsibility taking.

regards.

vincent