Checked exceptions within Block<T>

Tue Jan 15 21:41:29 PST 2013

Dynamic detection is used, for example, to ensure that someone doesn't stash the lambda under the rug and call it from another thread.  Or call it after the variable to be mutated has gone out of scope.  

On Jan 15, 2013, at 6:17 PM, Ricky Clarkson wrote:

> Wouldn't mutable local capture be an odd one out, as it shouldn't require any dynamic detection?  It exists in Scala, countless lexically-scoped Lisps, Ruby, Python, JavaScript, etc., without anything like an ArrayStoreException.  It is a source of surprises though and I'm happy enough that it won't be present.  I don't mind it in Scala because in Scala a mutable variable is more obvious (it has the word 'var' before it).
> 
> 
> On Tue, Jan 15, 2013 at 10:50 PM, Brian Goetz <brian.goetz at oracle.com> wrote:
> Just to add to Neal's notes: mutable local capture, nonlocal control flow, and exception transparency are, at some level, the same feature.  We have done enough research into this to convince ourselves that implementing these requires a combination of static analysis and dynamic detection (not unlike array covariance, where the compiler can prove some accesses are safe but the VM still needs to do an array store check sometimes and could throw ArrayStoreException.)
> 
> 
> 
> On Jan 15, 2013, at 5:11 PM, Neal Gafter wrote:
> 
> > I have a few points for you.
> >
> >   - The kinds of guarantees you're contemplating are difficult to encode
> >   in the language.  You haven't given me any hint as to how you would encode
> >   it into language rules, and neither does the message you reference in a
> >   link.
> >   - The rules you appear to contemplate and far stronger than they need to
> >   be to be as useful as you imagine.  There are many useful things that one
> >   can do using threads that would not violate any of the requirements that
> >   you imagine would be useful.
> >   - The most important guarantee that you appear to contemplate is that a
> >   closure does not outlive its enclosing control context.  But that guarantee
> >   is not nearly as useful as guarantees about other things that we do not
> >   guarantee, such as stack overflow and null references.  Languages with
> >   these kind of constructs do not as a practical matter suffer problems with
> >   closure lifetime extending past than enclosing control context, and they
> >   are easily handled as runtime exceptions.  In short, I don't think the
> >   problem you've not solved is not worth solving.
> >   - The transparencies that are valuable for lambdas are independent from
> >   and orthogonal to whether the lambda is used in the same thread in which it
> >   was created, or whether it is used concurrently.
> >
> > Cheers,
> > Neal
> >
> > On Tue, Jan 15, 2013 at 4:24 PM, Steven Simpson <ss at comp.lancs.ac.uk> wrote:
> >
> >> Hi Neal - did you mean to reply off-list?  If not, feel free to repost as
> >> if nothing had happened, and I'll do likewise, or just reply back on the
> >> list to the message below, as you see fit.
> >>
> >>
> >> On 15/01/13 22:12, Neal Gafter wrote:
> >>
> >>> What is a serial guarantee, and how do you imagine encoding it into the
> >>> language rules?
> >>>
> >>
> >> Short answer by example: a serial forEach promises to invoke its block on
> >> the caller's thread, and promises not to let it be invoked after returning
> >> to the caller.  I think you called it 'non-concurrent' on the list, at some
> >> point.  Longer answer...
> >>
> >> Some time ago, while it was being discussed, I was quite in favour of
> >> having various forms of transparencies that people were advocating, and
> >> then I've come round to the idea that lambdas shouldn't go that far,
> >> because a declared goal is to support idioms most suitable to parallel
> >> execution.
> >>
> >> I probably noticed that the use cases advocating the extra transparencies
> >> were often about flow-control abstraction, and that these are often serial
> >> in nature, i.e. the control abstraction didn't have to do anything
> >> concurrently to get its job done, e.g. (non-parallel) forEach, with(lock)
> >> {... }.  (I'm assuming that this observation is almost always true, but I
> >> am interested in seeing examples which contradict it, too.)
> >>
> >> Some advocated the user of these abstractions being able to turn on
> >> transparencies - e.g. using @Shared on a local to be mutated by the lambda
> >> - because they believe the call they are making promises not to invoke the
> >> supplied lambdas in 'complex' ways.  There's a risk that this will be used
> >> incorrectly, e.g. when the promise isn't made but is believed to exist, and
> >> I thought it would be a lower risk if instead the promise was formally
> >> declared by the method, and that turned on the transparencies.  Rather than
> >> the user having control over transparencies, and guessing when they would
> >> be safe, the provider of the method, who is in a position to decide whether
> >> to make the promise, chooses whether or not to declare it.
> >>
> >> So, a very weak serial guarantee that could be made by the receiver of a
> >> lambda (or other functional object) would be: "I will not invoke the
> >> functional object on any thread but the one you're invoking me with now."
> >> This would be enough to turn on mutable local capture, in some
> >> Javascript-like asynchronous environment, as was discussed here:
> >>
> >> <http://mail.openjdk.java.net/**pipermail/lambda-dev/2011-**
> >> July/003751.html<http://mail.openjdk.java.net/pipermail/lambda-dev/2011-July/003751.html>
> >>>
> >>
> >> A stronger guarantee would be as above plus: "I will not invoke the
> >> functional object after returning control to you."  This appears good
> >> enough to do control abstraction, where the method making the guarantee is
> >> really just offering to organize the user's code according to some usually
> >> serial idiom.  You'd additionally permit non-local returns, throws, breaks
> >> and continues.
> >>
> >> Further guarantees would be about the number of times something was to be
> >> invoked (at least once, exactly once, etc), which I believe you'd need to
> >> combine control abstraction with definitely-(un)assigned concepts,
> >> 'continue' and 'break'.
> >>
> >> How to express these guarantees...  I thought you could annotate the
> >> functional-interface parameters, e.g. @Callback for the weakest, @Serial
> >> for the next.  I don't know if that's good enough, because I'd expect to be
> >> able to use @Callback as if it were some sort of C-like type qualifier (and
> >> you'd then consider whether a @Callback Runnable was assignable to a plain
> >> Runnable), but that might be stretching the meaning, purpose and principles
> >> of annotations. Perhaps less of a problem for @Serial and stronger - I
> >> imagine these enabling a certain syntax, which emphasises that only a
> >> serial idiom is going to be applied, and turns on the corresponding
> >> transparencies.
> >>
> >> Are these guarantees compiler-verifiable?  Feels like not, generally.  It
> >> might not matter so much.  We already don't expect the compiler to detect a
> >> Comparator that just returns random values, or a Set that doesn't have set
> >> semantics.
> >>
> >> So, back to the point about Stream.  You only informally get serial
> >> guarantees with the one returned by Streamable.stream(). (Otherwise, what
> >> is parallel() for?)  If you declare a method to take a Stream (rather than
> >> a Streamable[1], for whatever reason), you can't assume that it has come
> >> from stream(), not parallel().  If you did know it was serial, you might
> >> feel free to implement your own ad-hoc transparencies that depend on this
> >> serial characteristic.  Since you don't know, you may as well assume it's
> >> the parallel one and not take the risk with ad-hoc transparencies - in
> >> which case, what's the point in having stream()?  At least, without having
> >> to add any further language changes, if stream() returned a SerialStream,
> >> you'd have as much a guarantee as you would that a supplied Set is not
> >> merely a Collection, and you could at least formally express that you
> >> require a SerialStream if a plain (potentially parallel) one won't do.
> >>
> >> Ignoring my ideas of serial guarantees for a moment, and narrow down the
> >> problem to just exception transparency, suppose you ressurect <throws E>.
> >> You could declare a serial forEach as:
> >>
> >>  <throws E> void forEach(Block<? super T, E> block) throws E;
> >>
> >> For a parallel version, you have to specify how multiple exceptions are
> >> handled by forEach, or require the block to handle them itself (and drop
> >> <throws E>).  If forEach handles them, does it throw away all but the
> >> first?  Merge them in some wrapping exception (which probably changes the
> >> signature)?  It seems, the most sensible options change the signature, and
> >> one signature belongs on Stream, and the other on SerialStream.
> >>
> >> [1] Hmm, seems I'm out of date already - no more Streamable.
> >>
> >> Cheers!
> >>
> >
> 
> 
> 