Checked exceptions within Block<T>

[Neal Gafter]

Concurrency+mutation=nondeterminism

On Tue, Jan 15, 2013 at 6:17 PM, Ricky Clarkson <ricky.clarkson at gmail.com>wrote:

> Wouldn't mutable local capture be an odd one out, as it shouldn't require
> any dynamic detection?  It exists in Scala, countless lexically-scoped
> Lisps, Ruby, Python, JavaScript, etc., without anything like an
> ArrayStoreException.  It is a source of surprises though and I'm happy
> enough that it won't be present.  I don't mind it in Scala because in Scala
> a mutable variable is more obvious (it has the word 'var' before it).
>
>
> On Tue, Jan 15, 2013 at 10:50 PM, Brian Goetz <brian.goetz at oracle.com>wrote:
>
>> Just to add to Neal's notes: mutable local capture, nonlocal control
>> flow, and exception transparency are, at some level, the same feature.  We
>> have done enough research into this to convince ourselves that implementing
>> these requires a combination of static analysis and dynamic detection (not
>> unlike array covariance, where the compiler can prove some accesses are
>> safe but the VM still needs to do an array store check sometimes and could
>> throw ArrayStoreException.)
>>
>>
>>
>> On Jan 15, 2013, at 5:11 PM, Neal Gafter wrote:
>>
>> > I have a few points for you.
>> >
>> >   - The kinds of guarantees you're contemplating are difficult to encode
>> >   in the language.  You haven't given me any hint as to how you would
>> encode
>> >   it into language rules, and neither does the message you reference in
>> a
>> >   link.
>> >   - The rules you appear to contemplate and far stronger than they need
>> to
>> >   be to be as useful as you imagine.  There are many useful things that
>> one
>> >   can do using threads that would not violate any of the requirements
>> that
>> >   you imagine would be useful.
>> >   - The most important guarantee that you appear to contemplate is that
>> a
>> >   closure does not outlive its enclosing control context.  But that
>> guarantee
>> >   is not nearly as useful as guarantees about other things that we do
>> not
>> >   guarantee, such as stack overflow and null references.  Languages with
>> >   these kind of constructs do not as a practical matter suffer problems
>> with
>> >   closure lifetime extending past than enclosing control context, and
>> they
>> >   are easily handled as runtime exceptions.  In short, I don't think the
>> >   problem you've not solved is not worth solving.
>> >   - The transparencies that are valuable for lambdas are independent
>> from
>> >   and orthogonal to whether the lambda is used in the same thread in
>> which it
>> >   was created, or whether it is used concurrently.
>> >
>> > Cheers,
>> > Neal
>> >
>> > On Tue, Jan 15, 2013 at 4:24 PM, Steven Simpson <ss at comp.lancs.ac.uk>
>> wrote:
>> >
>> >> Hi Neal - did you mean to reply off-list?  If not, feel free to repost
>> as
>> >> if nothing had happened, and I'll do likewise, or just reply back on
>> the
>> >> list to the message below, as you see fit.
>> >>
>> >>
>> >> On 15/01/13 22:12, Neal Gafter wrote:
>> >>
>> >>> What is a serial guarantee, and how do you imagine encoding it into
>> the
>> >>> language rules?
>> >>>
>> >>
>> >> Short answer by example: a serial forEach promises to invoke its block
>> on
>> >> the caller's thread, and promises not to let it be invoked after
>> returning
>> >> to the caller.  I think you called it 'non-concurrent' on the list, at
>> some
>> >> point.  Longer answer...
>> >>
>> >> Some time ago, while it was being discussed, I was quite in favour of
>> >> having various forms of transparencies that people were advocating, and
>> >> then I've come round to the idea that lambdas shouldn't go that far,
>> >> because a declared goal is to support idioms most suitable to parallel
>> >> execution.
>> >>
>> >> I probably noticed that the use cases advocating the extra
>> transparencies
>> >> were often about flow-control abstraction, and that these are often
>> serial
>> >> in nature, i.e. the control abstraction didn't have to do anything
>> >> concurrently to get its job done, e.g. (non-parallel) forEach,
>> with(lock)
>> >> {... }.  (I'm assuming that this observation is almost always true,
>> but I
>> >> am interested in seeing examples which contradict it, too.)
>> >>
>> >> Some advocated the user of these abstractions being able to turn on
>> >> transparencies - e.g. using @Shared on a local to be mutated by the
>> lambda
>> >> - because they believe the call they are making promises not to invoke
>> the
>> >> supplied lambdas in 'complex' ways.  There's a risk that this will be
>> used
>> >> incorrectly, e.g. when the promise isn't made but is believed to
>> exist, and
>> >> I thought it would be a lower risk if instead the promise was formally
>> >> declared by the method, and that turned on the transparencies.  Rather
>> than
>> >> the user having control over transparencies, and guessing when they
>> would
>> >> be safe, the provider of the method, who is in a position to decide
>> whether
>> >> to make the promise, chooses whether or not to declare it.
>> >>
>> >> So, a very weak serial guarantee that could be made by the receiver of
>> a
>> >> lambda (or other functional object) would be: "I will not invoke the
>> >> functional object on any thread but the one you're invoking me with
>> now."
>> >> This would be enough to turn on mutable local capture, in some
>> >> Javascript-like asynchronous environment, as was discussed here:
>> >>
>> >> <http://mail.openjdk.java.net/**pipermail/lambda-dev/2011-**
>> >> July/003751.html<
>> http://mail.openjdk.java.net/pipermail/lambda-dev/2011-July/003751.html>
>> >>>
>> >>
>> >> A stronger guarantee would be as above plus: "I will not invoke the
>> >> functional object after returning control to you."  This appears good
>> >> enough to do control abstraction, where the method making the
>> guarantee is
>> >> really just offering to organize the user's code according to some
>> usually
>> >> serial idiom.  You'd additionally permit non-local returns, throws,
>> breaks
>> >> and continues.
>> >>
>> >> Further guarantees would be about the number of times something was to
>> be
>> >> invoked (at least once, exactly once, etc), which I believe you'd need
>> to
>> >> combine control abstraction with definitely-(un)assigned concepts,
>> >> 'continue' and 'break'.
>> >>
>> >> How to express these guarantees...  I thought you could annotate the
>> >> functional-interface parameters, e.g. @Callback for the weakest,
>> @Serial
>> >> for the next.  I don't know if that's good enough, because I'd expect
>> to be
>> >> able to use @Callback as if it were some sort of C-like type qualifier
>> (and
>> >> you'd then consider whether a @Callback Runnable was assignable to a
>> plain
>> >> Runnable), but that might be stretching the meaning, purpose and
>> principles
>> >> of annotations. Perhaps less of a problem for @Serial and stronger - I
>> >> imagine these enabling a certain syntax, which emphasises that only a
>> >> serial idiom is going to be applied, and turns on the corresponding
>> >> transparencies.
>> >>
>> >> Are these guarantees compiler-verifiable?  Feels like not, generally.
>>  It
>> >> might not matter so much.  We already don't expect the compiler to
>> detect a
>> >> Comparator that just returns random values, or a Set that doesn't have
>> set
>> >> semantics.
>> >>
>> >> So, back to the point about Stream.  You only informally get serial
>> >> guarantees with the one returned by Streamable.stream(). (Otherwise,
>> what
>> >> is parallel() for?)  If you declare a method to take a Stream (rather
>> than
>> >> a Streamable[1], for whatever reason), you can't assume that it has
>> come
>> >> from stream(), not parallel().  If you did know it was serial, you
>> might
>> >> feel free to implement your own ad-hoc transparencies that depend on
>> this
>> >> serial characteristic.  Since you don't know, you may as well assume
>> it's
>> >> the parallel one and not take the risk with ad-hoc transparencies - in
>> >> which case, what's the point in having stream()?  At least, without
>> having
>> >> to add any further language changes, if stream() returned a
>> SerialStream,
>> >> you'd have as much a guarantee as you would that a supplied Set is not
>> >> merely a Collection, and you could at least formally express that you
>> >> require a SerialStream if a plain (potentially parallel) one won't do.
>> >>
>> >> Ignoring my ideas of serial guarantees for a moment, and narrow down
>> the
>> >> problem to just exception transparency, suppose you ressurect <throws
>> E>.
>> >> You could declare a serial forEach as:
>> >>
>> >>  <throws E> void forEach(Block<? super T, E> block) throws E;
>> >>
>> >> For a parallel version, you have to specify how multiple exceptions are
>> >> handled by forEach, or require the block to handle them itself (and
>> drop
>> >> <throws E>).  If forEach handles them, does it throw away all but the
>> >> first?  Merge them in some wrapping exception (which probably changes
>> the
>> >> signature)?  It seems, the most sensible options change the signature,
>> and
>> >> one signature belongs on Stream, and the other on SerialStream.
>> >>
>> >> [1] Hmm, seems I'm out of date already - no more Streamable.
>> >>
>> >> Cheers!
>> >>
>> >
>>
>>
>>
>