Exception transparency

Tue Jun 8 17:28:10 PDT 2010

Brian nailed exactly why my definition of purity is so "lame" - it doesn't
make matters worse and it can be retrofitted easily into existing java.
Anything more drastic is definitely outside the scope of project lambda.

I'm stil very confused about what you're suggesting. Why do we need @pure
and @impure? All non-closures are always pure and cannot be impure -
anonymous classes get closed and they can't even do any of the things on the
forbidden list (access mutables, transfer control, or transparently throw
checked exceptions). Closures are pure by default, unless they do one of
those 3 forbidden things. All of these 3 forbidden things can be easily
inferred; there is thus no need for annotations. Where would such an
annotation even go? project lambda does not allow you to annotate a closure
definition (an oversight that might have to be looked at more closure before
a final release!)

Also, you're mixing things around. All lambdas are *PURE* unless you do
something explicitly forbidden for pure closures, then they are
automatically impure. There's no messy syntax at all because there is no
@pure or @impure - closures are what they are.

There's a different way to look at it: Syntactically, all closures are
_impure_, unless the compiler detects that you aren't doing anything impure,
in which case it might as well be pure. So, the compiler understands what
you're _trying_ to do regardless of purity or lack thereof. This is similar
to how java works today. If you access a non-final variable from an
anonymous inner class, the compiler does NOT react with "variable not
found". Instead it knows exactly what you're trying to pull and says that
you can't do that. So, it understands you, it just refuses to compile it.

The notion of purity / impurity I'm suggesting works similar. If you throw
checked exception C from a closure, and the closure's signature doesn't
allow throwing C, but your lexical scope *DOES*, then the compiler will
always know exactly what you're trying to do. It may not compile it, though
- that depends on whether or not an impure closure is allowed there.

--Reinier Zwitserloot

On Tue, Jun 8, 2010 at 9:50 PM, Nathan Bryant
<nathan.bryant at linkshare.com>wrote:

> Brian,
>
> Ok, that makes some sense. If some of these ideas were put into practice,
> it seems to imply:
>
> -2 new annotations, @impure @pure
> -lambdas default to @pure if they capture only final locals (just like
> anonymous inner classes today)
>
> -Assuming the above, Iterable.forEach() defaulting to parallel semantics
> whenever it see @pure, seems too risky
> -But no problem if the programmer says Iterable.par().forEach() - they get
> what they ask for
>
> -or-
>
> -1 new annotation, @pure
> -all lambdas are impure unless @pure is explicitly stated
> -compiler error (warning?) if @pure is applied to a lambda that captures
> non-final locals
> -compiler warning (?) if @pure is applied to a function that calls an
> impure function
> -perceived messy syntax of adding @pure everywhere?
>
> -----Original Message-----
> From: Brian Goetz [mailto:brian.goetz at oracle.com]
> Sent: Tuesday, June 08, 2010 3:32 PM
> To: Nathan Bryant
> Cc: Reinier Zwitserloot; Paulo Levi; lambda-dev at openjdk.java.net
> Subject: Re: Exception transparency
>
> You're both right...
>
> Typically "pure functions" are side-effect-free, thread-safe, and all sorts
> of
> other goodness which enables them to be cached, lazified, eagerized,
> redundantly executed on different nodes for locality reasons, all sorts of
> other good stuff.
>
> But the constraints on "purity" in Java are too strong to be realistic.
>  Java
> is too happy to allow mutation and does not offer tools to prevent it
> (e.g.,
> 'const'.)
>
> Anonymous functions can do all sorts of evil and racy things.  But the
> rules
> (primarily that you can only capture final references) make the most common
> racy things harder.  The goal should be for closures to not dramatically
> *increase* the space of reasonable-looking but racy code.
>
> In that way Reinier's notion of purity is a good concept (but could use a
> better name), because it may enable us to add closures without greatly
> adding
> to the set of racy code constructs.  We can't take away the existing racy
> code
> constructs (and if we could, it wouldn't be under the scope of Lambda) but
> we
> can at least not make things worse.
>
> Your sum() example violates all the conventional notions of purity (has
> side
> effects, side effects are non-idempotent and non-commutative, etc) but fits
> into Reinier's notion of purity well enough: assuming that log.debug() is
> thread-safe, it does not express any constraint between the lambda and its
> capturing scope.  (We do not have a means for identifying either statically
> or
> dynamically whether log.debug() is thread-safe or not, which sucks, but
> this
> is not a problem that is new to closures.)
>
> On 6/8/2010 3:21 PM, Nathan Bryant wrote:
> > Reiner:
> >
> > Ok gotcha. I think.
> >
> > It seems to me that your particular definition of purity does not fit
> > Brian’s desire to use the notion of purity to allow us to determine
> > what’s thread safe. If a pure function were to call an impure function,
> > it can no longer be guaranteed that it’s thread safe: the impure
> > function might be mutating something, somewhere, either inside or
> > outside the VM, on the pure function’s behalf. That mutation might not
> > be thread safe. So in the most general sense, I/O is not pure.
> >
> > *From:* reinierz at gmail.com [mailto:reinierz at gmail.com] *On Behalf Of
> > *Reinier Zwitserloot
> > *Sent:* Tuesday, June 08, 2010 3:14 PM
> > *To:* Nathan Bryant
> > *Cc:* Paulo Levi; Brian Goetz; lambda-dev at openjdk.java.net
> > *Subject:* Re: Exception transparency
> >
> > Nope; anonymous inner classes are pure, and they can access locals from
> > outer scope. As long as they are final, though. The implementation shows
> > why this holds: When a variable is final, a copy is indistinguishable
> > from the original (and this is in fact how anonymous inner classes
> > accessing outer final locals is done by javac). Therefore, access to an
> > immutable variable from outer scope is no different from passing that
> > variable along as a parameter to the closure, which is conceptually
> > similar to currying that parameter into the closure. Which is clearly
> pure.
> >
> >
> > --Reinier Zwitserloot
> >
> >
> > On Tue, Jun 8, 2010 at 7:51 PM, Nathan Bryant
> > <nathan.bryant at linkshare.com <mailto:nathan.bryant at linkshare.com>>
> wrote:
> >
> > Reinier:
> >
> > You mean A) directly or indirectly, right? A pure function may not
> > access a mutable variable from another context… including statics,
> > threadlocals, any big resource pool that’s hanging off of some static
> > somewhere, or an object to which you have a reference… etc. In other
> > words, a pure function may only call pure functions. No?
> >
> > If this is not what you mean, then it seems to me that you are talking
> > about the “weaker, more complex definition of pureness” that I was
> > alluding to.
> >
> > *From:* reinierz at gmail.com <mailto:reinierz at gmail.com>
> > [mailto:reinierz at gmail.com <mailto:reinierz at gmail.com>] *On Behalf Of
> > *Reinier Zwitserloot
> > *Sent:* Tuesday, June 08, 2010 1:39 PM
> > *To:* Nathan Bryant
> > *Cc:* Paulo Levi; Brian Goetz; lambda-dev at openjdk.java.net
> > <mailto:lambda-dev at openjdk.java.net>
> > *Subject:* Re: Exception transparency
> >
> > Nathan, I don't think pure means what you think it means. Or I'm
> > horribly confused, also an option.
> >
> > "pure" at least in sofar I understand it within the confines of this
> > discussion, simply means: "Does not assume that the runtime stack
> > matches the lexical stack". No more, no less. So, everything is pure,
> > unless it is a closure that does one or more of the following things:
> >
> > A) Access a variable from lexical scope that is outside the closure
> > itself. The variable isn't final, and isn't effectively final.
> >
> > B) returns, breaks, or continues from inside the closure to outside the
> > closure; i.e. the closure breaks a for loop in the containing method.
> > (Not on the agenda for java7 closures so far, but nice to make possible,
> > if not for java7, then for java 8)
> >
> > C) throws a checked exception which is not handled by the closure (as
> > in, the closure type doesn't have a "throws TheCheckedException" in its
> > signature), but it IS handled by the containing method (either because
> > the closure is defined in a try/catch block that catches that exception,
> > or the containing method has a "throws TheCheckedException" clause).
> >
> > Any closure that engages in none of those 3 things is pure. Pure does
> > not mean "I have no side effects".
> >
> >
> > --Reinier Zwitserloot
> >
> > On Tue, Jun 8, 2010 at 5:47 PM, Nathan Bryant
> > <nathan.bryant at linkshare.com <mailto:nathan.bryant at linkshare.com>>
> wrote:
> >
> > A simple example:
> >
> > public @pure int sum(int a, int b) {
> >
> > log.debug(“sum!”);
> >
> > return a + b;
> >
> > }
> >
> > log.debug can never be strictly pure, but surely we don’t want to
> > disallow it!
> >
> > *From:* Paulo Levi [mailto:i30817 at gmail.com <mailto:i30817 at gmail.com>]
> > *Sent:* Tuesday, June 08, 2010 11:44 AM
> > *To:* Nathan Bryant
> > *Cc:* Brian Goetz; Reinier Zwitserloot; lambda-dev at openjdk.java.net
> > <mailto:lambda-dev at openjdk.java.net>
> > *Subject:* Re: Exception transparency
> >
> > Why is it not workable? Do programmers have trouble understanding that
> > they can't use private methods outside of their class to give a foolish
> > example?
> >
> >
> > Why would they have problems understanding that they can only use "pure"
> > methods in the body of a pure function -recursive obvs? (aside, here you
> > can see why constructors, and SAM types are still useful, mark a method
> > as pure, but leave the constructor to deal with the impurity).
> >
> > Seems like the most elegant solution, and frankly almost a requirement
> > for that the new parallel api's if they are to maintain a little sanity.
> > The alternative is to retrofit it later if the concept becomes
> > available, that will break code, and thus be used as a excuse to do
> > nothing, as usual.
> >
>